I have recently imported an SQL content pack, and effected the agent parameters. I am able to collect logs from the SQL server, however some logs are not normalized thus unreadable. Is this a configuration issue or I need a plug in that will normalize the data?

I installed the Microsoft - IIS content pack on my Log Insight cluster.  I noticed it requires certain fields to be enabled on the IIS server for the logs.  My question is, for it to work correctly, are these the only fields that "can" be enabled or do I just need to make sure that at least these ones are enabled?  Hope that makes sense.

Thanks,

Tim

IIS Prerequisites:

IIS content pack uses logs in W3C format, the following fields need to be enabled in IIS logs using IIS Manager:

• date
• time
• s-sitename
• s-ip
• cs-method
• cs-uri-stem
• cs-uri-query
• s-port
• cs-username
• c-ip
• cs (User-Agent)
• sc-status
• sc-substatus
• sc-win32-status

• time-taken

Dear all,

I have a requriement to query URLs like "/dac/rest/1.0/asset/list?workspaceId=701a551a-0e46-4e2b-b260-d8dcd4dda868"  in Log Insight.

Then I found the query returns nothing because there is a question mark inside the URL. In other words, I can query "/dac/rest/1.0/asset/list" and "workspaceId=701a551a-0e46-4e2b-b260-d8dcd4dda868" but I can't query "/dac/rest/1.0/asset/list?workspaceId=701a551a-0e46-4e2b-b260-d8dcd4dda868"

May I know whether we do have a workaround here?

Thanks

Silvester

I've installed Exchange content pack v3.2 configured all three scheduled jobs but nothing is showing on the Overview under Microsoft Exchange.  I've copied Exchange templates and added Exchange servers to the new template.  

https://solutionexchange.vmware.com/store/products/microsoft-exchange-server-log-insight-content-pack

Are the three scheduled jobs suppose to dump stats to ..\exchange\scripts\logs folder?  If so, it doesn't appear to work. 

I deploy Log Insight of OVF template and after start VM. In console VM I see the below

I can't connect this URL - WEB server in not started.

Why the Log Insight appliance reports as VMware vCenter Server Appliance?

Thank

Ales

Hi All,

The environment that I'm currently working in has a classification system for emails. It basically means that all emails must have [SEC=UNCLASSIFIED] in the subject line of the email or it is dropped.

Is there a way to edit the email subject lines in vRealize Log Insight alerts to append [SEC=UNCLASSIFIED]?

Thanks,

I see when you set up a forwarder with Log Insight you can select the ingestion API protocol or the syslog protocol.  Beyond that I don't see any way to customize the syslog protocol for a specific destination - for example a specific SIEM formal like LEEF or CEF.  Can this be done manually somewhere in the log insight config files or is this not possible with Log Insight?

Hi, I have created a vRealize dashboard that displays the Log Insight login screen. I am able from there to login to Log Insight but I heard there was a way to set it up to auto login from within vRealize, anyone know how this can be done?

thanks!

Morning,

i am failling to collect message and secure logs from one RHEL 6.8 server, The output of loginsight-agent is:

2016-07-18 10:43:14.900565 0x00007f5bb5eca700 <trace> AgentDaemon:389    | AgentDaemon Configuring...

2016-07-18 10:43:14.900595 0x00007f5bb5eca700 <trace> AgentDaemon:394    | Configuring queue...

2016-07-18 10:43:14.900622 0x00007f5bb5eca700 <trace> Config:211         | Read config param storage.max_disk_buffer = 200

2016-07-18 10:43:14.900918 0x00007f5bb5eca700 <trace> AgentDaemon:414    | Configuring collectors...

2016-07-18 10:43:14.900946 0x00007f5bb5eca700 <trace> EventCollector:22  | ConfigureAndStart invoked for collector: filelog

2016-07-18 10:43:14.900962 0x00007f5bb5eca700 <trace> EventCollector:25  | Collector filelog already started, trying to reconfigure

2016-07-18 10:43:14.900979 0x00007f5bb5eca700 <trace> EventCollector:37  | Stopping filelog

2016-07-18 10:43:14.931211 0x00007f5bb5eca700 <trace> EventCollector:39  | Stopped filelog

2016-07-18 10:43:14.931250 0x00007f5bb5eca700 <trace> EventCollector:47  | Configuring filelog

2016-07-18 10:43:14.931774 0x00007f5bb5eca700 <trace> EventCollector:49  | Configuration of filelog is done

2016-07-18 10:43:14.931803 0x00007f5bb5eca700 <trace> EventCollector:56  | Starting filelog

2016-07-18 10:43:14.931882 0x00007f5b8bfff700 <trace> Logger:188         | Thread "ThreadPool" has id 0x7f5b8bfff700

2016-07-18 10:43:14.932320 0x00007f5bb5eca700 <error> FLogCollector:209  | Could not subscribe to channel <syslog>. boost::filesystem::path codecvt to wstring: error

2016-07-18 10:43:14.932386 0x00007f5ba69f7700 <trace> Logger:188         | Thread "ThreadPool" has id 0x7f5ba69f7700

2016-07-18 10:43:14.932713 0x00007f5bb5eca700 <error> FLogCollector:209  | Could not subscribe to channel <com.linux.auth>. boost::filesystem::path codecvt to wstring: error

2016-07-18 10:43:14.932780 0x00007f5ba73f8700 <trace> Logger:188         | Thread "ThreadPool" has id 0x7f5ba73f8700

2016-07-18 10:43:14.933100 0x00007f5bb5eca700 <error> FLogCollector:209  | Could not subscribe to channel <com.linux.messages>. boost::filesystem::path codecvt to wstring: error

2016-07-18 10:43:14.933187 0x00007f5ba7df9700 <trace> Logger:188         | Thread "ThreadPool" has id 0x7f5ba7df9700

2016-07-18 10:43:14.933510 0x00007f5bb5eca700 <error> FLogCollector:209  | Could not subscribe to channel <com.linux.syslog>. boost::filesystem::path codecvt to wstring: error

2016-07-18 10:43:14.933594 0x00007f5ba5ff6700 <trace> Logger:188         | Thread "ThreadPool" has id 0x7f5ba5ff6700

2016-07-18 10:43:14.933905 0x00007f5bb5eca700 <error> FLogCollector:209  | Could not subscribe to channel <com.linux.cron>. boost::filesystem::path codecvt to wstring: error

2016-07-18 10:43:14.934009 0x00007f5ba55f5700 <trace> Logger:188         | Thread "ThreadPool" has id 0x7f5ba55f5700

2016-07-18 10:43:14.934352 0x00007f5bb5eca700 <error> FLogCollector:209  | Could not subscribe to channel <com.linux.secure>. boost::filesystem::path codecvt to wstring: error

2016-07-18 10:43:14.934454 0x00007f5ba4bf4700 <trace> Logger:188         | Thread "ThreadPool" has id 0x7f5ba4bf4700

2016-07-18 10:43:14.934746 0x00007f5bb5eca700 <error> FLogCollector:209  | Could not subscribe to channel <com.linux.maillog>. boost::filesystem::path codecvt to wstring: error

2016-07-18 10:43:14.934892 0x00007f5b8abfd700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b8abfd700

2016-07-18 10:43:14.934921 0x00007f5b8a1fc700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b8a1fc700

2016-07-18 10:43:14.934952 0x00007f5b897fb700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b897fb700

2016-07-18 10:43:14.934987 0x00007f5b88dfa700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b88dfa700

2016-07-18 10:43:14.935028 0x00007f5b6ffff700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b6ffff700

2016-07-18 10:43:14.935059 0x00007f5b6f5fe700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b6f5fe700

2016-07-18 10:43:14.935098 0x00007f5b6ebfd700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b6ebfd700

2016-07-18 10:43:14.935138 0x00007f5b6e1fc700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b6e1fc700

2016-07-18 10:43:14.935173 0x00007f5b6d7fb700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b6d7fb700

2016-07-18 10:43:14.935198 0x00007f5b6cdfa700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b6cdfa700

2016-07-18 10:43:14.935236 0x00007f5b57fff700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b57fff700

2016-07-18 10:43:14.935259 0x00007f5bb5eca700 <trace> EventCollector:59  | Started filelog

2016-07-18 10:43:14.935278 0x00007f5b575fe700 <trace> Logger:188         | Thread "FLogThreadPool" has id 0x7f5b575fe700

2016-07-18 10:43:14.935302 0x00007f5bb5eca700 <trace> AgentDaemon:419    | Configuring transport...

2016-07-18 10:43:14.935331 0x00007f5bb5eca700 <trace> Config:263         | Read config param server.proto = cfapi

2016-07-18 10:43:14.935349 0x00007f5bb5eca700 <trace> Config:263         | Read config param server.hostname = servername@domain.com

2016-07-18 10:43:14.935365 0x00007f5bb5eca700 <trace> Config:317         | Configuration key server.ssl is not specified. Using default: no

2016-07-18 10:43:14.935397 0x00007f5bb5eca700 <trace> Config:211         | Read config param server.port = 9000

2016-07-18 10:43:14.935420 0x00007f5bb5eca700 <trace> Config:211         | Read config param server.reconnect = 30

2016-07-18 10:43:14.935440 0x00007f5bb5eca700 <trace> AgentDaemon:444    | AgentDaemon configured successfully

Any ideas on what could be the issue?

Hi. I just deployed LI 3.3.2 today and all seems good until I leave the appliance for a while and then go back to it. I find that every login attempt gives me the following error - "HTTP Status 403 - CSRF nonce validation failed". A reboot id required to restore access. I've had to do this twice today.

Has anybody else seen this?

Cheers,

Neil.

Hello, guys.

Anybody know how audit change CPU, MEM and Disk virtual machine in Log Insight ?

It's possible ?

Hello,

Log Insight has great potential for our company but the maximum syslog message length of 10KB results in important log data being excluded from my queries. Is there any way to increase the maximum syslog size that Log Insight will ingest? If not, are there any workarounds available?

Thank you,

Dave

Hi Everbody,

I have a problem while trying to export the result of any query (with Log Insight 3.3.2).

After clicking on Export, no matter which format I choose, Export button shows no function, only Cancel works.

I tested that with Firefox and Chrome.

Do you have any ideas or suggestions?

THX for you help in advance....

Regards

Stefan

VMworld US is in Las Vegas this year and less than a month away. Recently the VMworld catalog was released allowing you to see all the session and plan your schedule. Once again, Log Insight is well represented with some knowledgeable people presenting on a variety of topics. Here are the top 5 sessions you should be sure to catch:

1. #MGT8040 Turning up the Noise: How VMware’s Private Cloud Team is Getting Closer to the Heartbeat of their Infrastructure, Billions of Events at a Time
2. #MGT8641 A Lot of Insight; No Power Point
3. #MGT7685 Insight into the World of Logs with VMware vRealize Log Insight
4. #INF8845 vSphere Logs Grow Up! Tech Preview of Actionable Logging with Log Insight
5. #MGT9615-SPO vRealize the Possibilities: Application Agility and Rapid Deployment with vRealize Automation, Orchestration, Operations and Log Insight

Also, for those of you attending TAM Day, be sure to catch the following exclusive Log Insight sessions where you will get information that VMworld attendees will not!

Refer to our Blog Post for more information: Top 5 Log Insight VMworld Sessions - VMware Cloud Management

Hi,

We are trying to forward events from Log Insight to a central syslog server - as syslog. Unfortunately it seems like Log Insight adds a VERSION 1 to the outgoing message, indicating that the message is RFC-5424, while its actually RFC-3164.

The following is a snippet from a TCP-dump on the Log Insight appliance:

185.xx.xx.7 is the Log Insight appliance

185.xx.xx.12 is the central syslog-server

10:41:44.606332 IP 185.xx.xx.6.45876 > 185.xx.xx.7.514: SYSLOG user.info, length: 193

E....U@.@.;..WQ..WQ..4..... <14>Aug  9 12:41:44 mgtnsxman01.xxx.xxx.xxxx.xxxxx.xxx 2016-08-09 12:41:44.605 CEST  INFO http-nio-127.0.0.1-7441-exec-163 UserSessionManager:43 - New session: XXXXXXXXXXXXXXXXXXXXXXXXXXX3F506

10:41:44.655030 IP 185.xx.xx.7.60149 > 185.xx.xx.12.514: SYSLOG user.info, length: 194

E...U4@.@....WQ..WR.........<14>1 Aug  9 12:41:44 mgtnsxman01.xxx.xxx.xxxx.xxxxx.xxx 2016-08-09 12:41:44.605 CEST  INFO http-nio-127.0.0.1-7441-exec-163 UserSessionManager:43 - New session: XXXXXXXXXXXXXXXXXXXXXXXXXXX3F506

Any reason Log Insight is not just forwarding syslog messages verbatim? Alternatively, convert to RFC-5424 format when forwarding.

Well, even better - fix the originator and have it log in RFC-5424 format in the first place ;-)

For syslog messages already in RFC-5424 format, when sent to Log Insight, things seems fine - although, PRI seems to have been changed!?

10:41:43.132566 IP 185.xx.xx.25.46324 > 185.xx.xx.7.514: SYSLOG local0.info, length: 376

E.....@.@....WQ..WQ.......:.<134>1 2016-08-09T10:41:04Z nsx-controller controller - api_request [niciraTag@39961 controller="df8ea526-2811-4260-9410-c0dd2c6db543" cluster="7ae1fa0d-ef26-469d-85ac-abeac48beacd"] ...185.xx.xx.6:56608 admin - [09/Aug/2016:10:41:04 +0000] "GET /ws.v1/control-cluster/node/df8ea526-2811-4260-9410-c0dd2c6db543 HTTP/1.1" 200 850 "-" "Jakarta Commons-HttpClient/3.0" 0.004210

10:41:43.180414 IP 185.xx.xx.7.60149 > 185.xx.xx.12.514: SYSLOG user.info, length: 374

E...U*@.@..n.WQ..WR......~.R<14>1 2016-08-09T10:41:04Z nsx-controller controller - api_request [niciraTag@39961 controller="df8ea526-2811-4260-9410-c0dd2c6db543" cluster="7ae1fa0d-ef26-469d-85ac-abeac48beacd"] ...185.xx.xx.6:56608 admin - [09/Aug/2016:10:41:04 +0000] "GET /ws.v1/control-cluster/node/df8ea526-2811-4260-9410-c0dd2c6db543 HTTP/1.1" 200 850 "-" "Jakarta Commons-HttpClient/3.0" 0.004210

Regards

Claus Albøge

Has anyone configured the SRM content pack yet in LI? I've installed it - but the text on the next steps are a little vague...

I see the SRM agent in the LI administration page - but nowhere to add IP addresses of SRM servers.

I seem to be missing the concept of how the Horizon View content pack is supposed to work since my charts are not populating even though syslog data is arriving from the connection broker at the log insight collector.

I see this information in the tech specs for the View content pack...

Tech Specs

Using the Log Insight Windows Agent, which is available for download from the Log Insight Administration --> Agents page, use this liagent.ini configuration:

[filelog|ViewMain]
directory=C:\ProgramData\VMware\VDM\logs
include=log-*.txt;debug-*.txt;pcoip_agent*.txt;pcoip_server*.txt
exclude=pcoip_perf*.txt;v4v*.log;wsnm_starts.txt

Make sure that agent is installed on the base image so that it runs on each View desktop, plus it should be installed on all the other servers as well including: ALL connection, security, & composer servers.

and looking at the content pack definition I see a number of fields with regex values defined, which are used to derive the graphs from information already in the syslog messages

example: vmw_view_agent_build = AGENTVERSION><AGENTBUILDNUM>-?\d+</AGENTBUILDNUM

...so I can't understand why would an agent be needed on the View servers and View desktops. Isn't the Log Insight agent only for pulling event logs and (I'm assuming) flat files? Given that Log Insight works with and learns from any text that has been shipped to it (in this case from syslog from the connection broker), and the fact that fields such as "appname" containing the value "view" are easy enough to examine, why would anything else be needed to populate the charts in the View content pack?

Like I said, I seem to be missing something.

Regards,

Ray

I have custom logs and each field seperated by a pipe character (|).  I wanted to create a custom regex field that should bring me back the 5th field.

Regex that tested in regex101.com is below and it is working fine with my log line.

^(?:[^|]*\|){5}([^|]*)

But when I use it loginsight custom field It returns unrelated fields (more that 1 pipes).

Any idea?

Log sample:

2016-08-25 10:19:29,011 INFO  [http-nio-7070-exec-710] [x.service.y]  - [#|INFO|25/08/2016 10:19:28.991|172.17.0.55|3454353453453|8.8.8.8|xyccc|1370|asdasdasd|#]

Hi all

after upgrading my log insight from version 3.3 to version 3.6 I am not able to logon to the web gui with the admin user.

It always says "Failed".

I have already reseted the admin password via ssh with script li-reset-admin-passwd.sh

Anyone having the same issue?

Regards

Hideyori

Hi,

We use Log Insight 3.6. vCenter 6 Update 2 Appliance with a seperate PSC.

We have configured vSphere intergration and the Log Insight vSphere dashboard is populated. Our ESXi Hosts are logging fine. Events, alarms etc are all received fine as well.

In the vCenter Server - Overview dashboard. No events are received. Nothing which appears in the VPXd log is being seen or can be found in log insight. No performance data is being received either in the vCenter Server - Performance dashboard.

vCenter Server appliance has been configured to forward UPD syslog traffic to Log Insight with the following configuration (IP has been removed) in the /etc/syslog-ng/syslog-ng.conf file.

source vpxd {

       file("/var/log/vmware/vpx/vpxd.log" follow_freq(1) flags(no-parse));

       file("/var/log/vmware/vpx/vpxd-alert.log" follow_freq(1) flags(no-parse));

       file("/var/log/vmware/vpx/vws.log" follow_freq(1) flags(no-parse));

       file("/var/log/vmware/vpx/vmware-vpxd.log" follow_freq(1) flags(no-parse));

       file("/var/log/vmware/vpx/inventoryservice/ds.log" follow_freq(1) flags(no-parse));

};

# Remote Syslog Host

destination loginsight {

       udp("x.x.x.x" port (514));

};

# Log vCenter Server vpxd log remotely

log {

        source(vpxd);

        destination(loginsight);

};

How can I troubleshoot this issue. I wish to be able to query VPXD logs in Log Insight.

Cheers