VMware Logging Guide for Snare Server v7.0 Intersect Alliance International Pty Ltd. All rights reserved worldwide. Intersect Alliance Pty Ltd shall not be liable for errors contained herein or for direct, or indirect damages in connection with the use of this material. No part of this work may be reproduced or transmitted in any form or by any means except as expressly permitted by Intersect Alliance International Pty Ltd. This does not include those documents and software developed under the terms of the open source General Public Licence, which covers the Snare agents and some other software. The Intersect Alliance logo and Snare logo are registered trademarks of Intersect Alliance International Pty Ltd. Other trademarks and trade names are marks' and names of their owners as may or may not be indicated. All trademarks are the property of their respective owners and are used here in an editorial context without intent of infringement. Specifications and content are subject to change without notice. Page 1 of 14
Table of Contents 1. About this Guide.................................................... 3 2. VMware Server Configuration.......................................... 4 3. Analysis........................................................... 6 Page 2 of 14
1. About this Guide This document details the steps required to configure VMware ESXi vsphere CLI to log to the Snare Server, and also highlights some basic analysis strategies for Snare Server v7. More details on the techniques used, are available in the Snare Server User Guide. Important Note These instructions have been tested on VMware ESXi 5.1, and should also apply to other versions of ESXi and ESX, as long as the syslog configuration can be modified to forward events to the Snare Server. Other Resources Other resources that may be useful to read include: Snare Server v7.0 Users Guide vsphere Command-Line Interface Documentation http://www.vmware.com/support/developer/vcli/ Intersect Alliance International Pty Ltd Page 3 of 14
2. VMware Server Configuration The following procedure assumes that you wish to configure the vsphere server via the command line. Logging functionality can also be modified using vclient GUI tools; please see the VMware documentation for detailed procedures. 2.1. Activate SSH, or access the vsphere Console 2.1.1. What you need... The DNS name or IP address of your vsphere server. Access to the vsphere console to enable SSH 2.1.2. Instructions Initial Screen On the vsphere console, use the F2 key to access the system configuration options. Hit F2 on your keyboard. Troubleshooting Use your cursor keys to choose the 'Troubleshooting options' menu option, and hit ENTER on your keyboard. Choose 'Troubleshooting options". Intersect Alliance International Pty Ltd Page 4 of 14
Activate SSH / Console You will need to either activate the vsphere console, or SSH. If you choose to activate the console, the keyboard sequence Alt+F1 will open a local console. Log in using your administrator account and password. If you choose to activate ssh, connect to your ESX machine using the IP address displayed on the first console screen. 2.1.3. Syslog delivery In order to activate remote delivery of VMware log data using the syslog protocol, several commands need to be run. 2.1.3.1. Firewall configuration Run the following commands, to allow syslog data to be sent through the ESX local firewall. esxcli network firewall ruleset set --ruleset-id=syslog --enabled=true esxcli network firewall refresh 2.1.3.2. Syslog configuration Configure the server to send syslog data to a remote server, using the syslog protocol. Substitute the IP address of your Snare Server, for the "10.11.12.13" in the following command: esxcli system syslog config set --loghost='udp://10.11.12.13:514' esxcli system syslog reload Intersect Alliance International Pty Ltd Page 5 of 14
3. Analysis If you are not familiar with the operation of the Snare Server, please refer to the Snare Server User Guide for more information. The Snare Server will receive data from your ESX/vSphere server, and add it to the generic syslog data source. The following series of screenshots provide an example of how to perform basic analysis on VMware vsphere/esx log data. Create a new objective called "VMWare ESX". Modify the objective configuration. Choose the "Change Type" button. Choose the "Analyse data from Generic Syslog logs" objective template, from the "Generic Syslog" group. Intersect Alliance International Pty Ltd Page 6 of 14
Once the objective template has been selected, the Configuration window will reappear, with log-type specific settings. Add a new match, and tell the Snare Server to look for logs from the ESX/vSphere server's hostname. Intersect Alliance International Pty Ltd Page 7 of 14
Add output components to the objective, such as the 15 minute pattern map, and Tabular Details. Modify the Table output configuration to include the fields of interest, save the configuration, and regenerate the objective. Intersect Alliance International Pty Ltd Page 8 of 14
We have some data returned from the objective. In this case, the data has arrived from the 'v5dev' server. You will notice though, that there are some interesting details within the body of the message, that we may be interested in analysing in more detail. In particular, you can see that the Date/Time presented in the event, is actually a little different than the time at which the Snare Server received the event. Usually, Snare is able to retrieve the date/time from within each event, but in this case, VMware are using a non-standard syslog date format, so the Snare Server has opted to preserve both the receive-time and the log-time in the event. However, we can pull out this information for our analysis. Intersect Alliance International Pty Ltd Page 9 of 14
Go back to your Configuration settings, and select the green 'Add New' button near the top of the window. A new window will pop up, asking for the "Field Name". Let's use "VMDATE". Next, we'll test a "regular expression" match to pull out the date from within the event body. For this, I have copied and pasted a sample event from the tabular output, into the 'sample log entry' field. Next, I have crafted a simple regular expression to pull out the date from the entry. In this case, the regular expression translates to: Grab the first 10 characters from the event, that contain numbers or dashes. Copy this expression to our 'Token', and save the result using the 'Create Field' button. Intersect Alliance International Pty Ltd Page 10 of 14
Regular expressions are very powerful. We could do almost the same thing by using the expression above, instead. (Grab the first 10 characters from the event, regardless of what they look like). While we're at it, we can grab the time (VMTIME). Since VMware's time format is reasonably consistent, a simple regular expression like the one above may be perfectly adequate. Next, we can pull out the syslog 'category' ("VMCATEGORY") Intersect Alliance International Pty Ltd Page 11 of 14
And the actual message content ("VMEVENT") These fields, can then be added to our Table output as required. Or featured as part of a graph component. Regenerate the objective once more once configuration has been completed. Intersect Alliance International Pty Ltd Page 12 of 14
Pie graph of sources. If we wanted to search for a particular subset of messages, such as commands executed by the root-level user, we could modify our configuration further. In this case, we've asked Snare to search for events from 'VMWareESX001' with a 'VMSource' of 'shell'. Regenerate the objective once done. Intersect Alliance International Pty Ltd Page 13 of 14
Output of a search for root-level command execution. Intersect Alliance International Pty Ltd Page 14 of 14