ArcSight Variable and Operators HOWTO

Transcription

1 ArcSight Variable and Operators HOWTO Descriptions and examples of some of the ArcSight variables and operators. Document Version

2 Copyright & Confidentiality Statements This document is Copyright Networks Ltd All rights reserved. The contents of this document may not be copied or duplicated in any form, in whole or in part, without the prior written permission of Networks Ltd. The information in this document is subject to change without notice. Networks Ltd shall not be liable for any damages resulting from technical errors or omissions which may be present in this document, or from use of this document. This document is an unpublished work protected by the United Kingdom copyright laws and is proprietary to Networks Ltd. Disclosure, copying, reproduction, merger, translation, modification, enhancement, or use of this document by anyone other than authorised employees, authorised users, or licensees of Networks Ltd without the prior written consent of Networks Ltd is prohibited. Document Information Name: ArcSight Variable and Operators HOWTO Version: Document Version 1.0 Publication Date: 23 September 2013 Classification: Prepared by: Mark Johnston Document Status: Final Document Control Version Date Author Change Summary Document Version /09/2013 Mark Johnston Final

3 Contents Copyright & Confidentiality Statements... 2 Document Information... 2 Document Control... 2 Introduction... 5 Background... 5 EvaluateVelocityTemplate... 5 General... 5 Location... 5 Example Search for usernames matching the pattern 1 character followed by 5 digits... 5 Example Compare ActiveList Entry to a regular expression (regex)... 6 Example String Substitution... 8 Example String Substitution Expanded (useful for activel:104)... 9 Example Sub Strings... 9 Example Conditional String Checks... 9 Example Others... 9 JavaMathematicalExpression General Location Notes Example Add 10 to the value of devicecustomnumber Example Calculate distance in KM s given latitude and longitude of two locations Active List Contains General Notes Like Logical Operator General Location Notes Example Simple Matching Matches Logical Operator General Location

4 Notes Example Search for an IP address in a String Example Search for the domain in a string Example Search for usernames matching the pattern 1 character followed by 5 digits

5 Introduction Background ArcSight provides a number of variables and operators that can be used within Content to help massage and search through event data. This document looks at some of the more useful variables and operators for which there is little existing documentation. EvaluateVelocityTemplate General Velocity is a java based template engine developed by the apache foundation. It provides a language to reference objects defined in java code. Further information on Velocity can be found at Below is a table of the most common operators used in Velocity. Specifically for ArcSight the evaluate_velocity_template variable allows interaction via the velocity language with ArcSight event information and other ArcSight variables. The variable is available for most resources. As an example, it could be used in an Active Channel (by implementing FieldSets) to give an Analyst an immediate view on rule fire information in a column. Location The EvaluateVelocityTemplate function is found under Variables (local or global) String EvaluateVelocityTemplate Example Search for usernames matching the pattern 1 character followed by 5 digits evaluate_velocity_template(#if($message.matches( [A-Za-z]\d{5} ))matches#else notmatches#end)

6 Example Compare ActiveList Entry to a regular expression (regex) Create an Active list with sample values Create global variable with local variables used to extract AL information Set actual global variable using local variables (calling $userdata which is local to the global) Set the event field reason with the global variable value

7 Insert some test data, and we see the reason field being set with appropriate info Global variable can also be used in conditions Setting the reason field to This works to test use of global variable in conditions

8 Channel with results This can also be done locally within the rule if running an older version of ESM Matches logical operator can also be used to test for the match Example String Substitution evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceall( x, y )) evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceall( [^x], y )) Assume message string user:mark:stuffhere:otherstuff:12345 and only want to return user value evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceall( user:([^:]+).*, $1 ))

9 Example String Substitution Expanded (useful for activel:104) Assume that activelist expiry comes from a list with two columns, rather than using indexof String1 is evaluate_velocity_template($devicecustomstring4.replaceall( ([^\ ]*)\ ([^\ ]*), $1 ) String2 is evaluate_velocity_template($devicecustomstring4.replaceall( ([^\ ]*)\ ([^\ ]*), $2 ) Example Sub Strings Assume message string of xx yy zz evaluate_velocity_template($message.substring(3,5)), which will return yy evaluate_velocity_template($message.substring(0,$message.indexof( ))), which will return xx evaluate_velocity_template($message.substring($message.lastindexof( ),8)) will return zz Example Conditional String Checks evaluate_velocity_template(#if($message== test )Matches#else Not_Matches#end evaluate_velocity_template(#if($message== test )ret_test#elseif($message== other )ret_other#else No_Match#end evaluate_velocity_template(#if($message!= test )No Match#else Match#end Example Others Multiple VM expressions in single template evaluate_velocity_template(#set($tempvar=$message.length())#if($tempvar > 10)Large#else Small#end

10 JavaMathematicalExpression General The function allows for Java mathematical expressions to be applied against fields (numerical only) within ArcSight. This function is not available under all resources, but can be used with Rules, Filters, Data Monitors and FieldSets. By virtue of being available in FieldSets, this function can thus be used within Active Channels. See for the available mathematical functions. Location The JavaMathematicalExpression function is found under Variables (local or global) Arithmetic JavaMathematicalExpression Notes ArcSight fields are referenced WITHOUT the $ within the expression. For example devicecustomnumber1 + 1 Example Add 10 to the value of devicecustomnumber1 evaluate_jep(devicecustomnumber1 + 10) Example Calculate distance in KM s given latitude and longitude of two locations The variables

11 The screenshot showing the results (validated with online calculator)

12 Active List Contains General Active list contains is not currently supported by ArcSight. However during our testing it was possible to extract regex expressions from an Active List and have those compared against respective string values. The screenshots below show the implementation. Notes It may be possible to achieve the activelist contains by using the ESM plugin. The plugin allows for the interaction of custom java code with events before they are written to or retrieved from the database. Create the activelist, must have a key field in order to be looked up. Insert all regex into regex expr field with a separator

13 Extract the regex expr values and then break them down (like activelist:104). Once broken down evaluate each regex expression against the field required. Screenshot showing the results

14 Like Logical Operator General The ArcSight LIKE rule operator functions exactly the same as the SQL LIKE condition with the exception of not being able to use escape characters (escape characters are used to match literal % or _ ). For the SQL LIKE condition there are two wildcard matches, these being % and _. The % allows matching of any string of any length (including zero length). The _ allows matching on a single character. Location Like is a logical operator and thus can be found under Event Conditions, in the Common Conditions Editor. Notes Variables cannot be used for comparison. For example Message Like $test Example Simple Matching Given event.message = hello the following will work: Message Like %ell% Message Like h% Message Like hell_ Message Like %ell_

15 Matches Logical Operator General The ArcSight MATCHES logical operator provides the capability to use regular expressions in rule conditions only for string matching. The regex engine is based upon Perl 5 syntax, and within ArcSight the regular expression must match the entire string to be true (wildcards can be used). Location The logical operator can only be used within Rules. Notes Commas cannot be used within the regular expression (ArcSight limitation). For example you cannot stipulate min and max occurrences \d+{1,3}. If shorthand character classes are used (or any other classes that use a \ ) e.g..*\d+\.\d+\.\d+\.\d+.* the rule editor will automatically insert the escaping \ and encapsulate the expression in quotes when the OK button is clicked. Example Search for an IP address in a String Assume event.message = Check for IP in this field Message Matches.*\\d+\\.\\d+\\.\\d+\\.\\d+.* Example Search for the domain in a string Assume event.message = From mark.johnston@accumuli.com to outside Message Matches.*@accumuli.com.* Example Search for usernames matching the pattern 1 character followed by 5 digits Assume event.destinationusername = I12345 Assume event.destinationusername = x12345 Destination User Name Matches [A-Za-z]\\d{5}