Lcatin f the map.x.prperties files $ARCSIGHT_HOME/current/user/agent/map File naming cnventin The files are named in sequential rder such as: Sme examples: 1. map.1.prperties 2. map.2.prperties 3. map.3.prperties The fllwing examples refer t the frmat f field names within prperties files. The frmat applies t every field in the ArcSight event schema. The name f each field is btained frm the grid within the ArcSight Cnsle. Yu can als search the Cnsle help t find a list f all f the event field names. Pay clse attentin t the capitalizatin in the prperties file. Field name in the grid Field name in the prperties file Name event.name Surce Address event.surceaddress Destinatin Prt event.destinatinprt Raw Event event.rawevent Device Custm Number 1 event.devicecustmnumber1 The fllwing example illustrates the frmat f the map files. The frmat fllws an "if, then" prcess. The first line declares the fields t be evaluated and mdified and the secnd line cntains the data. event.deviceeventclassid,set.event.categryobject,set.event.categrysignificance 10002,/Netwrk,/Hstile This basically declares, if the device event class id is 10002 then set the categry bject field t /Netwrk and the categry significance field t /Hstile. Sme general infrmatin: Is there a COMMENT flag that can be used within these cnfiguratin files (e.g. #, //, /** **/, etc...)? This file des nt have supprt fr cmments. Can multiple mappings be included per cnfiguratin file r are mappings file specific?
Multiple mappings can be used in a single file as lng as each mapping is n a separate line. Is there a LIMIT n the number f map.x.prperites that can be placed n a SmartAgent? Nt a hard limit. Each file crrespnds t a lkup that gets executed fr each event s it culd eventually affect perfrmance. I believe 1-10 files shuld nt be a prblem thugh. If there is a cnflict in data between tw map.x.prperty files, des ne f the mappings always take precedence (map.n.prperties r map.n+1.prperties) r is the result unpredicable? map.n+1.prperties takes precedence if there are cnflicts. If there are multiple matches in the same file nly the first ne will be used. Map Files Overview Map files are used by several Agent cmpnents (AgentInfAdder1, AgentNATPrcessr, and the categrizers) as well as map type extra prcessrs t set the values in ne r mre event fields based n the values in ther event fields (referred t as the "setter" and "getter" fields, respectively). The mapping data is stred in a CSV (cmma separated value) frmat as described in mre detail belw, althugh nt always in files with a.csv extensin. The lcatin and naming f the actual files varies depending n which Agent cmpnent is using it. Frmat As stated abve, the frmat is CSV. The first line (ignring cmment lines, which start with a pund sign (#)) designates which event fields are "setters" and which are "getters." Fr example: event.destinatinprt,set.event.applicatinprtcl In this example, there is ne "getter" field (destinatinprt) and ne "setter" field (applicatinprtcl). Later lines in the file must have the same number f clumns as the first line, and the values are assciated with the fields indicated in the crrespnding clumns in the header line. Using the same example as abve, the data culd be: 20,ftp 21,ftp 80,http 110,pp3 Nte that there is a duplicate in the "setter" clumn, which is allwed, but nt in the "getter" clumn, which wuld nt be. Nte als that additinal data "setters" are allwed, in the frm set.additinaldata.additinaldataname, but additinal data "getters" are nt.
Operatin When an event cmes in, the "getter" fields are extracted and lked up. If there is a match, then the values fr the "setter" fields are set. That is, they are if they are null r the cde is cnfigured t allw verwriting. If the cde is set t disallw verwriting, then a nn-null value in a "setter" field will be left unchanged. In the example shwn abve, if the event had 80 in the destinatinprt field and null in applicatinprtcl, then the applicatinprtcl field wuld be set t "http". If the event had 81 in the destinatinprt field, then applicatinprtcl wuld be unchanged since there is n match fr 81. Besides verwriting, ther cnfigurable bits in the cde include: * whether the "getters" are case sensitive r insensitive (nrmally case insensitive) * whether any leading r trailing whitespace characters are trimmed frm the "getter" values (nrmally trimmed) * whether any leading r trailing whitespace characters are trimmed frm the "setter" values (nrmally trimmed) * whether duplicate "getter" values are treated as warnings r as fatal errrs (nrmally as warnings) The underlying cde uses a HashMap, which means that the lkup is extremely fast even if the table is quite large, but als means that the entire table is kept in memry. Range "Getters" Besides simple "getter" fields, range "getters" are als allwed fr integer, lng, IP address, and MAC address fields. S rather than having smething like this (which wuld be 513 lines if it was shwn in its entirety): event.surceaddress,set.event.flexnumber1 1.1.0.0,0 1.1.0.1,0 1.1.0.2,0 1.1.0.3,0... 1.1.0.255,0 1.1.1.0,1 1.1.1.1,1 1.1.1.2,1 1.1.1.3,1... 1.1.1.255,1 Yu can instead have these three lines: range.event.surceaddress,set.event.flexnumber1 1.1.0.0-1.1.0.255,0 1.1.1.0-1.1.1.255,1 Nte als that lng numbers with the high-rder bit set must be entered as negative numbers, leading t ranges such as -9223372036854775808--1 (which is all f the values with the high-
rder bit set). MAC address ranges are entered in the frm hh:hh:hh:hh:hh:hh-hh:hh:hh:hh:hh:hh, where each hh is a ne r tw-digit hexedecimal number. Overlapping ranges It might nt be gd t depend n hw verlapping ranges wrk, but this sectin dcuments hw they wrk nw. Fr each range, lk at hw many high-rder bits match. Fr example, fr 1.1.1.0-1.1.1.255, the high-rder 24 bits all match. Ranges with mre matching bits are mre specific, and s they are chsen ver ranges with fewer matching bits. Fr example, 1.1.1.0-8.0.0.0 nly has 4 matching bits, s when lking fr IP address 1.1.1.128, the 1.1.1.0-1.1.1.255 range is chsen ver the 1.1.1.0-8.0.0.0 range even thugh bth ranges cntain the IP address. Cnsider ranges 1.1.1.0-8.0.0.255 and 1.1.1.0-10.255.255.255. Bth have 4 matching high-rder bits and the same value fr thse bits (zer). S they cllide and are stred tgether. In cases like this, the rder that the clliding ranges are fund in the map file matters, and if an IP address matches mre than ne f the clliding ranges, the first ne that includes the IP address will be chsen. All f these examples use IP addresses, but the same principles apply t integers, lngs, and MAC address. Regular Expressin "Getters" [This feature will be in the Agent release that cmes ut at the end f May r early June.] Regular expressin "getters" can be used n any string field. Here is an example: regex.event.surceusername,set.event.flexstring1.*?@arcsight.cm,arcsight.*?@micrsft.cm,micrsft In this example, if the surce user name field ends in "@arcsight.cm" r "@micrsft.cm", then the flex string 1 field will be set t "ArcSight" r "Micrsft", respectively. Any regular expressins supprted by the java.util.regex package are supprted. Regular expressins can easily "verlap," and in this event the first match will be chsen (unless there are range "getters" after the regular expressin "getters" -- see the Cmbining Range and Regular Expressin "Getters" sectin belw). There can als be mre than ne regular expressin clumn, which wrks as it shuld, namely that the first line where all f the clumns match, if any, will be chsen. Nte that the cde is mre efficient than yu might expect, nly having t evaluate ne cmbined regular expressin in the mst cmmn single clumn case. Cmbining Range and Regular Expressin "Getters" If a map file cntains bth a range "getter" and a regular expressin "getter," the rder f the clumns can matter. Generally it is better t have the first range "getter" precede the first regular expressin "getter," fr perfrmance. The reasn is that if the regular expressin is first and there is mre than ne regular expressin match, the range "getter" lkup must be repeated fr each regular expressin match until ne wrks r there are n mre, whereas if the range is first, the regular expressin cde will nly need t lk fr ne answer, picking the first ne.
Hwever, if there are verlapping ranges fr the range "getter," then putting the regular expressin befre the range may generate a match were the ppsite rder might nt. Fr example, cnsider this map file: range.event.surceaddress,regex.event.flexstring1,set.event.message 1.1.1.0-1.1.3.255,f.*,Map file 1 was here 1.1.1.0-1.1.3.255 f 1.1.3.0-1.1.3.0,f.*,Map file 1 was here 1.1.3.0-1.1.3.0 f 1.1.1.0-2.2.2.255,bar.*,Map file 1 was here 1.1.1.0-2.2.2.255 bar If the surce address field is 1.1.3.0, that matches all three ranges, but the cde will currently pick the 1.1.3.0-1.1.3.0 ne. But if the flex string 1 field is "bar", the map file will nt match since the 1.1.3.0-1.1.3.0 range has been chsen and there is n regular expressin matching "bar" fr that range. On the ther hand, cnsider a map file that lks like this: regex.event.flexstring1,range.event.surceaddress,set.event.message f.*,1.1.1.0-1.1.3.255,map file 1 was here 1.1.1.0-1.1.3.255 f f.*,1.1.3.0-1.1.3.0,map file 1 was here 1.1.3.0-1.1.3.0 f bar.*,1.1.1.0-2.2.2.255,map file 1 was here 1.1.1.0-2.2.2.255 bar In this case the regular expressin makes the nly valid match fr "bar", leaving nly a single range t match (which des in fact match). S there is a match where there was nt with the ther clumn rder. But again, if the ranges d nt verlap, putting the ranges first will give slightly better perfrmance. The "N Getter" Trick Smetimes yu just want t set a field t a value, regardless. It turns ut that yu can have a mapper with n "getter" fields at all, which means the "setter" fields will be set fr all events (mdul that verwrite thing). In this case, the file is always tw lines lng, like in this example: set.event.message Map file was here