For more information, see the Arcsight Locales and Encodings document available from the ArcSight Customer Support download site.

Transcription

1 0BArcSight SmartConnectors and Localization SUMMARY This document provides guidelines and helpful information related to ArcSight SmartConnectors that receive events from devices in languages other than English. This document describes the localization support for various types of parsers used by ArcSight SmartConnectors. For more information, see the Arcsight Locales and Encodings document available from the ArcSight Customer Support download site. WORKING WITH ARCSIGHT PARSERS TO HANDLE LOCALE-SPECIFIC ISSUES ArcSight SmartConnectors use different kinds of parsers. Guidelines using examples of key-value and regex parsers to handle events from devices in languages other than English are provided. This information is provided to explain both what is currently supported and what can be tailored for the parser in order to meet further localization needs for a particular language environment. Key Value Parsers ArcSight SmartConnectors that use key-value parsers provide a mechanism that can be used by end users to parse events coming from localized devices. Example: Assume the contents of a key-value parser are as follows and the name of the key that we are interested in is User: key.delimiter=&& key.value.delimiter== key.regexp=([^&=]+) token.count=1 token[0].name=user additionaldata.enabled=true event.destinationusername=user Case 1: Assume that the event is such that the key is in English and the value is also in English. {English key}={english value} or User=Mary In this case, the connector should work without any further changes and this event will be successfully parsed, because the contents of both the key and its value are in English. ArcSight SmartConnectors and Localization Technical Note 1

2 Case 2: Now assume that the event is such that the key is in English but the value is in a foreign language, say Chinese. {English key}={chinese value} or User={Chinese for Mary } In this case, the connector will be able to parse the values, because it already supports double bytes. However, for the data of the value to be displayed in the appropriate foreign language, make the following configuration changes: 1. Identify the character set or encoding of your values that are appearing in the foreign language. 2. In the agent.properties file, set the appropriate property to that character set. This property name is different for different types of connectors, so set the property accordingly. These properties apply to FlexConnectors as well. The property currently exists for the SNMP connector. As an example, for SNMP connectors you would add the following line in agent.properties: snmp.charset={your character set for the foreign language} 3. Restart the connector. Just because the value Mary is in Chinese, it does not mean that the event is coming from a localized device. It means that the device supports internationalization, but by itself it is not localized, because otherwise the key User would have also have appeared in Chinese. Case 3: Now, assume that the event is such that the key is in a foreign language, say Chinese, and the value is in English. {Chinese key}={english value} or {Chinese for User }= Mary This is an example of an event coming from a localized device, because the key User is not in English but it is localized in Chinese by the device. Here the connector will not be able to identify the key User, so all the information will go to additional data (because additional data has been enabled). So in this case, follow the configuration steps for reverse localization as follows: 1. Identify the character set or encoding of your events that are appearing in the foreign language 2. Contact ArcSight Customer Support for the key-value parser file for your connector, along with the name of the parent directory of the parser file. 3. Create a new file with the following name format: {ParserName}.{language id/locale id}.l10n.properties in the following directory: %ARCSIGHT_SMARTCONNECTORS_HOME%\user\agent\fcp\{Parent Directory name of connector identified in step 2} You can locate your language ID / locale ID from the table in the section "Localization of Date Formats." ArcSight SmartConnectors and Localization Technical Note 2

3 For example, for the Windows Event Log connector, create the file nt_advanced.ja.l10n.properties in the %ARCSIGHT_SMARTCONNECTORS_HOME%\user\agent\fcp\nt directory. Important: Create this file using the default system encoding of the localized device sending the events. 4. Specify the file header as follows: #!{DeviceEncoding} Example:!SJIS This file header specifies the encoding of the properties file and must match the encoding of the localized device sending the events. The header must: Be the first line in the file Exactly match the default system encoding of the device sending the localized events An example is the encoding for Russian characters. Note that Cyrillic (Russian) characters can be encoded using both UTF-8 as well as windows-1251 character sets. However, it is not necessary that both these encodings work as headers in the properties file. This is because the header must exactly match the device s default system encoding. Also, this exact encoding must be used when creating the properties file, which can most easily be done on the device itself. To detect the device s default system encoding, see "4BDetermining Default Encoding" on page Identify all the English keys in the original parser file provided by Customer Support. Identify the foreign language keys equivalent to the English keys. Add mapping entries in the file using either of the two syntaxes shown below: a. key.{english key}={chinese key} or b. localkey.{chinese key}={english key} If the English key has spaces in it, e.g. 'User Name' then each word in the English key needs to be appended by a trailing '\' character, except for the last word. Note that this is not required for the localized key. key.user\ Name={Chinese key for 'User Name'} If multiple localized keys need to be mapped to one English key, then you can map them as below using syntax (b). Note that only the following syntax will work, and not the other way round using syntax (a). localkey.{chinese key1}={english key} localkey.{chinese key2}={english key} localkey.{chinese key3}={english key} 6. Restart the connector. ArcSight SmartConnectors and Localization Technical Note 3

4 SmartConnector for Microsoft Windows Event Log Connector and Localization Support The SmartConnector for Microsoft Windows Event Log connector uses a key-value parser. ArcSight SmartConnectors provide reverse localization for different languages using the following encodings for the Windows Event Log connector: Language German Japanese Traditional Chinese Simplified Chinese ISO SJIS Big5 GB2312 Encoding A sample localization file for the Windows Event Log connector can be obtained by requesting ArcSight Customer Support. This file can be used as a template as it contains most of the English keys that need to be localized. Regex Parsers ArcSight provides a mechanism that can be used by end users to parse events coming from localized devices for connectors that use regex parsers. Example: Assume that the contents of a regex parser are as follows: regex=user is (\\S+) and machine is (\\S+) token.count=2 token[0].name=username token[1].name=machinename additionaldata.enabled=true event.destinationusername=username event.destinationaddress=machinename Case 1 Assume that the event is such that the keywords User is and and machine is are in English and the values are also in English. {English keywords}{english value} or User is Mary and machine is localhost In this case, the connector should work without any further changes and, because the contents of both the keywords and values are in English, this event will be successfully parsed. Case 2: Now assume that the event is such that the keywords are in English but the values are in a foreign language, say Chinese. {English keywords}{chinese value} or User is {Chinese for Mary } and machine is {Chinese for localhost } ArcSight SmartConnectors and Localization Technical Note 4

5 In this case, the connector will be able to parse the values, since it already supports double bytes. However, for the data of the value to be displayed in the appropriate foreign language, follow these configuration steps: 1. Identify the character set or encoding of your values that are appearing in the foreign language. 2. In the agent.properties file, set the appropriate property to that character set. This property name is different for different types of connectors, so set the property accordingly. These properties apply to FlexConnectors as well. Connector Type SNMP Regex Multiple Folder Follower Property Name snmp.charset agents[x].foldertable[x].encoding As an example, for SNMP connectors you need to add the following line in agent.properties: snmp.charset={your character set for the foreign language} 3. Restart the connector. Just because the value Mary is in Chinese, it does not mean that the event is coming from a localized device. It means that the device supports internationalization, but by itself it is not localized, because otherwise the key User would have also have appeared in Chinese. Case 3: Now, assume that the event is such that the keywords are in a foreign language, say Chinese, and the values are in English. {Chinese keywords}{english value} or {Chinese for User is } Mary {Chinese for and machine is } localhost This is an example of an event coming from a localized device, because the keywords are not in English, but are localized in Chinese by the device. Here the connector will not be able to identify the keywords, and so all the information will go to 'additionaldata' (since additional data has been enabled). In this case, follow these configuration steps: 1. Identify the character set of your events that are appearing in the foreign language. (To detect the device s default system encoding, see "4BDetermining Default Encoding" on page 12.) 2. Contact ArcSight Customer Support for the regex parser file for your connector, along with the name of the parent directory of the parser file. 3. Create another parser file with the same name and contents, and place it in a temporary directory. Create/save this file using the default system encoding of the localized device identified in step 1. An example is the encoding for Russian characters. Note that Cyrillic (Russian) characters can be encoded using both UTF-8 as well as windows-1251 charsets. However, it is not necessary that both these encodings work. ArcSight SmartConnectors and Localization Technical Note 5

6 This is because the parser override file encoding must exactly match the device's default system encoding, which can most easily be done on the device itself. 4. Identify the regular expression starting with the phrase 'regex=' in the new parser file. Replace the regular expression in English with its foreign language equivalent, depending upon which words need to be localized, say in Chinese. Save and close the file. regex={chinese for 'User is'}(\\s+){chinese for 'and machine is'}(\\s+) token.count=2 token[0].name=username token[1].name=machinename additionaldata.enabled=true event.destinationusername=username event.destinationaddress=machinename Note that the mappings remain the same. 5. Install JDK on your system from the following website link: 6. Open a command-prompt window and go to the following directory: {JDK installation directory}\bin 7. List the contents of the directory. You should see a file named: native2ascii.exe 8. The native2ascii tool is used to convert files with native encoded characters to a file with Unicode encoded characters. The syntax of this tool is as follows: native2ascii [options] [inputfile [outputfile]] Execute this tool with the following command: native2ascii encoding {Encoding name of the localized device identified in step 1} {File name with path information of the new parser file created in step 2} {%ARCSIGHT_SMARTCONNECTORS_HOME%\user\agent\fcp\{parent directory name identified in step 2}} Now, your localized parser file has been created. More information regarding the native2ascii tool can be found here: Language/Thenative2asciiTool.htm native2ascii tool for Solaris: native2ascii tool for Windows: ArcSight SmartConnectors and Localization Technical Note 6

7 The list of supported encodings for the native2ascii tool can be found at the following website link: Uhttp://java.sun.com/javase/6/docs/technotes/guides/intl/encoding.doc.htmlU 9. Restart the connector. Sometimes foreign languages such as Chinese or Spanish can have different grammatical versions for the same words. To support multiple mappings, you can create multiple properties files. Localization of Date Formats A number of connector tokens and operations are potentially affected by localization. For example, the Date token can optionally have a format specified, such as MMM or MMMM for month name. Also, the createtimestampbystarttimeelapsed operation expects a date format that includes the month name in MMM format. This date format can vary depending upon the locale. By default, the date is parsed on the basis of the U.S. locale, regardless of the configuration of the locale of the connector machine. So, for such operations to work, the agent.parser.locale.name property in the agent.properties file can be used to configure the locale of the connector to be used by the parser operators, such as createoptionaltimestampfromstring and tokens such as Date. By default, this property value is set to en_us, which means English for the United States. There are three choices: 1. Do nothing do not define this property in agent.properties. Because the agent.default.properties specifies the en_us locale (English for the United States) no matter where in the world the connector is running, the date is parsed according to US locale. 2. Define the locale to be blank in agent.properties. In this case, the platform's default locale is used. Look for the string "Locale is" in the agent.log file to check which locale was actually used. 3. Define the locale to be something other than en_us, such as de_de for German in Germany. If the specified locale is not understood by the SmartConnector, en_us is used and the error is noted in the agent.log file. ArcSight SmartConnectors and Localization Technical Note 7

8 Possible values for this property as of JDK 1.5.0_06 are as follows: ID Name Language Country Variant ar Arabic Arabic ar_ae Arabic (United Arab Emirates) Arabic United Arab Emirates ar_bh Arabic (Bahrain) Arabic Bahrain ar_dz Arabic (Algeria) Arabic Algeria ar_eg Arabic (Egypt) Arabic Egypt ar_iq Arabic (Iraq) Arabic Iraq ar_jo Arabic (Jordan) Arabic Jordan ar_kw Arabic (Kuwait) Arabic Kuwait ar_lb Arabic (Lebanon) Arabic Lebanon ar_ly Arabic (Libya) Arabic Libya ar_ma Arabic (Morocco) Arabic Morocco ar_om Arabic (Oman) Arabic Oman ar_qa Arabic (Qatar) Arabic Qatar ar_sa Arabic (Saudi Arabia) Arabic Saudi Arabia ar_sd Arabic (Sudan) Arabic Sudan ar_sy Arabic (Syria) Arabic Syria ar_tn Arabic (Tunisia) Arabic Tunisia ar_ye Arabic (Yemen) Arabic Yemen be Belarusian Belarusian be_by Belarusian (Belarus) Belarusian Belarus bg Bulgarian Bulgarian bg_bg Bulgarian (Bulgaria) Bulgarian Bulgaria ca Catalan Catalan ca_es Catalan (Spain) Catalan Spain cs Czech Czech cs_cz Czech (Czech Republic) Czech Czech Republic da Danish Danish da_dk Danish (Denmark) Danish Denmark ArcSight SmartConnectors and Localization Technical Note 8

9 ID Name Language Country Variant de German German de_at German (Austria) German Austria de_ch German (Switzerland) German Switzerland de_de German (Germany) German Germany de_lu German (Luxembourg) German Luxembourg el Greek Greek el_gr Greek (Greece) Greek Greece en English English en_au English (Australia) English Australia en_ca English (Canada) English Canada en_gb English (United Kingdom) English United Kingdom en_ie English (Ireland) English Ireland en_in English (India) English India en_nz English (New Zealand) English New Zealand en_us English (United States) English United States en_za English (South Africa) English South Africa es Spanish Spanish es_ar Spanish (Argentina) Spanish Argentina es_bo Spanish (Bolivia) Spanish Bolivia es_cl Spanish (Chile) Spanish Chile es_co Spanish (Colombia) Spanish Colombia es_cr Spanish (Costa Rica) Spanish Costa Rica es_do Spanish (Dominican Republic) Spanish Dominican Republic es_ec Spanish (Ecuador) Spanish Ecuador es_es Spanish (Spain) Spanish Spain es_gt Spanish (Guatemala) Spanish Guatemala es_hn Spanish (Honduras) Spanish Honduras es_mx Spanish (Mexico) Spanish Mexico es_ni Spanish (Nicaragua) Spanish Nicaragua ArcSight SmartConnectors and Localization Technical Note 9

10 ID Name Language Country Variant es_pa Spanish (Panama) Spanish Panama es_pe Spanish (Peru) Spanish Peru es_pr Spanish (Puerto Rico) Spanish Puerto Rico es_py Spanish (Paraguay) Spanish Paraguay es_sv Spanish (El Salvador) Spanish El Salvador es_uy Spanish (Uruguay) Spanish Uruguay es_ve Spanish (Venezuela) Spanish Venezuela et Estonian Estonian et_ee Estonian (Estonia) Estonian Estonia fi Finnish Finnish fi_fi Finnish (Finland) Finnish Finland fr French French fr_be French (Belgium) French Belgium fr_ca French (Canada) French Canada fr_ch French (Switzerland) French Switzerland fr_fr French (France) French France fr_lu French (Luxembourg) French Luxembourg hi_in Hindi (India) Hindi India hr Croatian Croatian hr_hr Croatian (Croatia) Croatian Croatia hu Hungarian Hungarian hu_hu Hungarian (Hungary) Hungarian Hungary is Icelandic Icelandic is_is Icelandic (Iceland) Icelandic Iceland it Italian Italian it_ch Italian (Switzerland) Italian Switzerland it_it Italian (Italy) Italian Italy iw Hebrew Hebrew iw_il Hebrew (Israel) Hebrew Israel ArcSight SmartConnectors and Localization Technical Note 10

11 ID Name Language Country Variant ja Japanese Japanese ja_jp Japanese (Japan) Japanese Japan ko Korean Korean ko_kr Korean (South Korea) Korean South Korea lt Lithuanian Lithuanian lt_lt Lithuanian (Lithuania) Lithuanian Lithuania lv Latvian Latvian lv_lv Latvian (Latvia) Latvian Latvia mk Macedonian Macedonian mk_mk Macedonian (Macedonia) Macedonian Macedonia nl Dutch Dutch nl_be Dutch (Belgium) Dutch Belgium nl_nl Dutch (Netherlands) Dutch Netherlands no Norwegian Norwegian no_no Norwegian (Norway) Norwegian Norway no_no_ny Norwegian (Norway,Nynorsk) Norwegian Norway Nynorsk pl Polish Polish pl_pl Polish (Poland) Polish Poland pt Portuguese Portuguese pt_br Portuguese (Brazil) Portuguese Brazil pt_pt Portuguese (Portugal) Portuguese Portugal ro Romanian Romanian ro_ro Romanian (Romania) Romanian Romania ru Russian Russian ru_ru Russian (Russia) Russian Russia sk Slovak Slovak sk_sk Slovak (Slovakia) Slovak Slovakia sl Slovenian Slovenian sl_si Slovenian (Slovenia) Slovenian Slovenia ArcSight SmartConnectors and Localization Technical Note 11

12 ID Name Language Country Variant sq Albanian Albanian sq_al Albanian (Albania) Albanian Albania sv Swedish Swedish sv_se Swedish (Sweden) Swedish Sweden th Thai Thai th_th Thai (Thailand) Thai Thailand th_th_th Thai (Thailand,TH) Thai Thailand TH tr Turkish Turkish tr_tr Turkish (Turkey) Turkish Turkey uk Ukrainian Ukrainian uk_ua Ukrainian (Ukraine) Ukrainian Ukraine vi Vietnamese Vietnamese vi_vn Vietnamese (Vietnam) Vietnamese Vietnam zh Chinese Chinese zh_cn Chinese (China) Chinese China zh_hk Chinese (Hong Kong) Chinese Hong Kong zh_tw Chinese (Taiwan) Chinese Taiwan 4BDETERMINING DEFAULT ENCODING To determine the default encoding of the system on which the connector is running when your localized device and connector are installed on the same system, you can search the agent.log file for the following text: Default platform character encoding is [Cp1252], will use [UTF8] The text xxxx within [xxxx] represents the platform encoding; for example, Cp1252. If the default platform character encoding is Cp1252, you do not need to specify the device encoding in the parser file header. Also, you can create/save the localized parser file using ANSI encoding on the system itself. ArcSight SmartConnectors and Localization Technical Note 12

13 Last Reviewed: 08/26/2008 Keywords: SmartConnector, Localization This technical note contains confidential information proprietary to ArcSight, Inc. Any party accepting this document agrees to hold its contents confidential, except for the purposes for which it was intended. Copyright ArcSight, Inc. All rights reserved. ArcSight, the ArcSight logo, ArcSight TRM, ArcSight NCM, ArcSight Enterprise Security Alliance, ArcSight Enterprise Security Alliance logo, ArcSight Interactive Discovery, ArcSight Pattern Discovery, ArcSight Logger, FlexConnector, SmartConnector, SmartStorage and CounterACT are trademarks of ArcSight, Inc. All other brands, products and company names used herein may be trademarks of their respective owners. Follow this link to see a complete statement of ArcSight's copyrights, trademarks and acknowledgements: The network information used in the examples in this document (including IP addresses and hostnames) is for illustration purposes only. This document is ArcSight Confidential. ArcSight SmartConnectors and Localization Technical Note 13