OpenSignature User Guidelines

Transcription

1 June 28, 2008 Overview Introduction The OpenSignature feature uses a flexible rules language that allows you to write customized, pattern-matching intrusion detection signatures to detect threats that are not detected by IBM ISS Intrusion Prevention System (IPS) products. Benefits Use the Open Signature feature to set up the following: Audit signatures for Layer 6 and 7 applications that are specific to your environment Signatures that name a specific attack variant for customized reporting purposes (instead of relying on the IBM ISS vulnerability protection, which uses a generic name for many threats) Requirements The OpenSignature feature is integrated into the IBM ISS Protocol Analysis Module (PAM) as a rule interpreter. OpenSignature relies on PAM and the IBM ISS intrusion prevention product you have installed on your network in order to work. Supported agents You can use the OpenSignature feature with the following agents: Proventia Intrusion Prevention Appliance versions 1.2 and later Proventia Desktop RealSecure Network Sensor Important: These guidelines assume that you are managing these agents through the SiteProtector Console, and that you are not manually editing configuration files. Syntax information Proper syntax in essential for well-constructed OpenSignature rules. The following guidelines are available: General Syntax on page 9 Content Keyword Modifiers on page 10 PCRE and Post-PCRE Keyword Modifiers on page 12 Optional Keyword Modifiers on page 14 IBM Internet Security Systems 1

2 Important Considerations Introduction The OpenSignature feature is flexible and you can use it to create rules for many purposes. However, that flexibility makes it possible for you to create rules that you did not intend to create. Important IBM ISS does not guarantee agent performance when you use OpenSignature rules. Use care when you create a rule Be careful when you create a new rule. Poorly written rules can impact agent performance or have other consequences. These consequences include, but are not limited to, the following: Causing PAM to run in a loop (thereby causing PAM to stop responding) Blocking all network traffic to that segment (inline mode, with or without a bypass) Having to reinstall the agent software locally Prevention or detection You can use OpenSignature rules for blocking only with Proventia Intrusion Prevention appliances. For Proventia Desktop and RealSecure Network Sensor, OpenSignature rules can detect events, but does not block them. Support limitations IBM ISS Customer Support cannot help you write or troubleshoot custom rules for your environment. If you need assistance to create custom signatures, please contact IBM ISS Professional Services. A poorly written signature can cause a performance issue that may not appear to be related to the signature. Customer Support may ask you to disable the OpenSignature feature, and then reproduce the performance issue, before troubleshooting the issue. 2

3 Enabling the OpenSignature Feature Enabling the OpenSignature Feature Introduction The parser that interprets your signature rules is disabled by default. To enable this parser, you must add a custom tuning parameter to a SiteProtector policy, and then apply the policy to your agent. This topic includes the information you need in order to add the custom parameter. Enabling OpenSignature for Proventia Network IPS GX6000 series appliances 1. On the Global Tuning Parameter page, click Add. 2. Complete or change the settings as indicated in the following table. engine.opensignature.enabled Value true Enabling OpenSignature for all Proventia Network IPS appliances (other than GX6000 series) 1. On the Global Tuning Parameter page, click Add. 2. Complete or change the settings as indicated in the following table. pam.trons.enabled Value true Enabling OpenSignature for Proventia Desktop 8.0 or In SiteProtector, go to the Proventia Desktop policy editor. 2. In Network Protection, select Default Settings Intrusion Prevention Settings Custom IPS parameters. 3. Click Add. 4. Complete or change the settings as indicated in the following table. Value pam.trons.enabled true Enabling OpenSignature for Proventia Desktop In SiteProtector, go to the Proventia Desktop policy editor. 2. In the Security Events Default Settings, go into Advanced Configuration. 3. Click Add. 4. Complete or change the settings as indicated in the following table. Value pam.trons.enabled true 3

4 Enabling OpenSignature for Network Sensor 1. In the SiteProtector Console, select the Sensor tab. 2. Right-click your Network Sensor, and then select Edit Properties. 3. Select the Advanced Parameters tab. 4. Click Add. 5. Complete or change the settings as indicated in the following table. Type Value pam.trons.enabled Boolean true Type a description for the rule Enabling OpenSignature for Network Sensor by policy 1. In the SiteProtector Console, select the Sensor tab. 2. Select the Network Sensors for which you want to apply the policy, and then select Apply Policy. 3. Select the Custom policy. 4. Select the X-Press Update tab within the policy editor. 5. Select the group within the XPU. 6. Click the Tuning button. 7. Click Add. 8. Complete or change the settings as indicated in the following table. Type Value pam.trons.enabled Boolean true Type a description for the rule 4

5 Adding an OpenSignature Rule Adding an OpenSignature Rule Introduction The procedure for adding a custom rule varies from product to product. This section contains the information you need in order to add OpenSignature rules for each product. Syntax information Use the following syntax information as you create your rules: General Syntax on page 9 Content Keyword Modifiers on page 10 PCRE and Post-PCRE Keyword Modifiers on page 12 Optional Keyword Modifiers on page 14 Adding a rule for Proventia Network IPS appliances 1. Go to the OpenSignature section in Proventia Manager. 2. Complete or change the settings as indicated in the following table. Enable Comments Rule String Event Throttling Select the check box to enable the rule. Type a description for the rule. Type the text string that tells the appliance when an event is triggered. Type an interval value in seconds. At most, one event that matches an attack is reported during the interval you specify. A value of 0 (zero) disables event throttling. Adding a rule for Proventia Desktop 8.0 or In SiteProtector, go to the Proventia Desktop policy editor. 2. In Network Protection, select Default Settings Intrusion Prevention Settings Custom IPS parameters. 3. Click Add. 4. Complete or change the settings as indicated in the following table. Value pam.trons.rules.n where n is an integer value. This value must be unique if you are entering multiple OpenSignature rules. pam.trons.rules.1, pam.trons.rules.2, pam.trons.rules.100 Type the rule. alert tcp /24 any (msg: This rule triggered on html ; content: html ; nocase; sid:10;) 5

6 Adding a rule for Proventia Desktop In SiteProtector, go into the Proventia Desktop 10.0 policy editor. 2. In the Security Events Default settings, go to Advanced Configuration. 3. Click Add. 4. Complete or change the settings as indicated in the following table. Value pam.trons.rules.n where n is an integer value. This value must be unique if you are entering multiple OpenSignature rules. pam.trons.rules.1, pam.trons.rules.2, pam.trons.rules.100 Type the rule. alert tcp /24 any (msg: This rule triggered on html ; content: html ; nocase; sid:10;) Adding a rule for Network Sensor 1. In the SiteProtector Console, select the Sensor tab. 2. Right-click the Network Sensor, and then select Edit Properties. 3. Select the Advanced Parameters tab. 4. Click Add. 5. Complete or change the settings as indicated in the following table. Type Value pam.trons.rules.n where n is an integer value. This value must be unique if you are entering multiple OpenSignature rules. s pam.trons.rules.1, pam.trons.rules.2, pam.trons.rules.100 Note: You can enter more than one rule at a time. You must encode multiple rules in BASE64 format, and then enter the rules as a single line of text. SiteProtector allows a maximum of 10,000 bytes of text in a field, which limits the number of rules that you can enter in each block. Select String. Type the rule. alert tcp /24 any (msg: This rule triggered on html ; content: html ; nocase; sid:10;) Type a description for the rule. 6

7 Adding an OpenSignature Rule Adding a rule for multiple Network Sensor installations 1. In the SiteProtector Console, click the Sensor tab. 2. Select the sensors, and then select Apply Policy. 3. Select the Custom policy. 4. Select the X-Press Update Tab within policy editor. 5. Select the group within the XPU. 6. Click the Tuning button. 7. Click Add. 8. Complete or change the settings as indicated in the following table. Type Value pam.trons.rules.n where n is an integer value. This value must be unique if you are entering multiple OpenSignature rules. s pam.trons.rules.1, pam.trons.rules.2, pam.trons.rules.100 Note: You can enter more than one rule at a time. You must encode multiple rules in BASE64 format, and then enter the rules as a single line of text. SiteProtector allows a maximum of 10,000 bytes of text in a field, which limits the number of rules that you can enter in each block. Select String. Type the rule. alert tcp /24 any (msg: This rule triggered on html ; content: html ; nocase; sid:10;) Type a description for the rule. 7

8 Setting up responses for OpenSignature rules Introduction You can use the Central Responses feature in SiteProtector to set up responses for OpenSignature rules. These responses include SNMP, SMTP, Log Evidence, and User Specified events. For detailed information about using this feature, see the SiteProtector Users Guide for Security Managers. Note: If you format an OpenSignature rule incorrectly, SiteProtector displays a PAM configuration error in the Sensor Analysis view. Setting responses on the Proventia Network IPS appliances You need to add some Global Tuning Parameters in order to set up blocking with an OpenSignature rule on the Proventia Network IPS appliances. alert tcp any any -> any any (msg:"yahoo accessed"; content:"yahoo"; nocase; sid:5000;) In this example, is the issue ID that you would see in the Alerts when the rule triggers. Use this issue ID in Global Tuning Parameters when you set the response. To set the block response for this sample rule, set the following values in the Global Tuning Parameters section of Proventia Manager: np.vs.0.issue =on np.vs.0.issue response=drop-packet,reset-intruder,resetvictim Use this approach to set the block response for any OpenSignature rules. Remember to use your rule ID instead of the ID ( ) provided in the example. Setting responses for Network Sensor You can assign responses to the OpenSignature rule by adding a response parameter in the sensor properties or in the policy using the issue.issue_id.response parameter. Use the issue ID that is created from the ID assigned in the rule. (See Rule identifier on page 9 for information about issue IDs are created.) 1. Open sensor properties or the policy you want to modify. 2. Complete or change the setting as indicated in the following table. Type the following value: issue.issue_id.response where issue_id is the issue ID created by the system (by adding 6,000,000 to the one to four digit identifier you select). Reference: The Advanced Tuning Parameters Reference document is available in the following location: Refer to the last section called Assigning Responses to an Issue for specific information on the responses for Network Sensor. 8

9 General Syntax General Syntax Introduction This topic covers general syntax for custom rules. Rule identifier Each rule requires a unique 1 to 4 digit identifier. The rules engine adds 6,000,000 to the identifier you provide in order to create a custom issue ID. For example, an identifier of 5000 would be assigned an issue ID of Syntax options General syntax options for custom rules are as follows: <action>: alert <protocol>: tcp, udp, icmp, ip <IP and netmask>: single IP address (a.b.c.d), range of IP addresses (a.b.c.dw.x.y.z), network address using CIDR notation (a.b.c.0/24) Complete syntax Use the following syntax for custom rules: <action> <protocol> <IP_and_netmask> <port> <direction_operator> <IP_and_netmask> <port> (<option1 keyword:argument>;<option2 keyword:argument>;) alert tcp any any -> any any (msg:"yahoo accessed"; content:"yahoo"; nocase; sid:5000;) In this example, the issue ID that appears in the Alerts is The negation operator An exclamation mark (!) is a negation operator. alert tcp! /24 In this example, the rule will trigger an alert when a tcp request comes from anywhere other than the specified IP address or range. 9

10 Content Keyword Modifiers Introduction This topic contains information you need in order to set up OpenSignature rules using content keyword modifiers. The content keyword The content keyword allows you to set rules that search for specific content in the packet payload, and also triggers a response based on the data provided. All OpenSignature rules require the content keyword. alert tcp any any -> any any (msg: Access google ; content: google ; nocase; sid:1000;) Optional data Option data for the content keyword can contain both text and binary data. Binary data is generally enclosed within the pipe character ( ) and is displayed as bytecode. Bytecode represents binary data as hexadecimal numbers. alert tcp any any -> any any (msg: Search google in binary form ; content: 77 2E 67 6f 6F 67 6c 65 ;nocase;sid:1000;) Content keyword modifiers Content specific keywords modify the content keyword. You must use a content keyword in the rule before you can use a content keyword modifier. Keyword Priority Depth Sets the priority of the alert to high, medium, or low. priority:high; priority:medium; priority:low; alert tcp any any -> any any (msg: Alert-found content containing the word BLUE ;content: blue ;priority:low; sid:5000;) Specifies how deep to search a packet to for the specified pattern. The depth keyword modifies the previous content keyword in the rule. A depth of 10 would look for the specified pattern within the first 10 bytes of the payload. The depth keyword modifies the previous content keyword, so a content keyword must be present in the rule before the depth is specified. depth:<number>; alert tcp any any -> any any (msg: Search for the depth of payload of google ;content: f 6e ;depth:10;sid:1000;) Table 1: Content keyword modifiers 10

11 Content Keyword Modifiers Keyword Offset Distance Within Nocase Specifies where to start searching for a pattern within a packet. The offset keyword modifies the previous content keyword in the rule. An offset of 10 would look for the specified pattern after the first 10 bytes of the payload. offset:<number>; alert tcp any any -> any any (msg: offset in payload of google ;content: google ;offset:20;sid:1000;) Specifies how far to go into a packet before searching for the specified pattern relative to the end of the previous pattern match. The distance keyword is similar to the depth keyword, but the distance is relative to the end of the last pattern match, not to the beginning of the packet. distance:<byte count>; alert tcp any any -> any any (msg: distance specified between content when accessing google ;content: google ;content: Content-Encoding ;distance:150;sid:1000;) Sets the maximum number of bytes allowable between pattern matches that use the content. It is used in conjunction with the distance rule option. within:<byte count>; alert tcp any any -> any any (msg: Search for the string in payload ;content: google ;content: 43 6f 6e e ;distance:64;within:30;sid:1000 Searches for the specific pattern without being case-sensitive. The nocase keyword modifies the previous content keyword in the rule. nocase; alert tcp any any -> any any (msg: Alert when access google ; content: google ;nocase;sid:1000;) Table 1: Content keyword modifiers (Continued) 11

12 PCRE and Post-PCRE Keyword Modifiers Introduction This topic describes rules that use PCRE keyword modifiers. The PCRE keyword The PCRE keyword allows you to use perl-compatible regular expressions to write rules. For details about PCRE, review the PCRE Web site at pcre:[!] (/<regex>/ m<delim><regex><delim>)[ismxaegrub] ; Rules that use PCRE examples The following examples show rules that use the PCRE keyword: alert ip any any -> any any (pcre: /FOO/i ;) alert tcp any any -> any any (pcre: /GET.*\.htm/i;) alert tcp any any -> any 80 (msg: Google Image Search ; pcre: / Host\:\simages.google.com\r\n/ism ; sid:1000;) Rules that use both PCRE and non-pcre keywords The following examples show rules that use both the PCRE keyword and non-pcre keywords: alert tcp any any -> any 80 (msg: Gator Agent Traffic ;content:! User- Agent\: Akregator ; pcre: /User-Agent\:[^\n]+Gator/i ;sid: 1000;) alert tcp any any -> any 80 (msg: Yahoo Mail Login ; content: yahoo ; pcre: /(Host\:)\s[a-zA-Z0-9.-]+(\.mail.yahoo.com\r\n)/ism ;sid:1000;) Rules for post-pcre modifiers Post-PCRE modifiers set compile time flags for the regular expression. The following table lists the rules for the post-pcre modifiers: Rule A E G i m s The pattern must match only at the start of the buffer (same as ^). Set $ to match only at the end of the subject string. Without E, $ also matches immediately before the final character if it is a newline (but not before any other newlines). A greedy algorithm dictates that the system should make the locally optimum choice at each stage of processing. The modifier inverts the greediness of the quantifiers so that they are not greedy by default, but become greedy if followed by?. Case-insensitive. By default, the string is treated as one line of characters. ^ and $ match at the beginning and ending of the string. When m is set, ^ and $ match immediately following or immediately before any newline in the buffer, as well as the very start and very end of the buffer. Includes newlines in the dot metacharacter Table 2: Rules for post-pcre modifiers 12

13 PCRE and Post-PCRE Keyword Modifiers Rule x Whitespace data characters in the pattern are ignored except when escaped or inside a character class Table 2: Rules for post-pcre modifiers (Continued) 13

14 Optional Keyword Modifiers Introduction This topic describes the optional keyword modifiers that you can use to configure OpenSignature rules. s Optional keyword modifiers are defined in the following table: Keyword ack byte_jump byte_test Dsize Checks for a specific TCP acknowledge number. <number>: alert tcp any any -> any (msg: ACK ; content: 0 ; ack:10;sid:1000;) Allows rules that skip over specific portions of length-encoded protocols and to detect threats in specific locations. byte_jump: <bytes_to_convert>, <offset> \ [,relative] [,multiplier <multiplier value>] [,big] [,little][,string]\ [,hex] [,dec] [,oct] [,align] [,from_beginning]; alert tcp any any -> any any (msg: Byte Jump ;content: HTTP ;byte_jump:1,12,relative ;byte_test:4,=, ,8,relative; sid:1000;) Tests a byte field against a specific value (with an operator). Can test binary values or convert representative byte strings to their binary equivalent and test them. byte_test: <bytes to convert>, [!]<operator>, <value>, <offset> \ [,relative] [,<endian>] [,<number type>, string]; alert tcp any any -> any any (msg: Byte Test ;content: HTTP/ 1.1 ;byte_test:4,=, ,27,relative;sid :1000;) Tests the packet payload size. dsize: [<>]<number>[<><number>]; alert tcp any any any any (msg: This rule triggered on packet payload size ; content: html ; nocase; dsize:300<>400; sid:1000;) Table 3: Optional keyword modifier descriptions 14

15 Optional Keyword Modifiers Keyword Flags Flow Check for specific TCP flag bits. The following bits may be checked: F - FIN (LSB in TCP Flags byte) S - SYN R - RST P - PSH A - ACK U - URG flags:[! * +]<FSRPAU>[,<FSRPAU>]; alert tcp any any any any (msg: This rule triggered on SYN and FIN flags ; content: html ; nocase; flags:sf,12; sid:1000;) Allows rules to apply only to certain directions of traffic flow in client-server system configurations. This option is used in conjunction with TCP stream reassembly. Options are as follows: to_client - Trigger on server responses from A to B to_server - Trigger on client requests from A to B from_client - Trigger on client requests from A to B from_server - Trigger on server responses from A to B established - Trigger only on established TCP connections stateless - Trigger regardless of the state of the stream processor (useful for packets that are designed to cause computers to crash) no_stream - Do not trigger on rebuilt stream packets (useful for dsize and stream4) only_stream - Only trigger on rebuilt stream packets flow: [(established stateless)] [,(to_client to_server from_client from_serv er)][,(no_stream only_stream)] alert tcp any any -> any 80 (msg Web server traffic detected ; content: html ; nocase; flow: to_server; sid:1000;) Table 3: Optional keyword modifier descriptions (Continued) 15

16 Keyword Flowbits Fragbits Icmp_id Allows rules to track states across transport protocol sessions. Used in conjunction with conversation tracking from the Flow preprocessor. Options are as follows: set - Sets the specified state for the current flow. unset - Unsets the specified state for the current flow. toggle - Sets the specified state if the state is unset, otherwise unsets the state if the state is set. isset - Checks if the specified state is set. isnotset - Checks if the specified state is not set. noalert - Cause the rule to not generate an alert, regardless of the rest of the detection options. flowbits: [set unset toggle isset,reset,noalert][,<sta TE_NAME>]; alert tcp any 143 -> any any (msg: IMAP login detected ; content: login ; nocase; flowbits: set,logged_in; sid:1000;) Checks for fragmentation and reserved bits in the IP header. The following bits may be checked: M - More Fragments D - Don't Fragment R - Reserved Bit The following modifiers can be set to change the match criteria + means match on the specified bits, plus any others * - means match if any of the specified bits are set! - means match if the specified bits are not set fragbits:[+*!]<[mdr]> alert ip any any -> any any (msg: Fragbits ; content: 0 ; fragbits:+md;sid:114007; sid:1000;) Checks for a specific ICMP ID value. icmp_id:<number>; alert icmp any -> any any (msg: Check icmp id ; content: 0 ;icmp_id:512;sid:1000;) Table 3: Optional keyword modifier descriptions (Continued) 16

17 Optional Keyword Modifiers Keyword Icmp_seq Icode Id Ip_proto Checks for a specific ICMP sequence value. icmp_seq: <number>; alert icmp any -> any any (msg: Check ICMP Seq ; content: 0 ; icmp_seq:256; sid:1000;) Checks for a specific ICMP code value. fragbits:[+*!]<[mdr]> alert icmp any -> any any (msg: Check icmp_id and icode and ; content: 0 ; icode:>4;sid:1000;) Checks the IP identification field for a specific value. Various tools (exploits, scanners, and other programs) set this field for different purposes. id:<number>; alert ip any -> any any (msg: Check IP ID ; content: 0 ; id:1245; sid:1000;) Allows checks against the IP protocol header. For a list of protocols that may be specified by name, see /etc/protocols. ip_proto:[!><] <name or number>; alert udp any any -> any any (msg: IP PROTO ; content: 0 ; ip_proto:17;sid:1000;) Table 3: Optional keyword modifier descriptions (Continued) 17

18 Keyword Ipopts Itype Msg seq sid Checks for a specific IP option. The following options may be checked: rr - Record route eol - End of list nop - No op ts - Time Stamp sec - IP security option lsrr - Loose source routing ssrr - Strict source routing satid - Stream identifier any - any IP options are set ipopts:<rr eol nop ts sec lsrr ssrr satid an y>; alert ip any any -> any any (msg: IPOPTS ; content: 0 ; ipopts:rr;sid:1000;) Checks for a specific ICMP type value. itype:[< >]<number>[<><number>]; alert icmp any any -> any any (msg: ITYPE ; content: 0 ; itype:13;sid:1000;) Provides the text for the message that is to be printed in an alert. msg: <message text> ; alert tcp any any any 80 (msg: This rule triggered on html ; content: html ; nocase; sid:1000;) Checks for a specific TCP sequence number. seq:<number>; alert tcp any any -> any any (msg: SEQ ; content: 0 ; seq:42;sid:1000;) An value to uniquely identify rules. sid:<n>; where n is the sid number. Table 3: Optional keyword modifier descriptions (Continued) 18

19 Optional Keyword Modifiers Keyword Tag tos ttl window Allows a rule to log more than just the single packet that triggered the rule. After a rule is triggered, additional traffic that involves the source or destination host is tagged. Tagged traffic is logged to allow analysis of response codes and postattack traffic. Available values are as follows: type - logs packets from the session or from the host that cause the tag to activate. Options are session or host. count - a number of units (units are specified in the <metric> field) metric - tags the host/session for a set number of packets or seconds direction tag: <type>, <count>, <metric>, [direction] s alert tcp any any <> any 80 (msg: Tag host ; content html ; nocase; tag:host,600,seconds,src; sid:1000;)session alert tcp any any -> any 80 (msg: Tag session ; content: html ; tag:session,10,seconds; sid:1000;) Checks the IP TOS field for a specific value. tos:<number>; alert ip any any -> any any (msg: IP PROTO and TOS ; content: 0 ; ip_proto:17;tos:16;sid:1000;) Checks the IP time-to-live value. Used to help detect traceroute attempts. ttl:[[<number>-]><=]<number>; alert udp any any -> any any (msg: TTL ; content: 0 ; ttl:10;sid:1000;) Checks for a specific TCP window size. window:<number>; alert tcp any any -> any any (msg: Windows ;content: HTTP ;window:5792;si d:1000;) Table 3: Optional keyword modifier descriptions (Continued) 19

20 Copyright IBM Corporation 2005, All Rights Reserved. IBM and the IBM logo are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. ADDME, Ahead of the threat, BlackICE, Internet Scanner, Proventia, RealSecure, SecurePartner, SecurityFusion, SiteProtector, System Scanner, Virtual Patch, X-Force and X-Press Update are trademarks or registered trademarks of Internet Security Systems, Inc. in the United States, other countries, or both. Internet Security Systems, Inc. is a wholly-owned subsidiary of International Business Machines Corporation. Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product and service names may be trademarks or service marks of others. References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates. 20