Version 1.0 TECHNICAL NOTE. Restricting Fingerprint DFA Size by Using Variables

Transcription

1 Version 1.0 TECHNICAL NOTE Restricting Fingerprint DFA Size by Using Variables Document version Aug 13, 2004 List of changes First version of the document for StoneGate IPS 1.0 with StoneGate Management Center 3.0.

2 Overview The StoneGate IPS regular expressions are used in fingerprints to match traffic stream byte sequences. Although the flexible syntax provides fine-detailed pattern matching, the regular expression syntax requires careful use. The wildcard characters can be used to represent any byte value. Therefore, the * or + wildcards used in a middle of a pattern can easily result to an expression that has a large number of different matching states. The computed matching pattern can grow exponentially, thus becoming too complex for efficient use on the Sensors. For example, the pattern.*expr-a.*expr-b should not be used because of the.* wildcard between EXPR-A and EXPR-B. Instead, the pattern needs to be divided into manageable chunks by using variables as in Illustration 1.1. ILLUSTRATION 1.1 Replacing wildcard in a middle of a pattern.*expr-a\p{a=1,ignore}.*expr-b\p{a==1->a=0} The wildcard is replaced by dividing the expression into two separate branches. After the first branch.*expr-a matches, the fingerprint does not match for the traffic stream until the second branch.*expr-b has also matched. The ignore keyword indicates that the corresponding first branch is only a partial match for the fingerprint. It does not trigger the response defined for the fingerprint, unless EXPR-B matches after EXPR-A has matched. The technical details are discussed in the following sections: Size Issues with Fingerprint DFAs, on page 2 describes the technical background on this issue. Using Variables in Fingerprint Expressions, on page 4 describes how to use variables in the fingerprints. Size Issues with Fingerprint DFAs The StoneGate IPS fingerprint language syntax is an extended form of regular expressions. Regular expressions are a powerful way of expressing pattern matching that can be divided roughly into two basic methods: deterministic and nondeterministic finite-state automaton (DFA and NFA, respectively). These methods differ in matching speed and memory usage: DFA consumes a lot of memory but it is superior in matching speed compared to NFA. NFA does not consume a lot of memory but the pattern matching is slow when compared to DFA. TECHNICAL NOTE 2

3 StoneGate IPS uses DFA to compile the fingerprint patterns into a binary data structure called fingerprint DFA. The size of this binary structure is directly comparable to the number of matching states it contains. Regular expressions allow using wildcard characters for flexible matching. When wildcards are not used in a middle of a fingerprint expression, the number of states in the fingerprint DFA grows roughly linearly with the number of characters in the expression. If wildcards are used in a middle of a pattern, it is quite possible to end up with a fingerprint DFA that has an exponentially growing number of states as in Illustration 1.2. ILLUSTRATION 1.2 Wildcard use that should be avoided.*expr-a.*expr-b This fingerprint matches when there are zero or more bytes in front of EXPR-A match and zero or more bytes between EXPR-A match and EXPR-B match. In this case, the exponential growth occurs due to the.* wildcard between EXPR-A and EXPR-B. The StoneGate IPS fingerprint syntax provides variables to deal with exponential growth of a fingerprint DFA. By using variables it is possible to reduce the amount of states in the resulting fingerprint DFA. The solution is to replace wildcard characters in a middle of a pattern by using variables. Especially, the * and + wildcards in a middle of a pattern should be replaced. Instead of using.* in the middle of the pattern, the same matching criteria can be expressed with variables as in Illustration 1.3. ILLUSTRATION 1.3 Using variables to replace wildcards in a middle of an expression.*expr-a\p{var_a=1,ignore}.*expr-b\p{var_a==1->var_a=0} The ignore keyword indicates that EXPR-A alone is not considered as a match. EXPR-B alone is not considered as a match either; EXPR-A has to match first so that the var_a is set to 1. Then, EXPR-B matches only when EXPR-A has already matched, as indicated by the condition that the var_a has to be 1 for matching EXPR-B. The var_a=0 expression resets the variable after a successful match. TECHNICAL NOTE 3

4 Using Variables in Fingerprint Expressions Variables can be used for defining related regular expression patterns. These relations can be expressed with the variables, so that the fingerprint matches when all the related patterns match. Each variable is unique within the fingerprint or fingerprint group where the variable is used. The name of a fingerprint variable can be anything consisting of alphanumeric and the underscore characters [A-Za-z_0-9]. The variable name must not begin with a digit. The variable value can be either 0 or 1. The values persist through each individual traffic stream. A variable extension can consist of conditional expressions, value setting expressions, or both. Variables are defined in the fingerprint expressions as in Illustration 1.4. ILLUSTRATION 1.4 Variable syntax for fingerprints \p{conditional_expressions} \p{value_setting_expressions} \p{conditional_expressions->value_setting_expressions} A value setting expression defines the values for one or more variables when the corresponding top-level branch matches. For example: \p{var_a=1} A conditional expression inspects the values defined for one or more variables so that the corresponding top-level branch matches, and the optional variable setting expressions are processed only if the conditional expression is true. For example: \p{var_b==1} When using both variable expression types for the same top-level branch, the -> implication operator must be used. For example: \p{var_a==1->var_a=0} Note that a single equal sign = sets a value for a variable, whereas two consecutive equal signs == evaluate the value of a variable. A variable extension is evaluated when the corresponding top-level branch matches. Variables can be used only in each top-level branch of a fingerprint. Therefore, variables cannot be used inside the () expression grouping parentheses. Also, the variable extension must always be defined at the end of the top-level branch. In Illustration 1.5, valid variable extensions are used in the two top-level branches that are separated by the logical OR symbol. ILLUSTRATION 1.5 Valid use of variables in a fingerprint.*expr-a\p{var_a=1}.*expr-b\p{var_a==1->var_a=0,var_b=1} TECHNICAL NOTE 4

5 Value Setting Expressions A value setting expression defines the values for one or more variables when the corresponding top-level branch matches. Variable values are defined as follows: A variable value is defined with variable_name=value. The variable name has to be unique in the fingerprint and the fingerprint group where it is used. For example: \p{var_a=1} Multiple variable values can be defined by separating them with a comma, for example: \p{var_a=0,var_b=1}. The ignore keyword indicates that the corresponding top-level branch is only a partial match for a fingerprint. It does not trigger the response defined for the fingerprint. For the example in Illustration 1.5, the response is triggered only if EXPR-B matches after EXPR-A has matched. ILLUSTRATION 1.6 Using the ignore keyword in fingerprint variables.*expr-a\p{var_a=1,ignore}.*expr-b\p{var_a==1->var_a=0} Conditional Expressions A conditional expression inspects the values defined for one or more variables so that the corresponding top-level branch matches. The variable setting expressions are processed only if the conditional expression is true. If the conditional expression does not match, no variable setting expressions are evaluated, and the top-level branch is ignored just as if the ignore keyword was used in a variable setting expression. The conditional expressions for variables are used as follows: variable_name==value matches when the value of the defined variable is equal to value. For example, \p{var_a==1}. Multiple conditional expressions can be defined by separating them with the && logical AND operator. For example: \p{var_a==1&&var_b==0}. Note that there is no logical OR operation for the conditional variable expressions. TECHNICAL NOTE 5

6 Trademarks and Patents Stonesoft, the Stonesoft logo and StoneGate are all trademarks or registered trademarks of Stonesoft Corporation. Multi-link technology, multi-link VPN, and the StoneGate clustering technology-as well as other technologies included in StoneGate-are protected by patents or pending patent applications in the U.S. and other countries. All other trademarks or registered trademarks are property of their respective owners. Copyright and Disclaimer Copyright Stonesoft Corporation. All rights reserved. These materials, Stonesoft products and related documentation are protected by copyright and other laws, international treaties and conventions. All rights, title and interest in the materials, Stonesoft products and related documentation shall remain with Stonesoft and its licensors. All registered or unregistered trademarks in these materials are the sole property of their respective owners. No part of this document or related Stonesoft products may be reproduced in any form, or by any means without written authorization from Stonesoft Corporation. Stonesoft provides this document for informational purposes only, is subject to change without notice and does not represent a commitment on the part of Stonesoft. Stonesoft assumes no liability for any errors or inaccuracies that may appear in these materials; use these materials at your own risk. Stonesoft does not warrant or endorse any third party products described herein. THESE MATERIALS ARE PROVIDED "AS-IS." STONESOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, AS TO, THE INFORMATION CONTAINED HEREIN. IN ADDITION, STONESOFT MAKES NO EXPRESS OR IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE OR USE WITH RESPECT THE INFORMATION OR TECHNIQUES CONTAINED IN THESE MATERIALS. IN NO EVENT SHALL STONESOFT BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL OR INCIDENTAL DAMAGES, INCLUDING, BUT NOT LIMITED TO, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING FROM THE USE OF THESE MATERIALS, EVEN IF ADVISED IN ADVANCE OF THE POSSIBILITY OF SUCH DAMAGES. Revision: International Headquarters Stonesoft Corp. Itälahdenkatu 22a FIN Helsinki, Finland tel fax. info.emea@stonesoft.com Business ID: VAT number: FI Americas Headquarters Stonesoft Inc. 115 Perimeter Center Place South Terraces, Suite 1000 Atlanta, GA tel fax. info.americas@stonesoft.com Asia Pacific Headquarters Stonesoft Corp. 90 Cecil Street # Singapore tel fax. info.asiapacific@stonesoft.com