Boundary Hash for Memory-Efficient Deep Packet Inspection

Similar documents
Highly Memory-Efficient LogLog Hash for Deep Packet Inspection

TriBiCa: Trie Bitmap Content Analyzer for High-Speed Network Intrusion Detection

TriBiCa: Trie Bitmap Content Analyzer for High-Speed Network Intrusion Detection

Automatic compilation framework for Bloom filter based intrusion detection

A Framework for Rule Processing in Reconfigurable Network Systems

Implementation of Boundary Cutting Algorithm Using Packet Classification

Multi-pattern Signature Matching for Hardware Network Intrusion Detection Systems

VARIABLE-LENGTH HASHING FOR EXACT PATTERN MATCHING

Hardware Implementation for Scalable Lookahead Regular Expression Detection

Selective Boundary Cutting For Packet Classification SOUMYA. K 1, CHANDRA SEKHAR. M 2

Efficient Packet Classification for Network Intrusion Detection using FPGA

Design of a Near-Minimal Dynamic Perfect Hash Function on Embedded Device

Novel FPGA-Based Signature Matching for Deep Packet Inspection

FlashTrie: Hash-based Prefix-Compressed Trie for IP Route Lookup Beyond 100Gbps

An Enhanced Bloom Filter for Longest Prefix Matching

Packet Inspection on Programmable Hardware

Pipelined Parallel AC-based Approach for Multi-String Matching

FPGA Implementation of Lookup Algorithms

Scalable Lookahead Regular Expression Detection System for Deep Packet Inspection

CHAPTER 4 BLOOM FILTER

The Encoding Complexity of Network Coding

LONGEST prefix matching (LPM) techniques have received

Scalable Lookup Algorithms for IPv6

Efficient Packet Classification using Splay Tree Models

A Connection between Network Coding and. Convolutional Codes

Multi-core Implementation of Decomposition-based Packet Classification Algorithms 1

Tree-Based Minimization of TCAM Entries for Packet Classification

Scalable Multi-Pipeline Architecture for High Performance Multi-Pattern String Matching

Regular Expression Acceleration at Multiple Tens of Gb/s

2386 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 6, JUNE 2006

Hash-Based String Matching Algorithm For Network Intrusion Prevention systems (NIPS)

Highly Compressed Aho-Corasick Automata For Efficient Intrusion Detection

Computational Geometry

Packet Classification Using Dynamically Generated Decision Trees

ID Bloom Filter: Achieving Faster Multi-Set Membership Query in Network Applications

Notes on Binary Dumbbell Trees

AN FPGA BASED ARCHITECTURE FOR COMPLEX RULE MATCHING WITH STATEFUL INSPECTION OF MULTIPLE TCP CONNECTIONS

A Framework for Rule Processing in Reconfigurable Network Systems

A Framework for Rule Processing in Reconfigurable Network Systems

Fast Reconfiguring Deep Packet Filter for 1+ Gigabit Network

Design and Implementation of DPI Mechanism for NIDS on FPGA

One Memory Access Bloom Filters and Their Generalization

Automation Framework for Large-Scale Regular Expression Matching on FPGA. Thilan Ganegedara, Yi-Hua E. Yang, Viktor K. Prasanna

COMP3121/3821/9101/ s1 Assignment 1

Routing Lookup Algorithm for IPv6 using Hash Tables

Switch and Router Design. Packet Processing Examples. Packet Processing Examples. Packet Processing Rate 12/14/2011

Performance of FPGA Implementation of Bit-split Architecture for Intrusion Detection Systems

A Robust Bloom Filter

arxiv: v3 [cs.ds] 18 Apr 2011

Advanced Pattern Based Virus Detection Algorithm for Network Security

would be included in is small: to be exact. Thus with probability1, the same partition n+1 n+1 would be produced regardless of whether p is in the inp

NOISE ELIMINATION USING A BIT CAMS

SWM: Simplified Wu-Manber for GPU-based Deep Packet Inspection

Project Proposal. ECE 526 Spring Modified Data Structure of Aho-Corasick. Benfano Soewito, Ed Flanigan and John Pangrazio

Highly Space Efficient Counters for Perl Compatible Regular Expressions in FPGAs

Efficient TCAM Encoding Schemes for Packet Classification using Gray Code

SSA: A Power and Memory Efficient Scheme to Multi-Match Packet Classification. Fang Yu, T.V. Lakshman, Martin Austin Motoyama, Randy H.

Large-scale Multi-flow Regular Expression Matching on FPGA*

ICS 691: Advanced Data Structures Spring Lecture 8

AN EFFICIENT AND SYSTEMATIC VIRUS DETECTION PROCESSOR FOR EMBEDDED NETWORK SECURITY

High-throughput Online Hash Table on FPGA*

REGULAR expressions are widely used in the network intrusion

Lecture 16. Reading: Weiss Ch. 5 CSE 100, UCSD: LEC 16. Page 1 of 40

Resource Efficient Multi Ported Sram Based Ternary Content Addressable Memory

Advanced Algorithmics (6EAP) MTAT Hashing. Jaak Vilo 2016 Fall

A Hybrid Approach to CAM-Based Longest Prefix Matching for IP Route Lookup

TOKEN-BASED DICTIONARY PATTERN MATCHING FOR TEXT ANALYTICS. Raphael Polig, Kubilay Atasu, Christoph Hagleitner

Advanced Pattern Based Virus Detection Algorithm for Network Security

Scalable Automaton Matching for High-speed Deep Content Inspection

Updating Designed for Fast IP Lookup

Parallel-Search Trie-based Scheme for Fast IP Lookup

Two-Stage Decomposition of SNORT Rules towards Efficient Hardware Implementation

The Encoding Complexity of Network Coding

Homework 1 Solutions:

Disjoint Superposition for Reduction of Conjoined Prefixes in IP Lookup for Actual IPv6 Forwarding Tables

FPGA Implementation of Token-Based Clam AV Regex Virus Signatures with Early Detection

TUPLE PRUNING USING BLOOM FILTERS FOR PACKET CLASSIFICATION

Symbol Table. Symbol table is used widely in many applications. dictionary is a kind of symbol table data dictionary is database management

Performance Evaluation and Improvement of Algorithmic Approaches for Packet Classification

Bloom Filters. References:

Lockless Hash Tables with Low False Negatives

Multiple-choice (35 pt.)

Design of a Weighted Fair Queueing Cell Scheduler for ATM Networks

Difference Bloom Filter: a Probabilistic Structure for Multi-set Membership Query

IP packet forwarding, or simply, IP-lookup, is a classic

Extensible Network Configuration and Communication Framework

Fast Lookup: Hash tables

ARELAY network consists of a pair of source and destination

NETWORK INTRUSION DETECTION SYSTEMS ON FPGAS WITH ON-CHIP NETWORK INTERFACES

LOW-DENSITY PARITY-CHECK (LDPC) codes [1] can

An Ultra High Throughput and Memory Efficient Pipeline Architecture for Multi-Match Packet Classification without TCAMs

A Real Time GIS Approximation Approach for Multiphase Spatial Query Processing Using Hierarchical-Partitioned-Indexing Technique

Load Balancing for Problems with Good Bisectors, and Applications in Finite Element Simulations

Tree-Structured Indexes

Introduction. hashing performs basic operations, such as insertion, better than other ADTs we ve seen so far

Error Detection and Correction by using Bloom Filters R. Prem Kumar, Smt. V. Annapurna

CHAPTER 9 HASH TABLES, MAPS, AND SKIP LISTS

Structural Framework for High Speed Intrusion Detection/Prevention Signature Based Systems

Data Warehousing & Data Mining

DATA STRUCTURES/UNIT 3

Transcription:

Boundary Hash for Memory-Efficient Deep Packet Inspection N. Sertac Artan, Masanori Bando, and H. Jonathan Chao Electrical and Computer Engineering Department Polytechnic University Brooklyn, NY Abstract Network Intrusion Detection and Prevention Systems (NIDPSs) are critical for network security. The Deep Packet Inspection (DPI) operation consumes a significant amount of resources in NIDPS. This is because to detect malicious activity DPI searches a database of signatures for each byte of every packet. In this paper, we develop a highly space-efficient data structure for hardware realization of Minimal Perfect Hash Functions (MPHFs). This data structure is simple to construct, requires 7n bits to represent the MPHF for a set of n keys and allows high-speed DPI. Index Terms Boundary Hash, Deep Packet Inspection, DPI, Minimal Perfect Hash Functions, Network Intrusion Detection and Prevention System, NIDPS, Network Security, Trie Bitmap Content Analyzer, TriBiCa, Snort. I. INTRODUCTION Network Intrusion Detection and Prevention Systems (NIDPSs) have a vital role in current state-of-the-art network security solutions [1], [2]. Deep Packet Inspection (DPI) is at the heart of these NIDPSs. DPI is the detection of malicious packets by comparing the packet payloads against excerpts from known intrusion packets, i.e., against the intrusion signatures database. DPI consumes a large portion of processing power and memory for the NIDPS. Yet, achieving high-speed DPI for a low cost is a continuing challenge as the line rates and the number of intrusions keep increasing. One way to address this challenge is to use Minimal Ferfect Hash Functions (MPHFs) to search the signature database [3]. An MPHF is a hash function that maps a set S of n keys into exactly n integer values (0...n 1) without any collisions [4]. MPHFs provide constant worst-case query time and minimal space; thus, they are very suitable for DPI. In this paper, we propose a low-cost MPHF called boundary hash (BH) suitable for hardware implementation. The proposed MPHF is based on our trie-based framework called TriBiCa (Trie Bitmap Content Analyzer) [3]. In this framework, the algorithm gradually decides on which key to compare the packet payload among a set of keys. For each query the algorithm provides at most one match candidate. The keys can be whole or partial signatures or some other information regarding signatures [3]. The algorithm starts with n keys at the root node of a trie and it partitions the keys into two equal-sized groups (each group with n/2 keys). Then, each of these new groups is placed into one of the child nodes of the root node and each new group is partitioned into two equalsized groups (each group with n/4 keys). This partitioning is repeated recursively until there are n nodes each with one key. To query for a key, the algorithm traverses the trie until a single candidate key is pointed at a leaf node. When the single key is found, only one comparison is needed (i.e., compare the queried key and the candidate key) to decide whether the queried key is actually the same as the candidate key. Based on the TriBiCa framework, we recently proposed a low-cost high-speed DPI architecture that requires a single commodity FPGA to do inspection at 10-Gbps [5]. The contribution of this paper is a new space-efficient node structure for the TriBiCa framework called boundary. In our previous proposals [3], [5], bitmaps are used as node structures to efficiently represent each group for queries. The proposed boundary hash (BH) completely eliminates the bitmaps in the nodes and replaces them with the boundary data structure. BH is simple to construct and can be represented with 7n bits in practice. It can complete queries in log 2 (n) pipelineable stages. The rest of the paper is organized as follows: Section II summarizes related work and states the problem addressed in this paper. Section III presents the proposed scheme. Section IV provides theoretical analysis of the memory usage and success probability of the proposed scheme. Section V summarizes the experiments executed to show the performance of the proposed scheme. Section VI concludes the paper. II. RELATED WORK AND PROBLEM DEFINITION For DPI in NIDPS, the data structure to store the intrusion signatures database should balance the requirements of highspeed, low-cost, and easy update. DPI approaches in software NIDPSs such as Snort [6] and Bro [7] are very flexible and support detection of sophisticated intrusions. However, they are not scalable for high speeds since they are running on general-purpose hardware, which is intrinsically slow and has limited parallelism. Hence, here we only consider the hardware approaches. DPI approaches on hardware can broadly be classified into two architectures based on their signature storage media: (1) off-chip memory [8], [9] and (2) on-chip memory and/or logic blocks [10] [17]. A survey of these architectures can be found in [3]. Architectures using off-chip memory for signature storage are fundamentally limited by the off-chip memory throughput and additional cost of memory chips. 978-1-4244-2075-9/08/$25.00 2008 IEEE 1732

As a result of these limitations of the off-chip storage, onchip storage gains attention, recently. Additionally, due to the high parallelism available on-chip, it is desirable to multiply the signature detection throughput by replicating the DPI data structures on a single chip to allow parallel detection. Unfortunately, the available on-chip storage is limited. This limitation forces highly space-optimized data structures, and the desired parallelism further constrains storage. Finally, the data structure should be simple enough to allow rapid detection to achieve high throughput to support tomorrow s line rates and large signature databases. Hence, considering the strict storage constraint and high-speed requirements, low-cost DPI is a continuing challenge. One reason that DPI consumes a large portion of NIDPS processing power is that the intrusion signatures can appear anywhere in a packet payload. To address this issue, in DPI, each packet payload is searched using a sliding window that slides one byte at a time. The window content is compared against the intrusion signatures to detect malicious activity. In the worst-case, this requires a comparison of the window content against all the signatures for each and every byte offset from the packet payload. Using hash functions, the number of possible matches to the window content can be reduced to a few possible signatures. The window content is then compared to these signatures to verify if there is a match. Although ordinary hash functions have a good average case speed especially when the memory utilization is low, they cannot guarantee the number of possible matches in the worst-case due to hash collisions. This is made worse if the memory utilization is high. A Perfect Hash Function (PHF) is a special type of hash function that eliminates all the collisions. To optimize both speed and storage, a minimal PHF (MPHF), which maps a given set S of n keys into exactly m = n memory slots without any collisions, can be used. In addition to the memory to store the keys, a hash function needs additional storage for its own representation. The information theoretical lower bound to represent an MPHF is approximately 1.4427n bits [18]. In [17], an efficient MPHF construction is given based on Bloom Filters [19], which requires 8.6n bits for representing the MPHF in practice. This approach, however, requires a complex addressing scheme for queries, where additional logic is required to calculate the address in the hash table. In [3], we propose an MPHF suitable for high-speed hardware implementation using 11n bits for representation. Our goal in this paper is to provide a low-cost and spaceefficient MPHF that is simple to construct and suitable for high-speed hardware implementation. III. BOUNDARY HASH To tackle the problem of constructing an MPHF for a set S = {K 1...K n } of n keys 1, let us first focus on a simpler goal. This simpler goal is defined by a binary decision 1 In this paper, we assume n is a power of two, however the proposed method can be applied to arbitrary size n as discussed in [3]. function, F over the queried key, q for the set S with n keys as follows. 0, q G l, G l = {K 1...K n/2 } F n (q, S) = 1, q G r, G r = {K n/2+1...k n } If q belongs to the left group (G l ), we say F n (q, S) =0 and if q belongs to the right group (G r ), we say F (q, S) =1. So, now the query is a membership query such as, does the queried key belong to the left or right group?, rather than a location query such as, which memory slot stores the queried key? Note that if q / S, the outcome of F is not specified. Also note that G l and G r are two disjoint subsets of set S. When we determine whether the queried key belongs to group G l or G r, we can find a narrower group with n/4 keys, which includes the queried key, by applying F n/2 on the queried key. For instance, if F n (q, S) =0i.e., q G l, then we can apply F n/2 (q, G l ) to find the group with n/4 keys including q. This operation can be repeated until the resulting group size is 1, i.e., until after applying F 2, which points out one candidate key that can match the queried key. Let us define a trie, where node N l,i at level l stores the i th subset of S with size n/2 l (i = 1...2 l, l = 0...log 2 (n) 1). Each node at level l uses one decision function F n/2 l and has two child nodes that store one of this nodes disjoint subsets each with n/2 l+1 keys. Let us define the function H n (q) as the concatenation of the log 2 (n) decision functions, {F n,f n/2,f n/4...f 2 }, (1) H n (q, S) =F n &F n/2 &F n/4...&f 2 (2) where & shows bitwise concatenation. Let H n [i] shows the i th bit of H n, where i =0is the most significant bit of H n. Theorem 3.1: The function H n (q, S) defines an MPHF on the set S of n keys. Proof: F n/2 l partitions the keys of a node at level l into two disjoint subsets, S L and S R. K L S L, H n [l] =0and K R S R, H n [l] =1. H n (K L,S) differs from H n (K R,S) at least at one bit location (namely at H n [l]), thus no K L S L collides with any K R S R. This principal can be applied recursively for all nodes starting from the root node, until we reach the final level, l f = log 2 (n) 1. Then, for any two keys K i and K j at two different nodes at level l f, H n (K i,s) and H n (K j,s) differ at least at one bit, thus no two keys from two different nodes collide. At level l f, F 2 partitions keys of each nodes into two disjoint sets, each with one key and since H n [l f ] is different for these two keys, no keys at the same node at level l f collide. Thus, H n is a PHF. The output of H n has log 2 (n) bits, since it is the concatenation of log 2 (n) functions each with 1-bit output. Thus, the range of H n is 0...n 1 and H n is an MPHF on the set S. Corollary 3.2: Constructing an MPHF on a set S of n keys is equivalent to constructing the decision functions {F n,f n/2...f 2 } recursively for the set S. 1733

A. Using PHFs To Construct an MPHF In this section, we show how to realize an MPHF for a set S of n keys using PHFs. From Corollary 3.2 it is sufficient to show the decision function F can be constructed from the PHFs, to show how to realize an MPHF. Let us first look at the operation of a PHF more closely with an example. Then, we will introduce the boundary concept for space-efficient realization of F. Assume a PHF h, with a range of 0...m 1, isusedfor storing and querying n keys without collisions in a memory with m slots (m >nby PHF definition). The operation is as follows. h is first calculated for each of the n keys and each key is inserted into the memory slot corresponding to its hash value. After all keys are stored, an index is assigned to each key in the memory incrementally from left to right such that the first key is K 1 and the last key is K n. Figure 1 shows the insertion of n =8keys (K 1...K 8 ) into a memory of m =16 slots using the PHF h. To make a query, h is calculated over the queried key and the corresponding memory slot is read. h(k 1 ) = 0 PHF h(k 8 ) = 13 K 1 K 2 K 3 K 4 K 5 K 6 K 7 K 8 0 boundary 15 b = 6 Fig. 1: Hashing a given set of n =8keys using a PHF, h with range m =16. The boundary, b =6is the index of the memory slot where the key K n/2 = K 4 is stored. To construct F from a PHF, we need to be able to distinguish between the keys in the left group (K 1...K 4 in the example) and the right group (K 5...K 8 ). To do so, after we store the keys, we additionally identify a memory slot index, we call a boundary, which is defined as follows: Boundary: The boundary, b is the index of the memory slot that stores the key, K n/2. We call this slot the boundary slot or the boundary bin (B b ). For instance, in Figure 1, the boundary is at b =6. Then, the following function F b is equivalent to the function F for a PHF, h. 0, h(q) b F b (q, S) = (3) 1, h(q) >b The function F b is equivalent to F since by definition, the keys are numbered from left to right in the bitmap and there are precisely n/2 keys in the bins (0...b) in total as well as in the bins (b +1...m 1). Thus, the boundary partitions the keys into two equal-sized groups and we can decide the group a queried key q belongs to by applying F b on q just like the function F. For a given query, the hash function is calculated. If the h(q) b, F b =0(left group, G l ); otherwise, F b =1 (right group, G r ). Since h is a PHF, there is no collision and each key has its own bin. Thus, the boundary always provides a clear distinction between G l and G r. Note that once the boundary is identified, to calculate F = F b the triple (m, h, b) is sufficient and the bins are not required anymore for the query operation. To emphasize this point, let us call the bins virtual bins. B. Relaxing the PHF Constraint In the previous section, we show by using PHFs, an MPHF can be realized. However, finding n 1 PHFs (one PHF per node of the trie) may be costly. In this section, we look at using other type of hash functions, not necessarily perfect. Assume we replace the PHF h of Section III-A with a hash function h u with a range of 0...m 1 from a universal hash function family, such as H 3 [20]. Since h u is not perfect, hashing n keys with h u may result in collisions. However, even when h u is used, we may realize F with F b. In fact, the only condition to realize F with F b is that the key K n/2+1 is not stored in the boundary bin. In other words, if the keys K n/2 and K n/2+1 are not in collision, F b is equivalent to F and we can use the triple (m, h u,b) to calculate F = F b.for instance in Figure 2a, F can be realized with F b and can be calculated using the triple (m =16,h u1,b=4). On the other hand in Figure 2b, the keys K n/2 = K 4 and K n/2+1 = K 5 are in collision and the triple cannot determine whether the keys in the boundary bin (K 3,K 4,K 5 ) belong to G l or G r, thus F F b. More formally, let B i show the number of keys in bin B i. Then, we can realize F with F b iff the boundary bin, B b satisfies, b 1 j=0 B j = n/2 ɛ and B b = ɛ, where ɛ>0. For B b >ɛ, we cannot determine whether the keys in the boundary bin belong to G l or G r. Note that when the hash function is a PHF, j, B j 1 and so ɛ =1. Thus, if a PHF is used, we can always realize F with F b. Given n keys, m bins, and a universal hash function h u, Figure 3 shows the pseudo-code to find the boundary bin. IV. ANALYSIS A. Space Complexity for Boundary Hash In the boundary hash trie, there are log 2 (n) levels. At the root node, there are n keys and at each level, the number of keys per node is halved compared to the previous level. Let us define the number of bins and number of keys in a node at level l as m l and n l, respectively. To keep the bin per key ratio in a node constant, the same halving relation can be applied to the number of virtual bins at a node between levels. Given m 0 = M R virtual bins at root node (root node is at level 0), there will be m l = M R /2 l virtual bins at any node at level l. There are 2 l nodes at level l in total. Thus, the total number of virtual bins at any level is constant and equal to M R, since, 2 l M R /2 l = M R. Fortunately, for boundary hashing the virtual bins are not stored. Instead a boundary is stored for each node. Since, there are M R /2 l bins in a node at level l, the memory requirement for such a node to store a single boundary is log 2 (M R /2 l ) bits. 1734

h u1 h u2 h u1 (K 1 ) = 0 h u1 (K 8 ) = 13 h u2 (K 1 ) = 0 h u2 (K 8 ) = 13 K 4 K 7 K 5 K 1 K 2 K 3 K 8 0 boundary 15 b = 4 (a) The first hash function h u1 results with a perfect boundary even if there are some collisions. K 6 K 5 K 4 K 7 K 1 K 2 K 3 K 8 0 No perfect 15 boundary K 6 (b) Collisions by h u2 prevent a perfect boundary. Fig. 2: Hashing a given set of n =8keys using two different universal hash functions, h u1 and h u2 with range m =16. 1: GetBoundary (S, h u, A) 2: // S is a set of n keys (K 1...K n ) 3: // h u is a universal hash function with range [0,m A 1] 4: // A is an array of m A counters initialized to zero 5: for i =1to n 1 do 6: A[h u (K i )] + + 7: end for 8: sum 0, i 0, issuccess false 9: while sum<n/2 and i<m A do 10: sum sum + A[i] 11: i i +1 12: if sum = n/2 then 13: issuccess true 14: end if 15: end while 16: B 1 i 1 // Boundary bin 17: return (issuccess, B 1 ) Fig. 3: Pseudo-code for partitioning a node in Boundary Hash Hence, the total space required to store all the boundaries for the 2 l nodes at level l is M l =2 l log 2 (M R /2 l ) (4) From (4), the total memory for the boundary hash trie with log 2 (n) levels is, M = Simplifying (5) we get, M = (2 i i) + log 2 (M R ) Definition 1: (From geometric series) j k=0 2 i log 2 (M R /2 i ) (5) k r k 1 = 1 rj+1 (j +1) rj (1 r) 2 1 r (2 i ) (6) (7) Definition 2: j 2 i =2 j+1 1 (8) Note that left part of M (let us define it as M left )in(6) can be rewritten as M left = (2 i i) =2 (i 2 i 1 ) (9) which is equivalent to Definition 1, given r = 2 and j = log 2 (n) 1. Then, M left becomes M left = (log 2 (n) 2) n +2 (10) From Definition 2, the right part of M (let us define it as M right ) in (6) can be rewritten as M right = log 2 (M R ) (2 i ) (11) = log 2 (M R ) (n 1) (12) Substituting (10) and (12) in (6) gives us M = log 2 (M R /n) n +2 n log 2 (M R ) 2 (13) If M R = O(n), then log 2 (M R /n) = O(1) and log 2 (M R )=O(log 2 (n)). Thus, the storage complexity for storing all the boundaries for the trie is O(n). Additionally, for each node, a hash function is used from a pool of universal hash functions. The worst-case universal hash function storage complexity is O(n (log 2 ( H )+H w )), where H shows the total number of universal hash functions used in the system and H w is the representation size of individual hash functions. Although, H = n 1 in the worst-case (one hash function per node), making worst-case hash function storage complexity O(n (log 2 (n)+h w )). It is much lower in practice, as shown in Section V. 1735

B. Success Probability of Boundary Hash In this section, the probability of successfully partitioning a set with n keys into two equal-sized groups using BH is derived. To achieve equal partitioning using BH, any key with an index greater than n/2 should appear to the right of the boundary bin. Equivalently, the total number of keys in the first (b 1) bins must be exactly equal to n/2 ɛ for successful equal-partitioning, where ɛ = B b. As explained in Section III-B, we can always achieve equal-partitioning if a PHF is used. If a single universal hash function with range 0...m 1 is used, B b can have up to n/2 keys (i.e., ɛ =1...n/2) and with any b =0...m 2 equal partitioning is possible. Then, the probability of successfully partitioning a set with n keys into two equal-sized groups using BH with a single universal hash function is, m 2 n/2 ( )( )( ) ɛ ( ) n n ɛ n 1 b 1 2 ɛ ( ) n m b 2 P = n b=0 i=1 2 ɛ ɛ m m m (14) This term inside the two sums shows that for a given (ɛ, b) pair, the probability that exactly ɛ keys hash to the boundary bin and exactly n/2 ɛ keys hash to the left of the boundary bin so that the remaining n/2 items hash to the right of the boundary bin. This value is then aggregated for all possible (ɛ, b) pairs by means of the two sums. Figure 4 shows this success probability for different key set sizes and different numbers of bins. operation can be pipelined to provide throughput of one query per clock cycle, regardless of the number of trie levels [5]. A C# program is developed to evaluate the first two parameters. In the first experiment, the boundary hash is constructed for different key set sizes (n =1K...1M) where each key is a 32-bit random number. For each set size, 100 different key sets are generated. Then the boundary hash is constructed for each of these key sets for various M R values 2 (M R = n...32 n). Figures 5 and 6 show the average and maximum number of hash functions required for each trie in this experiment, for different n and M R values, respectively. In Table I, the construction time, which increases linearly with number of keys, is also shown. Fig. 5: Number of hash functions required for various key set sizes for different memory configurations. Each data point shows an average value over the 100 key sets of the same size. Fig. 4: Success probability of equal partitioning for boundary hash using different memory configurations for different key set sizes. V. PERFORMANCE In this section the performance of the boundary hash is investigated. There are three main performance parameters for an MPHF; (1) Memory required for representation, (2) initial construction time, and (3) query time. The focus of this section is on the first two performance parameters. Since, the query time is determined by the number of levels in the trie and this is fixed to log 2 (n), we will not elaborate any further on the query speed but only note that in a hardware setting, the query Fig. 6: Number of hash functions required for various key set sizes for different memory configurations. Each data point shows maximum value over the 100 key sets of the same size. In the second experiment, boundary hash is constructed for the two main detectors (namely, 4-byte detector with 2724 keys and stateful detector with 3226 keys) as described in 2 Each node at level i is assigned M R /2 i virtual bins in accordance with the analysis in Section IV. 1736

TABLE I: Average boundary hash construction time for key sets with different sizes. Time is given in milliseconds. n 1K 2K 4K 8K 16K 32K Time 13 21 43 91 196 417 n 64K 128K 256K 512K 1M Time 886 1, 888 3, 979 8, 378 17, 399 detail in [5] to detect Snort intrusion signatures [6]. The experiment is repeated for 100 different hash sets. Figure 7 shows the maximum and average number of hash functions required for each trie in this experiment. Fig. 7: Number of average and maximum hash functions required for the two Snort key sets for different memory configurations. Each data point shows an average value over 100 different hash function sets. TABLE II: Memory required for BH representation in bits. n Boundary Hash Total 1K 2.99n 3.57n 6.56n 2K 2.99n 3.41n 6.40n 4K 3.00n 3.58n 6.58n 8K 3.00n 3.61n 6.61n 16K 3.00n 3.56n 6.56n 32K 3.00n 3.53n 6.53n 64K 3.00n 3.90n 6.90n 128K 3.00n 3.89n 6.89n 256K 3.00n 3.94n 6.94n 512K 3.00n 3.94n 6.94n 1M 3.00n 3.99n 6.99n 2M 3.00n 3.99n 6.99n 4M 3.00n 4.00n 7.00n Snort Detector 1 3.01n 3.33n 6.34n Snort Detector 2 3.00n 3.79n 6.79n There are two main components of the BH representation; (1) boundary storage and (2) hash function storage. For both experiments, the maximum total memory for representation is minimized at 7n bits for M R =2n as shown in Table II. VI. CONCLUSION Deep Packet Inspection (DPI) is the core operation of Network Intrusion Detection and Prevention Systems (NIDPSs). In this paper, a memory-efficient, simple to construct minimal perfect hash function suitable for hardware implementation called Boundary Hash is proposed for DPI. Boundary hash is developed over the trie-based minimal perfect hashing framework proposed in [3], and boundary hash provides a new bitmap-free node structure with a small size for this framework. The proposed method provides a minimal perfect hash function for a given static or pseudo-static set of n arbitrary keys using 7n bits. VII. ACKNOWLEDGMENTS The authors would like to thank to Yanming Shen for his insightful comments. REFERENCES [1] Sourcefire 3d. [Online]. Available: http://www.sourcefire.com [2] Fortinet. [Online]. Available: http://www.xilinx.com [3] N. S. Artan and H. J. Chao, TriBiCa: Trie Bitmap Content Analyzer for High-Speed Network Intrusion Detection, in 26th Annual IEEE Conference on Computer Communications (INFOCOM), 2007, pp. 125 133. [4] P. E. Black, Minimal Perfect Hashing, in Dictionary of Algorithms and Data Structures. U.S. National Institute of Standards and Technology, Jul. 2006. [Online]. Available: http://www.nist.gov/dads/html/minimalperfecthash.html [5] N. S. Artan, R. Ghosh, Y. Guo, and H. J. Chao, A 10-Gbps High-Speed Single-Chip Network Intrusion Detection and Prevention System, in 50th Annual IEEE Global Communications Conference, GLOBECOM 2007, Washington, DC, Nov. 2007. [6] [Online]. Available: http://www.snort.org [7] V. Paxson, Bro: A System for Detecting Network Intruders in Real- Time, Computer Networks, vol. 31, pp. 2435 2463, 1999. [8] F. Yu, T. Lakshman, and R. Katz, Gigabit Rate Pattern-Matching using TCAM, in Int. Conf. on Network Protocols (ICNP), Berlin, Germany, Oct. 2004. [9] H. Song and J. Lockwood, Multi-pattern Signature Matching for Hardware Network Intrusion Detection Systems, in 48th Annual IEEE Global Communications Conference, GLOBECOM 2005, St Louis, MO, Nov-Dec 2005. [10] C. Clark and D. Schimmel, Scalable Pattern Matching for High- Speed Networks, in IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM), Napa, California, 2004, pp. 249 257. [11] Y. H. Cho and W. H. Mangione-Smith, Fast Reconfiguring Deep Packet Filter for 1+ Gigabit Network, in FCCM, 2005, pp. 215 224. [12] Z. K. Baker and V. K. Prasanna, High-Throughput Linked-Pattern Matching for Intrusion Detection Systems, in Proc. of the First Annual ACM Symposium on Architectures for Networking and Communications Systems, Princeton, NJ, 2005, pp. 193 202. [13] J. Moscola, J. Lockwood, R. P. Loui, and M. Pachos, Implementation of a Content-Scanning Module for an Internet Firewall, in FCCM, 2003, pp. 31 38. [14] I. Sourdis, D. Pnevmatikatos, S. Wong, and S. Vassiliadis, A Reconfigurable Perfect-Hashing Scheme for Packet Inspection, in Proc. 15th International Conference on Field Programmable Logic and Applications (FPL 2005), August 2005, pp. 644 647. [15] L. Tan and T. Sherwood, Architectures for Bit-Split String Scanning in Intrusion Detection, IEEE Micro, vol. 26, no. 1, pp. 110 117, Jan-Feb 2006. [16] G. Papadopoulos and D. N. Pnevmatikatos, Hashing + Memory = Low Cost, Exact Pattern Matching, in Proc.15th International Conference on Field Programmable Logic and Applications (FPL), August 2005, pp. 39 44. [17] Y. Lu, B. Prabhakar, and F. Bonomi, Perfect Hashing for Network Applications, in IEEE Symposium on Information Theory, Seattle, WA, 2006, pp. 2774 2778. [18] F. C. Botelho, R. Pagh, and N. Ziviani, Simple and space-efficient minimal perfect hash functions, in WADS, 2007, pp. 139 150. [19] B. Bloom, Space/Time Trade-offs in Hash Coding with Allowable Errors, Communications of the ACM, vol. 13, no. 7, 1970. [20] M. Ramakrishna, E. Fu, and E. Bahcekapili, A Performance Study of Hashing Functions for Hardware Applications, in Proc. of 6th International Conference on Computing and Information, 1994, pp. 1621 1636. 1737