SI-DFA: Sub-expression Integrated Deterministic Finite Automata for Deep Packet Inspection

Transcription

1 2013 IEEE 14th Interntion Conferene on High Performne Swithing nd Routing SI-DFA: Sub-expression Integrted Deterministi Finite Automt for Deep Pket Inspetion Ayesh Khid, Rjt Sen, Anupm Chttopdhyy Institute for Communition Tehnoogies nd Embedded Systems (ICE), RWTH Ahen University, Germny ECE Deprtment, IIT Khrgpur, Indi Abstrt Finite utomt is widey used for Deep Pket Inspetion (DPI) of network trffi. Two types of utomt empoyed for this purpose re Non-deterministi Finite Automt (NFA) nd Deterministi Finite Automt (DFA). An NFA suffers from rge memory bndwidth per hrter due to mutipe tive sttes. A DFA, in omprison, ensures iner proessing time of O(1) for memory bsed rhitetures. However, the DFA stte exposion onditions ommony ourring in tody s NIDS rue-sets, render the utomt with prtiy infesibe memory spe requirements. To void stte bowup we propose semi-deterministi utomt, Sub-expression Integrted DFA (SI- DFA), tht ensures proessing time of singe stndrd DFA. Rues re broken into sub-expressions t bowup onditions nd ompied into singe DFA ong with n ssoition tbe, to orrety enpsute equivent utomt. We ist the rre ses in regur expressions for whih sub-expression Integrtion is inorret nd present methodoogy to detet their ourrenes. We evute SI-DFA on re-word rue-sets ike Bro, Snort nd Linux fiters nd ompre their performne with the stte-of-the-rt hybrid utomt soutions. SI-DFA renders 66-97% redution in proessing bndwidth, up to 68% ower spe requirement nd n improvement trend with inresing rue ompexity when ompred to the trdition soutions. Index Terms DPI, NIDS, DFA, NFA, stte bowup, keene strs, hybrid-utomt, regur expression mthing I. INTRODUCTION Network Intrusion Detetion Systems (NIDS) empoy Deep pket inspetion (DPI) for number of seurity bsed ppitions ike deteting nd monitoring viruses, miious trffi, un-uthorized ess, ttks nd protoos. The byte-by-byte inspetion of ontent t wire speed mkes DPI omputtiony henging probem. Moreover, prtiy fesibe spe requirements of the rue-sets utomt, despite the stteexposion, is required. Summrizing, the mjor desirbe however mutuy onfiting hrteristis of n effiient DPI system re s foows: 1) Fst nd deterministi proessing time with worst se time gurntees 2) Compt spe requirement, showing iner sbiity for inresing rue-sets Tody s rue-sets for DPI re given s regur expressions (regex) for ensuring fexibe yet ompt representtion. The worst se bndwidth nd spe requirements for representing n n-hrter regex s NFA nd DFA re given in Tbe I. Cery, there is spe-time trde-off; NFAs re ompt but sow, DFAs re fst but spe hungry. Spe requirement of n NFA is ompt with O(n) sttes, however, its proessing ompexity is hrd to mnge sine it my hve sttes tive simutneousy. A DFA gurntees O(1) proessing time but my suffer from bowup ses mking the worst se stte spe requirement Σ n (Σ being the phbet rnge). Current impementtions of regexes ptterns used in networking require gigbytes of memory, nd their performne remins imited to sub-gigbit prsing rtes; whih mkes it n importnt nd henging reserh re. Properties DFA NFA Tot Sttes Σ n n Storge ost O(Σ n ) O(n) Ative Sttes 1 n Proessing ost O(1) O(n 2 ) TABLE I COMPARISON OF NFA AND DFA FOR AN N-CHARACTER REGEX An ide finite utomt shoud thus hve the proessing speed of DFA nd spe requirements of n NFA. SI- DFA is n effort in this diretion. It voids stte bowup by integrting sub-expressions of the regexes in DFA, inked with n Assoition tbe. Theoreti nd prti ontributions of this pper re summrized s 1) We quntify stte bowup due to mutipe keene str in regexes nd propose shifter-dfa for n eonomi hnding of ounting onstrints. 2) We present nove utomt ed SI-DFA to void stte bowup. The proposed hybrid utomt is funtiony equivent to finite utomt with ery distint deterministi nd non-deterministi prts. The utomt s mjor portion is stndrd DFA tht not ony ensures deterministi proessing speed of O(1) but so n be subjeted to ompression nd enoding tehniques for DFAs [1] [4]. 3) We enist regex types for whih SI-DFA is not ppibe, give methodoogy for their detetion nd propose n terntive whenever possibe. 4) We vidte the resuts using re-word regexes typiy used for NIDS nd benhmrk the redution in onstrution time, spe nd bndwidth requirements ginst the proposed soutions. The reminder of the pper is orgnized s foows. Se. II quntifies the bowup enountered in NFA to DFA onversion. Se. III provides brief overview of the vrious proposed reted ontributions for tering stte bowup in pst nd justifies the motivtion for the proposed rhiteture. SI-DFA methodoogy is presented in Se. IV. Subsequenty we enist ertin ses in whih merging of sub-expressions is inorret nd ses in whih we hve n ternte soution. The subsequent Setion showses the experiment evution on re-word rue-sets fiters nd ompres the performne with /13/$ IEEE 164

2 NFA, DFA nd Hybrid-FA [5] impementtion. Bsed upon tht nysis onusion is presented in Se. VI. II. STATE-EXPLOSION QUANTIFICATION An NFA of n ext mth string retins the number of sttes fter its stndrd power-set onversion into DFA. A phenomenon ed exponenti stte bowup (or stte exposion) hppens when the regex orresponding to the NFA hs foowing onstruts. A. Counting Constrints A ounting onstrint is widrd repetition, tht my our in regexes s 1).{n,m} : widrd repetition between n ti m times. 2).{n,} : widrd repetition t est n times. 3).{n} : widrd repetition exty n times. Counting onstrints n ed to stte bowup during onstrution of DFAs, even when the orresponding regex is ompied in isotion [5]. To void stte bowup, Kumr et. proposed to keep ounter with the finite utomt to keep trk of these ength restritions [6]. Behi et. ter pointed out tht to gurntee the funtion orretness of this ounting utomt, tot of n-mny ounters woud be needed, n being the minimum repetition ount [7]. They proposed more eonomi soution to the probem by observing tht ony one ounter ong with (n 1)-ounter differenes wi be needed to mode n equivent ounter-dfa [7]. The orresponding DFA hs some ondition trnsitions bsed on number nd vues of differenti ounter bnk. We observe tht n n-bits shift register n serve the sme purpose, reping the ounter nd differene mrkers. Firing of new ounter is mrked by ping 1 t the hed of the shift register nd nt = n n be registered by monitoring 1 t the ti of the shift register. Bitwise OR-ing of the shift register vues n refer to presene of ny tive ounters. For quntifying the sving in terms of register bits, onsider regex expression1.{n}expression2. A og 2 (n)-bit ounter, pus (n 1) mny og 2 (n)-bit differene mrkers wi be needed for ounter-dfa [7], resuting in the synhronous ogi budget n.og 2 (n). A shift register, serving the sme purpose, wi redue this budget to n-bits. Sine ounting onstrints our frequenty for re-word regexes, sving in synhronous ogi budget n be signifint. Athough this is not the mjor ontribution of our work, to the best of our knowedge, suh shifter-dfa for hnding ounting onstrints hs not been proposed before. For the rest of the disussion, we exude the regexes with ounting onstrints sine they n be hnded better with shifter-dfa. Another exuded ss of regexes re the ones with Per Comptibe bk-referenes, whih n be hnded we by the tgging nd trvers reord keeping [7]. B. Keene Str (.*) Conditions Ourrene of unbounded widrd repetitions (or keene str) is frequent in the re-word regexes for DPI ppitions. The NFA of regex expression1.*expression2 hs sef-oop in pe of the keene str. Conversion into the DFA uses bowup ony if more thn one suh regexes re ompied together. In order to quntify the stte bowup, et us onsider k regexes. Let the i th regex with x i number of keene str onditions ed to DFA with n i sttes when ompied in isotion. The stte bowup shows stte number inrese mutipitive to the number of keene strs in the regexes dded to the utomt. The number of sttes N k in ombined DFA of k regur expressions n be expressed s: kx Y k N k = n i (x j + 1) (1) i=1 j=i+1 The eqution highights the severity of stte bowup emphsizing the need of soutions ike SI-DFA to ddress the probem. Moreover, the pseudo-widrd repetition onditions ourring in NIDS regexes so use bowup of omprbe mgnitude (e.g., expression1[ˆ\n\r]*expression2). For the rest of the pper keene str bowup ondition refers to.*,.+ or pseudo-widrd repetition uness otherwise mentioned. III. RELATED WORK An optim utomt for regex mthing is widey expored probem, both in ommeri nd demi ires, nd the reserh in this diretion ontinues ti dte. Bsed on the underying utomt, we n tegorize the shemes into three mjor sses: NFA, DFA nd Semi-deterministi utomt. One of the eriest NFA-bsed FPGA impementtion for regexes ws presented by R. Sidhu nd V. Prsnn [8]. They proposed one hot enoding of eh NFA stte by fipfop, supporting mutipe tive sttes, mutipe simutneous trnsitions nd O(1) proessing speed. Mny improvements foowed: bsed on vrious iruit optimiztions [9], [10], expoiting preism by subjeting expressions to individu ores[11], muti-strem dt proessing[12], muti-stride regex prsing [13] et. However most of the FPGA bsed NFA impementtions show sow reonfigurbiity nd inferior performne with rhiteture sbiity [14]. DFA bsed regex mthing engines ensure deterministi proessing bndwidth but hve huge memory requirements. Consequenty shemes trding DFA stte spe requirements with more thn one stte trvers per input byte hve been presented [1], [2]. By introdution of pipeined rhiteture, the probem of hving deterministi proessing rte hs so been soved by reent shemes, nmey, FEACAN [3] nd RCDFA [4]. In spite of sever DFA ompression shemes [1] [4] nd regex grouping shemes into mutipe DFAs for esping stte exposion [15], ompex regexes ompied soey into DFAs fi to generte n utomt of reizbe spe. Sever semideterministi utomt soutions hve been proposed to ddress this probem. A Hybrid-FA interrupts the subset onstrution of n NFA t sttes tht use bowup [5]. The utomt onstitutes hed-dfa with its bowup sttes inking it to sever ti DFAs/NFAs. Athough Hybrid-FA hs spe nd bndwidthrequirementsowerthnthtofsoedfandnfa soution respetivey, however, its proessing bndwidth vries with the network trffi with known worst se. Another simir hybrid utomt sheme espes stte exposion by 165

3 NFA segregtion into mutipe utomts t points bsed on probbiity distribution of utomt trnsitions [6]. It isotes the frequenty visited prefixes nd infrequenty visited suffixes of regexes nd mkes segregted utomts. This sheme however, requires reord keeping of stte trvers during trffi tring, tht hs signifint overhed for rge rue-sets. A tunbe semi-deterministi utomt, TFA, n be tuned t onstrution time to hve tive sttes bounded by b to ensure preditbe worst se performne [16]. Lrge onstrution ost of TFA is however prohibitive of its use, sine it reies on n NP-hrd Set Spit Probem for ny b 2. Moreover, in terms of memory requirements, it is seetivey inferior to Hybrid-FA [5]. A Semi-deterministi Finite Automt (SFA), with simir spirit s tht of TFA, ws proposed by Yng nd Prsnn [17]. They proposed greedy heuristi to sove the NP ompete probem of finding the minimum number of onstituent DFAs (-DFAs) required to void stte bowup ompetey for given set of regexes [17]. A more reent propos DC-DFA, proposes NFA stte set prtitioning suh tht eh onstituent stte set, when trnsted to DFA hs no bowups [18]. It surpsses SFA in terms of stte number redution nd system throughput. The semi-deterministi utomt shemes [5], [6], [16] [18] suessfuy evde the probem of stte-spe bowup by utotmt segregtion t the ost of inresed proessing bndwidth. For Hybrid-FA, the worst se memory bndwidth requirement is iner to the number of regexes ontining bowup onditions [5]. Hene for ompex regex set the number of simutneous tive utomt n in worst se inrese s muh s the number of rues. It my use sowing down of the system if the NIDS engine supports imited number of simutneous DFAs. With rue-sets pubiy vibe, miious trffi n be generted for sowing-down ttks on the system. Our proposed SI-DFA merges the sub-expressions into singe DFA, hene the worst se memory bndwidth requirement is independent of rue ompexity. Consequenty the system hs good memory effiieny nd requires proessing bndwidth orders of mgnitude ess thn hybrid soutions. Moreover its O(1) proessing mkes it resiient to sowing-down ttk, rendering it superior to the stte-of-the-rt soutions. IV. A SUB-EXPRESSION INTEGRATED DFA (SI-DFA) DFA stte bowup is inked with the ourrene of sefoop ike stte in the respetive NFA. In n NFA, the sefoop sttes wys remin tive fter first trvers. DFA stte bowup is due to its inbiity to tre mutipe prti mthes simutneousy nd is overome by stte repition using bowup. One wy of overoming this shortoming of DFAs without using stte bowup is brek n expression into prts t bowup onditions nd merge them into n integrted DFA. Methodiy speking we undergo foowing steps for SI- DFA retion 1) We brek regexes into prts ed sub-expressions using keene str onditions s deimiters. 2) We rete merged DFA for the sub-expressions. The epting sttes of DFA re beed s Fin Aepting Sttes (FAS) or Sub-expression Aepting Sttes (SAS) if they mth fin or non-fin sub-expression of regex respetivey. 3) A regex is epted if its onstituent sub-expressions re epted in the right order. This is ensured by reting dependeny of every sub-expression in regex (exept thefirstone)onitspreedingsub-expression.ainkbitis ssoited with every sub-expression tht is onditiony vidted provided its orresponding FAS/SAS is rehed nd its preeding sub-expression ink bit hs redy been rehed/vidted. Vidtion of n FAS reports mth of its respetive regex rue. 4) For pseudo-widrd bowup ses, the ourrene of orresponding forbidden hrter is inked to invidte the ink bit of preeding sub-expression. Consider regexes b.*d nd m for better understnding of SI-DFA retion methodoogy. With keene str s deimiter, 3 sub-expressions re extrted. A merged DFA of the subexpressions b, d nd m is reted, hving 7 sttes s shown in Fig /- b 1 2 5/1 d Fig. 1. SI-DFA for Sub-expressions b, d nd m Corresponding to the three sub-expressions we hve three ink bits whose ddresses re speified in ink bit ddress oumn of n Assoition Tbe shown s Tbe II. Stte 5 nd 6 re FAS for regex 1 nd 2 respetivey, whie stte 4 is n SAS. Whenever stte 5 is visited fter tring subexpression d its orresponding ink bit (bit 1) is vidted ony if its dependeny bit (bit 0) is redy vid, inditing former eptne of its preeding sub-expression b. FAS/SAS orresponding to the first sub-expression of regex hve no orresponding dependeny bit (stte 4 nd 6). Vidtion of bit orresponding to n FAS resuts in mth to rue number speified by the st oumn of the tbe. Consider trffi tre dbmd. Linked bits re vidted in order fter trversing b, m nd d of trffi respetivey, out of whih the st two ink bits report mthes of regex 2 nd 1 respetivey. Stte FAS Dependeny Link bit Mth # bit ddress ddress Rue # TABLE II ASSOCIATION TABLE SHOWING STATE DEPENDENCIES For pseudo-bowup ses, if forbidden hrters do not our in subsequent sub-expression e.g., b[ˆx]*d, forbidden hrter tbe is onstruted with ourrene of forbidden hrter x tied to invidte the ink bit orresponding to sub-expression b. One forbidden hrter my be inked 3 m 6/2 166

4 to number of ink bits for invidtion. Fortuntey, reword rue-sets hve few distint forbidden hrters, hene the strtegy is fesibe in terms of spe requirements. In prinipe our SI-DFA is osest in spirit to Hybrid-FA [5] but is superior in performne in foowing prospets. 1) A hybrid-fa for group of regexes with n bowup ses wi hve one hed nd n ti utomts, eding to n+1 tive sttes in worst se. In omprison, SI-DFA wi hve no more thn 1 tive stte. 2) Due to ommon prefix merging, mutipe sub-expressions when ompied into singe DFA my derese the numberofsttesompredtothesumofnumberofsttes in sever DFAs of sub-expressions. Consequenty, the memory requirements of n SI-DFA re esser thn tht of Hybrid-FA. A. Mpping SI-DFA on n Arhiteture The SI-DFA is essentiy DFA ouped with n Assoition tbe. This setion desribes the ogi mpping of SI- DFA into memory-bsed rhiteture s shown in Fig. 2. Fig. 2. SI-DFA Arhiteture Strutury one setion of SI-DFA is stndrd DFA. Input dt is fed into Trffi Pket Buffer, whih is shifted right fter onsuming eh hrter ( hrter is sid to be onsumed when is orresponding stte hnge is ompete). There is onfigurbe DFA tbe memory, tht hods stte trnsition dt. The DFA trnsition tbe is ompressed ording to the D2FA improved gorithm [2]. Consequenty the DFA tbe memory is represented in ompressed form, hving imited number of beed trnsitions, defut trnsition nd bit-mp to distinguish between the two kinds of trnsitions. A deoder proesses the dt from the urrent stte register nd the inoming hrter nd updtes the new stte. Whie proessing eh hrter, the next stte is found out by foowing the defut or beed trnsitions of the urrent stte orresponding to tht hrter. The seond portion of SI-DFA omprises of the Assoition tbe, whih hods the orretion between the SAS, FAS nd the sub-expression ink bit, ong with the forbidden hrter tbe. If the next stte is n SAS/FAS, the pointer is shifted to the prtiur row in the Assoition tbe. Condition vidtionofinkbitsisdonebsedonthedependenybitbeing vidted. During proessing of hrter it is simutneousy fed to the hrter tbe, s shown in Fig. 2. It is then heked for equity ginst the forbidden hrters nd if mth is found of its orresponding sub-expression ink bits re invidted. B. Probem in Sub-expression Extrtion Regexes hving keene str bowup onditions within seprted brnhes must be preproessed before being subjeted to n SI-DFA. For exmpe, nive division of regex (mno. xy)m b d t the bowup stte ondition wi resut in two meningess sub-expressions, (mno nd xy)m b d. As preproessing step, the regex is first onverted to mutipe regexes (mno. xy)m nd b d, preserving the equivene. The steps in breking down suh regexes re isted beow: 1) Prse the regex into symboi expression with (, ) nd s deimiters e.g., (mno. xy)m b d is onverted to (A)B C D where A = (mno. xy),b = m,c = b,d = d 2) M rk symboi hrters whose orresponding subexpression hs bowup ondition. In the bove exmpe symboi hrter A is M rk-ed. 3) Repe ontention of symbos with. i.e., (A)B C D = (A).B C D. 4) Postfix the symboi expression with. (AND) nd (OR) s opertors nd proess the expression. The OR-ing is done ony when the symbos in onsidertion re not mrked, otherwise they re kept seprtey s individu symbos. Steps re given for the exmpe. ) Postfix expression : AB.CD b) {A}.{B} => {AB} ) {C} {D} => {(C D)} d) {AB} {(C D)} => {AB, C D} e) Fin output : AB, C D 5) Repe the symbos with the orresponding subexpressions e.g., {(mno. xy)m, b d} C. Cses not Conforming with SI-DFA Methodoogy We enist here seetive ses in regexes for whih the SI-DFA is not ppibe nd give methodoogy of their detetion. 1) Pseudo widrd repetitions: Regexes with pseudo widrd repetition onditions suh tht forbidden hrters our in subsequent sub-expression nnot be hnded by SI-DFA e.g., b[ˆx]*xd. For the exmpe t hnd, onsider bmxd s input trffi tht shoud idey report mth with x put in the forbidden hrter tbe. However, the ourrene of the hrter x wi turn off the sub-expression bit orresponding to b nd subsequenty the regex wi not be mthed. Hene for suh ses, the ony terntive is to et the bowup hppen. Fortuntey these ses re rre in modern dy regexes s eborted ter. 2) Subsequent sub-expressions overp: A orrety working SI-DFA shoud strt mthing sub-expression ony fter subsequent sub-expression hs redy been epted. Some exmpes vioting this order re enisted.. Regex b.*b wi be wrongy mthed for trffi b. The overpping hrter between sub-expressions is b. b. Regex b.*d.*bd wi be wrongy mthed for trffi bd. Here, the overp exists between ontention of sub-expressions 1 nd 2 (bd) nd the fin subexpression (bd). 167

5 . Regex.b.*.d wi be wrongy mthed for trffi bd. Simiry regex b..*d wi be wrongy mthed for trffi bd. The exmpes emphsize the need of methodiy formuting overp detetion in subsequent sub-expressions in regexes so tht it my be utomted. Moreover, regex shoud be subjeted to overp hek between eh of its onstituent subexpressions, with the ontention of its preeding subexpressions before dering it fit to generte n SI-DFA. Overp: Let M 1(L1), M 2(L2) be the mhines epting nguges L1nd L2respetivey.Thereissidtobenoverp between the two, if there exists string beonging to L1 whih resuts in n non-initi stte when fed to the M2. We define n Intersetion( ) DFA M 3(L1 pref(l2)), denoting mhine epting intersetion of the two nguges L1 nd pref(l2) whie M(pref(L2)) epts possibe non-nu prefixes of L2 ( sttes other thn strting stte in M(pref(L2)) re epting sttes). Then M3 = (Q1 Q2, Σ, σ3, [q 1, q 2 ], F1 F2) (2) Where Q i, σ i, q i nd F i represent sttes, trnsition funtion, strting stte nd epting stte of i th mhine respetivey. S1 S2 is the Crtesin produt of sets S1 nd S2, ordered pirs, [q1,q2] is n ordered pir from Q1 Q2, σ3 is onstruted from σ3([x1, x2], ) = [σ1(x1, ), σ2(x2, )] (3) for in Σ nd [x1,x2] in Q1 Q2. If epting sttes of M3 i.e., F1 F2 re non-rehbe the two nguges hve no overp nd the regex n be subjeted to n SI-DFA. 3) Compete ontinment in subsequent sub-expressions: SI-DFA wi generte erroneous resut if sub-expression in regex is ompetey ontined in its foowing sub-expression. Some exmpes re enisted for ebortion.. Regex.*b.d wi be wrongy mthed for trffi bd, sine sub-expression is ompetey ontined in the next sub-expression. b. Simiry, regex b.*bd wi be wrongy mthed for trffi mbd. A methodi definition for detetion of these ses foows. Let M1(L1), M2(L2) be the mhines epting two subexpressions. We define M3(L1 L2), donting mhine eptingunionofthetwonguges L1nd L2.Ifthenumber of epting sttes in M3 is greter thn the sum of number of epting sttes in M1 nd M2, i.e., (F3 > (F1 + F2)), non-nu subset of L1 is ompetey ontined in L2. This ondition is neessry nd suffiient for heking ompete ontinment between two sub-expressions. For exmpe, when the sub-expressions of regex b.*bd re ompied in isoted DFAs, eh DFA hs one epting stte. However Union DFA of the two sub-expressions wi hve 3 epting sttes showing ompete ontinment. Detetion of overp nd ompete ontinment in subsequent sub-expressions is done s preproessing eimintion step for SI-DFA. These regexes n be det by other hybrid utomt methodoogies [5], [17]. D. Ext-mth remov in.+ Cses Mny re-word regexes hve unbounded widrd repetition terms defined by.+ (1 or more repetition of widrd). Converting suh regex (e.g., b.+d) into n equivent regex with keene str ondition gives b..*d. Breking tht t bowup ondition gives two overpping sub-expressions b. nd d, eving it unfit to be det by SI-DFA. We devie form workround for tking these ses. Notie tht the regex with dot-pus ondition, e.g., b.+d, n equivent utomt wi be the one tht mthes b.*d nd not mth bd. Hene we expoit the exusion of ext mth strings to ensure orretness. We define n utomt mthing the differene of two nguges L3 = L1 L2, for the exmpe L1={b, d} nd L2={bd} s shown in Fig. 3. Differene utomt is reted by first mking Union utomt of L1 nd L2 nd then onverting the epting stte due to L2 s non epting stte. Union DFA of b, d nd bd wi hve 3 epting sttes (3, 4 nd 6) but stte 6 is onverted to non-epting stte to ommodte subtrtion of bd. The resuting SI-DFA orrety mthes b.+d. 0 1 b 3/1 Fig. 3. Differene DFA for b, d nd bd mthing b.+d E. SI-DFA Constrution Any utomt-bsed NIDS hs two phses. A preproessing phse tht is genery done on gener purpose omputing CPU. It epts regexes nd genertes the respetive utomt representtion. Every new rue-set updte requires preproessing before kiking off the serhing phse. The serhing phse invoves IP trffi streming t muti-gig rtes to report mthes. This phse is genery off-oded to dedited sve hrdwre for eertion resons. SI-DFA retion methodoogy is expined s fowhrt in Fig. 4. First the rues hving ounting onstrints nd bkreferenes re seprted out to be det s Hybrid-FA [7] or SFA [17]. Rest of the rues undergo n OR-seprtion preproessing s expined in Se. IV-B. A inking fie is so generted to keep trk of the origin rue numbers. Next we identify the three types of rues for whih SI-DFA methodoogy nnot be ppied s given in Se. IV-C. Suh regexes re identified nd moved to nother fie to be det seprtey s Hybrid-FA [7] or SFA [17]. A the remining rues re divided into sub-expressions t the bowup onditions ong with the neessry inking informtion in seprte fies. For dot-pus onditions subsequent sub-expression re so dded to this fie with fgs for generting non epting sttes for these rues s expined in Se. IV-D. A these sub-expressions re then subjeted to stndrd DFA utomt genertion nd 5 2 d 6 d 4/2 168

6 Hybrid-FA [5] SI-DFA % Rue-set No of rues DFA size NFA size Hed DFA Ti DFAs Tot No of Border No of No of No of Redution in size size size tis Size Sttes SAS FAS Bndwidth Linux Bro > Snort 1 46 > Snort 2 16 > Snort 3 14 > TABLE III COMPARISON BETWEEN DIFFERENT AUTOMATA terms of utomt s onstrition time, proessing bndwidth requirement nd storge ompexity. A singe DFA for eh of these rue-sets exeeds 100K sttes due to stte bowup. We tune the Hybrid-FA [5] to hve minim impt of stte bowup by direting bowup onditions to ed to ti-fas. The sme rue-set when subjeted to SI-DFA methodoogy resuts in singe DFA with tot sttes nd SAS nd FAS sttes s speified in Tbe III. Fig. 4. SI-DFA Cretion Fowhrt ompression. Resut nysis (shown in dotted boundry) is n option step nd done ony for omprison of our sheme. V. PERFORMANCE EVALUATION SI-DFA ss ode (1.5K ines) ws deveoped in C++ nd inudes funtions for deteting regexes unfit for SI-DFA, utomt retion, trffi tring nd performne evution. Our testing ptform ws n AMD Phenom 1055T Proessor with 8 GB of RAM nd Linux operting system. We hose prti rue-sets extrted from Bro 2.0 [19], Snort [20] (snpshot updted on 10 th Juy, 2012), nd inux [21] rues for experiment evution. It is worth highighting tht the perentge of regexes from these rues, for whih the retion of SI-DFA is not possibe, s demonstrted by the st two oumns of Tbe IV, is very sm (the oumn % of overp represents the ses of both overp nd ompete ontinment). Note tht we onsider the regexes without ounting onstrints nd bk-referenes. No Highest No. Averge No. % % Rue-set of bowup ses bowup ses of of rues per rue per rue overp forbidden Bro % Linux % 0 Snort (Spywre) % 9.25 TABLE IV STATISTICAL STUDY OF CASES UNFIT FOR SI-DFA We extrt rue-sets Snort 1, Snort 2 nd Snort 3 from snort rue-set suh tht eh hs rues with predominnty one, two nd three bowup onditions respetivey (Tbe III). We ompre our SI-DFA performne with Hybrid-FA [5] in A. Memory Bndwidth Requirement The memory bndwidth requirement of n utomt is the number of its simutneousy tive sttes. It is proportion to the serh speed of trffi. Sine eh tive DFA requires one stte trvers per hrter, the mximum number of possibe tive DFAs for Hybrid-FA is given by the number of tis pus one (for Hed-DFA). SI-DFA gives remrkbe sving on simutneous memory esses required per hrter (66-97%) s it hs ony one DFA. The st oumn of the tbe shows this % redution, whih is uted s: (bndwidth HFA bndwidth SI-DFA) bndwidth HF A B. Storge Compexity 100 (4) Before ompring the storge requirements of the two utomt, we ompressed the Hed-DFA, the ti-dfas of Hybrid-FA nd the SI-DFA ording to the D2FA improved gorithm [2]. An n-stte DFA tbe when ompressed requires, for eh of its stte row, foowing mount of memory 1) A Stte ID hving w = og 2 (n)/8 bytes. 2) A defut trnsition of w bytes. 3) Lbeed trnsitions eh of w bytes. 4) A bitmp of /8 bytes speifying beed / defut trnsitions. For SI-DFA, Eh stte entry hs one ddition bit speifying if it is n FAS/SAS or not. In se it is n FAS/SAS there is pointer to speifi row of the Assoition tbe with width equ to Stte ID s expined beow. Assoition tbe for m number of FAS/SAS sttes is orgnized into the foowing dt segments: 1) An Stte ID hving og 2 (m)/8 bytes. 2) The IDs of ink bits to be vidted, eh hving memory footprint of b = og 2 (no bits)/8 bytes (these ink bits didn t exeed 9 per SAS/FAS for onsidered rue-sets) 3) The ID of dependeny ink bit of w bytes 4) Mth rue number in se of n FAS. One FAS my be inked to one or more rues to be mthed. For our experiment dt the Forbidden hrter tbe ws found to hve no more thn 10 hrters. Sine eh of the 169

7 forbidden hrter my be ssoited with one or more ink bits, the mximum footprint of this tbe for f hrters n be (f (hr id) + f ( og 2 (no bits)/8 )) bytes. Memory footprint omprison of SI-DFA nd hybrid-fa for the rue-sets in onsidertion is given in Fig. 5. It n be observed from tht SI-DFA outperforms onsiderby espeiy when the number of bowup onditions per ine inreses. Bro rue-set is modertey ompex. Its SI-DFA bets the respetive Hybrid-FA in memory footprint. Sine Linux rue-set hs ess frequent bowup onditions, SI-DFA hieves sving in terms of redued memory esses per hr proessing ony. Fig. 5 so shows trend in the SI-DFA memory footprint redution with inresing dt set ompexity. For the three Snort rue-sets, with inresing ompexity(eft to right) onsiderbe redution in memory ompred to Hybrid-FA is seen. Fig. 5. Memory Footprint Comprison of SI-DFA nd Hybrid-FA Fig. 6. Constrution Time Comprison of SI-DFA nd Hybrid-FA C. Automt onstrution time We ompre SI-DFA onstrution time on our testing ptform with tht of Hybrid-FA in Fig. 6. Note tht the time tken for testing regex being fit for SI-DFA hs been exuded. Tht is beuse t the time of reese of new rue-set, eh of its regex n be heked for its onformity to the SI- DFA methodoogy one nd tgged with this informtion. The trend of inresing performne with inresing ompexity understndby foows for utomt onstrution time too s seen for snort rue-sets. VI. CONCLUSION In this ontribution we present n SI-DFA tht merges sub-expressions t the bowup onditions into singe DFA. Thereby it not ony suessfuy espes the stte bowup but so keeps the proessing bndwidth equ to singe instne of stndrd DFA. We present ompete methodoogy of SI- DFA retion enisting ses for whih SI-DFA retion is not possibe nd form definition for detetion of these ses. Unike the hybrid shemes hving mutipe simutneousy tive DFAs, SI-DFA enpsutes their whoe essene in singe instne of DFA beting them by hving ower proessing bndwidth nd spe requirement with fster ompition of utomt for re-word NIDS rue-sets. Moreover with n inresing ompexity of regexes the performne improves further, mking it n ide hoie for prti NIDS. As future work we pn to ompre SI-DFA performne with other reported hybrid utomt shemes. We so intend to extend the SI-DFA methodoogy s prototype hrdwre impementtion. Aided with immense eertion, the proposed system n gurntee high-speed impementtion of NIDS more eonomiy. REFERENCES [1] S. Kumr, S. Dhrmpurikr, F. Yu, P. Crowey, nd J. Turner, Agorithms to eerte mutipe regur expressions mthing for deep pket inspetion, in ACM SIGCOMM Computer Communition Review, vo. 36, no. 4, 2006, pp [2] M. Behi nd P. Crowey, An improved gorithm to eerte regur expression evution, in Pro. of the 3rd ACM/IEEE Sym. on Arhiteture for networking nd ommunitions systems, 2007, pp [3] Y. Qi, K. Wng, J. Fong, Y. Xue, J. Li, W. Jing, nd V. Prsnn, FEA- CAN: Front-end eertion for ontent-wre network proessing, in Pro. of IEEE INFOCOM, 2011, pp [4] K. Wng, Y. Qi, Y. Xue, nd J. Li, Reorgnized nd ompt DFA for effiient regur expression mthing, in Pro. of IEEE Int. Conf. on Communitions (ICC), 2011, pp [5] M. Behi nd P. Crowey, A hybrid finite utomton for prti deep pket inspetion, in Pro. of the ACM CoNEXT, 2007, pp [6] S. Kumr, B. Chndrsekrn, J. Turner, nd G. Vrghese, Curing regur expressions mthing gorithms from insomni, mnesi, nd ui, in Pro. of 3rd ACM/IEEE Sym. on Arhiteture for networking nd ommunitions systems, 2007, pp [7] M. Behi nd P. Crowey, Extending finite utomt to effiienty mth Per-omptibe regur expressions, in Pro. of ACM CoNEXT, 2008, pp [8] R. Sidhu nd V. Prsnn, Fst regur expression mthing using FPGAs, in Pro. of 9th Annu IEEE Sym. on Fied-Progrmmbe Custom Computing Mhines (FCCM), 2001, pp [9] B. Huthings, R. Frnkin, nd D. Crver, Assisting network intrusion detetion with reonfigurbe hrdwre, in Pro. of 10th Annu IEEE Sym. on Fied-Progrmmbe Custom Computing Mhines, 2002, pp [10] Y. Yng, W. Jing, nd V. Prsnn, Compt rhiteture for highthroughput regur expression mthing on FPGA, in Pro. of the 4th ACM/IEEE Sym. on Arhitetures for Networking nd Communitions Systems, 2008, pp [11] A. Mitr, W. Njjr, nd L. Bhuyn, Compiing PCRE to FPGA for eerting snort IDS, in Pro. of the 3rd ACM/IEEE Sym. on Arhiteture for networking nd ommunitions systems, 2007, pp [12] Q. Yun, Y. Yng, nd V. Prsnn, Muti-strem regur expression mthing on FPGA, in Pro. of IEEE Int. Conf. on Reonfigurbe Computing nd FPGAs (ReConFig), 2011, pp [13] N. Ymgki, R. Sidhu, nd S. Kmiy, High-speed regur expression mthing engine using muti-hrter NFA, in Pro. of IEEE Int. Conf. on Fied Progrmmbe Logi nd Appitions (FPL), 2008, pp [14] C. Crk nd D. Shimme, Sbe pttern mthing for high speed networks, in Pro. of 12th Annu IEEE Sym. on Fied-Progrmmbe Custom Computing Mhines (FCCM), 2004, pp [15] F.Yu,Z.Chen,Y.Dio,T.Lkshmn,ndR.H.Ktz, Fstndmemoryeffiient regur expression mthing for deep pket inspetion, in Pro. of ACM/IEEE Sym. on Arhiteture for Networking nd Communitions systems, 2006, pp [16] Y. Xu, J. Jing, R. Wei, Y. Song, nd H. Cho, TFA: A tunbe finite utomton for regur expression mthing, Tehni Report, poytehni institute of new york university, [17] Y. Yng nd V. Prsnn, Spe-time trdeoff in regur expression mthing with semi-deterministi finite utomt, in Pro. of IEEE INFOCOM, 2011, pp [18] W. He, Y. F. Guo, nd H. C. Hu, Hybrid finite utomt-bsed gorithm for rge se regur expression mthing, vo. 263, pp , [19] [20] [21] 170