Efficient Regular Expression Grouping Algorithm Based on Label Propagation Xi Chena, Shuqiao Chenb and Ming Maoc

Transcription

1 4th Ntionl Conference on Electricl, Electronics nd Computer Engineering (NCEECE 2015) Efficient Regulr Expression Grouping Algorithm Bsed on Lbel Propgtion Xi Chen, Shuqio Chenb nd Ming Moc Ntionl Digitl Switching System Engineering & Technology Reserch Center, Zhengzhou, Chin Keywords: regulr expression, deep pcket inspection, grouping lgorithm, lbel propgtion. Abstrct. Regulr expression grouping is the prcticl wy to ddress the stte explosion problem of Deterministic Finite Automton (DFA). Previous grouping lgorithms hve poor grouping time, nd difficult to meet ppliction needs. In this work, we present n efficient regulr expression grouping lgorithm bsed on lbel propgtion. Through the deterministic initil grouping nd bsed on similrity of the propgtion process to chieve fster convergence. Experimentl results show tht. Compred with other lgorithms, GBLP lgorithm hs the minimum totl number of sttes nd grouping time for the sme number of groups. Introduction. With the incresing network security threts, Deep Pcket Inspection (DPI) is becoming more nd more importnt in security ppliction nd reserch. It describes the threts (e.g. spm, virus or nonymous intrusion) by defult description of the signtures, to identify the network flows with mlicious informtion in the pylod of their pckets. Nowdys, regulr expressions hve grdully become the min description lnguge of DPI signtures owing to its expressiveness nd flexibility. Regulr expression is usully chieved by Finite Automt (FA). The Deterministic Finite Automt (DFA) with its liner nd predictble mtching speed becomes the reserch hotspot. However, the DFA my require huge mount of memory become of the stte explosion problem. The number of DFA sttes cn be exponentil in size of regulr expression s shown in Fig.1. So mny lgorithms for the DFA storge optimiztion hve been proposed. V PV [^e] P Q b R c S d X T [^g] g PX Z ef.*gh SV d TV [^ce] e QX b RX g PY h h c RW [^g] g Y b.*cd RV f f f [^c] b PW W [^] QV [^e] e e c SX d TX [^cg] RY h PZ RZ /b.*cd/ nd /ef.*gh/ Fig.1 Exmple of stte explosion But in the current er of big dd, the scle nd complexity of the DPI rule set is lso incresing. The problem of the DFA stte explosion my cuse it difficult to generte. The rule set of L7-Filter is compiled to DFA which needs more thn 16GB of memory. The generl mchine is difficult to chieve. At this cse, most of the storge optimiztion lgorithms (D2FA, &FA, etc.) cnnot be pplied. But grouping lgorithm cn effectively void infltion between regulr expressions. However, s the demnd for the rel-time detection under the high-speed network environment is grdully prominent. The grouping time of the existing regulr expression grouping lgorithm is The uthors - Published by Atlntis Press 1054

2 longer nd it is difficult to be pplied in prctice. Therefore, in this work, n efficient regulr expression grouping lgorithm bsed on lbel propgtion (GBLP) is designed. It cn chieve the effective blnce between grouping time nd grouping effect. Finlly, we implement the GBLP lgorithm nd the other 3 kinds of grouping lgorithms in L7-Filter, Snort nd Cisco security ppliction. The experimentl results bsed on prcticl rule sets show tht under the sme number of clusters, the GBLP lgorithm hs improved the totl sttes number nd grouping time, especilly on grouping time reduced from 2 to 5 times. Relted Work DFA min memory consumption is n n (n is the number of stte, is lphbet of ASCII) of the Stte Trnsition Tble (STT). It occupies more thn 95% of the DFA storge spce. Therefore, the method of solving the DFA memory cn be divided into two directions: reduced stte trnsition edge [1-4] nd reduced stte number [5-10]. Ech of these Sttes is corresponding to the 256 trnsition edges, so the compression of the edge is limited. The reduction in the number of sttes cn void the spce explosion by the nture. In this work, we choose to design fst nd efficient lgorithm to reduce the number of sttes. F. Yu et l. [5] first proposed regulr expression grouping lgorithm, clled mdfa. However, the lgorithm only uses the mutul reltionship between the regulr expressions by 0 nd 1. Grouping strtegy is too simple. Becchi et l. [6] used the grouping method of dichotomy to chieve fster grouping time, but the prtil grouping effect is scrificed. Q. Xu et l. [7] defined the Dilte Rtio (DR) to describe the expnsion effect of regulr expressions, nd put forwrd kind of selective clustering lgorithm REGADG to improve the grouping effect. But the lgorithm my lose informtion. Rohrer [8] nd Fu Z [9] bstrcted the grouping problem, nd combined with the existing optimiztion lgorithm (integer liner progrmming, simulted nneling, genetic lgorithm nd nt colony optimiztion lgorithm) to chieve better effect of grouping. But even though Fu Z designed DFA estimtion lgorithm to ccelerte the execution speed, grouping time is still higher thn the trditionl mdfa. Liu Tingwen [10] proved tht the optimum k-grouping problem of regulr expression set ws NP-hrd, nd the locl optimiztion lgorithm GRELS ws designed. But the grouping time hs not been improved significntly. Obviously, the previous regulr expression grouping lgorithm hs grdully improved the grouping effect, less memory footprint. However, with the incresing demnd for network security nd service qulity, the importnce of dt rel-time detection is lso more prominent. And the grouping time of the grouping lgorithm is closely relted to detection efficiency. At the sme time, the nlysis found tht the previous grouping methods usully dopt the method of rndom trverse nd multiple itertions. And no deterministic initil group ws given. GBLP Algorithm. The regulr expression grouping is to divide the regulr expression set into multiple sub sets. The gol is to minimize the DFA expnsion sttes number. The lbel propgtion lgorithm is simple nd esy to implement. The implementtion time is short, especilly suitble for lrge scle dense grph grouping. Therefore, this work will reference the ide of lbel propgtion. We optimize the distribution of the initil lbel nd the trnsmission process. Finlly, the reduction of grouping time is relized in the premise of effective grouping. In prcticl ppliction, the prllel processing cpbility of hrdwre pltform will limit the mximum group number. In generl, the number of groups is determined by the mximum prllel processing cpbility of the device. So we set the known number of clusters K, minimizing the totl number of sttes. Introduction to Lbel Propgtion Algorithm. Lbel propgtion lgorithm [11] (LPA) is proposed by Zhu et l. in Its bsic ide is to construct smple connection digrm, which gives different initil lbel to ech vertex. Then ccording to the principle of voting, the lbel tht hs the lrgest number of tgs in the djcent 1055

3 vertices is trnsmitted to the vertex. Finlly, the sme lbel vertices re divided into group. Lbel propgtion lgorithm including initiliztion, lbel updte, nd dd the sme tg vertex to the sme group three prts. Ech vertex is ssigned unique lbel. And the optiml group is chieved fter multiple itertions. But in the process of trnsmission, lrge number of smll groups cn be generted. Mny meningful lbels re fuzzy. At the sme time, the communiction effect bsed on the principle of simple voting is poor. And simple voting principle is not suitble for complete grph of connected to ech other. Therefore, lbel propgtion lgorithm cnnot be directly pplied to the regulr expression grouping. We design fst nd efficient regulr expression grouping lgorithm by improving it. Minly includes two steps: deterministic initil group selection nd lbel propgtion bsed on similrity. Design nd Implementtion of GBLP. In view of regulr expression set R= {r 1, r 2,..., r n }, we construct undirected complete grph G= (V, E). Which V= {v 1, v 2,..., v n } for vertex set, where ech vertex v i represents regulr expression r i. Every pir of vertices v i nd v j hs n edge weight e ij. In order to describe the reltionship between the regulr expressions, the concept of the Expnsion Rte (ER) is introduced s the edge weight. Definition 1: For ny regulr expression r i nd r j, ER formul is s follows: Trr ( i, j) ER (r,r)= i j Tr ( ) + Tr ( ) (1) i j T(r i ) nd T(r j ) respectively corresponding to the regulr expression r i nd r j compiled into stte of DFA. T(r i, r j ) sid the regulr expression r i nd r j combined generted DFA stte number. When ER > 1, the corresponding regulr expression is merged nd the expnsion is generted. The greter the ER is, the more serious the stte expnds. The corresponding regulr expressions should be divided into different groups. When ER is less thn or equl to 1, sid the corresponding two regulr expressions do not interct. ER is close to 0, the corresponding regulr expressions re more likely to be clssified into the sme group. A. Initil Group Selection. The originl lbel propgtion lgorithm ssigns ech vertex unique lbel, which leds to mny smll groups in the itertive process. The meningful lbel cnnot be emerged lrge number t first. This reduces the convergent speed nd ccurcy limit problems. In the cse of the known pcket number K, the "criticl" K vertices my be selected s the initil group nd the initil tg is given. This cn effectively reduce the number of lbels, speed up the convergence rte, nd improve the ccurcy of the group. The selection of "criticl" K vertices, nmely choose K nd ll the vertices do not produce or produce the expnsion of the stte of the vertices s little s possible. The concept of Pek Intensity (PI) is introduced in this pper to describe the expnsion effect of the regulr expression in the vertices. Definition 2: For regulr expression r i corresponding vertex v i, PI formul is s follows: j= n PI( v ) = ER( r, r ) (2) i i j j= 1, i j PI is the weight of ll edges on the corresponding vertices. The greter the PI indictes tht the vertex nd the rest of the merger hve lrger expnsion rte. The vertex corresponds to regulr expression is more complex. On the contrry, the smller PI vertex corresponds to regulr expression is more simple, the smller the stte expnsion cused by. According to the bove nlysis, we first clculte the PI of n vertices nd prioritize. Then, ccording to the PI vlue, the minimum K vertices re selected s the initil group G i nd ssigned the initil lbels. B. Lbel Propgtion Grouping Process. In this process, the trditionl lbeling lgorithm is bsed on the principle of voting to select the 1056

4 common lbel. In order to void the lbel shocks, usully dopt the strtegy of synchronous to updte the lbel. The updting formul is s follows: C ( t) = f( C ( t),, C ( t), C ( t 1),, C ( t 1)) (3) vi vi1 vi( i 1) vi( i+ 1) vin Among them, v i1,, v in is the vertex v i of ll the djcent vertices, v i1,, v i(i-1) indictes tht the vertex of the lbel hs been updted before this itertion, v i(i+1),, v in sid the itertion of the lbel hs not updted the vertex, f function s described bove. But it cnnot be directly used in here. Ech vertex is connected to ech other in complete grph G. And we cn chieve the minimum of the expnsion stte for the purpose of grouping. So we need to modify it. The concept of vertex Similrity (S) is introduced. Definition 3: For vertex v i nd v j, S formul is s follows: 1 Sv ( i, vj) = ER( r, r ) i j (4) The S (v i, v j ) between v i nd v j is the reciprocl of ER (r i, r j ). Obviously, the greter the similrity between vertices, the corresponding two vertices should be for the sme group. So in order to chieve the lest number of expnsion stte. We design method of lbel propgtion bsed on similrity, lbel propgtion formul is: C ( t) = gc ( ( t),, C ( t), C ( t 1),, C ( t 1)) (5) vi vi1 vi( i 1) vi( i+ 1) vin In the bove formul, g returns the sum of the similrity nd the mximum of the neighbor vertices in the vertex v i. If the sum of similrity is equl, then rndomly choose one. The propgtion method cn effectively limit the spred of the lbel rndomness, nd improve the convergence precision. C. Algorithm Implementtion. Bsed on the bove two steps, the finl group ccording to the lbel informtion on ech vertex. Lbel propgtion grouping lgorithm steps re s follows: Step1: Clculte the PI of ech vertex in the complete grph G. Since the childhood prioritize put in X; Step2: Clculte the similrity between rbitrry vertices; Step3: According to the number of clusters K, PI's smllest K vertices re selected, nd the initil lbel is ssigned; Step4: Set up t =1, updte lbel, for(v i V), C ( t) = gc ( ( t),, C ( t), C ( t 1),, C ( t 1)) ; vi vi1 vi( i 1) vi( i+ 1) vin Step5: If ll vertices re ssigned lbels nd not chnge, the lgorithm ends, or set t =t+1, to Step4. Algorithm Anlysis. All kinds of regulr expression grouping lgorithms need to clculte expnsion reltionship between the rules. This step cn be considered s the preprocessing stge of the regulr expression grouping, nd the time complexity is O(n 2 ) (n number of rules in regulr expression set R). So we neglect this step in the process of nlysis. The time complexity of the grouping process is considered only. We put forwrd n effective regulr expression grouping lgorithm bsed on lbel propgtion. The initil grouping nd the lbel propgtion bsed on similrity re liner. The time complexity of the grouping process is O(n). For the previous grouping lgorithm, the expnsion reltionship needs to be clculted in ech itertion process. The time complexity is generlly O(n 2 ). Therefore the time complexity of GBLP is especilly improved. Experiment nd Anlysis. In this section, we conducted series of experiments to evlute the performnce of GBLP lgorithm. All experiments were performed on Virtul Mchine 8.0. The virtul mchine 1057

5 configurtion is s follows: opertion system: Ubuntu 11.10, internl memory: 8 GB, Intel dul core 2.5 GHz CPU. The Regulr Expression Processor [6] is used to clculte the ccurte number of sttes of regulr expressions distribution, s the metric of memory consumption. Three rule sets picked from L7-Filter Snort nd commercil corportion Cisco re used, s shown in Tble 1. Tble 1. RULE SETS INFORMATION Rule sets # of RegEx % RegEx w/ wildcrds (*,+) Snort-158 L7-Filter-80 Cisco We generted mdfa [5], Becchi [6], GRELS [10] nd GBLP respectively with different rule sets in Tble 1. We compred them on the totl sttes number of the DFAs constructed from groups nd grouping time. Tble 2. GROUPING RESULTS FOR Snort-158 Grouping lgorithm mdfa Becchi GRELS GBLP sttes number K=5 K=6 K=7 Grouping sttes Grouping sttes Grouping time(s) number time (s) number time (s) Grouping lgorithm mdfa Becchi GRELS GBLP Grouping lgorithm mdfa Becchi GRELS GBLP Tble 3. GROUPING RESULTS FOR L7-Filter-80 K=5 K=6 K=7 sttes Grouping sttes Grouping sttes Grouping number time (s) number time (s) number time (s) Tble 4. GROUPING RESULTS FOR Cisco-233 K=5 K=6 K=7 sttes Grouping sttes Grouping sttes Grouping number time (s) number time (s) number time (s) As shown in Tble 2. On Snort-158 rule set, under the sme group number, the sttes number of mdfa lgorithm is less thn tht of the Becchi lgorithm, but the grouping time is the longest. The GRELS lgorithm of locl optimiztion on the totl number of stte hs obviously improved thn the bove two groups, but the grouping time is still longer. While GBLP lgorithm proposed in this work is the smllest in two indictors. The totl number of sttes is reduced by bout 10% compred with the GRELS lgorithm. The grouping time is reduced by more thn 47% of the Becchi lgorithm. Tble 3 shows both effectiveness nd efficiency for L7-Filter-80 rule set. Although the regulr expression of the rule set is more complex, the stte expnsion is more serious. GBLP lgorithm is compred with the current GRELS lgorithm, on the bsis of the totl number of Stte to be reduced, the grouping time is reduced by more thn hlf. As shown in Tble 4, under the sme group number, the stte number of the existing grouping lgorithm is fr greter thn the GBLP lgorithm. GBLP lgorithm runs 5 times nd 3 times fster 1058

6 thn Becchi lgorithm nd GRELS lgorithm, respectively. By the bove nlysis, in the sme number of clusters, the sttes number of GBLP lgorithm nd the grouping time re improved compred with the previous grouping lgorithm. In similr bout the complex L7-Filter Snort rule sets, GBLP lgorithm cn reduce the number of sttes slightly, nd grouping time is reduced by bout hlf. While in the Cisco security pplictions nd other simple rule sets, the lgorithm performnce is more prominent. Compred with the ltest GRELS lgorithm, the grouping time decreses by 3 times. Conclusions In view of the current high-speed network environment, the grouping time of the existing regulr expression grouping lgorithms is longer. And they re difficult to meet the needs of rel-time detection in prcticl ppliction. We propose new fst nd efficient grouping lgorithm, which is efficient regulr expression grouping lgorithm bsed on lbel propgtion (GBLP). Lern from simple nd quick lbel propgtion ides. We give method to determine the initil groups, nd to design the grouping process bsed on the similrity. The experimentl results show tht the proposed GBLP lgorithm hs less number of sttes nd shorter grouping time thn the current grouping lgorithms, which cn chieve the effective blnce between memory occupncy nd grouping time. The future work is to consider the expnsion influence of different chrcteristics, nd further improve the grouping ccurcy of the lgorithm. Literture References [1] Kumr S, Dhrmpurikr S, Yu F, et l. Algorithms to ccelerte multiple regulr expressions mtching for deep pcket inspection[j]. ACM SIGCOMM Computer Communiction Review, 2006, 36(4): [2] Becchi M, Cdmbi S. Memory-efficient regulr expression serch using stte merging[a]. 26th IEEE Interntionl Conference on Computer Communictions (INFOCOM 2007)[C]. IEEE, 2007: [3] QI Y, Wng K, Fong J, et l. FEACAN: Front-end ccelertion for content-wre network processing[a] Proceedings IEEE INFOCOM[C]. IEEE, 2011: [4] Becchi M, Crowley P. A hybrid finite utomton for prcticl deep pcket inspection[a]. Proceedings of the 2007 ACM CoNEXT conference[c]. ACM, 2007:1-12. [5] Yu F, Chen Z, Dio Y, et l. Fst nd Memory-efficient Regulr Expression Mtching for Deep Pcket Inspection[C]. ANCS 06. New York, NY, USA: ACM, 2006: [6] Becchi M, Frnklin M, Crowley P. A worklod for evluting deep pcket inspection rchitectures[c]. In: Proc. of the IISWC Pisctwy: IEEE Computer Society. IEEE,2008: [7] Xu Qin, E Yue-peng, Ge Jing-guo, et l. Efficient regulr expression compression lgorithm for deep pcket inspection[j]. Journl of Softwre, 2009,20(8): [8] Rohrer J, Atsu K, Lunteren JV, et l. Memory-Efficient distribution of regulr expressions for fst deep pcket inspection[c]. CODES+ISSS 2009 Proceedings, Grenoble, Frnce, Oct 11-16, 2009: [9] Fu Z, Wng K, Ci L, et l. Intelligent grouping lgorithms for regulr expressions in deep inspection[c]. Computer Communiction nd Networks (ICCCN), rd Interntionl Conference on. IEEE, 2014: 1-8. [10] Liu Ting-wen, Sun Yong, Bu Dong-bo, et l. 1/(1-1/k)-Optiml lgorithm for regulr expression grouping[j]. Journl of Softwre, 2012,23(9):

7 [11] Zhu X, GHAHRAMANI Z. Lerning from lbeled nd unlbeled dt with lbel propgtion[r]. Technicl Report CMU-CALD , Crnegie Mellon University,