INT J COMPUT COMMUN, ISSN 1841-9836 Vol.7 (2012), No. 3 (Septembe), pp. 509-517 An Entopy-based Method fo Attack Detection in Lage Scale Netwok T. Liu, Z. Wang, H. Wang, K. Lu Ting Liu SKLMS Lab and MOE KLNNIS Lab, Xi an Jiaotong Univesity Xi an, Shaanxi, 710049, P.R.China E-mail: tingliu@mail.xjtu.edu.cn Zhiwen Wang, Haijun Wang, Ke Lu MOE KLNNIS Lab, Xi an Jiaotong Univesity Xi an, Shaanxi, 710049, P.R.China E-mail: wzw@mail.xjtu.edu.cn {hjwang,klu}@sei.xjtu.edu.cn Abstact: Intusion Detection System (IDS) typically geneates a huge numbe of alets with high false ate, especially in the lage scale netwok, which esult in a huge challenge on the efficiency and accuacy of the netwok attack detection. In this pape, an entopy-based method is poposed to analyze the numeous IDS alets and detect eal netwok attacks. We use Shannon entopy to examine the distibution of the souce IP addess, destination IP addess, souce theat and destination theat and datagam length of IDS alets; employ Renyi coss entopy to fuse the Shannon entopy vecto to detect netwok attack. In the expeiment, we deploy the Snot to monito pat of Xi an Jiaotong Univesity (XJTU) campus netwok including 32 C-class netwok (moe than 4000 uses), and gathe moe than 40,000 alets pe hou on aveage. The entopy-based method is employed to analyze those alets and detect netwok attacks. The expeiment esult shows that ou method can detect 96% attacks with vey low false alet ate. Keywods: Netwok Secuity, Entopy-based, IDS, Shannon Entopy, Renyi Coss Entopy. 1 Intoduction Netwok attacks ae defined as the opeations that disupt, deny, degade, o destoy infomation esident in compute netwoks o the netwoks themselves. In ecent yeas, moe and moe netwok attacks theatened the eliability and QoS of Intenet, compomised the infomation secuity and pivacy of uses. KSN (Kaspesky Secuity Netwok) ecoded 73 million Intenet bowses attacks on thei uses in 2009, and that numbe skyocketed to 580,371,937 in 2010 [1]. Symantec epoted that they ecoded 3 billion attacks fom thei global senso and client [2]. Intusion Detection System (IDS) is used to monito and captue intusions into compute and netwok systems which attempt to compomise thei secuity [3]. With the development of netwoks, a lage numbe of compute intusions occu evey day and IDSs have become a necessay addition to the secuity infastuctue of nealy evey oganization. Howeve, IDSs still suffe fom two poblems: 1) lage amount of alets. In fact, moe than 1 million alets ae geneated by Snot each day in ou eseach; 2) high false alets ate. Gina investigated the extent of false alets poblem in Snot using the 1999 DARPA IDS evaluation data, and found that 69% of total geneated alets ae consideed to be false alets [4]. These poblems esult in a huge challenge on the efficiency and accuacy of the netwok attack detection. Seveal methods have been applied to esolve the poblems of lage amount of alets and high false ate. Pietaszek used the adaptive alet classifie to educe false alets, which is tained with lots of labeled Copyight c 2006-2012 by CCC Publications
510 T. Liu, Z. Wang, H. Wang, K. Lu past alets [5]. Wheeas, it is difficult to label lage volume alets geneated in lage-scale netwok. In ode to educe the false alams, Mina popose the extend DPCA to standadize the obsevations accoding to the estimated means [6]. Spathoulas and Katsikas popose a post-pocessing filte based on the statistical popeties of the input alet set [7]. Cisa employ EWMA to detect attacks by analyzing the intensity of alets [3]. In ou eseach, 32 C-class subnets ae monitoed by Snot and moe than 1 million alets ae geneated evey day. Theefoe, we popose a method to spot anomalies which is moe toleable fo the opeato athe than educe false alets. In infomation theoy, entopy is a measue of the uncetainty associated with a andom vaiable, which is widely used to analyze the data and detect the anomalies in infomation secuity. Lakhina et al ague that the distibutions of packet featues (IP addesses and pots) obseved in flow taces eveal both the pesence and stuctue of a wide ange of anomalies. Using entopy as a summaization tool to analyze taffic fom two backbone netwoks, they found that it enables highly sensitive detection of a wide ange of anomalies, augmenting detections by volume-based methods [8]. Bauckhoff ind that entopy-based summaizations of packet and flow counts ae affected less by sampling than volume-based method in lage netwoks [9]. A. Wagne and B Plattne applied entopy to detect wom and anomaly in fast IP netwoks [10]. Relative entopy and Renyi coss entopy can be used to evaluate the similaity of diffeent distibutions. Yan et al use a taffic matix to epesent netwok state, and use Renyi coss entopy to analyze matix taffic and detect anomalies athe than Shannon entopy. The esults show Renyi coss entopy based method can detect DDoS attacks at the beginning with highe detection ate and lowe false ate than Shannon entopy based method [11]. Gu et al poposed an appoach to detect anomalies in the netwok taffic using Maximum Entopy estimation and elative entopy [12]. The packet distibution of the benign taffic was estimated using Maximum Entopy famewok and used as a baseline to detect the anomalies. In this pape, an entopy-based method is poposed to detect netwok attack. The Shannon entopy and Renyi coss entopy ae employed to analyze the distibution chaacteistics of alet featues and detect netwok attack. The expeimental esults unde actual netwok data show that this method can detect netwok attack quickly and accuately. The est of the pape is oganized as follows: the method is intoduced in Section 2, and the expeimental esults ae shown in Section 3. Section 4 is the conclusion and futue wok. 2 Methodology In this pape, Snot is used to monito the netwok and five statistical featues of the Snot alet ae selected. The Shannon entopy is used to analyze the distibution chaacteistics of alet that eflect the egulaity of netwok status. When the monitoed netwok uns in nomal way, the entopy values ae elatively smooth. Othewise, the entopy value of one o moe featues would change. The Renyi coss entopy of these featues is calculated to measue the netwok status and detect netwok attacks. 2.1 Snot Alet and Featue Selection Each Snot alet consists of tens of attibutions, such as timestamp, souce IP addess (sip), souce pot, destination IP addess ( dip), destination pot, pioity, datagam length and potocol, etc. Suppose thee ae n alets geneated in time inteval t. The alets set in time inteval t is denoted as Alet(t) = {alet 1, alet 2,..., alet n }. Assuming thee ae m distinct sip and k distinct dip in Alet(t), we can geneate the distinct souce IP addesses set (SIP) and distinct destination IP addesses set (DIP): S IP = {sip 1, sip 2,..., sip m }, DIP = {dip 1, dip 2,..., dip k }.
An Entopy-based Method fo Attack Detection in Lage Scale Netwok 511 Suppose the numbe of alets come fom sip i is snum i, and the numbe of alets send to dip i is dnum i. The alet numbe of each souce IP (S NUM) and destination IP (DNUM) can be calculated: S NUM = {snum 1, snum 2,..., snum m }, DNUM = {dnum 1, dnum 2,..., dnum k }. Thee ae 4 default pioities of Snot alet: 1, 2, 3 and 4. The theat seveity gadually weakens fom 1 to 4(high, medium, low, info). In ode to stengthen the theat degee of high seveity alets, the theat degee of the alet i is denoted as theat i = 5 (4 pioity alet i ) in pesent wok. Suppose the theat degee sum of all alets come fom sip i is stheat i, and the theat degee sum of all alets send to dip i is dtheat i. The theat degee of each souce IP (S T HREAT) and destination IP (DT HREAT) can be calculated: S T HREAT = {stheat 1, stheat 2,..., stheat m }, DT HREAT = {dtheat 1, dtheat 2,..., dtheat k }. The datagam length is the size of the packet that beaks the alam ules of Snot. We seach the distinct datagam length of all alets, and geneate the datagam length set DGMLEN = {dgmlen 1, dgmlen 2,..., dgmlen x }, whee x is the numbe of the distinct datagam length of all alets. Suppose the numbe of alets whose datagam length equal to dgmlen i is dgmnum i. The alet numbe with diffeent datagam length can be calculated: DGMNUM = {dgmnum 1, dgmnum 2,..., dgmnum x }. Above 5 featues (S NUM, DNUM, S T HREAT, DT HREAT, DGMNUM) ae selected to evaluate the alets and detect attacks. 2.2 Shannon Entopy-based Featue Analysis Shannon entopy is used as measues of infomation and uncetainty [13]. Fo a dataset X = {x 1, x 2, x 3,..., x n }, each data item x belongs to a class x C x. The entopy of X elative to C x is defined as n H(X) = p i log 2 p i (1) whee p i is the pobability of x i in X. The distibution chaacteistics of five featues ae analyzed using Shannon entopy. The entopies of S NU M and DNU M in time inteval t can be calculated H(S ip t ) = H(Dip t ) = The entopy of S T HREAT and DT HREAT can be calculated: H(S theat t ) = m m (snum i /n) log(snum i /n) (2) k (dnum i /n) log(dnum i /n) (3) theat_o f _sip(i) sum_theat ( ) theat_o f _sip(i) log sum_theat (4)
512 T. Liu, Z. Wang, H. Wang, K. Lu H(Dtheat t ) = k theat_o f _dip(i) sum_theat ( ) theat_o f _dip(i) log sum_theat whee theat_o f _sip(i) is the theat sum of the alets fom sip i, theat_o f _dip(i) is the theat sum of the alets to dip i, and sum_theat is the theat sum of all the alets in ALERTS which can be calculated using n sum_theat = theat i (6) The entopy of datagam length is H(Dgmlen t ) = x (dgmnum i /n) log(dgmnum i /n) (7) Afte calculating the entopies of above featues, we can use an entopy vecto V(t) = [H(S ip t ), H(Dip t ), H(S theat t ), H(Dtheat t ), H(Dgmlen t )] to epesent the netwok status of time inteval t. 2.3 Renyi Coss Entopy-based Attack Detection The Renyi entopy, a genealization of Shannon entopy, is a measue fo quantifying the divesity, uncetainty o andomness of a system. The Renyi entopy of ode α is defined as H α (P) = 1 1 α log 2 p α (8) whee 0 < α < 1, P is a discete stochastic vaiable, and p is the distibution function of P [14]. Highe values of α, appoaching 1, giving a Renyi entopy which is inceasingly detemined by consideation of only the highest pobability events. Lowe values of α, appoaching zeo, giving a Renyi entopy which inceasingly weights all possible events moe equally, egadless of thei pobabilities. The special case α 1 gives the Shannon entopy. The Renyi coss entopy of ode α is deived as I α (p, q) = 1 1 α log 2 p α q α 1 whee p and q ae two discete vaiables, p and q ae thei distibution functions [14]. If α = 0.5, the Renyi coss entopy is symmetic, which means I α (p, q) = I α (q, p). In the est of the pape, when efeing to the coss entopy we mean the symmetic case I 0.5 (p, q) = 2 log 2 p q (10) The Renyi coss entopy is used to fuse the values of diffeent featues. As mentioned above, we use an entopy vecto V(t) = [H(S ipt), H(Dipt), H(S theatt), H(Dtheatt), H(Dgmlent)] to epesent the netwok status of time t, thus the netwok status can be viewed as a time seies of entopy vecto V(1), V(2),..., V(t). Befoe calculating Renyi coss entopy, V(t) is unitized to whee V(t) = [ H(S ip t ), H(Dip t ), H(S theat t ), H(Dtheat t ), H(Dgmlen t )] (11) H(S ip t ) = H(S ip t )/H sum H(S theat t ) = H(S theat t )/H sum H(Dip t ) = H(Dip t )/H sum (12) H(Dtheat t ) = H(Dtheat t )/H sum H(Dgmlen t ) = H(Dgmlen t )/H sum (5) (9)
An Entopy-based Method fo Attack Detection in Lage Scale Netwok 513 and Hsum = H(S ip t ) + H(Dip t ) + H(S theat t ) + H(Dtheat t ) + H(Dgmlen t ). To detemine if thee is any change in the netwok at time t compae with pevious time t 1, we use the following equation to calculate the Renyi coss entopy of V(t) and V(t 1) I 0.5 ( V(t), V(t 1)) = 2 log 2 p (t 1)p (t) (13) We set η as the theshold of I0.5 ( V(t 1), V(t)) to test whethe thee is a change. The choice of theshold η is netwok dependent and it can be set as expeience. Since ou pupose is to detect netwok attack, it is not enough to compae netwok status of time t to its pevious time t 1, unless we make sue that no attack occus in time t 1. Thus, the aveage of the latest n nomalized Shannon Entopies is employed to eplace the t 1, called V(t, n) V(t, n) = 1 n n V(t i) (14) Then, we calculate the Renyi coss entopy of V(t) and V(t, n), and netwok attack is detected if its absolute is geate than η. I 0.5 ( V(t, n), V(t)) = 2 log 2 p (t, n)p (t) (15) 3 Expeiment Results 3.1 Data Collection In the eseach, we have used Snot to monito 32 C-class subnets in the Xi an Jiaotong Univesity campus netwok fo two weeks, which include moe than 4,000 uses. In this pape, we select the alets gatheed in 2010-12-6. Thee ae 862,284 alets with 65 signatues, which come fom 42,473 distinct souce IP addesses and send to 11,790 distinct destination IP addesses. Figue 1: The statistical esults of alets (2010-12-6). As shown in Fig.1, fou statistical featues of alets display the tend as the people living customs and habits (the time inteval set as 5 seconds). Few alets ae geneated in the middle night; then, moe alets ae detected fom 8:00 to 10:00 when students get up successively; the alets keep the same tend fom 10:00 to 23:30; the alets collapse at last 30 minutes, since netwok constaint due to the domitoy administating ules. At the same time, the statistical featues change abuptly in some time intevals. In geneal, these abnomal upheavals ae the sign of the faults o netwok attacks.
514 T. Liu, Z. Wang, H. Wang, K. Lu We select two alets sets in diffeent time peiod as taining and test data set: Taining data set includes 170,516 alets geneated fom 10:00 to 14:00. These alets come fom 13,148 IP addesses and send to 7,570 IP addesses. By analyzing these alets manually, we identify 87 host scan attacks, 5 pot scan attacks, 1 DoS attack and 1 host intusion. Test data set includes 578,389 alets geneated fom 14:00 to 23:30. These alets come fom 29,327 IP addesses and send to 10,590 IP addesses. By analyzing these alets manually, we identify 203 host scan attacks, 7 pot scan attacks, 6 DoS attack, 3 host intusion and 1 wom attack. 3.2 Entopy-based Attack Detection The taining data is evaluated by Shannon entopy, as shown in Fig. 2 (a). We emove the alets associated to tue attacks, which called as Attack Alet. The emaindes ae called as Flase Alet. We e-evaluate the Noise Alet in the taining data set, as shown in Fig. 2 (b). The Shannon entopies ae elatively smooth when no attack occus; othewise, one o some of the values would change abuptly. (a) All alets (b) False alets Figue 2: Shannon entopy. Although the Shannon entopies eflect the egulaity of netwok status, it is difficult to detect attack diectly by using five fixed thesholds. Because the Shannon entopy value vaies with the activities of end uses even the netwok uns in nomal way. In ou expeiment, the Renyi coss entopy is used to fuse the Shannon entopy of five statistical featues to detect attack. As shown in Fig. 3, we calculate the Renyi coss entopy of the alets in tain data set using (13). It is clealy shown that 1) the Renyi coss entopy will change shaply when the netwok ae attacked, see Fig. 3 (a); 2) the Renyi coss entopy will be close to 0 without the lage-scale netwok attacks and failues, see Fig. 3 (b). Thus, it is easy to detect attack using fixed theshold. In the expeiments, when η detect = 0.016, 84 attacks can be detected fom 94 attacks with 11 false detections. 81 host scan attacks can be detected fom 87 host scans. The missed scan attacks last fo a elative long time and with small scan density. 1 pot scan is detected fom 5 pot scans. 1 host intusion and 1 DoS attack ae detected successfully. Accoding to (14) and (15), the n and η ae impotant fo the accuacy of attack detection. In the expeiments, we set η base = { 0.001, 0.002, 0.003,..., 0.04} and n = {5, 10, 15,..., 200}. Fo each combination of η base and n, the taining data is analyzed in the following method. Fistly, each V(t) is unitized to V(t) using (11) and (12); Secondly, the Shannon entopy can be calculated using (14). Its unitized fom is V(t, n). Finally, V(t) is compaed with V(t, n) using (15) to calculate Renyi coss entopy value.
An Entopy-based Method fo Attack Detection in Lage Scale Netwok 515 (a) All alets (b) False alets Figue 3: Renyi coss entopy. In the expeiment, ROC (Receive Opeating Chaacteistic) is used to descibe the detection esults. ROC is a gaphical plot of tue positive ate and false positive ate [15]. Fig. 4(a) shows the ROC cuve of detection esults in taining data, whee the size of NTS n and base theshold η base equals (5, 0.005), (50, 0.02) and (100, 0.04) sepaately. When detection theshold η detect comes to 0, almost all the time intevals ae detected as netwok attack. Thus, the detection false positive ate and hit ate ae both nea 100%. A detection esult with high hit ate and low false ate is consideed to be a good esult. In this case, the ROC cuve is plotted at the top left cone, and the AUC value (Aea Unde ROC Cuve) has lage value. In this pape, we use AUC value to evaluate the detection esults. The best combination of n and?base can be obtained using taining data. As shown in Fig. 4(b), the AUC values of all the combinations ae calculated, and the highest AUC is 0.9962 when n = 95 and η base = 0.022. (a) ROC (b) AUC Figue 4: Detection esult on taining data set. 3.3 Testing The test data set is analyzed to detect the attacks using entopy-based method. As shown in Fig. 5, 211 attacks can be detected fom 220 attacks (detection ate is as high as 96%) with 8 false detections. 197 host scan attacks can be detected fom 203 host scans. 4 pot scans ae detected fom 7 pot scans. 3 host intusions, 1 wom attack and 6 DoS attacks ae detected successfully.
516 T. Liu, Z. Wang, H. Wang, K. Lu Figue 5: Attack detection esults on test data set. 4 Conclusion In this pape, a new netwok attack detection method based on entopy is poposed. The souce IP, destination IP, alet teat and alet datagam length ae selected fom tens of Snot alet attibutions. The Shannon entopy is used to analyze the alets to measue the egulaity of cuent netwok status. The Renyi coss entopy is employed to fuzz the Shannon entopy on diffeent featues to detect netwok attacks. In the expeiments, the netwok taffic of moe than 4000 uses in 32 C-class netwok ae monitoed using Snot. 748905 alets, geneated fom 10:00 to 23:30 Dec. 6 2010, ae selected and sepaated into taining data set and test data set. The expeiments show that the Renyi coss entopy value is nea 0 when the netwok uns in nomal, othewise the value will change abuptly when attack occus. The attack detection ate of entopy method is as high as 96% with only 8 false alets. In next step, moe alets fom diffeent time segments will be collected to test ou method and an attack classification method will be consideed. Acknowledgment This wok was suppoted by the National Natual Science Foundation (60921003, 60970121, 91018011), National Science Fund fo Distinguished Young Scholas (60825202) and the Fundamental Reseach Funds fo the Cental Univesities. Bibliogaphy [1] A. Gostev, "Kaspesky Secuity Bulletin. Malwae Evolution 2010," Kaspesky, 2011. [2] M. Fossi, G. Egan, K. Haley, E. Hohnson, T. Mack and A. Et, "Symantec Global Intenet Secuity Theat Repot Tends fo 2010," Symantec, 2011. [3] P. Cisa, S. Bosnjak and S. M. Cisa, "EWMA Algoithm in Netwok Pactice," Intenational Jounal of Computes, Communications & Contol, vol.5, pp. 160-170, 2010. [4] G. C. Tjhai, M. Papadaki, S. M. Funell and N. L. Clake, in Lectue Notes in Compute Science (including subseies Lectue Notes in Atificial Intelligence and Lectue Notes in Bioinfomatics), Tuin, Italy, 2008, pp. 139-150. [5] T. Pietaszek, "Using Adaptive Alet Classification to Reduce False Positives in Intusion Detection- Recent Advances in Intusion Detection," vol.3224, pp. 102-124, 2004.
An Entopy-based Method fo Attack Detection in Lage Scale Netwok 517 [6] J. Mina and C. Vede, "Fault detection fo lage scale systems using Dynamic Pincipal Components Analysis with adaptation," Intenational Jounal of Computes, Communications & Contol, vol.2, pp. 185-194, 2007. [7] G. P. Spathoulas and S. K. Katsikas, in 2009 16th Intenational Confeence on Systems, Signals and Image Pocessing, IWSSIP 2009, Chalkida, Geece, 2009. [8] A. Lakhina, M. Covella and C. Diot, in Compute Communication Review, New Yok, United States, 2005, pp. 217-228. [9] D. Bauckhoff, B. Tellenbach, A. Wagne, M. May and A. Lakhina, in Poceedings of the ACM SIGCOMM Intenet Measuement Confeence, IMC, Rio de Janeio, Bazil, 2006, pp. 159-164. [10] A. Wagne and B. Plattne, in Poceedings of the Wokshop on Enabling Technologies: Infastuctue fo Collaboative Entepises, WET ICE, Linkoeping, Sweden, 2005, pp. 172-177. [11] R. Yan and Q. Zheng, "Using Renyi coss entopy to analyze taffic matix and detect DDoS attacks," Infomation Technology Jounal, vol.8, pp. 1180-1188, 2009. [12] Y. Gu, A. McCallum and D. Towsley, "Detecting anomalies in netwok taffic using maximum entopy estimation," in Poc. 2005 Poceedings of the 5th ACM SIGCOMM confeence on Intenet Measuement, pp. 32. [13] C. E. Shannon, "A mathematical theoy of communication," SIGMOBILE Mob. Comput. Commun. Rev., vol.5, pp. 3-55, 2001. [14] C. E. Pfiste and W. G. Sullivan, "Renyi entopy, guesswok moments, and lage deviations," IEEE Tansactions on Infomation Theoy, vol.50, pp. 2794-2800, 2004. [15] A. P. Badley, "The use of the aea unde the ROC cuve in the evaluation of machine leaning algoithms," Patten Recognition, vol.30, pp. 1145-1159, 1997.