Zhiyong Huang* and Xiaoping Zeng

Transcription

1 44 Int. J. Internet Protocol Technology, Vol. 5, Nos. 1/2, 2010 Detectng and blockng P2P botnets through contact tracng chans Zhyong Huang* and Xaopng Zeng College of Communcaton Engneerng, Chongqng Unversty, Chongqng, , P.R. Chna E-mal: E-mal: *Correspondng author Yong Lu Department of Electrcal and Computer Engneerng, Polytechnc Insttute of NYU, 5 Metrotech Center, Brooklyn, NY 11201, USA E-mal: yonglu@poly.edu Abstract: Peer-to-peer (P2P) botnets have recently become serous securty threats on the nternet. It s dffcult to detect the propagaton of P2P botnets by solated montorng on ndvdual machnes due to ts decentralsed control structure. In ths paper, we propose a contact tracng chan-based framework to detect and block P2P botnets by tracng contact behavours among peers. In the proposed framework, the contacts of peers wth suspcous symptoms are traced and tracng chans are establshed to correlate contacts among peers wth ther abnormal symptoms. Peers are confrmed wth nfectons when the length of contact tracng chan that they belong to reaches a preset threshold. Under ths framework, we develop dfferent detecton, tracng and mmunsaton strateges. Through numercal smulatons, we demonstrate that the proposed contact tracng framework can quckly detect and block the propagaton of P2P botnets. Keywords: botnet; P2P; worm; contact tracng; transmsson chan; entropy; threshold; detecton; blockng; mmunsaton; smulaton; protocol. Reference to ths paper should be made as follows: Huang, Z., Zeng, X. and Lu, Y. (2010) Detectng and blockng P2P botnets through contact tracng chans, Int. J. Internet Protocol Technology, Vol. 5, Nos. 1/2, pp Bographcal notes: Zhyong Huang s currently a PhD canddate n the College of Communcaton Engneerng, Chongqng Unversty. He receved hs MS from Chongqng Unversty n In 2007, he was a Vstor Researcher at the Department of Electrcal and Computer Engneerng, Polytechnc Insttute of NYU. He currently works at the College of Communcaton Engneerng at Chongqng Unversty. Hs research nterests nclude network securty and network modellng, smulaton and analyss. Xaopng Zeng receved hs BS, MS and PhD from Chongqng Unversty n 1982, 1987 and 1996, respectvely. He s currently a Professor of College of Communcaton Engneerng at Chongqng Unversty. Hs research nterests nclude network securty, control and systems methods n communcaton networks. Untl now, he has publshed more than 70 techncal papers on relevant feld. Yong Lu has been an Assstant Professor at the Electrcal and Computer Engneerng Department of Polytechnc Insttute of NYU snce He receved hs PhD from Electrcal and Computer Engneerng Department at the Unversty of Massachusetts, Amherst n He receved hs Masters and Bachelors n the feld of Automatc Control from the Unversty of Scence and Technology of Chna n 1997 and 1994, respectvely. Hs general research nterests le n modellng, desgn and analyss of communcaton networks. Hs current research drectons nclude robust network routng, peer-to-peer IPTV systems, overlay networks and network measurement. He s a member of IEEE and ACM. Copyrght 2010 Inderscence Enterprses Ltd.

2 Detectng and blockng P2P botnets through contact tracng chans 45 1 Introducton In recent years, botnets have been frequently utlsed by attackers to launch malcous attacks on the nternet. A botnet s a network of compromsed peers (bots), whch are controlled by an attacker to launch DDOS attacks, dstrbute spam e-mals, etc. Usng botnets, attackers can launch attacks from thousands or even mllons of dstrbuted bots. Early botnets adopted the clent-server-based system archtecture. All bots n a botnet connect to some command and control (C&C) servers. Bots receve commands from C&C servers usng protocols lke Internet Relay Chat (IRC). One drawback of ths clent-server archtecture s that, due to the heavy traffc between servers and bots, the C&C servers can be easly detected by defenders. And the entre botnet wll be shut down once the C&C servers are blocked. More recent botnets employ the peer-to-peer (P2P) system archtecture. In a P2P archtecture, there s no fxed C&C servers. Every bot n the botnet acts as both clent and server. Bots communcate and exchange nformaton wth each other usng exstng or customsed P2P protocols. Attackers dsguse themselves as normal peers and dssemnate ther commands to all peers n the same P2P botnets. Due to the dstrbuted nature of P2P botnets, t s very challengng to detect and block them. In ths paper, we propose a novel contact tracng chan-based approach to the detecton and blockng of P2P botnets. The man challenges n botnet detecton are: 1 P2P botnet are now under wdespread development. Some P2P botnets use exstng P2P protocols, whle others develop customsed protocols (Grzzard et al., 2007). For example, Phatbot (Stewart, 2004) uses code from the WASTE project to mplement P2P, but Nugache (Lemos, 2006) and SpamThru (Stewart, 2004) use ther own P2P protocols for communcaton between peers. Most of the customsed protocols are encrypted. It s very dffcult to detect the sgnallng among bots n P2P botnets through protocol analyss. 2 To escape from traffc analyss-based detecton, some ntellgent P2P botnets delberately changed ther nfecton tactcs [e.g., reducng the sze of ther network (Hggns, 2007)]. It s hard to dstngush a P2P botnet from a normal P2P network by lookng nto the traffc volume and contacts between peers. 3 Snce there s no centralsed C&C servers, P2P botnets are very reslent. A bot can receve C&C sgnal from any of ts peerng neghbours. When the number of peerng contacts of each peer s reasonably large, the whole P2P botnet has very robust connectvty and can reman connected even after a large porton of bots are detected and removed from the botnet. To address the prevously descrbed challenges, we develop a contact tracng chan-based P2P botnet detecton and mmunsaton approach. Contact tracng has been successfully appled n dsease control (Huerta and Tsmrng, 2002; Hymana et al., 2003; Eames and Keelng, 2003). In our paper, we attempt to use contact tracng chans to detect bots. The contrbutons of our work are summarsed as follows: 1 We apply the concepts of contact tracng and transmsson chan to the detecton and mmunsaton of P2P botnets. We developed a tracng-chan-based framework that can effcently dentfy and block P2P botnets. 2 Smple threshold-based contact detecton scheme cannot detect ntellgent botnets. We develop a novel entropy-based detecton approach to dstngush botnets from normal P2P networks. 3 We propose several tracng-chan algorthms to strke the rght balance between the effcency and the accuracy of contact tracng. 4 We mplement a dscrete tme smulator and conduct extensve smulatons to study the effcency and robustness of the proposed contact tracng framework under dfferent network and system settngs. 2 Background and related works 2.1 P2P botnets P2P botnets are quckly becomng one of the most sgnfcant threats on the nternet today. P2P botnets are networks of compromsed peers controlled by attackers through P2P communcaton protocols. P2P botnets have robust network connectvty. They hde ther actvtes through sgnal encrypton and traffc dsperson. Consequently, t s very dffcult to trace and shut down P2P botnets. Grzzard et al. (2007) descrbed the development of P2P botnet and ntroduced several P2P control archtectures, such as Slapper, Snt, Phatbot and Nugache. Many P2P networks are becomng the favourte places for malware to spread (Dhungel et al., 2007; Lang et al., 2005). Worm s wdely adopted by P2P botnet n the wld due to ts automatc propagaton characterstcs. As an example, Storm Worm botnet s the frst major botnet to use P2P network for C&C (Porras et al., 2007). Smlar to other e-mal worms (e.g., Loveletter/ILOVEYOU and Bagle) (Holz et al., 2008), the e-mal body contans an embedded lnk or an attachment wth names such as FullVdeo.exe, Vdeo.exe and FullClp.exe. Attackers use socal engneerng technques to pretend to be a legtmate e-mal and trck the recpent nto openng the attachment or clckng on the embedded lnk. Once a vctm attempts to open the attached fle or clck the baleful lnk, t wll become an nfected peer. After the Trojan nstalls the ntal nfecton fles, the vctm wll attempt to connect peers n the Storm Worm botnet. Subsequently, t wll download the full payload and become a real bot under the control of the botmaster.

3 46 Z. Huang et al. Wthn the propagaton stage, the number of some net-packets (e.g., ICMP, UDP and SMTP) sharply ncreased (Kang et al., 2009), the hgh contact rate s the most mportant characterstc for worm detecton we consdered n ths paper; t wll be ntroduced n Secton 4. Some works have been done by researchers on P2P botnets defences. Zhou et al. (2005) gave a frst look at P2P worm defence. They proposed a framework where a small fracton of guardan nodes are employed to detect control flow hjacks of vulnerable programs usng schemes smlar to those n Crandall and Chong (2004), Newsome and Song (2005) and Suh et al. (2004), the approach whch deployed the same P2P network as P2P worm to propagate alerts s smlar to our strategy for buldng tracng chan. Gu et al. (2008) presented a general detecton framework (BotMner prototype system) that s ndependent of botnet C&C protocol and structure, and requres no a pror knowledge of botnets, the key of ths approach s focused on the net-behavour detecton; Holz et al. (2008) estmated the total number of the bots through nfltratng and analysng n-depth the Storm Worm botnet and mtgates ths botnet through dsruptng the communcaton channel, such as eclpsng content, pollutng; ths paper descrbed the typcal characterstc of Storm Worm. Our system adopted contact tracng scheme based on the net-behavour detecton of sngle node to dentfy the nfected nodes. 2.2 Contact tracng Contact tracng s a classc epdemc control method. It s frequently used to combat many nfectous dseases (such as TB and AIDS) and new nvadng pathogens (such as SARS). When an ndvdual s dentfed as havng a communcable dsease, any one has contacted wth the ndvdual has a hgh probablty to be nfected. Tracng based on contact hstory can quckly dentfy and block the propagaton of nfectons. Eames and Keelng (2003) used the parwse approach and full stochastc smulatons to nvestgate the effcency of contact tracng n dsease control. Hymana et al. (2003) used random screenng and contact tracng to reduce the spread of HIV. Huerta and Tsmrng (2002) developed the mean-feld model of contact tracng for the case of random graphs. The dea of contact tracng has recently been appled to network securty feld. Xong (2004) brought the concepts of contact tracng and transmsson chan nto e-mal network worm and vrus control. In our paper, ths approach s appled to detect and block P2P botnets. 3 Contact tracng framework In ths secton, we present the overall framework of contact tracng for P2P botnets detecton and control. Smlar to epdemc dsease control, we classfy peers nto dfferent states accordng to ther contact records and symptoms. In P2P botnets, a contact s defned as the establshment of a connecton between a par of peers. The nfectous symptom on a peer s defned as contacts at a rate hgher than a preset threshold. We classfy peers nto the followng fve possble states: 1 Normal: A peer that has no nfectous symptom s n the normal state. 2 Connected: A peer that has been contacted by a probable peer or a suspcous peer s n the connected state. 3 Suspcous: A peer that has nfectous symptom s n the suspcous state. 4 Probable: A suspcous peer s confrmed wth nfecton and declared as probable f t s on an establshed contact tracng chan (see detals below). 5 Immunsed: A probable peer that has been cleaned and patched wll change to the mmunsed state. Fgure 1 Dagram of peer state transtons Lnked Infectous symptom Chan s establshed Bot s mmunzed Fgure 1 shows the peer state transton: 1 Normal suspcous: A normal peer changes to a suspcous peer when t frst has the nfectous symptom. 2 Normal connected: A normal peer changes to a connected peer when t s frst contacted by a suspcous peer or a probable peer. 3 Connected suspcous: A connected peer changes to a suspcous peer when t has nfectous symptom.

4 Detectng and blockng P2P botnets through contact tracng chans 47 4 Suspcous probable: A suspcous peer changes to a probable peer when the tracng chan whch t belongs to s establshed. 5 Probable mmunsed: A probable peer changes to an mmunsed peer when t s cleaned and patched. The key of contact tracng-based detecton s to montor and control peer state transtons. It conssts of three components: detecton process, tracng process and mmunsaton process. The detecton process s to detect peer nfectous symptoms. The tracng process s to track the contact hstory of suspcous peers and establsh contact tracng chans to confrm peer nfectons. The mmunsaton process s to clean and patch the nfected peers. At frst, the detecton process montors the contact rate of peers. If a peer sends out more than a preset threshold M connectons wthn an nterval δ, t wll be dentfed as an abnormal peer and ts state wll be converted from normal to suspcous. Secondly, the tracng process tracks the contacts of all suspcous peers through recordng lnks between a suspcous peer and any peer that t contacts wth. If a normal peer s contacted by a suspcous peer, ts state wll be converted to connected. Consequently, f any of those contacted peers s detected wth nfectous symptom, that peer wll be also classfed as suspcous and the new contact lnks wll be recorded by the tracng process. If ndeed the nfecton propagates through peer contacts, the tracng process wll dentfy a chan of contact lnks. A complete tracng chan s establshed when the length of contact tracng chan reaches a preset threshold K and the tracng process can declare confrmed nfectons for all peers on the tracng chan. The peers on the chan wll convert ther state from suspcous to probable. Immunsaton wll then be appled to the newly dentfed probable peers. In the followng sectons, we present the detaled desgns of detecton, tracng and mmunsaton processes. 4 Symptom detecton process In the prevous secton, we talked about a smple threshold-based symptom detecton algorthm. However, the network condtons are so complex that we cannot always use a sngle fxed threshold to dstngush between legal traffc and llegal traffc. Many worms adopted new tactcs to escape from smple threshold-based detecton (e.g., low-rate contact strategy). It wll lead to a hgh false alarm rate n the detecton process (e.g., some peers runnng normal P2P software are mstakenly consdered as suspcous bots by the smple threshold-based detecton wth a low threshold, whereas some bots adopted low-rate contact strateges are mstakenly mssed by the smple threshold-based detecton wth a hgh threshold). Even though the transmsson chan approach can tolerate some false alarms on ndvdual contact tracng lnks 1, we stll want to mprove the detecton process to brng down the false alarm rate of the detecton process. In ths paper, we propose to dstngush P2P botnets from legtmate P2P systems by nvestgatng major contact-level characterstcs. If we have a long observaton perod of T, we dvde T nto n tme slots. V ( k ) denotes the number of connected peers n the tme slot k. We record p( k ), the rato of the number of newly connected peers wthn tme slot k to the total number of connected peers n the whole observaton perod T. For P2P botnets, pk ( ) takes large values only for the nfecton tme slots. For legtmate P2P system, pk ( ) s more unformly dstrbuted across all tme slots n the observaton perod. More specfcally, we can calculate the entropy (Shanon, 1948) of pk ( ): ( ) ( ) log ( ). n H p = n p k p k (1) k = 1 2 We can dstngush P2P bots from normal P2P systems by computng ther entropy values of pk ( ) n the observaton perod. Normal P2P systems have unformly low contact rate and thus they have a hgh entropy value. Bots usually have a smlar structure of worm propagaton. The entropy value of p( k ) s low. To further mprove the detecton accuracy, we augment the entropy-based detecton by nvestgatng addtonal peer nformaton, such as the contact ports, the arrval tme, the actve tme, the frequency and content length of contact (Husna et al., 2008). 5 Contact tracng chan establshment The core of the proposed contact tracng framework s to track the contactng behavours of suspcous peers and establsh contact tracng chans to confrm nfectons. The algorthms used to buld tracng chans determne the effcency and the accuracy of botnet detecton and blockng. On one hand, the algorthms should be effcent and can quckly dentfy and block ongong botnet propagaton. On the other hand, the algorthms should be accurate and rase as few false alarms as possble. In ths secton, we present three tracng chan algorthms to trade off the effcency and accuracy. 5.1 Basc chan algorthm We frst ntroduce a basc algorthm. In ths basc algorthm, each node s ntally assgned wth layer ID 0. For the frst tme a peer s contacted by a peer wth suspcous symptom, t changes ts layer ID from 0 to one plus the layer ID of the suspcous peer. A peer s layer ID wll not be changed for later contacts from other suspcous peers. A trace chan s establshed f some peer s layer ID reaches the chan length threshold K. Ths algorthm can be easly mplemented. However, t can only dentfy smple contact tracng chans consstng of contact lnks dscovered sequentally.

5 48 Z. Huang et al. For the network n Fgure 2, the contact threshold s M = 1 and the tracng chan threshold s K = 3. At tme 1, node B and C are contacted by node A, A s root node whose layer ID s 0. At tme 2, node B contacts node D, node B changes ts layer ID to 1. At tme 3, node B s contacted by node C, but B wll keep ts layer ID of 1. At tme 4, node E contacts node A. Node A wll not change ts ID and ths nfecton wll be gnored. No tracng chan s detected by ths algorthm wth threshold K = 3 even though we do have a chan of fve peers and four of them have suspcous symptoms. Ths wll largely slow down the botnet detecton speed. Fgure 2 Establshment of tracng chans n a smple network 5.2 Longest-chan algorthm To address ths problem, we present another algorthm that mproves the detecton speed by dentfyng the longest contact tracng chan for each peer. We buld the contact graph wth nodes beng peers and lnks beng contacts ntated by suspcous peers. A contact tracng chan then corresponds to a path n the contact graph. When a node s contacted by multple suspcous nodes, ths node s placed smultaneously on multple paths/chans n the contact graph. The longest-chan algorthm declares confrmed nfecton for a peer f the length of the longest contact path/chan that the peer s on reaches the chan length threshold K. To mplement ths algorthm usng peer layer ID, the layer ID of each peer s no longer fxed after the frst contact from a suspcous peer. When peer A s contacted by a suspcous peer B, peer layer IDs are updated as follows: f B s ID s larger than or equal to A s ID, set A s ID to one plus B s ID, also update the layer IDs of A s downstream peers n contact chans. More specfcally, each node has two state varables: ( P, L ), where P denotes the parent node of, n other words, P nfected, L s the layer of node. Intalsng L = 0 and P =. Assumng K s the preset chan length threshold, S s the set of nodes. When node m s contacted by a suspcous node n S, the longest-chan algorthm updates node states as n Table 1. Table 1 Algorthm Longest-chan tracng algorthm Node m s nfected by node n Procedure: Updatng the Tracng Chan f ( ) L L then n Pm m = n. L = L +1. m n Updatng the other nodes on ths chan. end f Procedure: Confrmng the Tracng Chan. for (all nodes r, r S) do f ( L ) r K then The suspcous nodes on ths chan are confrmed bots. end f end for Now we go back to the example n Fgure 2. The algorthm works the same as the basc algorthm up to tme 2. At tme 3, node B s nfected by node C, they all have layer ID 1, so node B s ID s changed to 2 accordng to our algorthm, node D s layer ID remans 0 because t has no suspcous symptom. At tme 4, root node A s nfected by node E, node E replaces node A to become the root node, correspondngly, node A s layer ID s set to 1, node C and B s layer IDs are set to 2 and 3, respectvely. The length of ths tracng chan reaches K = 3. Thus, a complete tracng chan s establshed and peer E, A, C and B are all confrmed wth nfectons. 5.3 Causal chan algorthm The longest-chan algorthm has a good tracng speed, but t has a weakness. It gnored the temporal order of contact lnks. As a result, the actual nfectons among peers do not necessarly follow the order that peers appear n a tracng chan. For the prevous example, E s the root node, but snce A, B and C all have symptom before E does. There s no causal relaton between E s symptom and symptoms on the other four nodes. Basc algorthm has consdered the causal relatons between nfectons. However, for each peer, t only consders ts frst nfecton lnk. Its detecton speed s too slow. To mprove the detecton effcency whle keep the causal relaton along tracng chans, we propose a causal chan algorthm that combnes the advantages of the basc algorthm and the longest-chan algorthm. Each node r S can potentally ntate the nfecton and be the root for a tracng chan. At the same tme, each

6 Detectng and blockng P2P botnets through contact tracng chans 49 node S, r can potentally be nfected by the nfecton orgnated at r and be placed on the tracng chan rooted at r. Therefore, each node should keep S layer IDs, one for each chan rooted at any r S. The state of each node can be descrbed by a tuple vector L r, P r, r S, L r denotes the layer of {( ( ) ( )) } where ( ) node relatve to root node r, and ( ) P r denotes the parent node of node relatve to root node r. The peer states are ntalsed as follows: 1 f r L ( r) = ; P ( r) = 0 f = r ( ) = 1 L r denotes node has not been nfected by any nodes on the tracng chan rooted at r. When = r, ( ) = 0 L r denotes every node s at layer 0 relatve to tself; and P ( r) = denotes node s the root node. Table 2 Algorthm Causal chan algorthm Node s nfected by node j Procedure: Updatng the Tracng Chan. ( ) =. P j j for (all nodes r, r S) do ( f Pj ( r) j and ( ) = 1) j ( ) = ( )( ) + 1. L r L r ( ) = j. P r Pj r L r then end f end for Procedure: Confrmng the Tracng Chan. f ( Lj ( r ) reaches the preset K ) then whle (j s not root node) do node j and Pj ( r ) are confrmed bots. node j state s ntalsed. j = P j ( r ). end f When node s nfected by node j, the algorthm can be descrbed as: 1 node obtans all the layer nformaton of node j and makes node j as ts parent node relatve to root node j 2 for any root node r that j connects to, f node has not been added to the tracng chan rooted at r( r j ), update the layer nformaton of node j and make node j as parent node relatve to root node r (2) 3 when L ( r ) reaches the preset threshold K, the nodes on the tracng chan rooted at node r are confrmed wth nfectons. Note the state of each confrmed node must be ntalsed mmedately because they have probablty to be contacted through another tracng chan rooted at the same node r. The detaled algorthm s descrbed n Table 2. Fgure 3 llustrates an example of the causal chan algorthm. In the example, there are seven nodes. The contact lnks between nodes are labelled wth the tme that contact happened, wth t 1 < t 2 < t 3 < t 4 < t 5 < t 6 < t 7. Each node s labelled wth non-trval state varables. Fgure 3 Causal chan nformaton passng among peers There are three potental tracng chans. Chan 1 conssts of nodes 1, 2, 3, 4 and 7. Chan 2 conssts of nodes 1, 2, 3, 6 and 4. Chan 3 conssts of nodes 5, 3, 6 and 4. If the chan length threshold s K = 3, chan 1 and chan 2 have reached the threshold. Thus, nodes 1, 2, 3, 4 and 6 are confrmed bots, note that because t 6 < t 7, there s no causal relaton between node 4 s symptom and node 6 s symptom, node 4 s confrmed through chan 1. Node 7 cannot be confrmed bot because t does not have nfectous symptom. It should be noted that node 6 was nfected by node 3 at t 5. Snce node 3 has been nfected twce by two dfferent nodes at t 2 and t 4, and t2 < t4 < t 5, thus, node 3 passes two sets of layer nformaton of chans 2 and 3 to node 6, and node 6 s confrmed wth nfecton through chan Dynamc chan length threshold In the tracng chan framework, the chan length threshold K s a very mportant parameter. To reduce false alarms, we should use a large K ; and to ncrease the dscovery speed, we can use a small K. In ths secton, we propose to dynamcally adjust K at dfferent stages of botnet propagaton to strke the rght balance between the detecton speed and false alarm rate. The typcal botnet propagaton perod can be dvded nto three stages: early, mddle and late. Let I be the number of already nfected nodes, Δ I be the new nfecton rate, whch s defned as the number of newly nfected peers wthn unt tme. Each perod can be charactersed as follows:

7 50 Z. Huang et al. 1 Early stage: The number of nfected peers I s small and the new nfecton rate Δ I s low. 2 Mddle stage: The number of nfected peers I s medum and the new nfecton rate Δ I s hgh. 3 Late stage: The number of nfected peers I s large and the new nfecton rate Δ I s low. Our strategy s to use dfferent K at dfferent stages. We use large K when the nfecton degree s low for accuracy. And we use small K when the nfecton degree s hgh for speed. 6 Immunsaton strateges Immunsaton s to use antvrus software to clean and patch the confrmed bots. Our focus s not on developng bot cleanng and patchng tools. Our goal s to nvestgate dfferent mmunsaton strateges and ther effectveness. We consder two mmunsaton strateges aganst P2P botnets. When bots are dscovered, the mmunsaton should be conducted as soon as possble. Due to resource constrants, not all bots can be mmunsed at the same tme. The order that bots are mmunsed affect the effectveness of the mmunsaton. The smplest strategy s random mmunsaton. All dscovered bots are treated equally and have the same probablty to be mmunsed. Each bot s selected at random to be mmunsed. Ths method does not dfferentated nodes and s easy to be mplemented. A more effectve strategy s preferental mmunsaton, whch ncludes two steps. At frst, the probable nodes confrmed by contact tracng chan are pcked out. Xong (2004) has ponted out the probable node has larger probablty to nfect other nodes. Secondly, we can choose the most-connected nodes from these dentfed probable nodes to mmunse. Ths s because nodes wth better network connectons are typcally connected to more nodes; they are more dangerous n nfectng other nodes. Ths method s dfferent from the random mmunsaton. It dfferentates bots based on ther nfecton potentals. We wll gve prorty to bots wth larger probablty and apply mmunsaton to them frst. Smlar preferental mmunsaton approach has been proposed by Wang et al. (2000) n the context of mmunsaton of computer vrus. They have shown that preferental mmunsaton can sgnfcantly slow down vrus propagaton. We wll study the performance of preferental mmunsaton of botnets usng smulatons n Secton 7. 7 Experments We developed a tme-stepped smulator to study the performance of contact chan tracng algorthm. It smulates botnet propagaton and the establshment of contact tracng chans by mplementng algorthms descrbed n the prevous sectons. To clearly montor the propagaton of botnet, we only consder one botnet n a smulated network wth S = 6,400 nodes. We defne P as the probablty that node s nfected by other nodes. We assume each node s behavour s ndependent of others. Smlar to Zou et al. (2007), when the peer populaton S s large, we can approxmate P by a 2 N μ, σ. In Gaussan random varable wth dstrbuton ( P P ) our experments, P N ( ) 0.07, For most smulaton experments, we perform 50 smulaton runs and report the average values. We gnore network delays and assume mmedate detecton of hgh rate contact symptom of an nfected node. Therefore, an nfected node wll be traced mmedately. Intally, the only nfected node s suspcous and all other nodes are n the normal state. At each consequent tme step, nfected nodes nfect other nodes wth probablty P. Once a node s contacted by a suspcous node, t wll be added to contact tracng chan accordng to the algorthms ntroduced n Secton 5. When a complete tracng chan s establshed, all nodes on the chan wll be confrmed wth nfecton. Then the mmunsaton strateges n Secton 6 wll be appled to clean and mmunse confrmed bots. 7.1 Tracng chan effcency We frst study the effcency of tracng chan algorthms usng smulatons. We compare the bot detecton speed of three chan algorthms ntroduced n Secton 5. We then study how the tracng chan effcency depends on three mportant factors n practce: chan length threshold K, node coverage rato of the tracng chan scheme and botnet node degree Comparson of chan algorthms Fgure 4 shows the performance comparson of three algorthms: basc algorthm, longest-chan algorthm and causal chan algorthm. The number of nfected nodes n the system and the numbers of bots detected by each algorthm are plotted as tme evolves. There s not much dfference among the three algorthms n the ntal stage of the botnet propagaton. Once the propagaton accelerates, t s found that the number of detected bots wth the longest-chan algorthm and causal chan algorthm are much more than the basc algorthm. Eventually, all bots can be detected by all three algorthms. Thus, the longest-chan algorthm and causal chan algorthm have much hgher bot detecton speed than the basc algorthm. Snce the causal chan algorthm consders the causal relatons between contacts and nfectons, ts accuracy s better than the longest-chan algorthm. From the smulaton results, we can conclude that causal chan algorthm has the best overall effcency and accuracy among the three algorthms; unless, all smulatons n the remanng of ths secton use the causal chan algorthm.

8 Detectng and blockng P2P botnets through contact tracng chans 51 Fgure Comparson of three tracng chan algorthms (see onlne verson for colours) degree s low, t has a lower tracng speed; when the nfecton degree s hgh, ts tracng speed accelerates dramatcally. Number of Bots Infected Bots Basc Algorthm Largest Chan Algorthm Causal Chan Algorthm Tme:t Chan length threshold The chan length threshold trade-offs the detecton speed and accuracy. We have ntroduced a dynamc chan length threshold algorthm n Secton 5.4. In our smulaton, as lsted n Table 3, the threshold K s dynamcally chosen from [3, 9] as a functon of the current nfecton degree Δ I. Table 3 Dynamc chan length table K = 3 K = 4 K = 5 K = 6 Δ I [50, ) [35, 50) [24, 35) [14, 24) K = 7 K = 8 K = 9 Δ I [9, 14) [4, 9) [0, 4) Fgure 5 Number of Bots Comparson of dscovery smulaton wth dfferent K (see onlne verson for colours) Infected Bots Dscovered Bots(K=4) Dscovered Bots(K=10) Dscovered Bots(Dynamc K) Tme:t Fgure 5 compares the performance of dynamc chan length threshold wth fxed length threshold at dfferent K = 4 and K = 10. Obvously, we see that the dynamc threshold has the best performance among three cases. When the nfecton Node coverage of tracng chan scheme As dscussed above, we assumed that every peer has nstalled the detecton software. In realty, only a subset of users wll nstall the detecton software. We call users who nstalled the software regstered peers and users who dd not nstall the software unregstered peers. Obvously, only the regstered peers can be detected and traced by our system. We must take nto account ths factor when usng tracng chan. It s clear that the number of regstered peers affects the effcency of bot detecton. The rato of regstered peers s a rato of the regstered peers to the total number of peers n the system: number of regstered peers q = total number of peers We defne the detecton rato as: number of detected bots R = (4) number of regstered bots We conduct smulatons usng the causal chan algorthm wth K = 4. Smulatons are conducted for q = 100%, q = 80%, q = 50% and q = 20%, respectvely. Fgure 6 shows that the number of regstered bots can affect the effcency of bots detecton. For the case of q = 100%, all the peers are regstered and all the bots can be detected by the tracng chan algorthm. For the case of q = 20%, there are about 60 regstered bots that cannot be detected because n ths case, there are about 80% peers are not regstered. Ths leads to the stuaton that a few nodes are solated and cannot be traced. But the number of solated nodes s nsgnfcant relatve to the detected bots. The detecton rato s stll hgh. In concluson, the tracng chan algorthm s pretty robust aganst low node coverage. Fgure 6 Number of Bots Effcency of dscovery smulaton wth dfferent rato of regster (see onlne verson for colours) Number of Regstered Bots(q=100%) Number of Recovered Bots(q=100%) Number of Regstered Bots(q=80%) Number of Recovered Bots(q=80%) Number of Regstered Bots(q=50%) Number of Recovered Bots(q=50%) Number of Regstered Bots(q=20%) Number of Recovered Bots(q=20%) (3) Tme:t

9 52 Z. Huang et al Node degree of bots The establshments of tracng chans are drven by contacts among peers. The node degree of bots n a botnet drectly affects the effcency of tracng chan algorthm. We generate random P2P botnets wth the average node degree of 100, 40, 30 and 10, respectvely. For each average node degree, we generate a group of 25 random network nstances and run the smulaton on each nstance. For each smulaton, we gradually vary the rato of regstered peer q from 1 to At each q value, we obtan the bots detecton rato R n each nstance. We then calculate the average detecton rato R for the group of botnets wth the same average node degree. Fgure 7 shows the average detecton rato at dfferent average degrees. Botnet group 1 has an average degree of E = 10, botnet group 2 has an average degree of E = 30, botnet group 3 has an average degree of E = 40 and botnet group 4 has an average degree of E = 100. For group 4, when q decreases gradually, the detecton rato stays at a hgh level ( R > 90% ) as long as q > 0.1. Botnet group 1 has a low average degree. The detecton rato s hgh ( R > 90% ) f q > 0.7. But the detecton rato decreases quckly when q gets smaller and reaches 0 when q > 0.2. The performance for groups 2 and 3 le n-between groups 1 and 4. We can see that the average node degree of botnet can affect the effcency of detecton. The hgher node degree, the hgher the detecton effcency. From the attacker pont of vew, hgher bot degree ncreases the reslence of botnets to the removals of ndvdual bots. However, our tracng chan algorthm can also explot hgh bot degree for effcent and robust detecton. Fgure 7 Detecton Rato:R Detecton effcency at dfferent node degrees (see onlne verson for colours) q Botnet 1(E=10) Botnet 2(E=30) Botnet 3(E=40) Botnet 4(E=100) Rato of Unregstered Peers:(1 q) 7.2 Immunsaton smulaton To study the performance of mmunsaton strateges, we smulate a statc botnet wth all nodes jon the system at tme 0 and only one node s nfected ntally. No node leaves the system durng the smulaton. We smulate two dfferent mmunsaton strateges: n the frst case, at each tme step, we randomly choose nfected nodes to mmunse wth a homogeneous probablty p ; n the second case, we use preferental strategy to mmunse the top p fracton of probable peers wth the hghest connectvty. Fgure 8 Number of Bots Comparson of mmunsaton strateges n statc botnets (see onlne verson for colours) p=0.1:random Immunzaton p=0.2:random Immunzaton p=0.2:preferental Immunzaton Tme:t Fgure 8 shows that both strateges can completely break the botnet. Comparng two strateges, the preferental mmunsaton s much more effectve than the random mmunsaton. It can always control the sze of botnet at a very low level. We vary p to smulate the effect of mmunsaton wth dfferent mmunsaton rato. The result s that, wth p = 0.1, the botnet reaches a larger sze than the botnet wth p = 0.2. Therefore, we can conclude that the hgh mmunsaton rato and good mmunsaton strategy lead to effcent mmunsaton. 8 Implementaton ssues A tracng chan mechansm ncludes three key steps: 1 dentfy the ndvdual peer wth symptoms 2 establsh the transmsson chan through contact tracng 3 mmunse the nfected peers. In ths secton, we dscuss the scheme to mplement those steps. ISPs are n a unque poston to montor and trace contact behavours of users n ther networks, on whch can nstall servers to detect abnormal traffc patterns of users and establsh tracng chans among users. The drawback of ths approach s that ISP servers have to montor a large number of users and the workload can be very hgh. In addton, to trace connecton traffc across dfferent ISPs, servers under dfferent admnstratons have to cooperate each other (Xong, 2004). In our paper, we adopt another archtecture called dstrbuted sensor archtecture. In ths archtecture, each computer s nstalled wth a symptom detecton software, called sensor. Dstrbuted

10 Detectng and blockng P2P botnets through contact tracng chans 53 sensors contnuously montor traffc on ther host computers and record connecton level nformaton (such as destnaton IP addresses and duratons of outgong connectons) n every tme nterval. When a peer behaves normally, the sensor only keeps the montored nformaton locally. Once a sensor detects suspcous actvtes, t wll fre and report the montored abnormal nformaton to a central server. The central server acts as a tracker whch can be confgured on the ISP servers. It collects abnormal nformaton from all sensors n the network and apples contact tracng algorthms to establsh nfecton chans. Once a chan s establshed, all the peers on the chan wll receve nfecton confrmaton messages from the server. Infected peers wll be cleaned and mmunsed by the mmunsaton software. Fgure 9 shows a dstrbuted sensors-based detecton system, whch conssts of a central server and dstrbuted sensors A, B,, F. Each sensor must nstall detecton software and become a regstered user. All regstered users communcate wth the central server. As shown n Fgure 9, when the contacts number of peer A and D reaches M = 4 n a tme nterval, sensors on A and D connect wth the server and the tracng process starts. Ths approach reduces the workload of central server because the server only needs to process a very lmted amount of data from dstrbuted sensors. Fgure 9 9 Conclusons Dstrbuted sensors archtecture (see onlne verson for colours) In ths paper, we proposed a novel contact tracng chan-based framework to detect and block the propagaton of P2P botnets. Our framework conssts of three components: detecton, tracng and mmunsaton. We developed new symptom detecton algorthms to dstngush P2P Bots from legtmate P2P systems and studed three contact tracng chan algorthms to trade off the tracng speed and false alarm rate. The performance of the proposed algorthms was evaluated usng dscrete-tme smulatons. We demonstrated that the proposed contact tracng framework can quckly detect and block the propagaton of bots n dynamc P2P network envronment. Of course, contact-tracng chan system s not perfect. As the bots are confrmed by establshng contact-tracng chan, once the tracng chan s attacked by botmasters, such as a DOS attack occurs n the mddle of a chan, the number of detected bots wll decrease greatly although our tracng algorthm can mantan a hgh bots detecton rato as dscussed n Secton 7. In our future work, we wll refne the proposed detecton and tracng algorthms to mprove the detecton effcency, and develop new chan length threshold adjustment rules to adaptvely acheve the balance between detecton speed and accuracy. The P2P contact tracng archtecture looks promsng, but t needs to be desgned robust aganst unexpected peer falures and reslent to malcous attacks by compromsed detecton and tracng agents on ndvdual peers. Acknowledgements Ths work s supported n part by CSTC Contract 2009AB2147 and by CSTC Contract 2009DA0001-A. References Crandall, J.R. and Chong, F.T. (2004) Mnos: control data attack preventon orthogonal to memory model, Proceedngs of the 37th Annual IEEE/ACM Internatonal Symposum on Mcroarchtecture, IEEE/ACM, December. Dhungel, P., He, X., Ross, K.W. and Saxena, N. (2007) The polluton attack n P2P lve vdeo streamng: measurement results and defenses, Proceedngs of the 2007 Workshop on Peer-to-Peer Streamng and IP-TV, Japan, August. Eames, K.T.D. and Keelng, M.J. (2003) Contact tracng and dsease control, Proceedngs of the Royal Socety, London, December. Grzzard, J. and Sharma, V., Nunnery, C. and Dagon, D. (2007) Peer-to-Peer botnets: overvew and case study, Proceedngs of the Frst Conference on Frst Workshop on Hot Topcs n Understandng Botnets, USA. Gu, G., Perdsc, R., Zhang, J. and Lee, W. (2008) BotMner: clusterng analyss of network traffc for protocol- and structure-ndependent botnet detecton, Proceedngs of the 17th USENIX Securty Symposum (Securty 08). Hggns, K.J. (2007) The world s bggest botnets, November, avalable at Holz, T., Stener, M., Dahl, F., Bersacky, E. and Frelng, F. (2008) Measurements and mtgaton of peer-to-peer-based botnets: a case study on Storm Worm, Frst USENIX Workshop on Large-Scale Explots and Emergent Threats. Huerta, R. and Tsmrng, L.S. (2002) Contact tracng and epdemcs control n socal networks, Physcal Revew E, November, Vol. 66. Husna, H., Phthakktnukoon, S. and Dantu, R. (2008) Traffc shapng of spam botnets, ICCNC th IEEE. Hymana, J.M., L, J. and Stanley, E.A. (2003) Modelng the mpact of random screenng and contact tracng n reducng the spread of HIV, Mathematcal Boscences, January, Vol. 181, pp.1 16.

11 54 Z. Huang et al. Kang, J., Zhang, J., L, Q. and L, Z. (2009) Detectng new P2P botnet wth mult-chart CUSUM, Internatonal Conference on Networks Securty, Wreless Communcatons and Trusted Computng. Lemos, R. (2006) Bot software looks to mprove peerage, avalable at Lang, J., Kumar, R., X, Y. and Ross, K.W. (2005) Polluton n P2P fle sharng systems, INFOCOM th Annual Jont Conference of the IEEE Computer and Communcatons Socetes. Proceedngs IEEE. Newsome, J. and Song, D. (2005) Dynamc tant analyss: automatc detecton and generaton of software explot attacks, Proceedngs of the 12th Annual Network and Dstrbuted System Securty Symposum (NDSS 2005), February. Porras, P., Sad, H. and Yegneswaran, V. (2007) A mult-perspectve analyss of the Storm (Peacomm) Worm, SRI Internatonal, October. Shanon, C.E. (1948) A mathematcal theory of communcaton, Bell System Techncal Journal, July, pp Stewart, J. (2004) Phatbot Trojan analyss, avalable at Stewart, J. (2004) Spamthru Trojan analyss, avalable at Suh, G.E., Lee, J. and Devadas, S. (2004) Secure program executon va dynamc nformaton flow trackng, Proceedngs of ASPLOS XI, October, pp Wang, C., Knght, J.C. and Elder, M.C. (2000) On computer vral nfecton and the effect of mmunzaton, 16th Annual Computer Securty Applcatons Conference (ACSAC 00), December. Xong, J. (2004) ACT: attachment chan tracng scheme for e-mal vrus detecton and control, Proceedngs of the 2004 ACM Workshop on Rapd Malcode, pp Zhou, L., Zhang, L., McSherry, F., Immorlca, N., Costa, M. and Chen, S. (2005) A frst look at peer-to-peer worms: threats and defenses, Proc. 4th Internatonal Workshop on Peer-to-Peer System. Zou, C.C., Towsley, D. and Gong, W. (2007) Modelng and smulaton study of the propagaton and defense of nternet e-mal worm, IEEE Transactons on Dependable and Secure Computng, Aprl, pp Notes 1 A complete tracng chan wll be establshed mstakenly only f all lnks along the chan are detected mstakenly.