Obstruction of Undesired Traffic by Filtering Source Prefixes on the Internet

Transcription

1 Obstruction of Undesired Traffic by Filtering Source Prefixes on the Internet Dhanya P J & M S Gayathri Department of Computer Science and Engineering, K C G College of Technology, Karapakkam, Chennai dhanya2112@gmail.com, gay3anand@gmail.com Abstract In recent days, the Internet has seen a continuous rise in malicious traffic including DDoS attacks,worm attacks and spam. One way to deal with this problem is to filter unwanted traffic at the routers based on source IP addresses. Filtering techniques are available at routers today via access control lists(acls).acls are typically stored in content addressable memory(tcam),which is more expensive and consumes more power. The size and cost of TCAM puts a limit on the number of filters. Filtering source prefixes instead of individual IP address helps to reduce the number of filters, but it cause collateral damage by blocking legitimate traffic originating from the filtered prefixes. Therefore, filter selection can be viewed as an optimization problem that tries to block many attack sources with minimum collateral damage as possible. One defense mechanism against malicious traffic is source address based filtering. This paper proposed a family of algorithms that optimize the tradeoff between the unwanted and legitimate traffic that is blocked. The main objective of this paper is to construct a source address blacklist per contributor that reflects the most probable set of attackers that will attack the target contributor and to select which source prefixes to filter so as to minimize the impact of bad traffic and can be accommodated with the given source budget. Keywords Filtering, Internet, Network security, Clustering algorithm I. INTRODUCTION In recent days, the Internet has seen a continuous rise in malicious traffic including DDoS attacks, worm attacks and spam. Filtering malicious traffic based on source IP address is one of the approaches to protect network infrastructure from attacks on the Internet. Protecting a host or network from malicious traffic is a hard problem that requires the coordination of several complementary components, including nontechnical (e.g., business and legal) and technical solutions ( at the application and/ or network level). Filtering support from the network is a fundamental building block in this effort. For example, an Internet service provider (ISP) may use filtering in response to an ongoing DDoS attack to block the DDoS traffic before it reaches its clients. Another ISP may want to proactively identify and block traffic carrying malicious code before it reaches and compromises vulnerable hosts in the first place. In either case, filtering is a necessary operation that must be performed within the network. Filtering capabilities are already available at routers today via access control lists (ACLs). ACLs enable a router to match a packet header against predefined rules and take predefined actions on the matching packets, and they are currently used for enforcing a variety of policies, including infrastructure protection. For the purpose of blocking malicious traffic, a filter is a simple ACL rule that denies access to a source IP address or prefix[3]. To keep up with the high forwarding rates of modern routers, filtering is implemented in hardware: ACL s are typically stored in ternary content addressable memory (TCAM), which allows for parallel access and reduces the number of lookups per forwarded packet. However, TCAM is more expensive and consumes more space and power than conventional memory. The size and cost of TCAM puts a limit on the number of filters, and this is not expected to change in the near future. In this paper filtering is based on source IP address of malicious traffic. The sources of malicious IP address form a blacklist. This paper proposed a new blacklist forecasting model called Highly Predictive Blacklisting (HPB)[2]. It is an approach to blacklist formulation. Under the HPB strategy, for every contributor, enumerate all sources of reported attackers and assign each of them a ranking score relative to its probability to attack the contributor in the future[6]. The ranking score is based on observation of the particular attacker s past activities, as well as the collective attack patterns exhibited by all other attackers in the alert repository. The key idea of HPB is to use peer-based correlations to 48

2 estimate attack probabilities of that an attack source poses to each contributor. In this paper proposed three algorithms used to block the malicious traffic on the internet. These algorithms take the input as a blacklist of IP addresses and select ranges to block. Algorithm Obstruct Entire Traffic blocks all blacklisted sources so as to minimize the collateral damage[2]. Algorithm Obstruct Some Traffic blocks some of the sources. These algorithms are optimal and also computationally efficient. II. PROBLEM DEFINITION There is a large and increasing amount of unwanted traffic on the Internet today, including phishing, spam, and distributed denial-of-service attacks. One mechanism that is used today to prevent unwanted traffic from reaching the victims, is to use access control lists (ACLs) at the routers to block packets that are considered unwanted. ACLs are rules that can classify packets according to a combination of fields in the IP header. In this paper, filtering is based on the source IP addresses of unwanted traffic. These sources are known and given as a blacklist. Several such blacklists are constructed from a proposed blacklist forecasting model. This paper looked at the problem: how to filter attack sources based on the source IP address so as to minimize the amount of legitimate traffic dropped, subject to constraints in the number of filters and the victim s access bandwidth. Filter all or most bad addresses; cause low or no collateral damage to legitimate traffic; stay within the budget in the number of filters and also the filter selection can be viewed as a optimization problem. III. EXISTING SYSTEM The existing system has been implemented with the source based filtering mechanism where we can filter all the malicious IP packets in the backbone IP network. The existing system is exposing with the model driven architecture which used all the network models as a subsystem. The existing system has been used ACL rules( Access Control list) rules but it an expensive method because it stores in the Content Addressable memory of the Network processor with minimum capabilities[3]. Filtering capabilities are already available at routers today via access control lists (ACLs). To keep up with the high forwarding rates of modern routers, filtering is implemented in hardware: ACLs are typically stored in ternary content addressable memory (TCAM), which allows for parallel access and reduces the number of lookups per forwarded packet. However, TCAM is more expensive and consumes more space and power than conventional memory. The size and cost of TCAM puts a limit on the number of filters. Existing system specifies three problems that correspond to different attack scenarios and operator policies[1]: blocking all addresses in a blacklist (BLOCK-ALL),blocking some addresses in a blacklist (BLOCK-SOME), blocking all/some addresses in a time-varying blacklist(time-varying BLOCK- ALL/SOME). For each problem, designed an optimal, yet computationally efficient, algorithm to solve it. It used the data from Dshield.org to evaluate the performance of algorithms in realistic attack scenarios. IV. PROPOSED SYSTEM In the proposed system filtering is based on source IP address of malicious traffic. The sources of malicious IP address form a blacklist. Filtering based on a blacklist involves constructing a set of ACL rules to block unwanted traffic so as to meet certain criteria[2].there are four practical source address filtering problems, depending on the attack scenario and the operator s policies: blocking all address in a Blacklist ( Obstruct Entire Traffic),blocking some address in a Blacklist (Obstruct Some Traffic),blocking all address in a time varying blacklist(dynamic Obstruction of All Traffic), blocking flows during a DDoS flooding attack to meet bandwidth constraints(flooding). This paper also proposed a blacklist forecasting model called Highly Predictive Blacklisting (HPB). It is an approach to blacklist formulation. Under the HPB strategy, for every contributor, enumerate all sources of reported attackers and assign each of them a ranking score relative to its probability to attack the contributor in the future. The ranking score is based on observation of the particular attacker s past activities, as well as the collective attack patterns exhibited by all other attackers in the alert repository[6]. V. SYSTEM IMPLEMENTATION When come to the module separation and implementation, this has been identified four distinct modules that together fulfill the entire functionality. And the main functionality that extends to the future expansion of the project. Mainly four important modules have been identified. They are, 1. Obstruct Entire Traffic 2. Obstruct Some Traffic 3. Dynamic Obstruction of Entire Traffic 4. Flooding 5. Highly Predictive Blacklisting Each module have well defined set of functionalities. Description of each module is explained 49

3 below. Fig.1 shows the architecture diagram of the system. It consist of mainly three blocks. Obstruct Entire/Some Traffic block filter all IP address in the blacklist and inputs to the block are blacklist, whitelist and maximum number of filters. In the second block, Dynamic Obstruction of Entire Traffic filter all the bad IP address in a time varying blacklist and input to the block is time varying blacklist and maximum number of filters. Third block is Flooding, it block flows during a DDoS flooding attack to meet bandwidth constraints.lcp Tree is build from the input blacklist and in a bottom up fashion compute the minimum collateral damage needed to block all bad addresses. A. Obstruct Entire Traffic Fig.1 : System Architecture Given a blacklist and a number of filters F, filter out all bad addresses, so as to minimize the collateral damage. Filter out all bad addresses in a blacklist is the natural first step. The blacklist is constructed by a Highly Predictive Blacklisting algorithm, that has identified and confirmed a consistent malicious behavior of the addresses that must be filtered out[2][1]. This problem is interesting only if F < N, otherwise we could filter out each individual address with a single filter. Proposed system developed a dynamic programming optimal algorithm that solves this problem. The goal of the problem is selection source address prefixes so as to block all malicious sources and minimize the collateral damage. Input to the problem is, a blacklist of malicious addresses BL a set of legitimate sources weights assigned to each legitimate source address I indicating the amount of traffic from that address a limit on the number of filters Fmax Algorithm1 build LCP-tree() for all leaf nodes leaf do end for z leaf (F) = 0 F [1, F max ] X leaf (F) = {leaf} F [1, F max ] level = level(leaf)-1 while level level(root) do for all node p such that level(p)==level do end for level = level - 1 end while z p (1) = gp X p (1) = {p} return z root (Fmax), X root (Fmax) z p (F) = min n=1,..f 1 {z sl (F n) + z sr (n)} F [2, F max ] X p (F) = X sl (F n) X sr (n) F [2, F max ] Algorithm 1, which solves Obstruct Entire Static and it consists of two main steps. First, build the LCPtree from the input blacklist. Second, in a bottom-up fashion, compute zp(f) p, F, i.e. the minimum collateral damage needed to block all malicious IPs in the subtree of prefix p using at most F filters. Following a dynamic programming (DP) approach, can find the optimal allocation of filters in the subtree rooted at prefix p, by finding a value n and by assigning F n filters to the left subtree and n to the right subtree, so as to minimize the collateral damage. It need to filter all malicious addresses (leaves in the LCP tree) implies that at least one filter must be assigned to the left and right subtree, i.e. n = 1, 2..., F 1. For every pair of sibling nodes, sl (left) and sr (right), with common parent node p, we have the DP recursive equation: z p (F) = min n=1,..f 1 {z sl (F n) + z sr (n)} F [2, F max ] with boundary conditions for leaf and intermediate nodes: zleaf (F) = 0 F 1, zp(1) = gp p 50

4 Once compute zp(f) for all prefixes in the LCP-tree, simply read the value of the optimal solution, zroot(fmax) and also use the variables Xp(F) to keep track of the set of prefixes used in the optimal solution. B. Obstruct Some Traffic Given a blacklist and a number of filters F, filter out some bad addresses, so as to optimize the achievable tradeoff between collateral damage (false positives) and unfiltered bad addresses (false negatives). The requirement of algorithm1 to filter out all the source IPs is too strict and may lead to large collateral damage if bad addresses are too spread apart in the address space. Algorithm2 differs from algorithm1 in that it tolerates leaving some bad sources unfiltered in exchange for a reduction in collateral damage. Instead, it tries to find and block only those subsets of bad addresses that have the highest negative impact on the network performance. Proposed system developed a dynamic programming optimal algorithm thats solves this problem. In the formulation, we provide a knob (namely, the weight wi assigned to an address i) that allows the administrator to express how much she values each address and thus control the tradeoff achieved by the optimal algorithm. The goal of the problem is block some source address prefixes so as to minimize the total cost, including the collateral damage and the benefit of blocking malicious addresses. Input to the problem is a blacklist of malicious addresses a set of legitimate sources weights assigned to all addresses a limit on the number of filters Fmax The algorithm is similar to Algorithm 1 in that it uses the LCP-tree and a similar DP approach. The difference is that not all addresses need to be covered and, at each step, we can assign n = 0 filters to the left or right subtree, i.e. in the 11 th step of the Algorithm 1: n = 0, 1..., F. It can recursively compute the optimal solution as before: z p (F) = min n=0,..f 1 {z sl (F n) + z sr (n)} with boundary conditions for intermediate (p) and leaf nodes: zp(0) = 0 p, zleaf (F) = bleaf F 1 zp(1) = min{gp bp, minn=0,1{zsl(1 n) + zsr (n)}} C. Dynamic Obstruction of Entire Traffic Bad addresses may change over time, new sources may send malicious traffic and, conversely, previously active sources may disappear. Proposed system developed a greedy algorithm to solve this problem. The goal is to filter out all /some bad addresses at all times, at minimum collateral damage. Given a set of blacklists BL = {BLt0,BLt1,.. }, and number of filters, F, find a set of filter rules {St0, St1, }, such that Sti solves problem obstruct entire/some static when the input list is BLt. Bad addresses may change over time New sources may send malicious trafficand, conversely, previously active sources may disappear. One way to solve the dynamic versions of Obstruct-Entire (Some) Static is to run the algorithms we propose for the static versions for the blacklist/whitelist pair at each time slot. Goal of the problem is to filter out all bad addresses at minimum collateral damage in every time slot. Input to the problem is a blacklist of malicious addresses BL weights assigned to each legitimate source address indicating the amount of traffic from that address a limit on the number of filters Fmax Apply the above algorithm1 for each individual arrival and departure. As long as the number of entering and departing addresses remains smaller than N, the computational cost of updating the list remains smaller than the cost of re-running algorithm1 from scratch for every instance. In the first time slot, run the algorithm1 and create a sorted list of collateral damage for filters of consecutive bad addresses. In subsequent time slots, update the sorted list and filtering choice by exploiting the greedy property. z root (Fmax )indicates the value of the optimal solution before the updating. D. Flooding In a flooding attack, a large number of compromised hosts send traffic to the victim and exhaust the victim s access bandwidth. In this case, our framework can be used to select the filtering rules that minimize the amount of good traffic that is blocked while meeting the access bandwidth constraint,in particular, the total bandwidth consumed by the unblocked traffic should not exceed the bandwidth of the flooded link. Input to the problem is a blacklist and a whitelist, where the absolute weight of each bad and good address is equal to the amount of traffic it generates; the number of available filters ; a constraint on the victim s link capacity (bandwidth) ; select filters so as to minimize collateral damage and make the total traffic fit within the victims link capacity. 51

5 Algorithm is similar to the one that solves Obstruct Entire Traffic, i.e., it relies on an LCP tree and a DP approach. However, now use the LCP tree of all the bad and good addresses. Moreover, when we compute the optimal filter allocation for each subtree, we now need to consider not only the number of filters allocated to that subtree, but also the corresponding amount of capacity (i.e., the amount of the victim s capacity consumed by the unfiltered traffic coming from the corresponding prefix) and can recursively compute the optimal solution bottom up as before z p (F,c) = min n=0,..f {z sl (F n,c-m) + z sr (n,m)} where z p (F,c) is the minimum collateral damage of prefix Ƥ when allocating F filters and capacity c to that prefix. E. Blacklist Forecasting Model Highly Predictive Blacklisting is a different approach to source address blacklist formulation. It present a probabilistic attacker ranking algorithm for blacklist formulation for use in centralized collaborative log sharing infrastructures, such as the DShield.org security log repository.. The ranking score is based on observation of the particular attacker s past activities, as well as the collective attack patterns exhibited by all other attackers in the alert repository.the heart of blacklisting system is the scoring algorithm. In formulating a blacklist for a contributor, it assigns scores to each attacker that are proportional to the estimation that the attacker will attack the contributor. Here, attackers represent class C addresses. Let ν and s denote the source and victim of an attack. Algorithm generates a customized blacklist per contributor (victim). R s (ν) used to denote the score for attacker s with respect to a victim (blacklist consumer)νv. R s (ν))is a sum of two ranking scores: RP s (ν) and RI s (ν). RP s (ν) is an estimation of s s attack probability given 1) s s past activities involving other victims ν, and 2) information on similarities among victims compiled over a collection of attack data. RI s (ν))is the estimation of s s attack probability based on s s previous activity involving only ν itself. First describe how to compute RP s (ν) and then continue to the calculation of RI s (ν). Correlation Graph Highly Predictive Blacklisting is to use peer-based correlations to estimate attack probabilities of that an attack source poses to each contributor. Model the correlation relationship between networks as a graph called correlation graph. The probability distribution is simulated by a random walk on the correlation graph: A source walks on the correlation graph, going from one node to another by following the correlations among the victim networks. Suppose we have a collection of past attacks made by a set of sources. An example is given in Table1. TABLE 1 ATTACK TABLE v 1 v 2 v 3 v 4 v 5 s 1 X X s 2 X s 3 X X X s 4 X X s 5 X s 6 X X s 7 The rows represent attack sources and the column represent the targeted networks (attack victims). An ``X'' in the table cell indicates that the corresponding source has reportedly attacked the corresponding network. Consider s 2 and s 7. Although they have attacked the same number of victims, from the viewpoint of v 1, one may say that s 2 is more likely to attack than s 7, because s 2 has attacked v 2, which shares more common attackers with v 1. Now compare the source s 5 to s 7. Both sources attacked only one network. None of these networks share common attacks with v 1. However, for v 1, s 5 and s 7 are not equal. Notice that v 2 shares common attacks with v 1, and v 3 shares common attacks with v 2. A path v 3 v 2 v 1 connects s 5 to v 1. One may say that s 5 is more likely to attack for v 1. Fig.3 shows the attack correlation as a graph. The correlation graph is a weighted directed graph G = (V, E). The nodes in the graph are the victims, i.e. V = {v 1, v 2,...} There is an edge from node v i to node v j if v i is correlated with v j. The weight on the edge is proportional to the strength of this correlation. Figure shows the correlation graph for the victims in Table1. X 52

6 Fig.3 : Correlation Graph corresponds to the attack table Rank an attack source, with respect to a victim, using the source's probability to attack the victim and estimate this probabilities in the following way: suppose we have an estimation on source s's probability of attacking victim v i. Following the outgoing edges of v i, A fraction of this probability can be distributed to the neighbors of v i in the graph. Each neighbor receives a share of this probability that is proportional to its strength of correlation with v i (i.e., proportional to the weight of the edge from v i to that neighbor.) Suppose v j is one of these neighbors in the correlation graph. A fraction of the probability received by v j is then further distributed, in the similar fashion, to its neighbors. The propagation of probability continues until the estimations for each victim reach a stable state. Such a probability-propagation process can be simulated by a random walk on the correlation graph. Let P s (v) be the estimate of the total probability that s attacks v. Let W ij be the correlation strength from victim v j to v i and B s (v i ) be an initial estimation based on whether s attacks v i in the attack table. The stable distribution of the random walk is the following: P s (ν i ) = B s (ν i ) + α W ij P s (ν j ) where B s (ν i ) is an initial estimation based on whether sattacks viin the attack table. Given a fixed source s, The sets of P s (ν i ) P(vi)and Bs(vi B s (ν i ))form vectors, which we denote by P s and B s respectively. Algorithm 2 Generate HPB for victim ν HPB_GEN(BL_Length, ν) foreach source s RP s s RANK _SCORE1(v, s) RI s Is RANK _SCORE2(v, s) R s RP s + RI s Sort s sin descending order according to their rank score R s Rs Return BL_ Length of s swith top R s Rs RANK _SCORE1(v,s) Obtain attack overlap from the attack table; Generate the standardized correlation matrix; Generate the final correlation matrix WW; Construct B s ; Solve linear system in Eq P s = B s + α WP s ; Return RP s (ν) RANK _SCORE2(v,s) Estimate W ij^i; Calculate B s ; Solve Eq. RI s (ν i ) = B s (ν i ) + α W ij RI s (ν i ) return RI s (ν i ) VI. PERFORMANCE ANALYSIS In this paper present a detailed analysis of how to filter the malicious traffic on the Internet. This section evaluate the performance of each algorithm.the algorithm strongly depends on the number of available filters and also on the inherent characteristics of the input blacklist, namely the degree of clustering. Considering at the attack sources in the blacklist, we verified that malicious sources are clustered in a few prefixes, rather than uniformly distributed over the IP space. In our simulations, we considered a blacklist to be the set of sources attacking a particular victim during a single day-period. The degree of clustering varied significantly in the blacklists of different victims and across different days. Fig. 2 : Sample graph for block all mechanism In Fig. 4, we consider two example blacklists corresponding to two different victims, each attacked by a large number of malicious IPs in a single day and take the blacklists with the highest and the lowest degree of source clustering observed in the entire data set, referred to as High Clustering and Low Clustering respectively. We run the algorithm and made the following observations. The optimal algorithm performs significantly better than a generic clustering algorithm that does not exploit the structure of IP prefixes. The degree of clustering in a blacklist depends the collateral damage and the number of filters used. 53

7 VII. CONCLUSION In this paper, designed an optimal algorithms that construct filtering rules to block IP address prefixes given a blacklist. There are several directions of future work including the process of applying algorithms to publicly available blacklists and Dshield data and using the filtering algorithms as a building block of a bigger system that effectively protects a network from unwanted traffic. VIII. REFERENCES [1] F. Soldo, A. Markopoulou, and K. Argyraki, Optimal filtering of source address prefixes: Models and algorithms, in Proc. IEEEINFOCOM, Rio de Janeiro, Brazil, Apr. 2009, pp [2] Dshield dataset, [3] J. Zhang, P. Porras, and J. Ullrich, Highly predictive blacklisting, in Proc. of USENIX Security 08 (Best Paper award), San Jose, C A, USA,Jul. 2008, pp [4] F. Soldo, A. Markopoulou, and K. Argyraki, Predictive Blacklisting as an Implicit Recommendation System, in Proc. IEEE INFOCOM, Rio de Janeiro, Brazil, Apr. 2009, pp [5] X. Liu, X. Yang, and Y. Lu, To filter or to authorize: Network-layer DoS defense against multimillion-node botnets, in Proc. ACM SIGCOMM, Seattle, WA, Aug. 2008, pp [6] High performance packet classification, HiPAC.org [Online].Available: ance_tests/results.html [7] E. Al-Shaer and H. Hamed, Firewall policy advisor, DePaul University,Chicago, IL, 2005 [Online]. Available: [8] S. Venkataraman, A. Blum, D. Song, S. Sen, and O. Spatscheck, Tracking dynamic sources of malicious activity at internet-scale, presented at the NIPS Whistler, BC, Canada, Dec [9] P. B. Z. Chen, C. Ji, Spatial-temporal characteristics of internet malicious sources, in IEEE INFOCOM Mini-Conference, Apr [10] P.Barford, R.Nowak, R. Willett, and V. Yegneswaran, Toward a model for sources of internet background radiation, in PAM, Mar