Low-rate and High-rate Distributed DoS Attack Detection Using Partial Rank Correlation

Transcription

1 Low-rate and High-rate Distributed DoS Attack Detection Using Partial Rank Correlation Monowar H. Bhuyan and Abhishek Kalwar Dept. of Computer Science & Engg. Kaziranga University, Jorhat , Assam D. K. Bhattacharyya Dept. of Computer Science & Engg. Tezpur University Tezpur , Assam, India J. K. Kalita Dept. of Computer Science University of Colorado Colorado Springs, CO 80918, USA Abstract Distributed Denial of Service (DDoS) attacks pose a serious threat to efficient and uninterrupted Internet services. During Distributed Denial of Service (DDoS), attackers make fool of innocent servers (i.e., slave) into reddening packets to the victim. Most low-rate DDoS attack detection mechanisms are associated with specific protocols used by the attacks. Due to slave, it has been found that the traffic flow for such an attack and their response flow to the victim may have linear relationships with another. Based on this observation, we propose the Partial Rank Correlation-based Detection (PRCD) scheme to detect both low-rate and high-rate DDoS attacks. Our experimental results confirm theoretical analysis and demonstrate the effectiveness of the proposed scheme in practice. Index Terms DDoS, rank correlation, attack, network traffic, low-rate, high-rate I. INTRODUCTION The Internet has an open architecture susceptible to various forms of network attacks. For example, Distributed Denial-ofservice (DDoS) attacks pose a serious threat to the security of cyberspace. Early attacks to well-known web sites, such as CNN, Amazon and Yahoo, in 2000 stopped normal services of these victims for hours [1], [2], [3], [4]. A recent report on DDoS attacks shows that the number of DDoS attacks has increased by 55% per year [5], and that the attacks have also increased in sophistication and severity. Attacks that flood packages are the most common and the most effective attack methods. Such an attack typically exhausts bandwidth, processing capacity, or memory of a victim machine or network. Such attacks also spread quickly on wired or wireless networks. It is classified as low-rate and high-rate attacks based on the attack rate dynamics. A low-rate DDoS attack is an intelligent attack as the attacker can send attack packets to the victim at a sufficiently low-rate to elude detection. Today, a large-scale DDoS attack is usually combined with multiple low-rate attacks, which are distributed on the Internet to avoid being detected by current detection methods. An attacker can use botnets to launch a low-rate DDoS attack, producing network behavior that appears normal. Therefore, it is difficult to detect and mitigate such attacks. Unlike typical DDoS attacks, a DDoS attack contains an army of attackers comprised of master zombies and slave Fig. 1. Architecture of DDoS attack zombies [6] (as shown in Figure 1). The network scenario in this type of attack is similar to that of typical DDoS attacks up to a degree. Attackers have control over master zombies, which in turn have control over slave zombies. The difference in this type of attack is that slave zombies are led by master zombies to send a stream of packets with the victim s IP address as the source to other uninfected machines (known as reflectors), exhorting these machines to connect with the victim. The reflectors follow by sending the victim a greater volume of traffic, as reply to its exhortations for the opening of new connections, because they believe that the victim was the host that asked for it. In this paper, we analyze network traffic near the victimend to detect both low-rate and high-rate DDoS attacks. In PRCD, we compute the partial rank correlation values for both legitimate traffic and anomalous traffic, and also find rank between them to raise an alarm with respect to a threshold value. The rest of the paper is organized as follows. Section II provides related work and comparison of existing methods. Our scheme for both low-rate and high-rate DDoS detection is presented in Section III. Section IV describes experimental results. Finally, we present the concluding remarks in Section V. II. RELATED WORK Only a few methods are available in the literature to detect low-rate DDoS attacks, although many more methods are available to detect general DDoS attacks. There are four

2 classes of detection strategies for defending against both lowrate and high-rate DDoS attacks [4] based on the deployment location in the network: source-end, victim-end, intermediate network and distributed. Detecting and stopping both low-rate and high-rate DDoS attacks at the source are the goals in the source-end defence mechanism. This mechanism detects malicious packets and prevents the possibility of flooding, but not on the victim side. It is best to filter or rate-limit malicious traffic with minimum damage within the legitimate traffic before it hits a potential victim. Moreover, a source-end defence mechanism acquires intelligence from a small amount of traffic and consumes few resources (i.e., processing power and buffer). The main difficulties of this mechanism are: (i) It cannot observe suspicious traffic at the victim-end because it has no interaction with the victim node, (ii) Sources are widely distributed and a single source behaves almost as in normal traffic, and (iii) Identification of deployment points are at the source-end. However, it cannot observe the effect at the victimend because it has no interaction with the victim. Hence, it could generate false alarms at a high-rate. In the victim-end defence mechanism, detection and response are generally performed at the routers of victim networks that provide critical Internet services. These mechanisms can closely observe the victim network traffic, model its behavior and detect anomalies. Detecting both low-rate and high-rate DDoS attacks in victim routers is relatively easy because of the high-rate of resource consumption. It is also the most practically applicable type of defence mechanism that can classify the attack traffic from legitimate traffic. But the main problems with this mechanism are: (i) During DDoS attacks, victim resources, e.g., network bandwidth, often get overwhelmed and cannot stop the flow beyond victim routers, and (ii) It can detect the attack only after it reaches the victim and detecting an attack when legitimate clients have already been denied is not useful. The intermediate network defence scheme balances the trade-offs between detection accuracy and attack bandwidth consumption, the main issues in source-end and victim-end detection mechanisms, respectively. It can be deployed in any network router connected to an ISP (Internet Service Provider). Such a scheme is generally collaborative in nature and the routers share their observations with other routers. Detection of attack sources is easy in this approach due to collaborative operation. Routers can form an overlay mesh to share their observations [7]. The main difficulty with this mechanism is the location of deployment. The unavailability of this mechanism in only a few routers may cause failure to the detection effort and the full practical implementation of this mechanism is extremely difficult because it will require reconfiguring all routers on the Internet. Most DDoS defence systems recently introduced are distributed in nature. Such systems are effective in keeping secure an organizational network due to cooperation among a large number of defence systems. Such a system is deployed in a distributed manner whenever required, as discussed above. It seems to provide a proper solution to discriminate both lowrate and high-rate DDoS threats from legitimate traffic. However, it requires support from multiple ISPs and administrative domains, which is usually difficult to obtain. Early detection of low-rate and high-rate DDoS attacks with high accuracy is essential to keep secure a system or a network. A comparison of DDoS defence mechanisms situated at different deployment locations is given in Table I. In the table, we observe that victim-end system is better because: It can closely observe the victim system or host to analyse the network traffic in near real-time, It is easy to deploy, and It is cheaper to detect both low-rate and high-rate DDoS attacks than other mechanisms. TABLE I FEASIBILITY OF DDOS DEFENSE AT DEPLOYMENT LOCATIONS Deployment Characteristics Rate limiting/ Defense vulnerability/ Deployment difficulty Filtering Robustness Source-end Very difficult Easy Low Highly difficult Victim-end Easy Difficult High Very easy Intermediate Difficult Difficult Medium Difficult network Distributed Difficult High High Difficult Paxson [8] analyzes different types of reflector attack defence mechanisms. These include significant threats to: DNS servers, Gnutella servers, and TCP-based servers. Al-Duwairi and Manimaran [9] introduce a packet pairing scheme to mitigate reflector based DDoS attacks. The scheme validates incoming reply packets in a distributed manner based on the request packet, which is performed at the edge routers of the ISP that connects to the victim. The scheme provides protection against two attacks: bandwidth exhaustion and resource exhaustion. They filter the attack traffic effectively with very little loss of legitimate traffic. Tsunoda et al. [10] present a robust scheme to detect DDoS attacks by confirming the validity of received packets with respect to the requestresponse relationship. This scheme can detect DDoS attacks accurately at a low cost. Yu et al. [11] discuss a suspicious flow discrimination algorithm using flow correlation coefficient. They demonstrate its effectiveness both theoretically and experimentally. Wei et al. [12] introduce a rank correlation based scheme to detect DDoS attacks. The simulation results demonstrate that the scheme can effectively differentiate reflection flows from legitimate flows. Zhang et al. [13] also present a nonparametric network traffic classification scheme based on flow correlation coefficient. They demonstrate that their scheme performs well even with a small set of training samples. Recently Ma and Chen [14] report using a variation of the Lyapunov exponent to detect network traffic anomalies. The authors make chaos-based analysis after pre-processing by an entropy based scheme and detect DDoS attacks. They combine source IP and destination IP address during their analysis. Based on our survey, we make the following observations. Most schemes analyze the relationship between the request and response packets to mitigate the DDoS attacks.

3 TABLE II NOTATIONS AND SYMBOLS Symbol/ Meaning Notation x network traffic data T time interval for processing t i i th time interval within T r partial rank correlation 1, 2 threshold for attack detection x i i th instance within x S sample traffic N total number of packets within full time interval T n smaller time interval t within T Fig. 2. PRCD: framework of the proposed scheme Existing schemes are mostly use victim-end deployment mechanisms. Most schemes work on both packet and flow level traffic. Due to lower cost, flow-level traffic is used for most analysis. III. PRCD: THE PROPOSED SCHEME Correlation coefficient is important in finding linear relationship between a pair of variables. Based on characteristics of normal and attack traffic, it would be useful information for detecting low-rate and high-rate DDoS attacks if there are such relationships in the attack traffic. Hence, we apply partial rank correlation to detect low-rate and high-rate DDoS attacks. Based on our knowledge, no one has applied partial rank correlation to detect low-rate DDoS attacks. The notations and symbols used to describe the proposed scheme are given in Table II. A framework of the proposed scheme is given in Figure 2. The major attractions of this scheme are the following. It can effectively identify malicious traffic from normal traffic using the rank value. Even though the cost is high, the detection accuracy is higher than competing methods. Before discussing the main scheme, we make the following assumptions. Routers have full control on in-and-out traffic flow, whenever found attacks it can immediately send a request to a router to control the network traffic. We sample the network traffic into 5 minute intervals and also further sub-sample into 10 sec time windows during processing. We assume that attack traffic obeys the Poisson distribution and normal traffic obeys the Gaussian distribution. Two instances of pure attack traffic have rank correlation value close to 1. The calculation may not be accurate due to background traffic. A. Partial Rank Correlation Pearson s correlation coefficient is a well-known metric suitable for explaining a linear relationship between pairs of instances. Based on background traffic and delay, linearity may not always be apparent. It has also been proved that Pearson s correlation is sensitive to outliers when bursty traffic occurs [12]. Based on experiments, we find that partial rank correlation is more suitable in detecting low-rate and high-rate DDoS attacks than Pearson correlation coefficient, because full linearity is not always possible in case of bursty traffic. Partial rank correlation coefficient computes the rank value based on the Pearson traffic statistics, where a value is converted to a rank value. We want to compute partial correlation coefficient between the random variables X, Y and Z. The expected values are µ X, µ Y and µ Z, the standard deviations are X, Y and Z. The coefficient r XY.Z is their covariance normalized by the standard deviation. r XY r XZ r YZ r XY.Z = p 1 r 2 XZ p 1 ryz 2 where r XY, r XZ and r YZ are the correlation coefficients. r XY is defined as r XY = cov(x, Y ) (2) X Y The range of r XY.Z is in between [-1, 1]. Values close to 1 represent stronger negative linear relationship while values close to 1 represent stronger positive relationship. 0 means there is no linear relationship between the pair. B. PRCD: Algorithm Let us assume that p X, p Y and p Z are three instances of pure malicious traffic. In such a case, partial rank correlation coefficient r XY.Z will be close to 1. Though on the Internet this assumption may not always hold due to background traffic, the partial rank correlation between the three malicious traffic instances should be strong in comparison to other traffic. To detect a DDoS attack, we use two thresholds 1 and 2 to justify whether both packets are malicious or not. If r XY.Z = 1, all three are DDoS attack traffic instances, which is decided based on the following. r XY.Z (x) = (1) ( 1, r XY.Z < 1 or r XY.Z 2 0, 1 apple r XY.Z apple 2 (3) The major steps of our scheme are given in Algorithm 1.

4 Algorithm 1 The DDoS attack detection algorithm Require: Network traffic x with respect to time window T and thresholds 1 and 2 Ensure: Alarm information (attack or normal) 1: Initialization: Sample period, T = 0, where i = 1, 2, 3, n, T = {t 1,t 2,t 3,,t N }, N is the full time interval. 2: Sample the network traffic X received from upstream router r based on sampling period T 3: Compute correlation coefficient r XY, r XZ and r YZ using Equation(2) for each sample instance within the sampling period T. 4: Compute partial rank correlation r XY.Z (x) using Equation(3) for each sample within sampling period T. 5: Compare rank value for suspicious traffic and make decision using Equation(3). 6: Check against threshold to see if r XY.Z (x) < 1 or r XY.Z (x) 2. If so then generate alarm; otherwise, router forward the packet to the downstream routers. 7: Go to step 2. C. Complexity Analysis The proposed scheme takes O(n 2 T ) time for detection of DDoS attacks, where n is the number of traffic instances within a sample. The time complexity is quadratic and the accuracy of the scheme is good in terms of low-rate and highrate DDoS attack detection. So, we say that the accuracy is effective though the cost is high. At a time only estimates rank correlation for two traffic instances. Fig. 3. Constant rate: comparison of partial rank correlation values IV. EXPERIMENTAL RESULTS We evaluate the proposed scheme using real-world DDoS datasets and compare with an entropy-based detection scheme [15], [12]. We use two different real-world datasets: MIT Lincoln Laboratory [16] and CAIDA DDoS 2007 [17] datasets. The MIT Lincoln Laboratory dataset is real-time and pure normal data. This dataset was acquired as tcpdump trace over a period of several weeks. It does not contain any attack traffic. The CAIDA DDoS 2007 dataset contains one hour of anonymized traffic traces from a DDoS attack on August 4, This dataset includes mainly two types of attacks: consumption of computing resources and consumption of network bandwidth. While the data was collected, the servers were connected to the Internet. According to Moore et al. [18], it is low-rate attack traffic if 1, 000 packets per second over the network covering 60% of attack traffic. If 10, 000 packets per seconds over the network covering more than 90% of attack traffic then it is known as high-rate attack traffic. We attempt to detect both low-rate and high-rate DDoS attacks within short time interval. A. Results We sample the 5 minute CAIDA traffic in 10 second intervals. We also sample the MIT Lincoln laboratory normal traffic into 10 second interval for our experiment. We apply partial rank correlation to both legitimate and malicious traffic Fig. 4. Variable rate: comparison of partial rank correlation values instances. We consider the following cases to validate our experimental results: (i) legitimate vs. legitimate, (ii) malicious vs. malicious, and (iii) legitimate vs. malicious. Figure 3 and Figure 4 show our results in terms of three different cases considered for experimentation. We observe the following. Though there is a huge amount of background traffic, correlation between two malicious traffic instances is still strong, whereas it is very weak between malicious and a legitimate traffic instance. It can t accurately differentiate between two traffic instances using entropy. Partial rank correlation coefficient provides stable rank value after about 100 time units, i.e., when the attack starts 0.1 second time point, only 10 seconds are needed to give the final alarm. To confirm the range of the threshold value, we estimate probability density against the partial correlation rank values when detecting both low-rate and high-rate DDoS attacks. We

5 relative entropy based scheme [12]. It is effective in terms of accuracy even it computes partial rank correlation over traffic instances in quadratic time. Currently we are working on detecting DDoS attacks using the extreme value theory. REFERENCES Fig. 5. Comparison of partial rank correlation (PRC) and probability density Fig. 6. found the following. Comparison of PRCD with Relative entropy metric The values of correlation between malicious vs malicious and legitimate vs. malicious can be used to distinguish the attack correctly. To achieve low false alarm rate, we choose the point of intersection as thresholds. From Figure 6, we derive the threshold as 1 = 0.30 and 2 = 0.25 for our experiment. V. CONCLUSION AND FUTURE WORKS We present a scheme that uses partial rank correlation to detect both low-rate and high-rate DDoS attacks. Once malicious traffic is found based on the rank value obtained from PRCD estimation, our scheme requests the edge router to stop forwarding the traffic to the downwards routers. The generation of an alarm is decided based on the two thresholds, 1 and 2, which are estimated heuristically. Experimental results using the proposed scheme outperforms a competing [1] L. Garber, Denial-of-Service Attacks Rip the Internet, Computer, vol. 33, no. 4, pp , April [2] M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, Survey on Incremental Approaches for Network Anomaly Detection, International Journal of Communication Networks and Information Security, vol. 3, no. 3, pp , [3], AOCD: An Adaptive Outlier Based Coordinated Scan Detection Approach, International Journal of Network Security, vol. 14, no. 6, pp , [4], Network Anomaly Detection: Methods, Systems and Tools, IEEE Communications Surveys Tutorials, vol. 16, no. 1, pp , [5] Neustar, The Danger Deepens: Neustars Annual DDoS Attacks and Impact Report, Neustar, Tech. Rep., [Online]. Available: [6] S. Gibson, DRDoS: Distributed Reflection Denial of Service, Description and analysis of a potent, increasingly prevalent, and worrisome Internet attack, CISCO, Tech. Rep., February [Online]. Available: [7] J. Mirkovic and P. Reiher, D-ward: A source-end defense against flooding denial-of-service attacks, IEEE Trans. on Dependable and Secure Computing, vol. 2, pp , [8] V. Paxson, An Analysis of Using Reflectors for Distributed Denial-ofservice Attacks, SIGCOMM Comput. Commun. Rev., vol. 31, no. 3, pp , July [9] B. Al-Duwairi and G. Manimaran, Distributed Packet Pairing for Reflector Based DDoS Attack Mitigation, Comput. Commun., vol. 29, no. 12, pp , August [10] H. Tsunoda, K. Ohta, A. Yamamoto, N. Ansari, Y. Waizumi, and Y. Nemoto, Detecting DRDoS Attacks by a Simple Response Packet Confirmation Mechanism, Comput. Commun., vol. 31, no. 14, pp , September [11] S. Yu, W. Zhou, W. Jia, S. Guo, Y. Xiang, and F. Tang, Discriminating ddos attacks from flash crowds using flow correlation coefficient, IEEE Trans. Parallel Distrib. Syst., vol. 23, no. 6, pp , June [12] W. Wei, F. Chen, Y. Xia, and G. Jin, A Rank Correlation Based Detection against Distributed Reflection DoS Attacks, IEEE Communications Letters, vol. 17, no. 1, pp , [13] J. Zhang, Y. Xiang, Y. Wang, W. Zhou, Y. Xiang, and Y. Guan, Network Traffic Classification Using Correlation Information, IEEE Trans. Parallel Distrib. Syst., vol. 24, no. 1, pp , January [14] X. Ma and Y. Chen, DDoS Detection Method Based on Chaos Analysis of Network Traffic Entropy, IEEE Communications Letters, vol. 18, no. 1, pp , [15] A. Rényi, On Measures of Entropy And Information, in Proc. of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, 1960, pp [16] MIT Lincoln Laboratory Datasets, MIT LLS DDOS 0.2.2, data/2000data.html, 2000, Massachusetts Institute of Technology, Cambridge, MA. [17] CAIDA, The Cooperative Analysis for Internet Data Analysis, [18] D. Moore, C. Shannon, D. J. Brown, G. M. Voelker, and S. Savage, Inferring Internet Denial-of-service Activity, ACM Trans. Computer System, vol. 24, no. 2, pp , May 2006.