Denial of Service (DoS) Attack Detection by Using Fuzzy Logic over Network Flows

Transcription

1 Denial of Service (DoS) Attack Detection by Using Fuzzy Logic over Network Flows S. Farzaneh Tabatabaei 1, Mazleena Salleh 2, MohammadReza Abbasy 3 and MohammadReza NajafTorkaman 4 Faculty of Computer Science and Information System, University of Technology Malaysia(UTM), Kuala Lumpur, Malaysia 1 (farzanehtabatabaei@gmail.com), 2 (mazleena@utm.my), 3 (ramohammad2@live.utm.my), 4 (rntmohammad2@live.utm.my) Abstract - Intrusion Detection System (IDS) is the tool that is able to detect occurrences of intrusion at host, network, as well as application. One of the most common network attacks is Denial of Service (DoS) attack. In DoS attack, a single host will send huge number of packets to one machine and thus make the operating of the network and host slow. There are several algorithms that have been proposed to detect DOS attacks and most of these solutions are based on detection mechanisms that have the potential of producing high number of false alarms. In addition, most of the solutions are monitoring and analyzing packets inside the network instead of network flow. In this paper, signature of selected attacks such as Smurf, Mail-Bomb and Ping-of-Death which are based on network flow is considered. The proposed engine monitors the network flows to detect attacks and the results show less false negative error during monitoring. In addition signature based IDS which use fuzzy decision tree for monitoring network flow proves that there are improvements on speed of detection and also performance of system. Keywords: DOS Detection, Fuzzy Logic, IDS 1 Introduction Reports of the internet usage showed that the number of internet users is increasing and unfortunately this phenomenon has attracted attacks on the network. Consequently this has raised the concern of the network security especially by the services providers and they are always looking for solutions to monitor and check packets being received from clients to avoid any kind of attacks. Security mechanism used in a network is to prevent the system from any kind of attack and to stray away from any unsecured state. As prevention mechanism could not capable to impede the attacks entirely, so new level of security will be needed and the goal is to detect and stop the attack as soon as possible [1]. An intrusion-detection system (IDS) dynamically monitors the actions taken in a given environment such as host or traffic of network and decides whether these actions are symptomatic of an attack or constitute a legitimate use of the environment [2]. The two most common detection techniques which could be applied in IDS are signature based detection and anomaly based detection [3]. Signature based detection technique in IDS is looking for characteristics of known attacks and IDS try to find the similarity between previous behavior of the system or network with characteristics of known attack in signature database but in this technique IDS cannot detect novel attacks [4] [5] [6] [7]. Anomaly detection technique adopts the normal condition of the network traffic or behavior of host as criteria of anomaly; by this approach it can detect unknown attacks. But this approach create a percentage of detection errors because of the difficulty to define the normal state of the network traffic precisely [4] [5] [6] [7]. Denial of Service (DoS) attack uses up the resources of host, network or both in the way that normal user as a client could not access to the Server [8]. Some researches use artificial intelligence [9] and data mining [10] and fuzzy [11] in IDS to detect intrusion. Recently fuzzy based intrusion detection systems have proved robustness to noise, self-learning capability, and the ability to build initial rules without the need for a priori knowledge [12]. Although a variety of approaches proposed to detect intrusion like DoS but still the accuracy and efficiency of detection needs more improvement. So information security experts are still trying to improve the mechanism for detection of DoS attacks by several algorithms. 2 Problem Statement According to Table1, the number of incidents increased rapidly and thus pushes researchers to give more and better effective way to stop those incidents. One of the solutions is to build IDS which can detect more intrusion with small false positive rate. According to [12] the rate of detection and false positive do not satisfy users especially in anomaly based IDS. This paper is about introducing an engine for IDS which detect some types of DoS attacks with better rate of detection

2 and less false positive errors. This mechanism will be signature based and the engine use fuzzy algorithm and it monitor the network flows for better performance. Number of security incident from CERT [13] website is shown in Table1. Given the widespread use of automated attack tools, attacks against Internet-connected systems have become so commonplace that counts of the number of incidents reported provide little information with regard to assessing the scope and impact of attacks [13]. Therefore, CERT stopped providing this statistic at the end of Table1: Number of security incident reports received by CERT Year Number of Incident , , , , , , , , ,412 3 Solution and Methodology The progress of providing a solution for the stated problem is divided to two phases; first, design and second, analyzing. After defining the objectives previous researches and methods which are used by different researchers are studied and then a system was designed which is based on these studies and the purpose is to reach the objectives. In second phase which is called analyzing, the consequence of the design and its effects on the system improvement is determined. This paper lead to design an application which use Fuzzy algorithm to detect DoS attacks in TCP and ICMP protocols, in this algorithm Fuzzy logic will process the data which was extracted from Network Flow Header to find the intrusion. Data collection after identifying the problem provided the idea as paper topic. Basic and general information about IDS is gathered and then a discussion about DoS attacks and their behaviors is conducted. These studies show two important problems in IDS, they are low speed and low detection rate for detecting DoS. The solution which is proposed is monitoring the network flow using fuzzy system to increase the speed and rate of DoS attack detection. 3.1 Design In this step, the study on related works is done and the mechanism of similar systems in details to find out what mechanism should be used to detect DoS attacks, is analyzed. There was an analysis between anomaly based detection and signature based detection in IDS, and finally signature based detection selected, because most of DOS attacks have their own signature and rate of detection is high in signature based detection. A review of previous researches was done to get a complete view about the proposed mechanisms for intrusion detection system for detecting DOS attacks and finally fuzzy decision tree was chosen to be the engine of IDS for analyzing the traffic and find DOS attacks. The system designed in a way to reach the objectives which declared in identifying problem phase and also considering knowledge from Gathering data. In this phase proposed architecture of the system is described in detail. Enough consideration should be taken to design the component to have a correct output from each module and whole system (attack report). 3.2 Analyzing The designed system monitors the network traffic and put all of the packets into the network flows but in the same time fuzzy engine finds suspicious packet and save those flows in the array. At the end, fuzzy decision tree will check headers of suspicious flow and in case of attack, the system generates the error. By applying fuzzy logic on traffic sample of Defense Advanced Research Projects Agency (DARPA) from Lincoln Laboratory of Massachusetts Institute of Technology (MIT) the system is tested and detection rates and performance of application is shown. The output of this phase is to achieve designing the algorithm to detect DoS attacks by Fuzzy Engine. 4 System Design Based on the findings from related researches and works the design of the system is introduced. There is an overview of the solution to show all the processes and how it can improve the performance or accuracy of the system. The system architecture is the first thing which is mentioned here, and then an overview of Fuzzy and Network Flow is mentioned. The full description about how to apply Fuzzy and Network Flow on IDS is followed by how these two improve the detection and speed. 4.1 Architecture

3 Figure 1 shows the whole process of the system with some details. In this Design the IDS collects all of the packets from Traffic Sample and put them inside of the flows to save inside of the memory. Meanwhile, the fuzzy engine is collecting any suspicious packet, and put them inside of suspicious flow. Whenever the suspicious flow is finished, the fuzzy engine will check it for the final attack report. 4.2 Preprocess Data TCP and ICMP packets from network are gauge and the network flows are constructed by the Network Flow Engine (NFE). Identification of flows for TCP packets are based on the numbers of packets from same Source, Destination, Source Port and Destination Port. It starts with SYN packet and it will finish when the FIN packet arrive. On the other hand for ICMP protocol, NFE could be defining two types of packets. First packet contains a request from one machine to another and second packet is the answer of request. The NFE will check the network flows for any anomaly behavior. Network Flow contains numbers of packet in one communication in the network. Traffic Sample Packets 4.4 Signature Based As mentioned before the objective of this paper was to detect 4 types of DOS attacks in DARPA traffic sample, so there is an explanation about each attack Land Attack protocol of incoming packet is TCP and Source IP and Destination IP are same as each other and Source Port is equal to Destination Port the Land attack will happen Mail-Bomb Attack There will be a TCP flow in this attack after establishment of one TCP connection between two computers. In this flow SMTP port will be used to send but the number of packets in one Flow is about 10,000 packets and size of each packet is about 1,000 byte. So size of flow will be about 10 MByte Smurf Attack There will be several ICMP flows in this attack, the number of packets in one flow is low but size of each packet is approximately 1,000 byte. However the number of flow will be high because several computers send a large packet to single computer. The packet contains Reply message but Request message never sent from victim. Network Flow Flow Engine Packets Fuzzy Decision Tree Suspicious Packets Suspisious Flow Ping of Death Attack There will be large number of oversize IP packets in one flow from one computer to another. Each packet is about 1,000 byte and size of attack flow if high, approximately 64,000 Bytes and it is under ICMP protocol which causes rebooting, freezing and crashing the victim machine. Network FLow Time of Last Packet Suspicious Successful Network Flow Final Fuzzy Engine Real Attack Attack Array Figure 1: DFD System Architecture Context Diagram 4.3 Design Issue Some of the issues in IDS are false positive error, false negative error, rate of detection, performance and speed. By using Network Flow as input and applying fuzzy decision tree as an engine for intrusion detection the result could have less False Positive error and better rate of detection and also better performance and speed. 4.5 Fuzzy Fuzzy sets just include 0 and 1, so there could be only two options, but in fuzzy logic by combination of several fuzzy set there could be several answers. Table 2 explains how fuzzy set and fuzzy logic combined in this system and made the fuzzy decision tree for detecting the 4 types of DOS attacks. Meanwhile the fuzzy engine looks for suspicious packets to change the status of flow from Normal to Suspicious to speed up the detection. This sub-process will be suspicious to packets which have following attribute (pseudo code form): For Land attack it is using these rules IF flowprtcl equal to TCP IF flowsrc equal to flowdest Record to Land attack array For MailBomb attack the rule which applied is IF flowprtc equal to TCP

4 IF flowdestport equal to SMTP IF flowsize >10 MB Record to MainBomb attack array For Smurf attack the rule applied is IF flowprtc equal to ICMP IF info contain Reply FOR Packet from last minute, to this Packet, go one by one IF info not contain Request from same machine Attack Prtc TCP Src Dest Record to Smurf attack array For Ping of Death attack rule applied is IF flowprtc equal to IP IF info contain ICMP Record to Ping of Death array Table 2: Combination of fuzzy set and fuzzy logic Flow Size Packet count No Flow Prtc ICMP Prtc IP Land Mail Bomb Mb packet Ping of Death b<M<10 or H>10Mb 60 packet Packet Size 1000b Smurf Figure 2 shows the full Fuzzy decision tree for detecting four types of DOS attacks. All of the rules in the Fuzzy decision tree are based on the attack signature which comes from DARPA website. 5 Analysis and Conclusion The design of fuzzy decision tree which can detect four types of DOS attacks by analyzing network flow is described. The proposed architecture is a guideline for implementation of the system. Experiments are conducted with the used of dataset from DARPA. Previous solutions on IDS were based on detection method which used packets data and resulted with high false errors. In this study the IDS design focused on solving the problem by applying fuzzy decision tree as processor and network flow as input of system. In this system, all of the packets are initially preprocessed and the subsequently the network flows are constructed. During this process, fuzzy engine will put all suspicious packets in to the memory. Finally, the flow header will be generated, the suspicious flow will be checked again by fuzzy engine and detected attacks will be printed. Using network flow as input of the proposed IDS was a method to increase detection rate of four types of DOS attacks, for example Land attack start with a flow which contain same source and destination IP, or in mail bomb attack the system must save the size of SMTP flow, attack like Smurf must be detected by counting number of flow to one machine and finally in ping of death attack number of packet in one flow must be high. Another method was using fuzzy decision tree inside of IDS. One of the main focuses in this project was to use simplest rules to detect four types of DOS attack; simple rules make time of process less so the speed of detection will be fast. Also for improvement of rate of detection the fuzzy decision tree applied rules from DARPA website, so in that case all of the signatures are 100% true and reliable. Table 3 shows the performance of fuzzy decision tree to detect DoS. Table 3: Performance of Fuzzy decision tree to detect DoS Name of attack False False Rate of Negative Positive Detection Land Attack 0% 0% 100% MailBomb Attack 0% 0% 100% Ping-of- Death 0% 0% 100% Smurf 0% 0% 100% In this solution the Land attack will be detected when one TCP packet which contain SYN come to the network with same source IP and destination IP and same source port and destination port, at that moment the system will report the alarm. Mail-Bomb attack will be detected when size of one SMTP flow exceeds a determined critical point and the system will generate alarm.

5 This system will generate alarm of Ping of Death attack when flow size in ICMP protocol exceeds certain number of bytes when packet size in one flow is high (more that 1000 bytes). Finally in Smurf attack the system will generate error when first N (certain number) network flows in ICMP protocol happen to one machine in short period of time when there is only Reply packet (no Request packet in last minute). These rules which mentioned above make this system fast enough to detect those DoS attacks. Figure 2 : Fuzzy Decision Tree 6 References [1] Molina, J., and Cukier, M. (2009). Evaluating Attack Resiliency for Host Intrusion Detection Systems. Journal of Information Assurance and Security, volume 4, no 1, [2] Debar, H., Dacier, M., Wespi, A. (1999). Towards a taxonomy of intrusion-detection systems. ACM Computer

6 Networks: The International Journal of Computer and Telecommunications Networking - Special issue on computer network security, Volume 31, Issue 8, [3] Sundaram, A. (1996). An Introduction to Intrusion Detection. ACM Crossroads - Special issue on computer security, Volume 2, Issue 4. [4] Anderson, D., Lunt, T. F., Javitz, H., Tamaru, A., Valdes, A. (1995). Detecting unusual program behavior using the statistical component of the next-generation intrusion detection expert system (NIDES), In SRI International Computer Science Laboratory Technical Report SRI-CSL [5] SPADE, Silicon Defense, [6] Mahoney, M. V., and Chan, P. K.,(2001) Detecting Novel Attacks by Identifying Anomalous Network Packet Headers. Florida Institute of Technology, Technical Report, CS [7] Waizumi, Y., Kudo, D., Kato, N., Nemoto, Y. (2005). A New Network Anomaly Detection Technique Based on Per- Flow and Per-Service Statistics, In Proceedings CIS IEEE, [8] Moore, D., Shannon, C., Brown, D. J., Voelker, G. M., Savage, S. (2006) Inferring Internet Denial-of-Service Activity. ACM Transactions on Computer Systems in 2006, Volume 24, No 2, [9] Frank, J., (2004). artificial intelligence and intrusion detection: current and future directions. In proceedings of the 17th national computer security conference. Volume 10. [10] Lee, W., Nimbalkar, R. A., Yee, K. Y., Patil, S. B., Desai, P. H., Tran, T. T., Stolfo, S. J.(2000). a data mining and CIDF based approach for detecting novel and distributed intrusions. In proceeding of 3rd international workshop on the recent advances in intrusion detection, Toulouse, France, Volume 1907, [11] Sap, M.N.M., Abdullah, A.H., Srinoy, S., Chimphle, S., Chimphle, W.,(2006). Anomaly Intrusion Detection Using Fuzzy Clustering Methods, Jurnal Teknologi Maklumat, FSKSM, UTM, Jurnal Teknologi Maklumat, Jld. Volume 18, [12] Fries, T. P. (2008). A Fuzzy-Genetic Approach to Network Intrusion Detection. Proceedings of the 2008 GECCO conference companion on Genetic and evolutionary computation, Atlanta, GA, USA, [13] CERT Coordination Center, CERT/CC Statistics ( );