Distributed Snort Network Intrusion Detection System with Load Balancing Approach

Transcription

1 Distributed Snort Network Intrusion Detection System with Load Balancing Approach Wu Yuan, Jeff Tan, Phu Dung Le Faculty of Information Technology Monash University Melbourne, Australia {Tennyson.Yuan, Jeff.Tan, Abstract As we enjoy the conveniences that the Internet or computer networks have brought to us, the problems are getting larger, especially network security problems. A Network Intrusion Detection System (NIDS) is one of the critical components in a network nowadays. It can monitor and analyze activities of network users, and then uses knowledge of attack patterns to identify and prevent such attacks. It can minimize damages that will be caused by attacks. This paper uses Snort, which is one of the most commonly used NIDS in industry. The paper presents an approach of Distributed Snort NIDS, which can coordinate multiple sensors across the Local Area Network to optimize usage of computational resources. The approach implements a Balance Control System (BCS) for each subnet, which monitors CPU usage of a particular Snort NIDS and, when the Snort IDS s CPU usage is too high, delegates analysis work to lightly loaded IDS host. Keywords-Network Security; Network Intrusion Detection System; Snort; IDS; Distributed Snort NIDS; Load Balancing. I. INTRODUCTION According to FBI s Internet Crime Complaint Centre 2009 annual report that financial loss has doubled in 2009 compared with 2008 [5]. As importance of information security has increased significantly during recent years, the term of Intrusion Detection (ID) has become more and more important in current environment of computer and network systems. An Intrusion Detection System (IDS) is a piece of software program, which is able to monitor malicious activities or policy violations in a specific network or a computer system, and give particular reactions or alerts based on pre-set rules or knowledge database [18]. There are three major types of IDS: Network intrusion detection system (NIDS), Host-based intrusion detection system (HIDS) and Stack-based intrusion detection system (SIDS) [18]. In this paper, Network intrusion detection system is the one, which will be discussed. According to Ptacek and Newsham [17], the Network Intrusion Detection System is a particular type of IDS, which is used to monitor activities in network traffic and large numbers of hosts. Normally, the NIDS is connected to a network hub, switch or tap for gaining network access. The system also includes multiple sensors across the whole network to collect information. The sensors are often located in a Demilitarized Zone (DMZ) or at network borders. Snort is one good example of NIDS. According to Sourcefire [20], Snort is one of the most commonly used network intrusion detection system in industry, which is provided lots of advantages, such as, open source, lightweight and rule-based. Currently, main issue we are facing is the system needs to spend more time to analyze each traffic pattern, because a database of knowledge of attacks patterns is getting obviously larger. Therefore, attackers may gain more time to perform some unauthorized or illegal activities in some components of a network. The ideal of Distributed Intrusion Detection Systems (DIDS) has been mainly used to increase efficiency of the NIDS, which distributes the IDS into different network segments to analyze and monitor network traffics in that specific network segment, or distribute the analysis work to a number of IDSs to increase the speed of traffic analysis. In this paper, we proposed a distributed Snort NIDS with load balancing mechanism, which can improve the performance of Snort NIDS to reduce the risks that may be brought by packets dropping in large network traffic environment. The rest parts of this paper are organized as follow. Section 2 provides an overview of Snort NIDS technology. Then, an overview of existing distributed IDS approaches are discussed in Section 3. In Section 4, our proposed Distributed Snort NIDS (DSNIDS) model with load balancing mechanism is introduced. The evaluation and testing of our approach are presented in the following section (Section 5). II. AN OVERVIEW OF SNORT NIDS Snort NIDS is one of the most widely used and the most famous Network Intrusion Detection System (NIDS) [20][3]. It is an open-source application, which provides packet sniffing, packet logging, and intrusion detection, which search and scan each network packet s contents to match pre-set intrusion rules. Another advantage of Snort is that it is a lightweight NIDS, because it has a small footprint, and it has less system resources requirement compare with a normal NIDS. It uses rules to detect any anomalous behavior and malicious activities in a network. If any network packet breaches the intrusion rules, an alert will be triggered. The Snort NIDS is able to perform realtime packet logging and traffic analysis on Internet Protocol (IP) network by applying content searching and content matching. As mentioned above, the IDS can be configured

2 to run in three modes, namely, sniffer mode, packet logger mode, and network intrusion detection mode [3]. According to Baker and Esler [3], the Sniffer mode allows the IDS to capture packets in the network, and display them on the console, and the Packet logger mode allows the IDS to log the packets to the disk. The network intrusion detection mode allows the IDS to analyze the network traffic against user pre-defined rules, and to perform actions that has defined in the rules. In our research, we are trying to improve the performance when the Snort NIDS is running at network intrusion detection mode. A. Snort System architecture Snort NIDS s architecture contains four main components: Sniffer: The sniffer is used to eavesdrop network traffic, which can be used in Network analysis and troubleshooting, performance analysis and benchmarking, and eavesdropping [3]. Preprocessor: Pre-processor in the Snort NIDS applies raw network packets that are captured by the sniffer. Detection Engine: After the raw packets processed by all enabled pre-processors process packets, they will then be handled by detection engine. The detection engine analysis those packets against a set of rules, if any particular rule matches the payload or data of a packet, then the packet will be sent to alert processor [3]. Alerting and Logging Component: alerting and logging component can save the alerts from the detection engine to a log file, or be sent to SNMP traps. An SQL database can be linked also [3]. In the basic concept, the Snort NIDS is a packet sniffier, and it is designed to capture network packets, and a preprocessor will process them, and then these captured packets will be checked against a set of rules by a detection engine [3]. The figure above (Figure 2.1) shows a basic view of the Snort NIDS s architecture. III. Figure 2.1 Snort NIDS architecture [3] AN OVERVIEW OF DISTRIBUTED NIDS APPROACHES Due to large network traffic on a broadcast LAN segment, and longer time consuming on packet analyzing in non-distributed IDS model, the IDS may drops large numbers of network packets, which gives a big opportunities for the anomalous behavior to be launched. Therefore, the non-distributed or centralized IDS model cannot satisfy current network and security environments. Distributing a number of Intrusion Detection Systems across the network is a way to significantly increase the capability of the intrusion detection system. In this section, two main Distributed Intrusion Detection approaches are discussed. A. Early Prototype of DIDS The concept and architecture of a Distributed Intrusion Detection Systems was created in 1991 at University of California by Snapp et al. [19]. In his DIDS architecture, the system contains three main components, DIDS director, host monitor and LAN monitor. The distributed monitors collect information, and send them to a centralized DIDS director to analyze. The host monitor is a kind of program, which is installed in each host computer. The monitor will analyze audit data in the host computer, and decide whether to forward the audit data to an expert system or the DIDS director to do further evaluation and analysis or not. Normally, critical information about the host computer is always sent to the centralized expert system or the DIDS director to evaluate. The LAN monitor is another component of this DIDS architecture. It is in charge of analyzing network traffic in one specific LAN segment. It can monitor network users activities, network connection and the volume of the traffic. Same as host monitor, the LAN monitor also can identify and analyze some certain events, and all the security information are sent to the centralized expert system for further analysis. Finally, the last component of the DIDS architecture is the expert system, which is similar to an intrusion detection system like NSM, and SNORT. It is a rule-based system, and written in prolog. The expert system uses rules that are generated by Intrusion Detection Model (IDM), which describes the pattern of an intrusion from the audit data that are collected by the host monitors and the LAN monitors. There are six different levels in IDM, they are data level, event level, subject level, context level, threat level and security level, and each of those levels represents a performance of transformation from audit records. Snapp et al. [19] develops an early prototype of a Distributed Intrusion Detection Systems. As the DIDS architecture distributed the monitors across the entire network, it can collect network information from different sources, and all those information is processed in the centralized expert system to prevent doorknob attack. However, As nowadays the number of rules is getting much larger, the system needs to spend much more time to analyze each traffic packet, and the prototype only distributes monitors across the network, and only uses one centralized expert system to analyze all information; therefore the overloaded IDS will drop some analysis and detection regularly, due to insufficient memory resources. It gives more chance to an attack to performance some damage activities [25].

3 Figure 3.1 EDL Signature Example [25] Figure 3.2 example of DIDS concept [25] B. DIDS with Multi Step Signatures The recent research by Vogel and Schmerl [25] has solved this problem, and significantly increased efficiency of a Distributed Intrusion Detection Systems by applying Multi Step Signatures, and arranging those signatures to different distributed IDSs. The approach uses Event Description Language (EDL) to define a multi-step signature. Figure 3.1 shows an example of an EDL signature; it consists of four places and three transitions. The place indicates the system states of an attack, there are four types of places: initial, interior, escape and exit. The transition represents changes of a state that are triggered by audit events. In this approach, a sensor or monitor logs audit events, and then separate them into different event types. The signature will also be separate into some small parts of signatures, and all those small parts are then assigned to different analysis unit, which is based on availability of analysis units. A filter is in charge of discarding those audit data that is not relevant to event types to minimize the communication, and transfers those relevant audit data to certain distributed analysis units. The Figure 3.2 shows an example of this concept. The research shows that this approach of Distributed Intrusion Detection Systems is 60 % faster than the centralized Intrusion Detection System. However, the research also indicates that analysis distribution is not well balanced. IV. THE PROPOSED DISTRIBUTED SNORT NIDS MODEL Snort is the most famous and widely used Network Intrusion Detection System; however, when it faces large network traffic, it may drops amount of network packets depend on hardware configuration that may increase false positive rate on attacking detection. The idea of the DSNIDS model is to coordinate all distributed Snort NIDS sensors together, and optimizes all available computation resources. In this section, we present the design of our DSIDS approach from the network architecture, the load balancing mechanism, and the internal communication methods. A. Network Architecture The Network architecture of our Distributed Snort NIDS approach is based on one of Baker s basic Snort NIDS network architectures, which installs one Snort NIDS for each component or subnet of the network. The details of Baker s Snort NIDS network architectures are discussed in the chapter three. We design two network architectures for our approach, one of them has two Snort NIDS sensors, one for each sub-network, and another one has one Snort NIDS sensor and one Balancing Control System (BCS) for each sub-network. The choice between these two network architectures depends on budget for hardware configuration.

4 Figure 4.1 Flow chat of Load Balancing mechanism B. Load Balancing Design The Figure 4.1 illustrates the program processing flow of the mechanism. The program runs on each Snort NIDS sensor, and each BCS. A detailed list of processes of our mechanism is provided below: 1) Each Snort NIDS runs a program, which checks CPU usage of the IDS, and save it to a file. 2) The CPU usage file is then sent to the controller. 3) The BCS captures network traffic into pcap file (the BCS only capture 10 seconds of network traffic), when the CPU usage of its home network IDS sensor is over 75%. 4) The BCS runs a program, which will get the both IDS sensors CPU usage files, and each BCS checks the CPU usage of its home network IDS sensor, if the usage is over 75%, and any of the IDS sensors in the network is below 50%. The BCS sends the pcap file to the idle Snort NIDS to analyze. 5) This process will restart every 10 seconds. C. Internal Communication methods From the Figure 4.2, we designed this private network to connect each Snort NIDS, each BCS, and a controller together, because exchanging CPU usage data and pcap files may use large amount of the network bandwidth, and it may affect performance of the packets capture. By using private network to connect these devices, data can be exchanged faster and more secure without too much security consideration. Figure 4.2 Load Balancing Design V. EVALUATION OF DSNIDS APPROACH This chapter presents the details of tests that we have done on the DSIDS approach. These tests illustrate performance improvement and benefits of our DSIDS approach with load balancing.

5 A. Traditional DSNIDS Figure 5.1 Traditional DSNIDS Figure 5.1 shows the testing network architecture of the traditional DSIDS. Testing scenario is when there is a large volume of network traffic in subnet, at same time client 2 uses FTP to transfer data to server 1 anonymously, which will cause IDS1 generates FTP anonymous user login attempt alert. Test Results: With large network traffic Test No. Test 1 Test 2 Test 3 Test 4 No. of attacks detected Test No. Without other network traffic Test 1 Test 2 Test3 Test4 No. of attacks detected Table 5.1 Traditional DSIDS We performed eight times tests; four of them are with large network traffic. The testing result is shown in Table 5.1, and from the result we can see that when the network with large number of traffic, the Snort NIDS drops large amount of packets. This is caused by the size of the packets queue is over the buffer size of the IDS. During the test, we also found out that the IDS1 s CPU usage reached 100% when the client 1 launched UDP flooding. However, the IDS2 still have plenty of resources are available. As we know, NIDS is hardware-sensitive, the testing results above is affected by limited resources, because we are running it as Figure 5.2 DSIDS Network Architecture with two BCS Test Details: Time: 5 mins. Total Number of FTP login attempted: 10 times Client 1 uses hping to generate large amoung of zero size UDP packets to keep IDS1 busy. numbers of virtual machines. B. The DSIDS Approach with Load Balancing Our DSIDS approach with load balancing mechanism makes sure computation recourses are coordinated together. To achieve that, we designed an assistant system, namely BCS, which is used to capture the network traffics in one particular subnet when the IDS is busy, and then the

6 network capture file sends to another free IDS to analyse. To test performance of our DSIDS approach, we create two testing environments. The first test uses our virtual network environment to test the performance of attacking detection in a small size LAN. The second test uses powerful cluster to run multiple snort sensors, and we give each of them a network capture file (each sensor gets different pcap file with different volume of network traffic) to analysis simultaneously. The test two is designed to show the performance of our approach, when there are multiple powerful snort sensors are available. 1) Test One Small size network The test one is applied on our virtual network environment to test the performance of attacking detection in a small size LAN (See Figure 5.2). However, due to limited computation resource, we only turned on one client machine for each subnet. The testing environment, and the IDS sensors hardware configuration are kept the same. Testing Scenario: the client in the subnet launches UDP flooding Denali of Service attack, which generates huge amount of empty UDP packets. At the same time, the client in the subnet uses FTP to send file to server 1 anonymously. Test Details: Time: 1 mins Total number of FTP logins: 6 times (Once per 10 seconds) Test Results: Testing for the DSIDS approach Test Test Total Number of UDP packets Total Number of UDP packets Test IDS 1 detected Total Number of UDP packets IDS 2 detected Total Number of FTP login IDS 1 detected Total Number of FTP login IDS 2 detected Table 5.2 the DSIDS approach From the Table 5.2, we can see that when IDS1 faces large amount of network packets, it drops large amount of network packets, as a result, the IDS 1 cannot detect any FTP anonymous login attempts. However, the BCS capture the network packets in the subnet for 1 min, and store them into a pcap file. The pcap file then send to IDS 2 to analyze. As a result, the IDS 2 can detect almost all attacks. 2) Test Two Multiple High Power Snort Sensor The second test uses six nodes (See Table 5.3) in a Campus HPC cluster to run multiple snort sensors, and we give each of them a network capture file (each sensor gets different pcap file with different volume of network traffic) to analyze simultaneously. In this approach, the computational hosts are not exactly sensors: they do not capture packets, but only analyze them. The HPC cluster is a Sun (Oracle) Grid Engine cluster, which has over 3,300 cores over 200 execute nodes, and over 2.5 TB RAM available. Test Two is designed to show the performance of our approach, when there are multiple powerful snort sensors available purely for analysis. Testing Nodes Details Hostname Architecture No. of Cores Total Memory gn116 lx24-amd G gn152 lx24-amd G gn60 lx24-amd G gn62 lx24-amd G gn63 lx24-amd G gn65 lx24-amd G Table 5.3 Testing Nodes Details Testing Scenario: Step one: We used the BCS to capture the network traffic ten times into ten pcap files, and the system listened for 5 mins each time. These pcap files contains different level of traffic volume, and different anomalous activities, such as UDP flooding, ICMP flooding, normal network traffic, large size of ICMP packets, FTP anonymous logins, etc. Then, these ten pcap files were arranged to the six nodes in the cluster, and run 10 Snort analysis jobs simultaneously. Step two: We double the numbers of pcap files to 20 by duplicating the pcap files from the step one, and then we assigned 20 Snort jobs to analysis these 20 pcap files simultaneously. Step three: We increase the numbers of pcap files to 100 by duplicating the pcap files from the step one, and then we assigned 100 Snort jobs to analysis these 100 pcap files simultaneously. Testing Results: Step One: Number of Done jobs 10 Number of failed jobs 0 Time Spent for all jobs 22 mins secs done Average job wall time 7 mins secs Maximum job wall time 20 mins secs Minimum job wall time secs Table 5.4 Step One Results

7 Step Two: Number of Done jobs 20 Number of failed jobs 0 Time Spent for all jobs 23 mins secs done Average job wall time 6 mins secs Maximum job wall time 20 mins secs Minimum job wall time secs Table 5.5 Step Two Results Step Three: Number of Done jobs 100 Number of failed jobs 0 Time Spent for all jobs 45 mins 4.54 secs done Average job wall time 5 mins secs Maximum job wall time 13 mins secs Minimum job wall time secs Table 5.6 Step Three Results From the testing results above, we can see, the total time spent for six multi-core nodes to run 20 Snort jobs is almost similar to when the six nodes run 10 jobs, and we also can see that the average time spent on each job is shorter than when they run 10 jobs. This is because the jobs are allocated to free cores that can therefore execute the analyses at the same time. Therefore, if the network has multiple Snort machines with powerful multi-core hardware, and assuming they are idle, the BCS in our approach can assign more than one analysis jobs to each machine at same time, and each Snort machine can run these jobs simultaneously to achieve higher performance. From Test three we can see that 100 snort jobs did not run simultaneously, because the total time spent is much larger than maximum job wall time. This is due to insufficient computational resources. However, the Maximum job wall time and minimum job wall time are significantly less than previous tests. VI. CONCLUSION Our research presents an approach of Distributed Snort NIDS, which can coordinate multiple sensors across the Local Area Network to optimize usage of computation resources. However the approach is unable to drop abnormal network packets, and it can only give alerts to the system administrators. Due to the approach uses BCS to duplicate network traffic when a particular IDS s CPU usage is high, which causes some amount of network traffic may be analyzed more than once by multiple Snort NIDS sensors. To extend the research, a filter should be designed and developed to solve some network packets may be analyzed more than once by different Snort NIDS sensors, and improving the performance when facing large amount of traffic flow. REFERENCES [1]. Abramson, D., Bethwaite, B., Enticott, C., Garic, S., Peachey, T., Michailova, A., et al. (2009). Robust Workflows for Science and Engineering. 2nd Workshop on Many-Task Computing on Grids and Supercomputers(MTAGS 2009). Portland. [2]. Andrzej, G. (1991). Distributed operating systems: the logical design. London: Longman Group. [3]. Baker, A. R., & Esler, J. (2007). Snort IDS and IPS Toolkit. Burlington: Syngress Publishing, Inc. [4]. Chakrabarti, S., Chakraborty, M., & Mukhopadhyay, I. (2010). Campus Network Security Study of Snort-based IDS. International Conference and Workshop on Emerging Trends in Technology. [5]. CyberInsecure.com. (2010, 03 16). Cybercrime Related Losses Doubled In 2009, Financial Losses Totaled Million. Retrieved 06 1, 2012 from losses-doubled-in-2009-financial-lossestotaled million/ [6]. Denning, D. (1987). An intrusion-detection model. leee Trans. on Soft- ware Engg., SE (13), [7]. Dineley, D., & Mobley, H. (2009). The Greatest Open Source Software of All Time. Retrieved 06 03, 2012 from InfoWorld, Inc.: [8]. Dowell, C., & Ramstedt.P. (1990). The COMPUTERWATCH data reduction tool. 13th National Computer Security Conference, (pp ). Washington, DC. [9]. Heberlein, L., Dias, G., Levitt, K., Mukherjee, B., Wood, J., & Wolber, D. (1990). A network security monitor Symposium on Research in Security and Privacy, (pp ). [10]. Hochberg, J. (1993). NADIR: an automated system for detecting network intrusion and misuse. Computers and Security, 12 (3), [11]. Javitz, H., & Valdez, A. (1991). The SRI IDES Statistical Anomaly Detector IEEE Symposium on Research in Security and F'rivacy. Oakland. [12]. Laing, B. (2000). How to guide-implementing a network based intrusion detection system. Reading: Internet Security Systems. [13]. Ling, J. (2012). Campus Network Security Program Based on Snort Network Security Intrusion Detection System. Advanced Materials Research, [14]. Mukherjee, B., Heberlein, L., & Levitt.K. (1994). Network Intrusion Detection. IEEE Network, 8 (3), [15]. Monash University. (2010, 07 28). About the Monash network. Retrieved 05 30, 2012 from

8 Monash University: / [16]. Newman, R. C. (2010). Computer Security: Protecting Digital Resources. Sudbury: Jones and Bartlett Publishers. [17]. Ptacek, T., & Newsham, T. (1998). Insertion, Evasion, and Denial of Service: Eluding Network Instrusion Detection. Secure Networks, Inc. [18]. Scarfone, K., & Mell, P. (2007). Guide to Intrusion Detection and Prevention Systems (IDPS). Gaithersburg: National Institute of Standards and Technology. [19]. Snapp, S., Brentano, J., Dias, G., & Goan, T. e. (1991). DIDS(Distributed intrusion detection system) motivation, architecture, and an early prototype. the Fourteenth National Computer Security Conference, (pp ). [20]. Sourcefire, Inc. (2011, 12 7). SNORT User Manual. Retrieved 02 29, 2012 from [21]. Sourdis, I., & Pnevmatikatos, D. (2003). Fast, Large-Scale String Match for a 10Gbps Fpga- Based Network Intrusion. FPL, 2003, [22]. Softpanorama. (2012, 11 4). TCP Performance Tuning. Retrieved 10 20, 2012 from Softpanorama: /Performance_tuning/tcp_performance_tuning.sht ml [23]. Tener, W. (1988). AI and 4GL: automated detection and investigation and detection tools. IFIP Security Conference. [24]. Tener, W. T. (1986). Discovery: An expert system in the commercial data security environment. In IFIP Security Conference. [25]. Vogel, M., & Schmerl, S. (2011). Efficient Distributed Intrusion Detection applying Multi Step Signatures. 17th GI/ITG Conference on Communication in Distributed Systems, (pp ). [26]. Weir, J. (2012, 07 20). Building a Debian\Snort based IDS.