Validation of the Network-based Dictionary Attack Detection

Transcription

1 Validation of the Network-based Dictionary Attack Detection Jan Vykopal Tomáš Plesník Institute of Computer Science Masaryk University Brno, Czech Republic Pavel Minařík Abstract This paper presents a study of successful dictionary attacks against a SSH server and their network-based detection. On the basis of experience in the protection of university network we developed a detection algorithm based on a generic SSH authentication pattern. Thanks to the network-based approach, the detection algorithm is host independent and highly scalable. We deployed a high-interaction honeypot based on VMware to validate the SSH dictionary attack pattern that is able to recognize a successful The honeypot provides several user accounts secured by both weak and strong passwords. All the communication between the honeypot and other hosts was logged at the host and even network layer (the relevant NetFlow data were stored too). After successful or unsuccessful break-in attempt, we could reliably determine detection accuracy (the false positive and negative rate). The pattern was implemented using a dynamic decision tree technique, so we can propose some modifications of its parameters based on the results. In addition, we could validate the pattern because the detection relies only on the NetFlow data. This study also discusses the performance details of detection method and reveals methods and behaviour of present successful attackers. Next, these findings are compared to the conclusions of the previous study. In our future work, we will focus on an extension of the detection method to other network services and protocols than SSH. Further, the method should also provide some reasons for the decision that the attack occurred (e. g., distributed dictionary attack). Keywords: dictionary attack, SSH, NetFlow, attack pattern, validation, honeypot. 1 Introduction In our previous paper [1], we proposed a SSH dictionary attack pattern that is able to recognize a successful We inspected logs on attacked hosts and then identified appropriate traffic in NetFlow data collected at the border of a university network or its subnets. As a result, we derived the following dictionary attack pattern at NetFlow level: TCP port of the victim is 22, TCP port of the attacker is generally random and greater than 1024, many flows (tens or hundreds) from the attacker to the victim in a short time window (5 minutes), the flows from the attacker are small: from 10 to 30 packets and from to bytes, victims' responses are small too (typically the same number of packets and bytes), flow duration is up to 5 seconds, the last flow is different in case of successful attempt. 128 Security and Protection of Information 2009

2 Next, we analysed network traffic of various applications that utilize the SSH protocol (ssh, scp, Putty, WinSCP, sftp and rsync) to eliminate false positives. In short, we did not find any traffic that fully corresponds to the attack pattern derived from real dictionary attacks. What is more, the attack pattern matches all flows during our simulated attacks and, of course, the attacks captured in log files at the beginning of our research. The main idea of the detection algorithm is to monitor and tune up key indicators (flow duration, number of packets and number of bytes transferred in victim's reply to the attacker) of the proceeding attack between attacker and its victim and observe significant change of these indicators. After a pre-specified number of attempts an unsuccessful attack is reported. Sudden and significant change of the flow characteristics followed by the stop of the attack may indicate successful While attacks may vary among different attackers and victims, we need really adaptable approach to detect successful According to these requirements we chose decision tree method to implement the algorithm. Using dynamic decision tree, we are able to store network traffic statistics, attack indicators and relevant detection parameters persistently and what is most important to build the tree according to attacks in progress. In this paper, we evaluate detection accuracy (the false positive and negative rate). We deployed a highinteraction honeypot and examined behavior of attackers. Incoming network traffic to a honeypot is assumed malicious by the definition. Common network traffic should not reach the honeypot. When the attacker conducts a successful dictionary attack we are notified via . We can immediately validate if the detection algorithm is accurate or not, whether detected the dictionary attack itself and whether correctly determined that the attack was successful or not. In case of the inaccurate detection, we can tune the parameters of the detection pattern, run the improved pattern and validate the results again. Details of the test bed is described in the following section. Section 3 contains experimental results. Related work is summarized and results are compared to our work in Section 4. Section 5 concludes the paper. Security and Protection of Information

3 2 Test bed To validate results of the described detection method, we deployed several high-interaction honeypots and NetFlow probes in the Masaryk university network. The whole test bed depicts Figure 1. Figure 1: Test bed. 2.1 Honeypots We decided to deploy high-interaction honeypots because they are real systems, not only software simulating real systems. In order to lower costs and minimize maintenance effort, we employed VMware Server as a virtualization platform (host) and installed five virtual honeypots (guests) upon it. The guests were runing Ubuntu 8.10 Server Edition with a patched OpenSSH 5.1p1 server. The SSH server was modified in the following way: standard log file (auth.log) contains user names and even passwords entered in authentication process, a copy of standard log file (auth.log) is stored in uncommon path in file system, alert is sent after successful authentication via password. Each guest machine provides ten user accounts and one superuser account for maintenance purposes. These accounts were reachable from the whole Internet. We chose common user names and passwords on the basis of our previous research [1] and other studies [3, 4]. Superuser account called 'root' was disabled. 130 Security and Protection of Information 2009

4 SSH daemon was listening to the TCP port 22. Any other services and daemons were disabled. The guest machines only reply to ICMP echo requests, reset TCP connections except the TCP port 22 and send ICMP port error messages in case of a connection attempt to UDP ports. All outbound traffic from the guests was shaped to 32 kbps by Traffic Control in the Linux kernel [5]. When we observed tens scans of the TCP port 80, we deployed lighttpd web server on all honeypots to attract other attackers. 2.2 NetFlow Probes and Collector All incoming traffic passes the edge router and is monitored by FlowMon [6] probe connected via SPAN (mirror) port. In addition, another FlowMon probe monitors all traffic between guests and other hosts in the university network or the Internet. As a result, all network traffic of honeypots (regardless of its source or destination) is captured in NetFlow data exported by both FlowMon probes. NetFlow collector nfdump [7] stores NetFlow records and serve NetFlow data to the detection module. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device. [8] The flow is commonly identified by the following 5-tuple: (srcip, dstip, proto, srcport, dstport). Particularly, there is no information about payload. NetFlow provides traffic aggregation (sum of packets, bytes, flow duration etc.) and thus is eligible for multigigabit networks. 2.3 Detection Module The dictionary attack detection module is generally based on decision tree technique. In contrast to traditional decision trees which are static sets of conditions organized in a structure, the tree presented will grow dynamically according to the classified data. Each tree node allows a set of conditions and operations to be connected. The satisfaction of given conditions controls data flow through the decision tree and the execution of operations. The detection module works with NetFlow data (processes individual flows). The main idea of the detection algorithm is to focus on key attack indicators (flow duration, number of packets and number of bytes transferred in victim's reply to the attacker) changes of the proceeding Sudden or significant change of the attack indicators followed by the stop of the attack may indicate successful The detection module starts with a set of input flows sorted by flow_start and processes flows one by one in a sequence. Detection also starts with generic bounds for attack indicators that fit to most of the attacks. For each pair (attacker, victim) arrays of attack indicators are build (duration array, packets array and bytes array) until attack attempts threshold is reached. New bounds for given attacker and victim are calculated using toleration parameters afterwards. From that moment each flow between the bounds is considered as unsuccessful attack and each flow out of bounds is considered as successful More detail and formal explanation of the detection algorithm is available in [1]. From a performance point of view the detection module based on decision trees is able to handle thousands of events per second on COTS (cost of-the-shelf) hardware. Particular results will depend on actual tree structure. An analysis of performance demonstrates that the approach presented is able to handle the detection of dictionary attacks real time in a large high-speed network ( computers, 10 Gigabit Ethernet interface, all traffic approximately 220 million flows per day, SSH traffic approximately 1.5 million flows per day). Particular performance results are presented in section Results First of all, we define some important notions. An unsuccessful attack is at least 20 repetitive attempts to log in a guest machine in a short time originating from the only one IP address. Two contiguous attempts Security and Protection of Information

5 must occur in 30 minutes otherwise they are considered to be two different attacks. The attack is successful if and only if the password provided by the attacker is correct and he or she is successfully loged in. TCP SYN or TCP SYN/RST scan (of port X) is a reconnaissance probe to a server (to the TCP port X) when no TCP connection is established (connection handshake is exploited). It originates from the only one IP address. Similarly, UDP scan is a UDP probe to a server originating from the only one IP address regardless of the server reply. 3.1 Behaviour of Attackers We observed totally 65 SSH dictionary attacks during a 23 day period. Despite the fact that the fifty user accounts on five machines were secured by weak passwords, only 3 attacks were successful (4,61 %). Next, we also observed less than 20 repetitive log-in attempts for 16 times. No traffic (including TCP and UDP scans) originating in the defended network and destined to the honeypots was not observed. On the contrary, there were logged totally 938 TCP and 501 UDP scans originating outside the network. Table 1 shows numbers and types of scans destined to particular honeypots. Honeypot Total number of TCP scans Number of TCP SYN scans Number of TCP SYN/RST scans Total number of UDP scans H H H H H Table 1: Numbers and types of scans destined to particular honeypots. The most popular TCP ports was 1433 (MS SQL) with 197 scans followed by 80 (HTTP) with 79 scans and 4899 (radmin) with 67 scans. Considering UDP ports, the majority of scans were aimed at ports 1026 and 1027 (106 and 83 scans). Considering scans of standard SSH port (TCP port 22), we observed that 21 of 34 scans were followed by SSH dictionary attacks originating from the same IP address. The time between the scan and the attack varied from 6 minutes to 2 hours. In case of the successful attacks, only one of three attacks was preceded by the scan as was defined above. Other two successful attacks was preceded by an establishing of TCP connection to the port 22 (further SSH scan) about 1 and 9 hours before the According to the log file, the attackers did not try any password. The log files say that sshd did not receive identification string from attacker's IP address. Next, we identified the following attack scenarios and groups (AGx in short) comprising both successful and unsuccessful attacks. We chose all attacks conducted by intruders that were successful for at least one time. The first successful attack (in AG1) was preceded by SSH scan of all five honeypots. Finally, the attacker was logged on the only one honeypot as guest by password guest123. He or she was successful in 1 minute and 4 seconds, after 44 log-in attempts. The vast majority of attempts were tried with password as same as username. After successful break-in, the attacker continued with the dictionary attack until the honeypot was shut down. The total number of log-in attempts was No other attacker's activity was observed (e. g., modification of filesystem or file downloading). 132 Security and Protection of Information 2009

6 The second successful attack (in AG2) was preceded by TCP SYN/RST scan of all five honeypots. Similarly to AG1, although the attacker conducted dictionary attacks against all honeypots, he or she was successful on the only one honeypot. After 56 attempts tried in 3 minutes and 6 seconds, the attacker was logged as guest by password Again, no other attacker's activity was observed. After 4,5 hours, the attacker performed another TCP SYN/RST scan of the same honeypot and then after 38 minutes tried to log in as other users than guest for 9 times. The third successful attack (in AG3) was preceded by SSH scan of four honeypots. Similarly to AG1 and AG2, the attacker conducted attacks against four honeypots, but succeeded on the only one host after 401 attempts in 21 minutes and 48 seconds. He or she was logged as test by password qwerty. Again, the majority of log-in attempts were tried with password as same as username and no other attacker's activity was observed. Further, the attacker continued with attacks against other three honeypots. 3.2 Parameters of Detection and Results The detection was performed with default parameters set a follows: bpplimitreplies = 250; Maximum of bytes per packet in a SSH login reply flow, other flows are ignored. bpplimitrequests = 150; Maximum of bytes per packet in a SSH login request flow, other flows are ignored. deltabytescoefficient = 1.2; Tolerance in difference of bytes in a flow to report a successful deltadurationcoefficient = 1.2; Tolerance in difference of flow duration to report a successful deltapacketscoefficient = 1.2; Tolerance in difference of packets in a flow to report a successful failedattackresponsebytes = "1400,5000"; Initial bounds for bytes in a flow representing an failedattackresponseduration = "0.400,5.000"; Initial bounds for duration of a flow representing an failedattackresponsepackets = "10,30"; Initial bounds for packets in a flow representing an manyattackattempts = 20; Minimal count of attempts to report an tsdeltathreshold = 1800; Minimal slack in seconds between two flows to distinguish between two different attacks from a single attacker to a single victim. The parameters are described in details in [1]. Security and Protection of Information

7 Next, we validate the detection of the three selected attack groups AG1, AG2 and AG3. All 14 SSH dictionary attacks (5 in AG1, 5 in AG2 and 4 in AG3) were detected. What is more, no other attacks were detected and all attacks were correctly labeled as successful or unsuccessful except one. The false positive is caused by the fact that the attacker continued with the attack after successful intrusion. This is opposed to our expectations. The detection was run within a time window of each attack on NetFlow data collected on the honeypot interface eth1 (see Figure 1). 3.3 Performance Analysis To obtain results of performance tests, we also used NetFlow data collected on SPAN port of the edge router as input. The processed traffic was a few times greater than on the honeypot interface. Unfortunately, the detection accuracy was not satisfactory in this case. It was caused by biased primary data - network packets provided by the SPAN port. For instance, the edge router mirrors the passing packets in nondeterministic way. Table 2 shows number of processed flows and duration in seconds of key operations of the whole detection and the performance measured in flows per second: data_delivery comprises loading data from nfdump and receiving to the detection module server, stored_in_memory stands for a storing data to the memory, pairs_created an operation of flow pairing according to [9], port_filtered filtering TCP traffic (port 22), break_detected SSH dictionary attack detection including storing results in a relation table. This measurement was done for all three attack groups. Operations Attack group AG1 AG2 AG3 data_delivery stored_in_memory pairs_created port_filtered break_detected Number of processed flows Performance of the detection module Total time Overall performance in flows per second Table 2: Duration of key operation and performance of the detection module for selected attacks. 134 Security and Protection of Information 2009

8 4 Related Work Our results concerning behaviour of attackers can be compared to results of [3] and [4]. Both studies utilize honeypots to create an attacker profile. In contrast to these studies, we did not observed any activities including downloading, installing, running malicious software or password change in case of three successful attacks. But we can confirm very low percentage of successful attacks. Generally, attempted username and password patterns are very similar as in [3] and [4]. Next, we observed the majority of attacks were preceded by TCP scans that is different to the findings in [4]. On the contrary, we confirm other findings in [4] that the attacks follow very simple and repetitive patterns such attacks continued although the attacker has already guessed the correct password. 5 Conclusions The achieved results show that network-based attack detection has a large potential to substitute traditional host-based methods. During the detection pattern evaluation we identified only one false negative when a successful attack was identified as an unsuccessful Another important result consists in validation of primary data quality according to probe wiring. Using SPAN port connection the quality of primary data goes down rapidly which influences results of the attack detection. From the performance point of view presented method is capable to process the whole university SSH traffic in real time. Concerning behaviour of successful attackers we observed no malicious activities on the host even in network traffic in hours after intrusions. Surprisingly, one attacker continued with the dictionary attack after successful log in. This behaviour could advert to low-skilled attacker. In our future work we will focus on the on-line detection method deployment which has been already started. We would also like to validate the SSH attack pattern on other authenticated services like FTP or web logins. Acknowledgement This work was supported by the Czech Ministry of Defence under Contract No. SMO02008PR980- OVMASUN References [ 1 ] Vykopal, J., Plesnik, T., and Minarik, P.: Network-based Dictionary Attack Detection, in Proc. Of ICFN 2009, Bangkok, pp , ISBN [ 2 ] VMware, Inc. web site. [ 3 ] Ramsbrock, D., Berthier, R., and Cukier, M.: Profiling Attacker Behavior Following SSH Compromises, in Proc. 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, pp , [ 4 ] Alata, E., Nicomette, V., Kaâniche, M., Dacier, M., and Herrb, M.: Lessons learned from the deployment of a high-interaction honeypot, in Proc. 6th European Dependable Computing Conference (EDCC-6), Coimbra, pp , [ 5 ] Linux Advanced Routing & Traffic Control. [ 6 ] FlowMon probe web site. [ 7 ] Nfdump web site. Security and Protection of Information

9 [ 8 ] Claise, B.: Cisco Systems NetFlow Services Export Version 9. RFC 3954 (Informational), [ 9 ] Trammell, B. and Boschi, E.: Bidirectional Flow Export Using IP Flow Information Export (IPFIX). RFC 5103, Security and Protection of Information 2009