A Study on Intrusion Detection Techniques in a TCP/IP Environment

Transcription

1 A Study on Intrusion Detection Techniques in a TCP/IP Environment C. A. Voglis and S. A. Paschos Department of Computer Science University of Ioannina GREECE Abstract: The TCP/IP protocol suite is the most used family of networking protocols. The implementation of these protocols emphasizes in performance rather, than in security. The last years, when the growth of Internet is explosive, many attacks have been realized, leading in loss of valuable data. In this paper we present an easyto-implement technique for detecting and preventing a great number of attacks which are based on a particular attack, namely IP spoofing. Keywords: TCP/IP protocols, active attacks, IP spoofing, SYN flooding 1 Introduction In our days more and more people connect their machines to the Internet, therefore using the TCP/IP protocol suite as the network operating system for their communications needs. This growing usage of the Internet revealed some deficiencies of the design and the implementation of the protocols which form the TCP/IP suite. These deficiencies made possible a number of attacks that may cause serious problems, and possibly the loss of sensitive data. In section 2 we briefly present the terms of TCP/IP that we will use in the following sections. In section 3 we will describe the most common security attacks in a TCP/IP environment. In section 4 we thoroughly examine methods to detect and prevent the most common attacks. In section 5 we present some preliminary results we collected by using a software tool which implements the methods and techniques presented in section 4. Finally, in section 6 some useful extensions to the existing tool are suggested. 2 TCP/IP terminology The TCP/IP protocol suite allows computers running totally different operating systems to communicate with each other. Like many other networking protocols, it is developed in layers, with each layer responsible for different facet of the communication. TCP/IP is a 4-layer system and consists of the following layers: link layer; handles the details of physical transmission network layer; handles the movement of packets around the network (IP) transport layer; provides a flow of data between two hosts for the application application layer; handles the details of a particular application(telnet, ftp...) The attacks that we describe in this paper concern the network and transport layer and exploit the nature and implementation of IP ([5]) and TCP ([7]). Each network interface of a computer connected on a TCP/IP network must have a unique Internet address (also called IP address). These addresses are 32-bit numbers that specify a physical connection to the Internet. That means, that a host must be assigned a unique IP address, in order to communicate with another host on the IP level. The protocols of transport layer (TCP, UDP) use a similar addressing technique in order to determine to which application protocol the incoming data are addressed. The TCP protocol identifies the applications using a 16-bit port

2 number; the pair (IP address, TCP port number) forms a unique end point in an internet. When an application sends data using TCP, TCP pass a unit of data called TCP segment to IP. The unit of data that IP sends to the network interface is called IP datagram. In conclusion, a connection in the TCP/IP environment is a combination of four numbers (remote IP address, remote port number, local IP address, local port number). This identification of the specific connection is transmitted through the network. IP addresses are located in the IP header and TCP ports are in the TCP header. TCP in particular provides a connection oriented, reliable, byte stream service. In order to achieve these goals each TCP segment carries a lot of information except the application s data. That additional data form the TCP header. The fields of TCP header that we are going to use in our analysis are: source and destination ports the sequence number, that identifies the byte order in the stream of data from the sending TCP to the receiving TCP the acknowledgment number, that contains the next sequence number that the sender of the acknowledgment expects to receive the flag field, that contains six flag bits that give special meaning to the TCP segments. The most common flags are: SYN: Denotes the beginning of a session and that TCP segment is carrying the Initial Sequence Number (ISN) ACK: The packet is an acknowledge packet RST: Immediately finishes a connection, i.e. destroys all the data structures in kernel that belong to this particular connection FIN: Gracefully finishes a connection In later sections we will call SYN packet an IP datagram that encapsulates a TCP segment with the SYN flag set. We are going to use the notation: (sip, sport, dip, dport)[flags][seq][ack] in order to describe a packet, where: sip: sport: Sender s IP address Sender s Port number dip: dport: flags: seq: ack: Destination s IP address Destination s Port number TCP header s flags TCP header s sequence number TCP header s acknowledge number 3 Common Active Attacks The attacks we will describe are called active attacks by the fact that the attacker acts upon the victim s system in order to change the data addressed to it. The most common attacks of this type are: Man in the middle attack. In this attack the attacker interfere with the data flow between two hosts (S and C). The attacker usually is located on another machine (A). In order to perform his task he uses a packet sniffer to watch the data hosts S and C are exchanging and a packet generator in order to masquerade himself as one of the communicating hosts. By using properly modified packets, the attacker manages to desynchronize the two hosts, so they cannot receive packets from each other. Of course, they can both receive the attacker s packets. This attack is described in detail in [3]. Blind attack. In this case the attacker pretends to be on a different host (C) and sends a SYN packet to the victim machine (S) trying to contact with it ([1]). The victim responds with its SYN packet, which contains its TCP initial sequence number from which he starts to count the bytes he will send to the specific connection. This packet goes to the address which the attacker pretends to have (C), and not to the attacker s real address (A), so the attacker has to develop a technique in order to calculate or guess the victim s initial sequence number. This can be achieved by monitoring the traffic using a packet sniffer. If the attacker knows this number, he can create the third packet which is needed to complete the connection establishment. This attack cannot be successful if the false address used by the attacker belongs to another host. This happens because when a packet with the initial sequence number arrives to host C, will be rejected, and a RST packet will be sent to host S, which in turn, will close the connection. Morris ([4]) pointed out that the attacker can block a specific port on host C, thus preventing C from receiving the answer from host S, and consequently, no RST packet will be generated.

3 SYN Flood attack. In this attack the attacker (A) sends a lot of SYN packets to a specific port number of the victim host (S). S tries to fulfill these requests by assigning the appropriate resources in order to handle them ([2]). The original TCP implementation did not use a timer in order to timeout these connections, so the attacker could initiate as many connections as needed to block this port. Any consequent packet addressed to this port would be rejected. The active attacks we described above, have something in common. They all use the same technique, namely IP spoofing. IP spoofing is the situation where a packet is transmitted in a TCP/IP network containing in its IP header an address other than the sending host. Of course IP spoofing implies that the attacker creates his own TCP header or alters an existing one (use another port, sequence number). 4 Detecting and Preventing the attacks In this section we are going to present some characteristics of each one attack, that make the attack detectable. But first we are going to explore how to detect IP spoofing, which is the first step of all the attacks we described so far. 4.1 Detection When an attacker impersonates another host, by using a false IP address (spoofed address) in the IP header instead of his IP address, one of the following four cases exists: C-1 The spoofed address does not yet corresponds to an internet host C-2 The spoofed address corresponds to an internet host that is not currently operating (it is down) C-3 The spoofed address corresponds to an internet host that has some TCP ports blocked (SYN flooding attack) C-4 The spoofed address corresponds to an internet host that is up and running From the above cases, case C-4 is the one that every attacker wishes to avoid (except for the case of man-in-the middle attack), because a functioning host can respond to any suspicious packet that arrives, by sending a RST packet. According to the protocol specifications, when a packet arrives to a specific port of a host, and there is not a process listening to this port, or that port does not belong to a server, the packet s arrival cause the generation of a RST packet (for example, the packet (IP1, Port1, IP2, Port2)[*][seq][ack] can create the packet (IP2, Port2, IP1, Port1)[RST][ack] that attempts to destroy the possible connection). In the three first cases the attacker s success is assured, because there is not going to be any response from a host. That way no disastrous for the attacker RST packets are generated. Now we can explain our method for IP spoofing detection. In every case we are using a packet sniffer, to capture the data in any Ethernet frame (IP datagram and consequently TCP segment). Our analysis depends on the source of the suspect packet: In the first case the packet is coming from a host inside our local network and in the second it is coming from a host outside our local network. 1. Packet is coming from the inside: In this case detection is somehow easy, because in a local network there is a factor that can be considered unchangeable, and cannot be spoofed. That factor is the Ethernet address of a host. Every host in a local network (eg. Ethernet segment) has a unique interface (Ethernet) address, which is written in the header of the interface (Ethernet) frame. Furthermore, every host in a local TCP/IP network has a unique IP address. We can therefore create a correlation (in a form of a two row table) between IP address and Ethernet address of any existing host in the local network (in general, there are IP addresses which belong to the local network s address space but there are not assigned to a machine). That correlation can be updated automatically in a specified time period (using ARP protocol), so we can capture any local changes (replacement of a defective NIC, starting the operation of a host etc.). The fact that this correlation contains only the IP addresses that are assigned to a machine in the local network, ensures that an IP address that belongs to the address space of the local network but is not assigned to a host, does not exists in that correlation (in the following, we will call this correlation, table). Using this table, we can detect any IP spoofing attempt from the local network, because:

4 (a) If a packet comes from a host inside our local network with IP address IP1, and the packet s header declares IP2 as source IP address (IP2 belongs to the local network s address space) i. If IP2 does not correspond to a host in the network, does not appears in the table ii. If IP2 corresponds to a host in the local network then, according to the correlation table, this host s Ethernet address is going to be different from the Ethernet address that the table returns, since [IP1, ETHER1] = [IP2, ETHER2]. (b) If a packet comes from a host inside the local network but impersonates a host outside of it, then it will be detected because it is not going to have the Ethernet address of the router (or one of the routers), of the local network. Instead, it will have the Ethernet address of another host in the network. (Nowadays, a router is usually a dedicated machine, which means that it cannot generate a packet). We have observed that if the packet s source is a host inside our local network, then there is a very high possibility to detect attacks that are based on the IP spoofing technique. Every one of the four cases at the beginning of this section, can be detected in a local network. 2. Packet coming from the outside: Unfortunately, in this case the interface address is not known, so we cannot use a technique like the previous one to identify the sending host. The only thing we know for the host is the IP address that can be spoofed easily. Cases C-1 and C-2 (the host does not exist or is not working), can be detected by sending towards the source address of the packets that arrive, one or more special packets to make sure that the sending hosts really exists and is currently up. To select what kind of packets we ll use for testing, we must first make sure that a host that is up and running will always respond to them. Such packets can be ICMP echo requests ([6]) or TCP SYN packets. ICMP echo request packets when arriving to a host cause it to respond immediately with an ICMP echo reply packet. The reason we prefer that type of packets is that they can often travel very quickly in an internet environment and a lot of firewalls allow them to pass, which is not always happening with TCP segments encapsulated in IP datagrams. The test is simple. If in a predefined period of time the sending hosts does not respond to our test packets, then there is a high possibility for the packet to be fake. In case C-3, the previous test is not going to work because the host, which corresponds to the source IP of the packet, exists and can respond with an ICMP echo reply. In this case a port on that hosts is probably blocked by a SYN flood attack, but all other services on that host function perfectly. According to the SYN flooding attack, an attacker can only block a port that is being used by a server. That means that in order an attacker to exploit the case C-3 he must also write in the TCP header of the sending TCP segment the port of a server that has already been blocked by a SYN flooding. Such servers listen to dedicated ports, usually in the range from 1 to 1023 (also known as well known ports ), so that every application knows the port that the server listens. It is also known that no server application can ask a service from another server application, which means that a server cannot initiate a connection. There is a strategy that we can use in order to detect spoofed packets of the case C-3, that is to consider spoofed packets any SYN packets that arrive, having source TCP port less than Using the above notation we conclude to the following rule: the packet (IP1,Port1,IP2,Port2)[SYN][isn1][*] is considered spoofed if port1 is in the range [0, 1023]. In the case C-4, when a packet arrives form outside the local network, it is very hard to detect a IP spoofing attempt. Cases like this must be faced according to the nature of the attack. According to the above analysis of all the possible IP spoofing techniques, we come to the conclusion that blind attacks and SYN flooding attacks can be detected successfully in almost every case, no matter the origin of the attack (inside or outside the local network). That is true because, using spoofed IP addresses that correspond to non-existing machines, or to hosts that are temporarily down, or to hosts with

5 some ports blocked by SYN flooding, is crucial for the success of these attacks. On the other hand, man-in-the-middle attack requires special attention when the attacker is outside the local network. In that case, IP spoofing is impossible to be detected (attacker impersonates both the two hosts that already communicate). It seems unachievable, in that case, to detect the particular attack. However studies of this kind of attack ([3]) have shown that this particular attack has some side effects, including special traffic patterns that are generated in the network, which can be used in order to detect the attack. Such a side effect is the generation of a large number of ACK packets without data, as a result of the desychronization state between client and server during the attack. For example, such an attack to the telnet server generates a vast number of ACK packets compared to these of a normal session. A possible detection of this attack can be implemented using a hash table to store every connection from outside (a connection in the TCP/IP environment can uniquely be defined by the quadruple [source IP, source Port, destination IP, destination Port]). For each packet that arrives in our host from outside the local network, we can find the corresponding position in the hash table (if it is not a SYN packet) and use it to record some statistics for the specific connection. Another way to deal with this attack is to calculate the statistics directly from specific structures in the kernel; TCP protocol holds a structure in the kernel (called tcpcb) in order to gather some specific statistics about the connection. The above analysis make us optimistic concerning the capability to implement a tool, as general as possible, that can detect those kind of attacks. 4.2 Prevention Having the detection of such attacks more or less insured, we can now move a step forward towards the prevention of these attacks. We use the term prevention to denote all the necessary actions we can take, in order to stop the attack from having disastrous effects on our machine. There are two general methods to prevent an attack: 1. Alert the system administrator that the host is under attack, in order to take the necessary precautions to stop the attack. This can be implemented by using some log techniques (like syslog daemon in UNIX). However, this method is relatively slow. Most of the attacks last only a few seconds and in that period is difficult for the administrator to stop the attack in time. Moreover, that method requires the supervision of a person for 24 hours a day. 2. Provide the tool the ability to take some action when it detects an attack. This method is much faster than the previous one (order of msec) and consequently more effective. Such an action is the transmission of RST packets when the tool detects a spoofed IP packet with SYN flag set, back to the host that protects. The reset packet is going to destroy all the protocol control blocks of the specific connection attempt. Another way to prevent an attack is to find the identification number (pid) of the process that corresponds to the specific connection. When we find that pid we can kill the process and stop the attack. The tool that we implemented uses both techniques for preventing an attack. Whenever it detects an attack it sends messages to the syslogd to alert the administrator and tries to destroy the attacker s connection to the server or kills any existing connection that the attacker is trying to compromise. 5 Results The test results of our tool in the local network of our University were very encouraging. The tool we implemented was tested against most types of attacks with very promising results. To be more specific, SYN flooding attack was detected and successfully prevented, whether the attack probes came from inside or outside our network. During the test, the attacker was sending SYN packets continuously, but he could not block a service. We dealt with the blind attack very successfully using our tool, no matter the origin of the probes (inside or outside the network). Every attempt to masquerade as a non existing host and establish a connection to the services of our protected host failed. The method we used to deal with man-in-themiddle attack was successful, when attacker was in the same local network. We have not yet tested the tool against such an attack coming from outside our local network, but from the results we collected when the attack was made from the inside, we believe that such attacks can be detected using TCP statistics (numbers of ACKS, total packets etc.).

6 6 Future work A very important addition to our tool is the ability not only to detect or prevent attacks against the host it runs on, but to detect and prevent attacks against every host in the local network. That is not impossible if we keep in mind that a packet sniffer can show us the traffic in the whole local network. We simply need to implement a mechanism that will allow the host, on which the tool runs, to communicate with all the other hosts in the local network. We can approach this goal by building appropriate client-server applications. The server will run on every machine we want to protect and the client on the machine where the tool runs. When our tool detects an attack against another host, it will send a message to that host, alerting it for the incoming attack. The server on the host is then responsible to handle the attack. 7 Conclusions In this paper we presented our ideas on the implementation of a general purpose tool that it can be used to detect and prevent active attacks against the TCP/IP protocols, attacks that are based mostly on the mechanism of IP spoofing. The methods that were presented here, can be easily implemented in any local network and can provide the desirable security level against those attacks. References [1] S. M. Bellovin. Security problems in the TCP/IP protocol suite. Computer Communication Review, 19(2):32 48, April [2] Marco de Vivo, Gabriela O. de Vivo, and Germinal Isern. Internet security attacks at the basic level. Operating Systems Reviews, pages 4 15, April [3] Laurent Joncheray. Simple active attack against TCP. In Proc. 5th USENIX UNIX Security Symposium, [4] Robert T. Morris. A weakness in the 4.2BSD Unix TCP/IP software. Computing Science Technical Report 117, AT&T Bell Laboratories, [5] J. Postel. RFC 791: Internet Protocol, September [6] J. Postel. RFC 792: Internet Control Message Protocol, September [7] J. Postel. RFC 793: Transmission control protocol, September [8] W. Richard Stevens. TCP/IP Illustrated. Vol 1: The Protocols. Addison-Wesley, 1994.