An active intrusion-confronting system using fake session and honeypot

Transcription

1 An active intrusion-confronting system using fake session and honeypot Myung-Sub Lee, Chang-Hyeon Park Department of Computer Engineering Yeungnam University, #214-1, Dae-dong, Kyungsan, Kyungbuk, , Korea Myung-Chun Ryoo, Joon-Ho Park Department of Computer Engineering Kyungwoon University, #55, Sandong-myeon, Gumi, Kyungbuk, , Korea Abstract In the coming age of information warfare, information security patterns need to be changed to use an active approach using offensive security mechanisms rather than the traditional passive approach in merely protecting against intrusions. In an active security environment, it is essential that, when detecting an intrusion, it is immediately confronted with methods such as analysing the intrusion situation in real-time, protecting information from the attacks, and even tracing the intruder. This paper presents an active intrusion-confronting system using a fake session and a honeypot. Through the fake session, attacks like DoS(Denial of Service) and port scan can be intercepted. By monitoring a honeypot system, in which the intruders are migrated from the protected system and an intrusion rule manager is activated, new intrusion rules are created and activated for confronting the next intrusions. Keywords :DoS, Honeypot, Fake session, Port Scan I. INTRODUCTION In today's infrastructure of information and communication consisting of closely connected unit structures, the problem of increasingly comprehensive network structure is intensified. In this environment, outflows of important information via intrusion and attacks to the information system itself become serious problems[1]. An information systems built-in security program has basically limited functions for protecting the system from a intruder, such as, access control, identification, authentication, authorization, and the collection of basic log information. Especially, since an information system has various vulnerabilities for security, it cannot provide a complete countermeasure to various attacks internal and external. To solve this problem, a variety of security technologies that confront the purpose of attack are necessary to be added into an information system. A representative security element for monitoring illegal behavior and preventing additional damage in an information system is the IDS(Intrusion Detection System)[2, 3,4]. IDS provides a monitoring function of a system and a network, based on the prescribed rules of detecting an intruder's attacks. However, it cannot stand against a variety of attacks and certain new attacks, which are not included in the prescribed rules, and its delayed counteraction is also a problem. In order to solve these problems, the conception of an honeypot appeared on the stage, by Professor David Clock at MIT University in the middle of 1990s. Honeypot consists of a disguised server to entice an intruder and a trace software[5, 6, 7, 8]. The disguised server stages virtual information that can attract intruders, and traces an intruder with an automatic trace software that operates upon intrusion. However, honeypot does not actively go into action, but just waits for an intruder to connect to the system. Because an independent system has such a problematic characteristic, it does not function smoothly, and reveals many problems still. This paper presents an active intrusion-confronting system using a fake session and a honeypot. Fake Session sends an illegal user, a presumable intruder, to honeypot by force, and stands against DoS(Denial of Service) and scan attacks imposed upon innumerable hosts on the network. Honeypot monitors an intruder's behavior and generates a new rule against intrusion by deducing a counteraction rule from the Intrusion Rule Manager. In addition, it uses a rule-based detection model to detect a intruder efficiently. This paper converts a rule generated from IRM(Intrusion Rule Manager) into a module, so it has an advantage to be updated dynamically during the execution of a program. II. AN ACTIVE INTRUSION-CONFRONTING SYSTEM The system, which is going to be proposed in this paper, is an active intrusion confronting system, and consists of a network intrusion detection system, a host intrusion detection system, a honeypot server, and an Agent. All the systems are implemented based on Linux kernel version

2 Fig. 1 The procedural diagram of an active intrusion confronting system In fig. 1, the Host-IDS monitors the behavior of users who connect to Telnet, and then records it in files. With the recorded audit data and using user executed files, directories, and return values for users' access, it inquires whether it is an illegal behavior or not. If it detects an illegal behavior through inquiries, it reports it to Agent. Since Audit Daemon that collects the audit data of a system does not exist in Linux at the moment, its implementation is similar to the Audit package that is included in the BSM(Basic Security Module) of Solaris, Audit is divided into six classes, such as, the behavior related to process running (PC), the behavior related to reading and writing(fr, FW), the behavior related to the changes of file attribute(fm), the behavior related to IPC(Inter Process Communication) among processes(ip), and the behavior related to log-in and log-out(lilo). Their detailed events are in table 1. Class PC FR FW FM LILO IP Table 1 The event related to a class Event EXIT, CHDIR, KILL, CHROOT, SETPGRP, SETUID, SETGID, etc. OPEN, AE_READ_LINK, etc. CREAT, LINK, MKNOD, SYMLINK, RENAME, TRUCATE, etc. CHMOD, CHOWN, FCNTL, FCHOWN, FLOCK, etc. LOGIN, CRONTAB, LOGOUT, SU, PASSWD, TELNET, etc. MSGCTL, MSGGET, MSGSND, SHMGET, SHMCTL, SHMAT, etc. When Audit daemon is executed in the background process, and a user-generated process carries out System Call that is in the kernel, the function of Audit that is in the System Call records events in /proc/audit. Referring the /proc/audit, Audit Daemon leaves a log file in the end. If the data used in Host-IDS are registered in /etc/security/audit.conf file, Audit daemon leaves only the registered class event in a log file. Network-IDS analyses a packet header and detects a denial of service. If there is a modulation after analyzing a packet header, it reports to Agent. In addition, if the overall size of the SYN-flag-checked packet in TCP header and UDP packet is bigger than the standard value, or the number of packets coming in during a certain period of time is more than the standard value, it regards them as a service rejection attack. In a honeypot system, a new telnet session is made on the basis of intruder information transmitted from Host-IDS, and then it prepares to accept the packet coming in after being made Destination-NAT in Network-IDS. And it moves the job from Host-IDS to a honeypot system without being noticed by the intruder in order to get intruder information. In Agent, row data that have been collected by Audit daemon of Host-IDS are collected, and then the collected data are analyzed to extract necessary data. It transmits the extracted data to all the systems, and executes relevant commands to confronting the instruction based on the outcomes that come from a detection system. 2.1 Intrusion detection procedure This paper uses a rule-based analysis mechanism for an intrusion detection technique, and makes every rule a module in order to be able to update it dynamically during the execution of the program. The mechanism of rule configuration uses an 'if' statement. In Network-IDS, a packet header is analyzed and DoS attacks are detected. A denial of service attack is checked when the overall size of the SYN-flag-checked packet in the TCP header and UDP packet is bigger than the standard value. In the case that the number of packets coming in during a certain period of time is more than the standard value, it is regarded as a DoS attack. The port scan function checks whether or not the input packet is five within three seconds according to rules. In this paper, the number of ports is limited to such a one that is necessary to be scanned from at least the same source address(scan_count_threshold) in order to detect a scan and such an attack that happens within the time set as DELAY between ports(scan_delay_ THRESHOLD). Upon detecting an intrusion, the information of a intruder is sent to Agent using Audit data. Upon receiving messages about intrusion detection, Agent divided it into 0, 1, 2 steps according to the significance of the outcomes of intrusion, and then decides an intrusion confronting action. 2.2 Intrusion confrontation procedure In this paper, the intrusion confrontation procedure is divided into three-step scenarios (scenario-a, B, C) and Fake Session. 1)The procedure of scenario-a shows fig.2. When Network -IDS detects a port scan, it reports the information of the intruder to Agent. And it chooses one of the closed

3 random ports and induces session to the honeypot server using an address translator. After randomly assigning the 5000th port in this paper and detecting a port scan, the session for the incoming packet is induced to the honeypot server. At the time of the port scan, it blocks the packet that a kernel sends out in reaction, and instead it makes a randomly made packet sent out. By setting the flag bit of the out going packet as SYN/ACK, it disguises the port as open, when it is closed in reality. sent to the honeypot daemon to prepare for receiving the intruder. Upon completing preparation, honeypot daemon executes the Iptables command of Network-IDS, and this command is implemented to be able to change the path of a real-time data packet. After the Audit daemon of Host-IDS detected a intruder, the process of its collection of information of a intruder and sending the information to honeypot is shown in fig. 5. Fig. 2 The procedure of scenario-a 2) The scenario-b monitors the behavior of a telnet server connecting the user in Host-IDS. If a user is detected as an intruder, Audit Daemon reports it to Agent. And then scenario B or C is selected according to the managers' policy. If scenario B is chosen, a packet discard command against an intruder is sent to Network-IDS. The process of a user, who is connected to the telnet server, is terminated. If the intruder requests re-connection to the telnet server, it is connected to the honeypot server. Fig. 3 shows the procedure of scenario B. Fig. 4 The procedure of scenario-c In fig. 5, Host-IDS' Audit Daemon executes a process(catchuserinfo) to get the information of a connector, and sends a intruder's IP and port, UID, and PID information to the honeypot server. In Host-IDS, UserInfo structure is delivered to iwishtorcv daemon which is for monitoring the connector of the honeypot server. The iwishtorcv daemon synchronizes the PID of an intruder's telnet process and shell process, which are going to be made newly, based on the information of UserInfo structure, transmitted by the catchuserinfo of Host-IDS. And after becoming Destination-NAT, to make the packet of an intruder sent to the honeypot server, it executes the Iptables command of Network-IDS. Besides, upon executing the makesession process, it synchronizes the information of an intruder with the information of Host-IDS. Fig. 3 The procedure of scenario-b 3) The scenario-c is the most crucial one in this paper, and has the mechanism in which a intruder is moved unawares to the honeypot server. Fig. 4 shows that the transfer of a telnet server connecting the intruder to the honeypot is executed while retaining the connection. If an illegal user is detected, the detected information of the user is Fig. 5 The move procedure of session

4 4) The Fake Session monitors the port scan of an intruder, and gives incorrect answering for the packet that is sent at the time of port scan, making the port scanning action of the intruder useless. As for the packet that comes to a designated port of Network-IDS from the port scan, among the packets sent by the kernel, of which flag bit is RST and ACK go into the discard. And instead, the packet made by a packet generator goes out. Although the flag bit of outgoing packets is set as SYN/ACK and a closed port, it is disguised as open. And connecting a certain port to honeypot server, it provides an intrusion confronting mechanism that is equivalent to scenario A. Table 2 shows the structure of Fake TCP header in Network-IDS. Table 2 The Fake TCP header TCP-RST TCP-SYN source=htons(sport) source=htons(sport) dest=htons(dport) dest=htons(dport) seq=htonl(seq) seq=htonl(seq) ask_seq=htonl(ask) ask_seq=htonl(ask) res1=0 rea1=0 doff=5 doff=th_offset window=htons(0) window=htons(tcp_window_size) fin=0 fin=0 syn=0 syn=1 ask=1 ask=1 rst=1 rst=0 III. EXPERIMENT In order to test the performance of Fake Session and the three scenarios (scenario A, B, C) proposed as a mechanism of active intrusion confronting in this paper, a simulation was executed. Scenario-A is a confronting mechanism against a scan attack and a DoS attack in Network-IDS. This clause presents the outcome of simulation against a DoS attack. The outcome of simulation against a scan attack will be presented. that after the SYN flooding attack is detected, the SYN packet, which comes from an intruder's IP, is discarded. Fig. 7 The screen of discarded SYN packet Scenario B, when a user, who is connecting to the telnet server, is detected as an intruder, Audit daemon sends the information of the intruder to Agent. If the intruder is not an internal user, scenario B is executed. Upon choosing scenario B, the packet discard command is sent to Network-IDS. In addition, the process of the user, who is connecting to the telnet server, is terminated. If the intruder requests re-connection to the telnet server, it is connected to the honeypot server. Fig. 8 is the executing screen of Audit daemon. Fig. 6 The screen of SYN flooding attack Fig. 6 shows that the SYN flooding attack packet in Network-IDS is captured using TcpDump[9]. Fig. 7 shows Fig. 8 The executing screen of Audit daemon Scenario C is a mechanism to migrate an intruder to the honeypot server unawares, when a user of the internal server of telnet is detected as an intruder. If an intruder, who is connecting to the telnet server, is detected as an illegal user,

5 the user information of the intruder is sent to the daemon of the honeypot server, in order for the honeypot server to prepare to receive the intruder. Fig. 9 shows the flow of packets at the time of executing scenario C. It shows that a new session is made by the makesession process of honeypot, and that the packet of which path is reset from telnet server to honeypot server by the tofirewall process. Fig. 9 The flow of packet at the time of executing the scenario-c The Fake Session makes the port scan of an intruder useless. For simulation, this paper used Nmap, a port scan tool[10]. Fig. 10 shows the outcome of port scan without Fake Session policy, while fig. 11 shows the outcome of port scan with the application of Fake Session policy. Fig. 10 The outcome of port scan without Fake Session policy Fig. 11 The outcome of port scan with Fake Session policy IV. CONCLUSION This paper implemented a honeypot system, an intrusion confronting system. The inducing mechanism of an intruder is as follows; First, when an intruder's request of connecting to telnet is detected in Network-IDS, it connects the intruder to the honeypot server, instead of the telnet server. Second, it disconnects an illegal user's connection, which is detected in Host-IDS, and connects the illegal user directly to the honeypot server, when re-connection is requested. Third, in the same situation as the second, it does not disconnect the illegal user' connection, only connects the illegal user to the honeypot server, while maintaining the initial connection. In addition, this paper implemented Fake Session, with which the scan attack that imposes upon numerous hosts on the network, for the present, can be made useless. There are some advantages in this system: First, realizing the forcible transfer of users, it can get sufficient knowledge on the intrusion contents of a intruder, and protects the information of a server safely. Second, by monitoring the behavior of an intruder, who is transferred to a honeypot server, it can detect new patterns of attacks and add new rules easily. Third, the vulnerability of a host is not easily exposed by this. REFERENCES [1] Crosbie M, Spafford E, "Applying Genetic Programming to Intrusion Detection", Technical Report, Purdue University, Department of Computer Science, [2] Stephen Northcutt, "Network Intrusion Detection-Third Edition", New Riders, September [3] William R. Cheswick, Steven M. Bellovin, "Firewalls and Internet Security", ISBN: , [4] W. Lee and S. J. Stolfo, "Data mining approaches for intrusion detection.", In Processing of the 7th USENIX Security Symposium, San Antonio, TX, January [5] Lance Spitzner, "Honeypots-Tracking Hackers", Addison-Wesley, October [6] Honeynet Project Members, "Know Your Enemy : Honeynets", September [7] Crosbie M, Spafford E, "Applying Genetic Programming to Intrusion Detection", Technical Report, Purdue University, Department of Computer Science, [8] CISCO, "NetRanger Intrusion Detection System", Technical Information, April, [9] TcpDump, [10] Nmap,