SYN Flood Attack Protection Technology White Paper

Transcription

1 Flood Attack Protection Technology White Paper Flood Attack Protection Technology White Paper Keywords: flood, Cookie, Safe Reset Abstract: This document describes the technologies and measures provided by H3C firewalls to protect servers against the flood attack, and the application restrictions of each protection measure. Acronyms: Acronym Full spelling DMZ TCB TCP De-Militarized Zone Transmission Control Block Transmission Control Protocol Hangzhou H3C Technologies Co., Ltd. 1/18

2 Flood Attack Protection Technology White Paper Table of Contents 1 Overview Background Benefits Attack Protection Implementation Flood Attack Inspection Technologies Half-Open Connections Inspection Connection Establishment Rate Inspection Flood Attack Protection Measures Blocking New Connection Requests Releasing Invalid Connections Cookie Safe Reset Application Scenarios Cookie Application Safe Reset Application Integrated Application of Flood Attack Protection Technologies Hangzhou H3C Technologies Co., Ltd. 2/18

3 Flood Attack Protection Technology White Paper 1 Overview 1.1 Background A flood attack sends a great number of packets to a target server to exhaust the server s half-open connection resources, resulting in denial of normal services. Typically, flood attacks use spoofed IP addresses and exploit the three-way handshake process for TCP connection establishment. When a client attempts to establish a TCP connection with a server, the client and server go through a three-way handshake process: (1) The client sends a packet to the server. (2) In response to the packet, the server replies with a / packet. (3) After the client receives the / packet, it sends an packet back to the server for final acknowledgment. A TCP connection can be established only if the three-way handshake succeeds, that is, the server receives the packet. Figure 1 illustrates the setup process of a TCP connection: Figure 1 Set up a TCP connection During the handshake process, after receiving the packet, the server needs to allocate a transmission control block (TCB) for the incomplete connection, which is called a half-open connection. The TCB allocated is released only when the server receives the from the client or the half-open connection times out. The client, on Hangzhou H3C Technologies Co., Ltd. 3/18

4 Flood Attack Protection Technology White Paper the contrary, allocates a TCB only after it receives the / packet. This resource allocation mechanism may be exploited by an attacker to launch a flood attack. Figure 2 Diagram for the flood attack As shown in Figure 2, the flood attacker initiates connections to the server by sending packets with a forged source IP address, such as a nonexistent address. The server replies to each connection request with a / packet but will never receive the expected packet because the destination address in the / packet is unreachable. Thus, the connections will stay in the half-open state until they are aged out. As long as the flood attacker sends such packets to the target server at a speed higher than that at which half-open TCP connections get aged out, the server will soon run out of TCB resources and be not able to service normal connection requests. To ensure that a server can provide TCP-based services normally, a firewall must be able to defend against flood attacks. 1.2 Benefits The H3C flood attack protection feature has four key benefits: 1. Security zone-based configuration The H3C flood attack protection feature supports security zone-based configuration. All attack inspection policies are configured on security zones. Therefore, the policy configuration of a firewall is simple and flexible. This not only Hangzhou H3C Technologies Co., Ltd. 4/18

5 Flood Attack Protection Technology White Paper reduces the configuration workload, but also satisfies the requirements of differentiated policies for different security zones. 2. Abundant alarm log information The H3C flood attack protection feature supports providing abundant alarm log information, which can be used by third party software. The log function and the audit function allow administrators to monitor attacks in real time, query and analyze attack history records, facilitating the trace of attack events. 3. Precise blocking of attack traffic The H3C flood protection feature uses an abnormality detection algorithm based on behavior patterns to check network traffic of the server to be protected. By monitoring the state machine negotiation process of TCP connections in real time, this feature can distinguish attack traffic from normal traffic and thereby block attack traffic precisely. 4. Flexible attack protection measures The H3C flood attack protection feature provides flexible measures for preventing flood attacks. You can configure the feature to log the attacks, drop the attack packets, enable the Cookie function, enable the Safe Reset function, or/and instruct the server to release the invalid half-open connections. 2 Attack Protection Implementation On the Internet, public servers are main targets of the flood attack. When any client on the Internet initiates a connection request to a public server, the destination IP address in the request must be a public IP address. The H3C firewall uses the attack detection technology to collect and analyze statistics on connections to public servers, so as to detect and identify attack packets. After detecting a flood attack aiming at a server, the H3C firewall can take multiple attack protection measures to defend against the attack. These measures fall into two categories: Hangzhou H3C Technologies Co., Ltd. 5/18

6 Flood Attack Protection Technology White Paper Connection limit: Uses the flood attack inspection technology to monitor the number of TCP half-open connections and the TCP connection establishment rate in real time. If the specified thresholds are reached or exceeded, it is considered that an attack is in process, and measures will be taken to block new connection requests or release invalid half-open connections. Connection proxy: Uses the Cookie technology or Safe Reset technology as the proxy for TCP connections. Such technologies can find attack packets through precise verification, and thus can filter malicious connection requests to the server without affecting normal services. The connection proxy technology can be configured not only for servers that have been attacked but also for servers that may be attacked. Thus, a firewall configured with such a technology can be used as a proxy for all traffic of the protected server, so as to prevent flood attacks from happening. 2.1 Flood Attack Inspection Technologies The flood attack inspection can be half-open connections inspection or connection establishment rate inspection, depending on the statistics objects Half-Open Connections Inspection 1. Principle If a client uses a forged source IP address to initiate the flood attack to a target server, there will be a large number of half-open connections on the target server. Such half-open connections will never become established, while normal half-open connections will become established when the three-way handshake processes are finished. To distinguish abnormal half-open connections from normal half-open connections, a firewall needs to record the number of all half-open connections from clients to the server and the number of half-open connections that become established. The difference between the two counts is the number of half-open connections that cannot become established. For a server not under attack, this number is within a relatively constant range. If this number increases sharply in a short time or even approaches the upper limit of the TCB resources, the server may be attacked by abnormal traffic. Hangzhou H3C Technologies Co., Ltd. 6/18

7 Flood Attack Protection Technology White Paper Client Firewall Server / / / / / / Half-open connections threshold Server under attack Figure 3 Diagram for half-open connections inspection As shown in Figure 3, you can set the threshold of the number of half-open connections according to the processing capability of the protected server. If the server cannot process connection requests from clients in time and the number of unfinished half-open connections exceeds the specified threshold, the firewall can determine that the server is under the flood attack. 2. Application restrictions The half-open connections statistics requires that the firewall can record the status of all connections from clients to a server. In other words, all packets from clients to a server need to be processed by the firewall. Therefore, to use the half-open connections inspection, you need to deploy the firewall in the key path of the ingress and egress of the server to be protected Connection Establishment Rate Inspection 1. Principle If a client initiates the flood attack to a target server, no matter the client uses a forged source IP address or its real source IP address, there will be a large number of packets destined for the target server in a short time. Among packets from the attacker to the server, some are connection requests and some are data packets of the established connections. An H3C firewall can count the number of connections established in a second, and then compare the number with Hangzhou H3C Technologies Co., Ltd. 7/18

8 Flood Attack Protection Technology White Paper the specified threshold. If the number reaches or exceeds the threshold, the firewall will conclude that the server is under the flood attack. Client Firewall Server / / / / Connection establishment rate threshold Server under attack Figure 4 Diagram for connection establishment rate inspection As shown in Figure 4, during the inspection on the server, the firewall counts the number of new connections initiated by the client to the server in a second, and regards the number as the current connection establishment rate. If the rate reaches or exceeds the specified threshold, the firewall concludes that the server is under the flood attack. 2. Application restrictions The connection establishment rate inspection requires that a firewall can count the number of connections from a client to the server. In other words, all packets from the client and the server need to be processed by the firewall. Therefore, to use the connection establishment rate inspection function, you need to deploy the firewall in the key path of the ingress of the server to be protected. 2.2 Flood Attack Protection Measures The H3C firewall supports the following measures for defending a server against the flood attack: Hangzhou H3C Technologies Co., Ltd. 8/18

9 Flood Attack Protection Technology White Paper Blocking new connection requests: The firewall can block the connection requests that are beyond the processing capability of the server to alleviate the suffer of the server from the attack. Releasing invalid connections: After detecting the existence of the flood attack, the firewall can inform the server to release the invalid connections to help the server restore normal service capability. Cookie and Safe Reset: The firewall checks the validity of the clients that initiate connections to the server, so as to protect the server away from the attack. Cookie and Safe Reset are applicable to different networking environments Blocking New Connection Requests 1. Principle The easiest way to defend against the flood attack is to temporarily block the connection requests from all clients to the server. A firewall will start to block new connection requests once it detects that the number of half-open connections or the connection establishment rate reaches or exceeds the specified threshold. Refer to Flood Attack Inspection Technologies for details about the flood attack inspection technologies. The following describes the blocking processes respectively when the attack is detected through the two inspection technologies. Blocking process based on half-open connections inspection As shown in Figure 5, after the firewall detects that the number of current half-open connections between the client and the server reaches or exceeds the specified threshold, the firewall drops all subsequent connection requests. New connections are allowed until the server finishes processing all the current half-open connections or the number of half-open connections drops below the threshold. Hangzhou H3C Technologies Co., Ltd. 9/18

10 Flood Attack Protection Technology White Paper Client Firewall Server / / / / / / Half-open connections threshold Drop packets Figure 5 Blocking process based on half-open connections inspection Blocking process based on connection establishment rate inspection As shown in Figure 6, the firewall keeps track of the connection establishment rate, and allows new connections to be established only when the connection establishment rate is under the specified threshold. If the rate reaches or exceeds the threshold, the firewall will drop the subsequent connection requests. New connections are allowed when the rate drops below the lower threshold. Client Firewall Server / / / / Connection establishment rate threshold Drop packets Figure 6 Blocking process based on connection establishment rate inspection 2. Application restrictions Blocking connection requests is a basic measure taken by the firewall after it detects the flood attack. When a server is under the flood attack, the firewall can Hangzhou H3C Technologies Co., Ltd. 10/18

11 Flood Attack Protection Technology White Paper blocking subsequent connection requests before the server processes them. This can weaken the affection of the attack on the server, but cannot improve the server s service capability when the server is under attack. Therefore, this function is used in cooperation of the flood attack inspection to prevent the server from suffering from intensive attack packets in a short time Releasing Invalid Connections 1. Principle As described in section Half-Open Connections Inspection, when there are too many half-open connections on a server, among the half-open connections being processed by the server, there are probably invalid ones initiated by an attacker client using forged source IP addresses. To prevent such invalid connections from taking up the server resources, the firewall needs to identify the invalid ones among all the half-open connections and notify the server to release these invalid connections. The firewall notifies the server to release invalid connections by sending RST packets that contain the five-tuple information of the invalid connections, including the source IP address, destination IP address, source port number, destination port number, and protocol type. 2. Application restrictions Releasing invalid connections is a basic measure taken by the firewall after it detects the flood attack. When the server is under the flood attack, this measure can be used to notify the server to release the resources occupied by the attack packets, so as to help the server resume normal operation faster. However, this measure cannot stop the attack. Therefore, this measure is usually used in cooperation of the flood attack inspection function to minimize the attack s affection on the server Cookie 1. Principle Cookie borrows the concept of cookie from HTTP. Using the Cookie technology, the firewall processes the TCP connection request, adds a cookie Hangzhou H3C Technologies Co., Ltd. 11/18

12 Flood Attack Protection Technology White Paper (authentication information for the client) in the /, and then checks the validity of the returned by using the cookie. As shown in Figure 7, using Cookie, the firewall acts as a proxy between the client and the server. Client Firewall Server 1) 2) / (Cookie) 3) Data (x) Data (y) Allocate TCB resources Request verified legal 4) 5) / 6) Proxy for subsequent packets 7) Data (x) Data (y) Figure 7 Prevent the flood attack using Cookie The process is as follows: (1) The client sends a packet to the server. (2) The firewall intercepts the packet and replies with a / packet on behalf of the server. The / packet contains a serial number, which is the cookie calculated by the firewall. The cookie is the encryption result of the encryption index and the client information of the connection, such as the IP address and port number. (3) After the client receives the /, it sends an back to the server for final acknowledgment. The firewall intercepts the packet, uses the encryption index and the client information to calculate the cookie again, and then compares the calculation result with the serial number of the packet. If they are the same, the firewall considers that the connection request is from a valid client. If they are not the same, or the firewall does not receive the expected packet at all, the firewall considers the client invalid, and will drop all subsequent packets from the invalid client without allocating TCB resources for them. (4) If the client is valid, the firewall will send a packet on behalf of the client to the server and, at the same time, allocate a TCB to record the description Hangzhou H3C Technologies Co., Ltd. 12/18

13 Flood Attack Protection Technology White Paper information of the connection. The TCB records the connection request sent from the firewall to the server as well as that sent from the client to the server. (5) The server replies to the firewall with a / packet. (6) After receiving the / packet, the firewall uses the existing connection description to construct an and sends the packet back to the server on behalf of the client for final acknowledgment. (7) After the above steps, two connections are established, one between the client and firewall, and the other between the firewall and the server. All the subsequent data packets between the client and the server will be forwarded by the firewall. The Cookie technology uses the authentication information carried in the / packet to authenticate the packet, so as to prevent the firewall from allocating TCB resources too early. As a result, malicious packets can lead to TCB resource exhausting on neither the server nor the firewall, and the flood attack is prevented effectively. In the flood attack prevention process, the firewall acts as a virtual server to interact with the client and the server respectively, filtering malicious connection requests to the server without affecting normal services. 2. Application restrictions As Cookie requires that the firewall act as a proxy for all packets between the client and server, the firewall must be deployed in the key path of the ingress and egress of the server to be protected Safe Reset 1. Principle Using Safe Reset, the firewall can interfere with the normal TCP connection establishment process to check the validity of a client. Figure 8 illustrates the implementation of Safe Reset. The firewall processes the TCP connection request packet (), modifies the packet serial number and adds the authentication information (called Cookie) in the /, and then checks the authentication information carried in returned packet to verify the client validity. During the process, the firewall forwards the packets of valid clients and drops the connection requests from the forged and malicious clients. Thus, the server will not Hangzhou H3C Technologies Co., Ltd. 13/18

14 Flood Attack Protection Technology White Paper allocate connection resources to such clients. In this way, the flood attack is prevented. Client Firewall Server 1) 2) / (Cookie) 3) RST Allocate TCB resources Packet verified legal Permit subsequent packets 4) / Figure 8 Prevent the flood attack using Safe Reset The process is as follows: (1) The client sends a packet to the server. (2) The firewall intercepts the packet and replies with a / packet on behalf of the server. The / packet contains an serial number that is not that expected by the client as well as a cookie. The cookie is the encryption result of the encryption index and the client information, such as the IP address and port number. (3) As required, the client replies to the server with an RST packet. The firewall intercepts the RST packet and uses the serial number in the packet for cookie authentication. If they are matched, the firewall considers the connection is valid and allocates a TCB to record the description information of the connection. If not, the firewall considers the connection invalid and drops all subsequent packets of the connection. (4) After the above process, the client reconnects to the server, and the firewall will check the validity of the connection request based on records of the existing connections and forward all valid packets. As shown in the above process, a firewall using Safe Reset authenticates a client by checking only its first connection request to the server, and normal services will not be affected even if the packets replied by the server to the client do not pass through the firewall. Therefore, Safe Reset is also referred to as unidirectional proxy. Normally, an application server will not initiate malicious connections to clients. Hangzhou H3C Technologies Co., Ltd. 14/18

15 Flood Attack Protection Technology White Paper Therefore, the firewall does not need to check the packets replied by the server to the client. Safe Reset only requires that the firewall monitor the packets from clients to the server in real time. For packets from the server to clients, you can determine whether to configure them to pass through the firewall as needed. Therefore, Safe Reset is more flexible for networking. 2. Application restrictions Safe Reset requires you deploy the firewall in the key path of the ingress of the protected server to check packets from all clients to the server. It does not require you configure the firewall to check packets from the server to clients. Therefore Safe Reset is more flexible than Cookie for networking. Because Safe Reset interferes with the TCP connection initiated by the client, it requires that the client comply with the standard TCP stack provisions. If a client does not comply with the standard TCP stack provisions, the client will not be able to connect to the server even if it is a valid client, because it cannot pass the verification of the firewall. A client will reconnect to a server after it sends an RST packet to the server. The Safe Reset technology is implemented based on this feature. Therefore, with the Safe Reset implemented, the time for a client to establish a connection with the protected server is longer than that a normal TCP connection process takes. As for how much longer the connection process will take, it is up to the TCP implementation on the client. Hangzhou H3C Technologies Co., Ltd. 15/18

16 Flood Attack Protection Technology White Paper 3 Application Scenarios 3.1 Cookie Application Attacker Application servers Cookie Internet Normal client Figure 9 Network diagram for Cookie application As shown in Figure 9, on the Internet, there are normal clients as well as attackers. Packets from the Internet that are destined for the application servers, as well as packets replied by the application servers are all processed by the firewall. With Cookie configured, the firewall will forward the TCP connection negotiation packets between the normal clients and the application servers after they pass the Cookie verification. Packets from attackers, on the contrary, cannot pass the Cookie verification and therefore will be dropped by the firewall. In this way, the application servers are protected. 3.2 Safe Reset Application Attacker Application servers Safe Reset Internet Normal client Figure 10 Network diagram for Safe Reset application Hangzhou H3C Technologies Co., Ltd. 16/18

17 Flood Attack Protection Technology White Paper As shown in Figure 10, on the Internet, there are normal clients as well as attackers. Packets from the Internet that are destined for the application servers are processed by the firewall, while the packets replied by the servers do not necessarily pass through the firewall. With Safe Reset configured, after the clients pass the Safe Reset verification, the firewall will forward the subsequent TCP connection request packets from them to the application servers. On the contrary, packets from attackers cannot pass the Safe Reset verification and therefore will be dropped by the firewall. In this way, the application servers are protected. 3.3 Integrated Application of Flood Attack Protection Technologies Figure 11 Network diagram for integrated application of flood attack protection technologies As shown in Figure 11, the internal network is in the Trust zone, the internal server is in the DMZ zone, and the external network is in the Untrust zone. Would-be attackers reside in the Untrust zone. Configure a security policy on the firewall to perform flood attack inspection for the server in the DMZ zone. Based on the actual traffic of the server, set the maximum connection establishment rate and the maximum number of half-open connections allowed on the server. If the server is under the flood attack, the Hangzhou H3C Technologies Co., Ltd. 17/18

18 Flood Attack Protection Technology White Paper firewall will output a flood attack log and, according to your configuration, act as a unidirectional or bidirectional TCP proxy to ensure that only normal TCP connection requests reach the server. Copyright 2008 Hangzhou H3C Technologies Co., Ltd. All rights reserved. No part of this manual may be reproduced or transmitted in any form or by any means without prior written consent of Hangzhou H3C Technologies Co., Ltd. The information in this document is subject to change without notice. Hangzhou H3C Technologies Co., Ltd. 18/18