Performance Analysis of IPsec Gateway

Transcription

1 VOL. 3, NO., APRIL Performance Analysis of IPsec Gateway Adam Tisovský, Ivan Baroňák Fakulta elektrotechniky a informatiky STU v Bratislave {tisovsky,baronak}@ut.fei.stuba.sk Abstract The paper outlines specific features of IPsec gateway performance related to the rate of offered traffic. It points out the role of hardware crypto accelerator in the security gateway and advises of pitfall when no hardware acceleration is implemented. It is essential to consider these effects when testing IPsec performance in order to obtain precise and reliable results. We propose recommendations and methodology for testing the performance of IPsec gateway. The evaluation is based on the values of IPsec forwarding rate and gateway s CPU utilization. Introduction Knowing information about the IPsec performance on a network device is important for both vendors and customers. It is often complicated to analytically calculate or to predict an actual performance of IPsec gateway for certain nature of network traffic. One of reliable methods how to obtain this information is to measure it on the devices either in-situ or in laboratory. We need results obtained in a confident, simple and mostly as-fast-as-possible way. To perform effective measurement with avoiding mistakes we need to know all specific phenomena related to the IPsec performance on a security gateway as well as to be equipped by a wise measuring tool. From our previous research [] we know that the capacity of IPsec processing expressed either in bits per second or in packets per second is not constant and not linear over the range of packet sizes. This introduces many difficulties into modelling the network parameters, e.g. traffic dimensioning, QoS policy, etc. Therefore there is higher demand for measuring the IPsec performance and also for researching the possibilities of performance modelling []. Another significant fact is that there are considerable differences in performance when IPsec is processed exclusively by the main CPU (i.e. cryptographic algorithm is executed on a software basis by the same processor responsible for switching packets, routing, resource management, etc.) and when the hardware crypto accelerator is employed to offload the main CPU. We can summarize these differences in two main statements: Performance with hardware crypto accelerator is much higher than without crypto accelerator. This apparent fact, however, covers other consequences there is an omissible difference in performance between all cryptographic algorithms, it is rather dependant on a security protocol (AH or ESP). The performance over the range of packet sizes is closer to a constant value of packets per second, yet still is not constant. Without hardware crypto accelerator IPsec forwarding rate decreases when the offered load exceeds the IPsec processing capacity (device is overloaded). In this paper we deal mainly with implications related to the second statement. We focus on how are the forwarding rate and CPU utilization of IPsec process dependant on the amount of offered load, with or without hardware crypto accelerator. Terminology In this paper we use terminology for device performance testing introduced mainly in RFC [], RFC 85 [3] and IETF draft Terminology for Benchmarking IPsec Devices []. These terms are: (offered traffic rate) the rate at which device under test receives the frames at a specified interface. [3] Forwarding rate the rate at which a device can be observed to successfully transmit to the correct destination interface in response to a specified offered load. It makes no explicit reference to frame loss. [3] Forward pressure situation in which overloaded device violates normal procedures in order to increase forwarding rate. [3] Throughput the maximum rate at which none of the offered frames are dropped by the device. [, ] Maximum forwarding rate (MFR) the highest forwarding rate of a device taken from an iterative set of forwarding rate measurements [3]. MFR is often recorded when the device is overloaded. Besides the terms throughput and maximum forwarding rate we propose to employ another term: Maximum forwarding rate same as offered load (MFRSOL) the highest forwarding rate of a device that is the same as offered load. According to our observations MFRSOL is more suitable to universally describe the performance of a device than throughput. The reason is that the throughput as described in RFC may be dependent on duration of the test. It varies between values of MFRSOL and MFR. As presented in this paper, a buffer preceding IPsec process may cause that no 39

2 VOL. 3, NO., APRIL packet is lost in a test of finite duration, despite an offered load is higher than forwarding rate. Due to buffer, forwarding pressure and interrupt coalescing in the device [5] throughput may be temporarily higher than MFRSOL. The device is, however, overloaded and the state of no packet loss is not infinitely sustainable. Conclusion is that the value of throughput is affected by many factors, as duration of the test, buffer length, interrupt-coalescing block size, ability of device to handle forward pressure, and therefore is not as universal as MFRSOL. However, all of aforementioned terms and their related methodologies of measuring may be interesting for specified demands and conditions in testing. 3 Experiment Environment We evaluated IPsec performance on Cisco 8 Integrated Services Router with integrated hardware crypto accelerator. Experiment environment was set as depicted in Fig.. Two Cisco 8 routers acted as security gateways. All device connections were made using Mbps Fastethernet links to ensure that capacity bottleneck is IPsec processing on the router, not the link interface. As a traffic generator and measurement tool was used Iperf.7. with Jperf.. GUI which was installed on two endpoints with operating system Linux Ubuntu version. one acting as a client (traffic transmitter), the second acting as a server (traffic receiver). UDP was used as a transport protocol because TCP employs flow and error control unneeded for our experiment. Input and output values in Iperf are related to the payload of Layer of OSI model (excluding L header), thus all the rates and sizes in this paper are related to UDP datagram payloads, too. Default settings in our scenario were these: 5 bytes UDP payload datagram (58 bytes IP packet) constant packet inter-departure time IPsec in tunnel mode ESP protocol for encryption 3DES encryption algorithm 3 seconds duration of the tests Fig.. Scheme of experiment environment Forwarding Rate. Hardware crypto accelerator enabled First we can take a look at a forwarding rate of IPsec process related to an offered load when the hardware crypto accelerator is enabled. In Table I are noted values for security combination ESP-3des and flow of 5 byte UDP payload datagrams. These values are plotted in Fig.. In this case forwarding rate is the same as offered load until it reaches MFRSOL at 7 Mbps. Until this point no packet is lost. Further increasing of offered load causes forward pressure on the router and in the effort to relieve the saturated queue router temporarily inhibit certain processes to increase the forwarding rate [3]. A number of packets are now being discarded as well. At offered load 9 Mbps we reach the MFR, which is 7.95 Mbps. When the rate of offered load overruns MFR the forwarding rate decreases very slightly (minus.6%). With a further increase of offered load it persists at the same level, however. This means that with omissible error we can measure MFR easily by setting offered load to any higher value than is the actual (expected) MFR. IPsec Forwarding Rate TABLE I ESP-3DES PERFORMANCE FOR 5 BYTES UDP DATAGRAM; HARDWARE ENCRYPTION Forwarding rate Packet loss Maximum forwarding rate same as offered load (MFRSOL) and also throughput Maximum forwarding rate (MFR) IPsec Forwarding Rate Offered Load Packet Loss Fig.. ESP-3des performance for 5 bytes UDP datagram; hardware encryption. Hardware crypto accelerator disabled If we disable hardware crypto accelerator, behaviour of forwarding rate will be different. The dependency of forwarding rate and a packet loss on the offered load is shown in Table II and plotted in Fig Packet Loss

3 VOL. 3, NO., APRIL First difference is that at a certain interval of offered load (3. Mbps to 3.58 Mbps in this example) forwarding rate is lower than the offered load, but no packet is lost. From the results we suppose that the reason is in buffer preceding the encryption process. Default duration of our measurements was set to 3 seconds, but when we increased this period, we experienced the packet loss. The less was difference between offered load and forwarding rate (i.e. the offered load exceeds current forwarding rate only slightly), the more time it took to record packet loss. And vice versa, if the offered load exceeds forwarding rate greatly, the packet loss occurs in a short time. The buffer, according to our observations, could be in this example quite large tens of kilobytes. In fact, analyzing the time when packet loss begins can be one of the methods how to estimate the buffer length when this information is not publicly available as was the case of our device as well. IPsec Forwarding Rate 3,5 3,5,5,5 TABLE II ESP-3DES PERFORMANCE FOR 5 BYTES UDP DATAGRAM; SOFTWARE ENCRYPTION Forwarding rate Packet loss Maximum forwarding rate that same as offered load (MFRSOL) Maximum forwarding rate (MFR) and also throughput IPsec Forwarding Rate 5 5 Offered Load Packet Loss Fig. 3. ESP-3des performance for 5 bytes UDP datagram; software encryption The second difference is that after reaching maximal forwarding rate (3.76 Mbps in this example), with further increase of offered load the forwarding rate decreases. This Packet Loss behaviour may be very risky the performance of security gateway might be decreased significantly if the amount of IPsec traffic is not designed correctly. Besides, we can obtain wrong numbers of device performance when we employ incorrect methodology of measurement. The reason of described behaviour is discussed below. 5 CPU utilization To reveal the reason of mentioned IPsec forwarding rate behaviour we take a look at CPU utilization. Cisco IOS provides this information using command show processes cpu which covers utilization of last 5 seconds, minute and 5 minutes. The utilization is split between operations on an interrupt level and on a process level. First we analyze CPU utilization of the gateway with hardware accelerator enabled. 5. Hardware crypto accelerator enabled Looking at CPU utilization we find out that with enabled hardware crypto accelerator the IPsec performance is determined only by amount of interrupts of the CPU. On a process level there is no CPU utilization by the cryptographic operations. As a result, whole CPU utilization, let be maximum 98%, consists of 96% CPU utilization on interrupt level and % on process level. This minor utilization on process level is the sum of common CPU not-intensive processes like Pool Manager, CDP Protocol, Per-Minute Jobs, DHCPD receive, Check Heaps, Load Meter, etc. The reason of high CPU utilization by interrupts during IPsec processing is that some device architectures do not allow processing of IPsec packet exclusively by the hardware crypto accelerator, but requires CPU cooperation to handle the packet. The role of CPU is mainly to control a packet passing through the device and to maintain software operations related to IPsec protocol (protocol stack, protocol driver, API, etc.) []. In Table III are listed values of CPU utilization for different offered loads for security combination ESP-3des and 5 byte UDP payload packet. For comparison, CPU utilization for plaintext (non-ipsec) traffic is % at Mbps. When plotting data from Table III into graph (Fig. ) we find out that the trend of CPU utilization is not linear over the range of offered load. The linearity appears only for offered load lower than about.3 Mbps, what is 58% of maximal measured forwarding rate at this point the CPU utilization is 88%. With further increasing of offered load the utilization rises very slowly and finally reaches 98% at maximal measured forwarding rate 7.75 Mbps. The interesting thing is that this CPU utilization behaviour does not affect forwarding rate at all it is strictly linear over the whole range of offered load (until it reaches maximal forwarding rate). This behaviour is not abnormal, however, as Cisco reports non-linear CPU utilization also for the plaintext packet switching for some of their devices []. For further understanding it is important to remind that maximal forwarding rate is achieved at almost full utilization of CPU by interrupts. As a result, forwarding rate does not decrease when the offered load exceeds maximal measured forwarding rate (as shown in Fig. ). This is in contrast with

4 VOL. 3, NO., APRIL CPU Utilization the case of crypto accelerator disabled (as was shown in Fig. 3 and also will be explained below). As was mentioned in the introduction, the capacity of IPsec processing is dependent on the packet length. For this reason we measured utilization for various packet lengths and verified that mentioned behaviour of CPU utilization is the same for all packet lengths. The trend keeps linearity up to circa 88% of CPU utilization. This corresponds to a forwarding rate of about 58% of MFR in all cases. Moreover, when we look at the forwarding rate that induces 5% CPU utilization we see that it equals around 3% of MFR for all packet lengths. Precise values are listed in Table IV. This assumption might help us to estimate performance for different packet lengths: either MFR from known CPU utilization at known offered load or vice versa CPU utilization for certain offered load from known MFR TABLE III CPU UTILIZATION FOR 5 BYTES UDP DATAGRAM, ESP-3DES; HARDWARE ENCRYPTION Total CPU utilization Interrupt utilization Total CPU Utilization Interrupt utilization Forwarding rate Offfered Load Fig.. CPU utilization at 5 bytes UDP datagram; hardware encryption Forwarding Rate UDP payload length [bytes] TABLE IV FORWARDING RATE AT 5% CPU UTILIZATION Maximum Forwarding Rate (MFR) Forwarding rate (FR) at 5% CPU utilization Ratio of FR / MFR % % % 5. Hardware crypto accelerator disabled Total CPU utilization of IPsec processing without hardware crypto accelerator consists of two main components CPU interrupts and encryption process. In Table V are listed values for ESP-3des encryption at 5 bytes UDP payload datagram. These values are plotted in Fig. 5 as well. Until maximum forwarding rate is reached, both main components of total CPU utilization (light orange) CPU interrupts (orange) and encryption process (dark orange) are directly proportional to the flow offered load. Interesting to note is that interrupt utilization keeps linear trend over the whole inspected range unlike the case with hardware crypto accelerator enabled. Maximum forwarding rate of IPsec processing is achieved at total CPU utilization of 99%, where 8.5% is consumed by the encryption process, 8% by the interrupts and the rest by minor processes. If we continue increasing offered load beyond this point, more CPU resources will be consumed by interrupts and less will be available for the encryption process itself. This leads into decrease of forwarding rate, as it was shown in Fig. 3, and makes measuring the capacity more difficult than when hardware crypto accelerator is enabled. There are two possible methods of measuring the capacity. First, estimate MFR by extrapolating the linear relation between CPU utilization and forwarding rate. Because the relation is linear and the line begins in zero, one measuring is needed, as depicted in Fig. 6. This method appears to be easy and convenient, but since extrapolation amplifies an error, we must be able to measure both CPU utilization and forwarding rate with enough precision. Our testing device provides values of CPU utilization rounded to integer only what could produce an error of several percent in a result. Inaccuracy may be lowered when measured point falls within high utilization. It is also possible to make several measurements with different offered loads to obtain more known points in extrapolation chart. Second method, a search algorithm with iterations of the offered load (its increase, decrease and a step change) in response to measured forwarding rate (or a packet loss) of the gateway. Using this method is possible to precisely measure MFRSOL, throughput and MFR. Disadvantage of the method might be its time consumption as we need several iterations until the result is approached. Therefore it is necessary to design effective iterative algorithm and wisely adjust its parameters (start point, step, tolerances limit).

5 VOL. 3, NO., APRIL CPU Utilization TABLE V CPU UTILIZATION FOR 5 BYTES UDP DATAGRAM, ESP-3DES; SOFTWARE ENCRYPTION Fig. 5. CPU utilization at 5 bytes UDP datagram; software encryption Total CPU Utilization 8 6 Total CPU utilization Measured point Interrupt utilization Extrapolated point 7%CPUat,3 Mbps Encryption utilization Encryption utilization Interrupt utilization Offered Load Total CPU utilization Forwarding Rate 99%CPUat 3,3Mbps 3 Offered Load Fig. 6. Throughput estimation by extrapolating the CPU utilization; estimation has % inaccuracy from the actual throughput value 3, Mbps,5 3,5 3,5,5,5 Forwarding rate It is necessary to say that for different packet lengths the relation between CPU utilization and forwarding rate utilization is not the same, as it was in case of hardware crypto accelerator enabled. However, presented linear trend of both components of CPU utilization CPU interrupts and encryption process is valid for every packet length. 6 Conclusion and future work In this paper we presented dependency of IPsec gateway performance on offered load and suggested considerations for effective and precise performance testing. According to our observations MFRSOL ( Maximal Forwarding Rate Same as Offered Load ) is more suitable term to universally describe the performance of a device than the throughput described in RFC, as this may be dependent on duration of the test. The paper has also outlined several interesting and potential fields of further research. We will aim to inspect IPsec capacity and CPU utilization for different nature of traffic more parallel flows or not-constant packet inter-departure time distribution, and mixed traffic, i.e. more packet lengths present in the concurrent flows. We will also examine the possibilities of estimation the IPsec buffer length. Acknowledgement This work is a part of research activities conducted at Slovak University of Technology Bratislava, Faculty of Electrical Engineering and Information Technology, Institute of Telecommunications, within the scope of the projects VEGA No. /86/ Modelling of Multimedia Traffic Parameters in IMS Networks and Support of Centre of Excellence for SMART Technologies, Systems and Services II., ITMS 69, co-funded by the ERDF. References [] Tisovský, A., Klúčik, S., "Method for Calculation the Packet- Size Dependent Throughput of a Computationally Intensive IPsec Process" In: Elektrorevue, ISSN 3-539, November, Art. no 98W. [] S. Bradner, RFC - Benchmarking Terminology for Network Interconnection Devices, IETF RFC, 99 [3] R. Mandeville, RFC - Benchmarking Terminology for LAN Switching Devices, IETF RFC, 998 [] M. Kaeo, T. Van Herck, M. Bustos, Terminology for Benchmarking IPsec Devices: draft-ietf-bmwg-ipsec-term-, IETF Draft, 9 [5] M. Zec, M. Mikuc, M. Žagar, Estimating the Impact of Interrupt Coalescing Delays on Steady State TCP Throughput, Proceedings of the th SoftCOM conference, [6] Khaled Salah, Integrated performance evaluating criterion for selecting between interrupt coalescing and normal interruption, International Journal of High Performance Computing and Networking, Volume 3 Issue 5/6, December 5 [7] M. Castelino, F. Hady, Network Processing Forum, Tutorial on NPF's IPsec Forwarding Benchmark. [Online].. Available: Benchmark 3

6 VOL. 3, NO., APRIL [8] M. Kaeo, Methodology for Benchmarking IPsec Devices, IETF Draft, 9 [9] S. Bradner, RFC 5 - Benchmarking Methodology for Network Interconnect Devices, IETF RFC, 999 [] B. Hickman et al., RFC 35 - Benchmarking Methodology for Firewall Performance, IETF RFC, 3 [] G. Waters, K. Stammberger, Understanding Crypto Performance in Embedded Systems. [Online]. 9. Available: WP_Rev.pdf [] Cisco Systems, Troubleshooting High CPU Utilization on Cisco Routers. Configuration Guide. [Online]. Available: oducts_tech_note986a8a7f.shtml?referring_site=body nav [3] T. Balogh, M. Medvecký, Comparison of Priority Queuing Based Scheduling Algorithms. In: Elektrorevue. - ISSN Roč. 5, 6.. (), art. no 93 [] M. Voznak, F. Rezac, M Halas, Speech Quality Evaluation in IPsec Environment, th International Conference on Networking, VLSI and Signal Processing (ICNVS '), FEB -, Univ Cambridge, Cambridge, ENGLAND, Source: RECENT ADVANCES IN NETWORKING, VLSI AND SIGNAL PROCESSING Book Series: Mathematics and Computers in Science and Engineering Pages: 9-53, Published: