TEST METHODOLOGY. SSL/TLS Performance. v1.0

Transcription

1 TEST METHODOLOGY SSL/TLS Performance v1.0

2 Table of Contents 1 Introduction The Need for SSL/TLS Performance Testing About This Test Methodology Inclusion Criteria SSL/TLS Functionality Validation Encryption/Inspection Validation Cipher Negotiation Performance Maximum SSL/TLS Handshakes per Second HTTPS Capacity with No Transaction Delay ,880 KB HTTP Response Size 40 Connections per Second KB HTTP Response Size 150 Connections per Second KB HTTP Response Size 600 Connections per Second KB HTTP Response Size 2500 Connections per Second Application Average Response Time: HTTP HTTPS Capacity with No Transaction Delay (HTTP Persistent Connections) ,880 KB HTTP Response Size 40 Connections per Second KB HTTP Response Size 150 Connections per Second KB HTTP Response Size 600 Connections per Second KB HTTP Response Size 2,500 Connections per Second... 7 Contact Information

3 1 Introduction 1.1 The Need for SSL/TLS Performance Testing The use of the Secure Sockets Layer (SSL) protocol, and its newer iteration, Transport Layer Security (TLS), has been on the rise with the ever-increasing need for privacy online. Modern cybercampaigns frequently focus on attacking users through the most common web protocols and applications. NSS Labs continues to receive inquiries from enterprise customers during their assessments of vendors that provide SSL/TLS decryption and protection technologies. To this end, NSS has developed a methodology to test the capabilities and performance of devices providing SSL/TLS protection. 1.2 About This Test Methodology NSS test reports are designed to address the challenges faced by enterprise security and IT professionals in selecting and managing security products. The scope of this particular methodology includes: Verification of SSL/TLS capability SSL/TLS performance Based on the needs identified in NSS research, the following capabilities are considered essential in any SSL/TLScapable device: Ability to perform SSL inspection Ability to negotiate to all modern ciphers and key sizes Ability to detect and block web-based (HTTP and HTTPS) attacks, including malware and exploits 1.3 Inclusion Criteria In order to encourage the greatest participation and to allay any potential concerns of bias, NSS invites all security vendors claiming SSL/TLS capabilities to submit their products at no cost. Vendors with major market share as well as challengers with new technology will be included. This test will be open to various technologies including next generation firewalls (NGFW), next generation intrusion prevention systems (NGIPS), secure web gateway (SWG), and others. 3

4 2 SSL/TLS Functionality Validation 2.1 Encryption/Inspection Validation In order to confirm that the device under test (DUT) is correctly decrypting and inspecting SSL/TLS traffic, a functionality test will be performed where a known malicious file is embedded in the encrypted traffic. NSS has access to various well-known malicious files, including but not limited to: EICAR file MS Aurora CVE Shellshock CVE Adobe Flash Player ByteArray With Workers Use-After-Free The device will be expected to properly detect the known malicious file. 2.2 Cipher Negotiation The DUT is expected to accurately negotiate and recognize modern SSL/TLS ciphers in order to decrypt and inspect SSL/TLS traffic. This test will be performed using a script that will attempt an SSL/TLS negotiation using both current and older versions of SSL/TLS protocols as well as all known ciphers. There are currently 323 cipher suites registered with IANA. The results of this test will be reported as a matrix of supported SSL/TLS versions and ciphers. 4

5 3 Performance This section measures the performance of the DUT using various traffic conditions that provide metrics for HTTPbased real-world performance. Individual implementations will vary based on usage; however, these quantitative metrics provide a gauge as to whether a particular DUT is appropriate for a given environment. Note: For these tests, the device will be configured in the same manner as for Test 2.1 (Encryption/Inspection Validation). This will ensure that in these tests traffic is not bypassing any inspection engine for the purpose of better performance. Each of the tests in this section will be performed multiple times, with the following configurations: No Encryption A comparative baseline will be established. TLS 1.2 cipher suites: o TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 using a 256-bit key strength o TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 using a 256-bit key strength o TLS_RSA_WITH_AES_128_GCM_SHA256 using a 2048-bit asymmetric key o TLS_RSA_WITH_AES_256_GCM_SHA384 using a 2048-bit asymmetric key o TLS_RSA_WITH_AES_128_GCM_SHA256 using a 4096-bit asymmetric key o TLS_RSA_WITH_AES_256_GCM_SHA384 using a 4096-bit asymmetric key 3.1 Maximum SSL/TLS Handshakes per Second This test measures a vendor s handshake performance using TLS v1.2 with the key sizes and ciphers referenced in section 3. This is purely to determine the number of handshakes that can be performed per second. No data will be transferred after the handshake is sucessfully completed, and the session will be terminated immediately. 3.2 HTTPS Capacity with No Transaction Delay The aim of these tests is to stress the HTTPS detection engine and determine how the DUT copes with network loads of varying average packet size and varying connections per second. By creating genuine session-based traffic with varying session lengths, the DUT is forced to track valid TCP sessions, thus ensuring a higher workload than for simple packet-based background traffic. This provides a test environment that is as close to real-world conditions as it is possible to achieve in a lab environment, while ensuring absolute accuracy and repeatability. Each transaction consists of a single HTTPS GET request,and there are no transaction delays (i.e., the web server responds immediately to all requests). All packets contain valid payload (a mix of binary and ASCII objects) and address data, and this test provides an excellent representation of a live network (albeit one biased towards HTTP traffic) at various network loads ,880 KB HTTP Response Size 40 Connections per Second Maximum 40 new connections per second per Gigabit of traffic with a 2,880 KB HTTPS response. With low connection rates and large packet sizes, all DUTs should be capable of performing well throughout this test. In 5

6 KB HTTP Response Size 150 Connections per Second Maximum 150 new connections per second per Gigabit of traffic with a 768 KB HTTPS response size. With low connection rates and large packet sizes, all DUTs should be capable of performing well throughout this test. In KB HTTP Response Size 600 Connections per Second Maximum 600 new connections per second per Gigabit of traffic with a 192 KB HTTPS response. With medium packet sizes and increased connection rates, this represents an average production network. In addition, the session is encrypted in order to ascertain performance degradation. This test will use a 2048-bit certificate and key size KB HTTP Response Size 2500 Connections per Second Maximum 2,500 new connections per second per Gigabit of traffic with a 44 KB HTTPS response size. With decreased packet sizes and increased connection rates, this may also represent an average production network. In 3.3 Application Average Response Time: HTTP The test traffic is passed through the DUT and directly back to the test generator. The latency results are recorded for each response size (44 KB, 192 KB, 768 KB, and 2,880 KB HTTPS) and at a load level of 90% of the maximum throughput with zero packet loss, as previously determined in section HTTPS Capacity with No Transaction Delay (HTTP Persistent Connections) The aim of these tests is to stress the HTTPS detection engine and determine how the DUT copes with network loads of varying average packet size and varying connections per second while decrypting and inspecting all traffic. By creating genuine session-based traffic with varying session lengths, the DUT is forced to track valid TCP sessions, thus ensuring a higher workload than for simple packet-based background traffic. This provides a test environment that is as close to real-world conditions as it is possible to achieve in a lab environment, while ensuring absolute accuracy and repeatability. This test will use HTTP persistent connections, with each TCP connection containing ten HTTP GETs and associated responses. All packets contain valid payload (a mix of binary and ASCII objects) and address data, and this test provides an excellent representation of a live network (albeit one biased towards HTTPS traffic) at various network loads. The stated response size is the total of all HTTP responses within a single TCP session ,880 KB HTTP Response Size 40 Connections per Second Maximum 40 new connections per second per Gigabit of traffic with a 2,880 KB HTTPS response. With low connection rates and large packet sizes, all DUTs should be capable of performing well throughout this test. In 6

7 KB HTTP Response Size 150 Connections per Second Maximum 150 new connections per second per Gigabit of traffic with a 768 KB HTTPS response size. With low connection rates and large packet sizes, all DUTs should be capable of performing well throughout this test. In KB HTTP Response Size 600 Connections per Second Maximum 600 new connections per second per Gigabit of traffic with a 192 KB HTTPS response. With medium packet sizes and increased connection rates, this represents an average production network. In addition, the session is encrypted in order to ascertain performance degradation. This test will use a 2048-bit certificate and key size KB HTTP Response Size 2,500 Connections per Second Maximum 2,500 new connections per second per Gigabit of traffic with a 44 KB HTTPS response size. With decreased packet sizes and increased connection rates, this may also represent an average production network. In 7

8 Contact Information NSS Labs, Inc. 206 Wild Basin Road Building A, Suite 200 Austin, TX USA This and other related documents available at: To receive a licensed copy or report misuse, please contact NSS Labs NSS Labs, Inc. All rights reserved. No part of this publication may be reproduced, copied/scanned, stored on a retrieval system, ed or otherwise disseminated or transmitted without the express written consent of NSS Labs, Inc. ( us or we ). Please read the disclaimer in this box because it contains important information that binds you. If you do not agree to these conditions, you should not read the rest of this report but should instead return the report immediately to us. You or your means the person who accesses this report and any entity on whose behalf he/she has obtained this report. 1. The information in this report is subject to change by us without notice, and we disclaim any obligation to update it. 2. The information in this report is believed by us to be accurate and reliable at the time of publication, but is not guaranteed. All use of and reliance on this report are at your sole risk. We are not liable or responsible for any damages, losses, or expenses of any nature whatsoever arising from any error or omission in this report. 3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY US. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT, ARE HEREBY DISCLAIMED AND EXCLUDED BY US. IN NO EVENT SHALL WE BE LIABLE FOR ANY DIRECT, CONSEQUENTIAL, INCIDENTAL, PUNITIVE, EXEMPLARY, OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 4. This report does not constitute an endorsement, recommendation, or guarantee of any of the products (hardware or software) tested or the hardware and/or software used in testing the products. The testing does not guarantee that there are no errors or defects in the products or that the products will meet your expectations, requirements, needs, or specifications, or that they will operate without interruption. 5. This report does not imply any endorsement, sponsorship, affiliation, or verification by or with any organizations mentioned in this report. 6. All trademarks, service marks, and trade names used in this report are the trademarks, service marks, and trade names of their respective owners. 8