T H E TOLLY G R O U P No. 199104 January 1999 Nortel Networks Contivity Extranet Switch 4000 Fast Ethernet-to-Fast Ethernet Layer 2 Tunneling Protocol Throughput Test Summary Premise: As savvy network administrators look to implement VPNs to develop cost-effective, secure networking across the Internet, they need to know how well their VPN tunnel servers perform while forwarding VPN traffic. Since Layer 2 Tunneling Protocol (L2TP) is a major emerging standard for establishing VPNs, The Tolly Group benchmarked the L2TP throughput of each L2TP network server under test. Nortel Networks commissioned The Tolly Group to benchmark the performance characteristics of each L2TP Network Server (LNS) under test, while handling up to 2,000 active L2TP tunnels in a Fast Ethernet environment. Switches and routers configured as LNSs are commonly used to establish secure tunnels across the Internet, enabling remote users to access internal resources at a central office. The Tolly Group examined the zero-packet-loss (+/- 2%) Fast Ethernet-to-Fast Ethernet L2TP throughput of Nortel s Contivity Extranet Switch 4000 (CES 4000), versus the Cisco 7206 router. The test configuration emulated VPN solutions that are also referred to as LAN-to-LAN or branch-toheadquarters VPN solutions. All tests used a single L2TP session per tunnel. Testing was performed in December 1998 and January 1999. Test results show that, with no encryption, the Nortel CES 4000 significantly outperformed the Cisco 7206 in every test. The results also show that the Throughput (Mbit/s) 90 80 70 60 50 40 30 20 10 0 83.3 43.1 Test Highlights ❻ Delivers substantially higher zero packet-loss (+/-2%) L2TP in every test conducted ❻ Achieves 82 Mbit/s and 53.4 Mbit/s more throughput than the Cisco 7206 while forwarding 1,450-byte and 512-byte IP datagrams, respectively, across two tunnels ❻ Demonstrates 41.6 Mbit/s and 22.6 Mbit/s more throughput than the Cisco 7206 while forwarding 1,450-byte and 512- byte IP datagrams, respectively, across 2,000 tunnels Zero Packet-Loss (+/- 2%)Throughput Fast Ethernet-to-Fast Ethernet Via the Layer 2 Tunneling Protocol (1,450-byte IP datagrams) 80.9 41.5 79.3 36.8 42.5 12.1 1.3 1.3 1.2 0.9 2 50 500 2,000 Number of VPN (L2TP) tunnels established Nortel CES 4000 Cisco 7206* Cisco 7206** *Note: Cisco 7206 tested with Experimental Version 11.3 (19981111:013259). **Note: Cisco 7206 tested with Version 11.3(6)AA1, Early Deployment Release Software. Source: The Tolly Group, January 1999 Figure 1 1999 The Tolly Group Page 1
Nortel CES 4000 delivered lower but consistent throughput with encryption enabled. The Cisco 7206 was not evaluated with encryption enabled. Zero packet-loss L2TP Throughput (1,450-byte datagrams) The zero packet-loss Fast Ethernet-to-Fast Ethernet L2TP results show the zero packet-loss (+/- 2%) throughput capability of the L2TP network server while routing 1,450-byte IP datagrams (void of any IP header information) through L2TP tunnels with encryption disabled (see figure 1). During tests involving two tunnels and 1,450-byte IP datagrams, the Nortel Contivity Extranet Switch (CES) 4000 delivered 64 times greater (IOS Version 11.3(6)AA1 Early Deployment Release Software) 83.3 Mbit/s versus 1.3 Mbit/s, respectively. The Nortel CES 4000 also delivered significantly greater throughput than the Cisco 7206 when the number of tunnels Zero Packet-Loss (+/- 2%)Throughput Fast Ethernet-to-Fast Ethernet Via the Layer 2 Tunneling Protocol (512-byte IP datagrams) Throughput (Mbit/s) 60 50 40 30 20 10 0 53.8 53.4 20.7 25.2 were increased to 50, 500 and 2,000, delivering 62, 66, and 47 times greater throughput than the Cisco 7206, respectively. 48.8 8.4 23.0 3.4 0.4 0.5 0.4 0.4 2 50 500 2,000 Number of VPN (L2TP) tunnels established Nortel CES 4000 Cisco 7206* Cisco 7206** *Note: Cisco 7206 tested with Experimental Version 11.3 (19981111:013259). **Note: Cisco 7206 tested with Version 11.3(6)AA1, Early Deployment Release Software. Source: The Tolly Group, January 1999 Figure 2 The second round of tests, conducted using IOS Experimental Version 11.3 (19981111: 013259) for the Cisco 7206, resulted in significantly higher performance Zero Packet-Loss (+/- 2%)Throughput Fast Ethernet-to-Fast Ethernet with Encryption Disabled System Under Test Nortel CES 4000 Cisco 7206* Cisco 7206** IP Datagram Size (bytes) Number of VPN Tunnels CES 4000 Average CES 4000 Average Throughput (Mbit/s) Throughput (Mbit/s) Throughput (Mbit/s) 512 2 0.9% 53.8 1.9% 20.7 0.0% 0.4 50 1.1% 53.4 0.6% 25.2 0.4% 0.5 500 0.9% 48.8 0.8% 8.4 0.0% 0.4 2,000 1.6% 23.0 0.0% 3.4 0.7% 0.4 1,450 2 1.1% 83.3 1.5% 43.1 0.1% 1.3 50 0.9% 80.9 0.2% 41.5 0.1% 1.3 500 0.4% 79.3 1.1% 36.8 1.0% 1.2 2,000 0.4% 42.5 1.0% 12.1 0.1% 0.9 *Note: Cisco 7206 tested with Experimental Version 11.3 (19981111:013259). **Note: Cisco 7206 tested with Version 11.3(6)AA1, Early Deployment Release Software. Source: The Tolly Group, January 1999 Figure 3 1999 The Tolly Group Page 2
for the Cisco product (see figure 1). Although throughput for the Cisco 7206 increased from roughly 1 Mbit/s to between 12.1 Mbit/s (across 2,000 tunnels) and 43.1 Mbit/s (across two tunnels), Nortel s CES 4000 maintained a significant performance advantage. While handling 2,000 tunnels, the Nortel CES 4000 delivered 3.5 times greater 42.5 Mbit/s versus 12.1 Mbit/s, respectively. While handling 500 tunnels, the Nortel CES 4000 delivered 2.2 times greater throughput than the Cisco 7206 79.3 Mbit/s versus 36.8 Mbit/s, respectively. The Nortel CES 4000 maintained the performance advantage, delivering 1.9 times greater throughput than the Cisco 7206, during tests involving two and 50 tunnels. See figure 3 for actual results. zero packet-loss L2TP Throughput (512-byte datagrams) The zero packet-loss Fast Ethernet-to-Fast Ethernet L2TP results show the zero packet-loss (+/- 2%) throughput capability of the L2TP network server while routing 512-byte IP datagrams through L2TP tunnels with encryption disabled (see figure 2). During tests involving two tunnels and 512-byte IP datagrams, Nortel s CES 4000 delivered 134 times greater (IOS Version 11.3(6)AA1 Early Deployment Release Software) 53.8 Mbit/s versus 0.4 Mbit/s, respectively. The Nortel CES 4000 also delivered significantly greater throughput than the Cisco 7206 when the number of tunnels were increased to 50, 500 and 2,000, delivering 107, 122 and 58 times greater throughput than the Cisco 7206, respectively. The second round of tests, conducted using IOS Experimental Version 11.3 (19981111: 013259) for the Cisco 7206, resulted in significantly greater performance for the Cisco product (see also figure 2). Although throughput for the Cisco 7206 increased from roughly 0.5 Mbit/s to between 3.4 Mbit/s (across 2,000 tunnels) and 25.2 Mbit/s (across 50 tunnels), the Nortel CES 4000 maintained a superior performance advantage. While handling 2,000 tunnels, the Nortel CES 4000 delivered 6.8 times greater 23 Mbit/s versus 3.4 Mbit/s, respectively. While handling 500 tunnels, the Nortel CES 4000 delivered 5.8 times greater 48.8 Mbit/s versus 8.4 Mbit/s, respectively. The Nortel CES 4000 maintained its performance advantage during tests involving two and 50 tunnels, delivering 2.6 and 2.1 times greater. For a complete set of results see figure 3. Nortel Contivity Extranet Switch 4000 (Encryption Enabled) To ascertain the zero packet-loss (+/- 2%) L2TP throughput of the Nortel CES 4000 in a secure environment, the same battery of tests were conducted with encryption enabled. Results show that the Nortel CES 4000 delivered consistent throughput while routing 512-byte and 1,450-byte IP datagrams through up to 2,000 tunnels. The Nortel CES 4000 delivered between 22.1 Mbit/s and 24.6 Mbit/s of throughput while routing 512- byte IP datagrams through 2,000 and two tunnels, respectively. The Nortel CES 4000 delivered between 46.4 Mbit/s and 50.6 Nortel Networks Contivity Extranet Switch 4000 L2TP Performance Nortel Networks Contivity Extranet Switch 4000 Product Specifications* Tunneling Protocols Supported ❻ PPTP (Including Compression and encryption) ❻ L2F ❻ IPSec (Including AH, ESP and ISAKMP/ Oakley) ❻ L2TP Authentication Services ❻ Internal or external LDAP ❻ RADIUS ❻ ❻ NT Domains Token Card Integration (Security Dynamics, AXENT) Encryption ❻ Up to 128-bit key length ❻ DES, Triple DES, and RC4 Filtering Criteria ❻ Individual user or group profile ❻ Source and destination IP address ❻ Port, service and protocol type ❻ SYN/ACK bit Bandwidth Management ❻ Four internal priority levels using Random Early Detection (RED) ❻ Four connection priority levels ❻ External Quality of Service (RSVP) Accounting ❻ Internal and External RADIUS databases ❻ Event, system, security and configuration ❻ Automatic archiving by month Management ❻ Full HTML and Java Configuration ❻ Scriptable SNMP alerts ❻ Four levels of administrator access ❻ Role-based management to separate service provider and end-user management For more information contact: Nortel Networks 4401 Great America Parkway Santa Clara, CA 95054 Phone: (800) 822-9638 URL: http://www.nortelnetworks.com *Vendor-supplied information not verified by The Tolly Group 1999 The Tolly Group Page 3
Mbit/s of throughput while routing 1,450-byte IP datagrams through 2,000 and two tunnels, respectively. (See figure 4.) Since the unencrypted performance of Cisco s 7206 already trailed the Nortel CES 4000 encrypted performance, The Tolly Group chose not to run additional comparisons with the Cisco 7206 with encryption enabled. Analysis Virtual private networks (VPNs) are emerging as one of the key technologies for implementing secure, cost-effective networking across the public Internet. As is typically the case with new technologies, customers need to know the performance of these technologies before deciding which product to deploy. This particular series of tests focused on a specific VPN protocol; the Layer 2 Tunneling Protocol (L2TP). The L2TP standard currently has an IETF draft status but all major networking vendors are offering L2TP solutions based on the current draft specification. The products under test for this report, Nortel s Contivity Extranet Switch (CES) 4000 and Cisco s 7206 router, are positioned as L2TP network servers, the L2TP term for the central site tunnel servers. The ability to handle a high number of tunnels with an acceptable level of performance is a key evaluation criterion for LNS devices. Testing showed that Nortel s Contivity Extranet Switch (CES) 4000 substantially outperformed the Cisco 7206 in every scenario tested (see methodology below). For the first series of tests, Cisco submitted IOS Version 11.3(5) AA1, Early Deployment Release Software (fc1) as the appropriate software level for these tests. However, due to the large disparity between Nortel and Cisco results (see figures 1 through 3), The Tolly Group solicited a review of results from Cisco. Upon reviewing the test results, Cisco submitted IOS Version 11.3(6)AA1 for a retest. The results in this case were only incrementally better for Cisco and upon further review, Cisco submitted IOS Experimental Version 11.3(19981111:013259) for another series of tests. With this version the Cisco results improved dramatically, however, despite this performance improvement, the Nortel CES 4000 still delivered significantly superior L2TP performance as shown in figures 1 through 3. While the Cisco 7206, as a general purpose router, may offer a broader functionality than the Nortel CES 4000, ISPs and large enterprises looking for VPNspecific solutions would find that the Nortel CES 4000 offers a significantly better LNS solution than the Cisco 7206. Although zero packet-loss (+/- 2%) L2TP throughput is an important baseline, network managers also need to know how well VPN products perform when handling encrypted traffic. To measure the impact of encryption on L2TP performance, additional tests were conducted with the Nortel CES 4000 to gauge the impact that encryption has on its L2TP performance. As shown in figure 5, the Nortel CES 4000 demonstrated consistent L2TP performance regardless of load, indicating that it is a scalable VPN solution while handling up to 2,000 active tunnels. Even with encryption enabled, in all cases except one, the Nortel CES 4000 demonstrated a better performance than the non-encrypted performance of the Cisco 7206. Test Configuration and Methodology The Tolly Group tested two L2TP network servers: Nortel s Contivity Extranet Switch 4000, version 2.0.53 (01Nov1998) and Cisco s 7206 router, Version 11.3(5)AA Early Deployment Release Software (fc1), Version 11.3(6) AA1, and Experimental Version 11.3 (19981111:013259). The LNS under test was configured with a pool of IP addresses assigned to virtual clients created by L2TP frame generators. IP routing was enabled and the LNS interface connected to the Fast Ethernet (i.e., the public network) was configured for L2TP tunneling. Although L2TP allows multiple sessions per tunnel, the tests were conducted using only one session per tunnel. Two Nortel CES 4000s (version 2.0.53) were used to set up and generate L2TP traffic. Engineers used a Micron Client Pro Mxe, 233- MHz Pentium configured with 64 Mbytes of RAM and Windows NT Server 4.0 SP3, as the test console to start the test and monitor the devices under test. The console was used to download the test configuration to the L2TP frame generators via telnet commands through a Xyplex MaxServer 1640-004 terminal server, version 6.0.2S13. Traffic streams were used so as to eliminate the impact of TCP and upper layer protocols on LNS performance measurement. In particular, no upper layer recovery was used for lost packets. The address range for the virtual clients 1999 The Tolly Group Page 4
CES 4000 Zero Packet-Loss (+/- 2%) L2TP Throughput Fast Ethernet-to-Fast Ethernet with Encryption Enabled IP Datagram Size (bytes) Number of VPN Tunnels CES 4000 Average CES 4000 Average Throughput (Mbit/s) 512 2 0.2% 24.6 50 0.3% 24.2 500 0.2% 23.7 2,000 1.1% 22.1 1,450 2 0.5% 50.6 50 0.8% 50.2 500 0.2% 49.0 2,000 0.9% 46.4 Source: The Tolly Group, January 1999 Figure 4 was set via GenInit to 10.2.128.0, 255.255.128.0. The delay between tunnel creation attempts was set via l2tpmultdelay to one second. This parameter is the actual delay between packets sent to create the tunnels during test setup. The packet generation delay counter was varied via blasterpause to generate traffic at specific Fast Ethernet rates and to determine the zero packet-loss (+/- 2%) throughput of the system under test. The IP data size was set via TunnelGen PktSize to either 512 bytes or 1,450 bytes. The starting address and number of clients was set via CreateL2tpMultTunnel Clients to 10.2.128.2, 10.2.0.2, l2tppap, 2; where 10.2.128.2 was the starting IP address, 10.2.0.2 was the IP address of the LNS interface, l2tppap was the name of the client account defined on the LNS under test, and the number of tunnels was set to two. Prior to running the test, The Tolly Group manually verified the number of active L2TP tunnels. The test was started via the GenStart(60) command, which denotes a test duration of 60 seconds. After the test completes, the two L2TP frame generators (i.e., Nortel CES 4000) were used to measure zero packetloss (+/- 2%) Fast Ethernet-to-Fast Ethernet L2TP throughput. The frame size on the trusted and untrusted networks differed due to the overhead (55 bytes) of the L2TP tunnels. The L2TP frame generators calculated throughput based on the data size of each packet void of any additional LAN header/trailer information. A Micron Millenia Lxr (Pentium 133-MHz processor with 64 MB of RAM) running Windows NT 4.0 Workstation and Network Associates NetXRay version 3.0.0 was used to capture VPN tunnel setup data and the ensuing performance traffic. See figure 5 for the test bed diagram. Equipment Acquisition and Support All competitive equipment was acquired through normal product distribution channels. The Tolly Group contacted executives at Cisco Systems and invited them to provide a higher level of support than available through normal channels. Cisco accepted The Tolly Group s offer and provided phone technical support to configure the Cisco 7206 for the tests executed by The Tolly Group. The Tolly Group verified product release levels and shared test configurations with Cisco in order to give the vendor an opportunity to optimize the Cisco 7206 for the tests. The Tolly Group shared test results for the Cisco 7206 with Cisco. For a more complete understanding of the interaction between The Tolly Group and Cisco, check out the Technical Support Diary for Competitive Products Tested posted on The Tolly Group s World Wide Web site at http:\\www.tolly.com (see 199104). 1999 The Tolly Group Page 5
NetXRay Analyzer Test Bed Untrusted Public Network Shared Fast Ethernet 100Mbit/s Connected to both L2TP Network Server (LNS) Under Test networks for traces L2TP Frame Generator #1 L2TP Frame Generator #2 Shared Fast Ethernet 100Mbit/s Trusted Private Network Telnet Sessions Terminal Server Test Console Telnet Session Source: The Tolly Group, January 1999 Figure 5 The Tolly Group gratefully acknowledges the providers of test equipment used in this project: Vendor Product Web address Network Associates, Inc. NetXRay Analyzer Version 3.0.0 www.networkassociates.com Since its inception, The Tolly Group has produced highquality tests that meet three overarching criteria: All tests are objective, fully documented and repeatable. We endeavor to provide complete disclosure of information concerning individual product tests, and multiparty competitive product evaluations. As an independent organization, The Tolly Group does not accept retainer contracts from vendors, nor does it endorse products or suppliers. This open and honest environment assures vendors they are treated fairly, and with the necessary care to guarantee all parties that the results of these tests are accurate and valid. The Tolly Group has codified this into the Fair Testing Charter, which may be viewed at http://www.tolly.com. Project Profile Sponsor: Nortel Networks Document number: 199104 Product Class: L2TP VPN Routers Products under test: ❺ Nortel Networks Contivity Extranet Switch 4000 ❺ Cisco 7206 router Testing window: December 1998 through January 1998 Software versions tested: ❺ Nortel Networks Contivity Extranet Switch 4000, version 2.0.53 (01Nov1998) ❺ Cisco 7206 router, IOS Version 11.3(5)AA1, Early Deployment Release Software (fc1) ❺ Cisco 7206 router, IOS Version 11.3(6)AA1, Early Deployment Release Software (fc1) ❺ Cisco 7206 router, IOS Experimental Version 11.3(19981111:013259) Additional information available: ❺ Technical support diary, configuration files and data files are available at The Tolly Group s web site at http:\\www.tolly.com. Internetworking technology is an area of rapid growth and constant change. The Tolly Group conducts engineering-caliber testing in an effort to provide the internetworking industry with valuable information on current products and technology. While great care is taken to assure utmost accuracy, mistakes can occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental, and consequential damages which may result from the use of information contained in this document. All trademarks are the property of their respective owners. Tolly Group doc. 199104 rev. clk 21 Jan 99 1999 The Tolly Group Page 6