TOLLY. Tasman Networks, Inc. commissioned. Tasman Networks, Inc and 1004 Routers. Test Summary

Transcription

1 Tasman Networks, Inc and 1004 Routers Competitive Performance Evaluation versus Cisco 1841, 2811 and 2821 Integrated Service Routers Premise: Wide-area network routers that provide T1/E1 access for branch offices and others to enterprise and service provider networks must be able to deliver high throughput, even with Quality of Service, IPSec VPN Security Services, Network Address Translation or other services active and vying for processor cycles. Tasman Networks, Inc. commissioned The Tolly Group to evaluate its Tasman 1004 and Tasman 1002 wide-area network routers with integrated network services such as Quality of Service (QoS), IPSec VPN with on-board hardware acceleration, stateful firewall, Network Address Translation (NAT) and Access Control Lists (ACLs) for enterprises and service providers. Tolly Group engineers measured the multilink Point-to-Point Protocol (MLPPP) zero-loss throughput of the Tasman 1004 router against Cisco 2811 and Cisco 2821 routers, with QoS, NAT and ACL features enabled in a scenario with multilink PPP traffic riding over four T1s. To test IPSec VPN throughput over a single 3DES/SHA1 tunnel with QoS and stateful firewall enabled, engineers tested the Tasman 1002 against the Cisco 1841 and the Cisco 2811 for Layer 3 throughput with multilink PPP traffic riding over two T1s. The T H E TOLLY G R O U P No December 2005 Throughput (Mbps) Test Highlights Tasman 1004 Router operated at or near wire-speed throughput and outperformed Cisco 2811 and 2821 routers, delivering 6X and 2X more throughput respectively, while simultaneously supporting active Quality of Service, Access Control List filters and Network Address Translation over four T1 lines Tasman 1002 and 1004 Routers demonstrated wire-speed performance for most packet sizes tested while simultaneously supporting active QoS, IPSec VPN and stateful firewall services over two or four T1 lines Tasman 1004 consistently outperformed the Cisco 2811 for all the packet sizes tested, especially at smaller packet sizes, when tested across four T1s with QoS, IPSec VPN and stateful firewall services, delivering 3X more throughput than its counterpart Tasman 1002 achieved wire-speed throughput at all packet sizes, while performance of Cisco 2811 and 1841 weaken when handling and 256-byte packets tested across two T1s with QoS, IPSec VPN and stateful firewall services The Tolly Group Page Packet Sizes (Bytes) Tasman 1004 Cisco 2821 Cisco 2811 Test Summary 4xT1 Multilink PPP (MLPPP) Aggregate WAN Layer 3 Throughput Zero-Loss Performance with QoS/ACL/NAT Enabled Source: The Tolly Group, September 2005 Figure 1 1.8

2 4xT1 Multilink PPP (MLPPP) Aggregate WAN Layer 3 Throughput Zero-Loss Performance with QoS/VPN/Firewall Enabled Throughput (Mbps) Packet Sizes (Bytes) Tasman 1004 Cisco 2811 Source: The Tolly Group, September 2005 Figure 2 Tolly Group also tested the Layer 3 IPSec VPN throughput of the Tasman 1004 versus Cisco 2811 in a scenario with multilink PPP traffic riding over four T1s. All tests were performed in September/October The Cisco 2811 and Cisco 2821 routers were installed with an optional IPSec VPN hardware acceleration module, while the Cisco 1841 router and Tasman 1002/1004 routers had on-board hardware acceleration support. Test results show that the Tasman 1004 and 1002 routers deliver superior throughput for the majority of packet sizes tested, especially with regards to smaller packet sizes (64 bytes to 256 bytes), delivering up to 6.4X greater throughput than the Cisco devices tested. Results Zero-Loss Throughput over Four T1s Tolly Group engineers tested the Tasman 1004 router against Cisco 2811 and Cisco 2821 routers to measure the fullduplex, zero-loss aggregate WAN throughput across a point-topoint router connection equivalent to a group of four T1 circuits combined using MLPPP under two scenarios: with QoS/Network Address Translation(NAT)/ACL enabled on the devices. with QoS/IPSec VPN/stateful firewall enabled on the devices With QoS, NAT and ACLs enabled, the Tasman 1004 router delivered wire-speed throughput for packet sizes greater than 256 bytes, and delivered more than 95% of wire-speed throughput for the more taxing 64- and 128- byte packet sizes. On the other hand, the Cisco 2811 router delivered significantly poorer throughput compared to Tasman 1004 router at all packet sizes tested. The Tasman 1004 router outperformed the Cisco 2811 router by achieving 1.7X to 6.4X greater throughput at all the packet sizes tested, most notably at the taxing smaller packet sizes of 64 to 512 bytes (See Figure 1). Moreover, the Tasman 1004 delivered anywhere from 32% to more than 130% greater throughput than the Cisco With QoS, IPSec VPN and stateful firewall enabled, the Tasman 1004 router was tested against the Cisco 2811 router. Results show the Tasman 1004 router achieving either wire-speed or near wire-speed throughput, 2005 The Tolly Group Page 2

3 depending upon the packet sizes tested. In comparison, the Cisco 2811 router could only achieve near wire-speed throughput for the large packet sizes of 1,024 and 1,400 bytes. For the remaining packet sizes, the Tasman 1004 router outperformed the Cisco 2811 router by achieving 1.5X to 3.5X more throughput, particularly at the 64- and 128- byte packet sizes, where the Tasman 1004 router demonstrated more than 3X the throughput of the Cisco 2811 router (See Figure 2). Zero-Loss Throughput over Two T1s Tolly Group engineers tested the Tasman 1002 router against the Cisco 2811 and the Cisco 1841 routers to measure the fullduplex, zero-loss aggregate WAN throughput across a point-topoint router connection equivalent to a group of two T1 circuits combined using MLPPP with QoS, IPSec VPN and stateful firewall services enabled on the devices. With QoS/IPSec VPN/stateful firewall enabled, the Tasman 1002 router achieved wire-speed throughput in all six packet scenarios with full-duplex traffic traversing a pair of T1 links supporting MLPPP. In contrast, the Cisco 1841 router offered near wire-speed performance only for the two largest packet sizes tested - 1,024 and 1,400 bytes, while at other packet sizes the Tasman 1002 router outperformed the Cisco 1841 by achieving 1.3X (with 512-byte packets) to as much as 4.4X (with 128-byte packets) greater throughput (See Figure 3). Similarly, the Cisco 2811 router achieved wire-speed throughput only for 512, 1,024 and 1,400-byte packet sizes, while the Tasman 1002 router consistently outperformed the Cisco 2811 router at packet sizes of 64 to 256 bytes with 1.2X to 3.4X greater throughput (See Figure 3). Analysis Testing shows that the Tasman Networks 1004 and 1002 routers possess an abundance of processing headroom to accommodate network services while simultaneously offering wire-speed throughput. Tests show that the Tasman 1004/1002 router can deliver wire-speed throughput at most packet sizes tested, while simultaneously processing a combination of QoS, NAT, ACL filters, IPSec VPN and firewall services. By contrast, tests show that the performance of the Cisco 1841/2811/2821 routers sag under the processing load, especially when smaller, more taxing packet sizes come into play. In head-to-head testing, the Tasman 1004/1002 routers consistently deliver better performance than the Cisco 1841/2811/2821 routers under almost all tested scenarios - at times delivering up to 6.4X the throughput of the Cisco devices tested (See Figure 1). Test Configuration and Methodology For the performance tests, The Tolly Group engineers measured the maximum zero-loss ( 0.001% acceptable frame loss) Layer 3 throughput using Spirent SmartBits and SmartFlow. For Tasman Networks, Inc. Tasman 1004, Tasman 1002 Zero-Loss, Layer 3 Throughput Tasman Networks 1004 and 1002 Routers Product Specifications* 1004/1002 Router hardware features Half rack-width 1U chassis (rack mount option available), Two 10/100 Ethernet ports, 256MB RAM, console and AUX ports 4 T1/E1 ports (1004) and 2 T1/E1 ports (1002) T1/E1s can be remotely enabled using software license keys On-board VPN encryption acceleration chip Wire-speed hardware platform with services enabled TiOS advanced software features Routing (RIP, OSPF, BGP4, PIM- SM/SSM, IGMP) Layer 2 features (VLAN tag/forward, Q- in-q, GRE) Services such as sophisticated Multi-class QoS, QoS on Layer 2 traffic Stateful firewall (60+ DDoS protection, 30+ ALG including SIP/H323) IPSec VPN (site-to-site, remote access, DES/3DES/AES, MD5/SHA1) Industry standard CLI and intuitive GUI for ease of management For more information contact: Tasman Networks, Inc Hellyer Ave, San Jose, CA USA Phone: (408) Fax: (408) URL: *Vendor-supplied information not verified by The Tolly Group 2005 The Tolly Group Page 3

4 2xT1 Multilink PPP (MLPPP) Aggregate WAN Layer 3 Throughput Zero-Loss Performance with QoS/FW/VPN Enabled Throughput (Mbps) Packet Size (Bytes) Tasman 1002 Cisco 2811 Cisco 1841 Note: Cisco ISR 1841 & 2811: IOS , 1841 with on-board VPN chip, 2811 with AIM-VPN/EPII-PLUS VPN module Source: The Tolly Group, September 2005 Figure 3 the 4XT1 Multilink PPP IPSec VPN throughput tests, engineers tested a Tasman 1004 outfitted with two 10/100 Base-T Fast Ethernet ports and up to four T1 ports. The Tasman 1004 was running OS Ver 8.2.1/ BootROM Ver. T1k Engineers tested the Tasman 1004 against a Cisco 2811 Integrated Service Router configured with an AIM- VPN/EPII-PLUS VPN module for VPN acceleration, and two dual T1 WIC modules and dual auto-sensing 10/100Base-T Fast Ethernet interfaces. The device was running OS Ver T1/ BootROM Ver. 12.3(8r)T7. The Tasman 1004 also was pitted against the Cisco 2811 Integrated Service Router described earlier, along with a Cisco 2821 Integrated Services Router in tests with QoS/ACL/NAT enabled on the devices. The Cisco 2821 Integrated Services Router was running OS Ver T1/ BootROM Ver. 12.3(8r)T7, with AIM-VPN/EPII- PLUS VPN module, two dual T1 WIC modules and was configured with two integrated 10/100/1000 Ethernet ports. For the 2XT1 Multilink PPP IPSec VPN throughput tests, the engineers tested a Tasman 1002 Router running OS Ver 8.2.1/ BootROM Ver. T1k against the Cisco 2811 Integrated Services Router (configured as mentioned earlier) and a Cisco 1841 Integrated Services Router configured with two Integrated 10/100 Fast Ethernet ports, and running OS Ver T1/ BootROM Ver. 12.3(8r)T8. The Cisco 1841 Integrated Services Router had an on-board chip for IPSec VPN acceleration while the Cisco 2811 was configured with AIM-VPN/EPII-PLUS VPN module. The Cisco routers utilized a dual T1 WIC module. Each router was tested for its Layer 3 throughput while connected back-to-back with an identical router using either four or two T1 WAN connectors. The WAN protocol used on the WAN link was Multilink Point-to-Point Protocol (MLPPP) over multiple T1 links. The throughput was measured using 64-, 128-, 256-, 512-, 1,024- and 1,400-byte packet sizes under two scenarios: with QoS/IPSec VPN/stateful firewall enabled on the devices for the 4xT1 tests with QoS/Network Address Translation(NAT)/ACL enabled 2005 The Tolly Group Page 4

5 Test Bed Test Scenarios : - QoS + ACL + NAT - QoS + Firewall + VPN WAN: 2xT1 or 4xT1 Multilink PPP Device Under Test Device Under Test Traffic generated by SmartBits , 128, 256, 512, 1024, and 1400-byte packets Spirent SmartBits 600 Traffic generated by SmartBits , 128, 256, 512, 1024, and 1400-byte packets Source: The Tolly Group, September 2005 Figure 4 on the devices for the 2xT1 tests. For both scenarios, the QoS configuration consisted of four traffic classes (DSCP 56, DSCP 44, DSCP 32 and DSCP 00) on each DUT. Spirent SmartBits 600 test traffic generator was used to generate 100 UDP packet flows with QoS priority bits configured as follow: 25 flows with Telnet (port 23) traffic for DSCP 56 (highest priority), 25 flows with FTP (port 21) traffic for DSCP 44, 25 flows with FTP-DATA (port 20) traffic for DSCP 32 and 25 flows with HTTP (port 80) traffic for DSCP 00. For the QoS/NAT/ACL tests, in addition to the QoS configuration as described above, the engineers configured 100 static IP addresses translation (NAT) on each DUT, along with 100 ACLs (first 99 blocks and last filter accepting any IP traffic) based on source and destination IP addressing scheme. For the QoS/IPSec VPN/Stateful firewall tests, in addition to the QoS configuration as described earlier, engineers configured a single site-to-site IPSec VPN tunnel with 3DES encryption and SHA1 authentication enabled and 10 stateful firewall rules (first 9 blocks and last rule accepting any IP traffic). Each DUT was connected backto-back with an identical router using either two or four T1 WAN links, as appropriate for the test case. The LAN ports on the DUT at each end were connected to the appropriate number of 10/100/1000 Ethernet ports on the SmartBits The Tolly Group Page 5

6 See Figure 4 for the test bed illustration. Spirent SmartFlow (ver. 4.60) was used to configure the tests. In the SmartFlow setup, the test duration was set as 60 seconds, acceptable frame loss percentage as less than or equal to 0.001%. Binary search algorithm was used. Using these settings in SmartFlow and the DUT configured as required for the desired WAN scenario, each test was run three times, and the results averaged to obtain the final throughput. Equipment Acquisition and Support The Cisco Integrated Service Routers with latest hardware versions (Models: 1841, 2811 and 2821) were acquired through normal product distribution channels. The Tolly Group contacted executives at Cisco Systems and invited them to provide a higher level of support than available through normal channels. Cisco never responded to the offer. The Tolly Group gratefully acknowledges the providers of test equipment used in this project. Vendor Product Web address Spirent Communications SmartBits Spirent Communications SmartFlow ver Terms of Usage USE THIS DOCUMENT ONLY IF YOU AGREE TO THE TERMS LISTED HEREIN. This document is provided, free-of-charge, to help you understand whether a given product, technology or service merits additional investigation for your particular needs. Any decision to purchase must be based on your own assessment of suitability. This evaluation was focused on illustrating specific features and/or performance of the product(s) and was conducted under controlled, laboratory conditions and certain tests may have been tailored to reflect performance under ideal conditions; performance may vary under real-world conditions. Users should run tests based on their own real-world scenarios to validate performance for their own networks. Commercially reasonable efforts were made to ensure the accuracy of the data contained herein but errors and/or oversights can occur. Project Profile The test/audit documented herein may also rely on various test tools the accuracy of which is beyond our control. Furthermore, the document relies on certain representations by the sponsor that are beyond our control to verify. Among these is that the software/hardware tested is production or production track and is, or will be, available in equivalent or better form to commercial customers. The Tolly Group provides a fee-based service to assist users in understanding the applicability of a given test scenario to their specific needs. Contact us for information. When foreign translations exist, the English document is considered authoritative. To assure accuracy, only use documents downloaded directly from The Tolly Group's Web site. Sponsor: Tasman Networks, Inc. Document number: Product Class: WAN router Products under test: Tasman Networks 1004 OS Ver 8.2.1/ BootROM Ver. T1k Tasman Networks 1002 OS Ver 8.2.1/ BootROM Ver. T1k Cisco 1841 Integrated Services Router OS Ver T1/ BootROM Ver. 12.3(8r)T8 Cisco 2811 Integrated Services Router OS Ver T1/ BootROM Ver. 12.3(8r)T7 Cisco 2821 Integrated Services Router OS Ver T1/ BootROM Ver. 12.3(8r)T7 Testing window: September 2005 Software status: Generally available For more information on this document, or other services offered by The Tolly Group, visit our World Wide Web site at send to sales@tolly.com, call (561) Information technology is an area of rapid growth and constant change. The Tolly Group conducts engineering-caliber testing in an effort to provide the internetworking industry with valuable information on current products and technology. While great care is taken to assure utmost accuracy, mistakes can occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental, and consequential damages which may result from the use of information contained in this document. All trademarks are the property of their respective owners. The Tolly Group doc rev. dmk 05 Dec The Tolly Group Page 6