T H E TOLLY G R O U P No. 206168 December 2006 Blade Network Technologies, Inc. Nortel Layer 2/3 10Gb Uplink Ethernet Switch Module & Nortel Layer 2/3 10Gb Ethernet Switch Module for IBM BladeCenter Competitive Performance Evaluation versus Cisco Catalyst 6509 Switch Test Summary Premise: The migration of voice, video, storage and data services onto a single network backbone places a premium on high performance and mission-critical availability of servers supporting applications and services. Blade server systems offer a compelling option for enterprise data centers by consolidating server resources in a centralized location while reducing costs for power, cooling, and data center footprint. Blade switches that integrate into the blade server chassis further enhance the value proposition if they are able to match or exceed the high performance, reliability and rich functionality offered by traditional external switches. B lade Network Technologies, Inc. (BLADE) commissioned The Tolly Group to evaluate the Layer 2/3 performance of Nortel Layer 2/3 10Gb Uplink Ethernet Switch Module (ESM) with three 10GbE ports and Layer 2/3 10Gb Ethernet Switch Module with 20 10GbE ports designed and manufactured by BLADE for IBM BladeCenter against Cisco s popular Catalyst 6509 Switch equipped with comparable switch modules. (BLADE was formerly a division of Nortel and is now an independent company licensed to use the Nortel brand.) Tests measured the Layer 2/3 throughput, latency and multicast performance of the switch modules in a variety of common network topologies. The Tolly Group also compared the price/ performance of the switch modules. Tests were conducted in October 2006. GH%0$#H=$1'I,'3">&'JC1&K (++,.-, -+, )-, +, /0%1&2'3)45 (+67'89:; E*'7D1&A -()'7D1&A (L-(M'7D1&A /0%1&2'3)45 (+67'<=2">? 89:; 30% 3CD&%')'GH%0$#H=$1 Test Highlights Exhibits up to 9X less latency than the Cisco solution tested for multiple frame sizes at Layer 2 and Layer 3 Sustains line-rate throughput with 10x1GbE ports transmitting to 1x10-GbE port bidirectionally 10Gb ESM achieves line rate, zero-loss throughput (per RFC 2544) with 20x10-GbE full-mesh ports for frames of 64 to 1,518 bytes Outstanding 10 GbE performance and price/performance compared to Cisco Catalyst 6509 equipped with comparable switching modules Layer 2/3 Throughput Comparison of Nortel Layer 2/3 10 Gb Ethernet Switch Module and Layer 2/3 10 Gb Uplink Ethernet Switch Module vs. Cisco Catalyst 6509 51% 50% @"AB0 @C1C2DA1 E-+F /0%1&2'3)45 (+67'89:; /0%1&2'3)45 (+67'<=2">? 89:; 30% 3CD&%'5'GH%0$#H=$1 51% 50% @"AB0 @C1C2DA1 E-+F Note: * = manufactured by Blade Network Technologies. Nortel L2/3 10 Gb ESM was tested with 20 10GbE ports in full mesh, while Nortel L2/3 10 Gb Uplink ESM and Cisco Catalyst 6509 were tested with 10 GbE ports transmitting to one GbE port bidirectionally. Source: The Tolly Group, October 2006 Figure 1 2006 The Tolly Group Page 1
Executive Summary IBM BladeCenter with a Nortel Layer 2/3 10Gb Uplink Ethernet Switch Module or a Nortel Layer 2/3 10Gb Ethernet Switch Module offers significantly better performance, as well as price/performance ratio, versus the Cisco Catalyst 6509 solution tested. Blade Switches make the cut Blade servers are making inroads into enterprise data centers by promising high server density, low data center footprint, and lower costs of power and cooling. Blade server systems are increasingly deploying Gigabit Ethernet and 10 Gigabit Ethernet (10GbE) connectivity, and this necessitates the availability of high bandwidth uplinks to the network core. 10GbE is moving further into the access layer and is a prerequisite for multi-core environments, bandwidth hungry applications, iscsi SAN, and link consolidation. While external chassis-based switching solutions from leading vendors like Cisco are commonplace in enterprise networks, these solutions do not always perform at wire speed and do not always offer low latency. IBM BladeCenter offers two high performance, low latency 10GbE switching solutions designed and built by BLADE. The Nortel Layer 2/3 10Gb Uplink Ethernet Switch Module (10Gb Uplink ESM) provides 10GbE connectivity to IBM BladeCenter chassis, with three external 10GbE uplink ports and one SFP GbE external port with 14 internal GbE ports. The Nortel Layer 2/3 10Gb Ethernet Switch Module (10Gb ESM) provides 10GbE connectivity to each server blade in a BladeCenter and BladeCenter H chassis by combining six external 10GbE ports with 14 internal 10GbE ports. Tests showed that these 10GbE switching solutions from BLADE for the IBM BladeCenter consistently outperformed the Cisco Catalyst 6509 configured with similar switch modules. The Cisco solution as tested did not use the optional DFC3 daughter card required to obtain the maximum performance claimed by Cisco. This was done in order to assess the out-of-box performance of all the devices under test. 3C/0>BD2E!A0BF %+ )+ *+ $+!+ +!"# $"( )"*,-./0123$4*!+562789: #)G6D/0A %!$G6D/0A!H%!&G6D/0A $"% $"% #"$,-./0123$4*!+562;<1=>? 789: 3CD0.2$23C/0>BD!!"&!'"( *'"% @=AB- @C/C1DA/2#%+' The 3x10G test was used to prove that 10-Gbps line rate is maintained when connecting the switch using three 10GbE links, for example, to two upstream core 10GbE switches for redundancy, and another 10GbE link connects to another blade chassis with a 10GbE switch. The One 10GbE port to 10 GbE ports bidirectional test proves that 10 streams at 1 Gbps from the blade servers can go out a single 10GbE link at line rate, and traffic from the network can come in a 10GbE link and out to the 10 blade servers at 1 Gbps line-rate. The 20 10GbE ports in fullmesh test on the Nortel 10Gb ESM shows that it delivers linerate throughput at the maximum network load. Latency Comparison @ 25% Line Rate of Nortel L2/3 10Gb Ethernet Switch Module and L2/3 10Gb Uplink Ethernet Switch Module vs. Cisco Catalyst 6509 (Lower numbers are better)!"# $"( )"*,-./0123$4*!+562789: $"% $"% #"!!!"'!'"( *'"#,-./0123$4* @=AB-!+562;<1=>? @C/C1DA/2#%+' 789: 3CD0.2*23C/0>BD Note: * = manufactured by Blade Network Technologies. Nortel L2/3 10 Gb ESM was tested with 20 10GbE ports in full mesh, while Nortel L2/3 10 Gb Uplink ESM and Cisco Catalyst 6509 were tested with 10 GbE ports transmitting to one GbE port bidirectionally. Source: The Tolly Group, October 2006. Figure 2 2006 The Tolly Group Page 2
These tests, combined with the price/performance analysis compared to chassis-based switching solutions like the Cisco Catalyst 6509, illustrate that integrated switching within the IBM Blade- Center chassis will not adversely impact end-to-end performance and, in many cases, improves performance. Cisco was invited to participate in the testing as per The Tolly Group s Fair Testing Charter (http://www.tolly.com/ftc.aspx) but Cisco s representatives did not respond to the invitation. Layer 2 Throughput and Latency Three 10GbE ports in full mesh In this test, the 10Gb Uplink ESM achieved zero-loss throughput (<=0.001% acceptable frame loss) for all standard % Line Rate 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Nortel L2/3 10Gb ESM* frame sizes (64 to 1,518 bytes) tested, with an average storeand- forward latency ranging from 1.86 µsec to 1.93 µsec. On the other hand, a similarly equipped Cisco solution only achieved a throughput ranging from 29.3% for 64-byte frames to 85.9% for 512-byte frames to 81.3% for 1,518-byte frames. The latency ranged from 8.1 µsec to 10.1 µsec for various frame sizes. See Figure 5 for detailed results. 10 GbE to One 10GbE Ports Bidirectionally Once again, the 10Gb Uplink ESM achieved zero-loss (<=0.001% acceptable frame loss) throughput for all frame sizes tested, while the Cisco Catalyst could only achieve throughput ranging from 30.2% to 52.3% for various frame sizes tested. Layer 2 Multicast Forwarding Throughput Comparison Nortel L2/3 10Gb Uplink ESM* 64 512 1518 Cisco Catalyst 6509 Note: * = manufactured by Blade Network Technologies. Nortel L2/3 10 Gb ESM was tested with one 10GbE port transmitting to 19 10GbE ports unidirectionally, while Nortel L2/3 10 Gb Uplink ESM and Cisco Catalyst 6509 were tested with one 10GbE port transmitting to 13 GbE ports transmitting to one GbE port bidirectionally. Source: The Tolly Group, October 2006 Figure 3 Blade Network Technologies, Inc. NorteI Layer 2/3 10Gb Ethernet Switch Modules for IBM BladeCenter Layer 2/3 Performance vs. Cisco Catalyst 6509 Product Specifications Vendor-supplied information not necessarily verified by The Tolly Group Blade Network Technologies, Inc. Nortel Layer 2/3 10Gb Uplink Ethernet Switch Module Uplinks: 4 ports: 3 10G (2 CX4, 1 XFP) and 1 1G RJ-45 SFP) for mgmt. or data Internal ports: 2 ports: 100Base-T management ports 14 ports: 1000Base-CX to servers Full line-rate performance: 90 Gbps full duplex Nortel L2/3 10Gb Ethernet Switch Module Uplinks: 7 ports: 6 10Gb (XFP) and 1 1Gb RJ- 45 for management or data Internal ports: 2 ports: 100Base-T mgmt. ports 14 ports: 10Gb to servers Full line-rate performance: 400 Gbps Common Software Features Availability/Resiliency: Ready for mission-critical applications VLANs: Customizable Virtual LAN support Traffic Management and Routing: Optimized for best performance Security: Filtering, HTTPS, SSH, RADIUS CLI: Alteon OS and Cisco-like CLI Cluster Management: Cluster of switches can be managed from one central location Secure Management: HTTPS, SSH v1/v2, SNMP v1-3 and more For more information contact: Blade Network Technologies, Inc. 2350 Mission College Blvd. Suite 600 Santa Clara, CA 95054 http://www.bladenetwork.net 2006 The Tolly Group Page 3
The average store-and-forward latency for the 10Gb Uplink ESM ranged from 2.5 µsec to 6.2 µsec, while the Cisco Catalyst experienced latency ranging from 11.8 µsec to 39.5 µsec. Store-andforward latency measures latency as the time from when the last bit of the frame enters the input port, to the time when the first bit of the frame exits the output port. See Figure 5 for detailed results. 20 10GbE ports in full mesh - RFC 2544 test This test topology was only tested for the 10Gb ESM due to the lack of a comparable switching solution from Cisco. The tests showed that the 10Gb ESM achieved throughput (absolute zero loss) for all the frame sizes tested, with the average LILO latency ranging from 1.6 µsec to 4.25 µsec. See Figure 5 for detailed results. LILO latency calculates the latency as the time from when the last bit of the frame enters the input port to the time when the last bit exits the output port. Layer 3 Throughput and Latency Three 10GbE ports in full mesh This test pitted the 10Gb Uplink ESM against a similarly equipped Cisco Catalyst 6509. Tests once again showed that the 10Gb Uplink ESM achieved zero loss (<=0.001% acceptable frame loss) for all frame sizes tested. In contrast, the Cisco device could only achieve between 29.3% to 85.3% for the various frame sizes. The average store-and-forward latency for the 10Gb Uplink ESM was only 1.84 µsec to 1.93 µsec for the various frame sizes. The 1;<=CF 1(33 14-3 1433 1/-3 1/33 1.-3 1.33 1-3 16 Price/Performance Comparison of Nortel and Cisco Products Tested 1/(233 78%9&: */;4'.3<='>?@A' Source: The Tolly Group, October 2006 Cisco Catalyst latency ranged from 8.2 µsec to 10.1 µsec. See Figure 5 for detailed results. 10 GbE to One 10GbE Ports Bidirectionally Once again, the 10Gb Uplink ESM achieved zero-loss (<=0.001% acceptable frame loss) throughput for all frame sizes tested, while the Cisco Catalyst only achieved 30.2% to 52.3% throughput. The store-and-forward latency for the 10Gb Uplink ESM ranged between 2.5 µsec to 6.1 µsec for the various frame sizes while the Cisco Catalyst latency ranged from 11.9 µsec to 39.6 µsec. See Figure 5 for detailed results. 20 10GbE ports in full mesh - RFC 2544 Test Once again, the tests showed that the 10Gb ESM achieved throughput (absolute zero loss) for all frame sizes tested, while the average LILO latency ranged from 1.6 µsec to 4.3 µsec. 1--2-( 78%9&:' */;4'.3<='BC:"+D >?@A 145- E"FG8 EH9H:IF9',-3J A'K&F"#+&L'H+L'MH+$NHG9$%&L'=I'O:HL&'7&9P8%D'Q&GR+8:8#"&FS'T+G2 Figure 4 Layer 2 Multicast forwarding 1 GbE port transmits to 13 GbE & 3 10GbE ports Multicast data was transmitted from one GbE port to a set of 13 GbE ports and three 10GbE ports acting as receivers. This test simulates a 1 to 16 fan-out with Layer 2 unidirectional multicast traffic with 1-Gbps test traffic of the standard Ethernet frame sizes between 64 bytes to 1,518 bytes. Both the 10Gb Uplink ESM, as well as the Cisco Catalyst 6509 solution, achieved throughput for all frame sizes. One 10GbE port transmits to 19 10GbE ports In this test, one 10GbE port transmits line-rate 10 Gbps unidirectional multicast traffic to a set of 19 10GbE receiver ports, for a total of 190 Gbps of multicast traffic. This test was performed only on the 10Gb ESM 2006 The Tolly Group Page 4
Source: The Tolly Group, October 2006 Figure 5 2006 The Tolly Group Page 5
due to the lack of a comparable switching solution from Cisco. Test results showed that the 10Gb ESM achieved line-rate throughput for all the frame sizes tested. Price-Performance Comparison This test evaluated the priceperformance ratio of the DUTs in terms of US Dollars per Gbps ($/ Gbps) of theoretical maximum throughput as claimed by the respective vendors. For this comparison, engineers followed a scenario where a user with an existing IBM BladeCenter blade server examined the cost of adding a 10GbE module to that device, versus deploying a standalone chassis-based switch outfitted with a 10GbE module, such as the Catalyst 6509. Based upon the scenario above, pricing for the BladeCenter solution involves a 10GbE module. For the Catalyst 6509 solution, pricing takes into account the cost of a Catalyst 6509 chassis, a 10GbE module, plus the cost of a a separate GbE port module to match the port configuration of the Blade server. Pricing also includes a Catalyst 6509 Supervisor 720 module. The prices of the Nortel 10GbE switching modules were obtained from IBM, while those for the Cisco Catalyst equipment were obtained directly from Cisco. The prices used here were the manufacturer suggested list prices in order to simplify the comparison. The 10Gb ESM offered the best price-performance ratio of the DUTs, with $24 per Gbps, followed by the 10Gb Uplink ESM at $55.54 per Gbps. The Cisco Catalyst solution as tested cost $375 per Gbps of throughput. See Figure 5 for detailed pricing used in these calculations. Test bed setup & methodology The test bed consisted of the DUT and a traffic generator to generate the test traffic as appropriate. See Figure 6 for the test bed diagram. Layer 2/3 Performance Test Bed Source: The Tolly Group, October 2006 Figure 6 The DUTs were Nortel Layer 2/3 10Gb Uplink Ethernet Switch Module (10Gb Uplink ESM), the Nortel Layer 2/3 10Gb Ethernet Switch Module (10Gb ESM), and the Cisco Catalyst 6509 Switch. The 10Gb Uplink ESM was configured as a custom-made test board with three 10GbE and one SFP GbE external ports along with 14 internal GbE ports, running OS version 1.0.1.1. The test board made the internal ports accessible as physical ports to link to the traffic generators. The 10Gb ESM was also configured as a custom-test board with six 10GbE external ports, and 14 internal 10GbE ports, running a beta OS since the device was not yet publicly available. The Cisco Catalyst 6509 switch was outfitted with a WS-X6704-10GE switch module with four 10GbE ports, a WS-X6148-GE- TX switch module with 48 10/ 100/1000 Base-T ports and a WS-SUP720 Supervisor Engine, running IOS version 12.2(18)SXD7a. Spirent TestCenter SPT-9000A test traffic generator was used with EDM-2003A (12-port Dual Media GbE) and MSA-2001A (two-port 10GbE) test modules, with TestCenter application version 1.30.1325. Tests also used a Spirent Smart- Bits 6000C test traffic generator with TeraMetrics LAN-3325A, XFP-3731A and XLW-3721A modules, along with the SmartFlow (ver 5.5) and SmartMulticastIP (ver 2.6) applications to generate the unicast and multicast test traffic, respectively. Test tool accuracy The Spirent TestCenter 9000A solution, using MSA-2001A 2006 The Tolly Group Page 6
modules, has been tested to have extremely low inherent latency (less than 100 nanoseconds) as measured by the back-to-back port latency test using two MSA- 2001A boards handling traffic at 99% of line rate. This allows latency to be measured with a single decimal microsecond of precision. Layer 2 Throughput and Latency Tests The tests were performed with test traffic consisting of one flow per port, consisting of frames of 64, 128, 256, 512, 1,024, 1,280 and 1,518 bytes, transmitted at the appropriate rate for each test scenario. A binary search algorithm was used to find the maximum throughput and store-andforward latency. The acceptable frame loss was set at < 0.001%. Tests were run three times, and the results averaged to obtain the published numbers. All the ports on the DUT were placed in one VLAN. Three 10GbE ports in full mesh For this test, the DUTs were the TOLLY INSIGHTS Nortel Layer 2/3 10Gb Uplink ESM and the Cisco Catalyst 6509. Three 10GbE ports on the DUT were connected to the 10GbE ports on the SmartBits 6000C. Test traffic of the chosen frame sizes was transmitted bidirectionally, with the ports configured in a full-mesh configuration. Latency tests were performed using the Store-and- Forward method. 10 GbE to One 10GbE Ports Bidirectionally The DUTs were the 10Gb Uplink ESM and the Cisco Catalyst 6509. On the 10Gb Uplink ESM, 10 internal GbE ports transmitted data bidirectionally to one external 10GbE port to simulate a backbone configuration. On the Cisco Catalyst 6509, 10 GbE ports (port numbers 1, 2, 9, 10, 23, 24, 25, 26, 47 and 48) on the WS-X6148-GE-TX module and one 10GbE module on the WS-X6704-10GE module were connected to the appropriate GbE and 10GbE ports on the Smart- Bits 6000C. The GbE ports were distributed in order to improve Blade servers and blade switches - Made for each other Blade servers promise to consolidate multiple physical servers into a single, small-footprint, easily manageable chassis solution to save on valuable data center space, power and cooling requirements. Typical blade server deployments in the data center serve applications like high performance cluster computing, iscsi SAN, etc. which require high bandwidth uplinks and inter-server communication links. Consolidating the network intelligence inside a multi-layer switching and routing module that could be plugged in a blade server chassis is the logical next step in realizing the promise of blade servers towards consolidating network resources. Blade servers and blade switching modules delivering 10GbE uplinks and data ports with Layer 2-7 intelligence are made for each other. the traffic distribution across Cisco s switch fabric. Additionally, CEF (Cisco Express Forwarding) was enabled and configured to use the default perdestination load sharing algorithm. The test traffic configuration in the SmartFlow consisted of the chosen frame sizes, with the ports configured in a backbone configuration. Latency tests used the Store-and-Forward method. 20 10GbE ports in full mesh - RFC 2544 test In this test, the only DUT was the 10Gb ESM. The six external 10GbE ports and the 14 internal 10GbE ports on the DUT were connected to appropriate 10GbE test ports on the Spirent TestCenter 9000A traffic generator. The test traffic consisted of all the RFC 2544 standard Ethernet frame sizes, with absolute zero acceptable frame loss. The test ports were configured in a full mesh configuration, and transmitted traffic bidirectionally. The latency tests used the Last In Last Out (LILO) algorithm. Layer 3 Throughput and Latency Tests This set of tests used the same DUTs, network topologies, test traffic mix and test bed setup as the corresponding Layer 2 tests mentioned previously with the exception that the DUTs were configured to forward the traffic at Layer 3, with each of their ports in a separate VLAN and a different IP subnet. Layer 2 Multicast Forwarding Tests This set of tests used the same set of frame sizes of 64, 128, 256, 512, 1,024, 1,280 and 1,518 bytes with unidirectional Layer 2 multicast forwarding with IGMP disabled on the DUTs. 2006 The Tolly Group Page 7
1 GbE port transmits to 13 GbE & 3 10GbE ports The DUTs for this test were the Nortel 10Gb Uplink ESM and the Cisco Catalyst 6509. The DUT forwarded the traffic based 10GbE ports acting as the receivers. On the Cisco Catalyst, one GbE port on the WS-X6148-GE-TX module acted as the transmitter, while 13 other GbE ports on the same module, and three 10GbE ports on the WS- X6704-10GE module acted as the receivers. Spirent Smart- MulticastIP test application and SmartBits 6000C test traffic generator were used to generate the traffic. One 10GbE port to 19 10GbE ports The test was the performed only on the Nortel 10Gb ESM, due to lack of similar switching modules with 20 GbE ports from competing vendors. One internal 10GbE port was configured as the transmitter and five other internal 10GbE ports and 14 external 10GbE ports acted as the receivers. These ports were connected to the appropriate 10GbE ports on the Spirent TestCenter 9000A chassis and TestCenter application was used to configure the test streams. The DUT forwarded the traffic based on the Layer 2 multicast addresses, and IGMP was disabled. The Tolly Group is a leading global provider of thirdparty validation services for vendors of IT products, components and services. The company is based in Boca Raton, FL and can be reached by phone at (561) 391-5610, or via the Internet at http://www.tolly.com, sales@tolly.com Test Equipment/Software Summary The Tolly Group gratefully acknowledges the providers of test equipment/software used in this project. Vendor Product Web Spirent Communications Spirent TestCenter 9000A Spirent TestCenter ver 1.30.1325 Spirent SmartBits 6000C Spirent SmartFlow 5.5 Spirent SmartMulticastIP 2.6 http://www.spirentcom.com Terms of Usage USE THIS DOCUMENT ONLY IF YOU AGREE TO THE TERMS LISTED HEREIN. This document is provided, free-of-charge, to help you understand whether a given product, technology or service merits additional investigation for your particular needs. Any decision to purchase must be based on your own assessment of suitability. This evaluation was focused on illustrating specific features and/or performance of the product(s) and was conducted under controlled, laboratory conditions and certain tests may have been tailored to reflect performance under ideal conditions; performance may vary under real-world conditions. Users should run tests based on their own real-world scenarios to validate performance for their own networks. Commercially reasonable efforts were made to ensure the accuracy of the data contained herein but errors and/or oversights can occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental and consequential damages which may result from the use of information contained in this document The test/audit documented herein may also rely on various test tools the accuracy of which is beyond our control. Furthermore, the document relies on certain representations by the sponsor that are beyond our control to verify. Among these is that the software/hardware tested is production or production track and is, or will be, available in equivalent or better form to commercial customers. When foreign translations exist, the English document is considered authoritative. To assure accuracy, only use documents downloaded directly from The Tolly Group s Web site. All trademarks are the property of their respective owners. 206168-JWFTW-KK-12Feb07 2006 The Tolly Group Page 8