TOLLY. No June Functional Verification and Performance Evaluation. Windows 2000 with IIS5 1 Web Server Performance.

Transcription

1 Premise: Web content providers seeking faster end-to-end performance are adopting Web acceleration devices to decrease client-side response times and ease overall strain on Web servers. While caching devices may help with static content, Web-based retailers rely on dynamically created Web pages to power their E-commerce sites. Web acceleration appliances can improve end-user transaction response time for both modem and broadband users, while decreasing the overall workload on the Web server. Redline Networks, Inc. commissioned The Tolly Group to evaluate its T X 21 Real-Time Acceleration Appliance, a purposebuilt device that intercepts HTTP requests from browser-based clients before they reach a Web server and accelerates Web page downloads by using a blend of compression, multiplexing of client requests over persistent HTTP connections and TCP processing offload to speed server response times. Tests were performed in February 21. Results show that the T X 21 dramatically improves end-to-end Web performance for client stations attempting to reach backend servers. Test results consistently show improved Web performance across test scenarios that employ popular Windows and Unix servers. T H E TOLLY G R O U P No June 21 Redline Networks, Inc. T X 21 Real-Time Acceleration Appliance Functional Verification and Performance Evaluation Speeds delivery of Web pages by up to 11X Accelerates connection time to a Web server from 1.4X to 18X Increases Web server capacity for error-free transmission by up to 2X on a Sun Enterprise 45 Web server running Apache Doubles the number of error-free connections per second supported by a Dell PowerApp.web 12 Web server running Internet Information Server 5. Time (seconds) Average HTTP Page Download Times 25 (Lower numbers represent faster downloads) Throughput Mbit/s Test Highlights A B C Products Test Summary Windows 2 with IIS5 1 Web Server Performance Number of Series1 HTTP GET Series2 requests per Series3 second 21 The Tolly Group Page Server baseline With T X 21 1 Microsoft Corp. Internet Information Server 5 Web server software Source: The Tolly Group, June 21 Figure 1

2 The Tolly Group Redline Networks, Inc. T X 21 7 Solaris 2.8 with Apache Web Server Running on a Sun Enterprise 45: Client Time to Connect with Server (Lower numbers represent faster connect times) 6 5 Time (seconds) New connections/second w ithout T X 21 w ith T X 21 Source: The Tolly Group, June 21 Figure 2 Tolly Group engineers established baseline results on a pair of Web servers: a Dell PowerApp.web 12 running Windows 2 Advanced Server and Microsoft s Internet Information Server, and a Sun Enterprise 45 running Solaris and Apache Web server software. Prior to benchmarking the T X 21, Tolly Group engineers first verified the functionality of Blitz, a load-generation test tool created by Redline Networks to generate load against Web servers and measure response time with greater accuracy than other commercially available tools. Blitz accurately simulates clients interacting with the server across links of various bandwidths (56 Kbit/s, ISDN, DSL, etc.), which is important in creating real-world demand on server resources. Tolly Group engineers validated the test tool by running a battery of stress tests on Blitz and found it to be capable of producing valid results when simulating real user load to stress and performancetest a Web server. Engineers subsequently used Blitz to generate Web server load conditions in the performance evaluation of the T X 21. Results Performance when Paired with a Dell PowerApp.web 12 Web Server The Tolly Group tested Web server performance by monitoring the number of concurrent connections built up over the life of the test and the number of pages served per second. Prior to activating the T X 21, engineers first established baseline performance results for each of the Web servers tested, in this case the Dell PowerApp.web 12, an 866-MHz Pentium III-class Web server with 128 Mbytes of memory, a 1/1 Mbit/s Fast Ethernet interface, and Windows 2 Advanced Server version SP1 and Internet Information Server version 5. Baseline tests of the Dell PowerApp.web 12 demonstrated connection times between.8 seconds and.9 seconds, time to first byte ranged from.431 seconds to 14 seconds, and total download times ranged from 9.8 seconds to 37.4 seconds for the test Web pages. (Time to first byte is the elapsed time from an HTTP GET request to the requestor receiving the first byte of data.) 21 The Tolly Group Page 2

3 The Tolly Group Redline Networks, Inc. T X 21 These average download times are derived from average download rates for a variety of link speeds including DSL, 56 Kbit/s and T1 connections. The Dell server transmitted pages error free through 15 connections per second, and ceased serving pages above 25 connections per second. Next, we activated the T X 21 in conjunction with the Dell server environment. Engineers observed almost a 1X acceleration of the connection time from the baseline, as connection times ranged between.8 and.95 seconds. The longest time to first byte was.292 seconds, or nearly 5 times faster than the 14 seconds in the baseline. Engineers observed marked improvement in Web page download times, which were between 1.9 seconds and 3.6 seconds with the T X 21, almost 2 times faster than the 37 seconds achieved in the baseline without the Redline device activated (see figure 1). With the T X 21 active, the Dell server continued to serve Web pages through 3 connections per second (at which point the 1-Mbit/s server NIC reached its saturation point), without error a 1% boost over the baseline results. Performance when Paired with a Sun Enterprise 45 Web Server Following the tests of the T X 21 alongside a popular Windows-based Web server, engineers next tested the device as it operated alongside an enterprise-class Unix server. As with the Dell server, engineers first established baseline operating results for the Sun Enterprise 45, a 3-MHz, quad-ultrasparc processor machine with 512 Mbytes of memory, a 1/1 Fast Ethernet interface and the Solaris 2.8 operating system running Apache Web Server software. Baseline tests of the Sun Enterprise 45 showed connection times that ranged from.9 seconds to 6., time to first byte that ranged from.45 seconds to 6.6 seconds, and total download times that ranged from 6.8 seconds to 1 seconds for the test Web pages. The server transmitted pages error free through 1 connections per second, and ceased serving pages above 75 connections per second. With the baseline results established, we activated the T X 21. We observed connection times improve to between.7 seconds to.5 seconds, which is a vast improvement over the baseline of up to six seconds for a connection (see figure 2). The interaction of the T X 21 reduced time to first byte by as much as 7% over baseline results, with data flowing between.2 seconds and 5.8 seconds. Engineers reported savings on the download times up to 72% versus the baseline, with the T X 21 achieving download times between 1.9 seconds and 9.8 seconds. With the T X 21 active, the server continued transmitting pages through 2 connections per second without error and continued to 3 connections per second with only.25% error (at which point the 1-Mbit/s server NIC reached its saturation point). See figure 3. This represents up to 2 times more connections served per second over the baseline. Redline Networks, Inc. T X 21 Real-Time Acceleration Appliance Functionality and Performance T X 21 Real-Time Acceleration Appliance Product Specifications* Features Improves end-to-end Web performance Increases by up to 2X a Web server s capacity to serve pages and support users Accelerates download of static and unique content to modem or broadband users by up to 1,11% Accelerates connect time by up to 5,8% Cuts bandwidth costs in half Decreases network latencies Decreases OPEX and CAPEX Increases density of Web hosting Dimensions: 2u, 19-inch appliance Weight: 23 lbs. 1/1 Mbit/s or Gigabit Ethernet interface Number of target hosts: 1 to 64 Operating temperature: F (5-35 C) Thermal output: 5 BTU/hr Relative humidity: 1%-9% non-condensing Dual hot-swappable 3W 1-24 VAC 6/5 Hz power supplies For more information contact: Redline Networks, Inc. 191 S Bascom Ave #8 Campbell, CA (phone) (fax) *Vendor-supplied information not verified by The Tolly Group 21 The Tolly Group Page 3

4 The Tolly Group Redline Networks, Inc. T X 21 Analysis At first glance, the Web acceleration derived from Redline Networks T X 21 may seem unbelievable. However, take into account the company s architectural approach to Web acceleration and you can begin to understand how the device generates such broad savings for Web page download rates, number of users supported and time to connect to the Web server. In effect, the T X 21 which sits in front of a Web server (or group of servers) acts as a front-end processor by off-loading TCP setup and teardown tasks from the Web server, relegating the server to handling HTTP GET requests received from the T X 21. This amounts to an enormous benefit for the servers, freeing them from dealing with each downstream client that comes in at a different speed ranging from slow dial-up rates to T-1, and freeing them from adjusting flow control to accommodate the various clients. Instead, the Web server communicates at Fast Ethernet rates over connections to the T X 21. Moreover, the server no longer handles the SYN-ACK negotiation to set up and tear down TCP connections, since that function now is offloaded to the T X 21. This relieves the Web server of an enormous processing burden. The T X 21 maintains a minimum number of persistent TCP connections between itself and the server, and between itself and downstream clients. Then, it multiplexes HTTP requests over those persistent connections to the server which returns data to the T X 21. The T X 21 then Error-Free Transmission Rate (Higher numbers represent more error-free transmissions) New connections/second determines which data streams need compression and blasts them back to the appropriate clients. The results, in our testing, were staggering. Testing of the T X 21 in conjunction with the Dell PowerApp.web 12 server demonstrated that the T X 21 achieves a faster time to first byte by 21% to 4,8%, and accelerates total download time by 51% to 1,13%. Also, the T X 21 extends the error-free transmission rate of the Dell PowerApp.web 12 server from 15 connections per second to 3 connections per second. 3 1 Dell PowerApp.web 12 Server Baseline w ithout T X 21 Testing of the T X 21 in conjunction with the Sun Enterprise 45 Web server concluded that the T X 21 achieves a faster time to connect by up to 5,8%, accelerates time to first byte by up to 33%, and decreases total download time by as much as 1 Sun Enterprise 45 Server T X 21 activated At 3 connections/sec, the NIC on the Dell server became saturated. 2 With T X 21, the Sun server supported up to 3 connections/sec (at which point the server NIC was saturated) with just a.25% error rate. Source: The Tolly Group, June 21 Figure 3 36%. Also, the T X 21 extends the error-free transmission rate of the Web server from 1 connections per second to 2 connections per second. Test Configuration and Methodology For this test, engineers used two separate Web servers. The device under test, the Redline Networks T X 21, was connected inline between the Web server under test and the Web load-generating tool, Blitz. All tests were run using an evenly distributed sampling of Web pages ranging from 22 Kbytes to 48 Kbytes in size. Also, the Blitz test tool simulated three different connection speeds to simulate live connection types: 12% of the test connections were set at 384 Kbit/s, 28% of the test connections were set at 28 Kbit/s, and 6% were set at The Tolly Group Page 4

5 The Tolly Group Redline Networks, Inc. T X 21 Blitz Test Tool Blitz is a testing tool developed by Redline Networks that generates load against Web servers and measures response time. Unlike other test tools that use a fixed number of simulated clients to make requests, Blitz is designed to allow the rate of requests to be fixed, and automatically will vary the number of simulated clients to maintain the desired request rate. Often, when a fixed number of clients is used, the server actually will throttle the load-testing tool because Web servers respond more slowly under load and new requests are dependent upon completion of the previous request. In the traditional fixedclient model, the rate of requests per second decreases as the server takes longer to respond. The result is test harness throttling and inaccurate results. Thus, it is important in a load-testing tool that connection rate is maintained as occurs in the real world independent of previous request time. Blitz provides this faculty. A second differentiating characteristic of Blitz is that it accurately simulates multiple groups of clients at various bandwidths (56 Kbit/s, ISDN, DSL, etc.) since it uses a kernel module to throttle below the TCP stack. Other testing tools which attempt to throttle at the application level cannot accurately simulate these slower devices because the TCP control packets (ACKs, SYNs, etc.) are sent out at the regular link speed (1 Mbit/s, etc.). The key is to delay the packets from reaching the IP stack so that the control packets flow out of the testing box at the slower link speed. This combination of features allows Blitz to exercise Web servers in a manner consistent with real-world scenarios. Moreover, Blitz s kernel module also allows packet losses/ drops and other delay artifacts to be modeled, thereby accounting for network anomalies common to WANs such as the Internet. Kbit/s. The Blitz test tool was set to run tests maintaining a constant user connection rate, and reported these results: time to connect, time to first byte, time to complete download and percentage of errors. All tests were run three times, and the results were averaged. The device under test was a Redline Networks T X 21 Real Time Acceleration Appliance. The T X 21 utilized an onboard 1 Base-T Ethernet adapter set to full duplex. The device under test was running RIOS.9 firmware during testing. The Blitz load-generation tool used during testing utilized an 85-MHz Pentium III processor, 128 Mbytes RAM, and an Intel 1/1 Ethernet adapter. Blitz was running a slightly modified kernel of the FreeBSD operating system. The Tolly Group engineers verified the functionality of the Blitz tool prior to testing the T X 21. One of the two Web servers tested in conjunction with the T X 21 was a Dell Computer Corp. PowerApp.web 12 with an 866- MHz Pentium III processor, 128 Mbytes RAM, and an 8255xbased PCI 1/1 network adapter. The Dell server was running Microsoft Corp. Windows 2 Advanced Server, Service Pack 1 operating system and Internet Information Server 5 Web server software. A second Web server used was a Sun Microsystems Enterprise 45 with four 3-MHz UltraSparc processors, 512 Mbytes RAM, and an on-board Sun 1/1 Ethernet adapter. The Sun server was running Solaris 8 operating system and the Apache Web server. All devices used in this test were connected to a Hewlett-Packard Co. Procurve 4M Ethernet switch equipped with four HPJ4111A 1/1 modules, four HPJ4115B 1/1 modules and a HPJ4121A management module (see figure 4). The switch was operating with firmware version C.8.3, and all ports were set to full duplex. An Acterna Corp. DA-35 DominoFastEthernet Internetwork Analyzer sat in-line with the Web server and the HP Procurve 21 The Tolly Group Page 5

6 The Tolly Group Redline Networks, Inc. T X 21 Test Bed Blitz test tool Redline Networks T X 21 Real-Time Acceleration Appliance Web Servers: Dell PowerApp.web 12 Sun Enterprise 45 Network Analyzer d i gi t a l Source: The Tolly Group, June 21 Figure 4 4M switch. A Dell Latitude CPx with a 5- MHz Pentium III, 64 Mbytes RAM and a Xircom Card Bus Ethernet 1 network adapter was used as a console for the DA-35. The Dell was running Windows 2 Professional, Service Pack 1 operating system and Domino NAS 2. software. Although it was not used to generate results, the DominoFE Internetwork Analyzer was used to monitor traffic for all tests. The Tolly Group gratefully acknowledges the providers of test equipment used in this project. Vendor Product Web address Acterna DominoFastEthernet Analyzer Since its inception, The Tolly Group has produced highquality tests that meet three overarching criteria: All tests are objective, fully documented and repeatable. We endeavor to provide complete disclosure of information concerning individual product tests, and multiparty competitive product evaluations. As an independent organization, The Tolly Group does not accept retainer contracts from vendors, nor does it endorse products or suppliers. This open and honest environment assures vendors they are treated fairly, and with the necessary care to guarantee all parties that the results of these tests are accurate and valid. The Tolly Group has codified this into the Fair Testing Charter, which may be viewed at Project Profile Sponsor: Redline Networks, Inc. Document number: Product Class: Web accelerator Products under test: T X 21 Real-Time Acceleration Appliance Testing window: February 21 Software versions tested: RIOS.9 Software status: Beta version (Version 1.1 is now generally available.) Additional information available: None For more information on this document, or other services offered by The Tolly Group, visit our World Wide Web site at send to info@tolly.com, call (8) or (732) Internetworking technology is an area of rapid growth and constant change. The Tolly Group conducts engineering-caliber testing in an effort to provide the internetworking industry with valuable information on current products and technology. While great care is taken to assure utmost accuracy, mistakes can occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental, and consequential damages which may result from the use of information contained in this document. All trademarks are the property of their respective owners. The Tolly Group doc rev. clk 26 Jun 1 21 The Tolly Group Page 6