TOLLY. No June Reflection for the Web v versus IBM SecureWay Host On-Demand v Web-to-Host Software Competitive Evaluation

Transcription

1 No June 2000 WRQ, Inc. Reflection for the Web v versus IBM SecureWay Host On-Demand v Web-to-Host Software Competitive Evaluation Premise: With the adoption of the Web browser as the client of choice for today s enterprise customers, Telnet 270 (TN270) is being replaced by Web-tohost (i.e., HTTP-to-TN270) software solutions built on the Java platform. These solutions offer several benefits over traditional, full-client solutions, including automatic distribution of software updates, platform independence and reduced desktop processing costs. However, with these benefits comes the requirement that Just in Time (JIT) distribution of applets be sufficiently rapid to ensure timely access to the target mainframe application. This is especially critical for users who must communicate with the source server across low-speed WAN connections. WRQ, Inc. commissioned The Tolly Group to benchmark the performance of its Web-to-host product, Reflection for the Web version 4.0. and compare the results to IBM Corp. s SecureWay Host On-Demand version Tests included delivery duration over a 10 Mbit/s Ethernet LAN, a 56 Kbit/s WAN connection and a 28.8 Kbit/s dial-up WAN connection. The tests measured the applet download time for the first connection; the time it takes to establish subsequent connections; and the time that elapses before the end-user is aware of download activity on the screen during the first connection. Testing was performed in June T H E TOLLY G R O U P Test Highlights Test Summary! Exhibits faster applet download times than its competitor in each configuration tested! Achieves six times faster speed than its competitor in initial applet download in "first-connection" tests! Indicates applet download activity on screen to user up to four times sooner in certain test cases Time (minutes) Web-to-Host Software: Applet Download for First Connection (minutes) (Lower Bars are Better) Mbit/s Ethernet LAN Kbit/s WAN Kbit/s Dial-up Speed/network topology WRQ Reflection for the Web 4.0. IBM SecureWay Host On-Demand (cached client) Source: The Tolly Group, June 2000 Figure The Tolly Group Page 1

2 The test results show that, in firstconnection tests, WRQ Reflection for the Web consistently delivered the host applet faster than the IBM SecureWay Host On-Demand. Furthermore, in tests of subsequent connections, WRQ Reflection for the Web was faster than IBM SecureWay Host On-Demand. Finally, WRQ Reflection for the Web indicated to the end-user the download in progress process much sooner than did its competitor. In accordance with The Tolly Group s Fair Testing Charter, The Tolly Group invited IBM Corp. to participate in the testing process. IBM agreed and recommended that The Tolly Group engineers test the IBM SecureWay Host On-Demand software s cached-client version. In addition, WRQ requested that The Tolly Group also test the thin-client version of IBM s product. (IBM refers to this as their thin-client version because software features are only downloaded as required.) Results Applet Download for First Connection Test results show that WRQ Reflection for the Web software could download the host connectivity applet during the initial connection at 0.16 minutes (.6 seconds) on a 10 Mbit/s Ethernet LAN. In comparison, the IBM SecureWay Host On-Demand (cached client) downloaded the applet in 0.67 minutes (40.2 seconds). Over a 56 Kbit/s WAN, the WRQ software downloaded the applet for the first connection in 1.2 minutes, while competitor IBM s software results show a download time of 8.2 minutes. Finally, the WRQ solution downloaded the applet during the initial connection over a 28.8 Kbit/s dial-up connection in 2.27 minutes, while IBM s solution Time (seconds) downloaded the applet in minutes. See figure 1. When engineers tested the thinclient version of IBM s solution, results show that it took 0.2 minutes (1.8 seconds) for the first applet download over an Ethernet LAN,.16 minutes over a 56 Kbit/s WAN, and 6.07 minutes over a 28.8 Kbit/s dial-up connection. This demonstrates that WRQ s Reflection for the Web software is also faster than IBM s thin-client solution. Subsequent Connections Web-to-Host Software: Subsequent Connections (seconds) (Lower Bars are Better) 10 Mbit/s Ethernet LAN 10 In a second series of tests, The Tolly Group measured the time to establish subsequent connections following the initial applet download and found that, over a Kbit/s WAN 1 Speed/network topology 28.8 Kbit/s Dial-up WRQ Reflection for the Web 4.0. IBM SecureWay Host On-Demand (cached client) Source: The Tolly Group, June 2000 Figure 2 16 Mbit/s Ethernet LAN, WRQ s Reflection for the Web took 7. seconds to connect, while IBM SecureWay Host On-Demand took 10 seconds to connect. Engineers also tested subsequent connections of the WRQ and the IBM solutions over a 56 Kbit/s WAN and results showed 8. seconds for WRQ s product and 1 seconds for IBM s product. When testing the software over a 28.8 Kbit/s dial-up connection, the WRQ software took seconds, while the IBM software expended 16 seconds. See figure 2. When engineers tested the thinclient version of IBM s software, results show that it took 11. seconds for the first connection over a 10 Mbit/s Ethernet LAN, 14.7 seconds over a 56 Kbit/s WAN, and 1. seconds over a The Tolly Group Page 2

3 Kbit/s dial-up WAN. This demonstrates that WRQ Reflection for the Web was again faster even than IBM s thin-client (feature on-demand) solution. End-User Awareness of Download Activity During tests of the initial applet download, engineers measured the time it took for the end-user screen to show signs of download activity such as a screen display to show the progress. WRQ Reflection for the Web took seconds before indicating activity when downloading the applet over an Ethernet LAN, while IBM SecureWay Host On-Demand took more than twice as much time at 6.7 seconds. Over a 56 Kbit/s WAN, the WRQ solution results showed screen activity in seconds, while IBM s solution results showed the first sign of screen activity after 6 seconds, more than four times that of its competitor. Finally, when The Tolly Group engineers tested the WRQ software over a 28.8 Kbit/s dial-up connection, the screen showed download activity in 15 seconds. The IBM software results show that it took 72 seconds for any activity to display on the screen to alert the end-user that the download had begun. See figure. When engineers tested the feature on-demand version of IBM SecureWay Host On-Demand, results show that it took 8 seconds for the first connection over an Ethernet LAN, 15 seconds over a 56 Kbit/s WAN, and 06 seconds over a 28.8 Kbit/s dial-up connection. This demonstrates that WRQ s Reflection for the Web software is faster than IBM s feature on-demand (thinclient) version. Analysis Customers who deploy Web-tohost connectivity software expect quick applet download times and quick host connect times. Moreover, if the initial applet download will take more than a few seconds, it should provide some early indication of its progress while the user awaits completion of the download to prevent the user from aborting the progress. These tests show that WRQ s Reflection for the Web software delivers faster applet download times than IBM s Host On-Demand cached-client version. This is partly because cached-client solutions download the entire applet the very first time that the user connects. Subsequent connection times incur no such delay. Even after the user has downloaded and cached either software s applet onto the system either in browser cache or application-specific local disk cache WRQ s Reflection for the Web still delivers faster connection times. Across a 56 Kbit/s WAN, WRQ s software connected to the Web server, confirmed its configuration, and subsequently connected to the mainframe in less than seconds, whereas IBM s Host On-Demand took 1 seconds to accomplish the same task. Similarly, across a 28.8 Kbit/s dialup connection, WRQ Reflection for the Web connected in seconds, whereas IBM Host On-Demand took 16 seconds. The tests also demonstrate that, as the download process progresses, WRQ s Reflection for the Web provides the user with more immediate feedback on the progress of applet download than does IBM s Host On-Demand cached-client software. Specifically, over a simulated 56 Kbit/s WAN, WRQ Reflection for the Web indicated that applet download started after WRQ, Inc. Reflection for the Web Competitive Evaluation WRQ, Inc. Reflection for the Web version 4.0. Product Specifications* Reflection for the Web is a centrally managed Webto-host solution that lets you Web-enable your enterprise for e-business without re-engineering. Reflection for the Web provides: Low-cost management! The Administrative WebStation, a selfcontained Web site, includes all the information and tools you need to set up and launch Web-to-host sessions! Deployment Director guides you through the steps of configuring Web-to-host sessions, generating Web pages, and uploading the files to your Web server! Point-and-click usage metering tool makes it easy to monitor the number of users running Reflection for the Web sessions High-performance remote access! Component-based architecture allows users to download only the features they need, resulting in smaller downloads and faster start-up times! Automatic applet caching means that terminal emulation components are stored locally and updated automatically, reducing the time it takes to get connected Integrated security! 168-bit Triple DES and 56-bit DES encryption provides the highest level of security available on the Web For more information contact: WRQ Offices: Corporate Headquarters: 1500 Dexter Avenue North Seattle, WA 810 USA Tel: (U.S. only) Fax: URL: *Vendor-supplied information not verified by The Tolly Group 2000 The Tolly Group Page

4 only seconds, whereas IBM s Host On-Demand provided no feedback on its activity until approximately 5 seconds into the process. Across a 28.8 Kbit/s simulated modem connection, WRQ s Reflection for the Web indicated the initiation of applet download after 15 seconds, whereas IBM Host On-Demand showed no activity until more than a minute into the process. (Modem times did not include modem dialup or modem hand-shaking times.) Test Configuration and Methodology The following two software products were tested by The Tolly Group in a series of Web-to-host tests: WRQ, Inc. s Reflection for the Web version 4.0. and IBM s SecureWay Host On-Demand version Both software products under test were installed on a Dell PowerEdge 200 equipped with dual 400-MHz Intel Corp. Pentium III processors with 12 Mbytes of RAM. In addition, the Web server hosted Microsoft Corp. Windows NT Server 4.0 with Service Pack 6 and a Com Corp. Fast EtherLinkXL PCI Bus Card (C05-TX). The Web server connected to an IBM Corp. PC Server 500 System/0 (P/0) mainframe (host) version VM V2R1 via a 10 Mbit/s half-duplex Ethernet connection. A Cisco Systems, Inc Series IP Router version 11.2 (16) release software (fcl) was configured to provide IP routing between the test bed and the mainframe. A Dell OptiPlex GXpro 200-MHz Intel Corp. Pentium Pro with 64 Mbytes of RAM with Internet Explorer was used as the client (Web browser). Microsoft Corp. Windows NT Workstation version 4.0 Service Pack 6 was running on Web-to-Host Software: End-User Awareness of Download Activity (seconds) (Lower Bars are Better) Time (seconds) the Dell that was also equipped with a Com Corp. Fast Ether- LinkXL PCI Bus Card (C05-TX). To simulate 56 Kbit/s WAN or 28.8 Kbit/s dial-up connections, engineers used Shunra Software, Ltd. s The Cloud version 2.0, a softwarebased WAN simulation tool, which ran on a 200-MHz Intel Corp. Pentium PC clone, with 64 Mbytes of RAM and a PCI Bus Card. The PC connected to the network via Ethernet and was running Microsoft Windows NT Workstation version 4.0, Service Pack 5. It was also equipped with a Compaq Computer Corp. Netelligent 10T PCI UTP Bus Card and a Com Corp. Etherlink 10/100 (C05B-TX) PCI Bus Card. 10 Mbit/s Ethernet LAN Kbit/s WAN 6 Speed/network topology Kbit/s Dial-up WRQ Reflection for the Web 4.0. IBM SecureWay Host On-Demand (cached client) Source: The Tolly Group, June 2000 Figure 72 Also between the client and its connection to the Web server and mainframe was an Agilent Technologies, Inc. Internet Advisor LAN Fast Ethernet model J200C, Rev , a full-duplex hardwarebased LAN analyzer. See figure 4. Engineers conducted the first applet-download connection tests and user-awareness tests simultaneously. Engineers measured the time required for each software under test to download and launch its HTTP/TN270 applet across a 10 Mbit/s Ethernet LAN and both the simulated 56 Kbit/s WAN and 28.8 dial-up connections. Before testing, engineers cleared the browser cache and then 2000 The Tolly Group Page 4

5 attempted to contact the host. Engineers measured the time until the first indication in the applet screen that the Web server had started downloading. The Tolly Group then recorded the user-awareness time as the time between the first request from the workstation to the host, to the first definite onscreen indication of host contact on the user s workstation. Engineers recorded the first applet download time as the time between the first request from the client to the host to the completion of the applet download where host-specific data displays on the screen. For subsequent connection times, The Tolly Group engineers recorded the time between the second request from the workstation to the host until reconnection. Engineers tested IBM s Host On- Demand software in each configuration with both its cached-client and thin-client versions. Each speed and topology was tested for three iterations for each system under test. All durations were measured with Big Clock running on a Palm, Inc. Palm IIIxe running Palm OS.5.0, and therefore include a margin of error of approximately +/- one second. Equipment Acquisition and Support When The Tolly Group contacted executives at IBM Corp. and invited them to provide a higher level of support than available through normal channels, they accepted. IBM, as their desired version for testing, supplied the IBM SecureWay Host On-Demand version IBM also provided telephone technical support to configure/tune the device for the test suites executed by The Tolly Group. The Tolly Group verified product release levels and shared test configurations with IBM in order to give them an opportunity to optimize their devices for the tests. Results were shared with IBM who acknowledged their accuracy and validity. For a more complete understanding of the interaction between The Tolly Group and IBM Corp., check out the Technical Support Diary for Competitive Products Tested posted on The Tolly Group s World Wide Web site at See document Just in Time Just in time (JIT) is a universal approach to resource management that dictates that resources should be delivered only when needed (as opposed to pre-loading typical with traditional software.) Dynamic Java applets enable JIT easily in software. In some cases, host connectivity software is only delivered to the end-user station at the moment it is required. Not all JIT is the same. Some vendors elect to download all functionality when host connectivity is required (IBM cached client). Another approach is to download smaller chunks of functionality as needed. IBM calls this its "thin-client" approach since it minimizes the amount of software and storage required at any given time on the end-user machine. It should be noted, however, that this definition is different from the common "thin client" definition as used by vendors such as Citrix Systems, Microsoft Corporation and Santa Cruz Operation. First Connect This initial connection occurs when there is nothing previously downloaded from the host and nothing in the cache, so that everything must be downloaded. This download typically takes longer than subsequent downloads. During its initial connection to the host the client must download the Java applet from the Web server and connect to the host via that applet. Prior to this first connection the client workstation will not have any of the relevant applet code in its cache. Therefore, the client must first download the entire applet before connecting to the host. Subsequent Connections This download refers to any use of the application after the initial download. At this point, functionality, which is often stored in cache and in the hardware, makes the download faster. User-Awareness Some JIT functions, especially an initial download over a low-speed WAN, can take minutes. Unless the user is notified of some activity taking place, the user might mistakenly believe that the session has hung and manually abort. Thus "user-awareness" is a term The Tolly Group coined to denote the moment a program begins showing screen activity of a download The Tolly Group Page 5

6 Test Bed Web Server: Dell PowerEdge 200 with Web-to-host software under test Client: Dell OptiPlex GXpro with Internet Explorer Web browser 1 2 Host system: IBM Corp. PC Server 500 System/0 (P/0) 10 Mbit/s halfduplex Ethernet 10 Mbit/s halfduplex Ethernet Shunra Software, Ltd. The Cloud simulating 28.8 or 56 Kbit/s dial-up connections LAN analyzer: Agilent Technologies, Inc. Internet Advisor Source: The Tolly Group, June 2000 Figure 4 The Tolly Group gratefully acknowledges the providers of test equipment used in this project. Vendor Product Web address Agilent Technologies, Inc. Internet Advisor Shunra Software, Ltd. The Cloud v Since its inception, The Tolly Group has produced highquality tests that meet three overarching criteria: All tests are objective, fully documented and repeatable. We endeavor to provide complete disclosure of information concerning individual product tests, and multiparty competitive product evaluations. As an independent organization, The Tolly Group does not accept retainer contracts from vendors, nor does it endorse products or suppliers. This open and honest environment assures vendors they are treated fairly, and with the necessary care to guarantee all parties that the results of these tests are accurate and valid. The Tolly Group has codified this into the Fair Testing Charter, which may be viewed at Project Profile Sponsor: WRQ, Inc. Document number: Product class: Web-to-host software Products under test: " WRQ Reflection for the Web v " IBM SecureWay Host On-Demand v Testing window: June 2000 Software status: " Generally available Additional information available: " Technical Support Diary For more information on this document, or other services offered by The Tolly Group, visit our World Wide Web site at send to info@tolly.com, call (800) -16 or (72) Internetworking technology is an area of rapid growth and constant change. The Tolly Group conducts engineering-caliber testing in an effort to provide the internetworking industry with valuable information on current products and technology. While great care is taken to assure utmost accuracy, mistakes can occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental, and consequential damages which may result from the use of information contained in this document. All trademarks are the property of their respective owners. The Tolly Group doc rev. clk 2 Jun The Tolly Group Page 6