Service Manager 9.31 web client end-to-end performance test report

Transcription

1 Technical white paper Service Manager 9.31 web client end-to-end performance test report Table of contents Introduction 2 Criteria 2 Test results 2 Conclusions 3 Appendix A: Test environment and tools 4 Hardware 4 Software 4 Network bandwidth 4 Test dataset 4 LoadRunner TruClient 4 Appendix B: Test results 6 No latency ms latency ms latency ms latency 8 Appendix C: Test scenarios 10 Appendix D: Research on response times 11 More information 12 Click here to verify the latest version of this document

2 Introduction The data and information supplied in this document serve as a reference before you implement the Service Manager (SM) web client. The performance test records the length of time required to perform 22 key scenarios for the SM web client in different network latency simulations. The test results reflect the end-to-end performance of the SM web client under different network latency circumstances. Criteria After reviewing research findings and usability test results, we consider the following response times to be acceptable when users open SM webpages in a web browser: 5 seconds (for most SM web client webpages) 10 seconds (for 'heavy' webpages with a large cache) Note The 'heaviest' webpage for the SM web client is the login page. When a user visits the login page the first time, the cache contents (such as JavaScript and CSS files) are loaded and saved by the web browser. Because these files are loaded only once, we have set the criteria for the heaviest page with a cache instead of without a cache. If the load time of a webpage exceeds these criteria, users are likely to consider the performance of the web application to be unacceptable. For more information about the research that supports these criteria, see Appendix D: Research on response times. Test results The following table summarizes the key test results. Network Latency* 125 ms 250 ms 400 ms Most pages < 4 seconds < 4 seconds < 5 seconds ESS login without cache < 4 seconds < 6 seconds < 9 seconds ESS login with cache < 3 seconds < 3 seconds < 3 seconds Login without cache < 5 seconds < 8 seconds < 11 seconds Login with cache < 4 seconds < 5 seconds < 6 seconds * Network latency refers to the round-trip latency between the SM web tier and the web browser. These test results are valid for up to 2,500 concurrent users. According to our tests, response times in both the single user scenario and the 2,500 concurrent user scenario are similar. For more information about the test environment and tools, see Appendix A: Test environment and tools. For detailed test results, see Appendix B: Test results. 2

3 Conclusions The following conclusions and recommendations are based on the performance test results and on the criteria that are defined in the "Criteria" section: The SM web client performs well when the network latency is less than 125 ms. The SM web client performs moderately when the network latency is less than 250 ms. We recommend that you run the SM web client when the network latency is less than 250 ms. The SM web client performs acceptably when the network latency is less than 400 ms. We do not recommend that you run the SM web client when the network latency exceeds 400 ms. Note These performance tests are based on the recommended configuration for an SM deployment. The actual performance in your implementation may vary, depending on a variety of factors. These include tailoring, data volume, environment (such as network latency and different web browsers), and the number of elements on a specific webpage. 3

4 Appendix A: Test environment and tools Hardware All tests were conducted by using the following physical assets. Server SM - primary server SM (9 servlets) 8cpu, 2.13GHz 32GB RAM SM - secondary server SM (12 servlets) 8cpu, 2.13GHz 32GB RAM SM - secondary server SM (11 servlets) 8cpu, 2.13GHz 32GB RAM DB server Oracle 11g release 2, 64-bit 8cpu, 2.13GHz 32GB RAM Operating system Windows 2008 R2 64-bit Windows 2008 R2 64-bit Windows 2008 R2 64-bit Windows 2003 R2 64-bit SP2 Tomcat server Tomcat x 10 instances, JVM - 1GB 8cpu, 2.13GHz 32GB RAM Windows 2003 R2 64-bit SP2 Apache httpd server Apache httpd cpu, 16GB RAM Windows 2003 SE 32-bit SP1 LoadRunner controller HP LoadRunner 11with TruClient Intel Xeon 2cpu 2.13GHz 4.0GB RAM Windows Server bit Software All tests were conducted by using the following software set: Oracle 11g Windows Server 2008 Enterprise 64-bit Windows Server 2003 Enterprise 32-bit and Windows Server 2003 Enterprise 64-bit HP LoadRunner 11.5 (with TruClient) Apache HTTP Server Apache Tomcat Server Java 1.6 for SM 9.31 web tier VE Desktop Shunra Network bandwidth In these performance tests, the maximum bandwidth available to the web browser when accessing the SM web tier server was limited to 4 Mbps. Test dataset All tests were based on the SM out-of-box (OOB) dataset. The OOB dataset only contains a small set of test data and 6,000 mock users. LoadRunner TruClient The LoadRunner Ajax TruClient protocol interactively records scripts as you navigate through your business process. This enables VuGen to record dynamic, complex web-based applications and to create user-friendly scripts. Scripts are created in real-time, and steps are displayed in the LoadRunner VuGen for TruClient sidebar as they are performed. Advantages of using Ajax Truclient Offers robust support for different Ajax controls, such as slider, calendar, and accordion Works on any framework, including, GWT, YUI, jquery, and ZX Ajax ExtJS Does not require correlations Provides a rich and visual UI 4

5 Works in the browser developing the script interacting with the application, viewing added steps while recording, and the ability to check and improve identification Provides insight into client behavior by using multiple transactions that are based on different events, such as "network complete," loading a DOM loaded, and loading a document. Challenges to using Ajax TruClient The latest version of LoadRunner Ajax TruClient enables you to record scripts by using either Mozilla Firefox or Internet Explorer with standard mode. Because the SM web tier only works on Internet Explorer with quirks mode, all of the test results are based on Firefox with LoadRunner TruClient. Protocol performance Meaningful comparisons to other protocol are not available. The performance of Ajax TruClient is application-specific. This is because the footprint is determined by the application client footprint, which is defined by the loaded JavaScript or DOM. 5

6 Appendix B: Test results For a description of the transactions, see Appendix C: Test scenarios. No latency Transaction Name Average Minimum Maximum Std. Deviation CM_Change_Save CM_Open_New_Change_Hardware ESS_Catalog_Order_Form ESS_Catalog_Order_Submit ESS_Login ESS_Login_Cached ESS_Logout ESS_Submit_Problem_Request ESS_View_Request_Help IM_Incident_Search IM_Incident_Update IM_New_Incident_Save Incident_New_Form OP_Login OP_Login_Cached OP_Logout PM_Open_Problem_Form PM_Problem_Save SD_Interaction_Search SD_Interaction_Update SD_New_Escalate_Next SD_New_Interaction_Escalate SD_New_Interaction_Escalate_Save SD_View_New_Interaction_Form ms latency Transaction Name Average Minimum Maximum Std. Deviation 6

7 CM_Change_Save CM_Open_New_Change_Hardware ESS_Catalog_Order_Form ESS_Catalog_Order_Submit ESS_Login ESS_Login_Cached ESS_Logout ESS_Submit_Problem_Request ESS_View_Request_Help IM_Incident_Search IM_Incident_Update IM_New_Incident_Save Incident_New_Form OP_Login OP_Login_Cached OP_Logout PM_Open_Problem_Form PM_Problem_Save SD_Interaction_Search SD_Interaction_Update SD_New_Escalate_Next SD_New_Interaction_Escalate SD_New_Interaction_Escalate_Save SD_View_New_Interaction_Form

8 250 ms latency Transaction Name Average Minimum Maximum Std. Deviation CM_Change_Save CM_Open_New_Change_Hardware ESS_Catalog_Order_Form ESS_Catalog_Order_Submit ESS_Login ESS_Login_Cached ESS_Logout ESS_Logout_Cached ESS_Submit_Problem_Request ESS_View_Request_Help IM_Incident_Search IM_Incident_Update IM_New_Incident_Save Incident_New_Form OP_Login OP_Login_Cached OP_Logout OP_Logout_Cached PM_Open_Problem_Form PM_Problem_Save SD_Interaction_Search SD_Interaction_Update SD_New_Escalate_Next SD_New_Interaction_Escalate SD_New_Interaction_Escalate_Save SD_View_New_Interaction_Form ms latency Transaction Name Average Minimum Maximum Std. Deviation 8

9 CM_Change_Save CM_Open_New_Change_Hardware ESS_Catalog_Order_Form ESS_Catalog_Order_Submit ESS_Login ESS_Login_Cached ESS_Logout ESS_Submit_Problem_Request ESS_View_Request_Help IM_Incident_Search IM_Incident_Update IM_New_Incident_Save Incident_New_Form OP_Login OP_Login_Cached OP_Logout PM_Open_Problem_Form PM_Problem_Save SD_Interaction_Search SD_Interaction_Update SD_New_Escalate_Next SD_New_Interaction_Escalate SD_New_Interaction_Escalate_Save SD_View_New_Interaction_Form

10 Appendix C: Test scenarios Transaction CM_Change_save CM_Open_New_Change_Hardware_Form ESS_Login ESS_Login_Cached ESS_Catalog_Order_Form ESS_Catalog_Order_Submit ESS_Logout ESS_Submit_Problem_Request ESS_View_Request_Help IM_Incident_Save IM_Incident_Search IM_Incient_Update IM_New_Incident_Form OP_Login OP_Login_Cached OP_Logout PM_Open_Problem_Form PM_Problem_Save SD_Interaction_Search SD_Interaction_Update SD_New_Interaction_Esclate SD_new_Interaction_Esclate_Next SD_New_Interaction_Esclate_Save SD_View_New_Interaction_Form Description In Change Management module, select Save & Exit to save a record. In Change Management module, open a new form to fill in a new record. Log in to ESS the first time without any cache previously saved by the browser. Log in to ESS with cached files. From ESS, select Submit to fill a form. From ESS, select Submit to send a catalog order. Log out from ESS. From ESS, select Submit to send a problem request. From ESS, select Request Help to open a new request form. In Incident Management module, save a new incident record. In Incident Management module, search for an incident record. In Incident Management module, update an existing incident record. In Incident Management module, open a form to enter a new incident. Log in to SM web client the first time without any cache previously saved by the browser. Log in to SM web client with cached files. Log out from SM. In Problem Management module, open a form to create a new record. In Problem Management module, select Save & Exit to save a problem record. In Service Desk module, perform a search for interaction. In Service Desk module, update an interaction record. In Service Desk module, select Escalate to escalate an interaction record. In Service Desk module, select Next to the next page when escalating an interaction record. In Service Desk module, select Save & Exit to save the escalation. In Service Desk module, select Register New Interaction to open a new interaction record. 10

11 Appendix D: Research on response times The following list details some research findings about response times: On average, web users maintain attention on the current task in a web-based application for 10 seconds. If a procedure takes longer than 10 seconds, it is likely that the user will leave the UI and do something else. This guideline has not changed for more than 40 years and is unlikely to change. Miller, R. B. (1968). Response time in man-computer conversational transactions. Proc. AFIPS Fall Joint Computer Conference Vol. 33, Card, S. K., Robertson, G. G., and Mackinlay, J. D. (1991). The information visualizer: An information workspace. Proc. ACM CHI'91 Conf. (New Orleans, LA, 28 April-2 May), Jakob Nielsen. Response Times: The 3 Important Limits. On average, web users will wait about 8 seconds for a webpage to download and be displayed. Zona Research, The Need for Speed II, Zona Market Bulletin, No. 5, April 2001, Zona Research, USA. Note With faster internet connections and lower tolerance levels among online shoppers, 4 seconds may now be the maximum length of time the average online shopper will wait for e-commerce websites. 11

12 More information Visit the HP Software Support Online web site at: This website provides contact information and details about the products, services, and support that HP Software offers. HP Software online support provides customer self-solve capabilities. It provides a fast and efficient way to access interactive technical support tools needed to manage your business. As a valued support customer, you can benefit by using the support web site to: Search for knowledge documents of interest Submit and track support cases and enhancement requests Download software patches Manage support contracts Look up HP support contacts Review information about available services Enter into discussions with other software customers Research and register for software training Most of the support areas require that you register as an HP Passport user and sign in. Many also require a support contract. To register for an HP Passport ID, go to: To find more information about access levels, go to: Sign up for updates hp.com/go/getupdated Share with colleagues Rate this document Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Firefox is a registered trademark of the Mozilla foundation. Internet Explorer is a registered trademark of the Microsoft corporation. SM 9.31, October 2013