Development of Performance Testing Suite Using Apache JMeter

Transcription

1 Development of Performance Testing Suite Using Apache JMeter Jidnyasa Agnihotri and Rashmi Phalnikar Abstract Testing a product has become one of the most important tasks for any organization (Be it small scale or large scale). Without testing the product, it is not delivered to the customer. Testing is an ongoing activity from the beginning of a product s development. A performance testing suite shall be developed using Apache JMeter for the purpose of testing a product. To perform performance testing on client- and server-type softwares, a 100% pure Java application named Apache JMeter is used. Apache JMeter is not a browser, it works at protocol level. Static and dynamic resources performance testing can be done using JMeter. A high level performance testing suite will be developed in capturing aspects of performance at UI and System level. Developing the testing suite helps in saving the time and cost of the organization. The discussion follows and describes benefits of performance testing and the performance testing suite. Keywords Silk performer Performance testing JMeter, etc. 1 Introduction Testing of any application, product, web applications, etc., has become very important to ensure that the product works efficiently as per the requirement of the customer. Testing department has gained a lot of importance for this reason. Every organization and company (large scale or small scale) does have testing department. The fast-growing world is leading to the demands of updating and implementing their software s as soon as possible to meet the requirements of the market and also J. Agnihotri (&) R. Phalnikar Department of IT, MITCOE, Pune, India jigyasathakurdas@gmail.com R. Phalnikar rashmiphalnikar@yahoo.co.in Springer Nature Singapore Pte Ltd S. Bhalla et al. (eds.), Intelligent Computing and Information and Communication, Advances in Intelligent Systems and Computing 673, 317

2 318 J. Agnihotri and R. Phalnikar to stay at lead in the race. Software should run correctly even if hundreds of people are trying to access it at one time. To stay away from bugs, software must undergo software testing. SDLC has software testing as one of its phase. Testing in itself is very vast and lengthy process. Testing ensures that the software will work properly as per the demands of the customer. Testing leads to successful completion and working of a product. In this paper, performance testing suite shall be developed using Apache JMeter. The tool shall be developed for a particular product, describing in detail the method to build the testing suite for any product. Also, the importance of testing and the testing suite for a particular product shall be discussed in detail. Rest of the paper is divided into the following section. Section 2 presents existing system. Section 3 Literature Survey Sect. 4 Architecture Sect. 5 Results and Analysis and Sect. 6 Conclusion. 2 Existing System Performance testing is a nonfunctional testing and is carried out to test the system under load conditions. A user can do performance testing on a product using various tools. Many tools are available online/in market for the purpose of performance testing. A tool which is easy to handle and is available will always be preferred by any organization. Silk performer tool [1] helps creating practical load and tests all the environments by creating huge stress. Performance testing on internet applications can be done using WebLOAD [2], a commercial tool for performance testing. A user can create practical load on websites and applications on web using LoadComplete [3]. A demo of original transactions is created for load testing by Rational Performance Tester [4]. Performance of your web application can be improvised using NeoLOAD [5]. Apache JMeter [6] is a Java platform application and can also create a functional test plan. Silk performer [1] can create huge load and requires very less size, because silk performer is a UI-based performance testing. WebLOAD [2] is the generated load from cloud and the machine present in the environment. Virtual users are created by LoadComplete [3] by first recording the actions performed by the user. These virtual users are created from the cloud or the machines. Thousands of users can be created and handled by HP LoadRunner [7]. It is a combination of more than 2 3 different machines. So, it is complex to handle. When user interaction is involved, only then Rational Performance Tester [8] can be used. NeoLOAD [5] can be of use only for testing web applications. Apache JMeter is a tool widely used for performance testing. Apart from testing the web applications, JMeter can also be used for testing the Servlets, JMS, LDAP, Web Services, Java classes, FTP, Pearl Scripts, and JAVA objects.

3 Development of Performance Testing Suite Using Apache JMeter Literature Survey Software testing of a product is done to make sure that the software is of a good quality. Software testing also makes sure that the software is reliable. Performance testing is performed on any application especially web application to understand how it behaves under heavy load. Cloud computing can be used in the field of testing for the purpose of overcoming the space issues [9]. Web services require testing on large scale. Extensive Human Intervention is required for calculating the performance of web service. Enhancing the components and creating new components also needs the obvious human interference. In paper [10], JMeter is used for web services testing which can be done without human intervention and dependencies are reduced so, in return, the speed of execution is increased. For measuring the reasons for the decline in the performance, in the paper [11], JMeter-based distributed test structure and point to point test structure are used to set up a computing environment. A fully automated testing service is made available recently for the ease of the testers. This service is available commercially. This service, known as Testing as a Service (TaaS), reduces the time and increases the efficiency. The paper [12] proposes a framework for performance testing. Problems generally faced by the testers are addressed quite neatly in this framework. Additionally generating and executing the test cases is completely automated. In this paper [12], JMeter is also one of the TaaS moreover it is easily available (open source). In [11], point to point test structure is developed, it helps in leading to the conclusion that the main reason of decline of performance is memory loss. This loss happens because of Tomcat server. JMeter is a very flexible tool and a tool with large number of possible variations which helps in testing of data from various different sectors [13]. Apache JMeter is a freely available feature rich tool and easy to set up and use. Flexibility of JMeter makes it use more compatible (Fig. 1). Apache JMeter executes its actions as shown in the flow above. Apache JMeter initially generates load on the server under test. It then collects the responses from these servers. The obtained responses are then analyzed and calculated. Report generation is the last step in the workflow which takes place after the response data is ready. Looking at this workflow and all the other factors form installing JMeter to generating the reports, it is understood that JMeter is the best tool to test the load as it is free, great load generation, and easy user interface. 4 Architecture Performance testing or load testing is a process of evaluating the quality or capability of a product (Fig. 2).

4 320 J. Agnihotri and R. Phalnikar Fig. 1 Workflow for JMeter (Source [14]) Fig. 2 Performance testing process (Source [15]) Figure above describes how the performance testing process is actually carried out. The tests are identified again and again to ensure that the system under test is giving appropriate output every time. After the test is complete, benchmark is noted and the same procedure is carried out again to note all the possible results. Also, no action should be left which does not undergo testing. Doing performance testing is a tedious task if it has to be done in the traditional way. Developing a testing suite will lead to time deduction. As in, the tester will

5 Development of Performance Testing Suite Using Apache JMeter 321 have the user interface ready, only the test should be started. So, from the point of view of an organization, it is a necessity to have a performance testing suite. The performance testing suite is developed in five modules. Each module is explained in detail ahead. Module 1: Learning the product Input Setup of the product Output Understanding the product Procedure In this module, the expert of the product will give the training. Understanding the product will lead to the clear idea of what has to be done. The detailed study of the product and all the flows in it are necessary to prepare the performance testing suite. This module will lead to finalization of the product for which the performance testing suite has to be developed. Module 2: Understanding the Scenarios for Performance Testing Input Scripts if available from previous test and different scenarios for Testing Output Various scenarios studied Procedure In this module, the possible scenarios will be studied. The performance suite developer will understand all the possible scenarios that are available and necessary at the moment. The scenarios which make any difference to the product s operations should be considered for script preparation. All the flows must be thoroughly studied by the developer to record the scripts correctly. Module 3: JMeter Input The product setup and the JMeter Setup Output Scripts for the Performance Testing Suite Procedure In this module, the possible scenarios are taken into consideration and those scenarios are recorded in JMeter. These scripts will be the important portion of the performance testing suite. Plugins required for the testing are in the development phase. The actual operations performed for the development of scripts include the following steps. (1) Initially, change the port number of the browser to be used. (2) Using the JMeter GUI s test script recorder start the recording. (3) Stop the recorder of JMeter when the necessary navigation is completed. (4) Change the recorded script and remove all the random generated values and hard coded values. (5) To check the results, add the appropriate listener. (6) Now, the initial script should be checked. (7) Enhancement of script wherever possible shall be done in this step. (8) Start the performance test, evaluate the results, repeat these actions for the application. The above steps are performed for every script/action of the product. The script with all the changes and modification which gives expected results is included in the suite.

6 322 J. Agnihotri and R. Phalnikar Module 4: Performance Testing Input Performance Testing Suite and the product setup Output Performance Test results Procedure At the end of this module, we will have performance results of how the system is performing under load conditions. The performance runs are usually carried out for 1 h, 4 h, 1 day, 4 days, 10 days, etc. The performance tests are performed on the product according to the requirement. The load generation is specified accordingly. Module 5: Verifying the Performance Results Input Results of the Performance tests Output Check the achieved accuracy Procedure At the end of this module, the results achieved are verified for their correctness. It is checked whether the system is behaving according to the expectations. If yes how to enhance is the next step. If there are certain problems in the system behavior, the changes to be done in the system are reported. For example, if a page should load in 5 s and if it taking 55 s, the issue should be reported and the necessary actions should be taken. Figures 3 and 4 are the visual description of the steps that are to be carried out in the development of performance testing suite. Various steps are combined together to put up in two figures. Figure 5 represents the graphical user interface of the performance testing suite developed. The load can be managed changing the parameters which appear in the thread group section. The graph represents the presence of threads on the particular timings. The scripts are present on the left-hand side. The performance run must be done through the non-gui mode for the run to be completed without causing any failure or problem to the system. Nineteen scripts are developed so far that are included in the suite. Fig. 3 Test script generation

7 Development of Performance Testing Suite Using Apache JMeter 323 Fig. 4 Test script execution Fig. 5 Performance testing Suite GUI 5 Results and Analysis Results obtained by the JMeter performance run are very simplified and can be viewed as a CSV file or statistics (in table format) or even as separate graphs for every operation performed. JMeter provides results by writing the results to the CSV file and to the folder which are provided in the command line of the non-gui mode.

8 324 J. Agnihotri and R. Phalnikar Fig. 6 Latencies over time Fig. 7 Statistics Figure 6 includes a graph which shows the latencies of the operations over time. Different color indicates different operation. Every operations graph can be viewed individually by disabling the other operations. The average time responses can be viewed using the statistics table, using graphs an approximate estimate of the times of the requests can be observed. The responses having incorrect reply can be pointed out easily using the graphs. Figure 7 shows the table which is produced as the output of the performance testing. The statistics are very precise, accurate, and give detailed information about the error, minimum time, maximum time, average time, throughput, etc. The graphs and the statistics tables are equally important to monitor the system. 6 Conclusion The performance testing is an important task to be done for every product or application. Having convenient means for this testing provides various benefits to the organization. Time and cost savings are the two most important benefits that can

9 Development of Performance Testing Suite Using Apache JMeter 325 be noted for having performance testing suite. Here, the tool used is also free so the cost reduction is to a high level. In this paper, we discuss the performance testing suite developed for SMGR (System Manager). For the testing of this product, initially silk performer was used which has high cost. The suite developed, gives accurate results when compared with the results of the silk performer. It is noticeable that the results are accurate up to 90%. The reports generated are also very detailed, precise, and give information about every single action performed during the run. Apache JMeter performance testing suite has a GUI which is more user friendly when compared to silk performer. The suite developed in here is specific to SMGR similarly it can be developed for any application (static or dynamic). Acknowledgements The work of researchers and publishers is meant a lot to the author during the whole process. Author expresses regards for the same. Also the help by the professors proved to be of great use. Author would also like to thank Avaya India Pvt. Ltd. for giving this opportunity of working for Avaya. The suite developed during this project is specific to SMGR (product of Avaya India Pvt. Ltd.) as the project is sponsored by Avaya. References 1. application_performance_testing/ w/silk_performer_trial/23695.tutorial-1-part-1-getting-started-with-silk-performer.aspx , Sharma, M., Iyer, V. S., Subramanian, S., & Shetty, A. Comparison of Load Testing Tools. 3. Vinayak Hegde, Pallavi. Web Performance Testing: Methodologies, Tools and Challenges , Vol. 45(3), pp , Brown AW, Iyengar S, Johnston S. A Rational approach to model-driven development. IBM Systems Journal , Vol. 6(5), Bhatia R, Ganpati A. In Depth Analysis of Web Performance Testing Tools Sep , pp , Gao Q, Wang W, Wu G, Li X, Wei J, Zhong H. Migrating load testing to the cloud: A case study. InService Oriented System Engineering (SOSE), 2013 IEEE 7th International Symposium on 2013 Mar 25. IEEE , Vol. 26, Issue 8, pp , Apte V, Hansen T, Reeser P. Performance comparison of dynamic web platforms. Computer Communications May , pp , Shenoy S, Bakar NA, Swamy R. An adaptive framework for web services testing automation using JMeter. InService-Oriented Computing and Applications (SOCA), 2014 IEEE 7th International Conference on 2014 Nov 17. IEEE , Vol. 5, pp , Jing Y, Lan Z, Hongyuan W, Yuqiang S, Guizhen C. JMeter-based aging simulation of computing system. In Computer, Mechatronics, Control and Electronic Engineering (CMCE), 2010 International Conference on 2010 Aug 24. IEEE , pp , Ali A, Badr N. Performance testing as a service for web applications. InIntelligent Computing and Information Systems (ICICIS), 2015 IEEE Seventh International Conference on 2015 Dec 12. IEEE.

10 326 J. Agnihotri and R. Phalnikar , pp. 1 5, Harikrishna P, Amuthan A. A survey of testing as a service in cloud computing. InComputer Communication and Informatics (ICCCI), 2016 International Conference on 2016 Jan 7. IEEE