A Test Automation Framework Based on WEB

Transcription

1 2012 IEEE/ACIS 11th International Conference on Computer and Information Science A Test Automation Framework Based on WEB Fei Wang,Wencai Du * College of Information Science & Technology, Hainan University Haikou, Hainan Island, ,China wencai@hainu.edu.cn Abstract Many software applications today are written as web-based applications to be run in an Internet browser. Selenium is a set of powerful different software tools working with many browsers, operating systems, programming languages, and testing frameworks each with a different approach to supporting automation test for testing web-based applications. JMeter is used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. JMeter operates at the protocol-level, on the other hand, Selenium works at the user-level.in this paper, authors have designed an automatic software testing framework for web applications based on the Selenium and JMeter. With the use of the software framework, we efficiently improve the extensibility and reusability of automated test. The results show that the new software framework improves software products quality and develop efficiency. This paper also illustrates how to design web-based test automation framework in details. Keywords-selenium; web; jmeter; automation testing framework I. INTRODUCTION The acceptance test of web applications often involves some manual tasks. Doing it manually, the operating personnel human error can occur easily. At the same, it wastes the time [1]. The web-based applications were bearing the more and more complex business logic and more and more huge information platform formation, along with the characteristics of short release cycle and the quick regeneration [2]. This requires the web application tests to be of comprehensiveness, expansibility and efficiency. The Browser carrying on web applications also was emerging in diversification and pluralism [3]. It also increases complexity and workload for the web application testing. More attention should be paid to the web application testing efficiency, reusability, and comprehensiveness. Aim at some problem mentioned above, the paper presents the test automation framework that was integrated by the two kinds of test automation tool Selenium and Jmeter. Common Web test automation tools, such as QTP,Win Runner Load Runner, Robot, SilkTest, Selenium are used widely in automation test [4]. Selenium can be said to be the most comprehensive of open source Web test automation tools, especially well support to the Web application which employed the BackBase framework [5]. *Corresponding Author Jmeter,Grinder,Multi-Mechanize, OpenSTA are common pressure test tools. However, Jmeter developed by Apache organization is an open source pressure test tool based on Java. It can be used on the server, the network or other object to simulate huge loading, in different pressure under a variety of testing their strength and analyze the overall performance, beyond that, it takes some kinds of test result reports[6]. Implement of the test automation framework shows that the test automation framework is convenient to carry on function test, performance testing, and loading test in web applications. It also shows that it has better test performance in reusability, extensibility, comprehensiveness, and feasibility. We are all known that Selenium and JMeter software testing framework tools work at the different levels. Selenium works at the user-level while JMeter operates at the protocol-level. In order to easy to carry on different tests for a web applications and to share testing procedures and test data between different kinds of tests, this paper presents one test automation framework by an integration of Selenium and Jmete. By implementing the new test automation framework showed that the test automation framework convenient to the web application doing function test, performance testing and loading test, to the ascension of the test reusability, scalability, comprehensive line, accuracy have good feasibility. II. BASICS OF SELENIUM AND JMETER Selenium Remote Control (RC) is allows you to write automated web application UI tests in any programming language against any HTTP website using any mainstream JavaScript-enabled browser [7]. The Selenium Server which launches and kills browsers, interprets and runs the Selenese commands passed from the test program, and acts as an HTTP proxy, intercepting and verifying HTTP messages passed between the browser and the AUT. Jmeter simulates the client to send request to server (web server, or database server), Can load and performance test many different server types [8] U.S. Government Work Not Protected by U.S. Copyright DOI /ICIS

2 III. TESTING AUTOMATION FRAMEWORK DESIGN AND IMPLENTATION The information about the same type of web applications abstracts way from the web-based test automation framework and form into a single configuration file. By doing so, we ensure that test automation framework and the application to be tested should be avoided interfere with each other, relatively independent. The test automation framework divides diversity interface elements and polygonal interface elements into non-variance elements and variable collection of space. It ensures that this test automation framework can be compatible with the same type of many applications, does not need any modification. We have designed a test automation framework by employing selenium test tools with supporting a variety of Web browser for solving web applications compatibility across the browsers (Figure1). It provides testers with a simple interface. It lets testers to configure different browsers conveniently for web application test. In order to provide testers with tool to simulate browser to send a variety of reasons of request functions to a web application server, Jmeter function was integrated into the test automation framework designed in this paper. It will do not cause any inconvenience to the performance test in which the browser was opened and PC resources were occupied. At the same time, a test automation framework can prepare a mass of data for performance test by directly configure data resources. In order to ensure universality, the test automation framework designed in our work not only provides testers with unified test cases, but also leaves the extensibility space for introducing other test tools in the future. Using the new test automation framework, testers can configure a test type easily just through modifying a test type and can switch among different tests without altering any elements related to test cases. Figure 1. A web-based test automation framework A. General Structure of the Integrated Automation Testing Framework The integrated auto testing framework designed in this paper includes five components listed the following (Figure 2): implement name of workers (Translator, Actionworker, Comparator, testresultcollector). Model: This component contains the object model which is converted from XML. Such as Action, Assertion, Element, ApplicationData, Cases, ToolsSetting. Translator has two steps. The first step is to translate each test case into sets of actions related to the different testing types, such as UI, backend, and loading test. The second step is to translate each set of actions into specific tools' commands/scripts depending on the tool employed in doing the actual testing. ActionWorker function is to call the corresponding tool to do the specific actions. Comparator function is to compare the test result with the expected result. Comparator has two parts. One is to collect the test result and expected result. Another is to compare against the expected result. Figure 2. Component Diagram a) Model (Overall Entity) These classes in the below are entity, and they relate to the testsuite XML, application background XML (Refer to Figure3). b) Function Description Functional tests are intended to help testers verify that a system, as a whole, functions as intended. Such tests verify that everything is wired together correctly. Therefore, we define a set of functions for the new test automation framework (Table 1). c) Overall Process Flow The testing framework has basic five steps: 1) By using xmlobjectconvertor() to convert the test case to TestCase entity, convert the test steps of test case to TestCaseAction entity, convert the assertion of test case to Assertion entity, convert the application background xml to ApplicatonData entity[9]; 2) To combined the test cases with application data; 3) By using translate() function to translate the case to TestToolCase; 4) To call the specific actionworker for specific testing tool to do the TestToolAction with the doaction() function; 5) To collect the test result for the specific testing tool by collecttestresult() function, and collect the expected test result by collectexpectedresult(), then compare the test result with the expected result with compare() function. B. The hierarchical structures of The Integrated Automation Testing Framework

3 3) To get an instance comparator from the getcomparatorinstance() function with the getcomparatorname() function; 4) To compare the test result with the expected result by compare() function, and then give the result of comparing. Figure 3. Overall Entities A) Translator Translator has two levels. Translate the test case which is prepared by tester to a tool test case which can be recognized by a specific test tool. (Refer to Table 1 function descriptions) The translator layer process flow like the following: 1) To get a translator name with a specific test type by gettranslatorname() function; 2) To translate the test case to a scope testcase with a specific translator name by gettranslatorinstance() and translate() functions ; 3) To get a translator name with a specific test tool by gettooltranslatorname() function; 4) To get a tool testcase from a scope test case by the translator by gettooltranlsatorinstance() and translate() functions. B) ActionWorker This component creates a specific actionworker and then does the specific actions. (Refer to TABLEI function description) The actionworker layer process flow like the following: 1) To get a actionworker name by getactionwokername() function; 2) To get a actionworker instance by getactionworkerinstance() function; 3) To do action by doaction() function with the actionworker in the last step. C) Comparator This component can make sure whether the test case is successful or not. This component collects the test result and collects expected result, and then do the comparison on the two kinds result. (Refer to Table 1 for function description) The main steps are listed in the following: 1) To get an instance test result collector to collecttestresult() from the gettestresultcollectorinstance() function with gettestresultcollectorname(); 2) To collect the expected result of the running test case by collectexpectedresult(); IV. TESTING THE FRAMEWORK The web-based test automation framework, like any another being developed, needs testing. As a new test automation framework, there are no tests to perform and all tests need. We must evaluate its performance of reusability, extensibility, compatibility, and universality in particular. In order to test the performance of sharing test processes and test data between UI test and performance test. The test processes are designed as following: Step1. To prepare all data for IU system and data for test tools configuration item, that is, to prepare the test cases and test type configuration file for both UI test and load test. Step 2. To launch the test automation framework, start UI test for IU system, modify default browser class of configuration items in test tools (TABLEII). Step 3. To Change test type into load test first, and then to launch the test automation framework, final modify test parameters of load test (TABLE III), for examples, each request spacing interval, How many requests launch at same time, How many threads need to implement these requests? After finished in steps above, a set of test results are obtained (TABLEIV). The results of evaluation of thee web-based test automation framework have showed that the framework provides testers a tool to configure different browsers conveniently for web application test, prepare a mass of data for performance test by directly configure data resources, configure a test type easily by modifying a test type, and switch among different tests without altering any elements related to test cases. V. CONCLUSION The new testing automation framework integrated by Selenium and Jmeter can share the test steps and test data among different testing, such as UI testing, backend testing, loading test and so on. It is convenient to switching in various types of testing for web applications. It supports multiple browsers and a variety of operating system. It can be widely used in web application test automation. ACKNOLEDGMENT This work is supported by NSFC(National Science Foundation of China) project (No ). REFERENCES [1] Ying Wu, Automation Testing Framework for web base on web base on selenium. Infomation Techonlogy[J],2011,9:

4 [2] YanQin. Lin, Function Auto Testing for web application [D] 2010 [3] McMahon, C., History of a Large Test Automation Project Using Selenium 2009,8 [4] LinLi. Fu, JunQiu.Dai, JinHe. Liu, Auto Test Solution for Web Application[J] 2010,4(39):23 [5] [ [6] [7] [8] [9] [10] TABLE I. FUNCTIONS DESCRIPTION Function Name xmlobjectconvertor() gettranslatorname() gettranslatorinstance() translate() gettooltranslatorname() gettooltranlsatorinstance() getactionwokername() getactionworkerinstance() doaction() gettestresultcollectorname() gettestresultcollectorinstance() collecttestresult() collectexpectedresult() getcomparatorname() getcomparatorinstance() compare() Description tester calls XMLConvertor in util component to convert xml to java object in model component tester calls applicationproperties to get the Translator implement name, ApplicationProperties is an entity in util component. gettranslatorinstance(), tester calls TranslatorFactory to create a Translator instance at first time, if the instance exists, just return the instance. tester calls TranslatorFactory to create a Translator instance at first time, if the instance exists, just return the instance tester passes the TestCase to translator, and translate the TestCase to TestToolCase which has the TestToolAction and TestToolAssertion, tester calls ToolTranslator to translate the TestScopeCase to TestToolCase.. tester calls ApplicationProperties to get the ToolTranslator implement name tester calls ToolTranslatorFactory to create a ToolTranslator instance at first time, if the ToolTranslatorInstance exists, just return the instance. tester calls ApplicationProperties to get the ActionWorker implement name, ApplicationProperties is an entity in util component tester calls ActionWorkerFactory to create an ActionWorker instance at first time, if the instance exists, just return the instance. tester passes the TestToolAction, and calls ActionWorker to doaction, when finish one case, return the finish case flag tester calls ApplicationProperties to get the testresultcollector implement name, ApplicationProperties is an entity in util component. tester calls TestResultCollectorFactory to create a testresultcollector instance at first time, if the instance exists, just return the instance. tester calls TestResultCollector to collect the test result tester calls ExpectedResultCollector to collect the expected result. tester calls ApplicationProperties to get the component implement name, ApplicationProperties is an entity in util component. tester calls ComparatorFactory to create an Comparator instance at first time, if the instance exists, just return the instance. tester calls comparator to do the comparison between the test result and expected result, return the compare result; tester pass the testresult and expectedresult to the comparator to call the compare fucntion, return the compare result.[10] TABLE II. THE UI TEST IN IU SYSTEM UI test Test cases browser type Switch between browsers Sharing a test case with load test IE, Firefox, Opera, Safria, Chrome Modifying a test tool configuration without change in a test case TABLE III. LOAD TEST IN IU SYSTEM Test cases load parameters load parameters alteration Load Test Sharing a test case with UI test 15 requests at a single thread; 30 requests at five threads; 50 requests at ten threads; 70 requests at 10 threads; Each request in a certain interval of 2ms Modifying a test tool configuration without change in a test case TABLE IV. COMPARING RESULT OF EXPERIMENT Test Reusability Test Expansibility Test Browser Test

5 Compatibility Comprehensiveness Traditional Test Test Library Architecture Framework (TLAF) Data-Driven Testing Framework(DDTF) Key words or table driven test framework The Hybrid Test Automation Framework A Test Automation Framework Based on WEB