13 th Annual International Software Testing Conference in India 2013

Transcription

1 13 th Annual International Software Testing Conference in India 2013 Cost Effective Solution for Cross Browser Testing Presented By: Vanitha Suresh Organization: Hexaware Technologies H5, SIPCOT IT Park, Navallur Post, Kancheepuram District, Chennai (India) Hexaware Technologies Limited. All rights reserved.

2 1. Abstract Test your site in any browser instantly. Cost associated in cross browser testing has always been a key problem where allocating dedicated systems, configuring and maintaining them had been challenging. This paper we seek to highlight the details and the benefits of a reliable yet inexpensive approach using Open Source tool integrated with Sauce Labs to reduce the cost involved in cross browser testing. 2. What and Why Cross Browser Testing With wide range of web browsers available, and with end users using different web browsers to access web applications, it has now become crucial to test web applications on multiple browsers. On different browsers, client components like JavaScript, AJAX requests, Applets, Flash and Flex etc. may behave differently. Also for different browsers you may have different ways on how requests are processed on server side based on the user inputs from client browser. So just testing your web application on single web browser is not enough. You need to make sure that your web application works fine across multiple browsers. Cross Browser Testing is a process to test web applications across multiple browsers. Cross browser testing involves checking compatibility of the application across multiple web browsers and ensures that the web application works correctly across different web browsers. Cross Browser testing involves testing both the client side and server side behavior of the Web application when it is accessed using different Web Browsers 3. Key Challenges Involved in Cross Browser Testing Lack of support to regular updates to browsers and new browser versions Cost overhead in setting up and maintaining complex test environments and allows running tests in parallel to accelerate deployment schedules Getting cross-browser test solution that supports all browsers across desktop and mobile OS Addressing run-time conflicts issues when running multiple versions of the same browser. Cost involved in allocation of system with different operating systems Cost involved in allocation of dedicated systems configured with particular browser and their versions Costs associated with specialized automation testers System utilization is only up to 20% - the systems allocated areutilized only during the execution phase where the dedicated machines with all configurations are idle most of the time Hexaware Technologies Limited. All rights reserved. Page 2 of 2 Page2 of 2

3 4. Open Source Automation Framework Integrated with Sauce Labs The proposed automation framework has been developed to cater to the challenges described above. The overall costs associated with cross browser testing are minimized by the use of Sauce Labs and Selenium. The functional automation tool Selenium along with Sauce Labs is used for automation. Some of the key benefits of the proposed framework are: One dedicated system with Selenium and Sauce Labs Test your site in any browser instantly SauceLabs provides cloud-based service which allows us to quickly test web applications over 150 browsers and platforms including Windows, ios, Android & Mac OS. Use of Sauce Labs helps each test to run on its own, freshly booted, secure virtual machine Virtual machines available at any time you need it A brand-new virtual machine guarantees security and repeatability for every test Free of settings from previous browser sessions Free of cached information and cookies All Sauce VMs with a particular OS/browser combination are identically configured, so you can be sure you're working with the same setup every time On running tests on Sauce, all browser requests are sent through Selenium's HTTP proxy server, which enables our browsers to accept any self-signed certificate Use of Selenium ensures that the automated script can be developed in the language in which resources are available within the organization, Selenium scripts can be exported and modified in Java, Groovy, C#, Perl, Ruby, Python and PHP Support available from Sauce Labs on any issue and also from their website Labs.com The automation framework is built using the following toolsets Selenium Automation tool (comprising of Selenium [Web driver], IDE and Driver files) Sauce Labs Hexaware Technologies Limited. All rights reserved. Page 3 of 3 Page3 of 3

4 5. Sauce Labs Framework Methodology Sauce leverages Selenium's client/server architecture and adds the network between the local scripts (made using Selenium's client libraries by us) and the Selenium Server (which is made available using cloud service from Sauce Servers, as are the browsers). It provides built-in video recording of every test, debugging tools and secure tunneling for local or firewalled testing, Sauce Labs makes running, debugging and scaling test suites easier than ever. Sauce supports automated functional testing with Selenium and JavaScript unit testing in all the popular programming languages, frameworks and CI systems. Test Script Sauce Labs Selenium Server Selenium Drivers Application Picture 1: Sauce Lab and Selenium Integration When several test scripts are sent, they communicate with Sauce servers where Server sends step-by-step instructions of what the browser needs to do Once the script is finished, machine will be closed by Sauce and a video and screenshot of every action taken by our test in the cloud will be made available to us. Sauce Labs Server Scripts Sent Video Recording and Screenshot of each action with Test Report Sauce Lab Freshly Created VMs Picture 2: Automation suite execution triggered from Sauce Labs Hexaware Technologies Limited. All rights reserved. Page 4 of 4 Page4 of 4

5 6. Sauce OnDemand Sauce OnDemand is a service that allows you to run your Selenium tests in our testing cloud. The IT burden of maintaining all those machines for Selenium Grid goes away and you can focus on writing tests. With Sauce OnDemand, you can: Run the same script against different configurations in parallel Instantly scale your testing efforts Watch your script run in real time Archive the results of a test after using the record function Share the video playback with stakeholders or customers to showcase new functionality 7. Running Selenium Scripts on Sauce OnDemand SinceSelenium scripts leverage an external framework for their structure, the actual running and results-gathering process is also framework dependent. If you want to use Sauce OnDemand from within a Selenium script, there is an extra bit of configuration necessary. Direct Sauce OnDemand Integration - The best way to run tests against Sauce OnDemand is to go directly to the Sauce Labs servers. Selenium scripts need to connect to a Selenium server in order to launch and control a browser. Normally this would be either localhost or another machine on your local network. For OnDemand, that server URL will be Labs.com port 80. To achieve this connection, additional information like the username and the access key is added as parameter while initializing the Selenium object. '{"username": "your username here", "access-key": "your access key here", "os": "Linux", "browser": "firefox", "browser-version": "3."}' Hexaware Technologies Limited. All rights reserved. Page 5 of 5 Page5 of 5

6 The Selenium object will look like selenium = new DefaultSelenium("localhost", "4444", '{"username": "your username here", "access-key": "your access key here", "os": "Linux", "browser": "firefox", "browser-version": "3."}', " Labs.com/"); selenium.start(); A mature suite of Selenium scripts will take a long time to run. Traditional Unit Tests are fast since they involve no disk or network access. Selenium, on the other hand, has that overhead in addition to the browser to contend with. Parallel test execution across machines is required to bring those times down. Picture 3: Snapshot of sample test results in Sauce Labs Hexaware Technologies Limited. All rights reserved. Page 6 of 6 Page6 of 6

7 8. The Selenium Framework Architecture The selenium framework helps to create robust, browser-based regression automation. Scale and distribute scripts across many environments Following activities are performed during automation of functionality: Create a new selenium test The locators of the various objects in the applications are obtained by either invoking the Selenium IDE or Firebug Create scripts using Selenium Webdriver Add verification points to your test Create Run Configuration for execution insauce Labs Run sauce connect to establish a server connection Execute the tests in different OS/Browsers using Sauce Labs View result via screenshots and video insauce Labs Apart from the script the Generic function library is created, this inherits the property file. The property file contains the details of the variables used. The Generic function library comprises of re-usable functions representing a functional flow like Login, Pop-up close, and Logout etc. The Generic function library is inherited by the various test packs. Note:Ports 80 and 443 has to be opened to run sauce connect Hexaware Technologies Limited. All rights reserved. Page 7 of 7 Page7 of 7

8 9. Conclusion The automation framework using Selenium and Sauce Labs is a simple and cost effective alternative for cross browser testing. The only costs associated with it are the costs of Sauce Labs which is low compared to maintaining many dedicated systems with required resources.the scripts can be built to run on different browsers without any additional effort. Almost all the activities that are performed by an expensive tool can be performed using the developed framework at the fraction of cost. As the issues around the costs of the tool have been overcome, the framework provides a viable approach for cross browser testing. 10. About the Author Name:Vanitha Suresh Position:Test automation consultant Software Testing Experience:6 Years Experience:Over 6 years of experience in Test Automation with expertise in client/server applications, SAP, Mainframe, People soft and web based applications. Having Strong knowledge in different automation frameworks and well versed in HP QTP, IBM RFT, Selenium, Silk Test and SAP TAO. Expertise in cross browser automation using Selenium and SauceLabs. Hexaware Technologies Limited. All rights reserved. Page 8 of 8 Page8 of 8