Critical Analogy of Automated Testing Tools

Size: px

Start display at page:

Download "Critical Analogy of Automated Testing Tools"

Stewart Lambert
5 years ago
Views:

Volume-6, Issue-2, March-April 2016 International Journal of Engineering and Management Research Page Number: 187-192 Critical Analogy of Automated Testing Tools Shivani Varshney 1, Himani Gupta 2 1,

1 Volume-6, Issue-2, March-April 2016 International Journal of Engineering and Management Research Page Number: Critical Analogy of Automated Testing Tools Shivani Varshney 1, Himani Gupta 2 1, 2 Research Scholar, Department of Computer Science, Galgotias University, Greater Noida, U.P., INDIA ABSTRACT Software aims at evaluating an attribute or capability of a program or system and determining that it meets its required results. Testing automation tools enables developers and testers to easily automate the entire process of in software development. It is to audit & customize source code. Performing in an effective manner produces arete software. With agile practices getting more popular Test Automation is an important factor in achieving continuous delivery. There are so many tools available in market for test automation. It includes both Open Source and Proprietary Tools/Framework. It is a very challenging task to pick up best tool for automation. The objective of the paper is to conduct a comparative study of automated tools such as Selenium free source, HP Quick test professional (QTP), Telerik Test Studio, Watir and Sahi. This work computes and compares these automated software tools to determine their application and effectiveness. Keywords--- Selenium free source, HP Quick test professional (QTP), Telerik, Watir and Sahi. I. INTRODUCTION Software is a process used to identify the correctness, completeness and quality of developed computer software. It is the process of exercising and evaluating a system or system components by manual automatic means to verify that it satisfies specified requirements or to identify differences between expected and actual results. The fundamental goal is to analyze the features supported by various functional tools that aid in minimizing the resources[1]. There are two ways of that are manual or automation. Manual carried out by the testers. Manual Testing is a process where a tester often follows a written test plan that leads them through a set of important test cases It requires a tester to play the role of an end user, and use most of all features of the application to ensure its correct behavior, plan that leads them through a set of important test cases. A test case in software is a set of conditions written for particular applications & tester run all these conditions to ensure the correct functionality of the software applications. Manual is a laborious activity that requires the tester to possess a certain set of qualities; to be patient, observant, speculative, creative, innovative, open-minded, resourceful, opinionated, and skillful. Manual helps discover defects related to the usability and GUI area. Any new application must be manually tested before its can be automated. Manual requires more effort, but is necessary to check automation feasibility The problems with manual are, it is very time consuming process, not reusable, has no scripting facility, great effort required, and some errors remain uncovered. Automation covers all the problems of manual. It increases the test execution speed, more reliable, repeatable, programmable, comprehensive. [2]Automation Testing is running test cases where manual intervention is not required to run each one. It uses special software to write & execute test cases to compare the actual outcome with the predicted outcome. Aim of present paper is to perceive the foremost automated tool.[3] The following section highlights a brief review of automated tool. The remaining sections are organized as follows. Section 3.0 gives an insight to the evolution of Automated Testing Tools and its components. Section 4.0 gives a description of HP QTP, Selenium, Sahi, Telerik Test Studio and Watir. A critical analogy of these automated tools is presented in Section 5.0. The results are depicted in Section 6.0 along with the scope for future work. and Section 7.0 presents the conclusion. II. TESTING TOOLS In software, test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of 187 Copyright Vandana Publications. All Rights Reserved.

actual outcomes and predicted outcomes[4]here the classification of software tools has been depicted.

I showing classification of tools MANUAL TESTING AUTOMATION TESTING If you have to run a set of tests repeatedly automation is If the Test

its most likely to perform manual It allows the tester to perform more ad-hoc(random Helping performing compatibility - the ).

Short term cost are reduced It gives you the ability to run automation scenarios to perform regressions in a short time The more time tester

Simultaneously on different machine with Automation can be carried out simulataneously different OS platform combination is not possible

combination Executing the Build Verification Testing(BVT) is very Automation is very useful for automating the mundane and tiresome in

HP QUICK TEST PROFESSIONAL QuickTest Professional is the solution for functional test and regression test automation.

and environments. It performs functional and regression through a user interface such as a native GUI or web interface.

2 actual outcomes and predicted outcomes[4]here the classification of software tools has been depicted.[5] Manual Testing Acceptance Black Box Whitebox Unit Automation Testing Selenium QTP Watir Sahi System Telerik Test studio Fig. I showing classification of tools MANUAL TESTING AUTOMATION TESTING If you have to run a set of tests repeatedly automation is If the Test cases have to be run a small number of times a huge gain. its most likely to perform manual It allows the tester to perform more ad-hoc(random Helping performing compatibility - the ).. software on different configuration. Short term cost are reduced It gives you the ability to run automation scenarios to perform regressions in a short time The more time tester spends a module the greater It gives you ability to run regressions on a code that is the odds to find real use bugs.. continuously changing. Simultaneously on different machine with Automation can be carried out simulataneously different OS platform combination is not possible using on different machine with different OS platform manual. combination Executing the Build Verification Testing(BVT) is very Automation is very useful for automating the mundane and tiresome in Manual Testing. Build Verification Testing(BVT). Table I Showing the difference between Manual Testing and Automated Testing. III. HP QUICK TEST PROFESSIONAL QuickTest Professional is the solution for functional test and regression test automation. In other words, QuickTest Professional is a tool used for automation of functional and regression tests for various software applications and environments. It performs functional and regression through a user interface such as a native GUI or web interface.[6] QuickTest Professional uses VBScript scripting language to specify the test procedure and to manipulate the objects and controls of the application under test. Although HP QuickTest Professional is usually used for "UI Based" Test Case Automation, it can also automate some "Non-UI" based Test Cases such as file system operations, database or Web services. HP QuickTest Professional 11 is current and latest available stable release from HP Software Division.[7] SELENIUM: 188 Copyright Vandana Publications. All Rights Reserved.

3 Selenium provides a rich set of commands for fully your web-app in virtually any way you can imagine. The command set is often called selenese. These commands essentially create a language.[8] In selenese, one can test the existence of UI elements based on their HTML tags, test for specific content, test for broken links, input fields, selection list options, submitting forms, and table data among other things. In addition Selenium commands support of window size, mouse position, alerts, Ajax functionality, pop up windows, event handling, and many other webapplication features. The Command Reference lists all the available commands.[9] A command is what tells Selenium what to do. Selenium commands come in three flavors : Actions, Accessors and Assertions.Actions are commands that generally manipulate the state of the application. They do things like click this link and select that option. If an Action fails, or has an error, the execution of the current test is stopped.many Actions can be called with the And Wait suffix, e.g. click And Wait. This suffix tells Selenium that the action will cause the browser to make a call to the server, and that Selenium should wait for a new page to load. Accessors examine the state of the application and store the results in variables, e.g. storetitle. They are also used to automatically generate Assertions. Assertions are like Accessors, but they verify that the state of the application conforms to what is expected. Examples include make sure the page title is X and verify that this checkbox is checked.[10] WATIR: Watir, is an open-source (BSD) family of Ruby libraries for automating web browsers. It allows you to write tests that. It is simple and flexible. Watir drives browsers the same way people do. It clicks links, fills in forms, press buttons. [10].Watir also checks results, such as whether expected text appears on the page. Watir has a multi browser (& OS) support. Watir has a rich API(Application Programming Interface).It is very convenient as the user do not need to have supplementary knowledge of programming..watir-classic makes use of the fact that Ruby has built in Object Linking and Embedding (OLE) capabilities. As such it is possible to drive Internet Explorer programmatically. Watir-classic operates differently than HTTP based test tools, which operate by simulating a browser. [10] Watir-webdriver is a modern version of the Watir API based on Selenium. Selenium 2.0 (seleniumwebdriver) aims to be the reference implementation of the Web Driver specification. In Ruby, Jari Bakken has implemented the Watir API as a wrapper around the Selenium 2.0 API. Not only is Watir-webdriver derived from Selenium 2.0, it is also built from the HTML specification, so Watir-webdriver should always be compatible with existing W3C specifications. SAHI: Sahi is an open source tool for web applications, with the facility to record and playback scripts. Developed in Java and Javascript, this tool uses simple Javascript to execute events in the browser. Sahi runs as a proxy server which intercepts traffic from the web browser and records the web browsing actions. Sahi can play back those recorded actions by injecting Javascript into the browser so it can access elements in the web page. This makes the tool independent of the website/ web application.[11] TELERIK TEST STUDIO: Telerik Test Studio is a automated software tool that offers a Visual Studio plug-in and a standalone application to help you build functional web and desktop tests, customize them in code within Visual Studio, or convert them to unit tests. [9] The tool provides industry-best test automation support for any application - Silverlight, WPF, AJAX, HTML5, and MVC included, as well as support for Visual Studio What s more, Test Studio is the best way to test RadControls for ASP.NET AJAX, Silverlight and WPF. The test recorder automatically detects the controls and exposes commonly used verifications. Use Test Studio standalone app for performance and load too.[12] IV. EVOLUTION AND CRITICAL ANALOGY OF AUTOMATED TESTING TOOLS Evolution of tools evolve from 1980 However the evolution of the Automated tool evolve from 1999 to Today, assuring the quality of any software has become very challenging. The size and complexity of applications have increased exponentially, while end-users are becoming more and more demanding. Under these circumstances, verification and validation activities tend to lose their momentum and make compromises this lead to the evolution of Automated Testing Tools. With every second new technology is changing and new technology is overcoming the problems that were present in the old technology. There are so many test automation tools (both open source and commercial) available in the market and it is extremely important to choose the right tool that best suits your overall requirements. But it is always wise to set a bench mark by comparing HP Quick Test Pro, Selenium, Sahi, Telerik Test Studio and Watir. The Goals of Automation can be 1.Speed up Test Execution 2.Achieved Improved Test Coverage 3.Ensures Consistency, Accuracy and Efficiency 4.Reliability and Transparency 5.Return on Investment (ROI) 189 Copyright Vandana Publications. All Rights Reserved.

4 6.Quality Repeatability and Re usability S NO. YEAR TESTING TOOL Software test automation First version of quick test Test Frame published Agile Manifesto is published JIRA 1.0 released SPECIFICATION Book by Mark Fewster and Dorothy Graham Early 1999mercury interactive release Quick test as an easier approach to creating and maintaining SAP automated test.it later become Quick test Professional The Test methodology Test Frame is published by Hans Buwaldo,Dennis Jansen and Iris Pinksterworking for British/Dutch IT company Logica CMG.Test Frame focuses on the uses of Test Automation A conference in Utah by 17 representatives from different development methodologies leads to the Agile Manifesto in which the Twelve Principles of Agile Software are published. The first version of the popular bug tracking tool JIRA is released by the Australian software company Atlassian Software Agile Testing Quadrant Selenium developed On his weblog Brian Marick publishes the Agile Testing Quadrant in August The quadrant combines two distinctions in ; the distiction between business facing or technology facing and the distinction between support programming or critique the product. The popular web application test tool Selenium is developed at Thought Works (Chicago) by Jason Huggins Sahi It is released under an Apache License 2.0 Open Source License Telerik Test Studio Telerik Test Studio is a Windows-based software tool for web and desktop functional, software performance, load and mobile application developed by Telerik Watir Watir (Web Application Testing in Ruby, pronounced water), is an open-source (BSD) family of Ruby libraries for automating web browser. Watir was primarily developed by Bret Pettichord and Paul Rogers Table II Evolution of Automated Testing Tools Success in any Test Automation (TA) effort lies in identifying the right tool for automation. There are so many test automation tools (both open source and commercial) available in the market and it is extremely important to choose the right tool that best suits your overall requirements. But it is always wise to set a bench mark by comparing HP Quick Test Pro, Selenium, Sahi, Telerik Test Studio and Watir. Our work has involved the analogy of the automated tools.this paper has compared Quick Test Professional, Selenium,Sahi, Watir and Telerik Test Studio. The comparison that is provided in this table is based on Licensing Cost, Script Creation time, Object oriented language support and scalability, support for operating system/platform, database framework, database application, report generation mobile support,client and drawbacks. On the above mentioned features we have provided the analogy of this work. 190 Copyright Vandana Publications. All Rights Reserved.

5 CRITICAL ANALOGY OF AUTOMATED TESTING TOOL FEATURE S Licensing Cost HP QTP SELENIUM SAHI TELERIK TEST STUDIO QTP License cost varies It is open Open source The cost of telerik from country to country, source.no software. No studio varies from depending upon the licensing cost licensing cost is version to version Test license type it is between is required required. Studio Functional cost $ $ The best. is $1,799. Test Studio place to inquire would be load cost is $1,799 and HP support. the cost of Test Studio Ultimate is $2,999. WATIR The pricing of sahi in India is Rs 22,275 per user and the cost of sahi outside India is $495 per user.bsd license required Script Creation Time Object Oriented Language support and Scalability Support for operating system/plat forms Framework Database application s Report Generation Less High Less Less Less Scripts can be Developed only in VBScript or JavaScript QTP supports Windows XP Easily integrated with HP Quality Center or HP ALM (separate commercial products) It supports relational databases, so anything that uses standard SQL is supported like SQLServer, Oracle, MySQL, Access, Quality Center has inbuilt awesome dashboards Supports Java,.Net, Perl, PHP, Python and Ruby Supports Windows PC/MAC/UN IX Platforms Selenium + Eclipse + Maven / ANT + Jenkins / Hudson & its plugins / Cruise Control + TestNG + SVN Access Data base using JDBC. Report is supported through test Sahi script,java,ruby sahi script Operating system and browser independent It easily automates applications built using ExtJS, ZK, Dojo, YUI or any other framework. Sahi supports database connectivity, where script can connect to database during runtime to execute any queries. Impressive reports and logs C#,ASP.net,VB.net Windows Server 2003 (users can run load tests against Windows Server 2003 but not on it); Windows Server 2008; Windows Server 2012; Windows Vista; Windows XP (minimum for load is XP SP3), Windows 7 NET 4.0 /.NET 4.5 as of version /.NET SP1 for Visual Studio plug-in Microsoft SQL Server 2008, 2008 R2, and Express, Developer, Professional, and Enterprise versions are all supported. Very rich image based reporting Ruby Operating System cross platform RSpec,Cucumber,Test by unit Ruby libraries for database support and it does not work with MS SQL,MY SQL Impressive reports 191 Copyright Vandana Publications. All Rights Reserved.

6 runner and various logs. Clients Hewelette Packard Blackberry10,ios driver,windo ws phone Mobile Different commercial Android, Support product i.e. iphone & HP UFT Mobile (formerly ipad, known as Mobile Cloud Blackberry for QTP).,Headless Drawbacks Cost is high License and maintenance WebKit Identifying locators that support common attributes like id, names,xpat H,javascript DOM etc Fiserv,HCL,Tesc o, Mphasis ipad Only used for Web Testing for now. Limited use, thus not many people know about it. Sony,IBM,Toshiba, Nestle,Microsoft Supports mobile devices Table III Critical Analogy of Automated tools Some features of Telerik Just Code are available only in particular languages Facebook,SAPYa hoo,bell Supports ios and android mobile Doesnot support browser plugin like Java applets,adobe flash and microsoftsilverlig ht V. CONCLUSION AND FUTURE WORK Testing automation tools enables developers and testers to easily automate the entire process of in software development. Automated Software is the best way to increase the effectiveness, efficiency and coverage of software.through this research paper a comparative and concise study of the evolution of automated tools has been given. We discussed various tools such as QTP, Selenium, Sahi, Telerik Test Studio and Watir.One can use the tool according to their need and budget. All tools have their own advantages and disadvantages. Analyzing the different tools makes us clear that the best tool among these tools is Telerik Test Studio. It is best for performing the Automation Testing. While other tools QTP, Selenium,Sahi,and Watir have more snag as compared to Telerik Test Studio.In the Telerik Test Studio more advancement is required as per the demand of. Test Studio provides better value if we do cost benefit analysis. REFERENCES [1] Innovative approaches of automated tools in software and Innovative approaches of automated tools in software and current technology as compared to manual -jan 2009 [2] Vishawjyoti, Sachin Sharma Study and analysis of automation techniques JGRCS Journal of Global Research in Computer Science Volume 3, No. 12, December 2012 [3] Mohd. Ehmer Khan, Different Forms of Software Testing Techniques for Finding Errors, IJCSI International Journal of Computer Science Issues, Vol. 7, Issue 3, No 1, May [4] Jovanovich and Irena, Software Testing Methods and Techniques, May 26, [5] Prof VN Maurya,Er. Rajendra Kumar Analytical study on Manual Testing vs Automated Testing using with simplistic cost model International journal of Electronics and Electrical Engineering Volume 2 Issue 1 Jan 2012 [6] Manjit Kaur,Raj Kumari Comparative study of Automation Testing tools Test Complete and Quick Test pro IJCA volume-24, no 1,June 2011 [7] Richa Rattan Comparative study of Automation Testing Tools Quick Test Pro and Selenium International Journal of Computer Science and Information Technology - Vol 3 No june 2013 [8] Nidhi Uppal,Vinay Chopra Design and Implementation in Selenium IDE with Web IDE International Journal of Computer Applications Vol 46- No-16 May 2013 [9] Prasanth Yalla,Dr.L.S. Reddy,M.Srinivas,T.Subha Mastan Rao Framework for Testing web application using Selenium tool with respect to integration IJCST Vol. 2,Issue 3,September [10] Nishi Gogna Study of browser based Automated Test Tools WATIR and Selenium International Journal of Information and Education technology Vol. 14,No 4,Aug 2014 [11] Sahi entry in Wikipedia: Http: //en.wikipedia.org [12] Telerik Test Studio entry in Wikipedia: Http: //en.wikipedia.org. 192 Copyright Vandana Publications. All Rights Reserved.

Automated testing in ERP using Rational Functional Tester

Automated testing in ERP using Rational Functional Tester A.Josephine Prapulla 1, Dr.L.Manjunatha Rao 2 1 Research Scholar, Bharathiar University. Coimbatore. 2 Professor and Director Dr. B. R. Ambedkar