Whitepaper on Test Automation Mechanisms for Mobile Apps

Transcription

1 Whitepaper on Test Automation Mechanisms for Mobile Apps Fiserv India Pvt. Limited A - 44/45, DLF IT Park, Sector - 62, NOIDA Uttar Pradesh INDIA By: Kafeel-Ahmad

2 Table of Contents 1. INTRODUCTION Purpose CHALLENGES IN MOBILE TESTING MOBILE APPLICATIONS MOBILE TESTING STRATEGY Emulators Simulators Real Devices MOBILE AUTOMATION Tools for Mobile Automation Automation using Open Source Automation using Real Devices Automation using Cloud CONCLUSION Whitepaper Zest 2013 ConfidentialInternal Page 2 of 11

3 1. Introduction 1.1. Purpose For everybody who uses a smartphone or tablet apps are indispensible.an app is an application that is developed for a specific environment.within the mobile context, the commonly used name is mobile app. For a long time, only apps for personal use were in the spotlight, but nowthis has changed. The usage of apps for business purposes is increasingsignificantly. Companies are using apps for banking, sales, marketing,or internal communication. In addition, B2B or enterprise apps, whichfacilitate interaction with back end tools and mobile devices by meansof web services or cloud platforms, are capturing market share. In thecourse of this development, the demand for methodical quality managementis increasing. This article will show various automation tools and technologies which are used for performing mobile automation using various tools available in market. 2. Challenges in Mobile Testing Due to the wide variety of display sizes and resolutions across mobile devices and models, the rendering and positioning of images canvary widely. The network latency will be unpredictable when applications communicateover network boundaries. Whitepaper STC 2013 ConfidentialInternal Page 3 of 11

4 Testing with real devices allows testers to verify that what is shown on the device s LCD looks cosmetically correct, but provides very little access to any type of diagnostic information other than whatis shown on the LCD of the real device. For most performance problems (e.g. slow page loads) testing with a real device does not provide the detailed root-cause analysis thatis needed to solve the problem. With an actual device, it is not possible for the tester to know theurl redirect/page inner text context. Test automation is not feasible in a seamless manner on an actual device. Multiple platforms to test with variety of connectivity modes and test execution across different browsers or environments is tedious Whitepaper STC 2013 ConfidentialInternal Page 4 of 11

5 3. Mobile Applications Native Apps Hybrid Apps Mobile Web Apps Single Platform Affinity Written with Platform SDK's Must be written for each platform Access to all native API's Faster graphics performance AppStore distribution Cross-platform affinity Written with web technologies (HTML5, CSS and JavaScript) Runs locally on the device, supports offline Access to native API's AppStore distribution Cross-platform affinity Written with web technologies (HTML5, CSS and JavaScript) or server side (PHP, ASP.Net, etc.) Runs on web server, viewable on mutiple devices Centralized updates 4. Mobile Testing Strategy Whitepaper STC 2013 ConfidentialInternal Page 5 of 11

6 Let s take a look at the kind of testing that we should target for mobileweb and apps. Usability testing Functional testing manual UE testing Regression testing automation Accessibility and security testing Performance testing All mobile testing centers are around the emulators, simulators, tools and devices. So, let s understand what an emulator and a simulator are! 4.1. Emulators An emulator is a hardware replica of an actual mobile device. The emulatorsimulates the mobile device and enables the tester to test a mobile appon a PC without having to test it on an actual mobile device. There are three types of emulator: Device emulators Operating System (OS) emulators Browser Emulators Whitepaper STC 2013 ConfidentialInternal Page 6 of 11

7 Advantages of using emulators are the following: One of the greatest advantages of testing with emulators is that anemulator will let you know exactly what is happening behind the devicelcd, allowing a tester to do a debug and actually open up a screen to seewhat is happening. The tester can also provide the developer with screenshots, high-levelinformation, and data messages. Emulators tend to be cost-effective because most of them are freeware. Because the virtual device (emulator) is in control of its own software stack, testers can collect important information on each component of a content page. Some examples of this information include inner text and page redirect URLs. Many types of content compatibility tests such as verifying image sizes or identifying broken links can be accurately and quickly performed Simulators Simulatorsmimic the software of the device for testing purposes. Testers typicallyuse the native browser of a PC for mobile browser simulation. To get a native browser to simulate a mobile browser, testers change the User Agent settings in the native browsers. This approach is typicallyused for automated functional testing. Whitepaper STC 2013 ConfidentialInternal Page 7 of 11

8 With simulators, testing can be done quickly and easily without the useof emulators. In addition, simulators are cost-effective because they canbe used without purchasing any new software. For Firefox browsers simulating iphone and Android, automation is possiblewith tools like Quick Test Professional. But this is mainly from the functional rather than look and feel perspectiveand is primarily used for functional automation testing. To reduce manualeffort/cost, one of the popular ways is to go for functional automation. For mobile web, this is the way automation can be successfully done andcan greatly reduce effort Real Devices By definition, devices are physical resources which need to be managed.unlike simulators, where additional devices can be either additionalsoftware installed or simple configuration, real devices are somethingphysical to own. Whitepaper STC 2013 ConfidentialInternal Page 8 of 11

9 5. Mobile Automation Because mobile testing must be performed on many different combinationsof devices, browsers, and operating systems, it is time-consuming and costly to do all of the testing manually. Test automation can be usedto reduce the time and costs associated with testing. In addition, testautomation can be used to increase the productivity of the test team. Butkey call out, automation testing is not intended to replace manual testingit is to reduce the effort/time to market for the product. There has to be a clear goal/objective defined before investing inmobile apps testing. Because it has its own challenge like tools support,personnel learning curve and infrastructure support as well Tools for Mobile Automation Here are some automation tools which are well recognized for mobileapp automation: Perfecto Mobile s MobileCloud Platform (plugs into QTP, supportsselenium) SeeTest Mobile Cloud (plugs into QTP, TestComplete, C#, RFT, Java,Perl, and Python). Keynote DeviceAnywhere Testdroid Cloud Jamo Solutions (plugs into QTP) SILK for Mobile FoneMonkey 5.2. Automation using Open Source The open source automation mechanism is based on getting the SDK for the android apps e.g. Robotium which gets integrated with Eclipse for script development. Similarly Monkey talk provides its own editor for script recording and playback which works also for an ios and Android apps. In open source automation the script is developed by referencing the android SDK and exposing the inbuilt support for the automation development. Whitepaper STC 2013 ConfidentialInternal Page 9 of 11

10 Features and Benefits: Native, Mobile Web, and Hybrid ios and Android Apps support. Emulators and Simulators are used for automation development and execution. Easy for a newbie to learn without any investment on devices or tools. Provides framework to work with API s and prepare their own scripts 5.3. Automation using Real Devices For performing mobile automation the real devices are usually preferred instead of the emulators or simulators since they perform the execution in real time. There are numerous tools in market for performing such type of automation where a real device is plugged in and the script execution is done on the actual device which gives real picture. Features and Benefits: Records on real devices (no jailbreak required). And on Emulators. Objects identified by Native / Web properties. Same test runs on different devices (e.g. android test runs on ios). Plugs into different tools like UFT (QTP), MSTest/VisualStudio/TFS, RFT, TestComplete, C#, Java, Perl, Python. Mobile device connectivity: local (USB to tester PC) Automation using Cloud It allows users to execute automated testing on real devicespresent anywhere in the world. Users can automate their testing using a web-based interface provided by the tool, which allows the creation of keyword-based scripts creation. Once the scripts are done, users anywhere in the world can access the devices from the cloud and run the scripts.qa teams can quickly create and automate test scripts to capture, verify and replay real user interactions on live mobile devices. Features and Benefits: Easy and efficient script creation. Utilization of coding flows including conditions, loops, flow commands etc. Detailed reporting of issues along with screenshot in readable format Saves time and money on problem reproduction by using real time online shared devices on cloud. Whitepaper STC 2013 ConfidentialInternal Page 10 of 11

11 6. Conclusion Test automation is a booming technology which has to be learnt as its importance is understood by the market trend which is tilting more towards smartphones and tablets usage replacing laptops and desktops. So for performing quick testing in a competitive market where multiple platform and devices support has to be tested for any app in very short span of time, automation is the only way which can save us in achieving the quality goals. Whitepaper STC 2013 ConfidentialInternal Page 11 of 11