Tech Mahindra & Experitest Co-webinar: Selenium vs UFT: Which is Better for Your Mobile Testing Strategy? Feb 2015
Speakers Guy Arieli CTO Experitest Mahendra Singh Project Manager (MATE) Tech Mahindra
Agenda 1 What is UFT? 5 Demo 2 UFT for Mobile 6 Seleinum vs UFT for Mobile -Which 3 What is Selenium? one is right for your testing team? 4 Selenium for mobile 7 Case studies
UFT and Selenium - Tool Assessment
What is UFT, QTP? UFT is the HP solution (previously called QTP) for Test Automation.. Runs on VB script A UI based automation tool. Focus on Web and desktop technologies
UFT for Mobile Experitest Mobile Add on for UFT Mobile Menu Devices & Applications Managing Full & complete integration into HP UFT software Record a test, edit it, run it, and view results (report) Integration of object repository (mobile object properties) Object Spy include Native & Web identification methods Integration into HP s Quality Center & Performance Center
Competitive Landscape- Plug in for UFT Mobility Capabilities Support all OS. Support all Mobile devices Support all app types Experitest Mobile Add on for UFT Enterprise readiness Integration to all CI stability scalability Remote capabilities HP End to End solution
What is Selenium? Selenium is an open source test automation framework that support cross browser test automation Selenium 2 also known as WebDriver Selenium supports perl, php, python, objective-c, java, Microsoft.net, ruby, and more. It is not a tool with a UI it uses a programming language to use it specifically This project gain extremely high popularity and support due to HP pricing and luck of support for SMBs
Selenium for Mobile Experitest Mobile Plug in for Selenium Experitest provide enterprise grade solution based on WebDriver API called MobilWebDriver Providers interface to interact with Mobile Application All function of Seetest Automation Tool can be executed in Java Selenium environment using MobilWebDriver
Competitive Landscape- Plug in for Selenium Mobility Capabilities Support all OS. Support all Mobile devices Support all app types Experitest plug in for selenium Enterprise readiness Integration to all CI stability scalability Remote capabilities End to End solution Selenium
UFT Vs Selenium Which is better for your testing team
UFT vs Selenium Which is better for Mobile Strategy? Supported language Mobile Add on for UFT VB Script Experitest Mobile Plugin for Selenium JAVA
Language Supported Java Using IDEs like Eclipse made writing Java easier then working with any other scripting language. Every student that graduate BSC in computer science knows Java. Java is Object Oriented, it can be very important in large automation projects. Microsoft VbScript Scripting Language. Lack full features of a programming language. Interpreted Considered to be easier to learn Vbscript cannot run on MAC. Easy to integrate with your enviourment backend
UFT vs Selenium Which is better for Mobile Strategy? Supported language Mobile Add on for UFT VB Script Experitest Mobile Plugin for Selenium JAVA Object Identification Descriptive Programing
Object Repository vs. Descriptive programming UFT uses Object Repository (by default). *** Selenium uses 'descriptive programming' (by default). With Experitest Mobile plugin for Selenium you can use object repository or descriptive programming in your WebDriver project. Object Repository- Pros When identification of an object changed, the update is done in a single place. Support internal tools like 'Object Spy'. Object Repository- cons Hard to source control. Hard to merge. Can reduce the test visibility. Can cause inconsistency in behavior.
UFT vs Selenium Which is better for Mobile Strategy? Supported language Object Identification Descriptive Programing Test Recording capabilities Xpath Object Spy Mobile Add on for UFT VB Script Experitest Mobile Plugin for Selenium JAVA
Programing skills? Yes / NO To build a successful automation project you will need to have programing capabilities No coding, approach is very limited and in many cases will fail Recording capabilities can ease the tool learning curve <!!!> Many testing team has programing knowledge in Vbscript Most of the computer science graduates has good Java knowledge
UFT vs Selenium Which is better for Mobile Strategy? Supported language Object Identification Descriptive Programing Test Recording capabilities Xpath Object Spy Parallel Execution Mobile Add on for UFT VB Script x Experitest Mobile Plugin for Selenium JAVA
Parallel Execution UFT can only perform one test per machine. Selenium can perform multiple & simultaneous tests
Demo
End-to-End Product Suite - Manual, Automation, Monitoring, Load, Network Virtualization, Manager G SeeTestManager U SeeTestAutomation Automation tool Same test runs on different devices Plugs into all testing & CI environments: Mobile Add-On for UFT Full & complete integration into HP UFT: Record a test, edit it, run it, and view results Integration of object repository and more SeeTestManual Manual testing tool Mini-recorder for automating short, repetitive, manual scenarios Report with screenshots of each step Manual to Automation Script Convertor Free Tool! N L SeeTestCloud Device Lab located onsite (in the customer VPN) Devices can be reserved and accessed by any remote user Used in conjunction with SeeTestAutomation/Manual for testing of remote devices SeeTestNetworkVirtualization Device performance under varying network conditions Simulating mobile network bandwidth latency and reception while on the move SeeTestLoad Real device generated mobile load Full integration to LoadRunner, Jmeter, etc.
Case Study Banking
Customer Profile: Multinational banking and financial services company Major global financial services provider engaged in personal banking, credit cards, corporate and investment banking and wealth and investment management. Headquartered in UK Spans over 50 countries across Europe, America, Africa & Asia ~150,000 employees What needed to be tested? The bank s personal and business account management apps Mobile applications in tens of languages Ranked in top 10 largest banks worldwide Revenue of ~$30 Bn.
Implementation Years working with the tool: 1 Over 25 SeeTestAutomation stations Number of Automation engineers working on the project: 25 2 geographical locations around the world (US, Europe, Africa and China) Number of tests before: 0 Number of tests today: ~2,000 Number of executed tests: ~200,000 (each test can run on more than one device) Build single test that can run on Chrome / IE / Firefox and Safari for Desktop And Chrome / Safari / IE for mobile (ios, Android and WP8.1).
Coverage Total coverage: ~90%. Number of executions of the full cycle: Once a day Full app test coverage for all OS-Cross platform testing of Android ios WindowsPhone8 Full end-to-end coverage of test scenario Support for multiple languages (including English, Spanish, French)
ROI What if we did it manually? A single QA engineer: 56 man years to finish the cycle (lifetime). For one day we would need 20,000 trained manual QA engineers. Another Advantage The R&D engineers can work faster and take more risks According to the bank R&D managers the development speed is 3-4 times faster!
Case Study Transportation
5 Customer Profile: Major National Railroad Passenger Corporation US national rail operator operating over 300 trains and an average of 85,000 passengers a day in USA & Canada. Last year served more than 30 million people. Headquartered in Washington DC, USA More than 20,000 employees Revenue: ~3 Bn
The needs Testing User s Applications Testing Integration with 3 rd Parties Amtrak mobile website Amtrak mobile native application Many user flows such as Save tickets to your phone calendar Integration with Apple Passbook or Samsung Wallet Book one-way and round-trip tickets Search for station information Check train status, board using eticket And more
The Requirements Test the Android, ios and WindowsPhone8 mobile application Have a full testing environment deployed onsite for end-to-end system testing including the integration with Apple Passbook or Samsung Wallet Monitoring capabilities (such as memory and CPU usage) of the Android/iOS devices to ensure large quantities of data do not hamper the app s functionality
SeeTestAutomation Deployment - Description Within 3 Months of Project Kick-Off Implementation ~10 floating licenses SeeTest Automation+ Experitest Mobile Add on for UFT Over 20 test machines running on ios, Android and WindowsPhone8 Onsite deployment with integration to the existing UFT/QC test environment Coverage Coverage of customer s app & website including the integration with Apple Passbook or Samsung Wallet Support for multiple languages (including English, Spanish) Parallel execution of tests Onsite mobile lab using SeeTestCloud 80% Automation; 20% manual On Premise Mobile Device Lab located in USA
Thank you