Perfecto Mobile. Deep testing on market devices and quick verification on as many market devices as possible

Transcription

1 October 25, 2013 Perfecto Mobile How to Build an Enterprise Grade Mobile Testing Strategy Presented by Carlo Cadet, Director Product Marketing blog.perfectomobile.com Conventional IT Projects cumulative business value Quick verification on emulators and Must devices Deep testing on market devices and quick verification on as many market devices as possible Continuous Regression Testing on new devices Service monitoring Requirements And Design Implementation Verification GA Maintenance time V 1.0 V Jeff Pa,on, all rights reserved, 1

2 Conventional IT Projects cumulative business value Ø End-to-End test cycle Ø On any device Ø By any team from any location Ø Highly secure and managed Release, Iteration, Nightly, Continuous Release, Iteration, Nightly, Continuous Release, Iteration, Nightly, Continuous GA Release, Iteration, Nightly, Continuous time V 1.0 Fully functional BEFORE Fully 3 featured Jeff Pa,on, all rights reserved, V 2.0 V 3.0 V 4.0 Agile Mandates Automation 4 2

3 Automation Requirements Team Friendly Cloud (Public/Private), HW Dependency (Local) App Style Mul4 - Na4ve, Hybrid, Web based Use Cases Integra4on to QC, Script support, Emulator support, DOM/OCR Object Recogni4on (Hybrid) Are you able to automate your process and testing requirements? Automation is a key for efficient testing across the SDLC Devices Cross Pla.orm support, Device varia4ons Cross Platform Differences Button Images Application Name Position Menu Panel Labels Text Colors and Images 6 3

4 How Is The Mobile Space Different? Mobile App Testing Requirements Functional Validation of Functionality Smoke / Regressions Testing Offline access testing Negative Testing Non Functional Network Strength / Outage / Recovery Different NW Types Peripheral Testing Interoperability (IOP) Voice / SMS interrupts Notifications Battery /Cable Removal Memory Leak Memory Usage Memory Leaks Garbage Collection Performance Testing CPU Usage testing Network Usage Page Render time or activity Render time Usability Testing User Experience Competitive Analysis Expert Review Installation Testing New App Install Uninstall and Reinstall Upgrade testing Security Testing OWASP Vulnerabilities Dynamic Testing Static Code Analysis Data Encryption Language Testing Validation for Locales Images and Text Currencies, time zones etc. Context 8 4

5 Fast & Constant Capability Upgrades AGILE ALM Performance Monitoring Automation Functional Application Types on Various Browsers Native Hybrid Web Sample Screen Sizes (Only Smartphone) QVGA: quarter VGA ( pixels) HVGA: half VGA ( pixels) WVGA: wide VGA ( pixels) FWVGA: full wide VGA ( pixels) nhd: one-ninth high definition ( pixels) qhd: onequarter high definition ( pixels) Various Device/Tablet OEMs Various OS Types and Versions 2.3.3, , , , Richer Engagement Language Tap & Multi tap Double Click / long tap Double click Drag Right click Swipe Drag Pinch & zoom Type Speak Type Rotate / Tilt Snapshot Authenticate Locate Communicate Text Field set / get Auto-complete / correct Text Field set / get Button/icon click Button/icon Radio / checkbox click / on-off Radio Combo / select List selection Checkbox Combo Table / List selection Popups Date / time selection Table Menu QR / barcode scan Securely Stepper approve Exchange information Speech recognition Tabs Voice identification Popups Date / time selection Call Groups / Answer Message send / receive Community tooltips update Contacts usage Navigate 5

6 Near Insatiable Appetite " Do more " Expect more " Express more 11 Building an Enterprise Grade Mobile Testing Strategy 6

7 Expect Rapid Transition Through the Mobile Maturity Stages Grow Coverage, Reliability & Governance Transform Predictability Start What fits my workflow? 13 Device Strategy Devices on/coming to Market Target User Device Strategy Usage Statistics Form Factors OS Versions 14 7

8 3M Device Selection Model But, Managing this is a Juggling Act Coverage Time to Market Locations & Networks Governance Devices 16 Existing Tools Service Level 8

9 The Importance of Mobile Test Automation In An Enterprise ALM Suggested Tips ü Install, Uninstall, Upgrades (With and W/O SD Card) ü Test for incoming events while your app is running ü Check for the clarity of your app error messages ü Check for Android and ios UI Guidelines compliance ü Test network loss & transition, low battery and other device constraints ü Localization Test with different languages (for relevant apps) ü Include HW during your app run (GPS, Camera) ü Think with the end in mind develop automation anchors for repetitive executions (Back to App Home i.e.) 9

10 What An Enterprise-Grade Mobile Testing Strategy Consists Of Maximize Your ROI Adopt The 5 Pillars 1. Inability to meet quality across all platforms 2. Inability to support continuous delivery 3. Inability to scale 4. Device chaos 5. Fail to leverage existing ALM/ tool investments Real Devices + Emulators (True) Mobile Cloud Cross Device Automatio n + Manual End to End Mobile App Lifecycle Extending Existing ALM 10

11 Building Mobile Application Quality Business challenges Better user experience Lowering delivery cost Accelerated delivery models Market dynamics Solutions Leverage managed device clouds Use real devices under real conditions Enable unattended continuous testing Extend ALM investment Business value Raise quality Faster delivery Support current processes Optimize performance Shrink MTTR Aligning strategy, organization, processes & tools True Case Study Top U.S Bank Read More: Perfecto Case Study 11

12 Friendly Tips Choose the right devices Define your test coverage goals Use both Visual and Real Objects Test your mobile app (native, web) across different networks Performance test high value transactions Don t forget interoperability (IOP) testing Monitor the end users are experience Plan and measure reuse How We Do it Differently " True Integrated Quality platform: The Mobile Cloud ü Distributed ü Sharable ü Secure ü Administration ü Scalable ü Assured & managed " Best in class automation (patented ScriptOnce ) " Broad use case support " Choice Local, private and public deployment options " A market leaders ecosystem designed for your workflow " Field Proven! Operating +70K hours of monthly usage " Complete end-to-end solution Manual Automated, Performance Monitoring Support 12

13 Ques%ons Q & A& A carloc@perfectomobile.com h,p:// Thank You 13