An overview of. Mobile Testing. By André Jacobs. A Jacobs

Transcription

1 An overview of Mobile Testing By André Jacobs

2 THE RISE AND RISE OF MOBILE

3 3 The Apple Story A look at the company that arguably has sparked the explosive growth in smart devices and mobile applications used by billions of people around the world today. Apple iphone is launched January 9, 2007 iphone was introduced at Macworld on January 9, 2007 in San Francisco.

4 4 1,000,000 iphones Sold September 10, 2007 Just 74 days after launch.

5 5 App Store launched July 9, 2008 App downloads July 14, 2008 The App Store opened via an update to itunes. It allowed Apple to control the quality of apps for the first time. App Store Downloads Top 10 Million

6 September 23, 2008, The First Production Android Device. T-Mobile announces the G1, also known as the HTC Dream, as the first device on the market to feature the Android 1.0 platform. 6 Apps a big thing September 9, 2008 Apps a really big thing April 24, 2009 ipad launched April 3, 2010 App Store Downloads Top 100 Million Worldwide. App Store Downloads Top 1 Billion. Apple introduces the ipad.

7 App Downloads In Billions Staggering numbers June 2, Years after the first iphone launch. Both the Apple App store and the Android Play store have roughly 1.2 million apps available for download.

8 8 Now

9 Mobile Adoption 9 Population Billion Active Internet Users Active Social Media Accounts Unique Mobile Users Billion Billion Billion

10 Social Media YouTube 40% Videos watched on mobile devices Billion videos watched daily 500 Million tweets daily 200 Million hours watched daily 70 Million active users Facebook 65% Twitter 80% Videos viewed on a mobile device (2 Billion). There are more than 284 million active Twitter users monthly, 80 percent of which are using mobile devices to tweet (227 Million).

11 Software Test Professional Polls 11 Do you currently have a mobile test initiative? 56% % 2015 March September March Now % 2014 Can you afford not to have a mobile testing strategy??

12 MOBILE CHALLENGES

13 Explosion of wearables Analysts project a boom in the sales of wearables. Some predict sales will climb to 170 million in Mobile Defined Handheld Handheld device or handheld computer. 13 User interface Touch or non-touch display screen and optional mini keyboard. Windows 10 One product, one platform, one store. Android 5.0 The smarts of Android on screens big and small. Portable Both compact and lightweight to is made for portability. Apple ios 8 Connections between apps and devices

14 Increased complexity 14 App Functionality Mobile Specific Operating Systems Devices

15 how it is developed and deployed the speed at which this happens; and the cost point Radically Changing Software Then Now 15 Large feature rich months 6 monthly Major IT installations $ Millions Application Dev Cycle Upgrades Distribution Costs Single purpose app 8-10 weeks Continuous deployment Download $ 50k - $ 100k

16 APPROACH TO MOBILE

17 Real Devices 17 Actual Accurate performance Can connect peripherals Device specific conditions Procurement Physical security Costs Difficult to share (procure multiple)

18 Remote Devices 18 Public / private clouds Web based Variations Physical security Privacy concerns Availability Speed Costs

19 Emulators 19 Simulates the device Use more often in early stages of testing Use existing infrastructure (cost effective) Can t connect peripherals Cannot do carrier testing Not a true reflection Availability of specific versions

20 Windows 20 Other Select an Operating System Android ios Blackberry

21 4.2.x 4.1.x x Select relevant OS versions 4.4.x 4.3.x

22 Specified Devices Samsung Galaxy Note 3, Samsung Galaxy S3 Screen Sizes (including tablets e.g.) 22 Select Devices Availability: Emulators, Real Devices; Remote Devices Supported OS versions

23 23 Combinations Costs Testing Types (add Test Automation?) System, Compatibility, Automation Build the Pyramid

24 IP Protection: Public / Private Private Security 24 Test Automation Solution Automation Tools Lab tools Costs Real / Emulators / Remote Real

25 25 Solution Present to client Costs Agree; Make a selection; and Execute Risk

26 Android 4.2 Android Android Android 4.3 Android 4.4 Android Android Android Android Android 5.0 Android Android Android 4.2 Android Android Android 4.3 Android 4.4 Android Android Android Android Android 5.0 Android Android Test case combinations (14+12) x releases 26 Samsung Galaxy S3 Test case Test case Samsung Galaxy Note 3 Test case Test case

27 27 Jelly Bean ( ) 49% Kit Kat (4.4) 34% Lollipop (5.0) 0% 4.1.x 21.3% 4.2.x 20.4% %

28 RECOMMENDATIONS

29 29 Manual Testing Test on emulators at first Test on selection of real devices Top, middle, bottom Different screen sizes Test on remote devices to fill gaps (might not be possible to procure devices or specific OS versions) Test Automation Build a lab Select a cloud provider Select a suitable test automation tool Performance Testing Focus on backend Don t try to test all devices Simulate bandwidth

30 Testing on emulators Don t rely on emulators only. Invest in real devices or remote devices. Think it is easy Mobile is not just another platform. Have a strategy, plan your approach. Not a once off investment Know that a continued investment is needed to continually procure devices, pay for licenses, rent remote devices. OS updates OS updates might catch you off guard, make sure you safeguard certain versions. (Research) 30

31 Take the leap 31 Success Look out Tips Look for emulators early, Look for operating system versions early. Procure devices early (consider second hand).

32 CASE STUDY

33 Case Study - Roche Diagnostics 33 Test Mobile Diabetes Application Identify issues with different Screen Sizes Multiple Android Versions Shorten Test Cycles; Decrease Cost Multiple Mobile Types Performance Test Backend

34 Case Study - Roche Diagnostics 34 SeeTest Cloud (Private); Remote Devices Mix of real devices and emulators SeeTest Manual Microsoft Visual Studio SeeTest Automation Neotys NeoLoad

35 So Many ilab, 21e Polo Crescent, Woodmead Office Park, Woodmead, Johannesburg