Introduction to mobile performance testing

Transcription

1 Introduction to mobile performance testing Igor Markov, Shlomi Zalma, Shane Evans, Wilson Mar June 11 & 13, 2013

2 HP LoadRunner components End User Client(s) Internet/WAN Application Server(s) Database 1 Author Scripts 3 Generate Load 4 Virtualize the Network 5 Measure Impact 6 Analysis, Diagnostics and Reporting 7 Stay Connected HP Live Network 2 Test Scenario Plan and Execute Tests 3

3 Incremental integrated development workflow Executive Scorecards Analytics Portal Check-in & Code Scan Fortify Source Code Analysis Test results LR Analysis Execute build Jenkins Provision devices ALM Lab Management 4 Application Security Assessment WebInspect Run test cases UFT / LoadRunner Deploy app Cloud Delivery Automation HP integrated suite

4 Workflow for performance engineering 1. Plan 2. Provision 3. Verify 4. Script 5. Scenarios Workload Templates Data Scenarios Attributes Cost vs. Benefit Domains Servers Ports IP Addresses SSL certs. Licenses Batch Jobs Network Variability Report Samples Transactions Resource Usage Variations Retry and Recovery Report Templates Reruns Baselines Comparisons Archival 5

5 Create a new script, mobile protocols 6 HP LoadRunner VuGen

6 Author mobile TruClient scripts

7 Recording methods HTTP/HTML App web server 8 TruClient Ajax User Agent > HP LoadRunner VuGen

8 Create a new script, mobile TruClient protocol 9 HP LoadRunner VuGen

9 TruClient recording 10 HP LoadRunner VuGen 11.52

10 Devices with User Agent strings Count: HP LoadRunner VuGen11.52

11 User Agent string example Sample User Agent for Samsung Galaxy S3 720 x 1280 Mozilla/5.0 (Linux; U; Android 4.0; xx-xx; GT- I9300 Build/IMM76D) AppleWebKit/ (KHTML, like Gecko) Version/4.0 Mobile Safari/ HP LoadRunner VuGen11.52

12 Invoke Firefox in VuGen for TruClient 13 HP LoadRunner VuGen11.52

13 HTTP/HTML mobile recording

14 Create a new script, HTML recording protocol 15 HP LoadRunner VuGen11.52

15 16 HP LoadRunner VuGen11.52

16 HTTP/HTML recording methods wizard HTTP/HTML Record & Analyze Traffic Analyze Traffic Record Emulator Proxy TruClient Recording 17 TruClient Ajax HP LoadRunner VuGen11.52

17 Record emulator recording method HTTP/HTML Android APK Record Emulator Device Emulator APK App web server 18 TruClient Ajax HP LoadRunner VuGen11.5x

18 Record emulator requirements HTTP/HTML Java Development Kit Android SDK with ADT Record Emulator 19

19 Record emulator HTTP/HTML Record Emulator 20 HP LoadRunner VuGen11.52

20 Record emulator options HTTP/HTML Record Emulator 21 HP LoadRunner VuGen11.52

21 Proxy recording HTTP/HTML Android APK Proxy Recording LoadRunner Proxy Port 80/443 Port 8080 ios App web server 22 HP LoadRunner VuGen11.5x

22 Proxy recording HTTP/HTML Proxy Recording 23 HP LoadRunner VuGen11.52

23 Record and analyze traffic HTTP/HTML Record & Analyze Traffic.pcap Wireshark APK Android Ad hoc network ios App web server 24 HP LoadRunner VuGen11.5x

24 Record and analyze traffic HTTP/HTML Record & Analyze Traffic 25 HP LoadRunner VuGen11.52

25 Record and analyze traffic HTTP/HTML Record & Analyze Traffic 26 HP LoadRunner VuGen11.52

26 Analyze traffic HTTP/HTML Analyze Traffic APK Android 4.1+ (rooted) Mobile Recorder.lrpcap App web server 27 HP LoadRunner VuGen11.52

27 Analyze traffic HTTP/HTML Analyze Traffic 28 HP LoadRunner VuGen11.52

28 Analyze traffic options HTTP/HTML Analyze Traffic 29 HP LoadRunner VuGen11.52

29 Scenario runs

30 Concurrency and rendezvous 32 [P10]

31 Wi-Fi vs. 3G response time 33 HP Network Virtualization by Shunra [P01]

32 3G vs. 4G latency spikes 34 HP Network Virtualization by Shunra

33 Login with more VUsers Bandwidth Utilization In (Down) Response Time Bandwidth Utilization Out (Up) 35

34 Ramp up and down Resources per user Response time Resources over time Back-end Resources Clean-up Resources Processes, Threads Memory (leaks?) CPU (saturation?) Disk I/O (thrashing?) Network I/O Transaction Throughput Etc. 36 HP LoadRunner Analysis [P04]

35 Capacity in 3 dimensions 37 [P04]

36 Scalability 38 [P01]

37 Isolate end-user performance by carrier 39 HP Business Service Management

38 Isolate end-user performance by device 40 HP Business Service Management

39 Thank you