First Results of the Performance Analysis

Transcription

1 First Results of the Performance Analysis User group meeting Diepenbeek: 29/09/2015 Gent: 30/09/2015 Michiel Willocx

2 Overall application performance Response times CPU usage Memory Usage Battery usage Disk Space Overview Performance Analysis Sensor access performance Hardware access performance Native API access performance GPS NFC Accelerometer Graphical Performance Local Storage Address book access Network capabilities

3 Table Of Contents Introduction Scope and approach of the performance analysis Results and evaluation Conclusions Future work

4

5 Classification of Cross-Platform-Tools

6 Classification of Cross-Platform-Tools Web Apps (JavaScript Frameworks) Web-to-native Wrappers Runtimes Source code translators App Factories

7 Classification of Cross-Platform-Tools Web Apps (JavaScript Frameworks) Web-to-native Wrappers Runtimes Source code translators App Factories

8 Web Apps è Mobile Websites Accessed in standard mobile browser (Chrome, Safari, ) Optimized for mobile device screen sizes Use JavaScript Frameworks UI Components Event handling, utility functions Use of design patterns

9 Web-To-Native Wrappers Web Apps, packaged as a native, stand-alone application Web code is displayed in a chromeless webview Wider range of native API calls compared to normal Web browser

10 Runtimes Application Code Runtime Environment Operating System Cross-platform compatibility layer Shields app form underlying differences between platforms Different strategies: o o Interpreted at runtime Compiled in advance (source code translators)

11 Source Code Translator Translation of code Different strategies: o o Translate to native source Translate to intermediary language o Translate to low level machine code Often used in combination with Runtime

12 Classification of Cross-Platform-Tools Web Apps (JavaScript Frameworks) Web-to-native Wrappers Runtimes Source code translators App Factories

13 Classification of Cross-Platform-Tools Web Apps (JavaScript Frameworks) Web-to-native Wrappers Runtimes Source code translators

14 Scope and approach of the performance analysis

15 Scope & Approach: CPTs Web Apps (JavaScript Frameworks) Web-to-native Wrappers Runtimes Source code translators

16 Scope & Approach: CPTs Web Apps (JavaScript Frameworks) Web-to-native Wrappers Runtimes Source code translators + native implementation

17 Phonegap Web-to-native Wrapper Most well-known cross-platform tool Supported OS: Android ios Windows Phone 8 BlackBerry Alternatives for PhoneGap as web-to-native wrapper? è discontinued, never used, bankrupt,

18 Ionic Based on èmvc Design pattern Focuses on look & feel and UI interaction No native skins

19 Xamarin Source code translator + Runtime Code written in C# Supported platforms: Android ios Windows Phone (OS X)

20 Scope & Approach: Platforms

21 Scope & Approach: Platforms

22 Scope & Approach: Devices

23 propcros

24 Measured Parameters o o o o o o Start time New page load time Pause and Resume Time Memory usage CPU usage Disk space

25 Measuring Methods Response Times Memory Usage CPU Usage Instruments tool: Time Profiler Instruments tool: Activity Monitor Instruments tool: Activity Monitor ADB Console Log Little Eye Tool ADB Shell: TOP command Disk Space Visible on device Visible on device

26 Results and Evaluation

27 Results and Evaluation Launch Time (in ms) 3000" 2500" 2000" 1500" 1000" Na:ve" Xamarin" PhoneGap"+"Ionic" 500" 0" Nexus"6" Acer"Liquid" iphone"6" iphone"4"

28 Xamarin: Android vs ios Source translated to Intermediary Language (IL) Just-In-Time (JIT) compilation Source translated to executeble binary code Ahead-Of-Time (AOT) compilation

29 Results and Evaluation New page load times (in ms) 700" 600" 500" 400" 300" 200" Na;ve" Xamarin" PhoneGap"+"Ionic" 100" 0" Nexus"6" Acer"Liquid" iphone"6" iphone"4"

30 Evaluation Resume and Pause (in ms) 80" 70" 60" 50" 40" 30" 20" 10" 40" 35" 30" 25" 20" 15" 10" 5" Na6ve" Xamarin" PhoneGap"+"Ionic" 0" Nexus"6" Acer"Liquid" 0" Nexus"6" Acer"Liquid" (only in Android)

31 Evaluation Memory Usage: Measured at 4 different points in the lifecycle of the application o o o o Right after launch In background, right after launch After using the application In background, after using the application

32 Results and Evaluation Memory usage: after launch (in MB) 250" 200" 150" 100" Na9ve" Xamarin" PhoneGap"+"Ionic" 50" 0" Nexus"6" Acer"Liquid" iphone"6" iphone"4"

33 Evaluation Memory usage: in background after launch (in MB) 160" 140" 120" 100" 80" 60" Na9ve" Xamarin" PhoneGap"+"Ionic" 40" 20" 0" Nexus"6" Acer"Liquid" iphone"6" iphone"4"

34 Results and Evaluation Memory usage: after usage (in MB) 450" 400" 350" 300" 250" 200" 150" Na:ve" Xamarin" PhoneGap"+"Ionic" 100" 50" 0" Nexus"6" Acer"Liquid" iphone"6" iphone"4"

35 Evaluation Memory usage: after usage, in background (in MB) 400" 350" 300" 250" 200" 150" Na:ve" Xamarin" PhoneGap"+"Ionic" 100" 50" 0" Nexus"6" Acer"Liquid" iphone"6" iphone"4"

36 Results and Evaluation Cpu usage (in %) 90" 80" 70" 60" 50" 40" 30" Na=ve" Xamarin" PhoneGap"+"Ionic" 20" 10" 0" Nexus"6" Acer"Liquid" iphone"6" iphone"4"

37 Results and Evaluation Disk space: Installer size (in MB) 12" 10" 8" 6" 4" Na2ve" Xamarin" PhoneGap"+"Ionic" 2" 0" Android" ios"

38 Battery Usage

39 All measurements are available online

40 Conclusion of the Performance Analysis

41 Reflection and Conclusion CPT applications always come with a performance penalty è Often acceptable, especially on high-end devices

42 Reflection and Conclusion Performance penalty is not linear to the complexity of the application è CPT disadvantages decrease as the size of the application increases e.g. runtime, webview,

43 Reflection and Conclusion The CPT selection strategy can depend on the type of the application Lots of executed code è Xamarin Advanced, reusable user interface è PhoneGap

44 Reflection and Conclusion App performance does not solely depend on CPT characteristics, but also on underlying OS support

45 Future Work

46 Next user group meeting: Extended performance analysis More CPTs Add Windows Phone Adjust Parameters Test empty application for each CPT

47 !! Extra CPTs: Available PropertyCross implementations Runtimes)&)Source)Code)Translators)! Adobe!Air! Delphi!!!!!!!NeoMAD!! Titanium!! Xamarin! Xamarin!MVVMCross!!!!! Web)Apps)/)Web9To9Native)/)JavaScript)Frameworks) Emy!! Enyo!! Famo.us!! Intel!App!Framework Ionic!! jqtouch!! jquery!mobile!! Kendo!UI! Lavaca!! Lungo!! Mgwt!! OnsenUI!! PhoneJS!! Sencha!Touch2!

48 ! Extra CPTs: Selected PropertyCross implementations Runtimes)&)Source)Code)Translators) Adobe!Air! NeoMAD!! Titanium!! Xamarin!!!!!!! Delphi!! Xamarin!MVVMCross!!!!! Web)Apps)/)Web9To9Native/JavaScript)Frameworks)!Ionic!!!jQuery!Mobile!!!Famo.us!!!Sencha!Touch2!!Mgwt!!!Intel!App!Framework!!!!Emy!!!Enyo!!!Kendo!UI!!!Lavaca!!!Lungo!!!PhoneJS!!!OnsenUI!!!jQTouch!! + Native implementation for Android, ios and Windows Phone

49 Add Windows Phone

50 Adjust Parameters Response times o Pause and resume times o Time to return to previous page of the application Memory o o Memory usage in background Influence of garbage collection CPU o Global CPU usage of the application

51 Analysis of an empty application Implementations PhoneGap NeoMAD Titanium Xamarin Adobe Air Parameters Launch time Memory usage Installer size Native Android Native ios Native Windows Phone

52 Later: Security Analysis Do cross platform tools achieve the same security as native applications?

53 Comparison of security related APIs. e.g. Storage o Local o o o Cloud Backup mechanisms Unlocking of stored data Are all native options available in the CPT? What are the default settings? Are they secure?

54 Evaluation of security mechanisms e.g. Secure communication

55 Questions / Remarks?