Integrating Mobile Applications - Contrasting the Browser with Native OS Apps. Cary FitzGerald

Transcription

1

2 Integrating Mobile Applications - Contrasting the Browser with Native OS Apps Cary FitzGerald caryfitz@employees.org

3 Workshop Leaders Peter Dunkley Acision Adam van den Hoven Frozen Mountain

4 Integrating Mobile Applications Contrasting the Browser with Native OS Apps

5 Browser vs. Native?

6 Browser vs. Native The Web is more than Browsers AJAX & REST transcends browsers JSON is the standard data exchange format Real Time communication is more than Chat The challenge and opportunity is to find unique ways to leverage WebRTC 6

7 Native Apps Compiled, Packaged app True Native or Hybrid Deployed separate from use Not just Phones and Tablets Augmented / Virtual Reality Embedded Systems Desktop Apps Servers 7

8 Browsers PROS Leverage existing expertise Develop once, use everywhere (in theory) Simple API Simple deployment CONS Design Complexity Limited API ios not supported Windows Phone not supported IE not supported 8

9 Native Aps Pros Background operation Optimized UI and I18N Preapproved access to resources Easy access to device features Cons Many languages to support Little consistency More complex deployment Mostly DIY* 9

10 When to use a Browser Secondary Feature of Larger App Very Simple App Minimally Viable Product 10

11 Primary Interaction is Real Time Communication while in the background Offline notifications Switch Camera (front/back) 11

12 Headless Peers Recording peer Embedded Systems 12

13 Secure Communications Guarantee the behaviour of peers 13

14 Augmented Reality Great opportunity for unique interactions Expert assistance overlay Live Games 14

15 Non-camera and multiple sources Telemedicine Education 15

16 Managing multiple platforms Just bite the bullet Translation Hybrid Cross-compile with Xamarin 16

17 Demo 17

18 Integrating Mobile Applications Contrasting the Browser with Native OS Apps Peter Dunkley Acision

19 Acision at-a-glance Heritage & history 20 year history 270 customers globally 2 trillion messages/ year Innovation 1 st to launch commercial SMSC Cloud & virtualized services Multi-Channel messaging + WebRTC Global presence & reach 35+ offices across 6 continents 850+ employees 100+ support team Operational Excellence Data and insights Carrier grade security 24/7 Support

20 THE STATE OF WEBRTC ON MOBILE TODAY

21 WebRTC in browsers on mobile devices Android Supported in Google Chrome, Mozilla Firefox, and Ericsson Bowser* for Android Mobile browser support trails desktop browser support WebRTC in WebViews supported in Android Lollypop ios Supported in Ericsson Bowser* for ios * Ericsson Bowser is not yet (and might never be) a platform for consumers to use

22 WebRTC in browsers on mobile devices Blackberry 10 Natively supported within the OS Google Chrome rendering engine in built into BB10 Windows Mobile No support today

23 WebRTC in browsers on mobile devices Support for WebRTC in mobile browsers is inconsistent across the various platforms Even on Android it lags behind what is available in desktop browsers Other features needed by many apps (for example, mobile push notifications) still require native apps For the rest of this presentation I will only consider Android and ios

24 WebRTC libraries for mobile devices Google Media Engine (GME) Liberal open-source license Used within Google Chrome and parts used within Mozilla Firefox G.711 and Opus audio VP8 video DataChannel support Widely used today OpenWebRTC (from Ericsson) Liberal open-source license (BSD) Used within Ericsson Bowser G.711 and Opus audio H.264* and VP8 video No DataChannel support (yet) Not widely used today * Does not confer a license to use H.264

25 WebRTC libraries for mobile devices The GME is the best known and widest used library today The only option if you want to use DataChannels right now OpenWebRTC is worth watching The fact that it contains an H.264 implementation doesn t mean developers can just use it For the rest of this presentation I will only consider the GME

26 WHAT CAN I DO WITH WEBRTC IN A MOBILE BROWSER TODAY?

27 Android Click-2-call style apps can be used in Google Chrome on Android You can t receive sessions when the browser is closed You can t write WebRTC apps that require mobile push notifications Some hybrid apps (combination of native code and WebViews using WebRTC) are possible on Android Lollypop

28 ios Nothing today

29 THE GOOGLE MEDIA ENGINE

30 WebRTC Architecture Your web app #1 Your web app #2... Your web app #3 The web WebRTC API RTCWeb WebRTC C++ API (PeerConnection) Session management / Abstract signalling (Session) Voice Engine G.711/OPUS Codec Video Engine H.264/VP8 Codec Transport SRTP Your browser NetEQ for Voice Video Jitter Buffer Multiplexing Echo Canceller / Noise Reduction Image enhancements P2P STUN + TURN + ICE Audio Capture/Render Video Capture Network I/O Based on the diagram from

31 WebRTC C++ API The C++ API documentation at is very basic There is a close mapping between the functions defined in the W3C WebRTC draft (for JavaScript) and the C++ API You can infer the C++ API behaviour by interpreting the W3C draft Getting the WebRTC C++ API working on a native mobile device is not easy

32 The media engine isn t enough Even once you ve gotten the media engine working you need more to establish your sessions Discovering peers and exchanging media capabilities and connection details needs to happen before the media session can be established You need to consider user management and provisioning too There are many developer toolkits and SDK available to help you here (each with slightly different benefits) take advantage of them

33 Authentication Code Access Token Request Identity Identity Consider using OAuth2 if you can 1 OAuth2 Provider Authentication Code 8 3 Identity Web Service

34 Consider using OAuth2 if you can Web Service UA 5: Shared secret Communication link not required You don t have to create or manage accounts on the Signalling Server Signalling Server

35 CROSS PLATFORM CODE EXAMPLE Based on the Acision Forge SDK

36 Android Example AcisionSdkConfiguration config = new AcisionSdkConfiguration("MYAPIKEY ", "user", "password"); config.setapplicationactivity(this); AcisionSdk acisionsdk = new AcisionSdk(config, new AcisionSdkCallbacks() public void onconnected(acisionsdk acisionsdk) { session = acisionsdk.getwebrtc().connectto(" ", new WebrtcSessionCallbacks() public void onconnected() { session.setlocalview(localvideoview); session.setremoteview(remotevideoview); } }, new WebrtcOptions()); } });

37 ios Example ACSAcisionSDKConfiguration *config = [ACSAcisionSDKConfiguration acisionsdkconfigurationwithapikey:@"myapikey username:@"user" password:@"password"]; self.acisionsdk = [ACSAcisionSDK acisionsdkwithconfig:config delegate:self]; - (void)onconnected:(acsacisionsdk *)acisionsdk { ACSAcisionSDKWebRTCOptions* options = [[ACSAcisionSDKWebRTCOptions alloc] init]; self.currentsession = [self.acisionsdk.webrtc delegate:self options:options]; } -(void)onconnected { self.currentsession.localview = self.localview; self.currentsession.remoteview = self.remoteview; }

38 JavaScript Example var lib = new AcisionSDK("MYAPIKEY", { onconnected: function() { var session = lib.webrtc.connect(" "); session.remotevideoelement = document.getelementbyid("vidremote"); session.remoteaudioelement = document.getelementbyid("audremote"); } }, { username: "user", password: "password" } );

39 IOS DEEP DIVE Based on the Acision Forge SDK

40 Noughts and Crosses