WebRTC: The Journey From Browser Plugins to Thin Client Communications

Size: px

Start display at page:

Download "WebRTC: The Journey From Browser Plugins to Thin Client Communications"

Rosanna Murphy
6 years ago
Views:

2 WebRTC: The Journey From Browser Plugins to Thin Client Communications Keith Griffin Ph.D Principal Engineer Office of CTO Collaboration Technology Group BRKCOL-2556

3 Agenda Introduction Our Journey to WebRTC Browser Plugins The End of The Beginning WebRTC as an Enterprise Media Termination Solution WebRTC in Cisco Collaboration Products Demo s State of the art and timing Summary 3

4 Introduction 4

5 About WebRTC What is WebRTC: WebRTC is an API definition being drafted by the World Wide Web Consortium (W3C) It is a free, open project that enables web browsers with Real-Time Communications (RTC) capabilities via simple JavaScript APIs What is the merit of WebRTC: WebRTC enables applications such as voice calling, video chat and P2P file sharing inside the browsers without plugins (or separate clients) BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 5

7 But... Proprietary no interoperability Requires 3rd party plugins Difficult to deploy (permissions, etc...) Not available on all platforms Needs to updated BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 7

8 Browser Plugins for Media Termination 8

11 The end of the beginning Timelines to deprecate plugins:

12 Different Browsers, Different Plugins NPAPI Netscape Plugin API A cross platform browser plugin architecture in: Chrome Firefox Safari ActiveX Today s browsers are speedier, safer, and more capable than their ancestors. Meanwhile, NPAPI s 90s-era architecture has become a leading cause of hangs, crashes, security incidents, and code complexity. Because of this, Chrome will be phasing out NPAPI support over the coming year. A browser plugin architecture created by Microsoft based on its COM (Common Object Model) and OLE (Object Linking and Embedding) technologies Internet Explorer Browser Plugin Technologies stem from developments in the mid-nineties BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 12

13 Browser Market Share Usage share of PC browsers for December 2016 Source Chrome IE Firefox Safari Edge Others NetMarketShare 56.43% 20.84% 12.22% 3.47% 5.33% 1.70% StatCounter 62.66% 9.86% 14.95% 5.1% 3.37% 4.06% W3Counter 56.6% 7.7% 11.1% 14.5% 2% 8.1% BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 13

14 WebRTC as a Media Termination Solution for Enterprise Collaboration 14

WebRTC: Web Real Time Communications Browser + Voice + Video + Data in Real Time What WebRTC enables: Browsers with interactive voice and video communications Real time

15 WebRTC: Web Real Time Communications Browser + Voice + Video + Data in Real Time What WebRTC enables: Browsers with interactive voice and video communications Real time media and data directly between browsers Interoperability with existing voice and video systems BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 15

STORAGE Application Cache IndexedDB localstorage STYLING border-radius columns

RTCWeb <audio> <video> <canvas> WebGL PERFORMANCE XMLHttpRequest Web Workers window.

16 WebRTC in a HTML5 Context SEMANTICS <!DOCTYPE html> <audio> <section> MathML <video> <output> <footer> <header> <time> OFFLINE / STORAGE Application Cache IndexedDB localstorage STYLING border-radius columns border-image backgrounds CSS3 transitions animations shadows MEDIA & GRAPHICS WebRTC RTCWeb <audio> <video> <canvas> WebGL PERFORMANCE XMLHttpRequest Web Workers window.history Drag & Drop Web Sockets SVG INTEGRATION Geolocation Touch Events Full Screen BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 16

17 WebRTC Native Browser Architecture Collaboration Apps WebRTC Javascript API WebRTC WebRTC Native API (C++) Session Management (SDP) Voice Engine Voice Codecs Noise Reduction Echo Cancellation Video Engine Video Codec Jitter Buffer Image Enhancements Transport Encryption / Security Packetisation Multiplexing Connectivity ICE, STUN, TURN Adapted from WebRTC architecture diagram BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 17

HTML, CSS, JavaScript) Web Server GW to SIP SIP SIP Proxy

18 Browser to Non-Browser Endpoint High-level Real-time Communications Architecture Voice, Video via SRTP Web App via HTTP/HTTPS (e.g. HTML, CSS, JavaScript) Web Server GW to SIP SIP SIP Proxy BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 18

19 Example: Consumer to Expert with Video (Browser to End Point) High Level Architecture Public Internet Firewall Traversal Voice/Video Call Control Video Chat Contact Centre Consumer Softclient BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 19

What can we really do with this technology?

21 WebRTC in Cisco Collaboration Products 21

22 WebRTC in the Collaboration Portfolio Product Coverage Cisco Spark Web Client Cisco Spark SDK Webex Web App via WebRTC VoIP/Opus Attendee Mode Webex on Chromebook Variation of Webex Web App WebRTC attendee video support via hardware enabled H.264 Remote Expert Mobile / Jabber Guest Enablers: Industry components such as browsers directly impact H264 based WebRTC support in browsers product direction enablers must be fully capable Firefox currently supports H.264 before product can adopt Chrome currently adding H.264 Microsoft Edge adding support WebRTC/ORTC/H.264»Currently in Edge Preview Build» BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 22

23 Cisco Spark Web Client and SDK Olivier Proffit Cisco Spark SDK Team 23

Cisco Spark SDK for JavaScript Resources Quick Start Guide https://ciscospark.github.

io/spark-js-sdk/app/ Open source code https://github.

25 Embed the Cisco Spark SDK How to start 1. Register you app on developer.ciscospark.com/apps.html 2. Get your Oauth query parameters BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 25

26 Spark SDK - Features Calling Call & Media Controls Other Functions Specs Calling Call Events Call Control Audio Control Video Control Persistent registration Feedback ios SDK - swift ios 9 and 10 JS SDK - WebRTC Chrome - current Firefox - current CODECS H264 Opus BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 26

27 Cisco Spark SDK Calling Capabilities Cisco Spark SDK can dial Spark SDKs Spark clients Spark phones Spark rooms SIP URI Cisco or any 3 rd party PSTN number through Spark hybrid services with CUCM or HCS PSTN number through Spark Call (US only) PSTN number through Tropo (SIP call) BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 27

28 Cisco Spark SDK Calling Capabilities Spark Hybrid services Your App & Spark SDK Pocket Cisco Spark Service Message Meeting Call Call Service Connect Directory Service Cisco Collaboration Cloud Internet Cisco Hybrid Boardroom Cisco Collaboration Edge Architecture Unified Communications Customer Care Conferencing Cisco On-Premises and Partner-Hosted HCS PSTN End to End Security BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 28

Cisco Spark SDK Calling Capabilities High level architectures examples Consumer to Consumer Enterprise collaboration with Hybrid your App + Spark SDK CISCO CLOUD your App & Spark SDK your App + Spark

29 Cisco Spark SDK Calling Capabilities High level architectures examples Consumer to Consumer Enterprise collaboration with Hybrid your App + Spark SDK CISCO CLOUD your App & Spark SDK your App + Spark SDK CISCO CLOUD Customer engagement your App + Spark SDK Business agents CISCO CLOUD SPARK CALL - US OR TROPO PSTN Spark SDK SIP PSTN Internal participants OPTIONAL EXISTING ON PREMISE CUCM PSTN SIP External participants Customers BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 29

browsers ios SDK uses APNS Incoming calls trigger a Spark membership webhook Webhook registered to

30 Incoming Call Notification JS SDK uses WebSocket Supported by all browsers Notifications sent to website opened in a running browser window Push notifications differ and are supported by few browsers ios SDK uses APNS Incoming calls trigger a Spark membership webhook Webhook registered to developer s server which calls APNS BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 30

31 Roadmap DATES & FEATURES SUBJECT TO CHANGE WITHOUT NOTICE ios and WebRTC SDK Beta Audio / Video 2-party Limited screen sharing (receive on ios) ios and WebRTC SDK General availability for Spark subscribers Audio / Video 2-party Limited screen sharing (receive on ios) ios and WebRTC SDK Multiparty Advanced screen sharing Android SDK Beta CY17 Q1 CY17 Q2 CY17 Later BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 31

32 Spark SDK positioning Jabber SDK Vs Jabber Guest Perpetual & REM Go-To-Market pay upfront Jabber Guest SDK Perpetual pay upfront Remote Expert Mobile SDK Perpetual pay upfront Spark SDK Subscription Mobile OS No mobile ios, Android ios, Android ios, Android CY17 WebRTC No WebRTC No WebRTC Firefox, Chrome IE*, Safari* Firefox, Chrome Cloud Offer OnPrem & HCS OnPrem OnPrem Cloud & Hybrid B2C Call Scenarios No Inbound only Yes Yes Secure Messaging Yes No CafeX add-on Presence (Workflow) Yes No CafeX add-on H1 CY17 Messaging widget CY17 Presence API Content Sharing No 1 way Co-browse CY17 Multiparty CUCM CUCM Non-Cisco or CUCM CY17 Cloud * BRKCOL-2556 With plugin 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 32

33 Webex Clients 33

34 Webex Client High Level Browser Architecture Plug-in Interface HTML / Javascript WebEx Meeting Client WebRTC Voice Video Share BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 34

35 Desktop Join Experience Browser Internet Explorer Firefox Chrome Edge Safari Meeting Join ActiveX NPAPI (Whitelist) -> Extension Chrome Extension and Native Messaging TFS Plugin BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 35

36 Introducing the WebEx Web App (Limited Rollout) Join meetings directly within the browser Fast!--No download or plugins to install Key meeting features: o Integrated audio o View shared content o Interact with meeting participants o Send and receive video (Firefox, Chrome) o Seamlessly switch to the desktop app Join directly from your browser Interact with meeting participants Contact your Cisco Customer Success Manager to have the web app enabled on your site View shared content 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 36

for better view of shared content Video support on Chrome and Chromebook Join-before-host

Manager to have the web app enabled on your site Introducing the WebEx App for Chrome A

37 WebEx Web App (Limited Rollout) Improved user interface and panel management Collapse the right panel for better view of shared content Video support on Chrome and Chromebook Join-before-host support Optimised join flows for SSO-enabled sites Contact your Cisco Customer Success Manager to have the web app enabled on your site Introducing the WebEx App for Chrome A downloadable version of the web app available in the Chrome web store with the ability to join by meeting number 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 37

38 WebEx Web App (Formerly known as Browser Client ) 38

39 What is the WebEx Web App? The Web App allows you to join meetings fast, with no installation on your computer (no downloads/plugins). Based on HTML5/WebRTC technology, this version is available for attendees only. (Hosts will only be able to use the desktop app (full client experience).) Currently in Limited Rollout by request To join using browser, simply click the Join using browser link on the meeting join or Personal Room page. Join Page 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 39

40 What functionality is available if I join a meeting using the Web App? Fast join with nothing to install (no plug-in) Simple & intuitive user experience for a view-only attendee: o View shared content (zoom in/out) o Active speaker notification o View list of participants o Recording icon when the meeting is recorded o Indication of who is sharing and what is being shared Chat (private/group) Integrated audio (Note: VoIP is only supported on Chrome and Firefox. IE and Safari support telephony only.*) Send and receive video (Firefox only*) Switch to full meeting experience on Windows/Mac (allows user to switch to the Desktop App if they need other features) More info can be found on Help Central: *VoIP and Video functionality will come to other browsers as they gain WebRTC support 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 40

41 What are the current limitations/known issues? No host/presenter functionality (must use or switch to full experience when prompted) Sharing a media file cannot be seen by the Web App (Web App users will be prompted to switch to the full experience.) Even if join before host is enabled, if the first attendee to join joins through the Web App, the meeting will not start until the host (or the first attendee who joins through the full experience) joins the meeting. (A message notifies the user to wait until the host joins.) The Web App is specifically for the desktop experience (mobile will continue to use the WebEx mobile app) No VoIP on TSP (Telephony Service Provider)-based sites Does not support sites with E2E (end-to-end) encryption enabled Must be running WBS30 or higher More info can be found on Help Central: Cisco and/or its affiliates. All rights reserved. Cisco Public 41

42 Supported Platforms/Browsers in 2.0 Microsoft Windows 7 or Later Feature IE10 or Later Chrome 49 or Later Firefox 45 or Later Join a meeting Yes Yes Yes Switch to the full meeting experience Yes Yes Yes View shared screen, applications, or files Yes Yes Yes View media files No No No View whiteboard and annotation Yes Yes Yes Chat Yes Yes Yes Telephony (Call-in/Call Me) Yes Yes Yes Call back to a video system Yes Yes Yes Call using a computer No Yes Yes Receive video (multi-way video, 720p) No No Yes Send video No No Yes (480p) Mac OSX 10.8 or Later Feature Safari 7.0 or Later Chrome 49 or Later Firefox 45 or Later Join a meeting Yes Yes Yes Switch to the full meeting experience Yes Yes Yes View shared screen, applications, or files Yes Yes Yes View media files No No No View whiteboard and annotation Yes Yes Yes Chat Yes Yes Yes Telephony (Call-in/Call Me) Yes Yes Yes Call back to a video system Yes Yes Yes Call using a computer No Yes Yes Receive video (multi-way video, 720p) No No Yes Send video No No Yes (480p) Linux (Ubuntu 14.x, OpenSuSE 13.x, Fedora 18, 19, Red Hat 6,7, and Debian 8.x) Feature Firefox 45 or later Chrome 49 or later Chromium Join a meeting Yes Yes Yes Switch to the full meeting experience Yes N/A N/A View shared screen, applications, or files Yes Yes Yes View media files No No No View whiteboard and annotation Yes Yes Yes Chat Yes Yes Yes Telephony (Call-in/Call Me) Yes Yes Yes Call back to a video system Yes Yes Yes Call using a computer Yes No No Receive video (multi-way video, 720p) Yes No No Send video Yes (480p) No No Chromebook Feature Join a meeting Switch to the full meeting experience View shared screen, applications, or files View media files View whiteboard and annotation Chat Telephony (Call-in/Call Me) Call back to a video system Call using a computer Receive video (multi-way video, 720p) Send video Chrome 49 or Later Yes N/A Yes No Yes Yes Yes Yes Yes No No Full details: Cisco and/or its affiliates. All rights reserved. Cisco Public 42

43 Web App User Flow 43

Attendee clicks Join using browser link Web App will launch in the current browser window

45 First-time Experience The first time a user joins through the browser, they will see a screen introducing the meeting controls Click OK to dismiss the FTE and enter meeting Cisco and/or its affiliates. All rights reserved. Cisco Public 45

New Video-centric User Experience Meeting info Active Speaker Window Avatars Display for users who do not have video turned on Meeting Controls Automatically hide

46 New Video-centric User Experience Meeting info Active Speaker Window Avatars Display for users who do not have video turned on Meeting Controls Automatically hide when not in use, reappear by moving mouse Participant thumbnails (Display up to 6 at a time) 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 46

Connect to Audio Familiar Audio Connection

Chrome and Firefox only 2017 Cisco and/or

47 Connect to Audio Familiar Audio Connection menu under the button with the phone icon CMR-enabled sites have the option to call back to a video end-point VoIP supported on Chrome and Firefox only 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 47

Allow Browser to Access Mic/Camera Users can select Always Share in order to skip this step the

will appear in active speaker window 2017 Cisco

49 Turn on Video (Firefox & Chrome Only) Video preview prior to sending Once connected, user s selfview will appear in active speaker window 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 49

View Participants List Toggle the participants list on/off (view/hide)

controls Content/Video Window Resize 2017

over individual users to send private chat messages 2017

52 Chat Chat publicly with everyone in the meeting or privately with individual attendees Toggle the Chat panel on/off Hover over individual users to send private chat messages 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 52

53 Switch to the Full Meeting Experience At any time, the user can switch seamless to the desktop app in order to share, view media files, annotate, etc. Switch to the Desktop App 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 53

54 Demo Webex Web App

55 WebRTC - State of the art and timing

56 Standards Efforts RTCWeb Working Group Cullen Jennings of Cisco is co-chair Defining how browsers communicate with others largely re-using existing protocols Notable documents draft-ietf-rtcweb-audio draft-ietf-rtcweb-jsep draft-ietf-rtcweb-qos draft-ietf-rtcweb-security-arch draft-ietf-rtcweb-data-channel draft-ietf-rtcweb-overview draft-ietf-rtcweb-rtp-usage draft-ietf-rtcweb-use-cases-and-requirements WebRTC Working Group Cullen Jennings co-authors RTCWeb draft Keith Griffin co-authors Screen Share draft Defining how Web applications access browser real-time communications, i.e. API s Notable documents WebRTC 1.0: Real-time Communication Between Browsers Media Capture and Streams Media Capture Scenarios BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 56

57 Standards Technology Progress Converging Audio Codecs.. G.711, Opus Signaling: SDP-based offer/answer using JavaScript Firewall/NAT Traversal ICE, STUN, TURN Media Encryption: DTLS-keyed SRTP Media Consent: ICE/STUN Identity: Identity Provider Model QoS DiffServ Code Point markings to enhance WiFi, residential GWs, LTE links Both Video Codec(s) VP8 Vs H.264 are supported, IETF Decision Made and implementation underway OpenH264 In Development Cisco Announced Free OpenH264 Project Mozilla Firefox using OpenH264 Chrome H.264 implementation underway Edge Browser H.264 implementation underway Edge Browser WebRTC 1.0 support underway Congestion Control Goals = minimise latency, quick reaction, consistent data flow Screen/Application Sharing BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 57

58 W3C Share Specification Share a users screen via WebRTC Share the visible or logical display Considerations Security Authorisation: User consent Proprietary Screen Share is possible in most browsers. BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 58

WebRTC Screen Share Introduction of getdisplaymedia method on the NavigatorMediaDevices interface (similar to getusermedia) NavigatorUserMedia { getdisplaymedia (MediaStreamConstraints constraints);

59 WebRTC Screen Share Introduction of getdisplaymedia method on the NavigatorMediaDevices interface (similar to getusermedia) NavigatorUserMedia { getdisplaymedia (MediaStreamConstraints constraints); }; MediaTrackConstraintSet { ConstrainDOMString displaysurface; ConstrainBoolean logicalsurface; }; enum DisplayCaptureSurfaceType { "monitor", "window", "application", "browser" }; BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 59

60 Non-WebRTC Screen Share Status Feature Firefox Chrome Application Type Drive by web / Web App Chrome Packaged App Share Screen Yes Yes Share Window Yes Yes Share Application Yes No Share Tab No Yes User Consent Required Yes Yes Selection UI Drop-down Pop-out Screen Share UI Top Centre of Screen Strip below browser BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 60

61 Demo Building a WebRTC Client Demo Code

Browser Implementations of WebRTC Initial Versions Coming to Market Google Chrome Current Chrome versions are WebRTC capable Actively contributing to standards efforts Contributing to

264 as IETF MTI in Chrome 52 Mozilla Firefox Actively working on open source implementation & contributing to standards efforts Cisco contributed open source development, e.g. RFC4566- compliant SDP engine, call control application logic Adopted OpenH.

62 Browser Implementations of WebRTC Initial Versions Coming to Market Google Chrome Current Chrome versions are WebRTC capable Actively contributing to standards efforts Contributing to open source, e.g. WebRTC.org Implementing H.264 as IETF MTI in Chrome 52 Mozilla Firefox Actively working on open source implementation & contributing to standards efforts Cisco contributed open source development, e.g. RFC4566- compliant SDP engine, call control application logic Adopted OpenH.264 Microsoft Internet Explorer Contributing to standards efforts Windows 10 Edge Browser to support WebRTC and H.264 Apple Safari In Development - Webkit No updates available BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 62

63 WebRTC: Browser Readiness for Collaboration IE/Win (10+ only) Win 10 / Edge Safari/Mac (7+ only) FireFox (WebRTC) Chrome (WebRTC) Chromebook (WebRTC) VoIP No WebRTC Plugin Only Video No WebRTC Plugin Only Share No WebRTC Plugin Only WebRTC/ORTC No WebRTC Plugin Only WebRTC/ORTC No WebRTC Plugin Only TBD No WebRTC Plugin Only Opus / G711 - WebRTC Opus / G711 - WebRTC Opus / G711 - WebRTC H.264** / VP8 VP8 / H.264 H.264 / VP8 WebRTC (TBD*) WebRTC (TBD*) WebRTC (TBD*) Telephony Callback Callback Callback Callback Callback Callback * WebRTC Share standard extension definition underway ** OpenH.264 currently in Firefox Considers interoperability BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 63

65 Wrap Up

67 Summary WebRTC can change the way we communicate in browsers, mobile and fixed endpoints. Standards and Industry Direction continues to evolve Emerging interoperable proof points Product Development underway Enabling real product development as browsers adopt Progress and adoption is good but much more to do. Dependency on browser adoption and more BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 67

68 Q & A

Complete Your Online Session Evaluation Give us your feedback and receive a Cisco Live 2017 Cap by completing the overall event evaluation and 5 session evaluations.

69 Complete Your Online Session Evaluation Give us your feedback and receive a Cisco Live 2017 Cap by completing the overall event evaluation and 5 session evaluations. All evaluations can be completed via the Cisco Live Mobile App. Caps can be collected Friday 10 March at Registration. Learn online with Cisco Live! Visit us online after the conference for full access to session videos and presentations. BRKCOL Cisco and/or its affiliates. All rights reserved. Cisco Public 69

70 Thank you

2017 Cisco and/or its affiliates. All rights reserved. Cisco Public

2017 Cisco and/or its affiliates. All rights reserved. Cisco Public Embed Spark SDK in Your App Olivier Proffit, Product Manager Jonathan Field, Business Development Manager 2017 Cisco and/or its affiliates.