Standard approaches and a case study

Size: px

Start display at page:

Download "Standard approaches and a case study"

Theodore Stokes
5 years ago
Views:

1 Standard approaches and a case study Corso di Sistemi Multimediali A.A Prof. Antonio Picariello Corso di Applicazioni Telematiche A.A Prof. Simon Pietro Romano Università degli Studi di Napoli Federico II Facoltà di Ingegneria 9/6/2011 1

2 Outline Web Conferencing Introduction History and technologies IETF standardization efforts Milestones and ongoing works in the RAI area Centralized Conferencing, Media Control, Session Recording, Via-browser conferencing, Telepresence Systems From theory to practice: the Meetecho platform A fully-fledged standard conferencing system made in unina 2

3 A Web Conferencing definition The term Conference can be used to describe any meeting of people that confer about a certain topic Web Conferencing is used to conduct live meetings or presentations over the Internet 3

4 Features VoIP (Voice over IP) Live video Text chat Slide presentations Whiteboard with annotation Screen/desktop sharing Application sharing Recording Polls and surveys 4

5 History Tele-Conferencing Conference calls (Audio Tele-Conferencing) Video conferences (Video Tele-Conferencing) IP-Conferencing Text Conferencing Audio and Video Conferencing Data Conferencing 5

6 Audio and Video Conferencing Evolution Analog Phone Lines (PSTN) Conference calls Three-way calling Conference bridges Video Telephony Digital Telephony (ISDN) ITU-T H.320 umbrella recommendation Next-Generation Networks & VoIP technologies 6

7 VoIP protocols SIP Session Initiation Protocol, RFC 3261 Call signalling Session intitiation, managing and termination Codec negotiation SDP Session Description Protocol, RFC 4566 Session media stream description RTP Real-time Transport Protocol, RFC3550 Media streams transfer 7

lists Synchronous Meetings Live text communication Internet Relay Chat

8 Text Conferencing Evolution Asynchronous Meetings Posted text messages (not live) Message/Bulletin Boards Forums Network Newsgroups/Mailing lists Synchronous Meetings Live text communication Internet Relay Chat (IRC) Web-based Chat (CGI/Java) Instant Messaging (Skype/MSN/XMPP/etc.) 8

Presentation Question & Answers session Ad-hoc and Reserved Conferences

9 Typical scenarios From Point-to-Point Calls to Multipoint Calls Three-way calling A Coaching scenario A+B C A+B+C B Lecture-mode Conferences Presentation Question & Answers session Ad-hoc and Reserved Conferences Conference-aware/-unaware participants Manage conference/users/media/policies Sidebars/Whispers 9

Issues Call signalling Gateway functionality Client

Terminal capabilities User media profiling Coaching

10 Issues Call signalling Gateway functionality Client A Server SIP H.323 Client B Control and Management Tone detection (DTMF) Dedicated protocols Mixing and Transcoding Terminal capabilities User media profiling Coaching scenario Videoswitching Client A B Server Media A Media C A A Media B Client B B A+B Client C 10

Standardization Efforts Geographically dispersed businesses, the rising cost of energy, and the development of NGN lead to the proliferation of proprietary conferencing solutions No standardization

11 Standardization Efforts Geographically dispersed businesses, the rising cost of energy, and the development of NGN lead to the proliferation of proprietary conferencing solutions No standardization for many years Lack of interoperability Platform dependency Security issues Cost Market segmentation Standardization Bodies ITU (International Telecommunication Union) 3GPP (3rd Generation Partnership Project) IETF (Internet Engineering Task Force) 11

12 Standardization Efforts: ITU Established to standardize and regulate international radio and telecommunications International Standards referred to as Recommendations ITU-T: Telecommunication Sector G: Transmission Systems and Media G.71x (Audio compression, mu-law and a-law) G.72x (Audio compression, ADPCM) H: Audiovisual and Multimedia Systems H.320 (PSTN/ISDN, Telephone Systems) H.323 (IP, Packet-based Communication Systems) T: Terminals for Telematic Services T.120 (Data Sharing Protocols) T.140 (RTP Interactive Text) 12

13 Standardization Efforts: IETF Under the umbrella of the Internet Society Develops and promotes Internet Standards Deals in particular with standards of the TCP/IP suite Organization Working Groups (WG) Internet Drafts Requests for Comments (RFC) Rough consensus, running code We will focus on some RAI area activities... 13

14 IETF RAI area RAI Real time Applications and Infrastructure Legacy standards Standard VoIP protocols SIPPING XCON MediaCtrl Brand-new standardization activities SIPREC RTCWEB CLUE 14

15 SIPPING Working Group Session Initiation Protocol Proposal INvestiGation Documents the use of SIP for several applications related to telephony and multimedia SIP Conferencing Models Loosely-Coupled Conference Fully Distributed Multiparty Conference Tightly-Coupled Conference Main actors: - Participants - Focus - Mixer - Policy Server - Notification Server Sip Conferencing Framework 15

16 XCON Working Group Centralized Conferencing ( X CON) A star topology on the signalling plane Advanced conferencing features VoIP + video & data sharing Signalling-agnostic Not only SIP H.323, IAX, Defines : Conference data model and lifecycle A suite of client-server protocols to realize sophisticated conferencing scenarios 16

17 Conference Object &Data Model Template conference object (blueprint) <conference-info> <conference-description> cloning creation <host-info> delete Registered conference object <conference-state> <users> last leave first join <floor-information> delete Active conference object <sidebars-by-val> <sidebars-by-ref> 17

18 Conference object example Conference ID Title Conference state Password / PIN Allowed join modality Floor handling policy Participants info User ID Nickname Signalling URI Floor info Floor ID Associated media Moderator ID Max number of floor owners <info:conference-info entity=" "> <info:conference-description> <info:display-text>my Full-Media Conf</info:display-text> <info:conf-uris> <info:entry> <info:uri>xcon: @meetecho.com</info:uri> <xcon:conference-password>3903/0</xcon:conference-password> </info:entry> </info:conf-uris> <info:available-media> <info:entry label="audiolabel"> <info:type>audio</info:type> </info:entry> <info:entry label="videolabel"> <info:type>video</info:type> </info:entry> </info:available-media> </info:conference-description> <info:conference-state> <info:active>false</info:active> </info:conference-state> <info:users> <info:user entity="13"> <info:display-text>alex</info:display-text> <info:endpoint entity="sip:alex@ :5060"/> </info:user> <info:user entity="14"> <info:display-text>user134</info:display-text> <info:endpoint entity="sip:user134@ :5080"/> </info:user> <xcon:join-handling>allow</xcon:join-handling> </info:users> <xcon:floor-information> <xcon:floor-request-handling>confirm</xcon:floor-request-handling> <xcon:conference-floor-policy> <xcon:floor id="11"> <xcon:media-label>audiolabel</xcon:media-label> <xcon:max-floor-users>8</xcon:max-floor-users> <xcon:moderator-id>13</xcon:moderator-id> </xcon:floor> <xcon:floor id="22"> <xcon:media-label>videolabel</xcon:media-label> </xcon:floor> </xcon:conference-floor-policy> </xcon:floor-information> 18 </info:conference-info>

19 XCON Protocols Signalling SIP, H.323, IAX2, et al. Floor Control BFCP (Binary Floor Control Protocol) Notification Xcon Event Package Conference Control CCMP (Centralized Conferencing Manipulation Protocol) University of Napoli highly active in this field 19

CCMP Conference Control Conference CRUD Manipulation Creation From a client-provided model or by cloning system blueprints Retrieval Update User profiles,

20 CCMP Conference Control Conference CRUD Manipulation Creation From a client-provided model or by cloning system blueprints Retrieval Update User profiles, multimedia flow features (audio volume, video layout, ),... Deletion XML- based Carrying datamodel fragments Single verb HTTP + CCMP body transport solution 20

21 BFCP Floor Control Aimed at coordinating access to a set of shared resources A Floor is a token, a temporary permission to access or manipulate a specific shared resource or set of resources Standardized in RFC 4582 Identifiers (Conferences/Floors/Users) Floor Control Server Floor Control Participant Floor Chair Negotiation of BFCP connections within SIP/SDP standardized in RFC 4583 Only existing implementation to date: COMICS/Ericsson 21

22 BFCP in action 1) Floor Request 2) Notify Chair Decision 3 3) Chair decision 4) Decision 5) Floor Granted/Denied 6) Notify 4 Decision Notify Floor Request Notify Floor Granted Or Denied 5 22

23 S I P MediaCtrl Working Group Defines an architecture in which an Application Server can control a Media Server A Media Server could be completely devoted to perform onerous online/offline operations on media flows: Media Processing Mixing/Transcoding Playing/Recording Storing/Retrieving Detecting Tones (DTMF) Interactive Voice Response (IVR)/VoiceXML Text-to-Speech/Speech Recognition RTP Streams Manipulation S I P S I P R T P A p p l i c a t i o n S e r v e r C o n t r o l C h a n n e l R T P Client B Exploitable approach within conferencing frameworks M e d i a S e r v e r 23

24 SIPREC Working Group SIPREC - SIP-based RECording Designing a protocol based on SIP-extension and a metadata model to deal with multimedia session recording A recorded session can play an important role in different scenarios E-learning, minutes taking, Business analytics Media processing (online/offline) Some media processing actions may require separation between different media streams E.g.: Word-spotting, Speaker identification Emotion-detection 24

25 SIPREC - Architecture Session Recording Client (SRC) Logical entity able to acquire the communication session to be recorded Sends the recording media streams to the SRS Session Recording Server (SRS) Records the received session Recording Session (RS) is made of: CS Multimedia Content CS Metadata 25

26 SIPREC - Requirements Active recording The SRC proactively demands for registration No passive recording / sniffing / wiretapping Session participants MUST be recording-aware Notification mechanisms are needed to inform participants Start, Pause, Resume and Stop controls Define a protocol between SRC and SRS to manage the recording session 26

27 SIPREC Recording Metadata Information that describes recorded media and the Communication Session (CS) to which they relate They have to allow for: Searching and filtering the archived session Performing a meaningful playout e.g., with correct synchronization between media streams 27

28 Recording Metadata Example <recording-metadata xmlns='...:siprec'> <recording id=""> </recording> <group id="" recording=""></group> <session id=""group=""></session> <participant id=""session=""></participant> <stream id=""session=""></stream> <extensiondata id="" parent=""> </extensiondata> </recording-metadata>

29 Recording Metatada Elements Recording Session Features: ID, recording type (selective, persistent), recording requestor (SRC or SRS), One or mode associated CSG Communication Session Group (CSG) Identified by a unique ID Group of one or more CS belonging to the same macro-session Communication Session (CS) Features: ID, call termination reason, start time, end time, Can involve one or more Participant elements and one or more Media Stream elements Participant Features: name, AoR, participant role, Media Stream Features: start time, stop time, codec, SDP m-line reference,.. Extension Data Envisioned the optional presenxe of additional data not specified as part of SIPREC 29

30 RTCWEB Working Group Real Time Communication on the World Wide Web Goal: standardizing basic components enabling interactive rich communication using audio, video, collaboration, games, etc. between two peers' web-browsers Direct browser-to-browser communication Nowadays several different methods exists, but they are not interoperable since they rely on incompatible browser plug-in platform Collaboration with the W3C WEBRTC group : IETF architecture and requirements W3C Javascript API development The idea: client-side applications, typically written in HTML and Javascript, interacting with web browser through a yet-to-be-defined API 30

31 RTCWEB IETF issues Identification and definition of network-related aspects Control protocols Connection establishment and management Connection-less transport Selection of the most suitable encoders/decoders Codec negotiation New threats and security issues Firewall and NAT traversal Interoperation with legacy VoIP equipment, keeping the balance with practicality of browser deployment 31

CLUE Working Group ControLling multiple streams for telepresence Telepresence systems provide high definition, high quality audio/video enabling a "being-there"

32 CLUE Working Group ControLling multiple streams for telepresence Telepresence systems provide high definition, high quality audio/video enabling a "being-there" / immersive experience Use of multiple displays permitting life size image reproduction using multiple cameras, encoders, decoders, microphones and loudspeakers 32

33 CLUE issues specifications for SIP-based conferencing systems to enable communication of information about media streams necessary to make reasonable decisions about transmitting, selecting, and rendering media streams: Spatial relationships of cameras, displays, microphones, and loudspeakers Viewpoints - field of view/capture for camera/microphone/display/loudspeaker Senders and intermediate devices use them to understand how best to compose streams for receivers Receivers can know the characteristics of the received streams Usage of the stream (presentation, document camera output, ) Aspect ratio of cameras and displays Sources selection receiver-driven 33

34 Meetecho Platform Web conferencing and on-line collaboration tool Academic spin-off since Spring 2011 Developed by COMICS members and Unina students Main features: Scalable and Performing Client-Server Solution IETF standards compliant (XCON, Media Ctrl) Open-source Java-based Desktop Client Mobile device access support Android iphone, ipad BlackBerry Symbian Windows Phone Web browser access support (Meetecho Weblite) 34

35 Meetecho capabilities PSTN audio Voice over IP audio Live video Audio & Video Moderation Presentation delivery Desktop/screen sharing Text chat Presence File sharing Whiteboard Sharing Basic security Remote Control Session Recording Polls & Surveys Mobility support Web access support

36 Meetecho in action

37 Audio/Video controls panel Mirror On/Off Pause Audio Pause Video Listen/Talk Volume Selection of the desired contribution

Moderated Scenarios Realized by means of BFCP The conference chair decides to grant or not audio and/or video floor to the requestor The Meetecho

38 Moderated Scenarios Realized by means of BFCP The conference chair decides to grant or not audio and/or video floor to the requestor The Meetecho server allows only granted users to contribute their media within the conference media mix Adaptive video layout depending on granted participant number

Slide sharing Slides are converted in background to a series of images made available on a web server All kind of presentation are supported Microsoft Office (ppt, pptx, doc, docx, xls, xlsx)

39 Slide sharing Slides are converted in background to a series of images made available on a web server All kind of presentation are supported Microsoft Office (ppt, pptx, doc, docx, xls, xlsx) OpenOffice (odp) Pdf Slides previously exported in image formats as (gif, jpg, png, ecc.) The presenter is the one who controls slide progress Slides can be sent to the whiteboard for online collaborative modifications

40 Whiteboard Shared between conference participants Allows to Draw illustrative sketches Modify slides Insert clip-art Save file modifications

41 Text Chat Based on XMPP (extensible Messaging and Presence Protocol) Allows instant messaging during conference sessions between conference participants Features Roster management Presence info Single file sharing Conversation chronology

42 Polling Creating, answering and consulting real-time surveys for conference participants

43 Desktop sharing Meetecho users can Show to other participants their desktop activities Render desktop control to other participants Set visualization quality

44 Video mixing (1) Up to 16 parallel video streams in a single conference Mixing strategy: The mixer receives participant video contributions Video streams are composed in a mosaico stream The mixer sends to each participant the single mosaico stream and not each single video contribution That allows for Constant load Constant bandwidth consumption

45 Video mixing (2) Other tools overwhelm clients with as many flows as the contributing participants number Users Server User Meetecho mixes video streams and sends always a single flow to each users Users Meetecho Server User

46 Meetecho codecs Supported audio codecs: GSM G.711 u-law Supported video codecs: H.323 Q-CIF (176 x 144 pixel) H.323 CIF (352x288 pixel) 46

47 Managing Scalability Centralized case A single focus for N users Distibuted case Cascaded Conferencing Load-balancing users between more interoperating focuses Each focus is seen as a participant by the other focus Orchestration of a set of XCON clouds Experimental campaign showed performance benefits whit the increase of the participant number 47

48 Meetecho Session Recording Meetecho Session Metadata are encoded using SMIL (Synchronized Multimedia Integration Language) Well known and established standard specification Recordings are available in a standard fashion Media timing aware Enabling synchronized playout Allowing media stream separation A Recording Server is envisioned To receive all conference media streams and record them To encode stream-related metadata info in the standard SMIL format 48

49 Server-side recording Client-side recording may be realized by means of ordinary desktop recording softwares Available tools: Adobe Captivate, Wink, Istanbul, Byzanz, Simple and fast Conferencing platfom independent Client-side recording drawbacks: Only a single perspective is taken into account Recorded media flattened to a single, very large video Output files usually in proprietary formats Flash, OGGTheora, Meetecho server-side recording approach uses standard formats is more suitable for multimedia session with several media and etherogeneous devices involved 49

Recording audio & video Both Audio and video from each participant are aggregated in the central mixer the mixer can record and store them locally or on a remote storage SIP XCON AS MEDIACTRL SIP

50 Recording audio & video Both Audio and video from each participant are aggregated in the central mixer the mixer can record and store them locally or on a remote storage SIP XCON AS MEDIACTRL SIP UA-A RTP Mixer RTP UA-B Transcoding, adaptation and other operation may be carried out on the streams RTP UA-A (Rx + Tx) RTP UA-B (Rx + Tx) The output is a single file containing both audio and video (AVI file with specific codecs) Recorder

51 Recording IM & events Chat contributions are saved together with timestamps and info on the employed IM protocol C (MSRP) UAC-C 10:11:24 Hi! UAC-A A (XMPP) 10:11:26 Hey C Recorder B (IRC) 10:11:30 Hello UAC-B Textual event notification may be treated as well Slide changes Join /leave events Poll announcement [ ] 10:11:24 <User C> Hi! 10:11:26 <User A> Hey C 10:11:30 <User B> Hello [ ] 51

52 Recording slide presentations Presentations are considered assimple slideshows of static images Slide change events are triggered and notified via XMPP, registered as text Slides are registered as a series of images together with metadata info (associated events) 52

53 Recording whiteboarding UAC-C C (XMPP) 10:11:20 line UAC-A A (XMPP) 10:10:56 circle Whiteboard Server B (XMPP) 10:12:30 text UAC-B [ ] 10:10:56 <User A> circle 10:11:20 <User C> line 10:12:30 <User B> text [ ] 53

54 Putting It All Together: SMIL Standard XML-based markup language for describing presentations involving heterogeneous media SMIL metadata file <head> Generic metadata and layout composition details <body> References to media to be included (URL) Layout regions info Timing info 54

55 Media metadata SMIL allows for both standard and non-standard media to be included in a presentation Standard media streams have dedicated standard tags in the media part of the <body> <video> the A/V Meetecho recording AVI file <textstream> IM, event notifications, converted in Real-Time Text docs <img> slides Non-standard media are included in the <ref> tag whiteboarding case 55

56 A Meetecho conf SMIL file 56

57 Playing out the recording (1) 1. START 7. SHOW SMILenabled player 2. get conf45.smil Session database Web server 3. get audio/video 4. get RealText file 5. get slide images 6. get whiteboard packets video text slides whiteboard 57

58 Playing out the recording (2) General approach: 1) Retrieval of the SMIL file (HTTP or out-of-band mechanism) 2) SMIL file is passed to a compliant media player preparying the presentation layout 3) The player retrieves and reproduces each supported media SMIL players examples : RealPlayer, Ambulant, Standard stream support Don t play Meetecho whiteboarding Meetcho players! 58

59 Playing out the recordings (3) 3 options 1) Watch it online by means of a JNLPbased custom player 2) Watch it online with HTML5 3) Download the registration and watch it offline (jar file) 59

60 Session recordings available at: 60

Chapter 11: Understanding the H.323 Standard

Página 1 de 7 Chapter 11: Understanding the H.323 Standard This chapter contains information about the H.323 standard and its architecture, and discusses how Microsoft Windows NetMeeting supports H.323