Lecture 14: Multimedia Communications Prof. Shervin Shirmohammadi SITE, University of Ottawa Fall 2005 CEG 4183 14-1 Multimedia Characteristics Bandwidth Media has natural bitrate, not very flexible. Packet Loss Affects audio more severely than video. Delay End-to-end delay has a certain upperbound in most audio/video applications. Jitter: variance of delay Adversely affects synchronization (lip sync), collaborative real-time applications (worst than delay in these cases). Fall 2005 CEG 4183 14-2 1
Multimedia Application Categories Presentational: the media is simply rendered to the user, interaction limited to play/pause/open/close/ ; e.g., video-on-demand Conversational: users interact with the content and with each other; e.g., video conferencing, whiteboarding, collaborative virtual environment, multiplayer games, Network lag (delay, loss, jitter) affects both categories, but it s harder to resolve it in conversational applications due to tighter synchronization requirements. Fall 2005 CEG 4183 14-3 Requirements Application file transfer e-mail Web documents real-time audio/video stored audio/video interactive games Bandwidth flexible flexible flexible audio: 5Kb-1Mb video:10kb-5mb same as above Time Sensitive no no no yes, 100 s msec yes, few secs yes, 100 s msec Will TCP be suitable for the transport of real-time media such as audio and video? What about UDP? Fall 2005 CEG 4183 14-4 2
Real-time Transport Protocol: RTP RFC 1889, 1996; ITU-T Standard H.225.0 It is usually implemented as application-level framing, sitting on top of UDP. NOTE: RTP is named as such because it is used to carry real-time data: RTP does not guarantee real-time delivery by itself. Real-time delivery depends on the underlying network; a transport-layer or application-layer protocol cannot guaranty real-time delivery because it can t control the network. Does IP provide real-time delivery? Can we have real time delivery on the Internet? IP header UDP header RTP header media Fall 2005 CEG 4183 14-5 RTP Packet Version: 2 bits, this field identifies the version of RTP. The version defined by this specification is two (2). CS count (CC): 4 bits, the CS count contains the number of CS identifiers that follow the fixed header. Payload Type : 7 bits, this field identifies the format of the RTP payload and determines its interpretation by the application. Fall 2005 CEG 4183 14-6 3
RTP Packet ( ) sequence number : 16 bits, The sequence number increments by one for each RTP data packet sent, and may be used by the receiver to detect packet loss and to restore packet sequence. timestamp: 32 bits, The timestamp reflects the sampling instant of the first octet in the RTP data packet. The sampling instant must be derived from a clock that increments monotonically and linearly in time to allow synchronization and jitter calculations SS : 32 bits, The SSRC field identifies the synchronization source. CS list : 0 to 15 items, 32 bits each. The CSRC list identifies the contributing sources for the payload contained in this packet. The number of identifiers is given by the CC field. Fall 2005 CEG 4183 14-7 RTCP Real-time Transport Control Protocol. Used in conjunction with RTP. Similar to RTP, it uses UDP as its delivery mechanism. provides information (control data) to session participants: QoS feedback. Receivers report back the quality of their reception from each sender (number of lost packets, jitter, round-trip delays). This information can be used by senders to adapt the source (if possible). Intermedia synchronization. Necessary information for the synchronization of sources (audio/video), even if originating from different servers, is provided by RTCP. Identification. information such as the e-mail address, phone number, and full name of the participant. Session Control. Participants can also send small notes to each other, such as ``stepping out of the office', or indicate they are leaving (BYE message). Fall 2005 CEG 4183 14-8 4
RTCP Packet Sender Report Fall 2005 CEG 4183 14-9 RTSP: Real Time Streaming Protocol RFC 2326, 1998 Similar to HTTP in terms of role and position in the communications stack. It is used to send commands between client and media source (server). Play, pause, stop, Doesn t transport the media itself (that s the job of RTP or another protocol). Again, it s not really real-time. Real-time delivery depends on the network itself. Usually uses port 554 Fall 2005 CEG 4183 14-10 5
Method Options SETUP ANNOUNCE DESCRIBE PLAY RECORD REDIRECT PAUSE SET-PARAMETER TEARDOWN RTSP Methods Description Get available methods Establish transport Change description of media object Get description of media object Start playback, reposition Start recording Redirect client to new server Halt delivery, but keep state Device or encoding control Remove state RTSP State Fall 2005 CEG 4183 14-11 Overall Operation client GET pointer to audio SETUP PLAY audio data QoS feedback TEARDOWN Web server audio server HTTP RTSP RTP RTCP Web server and audio server might or might not be the same machine. Fall 2005 CEG 4183 14-12 6
Voice over IP ITU-T H.323: Visual Telephone Systems and Equipment for Local Area Networks Which Provide A Non-Guaranteed Quality of Service The H.323 architectural model for Internet telephony. Fall 2005 CEG 4183 14-13 ITU-T H.323 The H.323 protocol stack. POTS protocol Protocol negotiation PC to Gatekeeper RAS: Registration/Admission/Status Fall 2005 CEG 4183 14-14 7
Voice over IP: SIP The Session Initiation Protocol IETF RFC 3261 Only handles setup and management, it s up to the parties themselves to exchange messages (over RTP/RTCP, for example). SIP packets can be transported over UDP or TCP. Phone numbers are represented as URLs; e.g., sip:isle@cs.university.edu The SIP methods defined in the core specification: Fall 2005 CEG 4183 14-15 SIP Operation Use of proxy and redirection servers with SIP. Fall 2005 CEG 4183 14-16 8
Video on Demand Overview of a video-on-demand system. Fall 2005 CEG 4183 14-17 Video Servers Serving video streams (bulky media) at fast rates requires special storage and retrieval methods. A video server storage hierarchy: the higher in the pyramid, we get less capacity but faster access. The idea is to put the more popular videos higher in the pyramid. Fall 2005 CEG 4183 14-18 9
Communications in Virtual Worlds 3D Virtual Environment Immersive VR CAVE Augmented Reality 3D Video Fall 2005 CEG 4183 14-19 Tele-Haptics How to transmit information about touch over the network? Can be remote-manipulation (tele-operation) or pure simulation. Other human senses? Fall 2005 CEG 4183 14-20 10