Lecture 6: Internet Streaming Media

Lecture 6: Internet Streaming Media A/Prof. Jian Zhang NICTA & CSE UNSW Dr. Reji Mathew EE&T UNSW COMP9519 Multimedia Systems S2 2010 jzhang@cse.unsw.edu.au

Background So now you can code video (and audio) But how to transfer this video to your mates? Possibilities File Transfer TCP Streaming for real-time delivery TCP, UDP Protocols for Enabling Streaming RTP Transport RTCP Control SDP - Description RTSP - Signaling COMP9519 Multimedia Systems Lecture 6 Slide 2 J. Zhang & R. Mathew

Lecture Outline Introduction to Streaming Streaming Protocols RTP/RTCP System Overview System for streaming video and audio on the Internet Overview of important components Sender side issues Receiver side issues Client Architecture : An Example Session Description Protocol How to describe a multimedia session? SDP example COMP9519 Multimedia Systems Lecture 6 Slide 3 J. Zhang & R. Mathew

Introduction Facts on TCP (Transmission Control Protocol) Data sent on a TCP connection is delivered reliably and in order at the destination. Transmission made reliable via use of sequence numbers and acknowledgments. Little control over the timely arrival of data Arbitrary number of retransmissions Not suitable for real-time streaming Real-time applications have stringent limits on acceptable delay eg video conferencing COMP9519 Multimedia Systems Lecture 6 Slide 4 J. Zhang & R. Mathew

Introduction Facts on UDP (User Datagram Protocol) Datagram mode of communication Send messages with a minimum of protocol mechanism Unordered datagram service delivery and duplicate protection are not guaranteed UDP is unreliable But used when timeliness is more important you have control over when data is sent Preferred for streaming TCP is more reliable Used when reliability is more important (than timeliness) Eg, file download COMP9519 Multimedia Systems Lecture 6 Slide 5 J. Zhang & R. Mathew

Introduction File Transfer Examples : HTTP, FTP Using TCP internet HTTP HTTP Server Request for a file Download whole file Wait for file to be downloaded Lengthy delay for large files Not suitable for real-time applications Play File COMP9519 Multimedia Systems Lecture 6 Slide 6 J. Zhang & R. Mathew

Introduction Streaming Using TCP (not preferred) Wait until some content is downloaded to a buffer Start playing out from buffer while server/client run recive/ack procedure If data rate drops then will need to stop play out Re-transmission due to congestion and packets loss Frequent start and stop of play out for Resync. internet HTTP TCP r bits/s Play Out Buffer HTTP Server File Viewing Time = T s File Size = D bits Ack/receiver procedure COMP9519 Multimedia Systems Lecture 6 Slide 7 J. Zhang & R. Mathew d bits (D - d) / r < T

Introduction Streaming Using UDP (preferred) Transmit data rate matches coded rate of media Eg Transmit rate = R = D/T Start playing out as long as data arriving Small delay (smaller than when using TCP) Playing out from buffer regardless the traffic congestion and packets loss Play out rate = R Tx File internet UDP Play Out Buffer File Viewing Time = T s File Size = D bits COMP9519 Multimedia Systems Lecture 6 Slide 8 J. Zhang & R. Mathew

Introduction So we choose UDP/IP (over TCP/IP) for streaming But multimedia data have other requirements Sequencing: to reorder packets at the receiver that are out of order to detect lost packets Intra-media synchronization: timing information for play out of successive packets Eg, period of time to show/hold a single video frame Inter-media synchronization: timing information to synchronize multiple media streams Eg, synchronize audio and video streams (lip sync) COMP9519 Multimedia Systems Lecture 6 Slide 9 J. Zhang & R. Mathew

Introduction Payload identification: Communicate the codec being currently used Allow codec switching to adjust to bandwidth or user capabilities Frame indication: To indicate start and end of a frame or access unit Helps in delivery of data to higher layers COMP9519 Multimedia Systems Lecture 6 Slide 10 J. Zhang & R. Mathew

Internet Streaming Protocols How to meet requirements for multimedia streaming Solution - Real Time Transport Protocol (RTP) - Real Time Control Protocol (RTCP) RTP/RTCP Defined in a generic way Developed by IETF RFC 3350 RTP: A Transport Protocol for Real-Time Applications More info ftp://ftp.rfc-editor.org/in-notes/rfc3550.pdf Designed to be multi-cast friendly Generally used with UDP Although designed to be independent of underlying transport And network layers COMP9519 Multimedia Systems Lecture 6 Slide 11 J. Zhang & R. Mathew

RTP Overview compressed bit stream fragment RTP Attach RTP Header Time stamping Sequence numbering Payload ID Frame indication Decoder RTP UDP UDP IP COMP9519 Multimedia Systems Lecture 6 Slide 12 J. Zhang & R. Mathew

RTP Header Fields Marker (M) : 1 bit The first twelve bytes (octets) are present in every RTP packet. Intended to allow significant events such as frame boundaries to be marked in the packet stream MPEG-4 Example : set to 1 to indicate last RTP packet of a VOP. One compressed frame or VOP Broken up into 3 packets M=0 M=0 M=1 COMP9519 Multimedia Systems Lecture 6 Slide 13 J. Zhang & R. Mathew

RTP Header Fields Payload Type : 7 bits PT This field identifies the format of the RTP payload (eg payload is MPEG-4 video or u-law audio.) Static mapping Payload Type value codec COMP9519 Multimedia Systems Lecture 6 Slide 14 J. Zhang & R. Mathew

RTP Header Fields Sequence Number : 16 bits The sequence number is incremented by one for each RTP packet sent. Usually made to start at a random number. Sequence number can be used by the receiver to reorder out of order packets and detect packet loss. COMP9519 Multimedia Systems Lecture 6 Slide 15 J. Zhang & R. Mathew

RTP Header Fields Time Stamp : 32 bits The timestamp specifies the sampling instant of the first octet in the RTP data packet. Example : two audio samples in one packet, timestamp refers to the sampling instant of the first sample. Timestamp of packet = sample time of first packet one RTP packet COMP9519 Multimedia Systems Lecture 6 Slide 16 J. Zhang & R. Mathew

RTP Header Fields Time Stamp : 32 bits The timestamp is derived from a clock that increments monotonically and linearly in time. The clock frequency is dependent on the payload format. This means RTP timestamps are in units which are meaningful for the format of media being carried. Examples : 8KHz sampled u-law audio would use an 8KHz sample driven clock for timestamps. MPEG video would use the MPEG 90KHz clock. COMP9519 Multimedia Systems Lecture 6 Slide 17 J. Zhang & R. Mathew

RTP Header Fields Time Stamp : 32 bits Random start timestamp audio clock = 8kHz y y + num_cycles video clock = 90kHz RTP Timestamps can be used at the receiver for intra media synchronization. x x + num_cycles At the receiver inter media synchronization uses RTP Timestamps with the help of RTCP. clocks for audio and video streams from the same source use different timebases. RTCP relates these different clocks to a reference clock (wall clock), so audio-video synchronization can be performed. COMP9519 Multimedia Systems Lecture 6 Slide 18 J. Zhang & R. Mathew

RTP Header Fields SSRC : 32 bits Synchronization source identifies an RTP sender value chosen randomly identify sources in multi-sender sessions ssrc = id1 ssrc = id2 AMR AMR translator u-law u-law id1 id2 mixer ssrc = id3 COMP9519 Multimedia Systems Lecture 6 Slide 19 J. Zhang & R. Mathew ssrc = id3 csrc = id1, id2

RTP Header Fields CSRC : 32 bits each Contributing Sources Identifies contributing sources number of contributing sources specified by CC field CSRC identifiers are inserted by mixers using SSRC of contributing sources ssrc = id1 AMR u-law id1 ssrc = id3 csrc = id1, id2 ssrc = id2 AMR u-law translator id2 mixer ssrc = id3 COMP9519 Multimedia Systems Lecture 6 Slide 20 J. Zhang & R. Mathew

RTP RTP Header Fields CSRC : 32 bits each Contributing Sources Example use of CSRC Audio conference where multiple audio streams are mixed together to form an output packet stream. The CSRC list identifies all the sources that have been mixed together thereby allowing identification of current participants or talkers in the session. Note that mixers will generate its own timing for the combined stream, therefore a mixer is identified by its own SSRC COMP9519 Multimedia Systems Lecture 6 Slide 21 J. Zhang & R. Mathew

RTCP RTP allows for transport of media data But. How do we know the packets are being received? packet loss rate? available bandwidth? What are the bounds on delay and jitter? Need some feedback on network performance Real-Time Control Protocol (RTCP) Used in conjunction with RTP Primary function feedback on quality of data distribution COMP9519 Multimedia Systems Lecture 6 Slide 22 J. Zhang & R. Mathew

RTCP Control information exchanged between session members by RTCP packets Five RTCP packet types defined RTCP packets begin with a fixed part Similar to RTP data packets RTCP packets are sent periodically from all participants of a session multicast example RTCP packets are stackable compound packets concatenate multiple packets Usually sent over UDP (same as RTP) Using a port number = RTP_port + 1 COMP9519 Multimedia Systems Lecture 6 Slide 23 J. Zhang & R. Mathew

RTCP RTCP packet types to carry control information RTCP SDES : Source description items sent by all session members includes a persistent identifier of sources in the session RTCP SR : Sender Report sent by active senders report of transmission and reception statistics RTCP RR : Receiver Report Sent by receivers (not active senders) report of reception statistics RTCP BYE : Packet to indicate end of participation RTCP APP : Application specific functions COMP9519 Multimedia Systems Lecture 6 Slide 24 J. Zhang & R. Mathew

RTCP RTCP SDES Packet : Source description items Source identifier item called CNAME is mandatory CNAME canonical name Receivers map SSRC and CNAME Note : SSRC can change but CNAME is persistent CNAME also required by receivers to associate multiple streams from a single participant (eg synchronize auido + video) Other descriptions items include NAME : of user EMAIL : of user PHONE : number of user LOC : geographic user location, depends on application TOOL : name of application generating the stream COMP9519 Multimedia Systems Lecture 6 Slide 25 J. Zhang & R. Mathew

RTCP RTCP Sender Report Packet SSRC ID for the originator of this Sender Report Network Time Protocol timestamp (wallclock time) when this report was sent Associated RTP time stamp (same instant as NTP timestamp). Total number of RTP packets sent since start of transmission Used for synch at the receiver Total number of payload octets sent since start of transmission COMP9519 Multimedia Systems Lecture 6 Slide 26 J. Zhang & R. Mathew

RTCP RTCP Receiver Report Packet SSRC ID for the originator of this RR SSRC ID of sender Fraction of RTP packets lost since last Receiver Report Total number of RTP packets lost since start of transmission PT=RR=201 Highest RTP Sequence number of packets received Delay between receiving Sender Report and sending this Receiver Report RTP packet inter-arrival time RTP timestamp of last Sender Report received from source COMP9519 Multimedia Systems Lecture 6 Slide 27 J. Zhang & R. Mathew

RTCP RTCP Bandwidth Utilization If transmission rate of RTCP packets were fixed then RTCP traffic would grow linearly with number of receivers RTCP rate dependents on the number of participants Increase in participants (eg receivers) then decrease rate Need to keep control traffic to a small fraction at all times So as not to impair the primary function of data transport Need to have common known rules to follow in calculating the bandwidth used for control traffic So that each participant can independently calculate its share of the control bandwidth COMP9519 Multimedia Systems Lecture 6 Slide 28 J. Zhang & R. Mathew

RTCP RTCP Bandwidth Utilization (continued) RTCP rate dependents on the number of participants Try to keep control traffic to 5% session bandwidth Senders collectively allocated 25% of control bandwidth Receivers equally share the remaining 75% of control bandwidth Interval between RTCP packets is varied randomly To avoid unintended synchronization of all participants. Varied [0.5,1.5] x interval Interval between RTCP packets is required to be > 5 s COMP9519 Multimedia Systems Lecture 6 Slide 29 J. Zhang & R. Mathew

System Overview compressed bit stream Reorder packets Identify lost packets Detect payload type Detect participants Packets to Frames fragment RTP Attach RTP Header Time stamping Sequence numbering Payload ID Frame indication Source ID Decoder RTP UDP UDP IP COMP9519 Multimedia Systems Lecture 6 Slide 30 J. Zhang & R. Mathew

System Overview MPEG-4 File / Encoder Receiver Side Issues -Buffering input packetize data -Error concealment -Inter media synchronization RTCP RTP Sender Side Issues RTCP Decoder RTP MPEG-4 Video -Conforming to Network Bandwidth -Adaptive Rate Control -Error Resilience -Packetization strategy UDP UDP COMP9519 Multimedia Systems Lecture 6 Slide 31 J. Zhang & R. Mathew IP

System Overview : Sender Side Issues Conforming to Network Bandwidth Example : live video encoding Rate Control to match available bandwidth MPEG-4 Encoder Rate Control 64 k bits/s 128 k bits/s Example : Pre-recorded video Multiple copies of content at various bit rates Choose appropriate version to match bandwidth 256 k bits/s 64 k bits/s 128 k bits/s 256 k bits/s COMP9519 Multimedia Systems Lecture 6 Slide 32 J. Zhang & R. Mathew

System Overview : Sender Side Issues Conforming to Network Bandwidth For multiple participants with different bandwidth access Scalable coding is an option Example : 3 layer coding Base Layer + 2 Enhancement layers MPEG-4 Encoder Rate Control internet 110 k bits/s MPEG-4 scalability options Temporal, SNR, Spatial Rate control for each layer to ensure combined rates meet bandwidth COMP9519 Multimedia Systems Lecture 6 Slide 33 J. Zhang & R. Mathew 90 k bits/s 64 k bits/s

System Overview : Sender Side Issues Adaptive Rate Control Even though rate control meets nominal bandwidth rates Fluctuations of available network bandwidth still occur Schemes to dynamically control output rate are required Make rate control dynamically adjust to network conditions RTCP provides some feedback about network Therefore can formulate adaptive rate control using RTCP Example : [a] On End-to-End Transport Architecture for MPEG-4 Video Streaming over the Internet, D Wu et al, IEEE CSVT, Vol. 10, No. 6, September 2000 COMP9519 Multimedia Systems Lecture 6 Slide 34 J. Zhang & R. Mathew

System Overview : Sender Side Issues Adaptive Rate Control MPEG-4 Encoder Rate Control RTP RTCP (SR) Use packet loss ratio returned in the RTCP packet This is the packets loss experienced by the receiver since the last RTCP packet IF small loss ratio increase output rate ELSE reduce output rate RTCP (RR) COMP9519 Multimedia Systems Lecture 6 Slide 35 J. Zhang & R. Mathew IF (P loss <= P threshold ) r = min{ ( r+ ), Peak_Rate } ELSE r = max{ (α x r), Min_Rate }

System Overview : Sender Side Issues Adaptive Rate Control IF (P loss <= P threshold ) r = min{ ( r+ ), Peak_Rate } ELSE r = max{ (α x r), Min_Rate } Trying to adjust output rate (r) to maintain packet loss ratio (P loss ) below a threshold (P threshold ) Additive increase in rate conservative increase If there is no congestion and hence low loss ratio Multiplicative decrease in rate fast adaptation Swift reduction necessary to shorten congestion period and reduce packet loss. COMP9519 Multimedia Systems Lecture 6 Slide 36 J. Zhang & R. Mathew

System Overview : Sender Side Issues Adaptive Rate Control Results from [a] for peer-to-peer network. Source rate and link capacity Rate (kbps) vs Time (s) Link utilization and packet loss ratio Percentage(%) vs Time (s) COMP9519 Multimedia Systems Lecture 6 Slide 37 J. Zhang & R. Mathew

System Overview : Sender Side Issues Error Resilience Packet loss causes degradation of perceptual quality at the receiver Need to ensure source coding includes mechanisms to be resilient to error For MPEG-4 examples of limiting the effects of packet loss include [a] regular intra frame coding or [b] intra block refresh schemes. [a] [b] I P P P P I P I P P P Intra frame stops the propagation of error Intra block refresh schemes also limit the propagation of error but with a smoother bit rate allocation among the frames COMP9519 Multimedia Systems Lecture 6 Slide 38 J. Zhang & R. Mathew

System Overview : Sender Side Issues Packetization Strategy Must meet network MTU limits Maximum Transmission Unit Minimum MTU of all links from sender to receiver Minimize number of packets required Thereby minimizing overhead Each packet overhead IP, UDP and RTP header Minimize dependency between consecutive packets For better error resilience COMP9519 Multimedia Systems Lecture 6 Slide 39 J. Zhang & R. Mathew

System Overview : Sender Side Issues Packetization Strategy Must meet network MTU limits Example : MTU = 1500 bytes Packets larger than MTU will result in IP fragmentation Resulting in overhead for each fragmented packet Loss of one fragment will corrupt the whole packet MPEG-4 : Try to allocate a single VOP (frame) per packet If packet is greater than MTU then break into multiple packets Can break any where - arbitrary byte positions Best to break at GOB or slice boundaries COMP9519 Multimedia Systems Lecture 6 Slide 40 J. Zhang & R. Mathew

System Overview : Sender Side Issues Packetization Strategy Minimize number of packets required Less packets means less bits spend on packet headers Example : 20 byte IP Header 8 byte UDP Header 12 byte RTP Header Minimize dependency between packets So that even if one packet is lost, the next packet can still be decoded Example : MPEG-4 use VOP boundaries to start / stop packets COMP9519 Multimedia Systems Lecture 6 Slide 41 J. Zhang & R. Mathew

System Overview : Sender Side Issues Packetization Strategy Minimize dependency between packets (continued) VOP 1 VOP 2 VOP 3 loss VOP 1 VOP 2 VOP 3 VOP 1 VOP 2 VOP 3 COMP9519 Multimedia Systems Lecture 6 Slide 42 J. Zhang & R. Mathew

System Overview : Receiver Side Issues Buffering Input Data Constant Transmit Rate Mean Receive Rate Constant Play Out Rate Number of Packets Packets Generated Packets Received Received Data Played out T N : Network transfer delay T B : Buffer Delay T N T B Time COMP9519 Multimedia Systems Lecture 6 Slide 43 J. Zhang & R. Mathew

System Overview : Receiver Side Issues Buffering Input Data Jitter can be removed by buffering received packets Receiver plays out packets T N + T B (milliseconds) after generation of the packet at the sender There exists a tradeoff between delay (due to buffering) and packet loss Packets received late (> T N + T B ) are ignored (i.e. lost) Live streaming applications have more stringent requirements on delay so this limits the amount of buffering COMP9519 Multimedia Systems Lecture 6 Slide 44 J. Zhang & R. Mathew

System Overview : Receiver Side Issues Error Concealment A streaming service will always be affected by lost packets. How to conceal errors due to missing packets? So as to minimize loss of quality Reduce impact to user perceived quality of image, video or audio service Error Resilience Techniques employed at the receiver to conceal errors These techniques may vary from receiver to receiver COMP9519 Multimedia Systems Lecture 6 Slide 45 J. Zhang & R. Mathew

System Overview : Receiver Side Issues Error Concealment Example : MPEG-4 One possibility is to replace a missing group of blocks / slice of the current frame with group of blocks / slice from the previous frame. Missing blocks t-1 Copy from previous frame t Not part of the MPEG-4 Standard Various techniques proposed and used by decoder developers COMP9519 Multimedia Systems Lecture 6 Slide 46 J. Zhang & R. Mathew

System Overview : Receiver Side Issues Inter Media Synchronization Need to synchronize multiple media streams Synchronize video and audio streams lip synch Synchronize video, audio and text RTP time stamps Start at a random number random offset for each stream May advance at different rates Audio and video clock rates could be different To synchronize streams at the receiver Requires information sent in the RTCP SR (sender reports) COMP9519 Multimedia Systems Lecture 6 Slide 47 J. Zhang & R. Mathew

System Overview : Receiver Side Issues Inter Media Synchronization Audio De-packetizer and Decoder Video De-packetizer and Decoder 1600 1760 1920 2080 2240 99000 108000 117000 SR 1680 : 15:2:32.00 SR 106200 : 15:2:32.01 Audio clock = 8000Hz (eg u-law) 1760 = 32.00 + 0.01 1920 = 32.00 + 0.03 2240 = 32.00 + 0.07 ---------------------------- Video clock = 90000Hz (eg MPEG-4) 108000 = 32.01 + 0.02 117000 = 32.01 + 0.12 COMP9519 Multimedia Systems Lecture 6 Slide 48 J. Zhang & R. Mathew

System Overview : Receiver Side Issues Inter Media Synchronization RTCP SR Maps RTP timestamps with the NTP time stamps Relates to the instance when the SR is sent More information on Network Time Protocol Synchronizing audio with Video Must meet human perception limits Example : Audio can lead video by 40 ms Video can lead audio by 120 ms COMP9519 Multimedia Systems Lecture 6 Slide 49 J. Zhang & R. Mathew

System Overview MPEG-4 Video AMR Audio Synchronized MPEG-4 Video and AMR Audio Presentation packetize packetize RTCP RTP RTCP RTP Audio Decoder Video Decoder RTCP RTP RTCP RTP UDP UDP IP COMP9519 Multimedia Systems Lecture 6 Slide 50 J. Zhang & R. Mathew

System Overview Inter Media Synchronization Separate RTP & RTCP streams for video and audio Synchronized display at receiver Both streams part of one session Information on AMR Audio IETF, Request for Comments: 3267 Real-Time Transport Protocol (RTP) Payload Format and File Storage Format for the Adaptive Multi-Rate (AMR) and Adaptive Multi-Rate Wideband (AMR-WB) Audio Codecs ftp://ftp.rfc-editor.org/in-notes/rfc3267.txt COMP9519 Multimedia Systems Lecture 6 Slide 51 J. Zhang & R. Mathew

Client Architecture : An Example Reference Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 Review of Streaming Video, RTP, MPEG-4 Client Architecture for streaming MPEG-4 video Multilevel buffer system COMP9519 Multimedia Systems Lecture 6 Slide 52 J. Zhang & R. Mathew

Client Architecture : An Example Multiple buffer architecture to allow efficient control of media processing and presentation. Three Modules Network Module : Packet buffer Codec Module : Frame Buffer Presentation Module : Image Buffer Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 COMP9519 Multimedia Systems Lecture 6 Slide 53 J. Zhang & R. Mathew

Client Architecture : Multiple Buffers Packet Buffer Frame Buffer Image Buffer Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 COMP9519 Multimedia Systems Lecture 6 Slide 54 J. Zhang & R. Mathew

Client Architecture : Multiple Buffers Packet Buffer Reorder packets Identify missing packets (allow for easy insertion) Ignores delayed packets (outside moving window) Frame Buffer Stores video packets (smoothing out network jitter) Allows dropping of frame (when lacking CPU resources) Identify frames to be dropped (B, P) Image Buffer Stores decoded frames for rendering Smoothing decoders speed COMP9519 Multimedia Systems Lecture 6 Slide 55 J. Zhang & R. Mathew

Client Architecture : Rendering Rendered frames rates with and without the image buffer Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 COMP9519 Multimedia Systems Lecture 6 Slide 56 J. Zhang & R. Mathew

Client Architecture : Conclusion The MPEG-4 MLBA subsystem facilitates three player-related activities: precise A/V synchronization, client-based QoS management, and improved rendering performance through an image buffer. Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 COMP9519 Multimedia Systems Lecture 6 Slide 57 J. Zhang & R. Mathew

Introduction to SDP (Control Info) How to describe content and session info to client Session Description Protocol -- SDP Example : An existing live multicast session Video and Audio streams Transport - RTP/UDP/IP, Control RTCP/UDP/IP A new client wanting to join the multicast session Needs to know multicast IP address and port Media streams in a session (e.g. video only or audio + video) Payload format (e.g. MPEG-4 video, AMR audio) Initialization data for video and audio decoders Transport protocol used Other information. Need a way to describe a multimedia session To enable new clients to easily join the session COMP9519 Multimedia Systems Lecture 6 Slide 58 J. Zhang & R. Mathew

SDP Session Description Protocol (SDP) IETF RFC2327 www.ietf.org/rfc/rfc2327.txt For describing multimedia sessions To communicate the existence of a session To convey sufficient information to join a session Simple text format Defined to be general purpose Can be used for a wide range of network environments And applications COMP9519 Multimedia Systems Lecture 6 Slide 59 J. Zhang & R. Mathew

SDP SDP includes Session name and purpose Time the session is active The media comprising the session Information to receive media (addresses, ports, formats) Information about bandwidth to be used Contact information of a person responsible for the session SDP is used by other signaling / initiation protocols SIP : Session Initiation Protocol RTSP : Real-time Streaming Protocol COMP9519 Multimedia Systems Lecture 6 Slide 60 J. Zhang & R. Mathew

SDP session level description v= (protocol version) o= (owner and session identifier) s= (session name) i=* (session information) u=* (URI of description) e=* (email address) p=* (phone number) c=* (connection information) b=* (bandwidth information) z=* (time zone adjustments) k=* (encryption key) a=* (zero or more session attribute) t= (time the session is active) r=* (zero or more repeat times) SDP session description consists of a number of lines of text of the form <type>=<value> <type> is always exactly one character and is case-significant. media level description m= (media & transport address) i=* (media title) c=* (connection information) b=* (bandwidth information) k=* (encryption key) a=* (zero or more media attribute) COMP9519 Multimedia Systems Lecture 6 Slide 61 J. Zhang & R. Mathew

SDP Version number increased when a modification is made to the session data. Recommended that an NTP timestamp is used v=0 o=nicta 2890844526 2890842807 IN IP4 129.94.135.201 s=camera Originator ONE information i=video stream for Session realtime name surveillance u=http://www.nicta.com/mmvc/demos/surveillancevideo.pdf (address of Session machine Information from which the session was created) e=jian.zhang@nicta.com.au (text description / title for (Jian session) Zhang) URL for more information about the session c=in IP4 225.0.0.37/2 Contact person e-mail t=0 0 (person responsible Connection not necessarily Details the creator) a=recvonly <network type> <address type> <connection address> m=video IN 20000 (internet) RTP/AVP IP4 (IP v4) Time 98 address Information (multicast address/ttl) a=rtpmap:98 MP4V-ES/90000 Session level attribute <start time> <end time> (operate in receive only mode) a=fmtp:98 profile-level-id=1; (ntp time, 0,0 implies config=000001b001000001 permanent session) Media Media attribute Announcement (rtp map) a=orient:portrait <media> <port> <transport> <fmt list> <username> <session id> <version> <network type> <address type> <address> <payload type> (fmt : refers Media <encoding to media attribute name>/<clock format (format specific specific rate> (map RTP dynamic payload number to information) parameters) media format and clock rate) (e.g. (info Dynamic regarding payload mpeg-4 number) media, profile, level & initialization data) Media attribute (orientation) (only used in some applications, example landscape or portrait) COMP9519 Multimedia Systems Lecture 6 Slide 62 J. Zhang & R. Mathew

System Overview Example Using SDP to join a multicast session Request SDP file via HTTP Retrieve information from downloaded SDP file Receive RTP streams on SDP specifed address & port Decode and display specified media HTTP GET SDP file http://www.nicta.com/lecture.sdp Media Server RTP Media Stream Client RTCP COMP9519 Multimedia Systems Lecture 6 Slide 63 J. Zhang & R. Mathew

System Overview Example Traffic Monitoring Continuous multicast streaming of video A client can receive the stream by downloading SDP via http (web browser) Provide SDP file to QuickTime player Player initializes and waits for stream data Easily support multiple client COMP9519 Multimedia Systems Lecture 6 Slide 64 J. Zhang & R. Mathew

References and Further Reading RTP / RTCP : IETF, RFC 3350 RTP: A Transport Protocol for Real-Time Applications Streaming Video over the Internet: Approaches and Directions, Dapeng Wu, Yiwei Thomas Hou, Wenwu Zhu, Ya-Qin Zhang, and Jon M. Peha, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 11, NO. 3, MARCH 2001 On End-to-End Transport Architecture for MPEG-4 Video Streaming over the Internet, D Wu et al, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, Vol. 10, No. 6, September 2000 COMP9519 Multimedia Systems Lecture 6 Slide 65 J. Zhang & R. Mathew