Lecture 7: Internet Streaming Media

Similar documents
Lecture 7: Internet Streaming Media. Reji Mathew NICTA & CSE UNSW COMP9519 Multimedia Systems S2 2007

Internet Streaming Media. Reji Mathew NICTA & CSE UNSW COMP9519 Multimedia Systems S2 2006

Internet Streaming Media

Internet Streaming Media. Reji Mathew NICTA & CSE UNSW COMP9519 Multimedia Systems S2 2007

Internet Streaming Media

Lecture 6: Internet Streaming Media

EDA095 Audio and Video Streaming

Popular protocols for serving media

Multimedia Communication

EDA095 Audio and Video Streaming

EDA095 Audio and Video Streaming

Mohammad Hossein Manshaei 1393

Session Initiation Protocol (SIP) Overview

CSCD 433/533 Advanced Networks Fall Lecture 14 RTSP and Transport Protocols/ RTP

Multimedia Applications. Classification of Applications. Transport and Network Layer

Overview : Image/Video Coding Techniques

Internet Technologies for Multimedia Applications

Z24: Signalling Protocols

Media server and QoS (so far)

Using RTSP with Firewalls, Proxies, and Other Intermediary Network Devices

Real Time Protocols. Overview. Introduction. Tarik Cicic University of Oslo December IETF-suite of real-time protocols data transport:

TSIN02 - Internetworking

Understanding SIP exchanges by experimentation

TSIN02 - Internetworking

Kommunikationssysteme [KS]

Session Initiation Protocol (SIP) Overview

Real-time Services BUPT/QMUL

Media Communications Internet Telephony and Teleconference

TSIN02 - Internetworking

Lecture 14: Multimedia Communications

Transporting Voice by Using IP

Department of Computer Science. Burapha University 6 SIP (I)

Video Streaming and Media Session Protocols

Overview : Image/Video Coding Techniques

Multimedia in the Internet

Multimedia Protocols. Foreleser: Carsten Griwodz Mai INF-3190: Multimedia Protocols

MISB RP 1302 RECOMMENDED PRACTICE. 27 February Inserting KLV in Session Description Protocol (SDP) 1 Scope. 2 References

Telecommunication Services Engineering Lab. Roch H. Glitho

ITTC Communication Networks The University of Kansas EECS 780 Multimedia and Session Control

Voice over IP (VoIP)

Computer Networks. Wenzhong Li. Nanjing University

EVC-IP-1,3-2MP. (Audio, Video Player Development Guide)

13. Internet Applications 최양희서울대학교컴퓨터공학부

Request for Comments: Category: Standards Track Columbia U. G. Camarillo Ericsson A. Johnston WorldCom J. Peterson Neustar R.

Real-time Services BUPT/QMUL

Digital Asset Management 5. Streaming multimedia

Chapter 28. Multimedia

Streaming Technologies Delivering Multimedia into the Future. May 2014

Session Initiation Protocol (SIP)

CS 218 F Nov 3 lecture: Streaming video/audio Adaptive encoding (eg, layered encoding) TCP friendliness. References:

Outline. QoS routing in ad-hoc networks. Real-time traffic support. Classification of QoS approaches. QoS design choices

Chapter 3: IP Multimedia Subsystems and Application-Level Signaling

Multimedia networking: outline

Applications real time/non-real time

Interoperability Test Guideline. For SIP/MPEG-4 Multimedia Communication System

CSC 4900 Computer Networks: Multimedia Applications

Multimedia Streaming Protocols RTSP/RTP RTCP. Prof. Lin Weiguo Copyleft 2009~2017, School of Computing, CUC

RTP. Prof. C. Noronha RTP. Real-Time Transport Protocol RFC 1889

4 rd class Department of Network College of IT- University of Babylon

Provide a generic transport capabilities for real-time multimedia applications Supports both conversational and streaming applications

Medical Sensor Application Framework Based on IMS/SIP Platform

Lecture 9: Media over IP

S Postgraduate Course in Radio Communications. Application Layer Mobility in WLAN. Antti Keurulainen,

Outline. Multimedia is different Real Time Protocol (RTP) Session Description Protocol (SDP) Session Initiation Protocol (SIP)

Networking Applications

Overview. Slide. Special Module on Media Processing and Communication

INTERFACE SPECIFICATION SIP Trunking. 8x8 SIP Trunking. Interface Specification. Version 2.0

AARNet Copyright SDP Deep Dive. Network Operations. Bill Efthimiou APAN33 SIP workshop February 2012

Multimedia networks. Additional references. Jargon. Analog to Digital (S5 4.3) KR: Kurose and Ross chapter 7 (KR3: 3 rd ed)

MRCP Version 1. A.1 Overview

CS640: Introduction to Computer Networks. Application Classes. Application Classes (more) 11/20/2007

MISB EG Motion Imagery Standards Board Engineering Guideline. 24 April Delivery of Low Bandwidth Motion Imagery. 1 Scope.

Tech-invite. RFC 3261's SIP Examples. biloxi.com Registrar. Bob's SIP phone

in the Internet Andrea Bianco Telecommunication Network Group Application taxonomy

Service/company landscape include 1-1

ENSC 833-3: NETWORK PROTOCOLS AND PERFORMANCE. Implement Session Initiation Protocol (SIP) User Agent Prototype

Real-Time Control Protocol (RTCP)

IETF Video Standards A review, some history, and some reflections. Colin Perkins

CS519: Computer Networks. Lecture 9: May 03, 2004 Media over Internet

VoIP. ALLPPT.com _ Free PowerPoint Templates, Diagrams and Charts

File transfer. Internet Applications (FTP,WWW, ) Connections. Data connections

Streaming (Multi)media

Multimedia Networking

Technical Guide G6. RTSP Streaming. Network Camera October 26, 2012 G6TG005 Revision 1.0.0

Internet Streaming Media Alliance Hyperlinked Video Specification Version 1.0 September 2006

Transporting Voice by Using IP

OpenVideo Streaming API RTSP Connection Methods

Verifying the Internet Streamer CDS

Multimedia Networking

Multimedia Communications

Reserving N and N+1 Ports with PCP

The Session Initiation Protocol

Compliance with RFC 3261

Quality of Service. Qos Mechanisms. EECS 122: Lecture 15

IMS Client Framework for All IP-Based Communication Networks

Information About SIP Compliance with RFC 3261

Today. March 7, 2006 EECS122 Lecture 15 (AKP) 4. D(t) Scheduling Discipline. March 7, 2006 EECS122 Lecture 15 (AKP) 5

Journal of Information, Control and Management Systems, Vol. X, (200X), No.X SIP OVER NAT. Pavel Segeč

A SIP of IP-telephony

The Transport Layer: User Datagram Protocol

Transcription:

Lecture 7: Internet Streaming Media Reji Mathew NICTA & CSE UNSW COMP9519 Multimedia Systems S2 2007 Notes on Previous Lecture RTCP Packets SR and RR can be used for independent network management Payload independent monitors that receive only the RTCP packets to evaluate the performance of their networks for multicast distribution COMP9519 Multimedia Systems Lecture 7 Slide 2 Reji Mathew Introduction The story so far. Video Coding MPEG-4 Audio Coding u-law, AMR Packetization and Transport of media RTP Transport Control / QoS Feedback - RTCP Lecture Outline Literature Review Client Architecture : An Example Next Further Protocols Enabling Streaming SDP - Description RTSP Signaling SIP Client Architecture : An Example Reference Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 Review of Streaming Video, RTP, MPEG-4 Client Architecture for streaming MPEG-4 video Multilevel buffer system COMP9519 Multimedia Systems Lecture 7 Slide 3 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 4 Reji Mathew

Client Architecture : An Example Multiple buffer architecture to allow efficient control of media processing and presentation. Client Architecture : Multiple Buffers Packet Buffer Three Modules Network Module : Packet buffer Codec Module : Frame Buffer Presentation Module : Image Buffer Frame Buffer Image Buffer Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 COMP9519 Multimedia Systems Lecture 7 Slide 5 Reji Mathew Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 COMP9519 Multimedia Systems Lecture 7 Slide 6 Reji Mathew Client Architecture : Multiple Buffers Packet Buffer Reorder packets Identify missing packets (allow for easy insertion) Ignores delayed packets (outside moving window) Client Architecture : Rendering Frame Buffer Stores video packets (smoothing out network jitter) Allows dropping of frame (when lacking CPU resources) Identify frames to be dropped (B, P) Image Buffer Stores decoded frames for rendering Smoothing decoders speed Rendered frames rates with and without the image buffer Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 COMP9519 Multimedia Systems Lecture 7 Slide 7 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 8 Reji Mathew

Client Architecture : Conclusion The MPEG-4 MLBA subsystem facilitates three player-related activities: precise A/V synchronization, client-based QoS management, and improved rendering performance through an image buffer. Lecture Outline Session Description Protocol How to describe a multimedia session? SDP example RTSP Internet VCR controls Stop, Pause, Play, Fast Forward over the internet RTSP Methods Signal Timing Diagram Protocol stack for a total streaming system SIP Quick Overview MPEG-4 File Format What s so good about MP4? File Format and Structure Hinting Client architecture for MPEG-4 streaming, Haifeng Xu Diamand, J. Luthra, A. IEEE Multimedia, April-June 2004, Vol 11, Issue: 2, pages 16-23 COMP9519 Multimedia Systems Lecture 7 Slide 9 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 10 Reji Mathew Introduction Example : An existing live multicast session Video and Audio streams Transport - RTP/UDP/IP, Control RTCP/UDP/IP A new client wanting to join the multicast session Needs to know multicast IP address and port Media streams in a session (e.g. video only or audio + video) Payload format (e.g. MPEG-4 video, AMR audio) Initialization data for video and audio decoders Transport protocol used Other information. Need a way to describe a multimedia session To enable new clients to easily join the session SDP Session Description Protocol (SDP) IETF RFC2327 www.ietf.org/rfc/rfc2327.txt For describing multimedia sessions To communicate the existence of a session To convey sufficient information to join a session Simple text format Defined to be general purpose Can be used for a wide range of network environments And applications COMP9519 Multimedia Systems Lecture 7 Slide 11 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 12 Reji Mathew

SDP SDP includes Session name and purpose Time the session is active The media comprising the session Information to receive media (addresses, ports, formats) Information about bandwidth to be used Contact information of a person responsible for the session SDP is used by other signaling / initiation protocols SIP : Session Initiation Protocol RTSP : Real-time Streaming Protocol COMP9519 Multimedia Systems Lecture 7 Slide 13 Reji Mathew SDP session level description v= (protocol version) o= (owner and session identifier) s= (session name) i=* (session information) u=* (URI of description) e=* (email address) p=* (phone number) c=* (connection information) b=* (bandwidth information) z=* (time zone adjustments) k=* (encryption key) a=* (zero or more session attribute) t= (time the session is active) r=* (zero or more repeat times) COMP9519 Multimedia Systems Lecture 7 Slide 14 Reji Mathew SDP session description consists of a number of lines of text of the form <type>=<value> <type> is always exactly one character and is case-significant. media level description m= (media & transport address) i=* (media title) c=* (connection information) b=* (bandwidth information) k=* (encryption key) a=* (zero or more media attribute) increased when a modification is made to the session data. SDP Version number Recommended that an NTP timestamp is used v=0 o=mmvc 2890844526 2890842807 IN IP4 129.94.135.201 s=camera Originator ONE information i=video <username> stream <session for Session realtime id> name <version> surveillance <network type> <address type> <address> u=http://www.nicta.com/mmvc/demos/surveillancevideo.pdf (address of Session machine Information from which the session was created) e=rmathew@nicta.com.au (text description / title (Reji for session) Mathew) URL for more information about the session c=in IP4 225.0.0.37/2 Contact person e-mail t=0 0 (person responsible Connection not necessarily Details the creator) a=recvonly <network type> <address type> <connection address> m=video IN 20000 (internet) RTP/AVP IP4 (IP v4) Time 98 address Information (multicast address/ttl) a=rtpmap:98 MP4V-ES/90000 Session level attribute <start time> <end time> (operate in receive only mode) a=fmtp:98 profile-level-id=1; (ntp time, 0,0 Media implies config=000001b001000001 Announcement permanent session) Media attribute (rtp map) a=orient:portrait <payload <media> type> <port> <encoding <transport> name>/<clock <fmt list> rate> (fmt : refers Media to media attribute format (format specific specific information) parameters) (map RTP dynamic payload number to media format and clock rate) (e.g. (info Dynamic regarding payload mpeg-4 number) media, profile, level & initialization data) Media attribute (orientation) (only used in some applications, example landscape or portrait) COMP9519 Multimedia Systems Lecture 7 Slide 15 Reji Mathew System Overview Example Using SDP to join a multicast session Request SDP file via HTTP Retrieve information from downloaded SDP file Receive RTP streams on SDP specifed address & port Decode and display specified media Media Server HTTP GET SDP file RTP Media Stream RTCP COMP9519 Multimedia Systems Lecture 7 Slide 16 Reji Mathew http://www.nicta.com/lecture.sdp Client

System Overview Example Traffic Monitoring Continuous multicast streaming of video A client can receive the stream by downloading SDP via http (web browser) Provide SDP file to QuickTime player Player initializes and waits for stream data Easily support multiple client Demo Link Lecture Outline Session Description Protocol How to describe a multimedia session? SDP example RTSP Internet VCR controls Stop, Pause, Play, Fast Forward over the internet RTSP Methods Signal Timing Diagram Protocol stack for a total streaming system SIP Quick Overview MPEG-4 File Format What s so good about MP4? File Format and Structure Hinting COMP9519 Multimedia Systems Lecture 7 Slide 17 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 18 Reji Mathew RTSP Now we can describe and transport media streams But how to control these streams Start, Stop, Pause, Fast Forward, Rewind internet VCR Solution RTSP Real Time Streaming Protocol Establishes and controls one or more continuous media streams - such as audio and video Similar in syntax and operation to HTTP/1.1 Client Server protocol Text based IETF RFC 2326 COMP9519 Multimedia Systems Lecture 7 Slide 19 Reji Mathew RTSP More Info www.ietf.org/rfc/rfc2326.txt www.rtsp.org Example Play Request (TCP/IP) Play Response (TCP/IP) Media Server RTSP/1.0 200 OK CSeq: 833 Date: 23 Jan 1997 15:35:06 GMT Range: smpte=0:10:22-;time=19970123t153600z PLAY rtsp://audio.example.com/twister.en RTSP/1.0 CSeq: 833 Range: smpte=0:10:20-;time=19970123t153600z COMP9519 Multimedia Systems Lecture 7 Slide 20 Reji Mathew Client

RTSP Protocol Properties Media Transport Independent RTSP does not depend on the transport mechanism used to carry the continuous media Media transport can be via RTP or some other mechanism Transport-independent: RTSP may use either an unreliable datagram protocol (UDP) or a reliable stream protocol such as TCP Control of recording devices: Can control both recording only and playback only devices, As well as devices that can alternate between the two modes RTSP Protocol Properties Extendable: New methods and parameters can be easily added to RTSP. Easy to parse Multi-server capable: Each media stream within a presentation can reside on a different server. HTTP-friendly: RTSP reuses HTTP concepts, so that the existing infrastructure can be reused. COMP9519 Multimedia Systems Lecture 7 Slide 21 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 22 Reji Mathew RTSP Protocol Operation Text based messages between client and server Messages can be : Requests Responses Request Messages Request-line Message - header Message - header CRLF Message - body Contains URI of the resource The method to be applied to the resource The protocol version in use General and request headers May or may not be present COMP9519 Multimedia Systems Lecture 7 Slide 23 Reji Mathew RTSP Protocol Operation Response Messages Request-line Message - header Message - header CRLF Message - body The protocol version in use Status code (3 digit code) Textual description of status code General and response headers May or may not be present Next : examples of Request and Response messages Corresponding to five important request methods Methods : Describe, Setup, Play, Pause, Teardown Unicast example COMP9519 Multimedia Systems Lecture 7 Slide 24 Reji Mathew

RTSP : Describe Method Retrieves the description of a presentation, identified by the request URL, from a server. The DESCRIBE reply-response pair constitutes the media initialization phase of RTSP. C->S: DESCRIBE rtsp://nicta.com/twister RTSP/1.0 Version Request line CSeq: 1 Presentation URL S->C: RTSP/1.0 200 OK Response line: Version, Status code 200 with description Method CSeq: 1 Same CSeq number as request Content-Type: application/sdp Content-Length: 164 v=0 o=- 2890844256 2890842807 IN IP4 172.16.2.93 s=rtsp Session i=an Example of RTSP Session Usage a=control:rtsp://nicta.com/twister t=0 0 m=audio 0 RTP/AVP 0 a=control:rtsp://nicta.com/twister/audio m=video 0 RTP/AVP 26 a=control:rtsp://nicta.com/twister/video CSeq header field Sequence number for an RTSP request-response pair Must be present in all requests and responses COMP9519 Multimedia Systems Lecture 7 Slide 25 Reji Mathew SDP Information (message body) Aggregate control possible (session level control attribute) RTSP : Setup Method Specifies the transport mechanism to be used for the streamed media C->S: SETUP rtsp://nicta.com/twister/audio RTSP/1.0 CSeq: 2 Transport: RTP/AVP;unicast;client_port=8000-8001 S->C: RTSP/1.0 200 OK CSeq: 2 S->C: RTSP/1.0 200 OK CSeq: 3 Transport: RTP/AVP;unicast;client_port=8002-8003; server_port=9004-9005 COMP9519 Multimedia Systems Lecture 7 Slide 26 Reji Mathew unicast RTP/RTCP port pair on which the client has chosen to receive media data and control information Transport: RTP/AVP;unicast;client_port=8000-8001; server_port=9000-9001 RTSP session ID unicast RTP/RTCP port C->S: SETUP rtsp://nicta.com/twister/video RTSP/1.0 pair on which the server CSeq: 3 has chosen to send Transport: RTP/AVP;unicast;client_port=8002-8003 media data and control information Setup operates on each stream separately Same session ID used RTSP : Play Method Tells the server to start sending data via the mechanism specified in SETUP request/response messages C->S: PLAY rtsp://nicta.com/twister RTSP/1.0 CSeq: 4 Range: npt=0- S->C: RTSP/1.0 200 OK CSeq: 4 RTP-Info: url=rtsp://foo/twister/video; seq=9810092;rtptime=3450012 Aggregate control of all media streams in the presentation. The PLAY request starts streaming both audio and video streams. Range header used to specify time ranges (start / stop) for playback Normal play time (NPT) indicates the stream absolute position relative to the beginning of the presentation. RTSP : Pause Method Causes the stream delivery to be interrupted (halted) temporarily. C->S: PAUSE rtsp://nicta.com/twister RTSP/1.0 CSeq: 5 S->C: RTSP/1.0 200 OK CSeq: 5 Aggregate control of all media streams in the presentation. The PAUSE request halts the streaming of both audio and video streams. COMP9519 Multimedia Systems Lecture 7 Slide 27 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 28 Reji Mathew

RTSP : Teardown Method Stops the stream delivery for the given URI, freeing the resources associated with it C->S: TEARDOWN rtsp://nicta.com/twister RTSP/1.0 CSeq: 892 S->C: RTSP/1.0 200 OK CSeq: 892 RTSP : Signal Timing Diagram Media Client DESCRIBE Response (SDP) SETUP (Video stream) Response SETUP (Audio stream) Response Media Server PLAY Response PAUSE Response COMP9519 Multimedia Systems Lecture 7 Slide 29 Reji Mathew TEARDOWN Response COMP9519 Multimedia Systems Lecture 7 Slide 30 Reji Mathew Summary of RTSP Methods OPTIONS : SETUP : ANNOUNCE : DESCRIBE : PLAY : RECORD : REDIRECT : PAUSE : get available methods establish transport change description of media object get description of media start playback, reposition start recording redirect client to new server pause delivery, while keeping state SET PARAMETER : device or encoding control TEARDOWN : remove state / close RTSP : Comparison with HTTP Similar to HTTP, but some important differences for RTSP are : Data delivery takes place out-of-band in a different protocol e.g. RTP Both the media client and media server can issue requests Client > Server : SETUP, PLAY,. Server > Client : ANNOUNCE, REDIRECT,. Requests are not stateless : client & server state machines client changes state on receipt of replies to requests server changes state on receiving requests Init Ready Playing Recording COMP9519 Multimedia Systems Lecture 7 Slide 31 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 32 Reji Mathew

Protocol Stack Lecture Outline MPEG-4 File / Encoder RTCP packetize RTP SDP RTSP SDP RTSP RTCP Decoder RTP MPEG-4 Video Session Description Protocol How to describe a multimedia session? SDP example RTSP Internet VCR controls Stop, Pause, Play, Fast Forward over the internet RTSP Methods Signal Timing Diagram Protocol stack for a total streaming system SIP Quick Overview UDP TCP TCP UDP MPEG-4 File Format What s so good about MP4? File Format and Structure Hinting IP COMP9519 Multimedia Systems Lecture 7 Slide 33 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 34 Reji Mathew SIP Session Initiation Protocol (SIP) Another session control protocol IETF RFC 3261 www.ietf.org/rfc/rfc3261.txt Protocol that can establish, modify and terminate multimedia sessions. Applications IP telephony, multimedia conferences SIP, like RTSP, Uses text-based request/response transaction model Requests contain Methods and Header fields Responses include 3 digit status codes (eg 200 OK ) Is Transport layer independent Uses other protocols for media delivery (eg RTP). SIP SIP Methods INVITE, ACK, CANCEL : for setting up sessions BYE : for terminating sessions REGISTER : for registering contact information OPTIONS : for querying servers about capabilities INVITE User agent client initiates a session with INVITE Invitation can be accepted by a user agent server SIP invitations convey session descriptions that allow participants to agree on a set of compatible media types Offer/answer model ACK Response confirmation to INVITE COMP9519 Multimedia Systems Lecture 7 Slide 35 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 36 Reji Mathew

SIP INVITE Example INVITE sip:bob@biloxi.com SIP/2.0 Via:SIP/2.0/UDPpc33.atlanta.com;branch=z9hG4bK776asdhds Max-Forwards: 70 To: Bob <sip:bob@biloxi.com> From: Alice <sip:alice@atlanta.com>;tag=1928301774 Call-ID: a84b4c76e66710@pc33.atlanta.com CSeq: 314159 INVITE Contact: <sip:alice@pc33.atlanta.com> Content-Type: application/sdp Content-Length: 142 (Alice's SDP not shown) COMP9519 Multimedia Systems Lecture 7 Slide 37 Reji Mathew SIP SIP supports user mobility With proxy servers to help route requests to user s current location Registration function allowing users to upload current locations for use by proxy servers COMP9519 Multimedia Systems Lecture 7 Slide 38 Reji Mathew SIP atlanta.com proxy biloxi.com proxy Alice INVITE INVITE Bob SIP Response Example SIP/2.0 200 OK Via: SIP/2.0/UDP server10.biloxi.com ;branch=z9hg4bknashds8;received=192.0.2.3 Via: SIP/2.0/UDP bigbox3.site3.atlanta.com ;branch=z9hg4bk77ef4c2312983.1;received=192.0.2.2 INVITE Via: SIP/2.0/UDP pc33.atlanta.com ;branch=z9hg4bk776asdhds ;received=192.0.2.1 200 OK 200 OK 200 OK To: Bob <sip:bob@biloxi.com>;tag=a6c85cf From: Alice <sip:alice@atlanta.com>;tag=1928301774 ACK Media Session BYE Call-ID: a84b4c76e66710@pc33.atlanta.com CSeq: 314159 INVITE Contact: <sip:bob@192.0.2.4> Content-Type: application/sdp 200 OK COMP9519 Multimedia Systems Lecture 7 Slide 39 Reji Mathew Content-Length: 131 COMP9519 Multimedia Systems Lecture 7 Slide 40 Reji Mathew (Bob's SDP not shown)

Streaming Applications Streaming Media Players Video on Demand VoIP Video Conf (3G) IPTV Qucik Time Player Real Player Microsoft Media player Packet Video MPEG SDP RTP RTCP RTSP SIP Players for various platforms PC Pocket PC 3G Phones IP COMP9519 Multimedia Systems Lecture 7 Slide 41 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 42 Reji Mathew Lecture Outline Session Description Protocol How to describe a multimedia session? SDP example RTSP Internet VCR controls Stop, Pause, Play, Fast Forward over the internet RTSP Methods Signal Timing Diagram Protocol stack for a total streaming system SIP Quick Overview MPEG-4 File Format What s so good about MP4? File Format and Structure Hinting File Format Now we can code and stream video and audio Next we look at file formats for exchanging multimedia MPEG-1 & MPEG-2 content typically exchanged as files that represent a stream ready to be delivered Embedded absolute time stamps Fragmentation of media for some preferred transport Random Access could be difficult Need a new file format that can be used when Storing captured media Editing stored media Combining multiple files Delivering as File Download or Streaming A life-cycle file format COMP9519 Multimedia Systems Lecture 7 Slide 43 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 44 Reji Mathew

File Format MPEG-4 file format : MP4 life-cycle file format defined as part of MPEG-4 Based on Apple s QuickTime format ( mov files ) Due to MP4 s flexibility and extensibility Industry acceptance for exchange of MPEG-4 video and audio Basis for Motion JPEG 2000 file format Basis for 3GPP file format used in MMS Multimedia Messaging Service MP4 : Main File Format Concepts The Media data is stored separately from Meta data Media data : Audio, Video samples Metadata : Data describing the media Examples : timing info, number of bytes required for a frame Timing information specified by relative numbers (durations) rather than absolute numbers Allows editing to be easier eg insertion of a new frame Able to store media data distributed over several files Use URLs to point to media data stored at various locations COMP9519 Multimedia Systems Lecture 7 Slide 45 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 46 Reji Mathew MP4 : Main File Format Concepts The Media data is stored separately from Meta data Timing information specified by relative numbers (durations) rather than absolute numbers Able to store media data distributed over several files Locating media data by means of data offsets and length information Metadata tables mapping media sample number to location in a file Support streaming protocols through optional hint tracks Metadata information for packetization and header data Example hints for RTP streaming stored as a separate track COMP9519 Multimedia Systems Lecture 7 Slide 47 Reji Mathew MP4: File Structure The MP4 file format is composed of object-oriented structures called atoms. A unique tag and a length identify each atom. Metadata atoms : A hierarchy of metadata giving information such as index points, durations, and pointers to the media data. This collection of atoms is contained in an atom called the movie atom. Media Data Atoms : The media data is contained in one or more mdat or media data atoms, or could be located outside the MP4 file; referenced via URL s. COMP9519 Multimedia Systems Lecture 7 Slide 48 Reji Mathew

MP4: File Structure Metadata Hierarchy [5] Movie Atom moov MP4: File Structure Media data mdat Atom Contains interleaved, time ordered video and audio samples referred to as Access Units (AU) Several frames from the same track are often stored contiguously in chunks. Audio track trak Video track trak Hint track trak Other MPEG-4 track trak Media data mdat Sample table stbl Contains tables of sample timing and location information Frame 1 Frame 2 Frame 3 Frame 1 Frame 2 Frame 4 Video chunk Audio chunk COMP9519 Multimedia Systems Lecture 7 Slide 49 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 50 Reji Mathew MP4: File Structure Basic structure of MP4 file Metedata moov atom Each stream has a track trak atom which contain tables Tables with info on AU s Timing, size, location (offsets), synchronization points Tables used to point to AU in mdat MP4: File Structure [4] Media Data mdat atom Contains AU of audio and video Putting it together See next slide COMP9519 Multimedia Systems Lecture 7 Slide 51 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 52 Reji Mathew

MP4: Hinting MP4 file format is a streamable format, as opposed to a streaming format. The file format facilitates streaming without ever being streamed directly File format does NOT define an on-the-wire protocol, and is never actually streamed over a transmission medium. Metadata in the file known as hint tracks provide instructions, telling a server application how to deliver the media data over a particular delivery protocol. There can be multiple hint tracks for one presentation, describing how to deliver over various delivery protocols. MP4: Hinting Remember for streaming, need to format data into packets, with each packet containing a Header, and Samples or Media data Example RTP for MPEG-4 Hint Track Tables pointing to header information The header information is stored in mdat Example RTP header information Then points to appropriate fragment of sample data Example one video frame for an RTP packet See next slide COMP9519 Multimedia Systems Lecture 7 Slide 53 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 54 Reji Mathew MP4: Hinting [4] MP4: Hinting What to do to accommodate different transport protocols? Multiple hint tracks for one media Advantage to the media server? The server can be dum No knowledge of transport protocol required Header information already stored in the MP4 file No need to do media fragmentation Pointers to appropriate media fragments already created No knowledge of media codec required COMP9519 Multimedia Systems Lecture 7 Slide 55 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 56 Reji Mathew

MP4: Hinting No need to parse the coded media bitstream No need to perform fragmentation No need to create headers Low server complexity Media Server clip.mp4 retrieve References and Further Reading 1. IETF RFC 2327 SDP: Session Description Protocol 2. IETF RFC 2326 Real Time Streaming Protocol (RTSP) 3. IETF RFC 3261 SIP: Session Initiation Protocol 4. David Singer, QuickTime Engineering Quick Time etc. Introduction for UC Davis, Feb 2004 Client Generic server design 5. MPEG-4 Overview, ISO/IEC JTC1/SC29/WG11 N4668 www.chiariglione.org/mpeg/standards/mpeg-4/mpeg-4.htm COMP9519 Multimedia Systems Lecture 7 Slide 57 Reji Mathew COMP9519 Multimedia Systems Lecture 7 Slide 58 Reji Mathew