CS 414 Multimedia Systems Design Lecture 27 DASH (Dynamic Adaptive Streaming over HTTP) Klara Nahrstedt Spring 2012
Administrative MP2 posted MP2 Deadline April 7, Saturday, 5pm.
APPLICATION Internet Multimedia Protocol Stack DASH HTTP Synchronization Service SIP RTSP RSVP RTCP Media encaps (H.264, MPEG-4) RTP Layer 5 (Session) KERNEL TCP DCCP IP Version 4, IP Version 6 UDP Layer 4 (Transport) Layer 3 (Network) AAL3/4 AAL5 MPLS Layer 2 (Link/MAC) ATM/Fiber Optics Ethernet/WiFi
Problems with Internet Video Video not accessible Video Behind firewall Plugins not available Bandwidth not sufficient Wrong and non-trust device Wrong format Low Quality of Experience Long start-up latency Frequent rebuffering Low playback quality No lip-sync
Mobile Video Streaming Challenges Mobile Internet use is expanding dramatically Video traffic is growing exponentially Challenges: Mobile users expect high quality video experience Network operators need to offer quality experience affordably
Growth of Mobile Internet/Video Thomas Stockhammer, Qualcomm, DASH Design Principles and Standards, MMSys 2011
Adaptive Streaming One approach to tackle problems/challenges
Adaptive Streaming Concept Adaptive Streaming technologies enable Optimal streaming video viewing experience for diverse range of devices over broad set of connection speeds Adaptive streaming technologies share Production of multiple files from the same source file to distribute to viewers watching on different powered devices via different connection speeds Distribution of files adaptively, changing stream that is delivered to adapt to changes in effective throughput and available CPU cycles on playback stations Transparent operation to the user so that the viewer clicks one button and all streams switch/adapt behind the scenes.
Adaptive Streaming One Approach of Adaptive Streaming 1. Server sends first the high important video information (e.g., I frames) And after the high importance video information is sent, lower importance video information follows (e.g., P and B frames) if bandwidth and time allows Second Approach of Adaptive Streaming 1. Server sends with high quality part of the frame and only progressively,if bandwidth and time allow, it sends the rest of the frame information Third Approach of Adaptive Streaming 1. At server video is encoded in multiple bitrates and depending on the device Bandwidth it adjusts at what rate Other approaches exist Source: http://www.dicomdistribution.com/adaptive%20streaming.html
Standardization History Thomas Stockhammer, Qualcomm, DASH Design Principles and Standards, Presentation at MMSys 2011
Adaptive HTTP Streaming System (Protocol) Server Can be standard web server Media segment can be prepared in-line or offline Client Sends series of HTTP GET segment requests and receives segments Performs rate adaptation before sending a new GET segment request
Client-centric approach Client has best view of network conditions No session state in network Redundancy Scalability Faster innovation and experimentation But, relies on client for operational metrics Only client knows what really happens
Terms and Definitions of Adaptive HTTP Streaming Need Media Presentation Description (MDP) which provides metadata For requesting (GET request) media segments For rate adaptation purpose Segment which may include media data or metadata to decode Need DASH
DASH Dynamic Adaptive Streaming over HTTP Dash is NOT System, protocol, presentation, codec, interactivity What is DASH Enabler which provides formats to enable efficient and highquality delivery of streaming services over the Internet Component of end-to-end service Enabler to reuse existing technologies (containers, DRM (Digital Rights Management), codecs) Enabler for deployment on top of HTTP-CDNs Enabler for very high user experience (low start-up, no rebuffering) Provides simple inter-operability points (profiles)
DASH Client Thomas Stockhammer, Qualcomm, DASH Design Principles and Standards, Presentation at MMSys 2011
Information Classification DASH uses MPD (Media Presentation Descriptor) and Index Information as metadata for DASH Access Client Initialization and Media Segments for Media Engine Reuse of existing container format Source: MMSys 11
Media Presentation Data Model MDP - description of accessible segments and corresponding timing Source: Stockhammer, Qualcomm, DASH Design Principles and Standards, Presentation at MMSys 2011
MDP Information Includes redundant information of media streams to initially select or reject groups or representations Includes access and timing information Content addressing via HTTP-URLs Byte range for each accessible segment Segment availability start and end time in wall-clock time Approximate media start time and duration Instructions on starting playout (for live service) Includes switching relations across representations
Media Segments (1) Contain information to map segment into media presentation timeline for switching and synchronous presentation with other representations Can be short (~ 1-10seconds) Can be long (~10sec 2 hours)
Media Segments (2) Media segment duration Short duration Long duration advantages Commonality with live high switching granularity on segment level - Small number of files - Small number of URLs - High switching granularity - Flexible request sizes - Improved cache performance disadvantages - Large number of files - Large number of URLs - Fixed request size - Switching granularity on segment level - Need for segment index - Difference from live
Segment Indexing Provides information in ISO box structure on Accessible units of data (e.g., frames) in media segment Byte range in segments (easy access through HTTP GET) Accurate presentation duration (seamless switching) Presence of representation access positions Provides compact bitrate-over-time to client Can be used for intelligent request schedule Generic data structure Hierarchical structuring for efficient access
Media Segment with Segment Index Source: Stockhammer, Qualcomm, DASH Design Principles and Standards, Presentation at MMSys 2011
DASH Capabilities Enables live, on-demand and time-shift services Allows independency of request sizes and segment sizes Allows segment formats that are ISO base media formats ISO BMFF or MPEG-2 TS ISO BMFF -.. File Format (extensions) MPEG-2 TS -.. Transport Stream (extensions) Guidelines for integrating any other format Codec independent Supports server/client component synchronization (e.g., separate and multiplexed AV) Enables targeted ad insertion Provides definition of quality metrics Enables content descriptors for protection, accessibility, rating,
Composition of Media Presentation (CMP) Consider layer on top of MDP which specifies initial user and device options Reasons: Size of MPD expands very quickly (different camera views/angles, subtitles, audio languages, ) Device pre-configuration (codec, resolution ) User pre-configuration due to her preferences (camera angle, subtitle, ) Flexibility and compatibility with existing repository formats
CMP Protocol Sequence (1) At Client Request CMP from Server (1) Once CMP received, check and Configure CMP at client if needed (3) Request for MDP that fulfills Requirements (4) Start standard DASH complaint steps Source: Muller et al, MMSys 11
DASH featuring Session Mobility (Problem Description) Problem: User runs DASH streaming session on laptop and decides to switch to Another mobile device Source: Muller et al, MMSys 11
Session Mobility Protocol (1) Initiate session transfer (2) Digital Item Adaptation (DIA) Starts and preserves current state of Digital Item (segment) (3) Transfer Context digital item To selected device (4) Download CMP (5) Reconfigure device (6) Continue with DASH-compatible protocol Source: Muller et al. MMSys 2011
Adaptive streaming in practice Source: Watson, MMSys 11
Adaptive streaming in practice
MPEG DASH Summary Is rich and simple at the same time Supports both un-chunked and chunked Supports both separate and combined AV Index formats for efficient byte range operation ISO base media file format w/common encryption Many useful stream and track annotations Currently best candidate for open standard for adaptive streaming
Conclusion Metadata used by DASH and session mobility approach use MPEG-21 international standard Facilitates MPEG-21 Digital Item Leverages existing media repositories such as UPnP One possible implementation of DASH http://www-itec.uni-klu.ac.at/dash