Anatomy of a DASH Client. Ali C. Begen, Ph.D.

Transcription

1 Anatomy of a DASH Client Ali C. Begen, Ph.D.

2 Video Delivery over HTTP Enables playback while still downloading Server sends the file as fast as possible Pseudo Streaming Enables seeking via media indexing Server paces transmission based on encoding rate Content is divided into short-duration chunks Enables live streaming and ad insertion Adaptive Streaming Multiple versions of the content are created Enables to adapt to network and device conditions Progressive Download Chunked Streaming 2

3 Dead, Surviving, Maturing and Newborn Technologies Move Adaptive Stream (Long gone) Microsoft Smooth Streaming (Legacy) Adobe Flash (Almost dead) Adobe HTTP Dynamic Streaming (Legacy) Apple HTTP Live Streaming (The elephant in the room) (Soon to be an RFC) MPEG DASH and CMAF (The standards) 3

4 Adaptive Streaming over HTTP Multi-rate Encoder Packager Origin (HTTP) Server Server Storage Content Ingest (Live or Pre-captured) HTTP GET Request Response Media Buffer Streaming Client Decoding and Presentation 4

5 Scope of MPEG DASH Shown in Red MPD... MPD... MPD... MPD Transport Control Engine MPD Parser Segment Parser Media Engines HTTP 1.1 HTTP Client HTTP Server DASH Client 5

6 DASH Manifest Media Presentation Description (MPD) List of Accessible Segments and Their Timings MPD Period id = 1 start = 0 s Period id = 2 start = 100 s Period id = 3 start = 300 s Period id = 4 start = 850 s Period id = 2 start = 100 s Adaptation Set 0 subtitle turkish Adaptation Set 1 video Adaptation Set 2 audio english Adaptation Set 3 audio italian Adaptation Set 1 BaseURL= Representation 1 Rate = 500 Kbps Representation 2 Rate = 1 Mbps Representation 3 Rate = 2 Mbps Representation 4 Rate = 3 Mbps Representation 3 Rate = 2 Mbps Resolution = 720p Segment Info Duration = 10 s Template: 3/$Number$.mp4 Segment Access Initialization Segment Media Segment 1 start = 0 s Media Segment 2 start = 10 s Splicing of arbitrary content like ads Selection of components/tracks Selection of representations Well-defined media format Chunks with addresses and timing 6

7 Smart Clients - Client fetches and parses the manifest - Client uses the OS-provided HTTP stack (HTTP may run over TCP or QUIC) - Client uses the required decryption tools for the protected content HTTP Server Request Response (One can also multicast media segments) Client Client monitors and measures - Size of the playout buffer (both in bytes and seconds) - Chunk download times and throughput - Local resources (CPU, memory, window size, etc.) - Dropped frames Client performs adaptation Client measures and reports metrics for analytics 7

8 Tradeoffs in Adaptive Streaming Overall quality Quality stability Proximity to live edge Stalls Zapping/seeking time 8

9 Types of Browser-Based Playback Type 1: Minimum control architectural model for HTML5 support of adaptive streaming where manifest and heuristics are managed by the user agent Type 2: Adaptation control architectural model for HTML5 support of adaptive streaming providing script manageable features Type 3: Full media control architectural model for HTML5 support of adaptive streaming allowing script to explicitly send the media segments (MSE + EME) Source: CTA WAVE 9

10 MSE Support in Web Browsers Source: 10

11 DRM Support on Desktop Browsers Browser OS EME/CDM Flash Player NPAPI/ Silverlight 5 Chrome Win Widevine (Yes) No OS X (Yes) No Linux (Yes) No Firefox Win Widevine Yes No OS X Yes No Linux Yes No Safari > OS X Yosemite (FairPlay) Yes Yes < OS X Yosemite No Yes Yes IE/Edge < Win 7 No Yes Yes Win 10 PlayReady Yes No 11

12 DRM Support on Mobile Platforms Platform Browser Native App / DRM ios No MSE/EME yet HLS (AES-128 CBC) via <video> Native SDK and WebView: FairPlay Android MSE/EME Native + Android SDK (MediaDRM APIs) (Widevine + OMA v2) ExoPlayer Native + WebView with Widevine Windows 10 MSE/EME WebViews/Hosted Web Apps with PlayReady 12

13 Streaming over HTTP The Promise Leverage tried-and-true Web infrastructure for scaling Video is just ordinary Web content! Leverage tried-and-true TCP Congestion avoidance Reliability No special QoS for video It should all just work J 13

14 Does It Just Work? When a streaming client competes with other types of traffic, mostly yes When streaming clients compete with each other, we begin to see problems: The clients adaptation behaviors interact with each other The competing clients form an accidental distributed control-feedback system Unexpected behaviors will result in places like Multiple screens within a household ISP access and aggregation links Small cells in stadiums and malls 14

15 Demystifying a Streaming Client A Single Microsoft Smooth Streaming Client under a Controlled Environment 5 Bitrate (Mbps) Steady State Periodic requests Buffer-filling State Back-to-back requests Time (s) Available Bandwidth Requests Chunk Tput Average Tput Reading: An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP, ACM MMSys

16 A Simple Test Scenario 10 (Commercial) Streaming Clients Sharing a 10 Mbps Link Requested Bitrate (Kbps) Time (s) Client1 Client2 Client3 16

17 Selfishness Hurts Everyone Viewer Experience Statistics Source: Conviva Viewer Experience Report,

18 Inner and Outer Control Loops Manifest Media HTTP Origin Module Request Response Manifest Resource Monitors Streaming Application TCP Sender HTTP Server Data / ACK TCP Receiver Streaming Client There could be multiple TCPs destined to the same or different servers 18

19 Streaming with Multiple TCP Connections Using multiple concurrent TCPs Should not be used to greedily get a larger share of the bandwidth Can help mitigate head-of-line blocking Allows fetching multiple (sub)segments in parallel Allows to quickly abandon a non-working connection without having to slow-start a new one System performance deteriorates very quickly if many clients adopt this approach without limiting the aggregated bandwidth consumption 19

20 Understanding the Root Cause Two Competing Clients Depending on the timing of the ON periods: Unfairness, underutilization and/or instability may occur Clients may grossly overestimate their fair share of the available bandwidth Clients cannot figure out how much bandwidth to use until they use too much Reading: What happens when HTTP adaptive streaming players compete for bandwidth?, ACM NOSSDAV

21 How to Solve the Issues? Fix the clients (Use a better algorithm like PANDA or BOLA) Solution Approaches Get support from the network (QoS in the core/edge, SDN, etc.) Enable a control plane (E.g., SAND) 21

22 Commercial OTT Issues Content format issues Each asset is copied to multiple media formats different video codecs different audio codecs different (regional) frame rates Cost to content creators and distributors Inefficiencies in CDNs and higher storage costs Platform issues Lack of consistent app behavior across platforms Varying video features, APIs and semantics across platforms Playback issues Codec incompatibility Partial profile support Switching bitrate glitches Audio discontinuities Ad splicing problems Long-term playback instability Request protocol deficiencies Memory problems, CPU weaknesses Scaling (display) issues Variable HDR support Unknown capabilities 22

23 That is Why We Need Analytics OS Vendor: Your Internet connection must be bad Service Provider: Your video or CDN provider must be slow Device/App Vendor: It must be the OS Video/CDN Provider: Your home network must be slow Consumer: The device or the app is slow 23

24 Netflix s Content-Based Encoding Method 24

25 Segments Have Different Complexities Quality Video Segment #1 Consistent Quality Video Segment #2 Equal Bitrate Allocation among Segments Bitrate 25

26 Adaptation Feature Does Not Deliver Consistent Quality Guidelines Limited Bitrate Variability to (Mostly) 10% So Far Segment Size Large variation in quality Segment Quality S Small variation in encoding bitrate Easy Moderate Easy Easy Easy Difficult Difficult Difficult Moderate Moderate Moderate Moderate Q CBR Time (s) If there is something worse than having to watch a video at a lousy quality, it is to watch that video with varying quality 26

27 What if We Encode in a More Subtle Fashion? Segment Size Small variation in quality Segment Quality S Q VBR Easy Moderate Easy Easy Difficult Difficult Difficult Moderate Moderate Moderate Moderate Large variation in Easy encoding bitrate Time (s) While we spend the same total amount of bits, we not only increase average quality but also reduce quality variation Note: HLS authoring spec for ATV allows 2x capping rate for VoD. For linear content, variability is limited to 10-25% range. 27

28 Generating VBR-encoded segments is easy, but streaming them is not! Content Aware Encoding Content Aware Streaming 28

29 Multiple Representations Naturally Enable Cherry-Picking 4 Mbps 3 Mbps 2 Mbps 1 Mbps Mbps Network HTTP Server Streaming Client Segment Size Segment Quality Q VBR S Reading: Streaming video over HTTP with consistent quality, ACM MMSys 2014 Time (s) 29

30 Dimensions In-Stream vs. Across-Streams Same principle applies to both: In-stream: Temporal bit shifting between segments Across-streams: Bit shifting between streams sharing a bottleneck link Quality Video Segment #1 Quality Stream 1 (News) Consistent Quality Video Segment #2 Consistent Quality Stream 2 (Sports) Equal Bitrate Allocation among Segments Bitrate Equal Bitrate Allocation among Streams Bitrate Reading: Spending quality time with the Web video, IEEE Internet Comput.,

31 Any Questions?