Scaling Adaptive Streaming Systems with Network Support IEEE ComSoc Distinguished Lectures Seattle, WA, Aug. 2017 Ali C. Begen, Ph.D. acbegen@ieee.org
A Bit about Myself Electrical engineering degree from Bilkent University (2001) Ph.D. degree from Georgia Tech (2006) Video delivery and multimedia communications Research, development and standards at Cisco (2007-2015) IPTV, content delivery, software clients Transport and distribution over IP networks Enterprise video Consulting at Networked Media since 2016 Assistant professor at OzU and IEEE Distinguished Lecturer since 2016 2
Agenda HTTP adaptive streaming recap Issues and deployment challenges Network-based and control plane solutions Areas to explore further 3
Internet (IP aka OTT) Video Essentials Reach Reach all connected devices Scale Enable live and on-demand delivery to the mass market Quality of Experience Provide TV-like consistent rich viewer experience Business Enable revenue generation thru paid content, subscriptions, targeted advertising, etc. Regulatory Satisfy regulations such as captioning, ratings and parental control 4
Creating Revenue Attracting Eye Balls High-End Content Hollywood movies, TV shows Sports Excellent Quality HD/3D/UHD audiovisual presentation w/o artifacts such as pixelization and rebuffering Fast startup, fast zapping and low glass-to-glass delay Usability Navigation, content discovery, battery consumption, trick modes, social network integration Service Flexibility Linear TV Time-shifted and on-demand services Reach Any device, any time 5
Video Delivery over HTTP Enables playback while still downloading Server sends the file as fast as possible Pseudo Streaming Enables seeking via media indexing Server paces transmission based on encoding rate Content is divided into short-duration chunks Enables live streaming and ad insertion Adaptive Streaming Multiple versions of the content are created Enables to adapt to network and device conditions Progressive Download Chunked Streaming 6
Adaptive Streaming over HTTP Multi-rate Encoder Packager Origin (HTTP) Server Server Storage Content Ingest (Live or Pre-captured) HTTP GET Request Response Media Buffer Streaming Client Decoding and Presentation 7
Adaptive Streaming over HTTP Adapt Video to Web Rather than Changing the Web Imitation of Streaming via Short Downloads Downloads small chunks to minimize bandwidth waste Enables to monitor consumption and track the streaming clients Adaptation to Dynamic Conditions and Device Capabilities Adapts to dynamic conditions in the Internet and home network Adapts to display resolution, CPU and memory resources of the streaming client à Facilitates any device, anywhere, anytime paradigm Improved Quality of Experience Enables faster start-up and seeking, and quicker buffer fills Reduces skips, freezes and stutters Use of HTTP Well-understood naming/addressing approach, and authentication/authorization infrastructure Provides easy traversal for all kinds of middleboxes (e.g., NATs, firewalls) Enables cloud access, leverages the existing (cheap) HTTP caching infrastructure 8
Dead, Surviving, Maturing and Newborn Technologies Move Adaptive Stream (Long gone) http://www.movenetworks.com Microsoft Smooth Streaming (Legacy) http://www.iis.net/expand/smoothstreaming Adobe Flash (Almost dead) http://www.adobe.com/products/flashplayer.html Adobe HTTP Dynamic Streaming (Legacy) http://www.adobe.com/products/httpdynamicstreaming Apple HTTP Live Streaming (The elephant in the room) http://tools.ietf.org/html/draft-pantos-http-live-streaming (Soon to be an RFC) MPEG DASH and CMAF (The standards) http://mpeg.chiariglione.org/standards/mpeg-dash http://mpeg.chiariglione.org/standards/mpeg-a/common-media-application-format 9
An Example Manifest Format List of Accessible Segments and Their Timings MPD Period id = 1 start = 0 s Period id = 2 start = 100 s Period id = 3 start = 300 s Period id = 4 start = 850 s Period id = 2 start = 100 s Adaptation Set 0 subtitle turkish Adaptation Set 1 video Adaptation Set 2 audio english Adaptation Set 3 audio italian Adaptation Set 1 BaseURL=http://abr.rocks.com/ Representation 1 Rate = 500 Kbps Representation 2 Rate = 1 Mbps Representation 3 Rate = 2 Mbps Representation 4 Rate = 3 Mbps Representation 3 Rate = 2 Mbps Resolution = 720p Segment Info Duration = 10 s Template: 3/$Number$.mp4 Segment Access Initialization Segment http://abr.rocks.com/3/0.mp4 Media Segment 1 start = 0 s http://abr.rocks.com/3/1.mp4 Media Segment 2 start = 10 s http://abr.rocks.com/3/2.mp4 Splicing of arbitrary content like ads Selection of components/tracks Selection of representations Well-defined media format Chunks with addresses and timing 10
Example Representations Vancouver 2010 Sochi 2014 FIFA 2014 Encoding Resolution Encoding Resolution Encoding Resolution Bitrate Bitrate Bitrate Rep. #1 3.45 Mbps 1280 x 720 Rep. #2 1.95 Mbps 848 x 480 Rep. #3 1.25 Mbps 640 x 360 Rep. #4 900 Kbps 512 x 288 Rep. #5 600 Kbps 400 x 224 Rep. #6 400 Kbps 312 x 176 Rep. #1 3.45 Mbps 1280 x 720 Rep. #2 2.2 Mbps 960 x 540 Rep. #3 1.4 Mbps 960 x 540 Rep. #4 900 Kbps 512 x 288 Rep. #5 600 Kbps 512 x 288 Rep. #6 400 Kbps 340 x 192 Rep. #7 200 Kbps 340 x 192 Rep. #1 3.45 Mbps 1280 x 720 Rep. #2 2.2 Mbps 1024 x 576 Rep. #3 1.4 Mbps 768 x 432 Rep. #4 950 Kbps 640 x 360 Rep. #5 600 Kbps 512 x 288 Rep. #6 400 Kbps 384 x 216 Rep. #7 250 Kbps 384 x 216 Rep. #8 150 Kbps 256 x 144 Source: Vertigo MIX10, Alex Zambelli s Streaming Media Blog, Akamai 11
Smart Clients - Client fetches and parses the manifest - Client uses the OS-provided HTTP stack (HTTP may run over TCP or QUIC) - Client uses the required decryption tools for the protected content HTTP Server Request Response (One can also multicast media segments) Client Client monitors and measures - Size of the playout buffer (both in bytes and seconds) - Chunk download times and throughput - Local resources (CPU, memory, window size, etc.) - Dropped frames Client performs adaptation Client measures and reports metrics for analytics 12
Tradeoffs in Adaptive Streaming Overall quality Quality stability Proximity to live edge Stalls Zapping/seeking time 13
Streaming over HTTP The Promise Leverage tried-and-true Web infrastructure for scaling Video is just ordinary Web content! Leverage tried-and-true TCP Congestion avoidance Reliability No special QoS for video It should all just work J 14
Does It Just Work? When streaming clients compete with other traffic, mostly yes When streaming clients compete with each other, we begin to see problems: The clients adaptation behaviors interact with each other The competing clients form an accidental distributed control-feedback system Unexpected behaviors will result in places like Multiple screens within a household ISP access and aggregation links Small cells in stadiums and malls 15
Demystifying a Streaming Client A Single Microsoft Smooth Streaming Client under a Controlled Environment 5 Bitrate (Mbps) 4 3 2 1 0 Steady State Periodic requests Buffer-filling State Back-to-back requests 0 50 100 150 200 250 300 350 400 450 500 Time (s) Available Bandwidth Requests Chunk Tput Average Tput Reading: An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP, ACM MMSys 2011 16
Typical Example 10 (Commercial) Streaming Clients Sharing a 10 Mbps Link Requested Bitrate (Kbps) 1400 1200 1000 800 600 400 200 0 0 100 200 300 400 500 Time (s) Client1 Client2 Client3 17
Selfishness Hurts Everyone Viewer Experience Statistics Source: Conviva Viewer Experience Report, 2015 18
Inner and Outer Control Loops Manifest Media HTTP Origin Module Request Response Manifest Resource Monitors Streaming Application TCP Sender HTTP Server Data / ACK TCP Receiver Streaming Client There could be multiple TCPs destined to the same or different servers 19
Streaming with Multiple TCP Connections Using multiple concurrent TCPs Should not be used to greedily get a larger share of the bandwidth Can help mitigate head-of-line blocking Allows fetching multiple (sub)segments in parallel Allows to quickly abandon a non-working connection without having to slow-start a new one System performance deteriorates very quickly if many clients adopt this approach without limiting the aggregated bandwidth consumption 20
Streaming with Multiple TCP Connections Similar to Driving on a Multi-Lane Highway TCP 1 TCP 2 TCP 3 Using multiple concurrent TCP connections helps mitigate the HOL blocking and allows to fetch multiple (sub-)segments in parallel 21
Understanding the Root Cause Two Competing Clients Depending on the timing of the ON periods: Unfairness, underutilization and/or instability may occur Clients may grossly overestimate their fair share of the available bandwidth Clients cannot figure out how much bandwidth to use until they use too much Reading: What happens when HTTP adaptive streaming players compete for bandwidth?, ACM NOSSDAV 2012 22
How to Solve the Issues? Fix the clients Use a better algorithm like PANDA or BOLA Enable a control plane E.g., 23009-5 SAND Get support from the network QoS in the core/edge, SDN, etc. 23
One Slide on Software Defined Networking (SDN) Control and data planes are decoupled, network intelligence and state are logically centralized, and the underlying network infrastructure is abstracted from the apps 24
SDN-Based Bitrate Adaptation Reading: SDNDASH: improving QoE of HTTP adaptive streaming using software defined networking, ACM MM 2016 25
Testing the Idea SDNdash.js is tested against BBA SARA QDASH Unmodified dash.js 26
Gains Compared to QDASH, BBA, SARA and dash.js Better fairness: 40%, 25%, 30% and 35% Better utilization: 12%, 13%, 5% and 52% Better video stability: 40%, 20%, 60% and 30% Less bitrate switches: 87%, 16%, 45% and 22% Higher and more consistent quality: 42%, 7%, 41% and 24% Better QoE: 35%, 25%, 30% and 28% 27
Control Points in the Ecosystem Content Provider Advertiser CDN Provider ISP Device/ App Consumer I want to make sure that my content is protected and looks awesome on any device I want to make sure that my ads are viewed, trackable and measurable I want to make sure that my servers are properly used and latency is low I want to control the QoE of all my customers, differentiate my own services, make $$$ from OTT services I want to make sure that my device/app provides the best possible video quality I want the best quality for minimal $ 28
A Control Plane Approach DASH Part 5: Server and Network Assisted DASH (SAND) DANE (Analytics) Media PER Messages Metrics and Status Messages PED Messages DANE (Origin) DANE Regular Network Element DASH Client DANE: DASH-assisting network element PER: Parameters for enhancing reception PED: Parameters for enhancing delivery 29
Four Major Areas of Further Exploration Content Preparation How to choose bitrate/resolution pairs to make up/downshifts least visible How to pick the segment durations How to achieve content aware streaming and not just content aware encoding Distribution and Delivery What information could the network provide to streaming clients How to achieve controlled unfairness MPTCP or QUIC better than TCP? What about multicast and HTTP/2? QoE Modeling and Client Design How to model streaming dynamics for different genres How to model the impact of faster zapping and trick modes on the QoE Understanding the impact of QoE on viewer engagement Analytics, Fault Isolation, Diagnostics Understanding the interaction of adaptive streaming with caching in CDNs Extracting actions based on real-time analytics Fixing issues faster and remotely 30
Thank You Visit ali.begen.net for the slides and more Television! Teacher, mother, secret lover Homer Simpson