A Comparative Case Study of HTTP Adaptive Streaming Algorithms in Mobile Networks

Transcription

1 Theodoros Karagkioules, Cyril Concolato, Dimitrios Tsilimantos and Stefan Valentin LTCI Telecom ParisTech Universite Paris-Saclay Mathematical and Algorithmic Sciences Lab France Research Center Huawei Technologies France SASU

2 Outline Introduction Motivation Scope Adaptive Bitrate Algorithms Throughput-based adaptation Buffer-based adaptation Time-based adaptation Experimental Framework Network profiles QoE Content Performance evaluation Experimental Validation Adaptability Instability Un-smoothness Overview Conclusion Page 2

3 Motivation Video made 60% of all mobile data traffic in 2016, predicted to increase to 78% by 2021 [1] This traffic is dominated by the Dynamic Adaptive Streaming over HTTP a.k.a MPEG-DASH [2] A HAS video stream is available in multiple qualities, each divided into segments or chunks Segments are downloaded using HTTP on top of TCP or UDP (in combination with QUIC) Video client controls bitrate by adapting (i) video quality and (ii) segment download time Bandwidth Quality High Medium Network conditions Medium Low Low Time Time Web server with various Request for Received segments at qualities per segment specific quality streaming client [1] Cisco, Visual Networking Index: Forecast and Methodology, , White Paper, Feb Time HAS policy not standardized Quality High [2] ISO/IEC, Dynamic adaptive streaming over HTTP (DASH), International Standard DIS , ISO/IEC, 2012 Page 3

4 Scope Extensive state of the art in literature Not only heuristics Research is shifting towards control theoretic approaches Classification of ABR algorithms Throughput-based Buffer-based Time-based Too many comparisons in literature? A Comparison per class is missing Also a comparison that takes the application type (VoD and Live streaming) is missing Performance evaluation of 5 algorithms, based or real field mobile network data Page 4

5 Outline Introduction Motivation Scope Adaptive Bitrate Algorithms Throughput-based adaptation Buffer-based adaptation Time-based adaptation Experimental Framework Network profiles QoE Content Performance evaluation Experimental Validation Adaptability Instability Un-smoothness Overview Conclusion Page 5

6 Throughput-based adaptation In throughput-based adaptation the decision on the bit-rate and the scheduling of every segment happens in four steps: 1. Throughput estimation 1. Where the TCP throughput is estimated through probes 2. Smoothing 1. Where the estimated TCP throughput is smoothed, to avoid estimation errors 3. Quantization 1. Where the smoothed estimated throughput is mapped to the discrete set of the video representations 4. Scheduling 1. The next segment request us scheduled once the inter-request time is estimated. In our study we have implemented two algorithms (Conventional and PANDA) as specified in [3] [3] Z. Li, X. Zhu, J. Gahm, R. Pan, H. Hu, A. C. Begen, and D. Oran Probe and Adapt: Rate Adaptation for HTTP Video Streaming At Scale. IEEE J. Sel. Areas Commun. 32 (April 2014). Page 6

7 Buffer based adaptation In buffer-based adaptation the decision on the bit-rate and the scheduling of every segment is made based on the instantaneous buffer level. According to BBA [4] the average segment size of each corresponding bitrate is mapped with the instantaneous buffer level, in a linear manner with two fixed points for the lowest and highest bit-rate. BOLA [5] deploys Lyapunov optimization in order to indicate the video bit-rate of each segment. The algorithm is designed to maximize a joint utility function that rewards an increase in the average quality and penalizes potential re-buffering occurrences. [4] Te-Yuan Huang, Ramesh Johari, Nick McKeown, Mathew Trunnell, and Mark Watson A Buffer-based Approach to Rate Adaptation: Evidence from alarge Video Streaming Service. In Proc. ACM SIGCOMM. [5] K. Spiteri, R. Urgaonkar, and R. K. Sitaraman BOLA: Near-optimal bitrate adaptation for online videos. In IEEE INFOCOM Page 7

8 Time-based adaptation Since the download time is considered as a higher level parameter than throughput, in this study, time-based adaptation is treated as a separate class of algorithms. ABMA+ [6] is an adaptation and buffer management algorithm, which selects the video representation based on the predicted probability of video stalling. The algorithm continuously estimates the segment download time and uses a precomputed play-out buffer map to select the maximum video representation, which guarantees smooth content play-out. The segment download time estimation is based on the same probing mechanism as the throughput-based method [6] A. Beben, P. Wisniewski, J. Mongay Batalla, and P. Krawiec ABMA+: Lightweight and Effecient Algorithm for HTTP Adaptive Streaming. In Proc. Int.ACM Conference on Multimedia Systems (MMSys). 2:1 2:11. Page 8

9 Outline Introduction Motivation Scope Adaptive Bitrate Algorithms Throughput-based adaptation Buffer-based adaptation Time-based adaptation Experimental Framework Network profiles QoE Content Performance evaluation Conclusion Experimental Validation Adaptability Instability Un-smoothness Overview Page 9

10 Network profiles Mobile networks are characterized by their intense bandwidth and coverage fluctuation To test the realistic performance of the ABR algorithms we used actual 3G traces [7]. 2 different network profiles where studied to emulate all coverage situations: Challenging (Underground) Normal (Bus) 1 Controlled (DASH-IF) scenario was used for the experimental validation [7] C. Griwodz P. Halvorsen H. Riiser, P. Vigmostad Commute Path Bandwidth Traces from 3G Networks: Analysis and Applications. Proc. of MMSys 5, 1 (March 2013), Page 10

11 QoE metrics (1/2) A unified QoE model is missing from the literature. In this study we considered 5 metrics that can be combined in 3 categories For a video of K segments and a set of video bitrates R ϵ {R 1,, R N }, the QoE is characterized as: Adaptability: Instability, which consists of the adaptation frequency (AF) and the adaptation amplitude (AA): Page 11

12 QoE metrics (2/2) Un-smoothness, which consists of the re-buffering duration and the rebuffering frequency. Where L is the length of the video, ω is the rebuffering threshold and β a binary operator regarding the occurrence of a rebuffering event. All the metrics are normalized and averaged over the complete set of traces. Page 12

13 Outline Introduction Motivation Scope Adaptive Bitrate Algorithms Throughput-based adaptation Buffer-based adaptation Time-based adaptation Experimental Framework Network profiles QoE Content Performance evaluation Experimental Validation Adaptability Instability Unsmoothness Overview Conclusion Page 13

14 Experimental Validation 5 ABR algorithms (Conventional, PANDA, BBA, BOLA, ABMA+) 12 mobile traces and 2 network profiles 5 Challenging scenarios 7 Normal scenarios 3 different movies (encoded according to the quintiles of the total throughput CDF) Big buck bunny (Animated content) Tears of Steel (High motion animated and non animated scenes) Red bull play streets (Sport) Both live and VoD is investigated in our study as B max is considered as an additional factor VoD (B max =4 segments) Live (B max =23 segments) 5 QoE metrics (Adaptability, Instability (2), Unsmoothness (2)) Page 14

15 Adaptability Buffer-based algorithms achieve higher adaptability in normal conditions. They are more successful, by design, in conserving high representation levels, even higher than the available throughput. Throughput-based and time-based algorithms show a slightly diminished ability to match the representation to the available average throughput This is due to the significant throughput variation that characterizes the selected network profiles. Page 15

16 Instability Buffer-based algorithms are about 40% more probable of making a quality switch when the buffer is small. BOLA is optimized to achieve a high bit-rate but, the stability aspect is not considered in the optimization, since it is addressed with a heuristic in a second phase. BBA has a pre-selected constant higher buffer threshold which makes the segment map less agile to throughput variation when the maximum buffer is small. Throughput-based and time-based algorithms appear to switch quality less often. Page 16

17 Unsmoothness Re-buffering probability is slightly higher in the cases of a small buffer (i.e. live streaming). Challenging profiles: buffer-based algorithms, along with PANDA, are slightly more probable to experience a re-buffering event. Normal scenarios: smooth streaming from almost all algorithms, due to the absence of long throughput outages of this profile. The duration (amplitude) of the re-buffering events, lasts about 25% of the video duration. (underground areas, may cause network outages, for 1/4 of the trace.) Page 17

18 General comments on the results No metric should be treated separately Only the combination of all metrics allows our comparison to be insightful. Overall, our results match those in literature In Table 2 we have gathered the best performing classes of algorithms, per QoE element. This table can serve as insight to the selection of the most appropriate algorithmic class, depending on the application parameters (live, VOD, etc.) and the commonly experienced network conditions. Page 18

19 Outline Introduction Motivation Scope Adaptive Bitrate Algorithms Throughput-based adaptation Buffer-based adaptation Time-based adaptation Experimental Framework Network profiles QoE Content Performance evaluation Conclusion Experimental Validation Adaptability Instability Un-smoothness Overview Page 19

20 Conclusion Per class comparison of 5 state-of-the-art ABR algorithms Studied Normal and challenging network situations LIVE and VoD Unified QoE framework missing Optimized parameterization is necessary Future work! Future work! The target buffer level is a critical classifier for the studied HAS algorithms. Overall, we believe that our findings provide valuable insight for the design and choice of HAS algorithms according to networks conditions and service requirements. Page 20

21 Thank you For any questions and/or comments/remarks please contact me at: Page 21