Multi-Video Streaming with DASH

Transcription

1 Linköping University Department of Computer Science Bachelor thesis, 16 ECTS Datateknik 217 LIU-IDA/LITH-EX-G--17/71--SE Multi-Video Streaming with DASH Multi-video streaming med DASH Sebastian Andersson Markus Johansson Supervisor : Niklas Carlsson Examiner : Nahid Shahmehri Linköpings universitet SE Linköping ,

2 Upphovsrätt Detta dokument hålls tillgängligt på Internet eller dess framtida ersättare under 25 år från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida Copyright The publishers will keep this document online on the Internet or its possible replacement for a period of 25 years starting from the date of publication barring exceptional circumstances. The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its www home page: Sebastian Andersson and Markus Johansson

3 Students in the 5 year Information Technology program complete a semester-long software development project during their sixth semester (third year). The project is completed in mid-sized groups, and the students implement a mobile application intended to be used in a multi-actor setting, currently a search and rescue scenario. In parallel they study several topics relevant to the technical and ethical considerations in the project. The project culminates by demonstrating a working product and a written report documenting the results of the practical development process including requirements elicitation. During the final stage of the semester, students create small groups and specialise in one topic, resulting in a bachelor thesis. The current report represents the results obtained during this specialisation work. Hence, the thesis should be viewed as part of a larger body of work required to pass the semester, including the conditions and requirements for a bachelor thesis.

4 Abstract Multi-video streaming allows the viewer to interact with the stream by choosing and switching between several different camera angles to view the stream from. For this report we implement and evaluate adaptive multi-video streaming with dash.js. With the help of dash.js and self-made additions, multiple parallel video streams which are synchronized in time are implemented to provide a good user experience with smooth switching between streams. These streams will be delivered according to dash.js own HTTP-based Adaptive Streaming algorithms to achieve adaptive streams for varying conditions. In order to optimize the usage of the available bandwidth in terms of video quality in a multi-video environment we implement probabilities of camera switching to adapt qualities and allocated bandwidth of streams. By utilizing the functions of dash.js we create two prefetching policies and analyze these results together with the standard non-prefetch dash.js implementation in a multi-view video environment. Our results present the improvements in terms of stalling with a prefetch implementation and the possibility of a good policy to further optimize a multi-view video implementation in terms of stalling, quality and bandwidth usage. Evaluation of dash.js compatibility for a multi-view video environment is also discussed where pros and cons of dash.js in their current state are presented.

5 Contents Abstract Acknowledgments Contents List of Figures List of Tables iv v v vi vii 1 Introduction 1 2 Background Adaptive Streaming Multi-View Video Streaming DASH and dash.js Related Works 6 4 System Design & Implementation Implementation Overview Testbed and Evaluation Framework Detailed System Changes Evaluation Methodology Testbed and Evaluation Framework Baseline Policies Bandwidth Profiles and Test Scenarios Results Example Run Results Based on Means Impact of Available Bandwidth Proof of Concept: On Switch Prefetch Delay, Buffer Level and Segment Duration Discussion Results Method The Work in a Wider Context Conclusion 29 Bibliography 3 v

6 List of Figures 2.1 Illustration of the segmentation of media Example of a manifest, with some of the required properties specified Throttle techniques Testbed Timeline of tests for default bandwidth profile Flowchart of the track switch Theoretical behavior of delay Policy illustration Proof of concept detailed illustration Proof of concept graphic illustration Buffer occupancy of the three implementations Quality occupation for the three implementations Average buffer occupancy Average quality index Means based on bandwidth Average buffer occupancy for on switch prefetch Delay and buffer length Delay and segment time left vi

7 List of Tables 4.1 Video table Notion table Buffer standard deviation Stall time Quality standard deviation Data for on switch prefetch vii

8 1 Introduction Today streaming of multimedia is more popular than ever and it is still growing. Not only are we streaming media into our homes, but it has also become a part of our daily life outside of it and in many cases this is done while we are mobile. A mobile connection is on the other hand often less stable than the ones in our homes, which creates new problems for the possibility of streaming. Varying connections have led to the development of adaptive streaming solutions, which means that the quality of the video varies to match the current network conditions. One standard for adaptive streaming is DASH 1, which was developed by the Moving Picture Experts Group 2. This thesis presents a dash.js implementation of an adaptive multi-video player, as well as tests of the implementation s performance. Dash.js is an implementation of DASH, built upon JavaScript and was developed as an initiative by the DASH Industry Forum. Our player provides a reliable way to prefetch and switch between different streams of video. The implementation could in practice be used for something like switching between cameras in a film or between cameras used in video surveillance. With the results of the multi-video prefetching implementation we want to answer the following research questions: How good is the compability with multi-video for dash.js in its current state? Does prefetching provide beneficial results for multi-video in dash.js? Can the prefetching implementation be improved through different policies? Prior work [1] has demonstrated the concept of stream bundles, with the use of an implementation based on OSMF 3, to evaluate the performance of an optimized prefetching framework. In this thesis, we incorporate these ideas into dash.js. In particular, we present how an implementation can be made, as well as how different streams can be prioritized in order to provide a smooth transition between streams. Streams that the user is likely to switch to will therefore have a higher quality for its prefetching than the ones that are less likely to 1 DASH: Dynamic Adaptive Streaming over HTTP, also known as MPEG-DASH 2 Website: 3 OSMF: Open Source Media Framework. A framework for Adobe Flash Player and Adobe AIR, created by Adobe, for developing media players 1

9 be switched to. This can be done both by having a probability based on the user s previous switches, the closeness of the stream/camera or a combination of both. This thesis begins with a description of the fundamental parts of the dash.js-player, followed by a description of how the implementation was made. We then present different methods for prioritizing different streams. Finally, the last part will focus on the testing and evaluation of the implementation, before completing the thesis with a discussion and conclusions. 2

10 2 Background This chapter provides an overview of the concepts of adaptive streaming, multi-view video streaming, and DASH. 2.1 Adaptive Streaming Adaptive streaming provides the ability to adapt the quality of a multimedia-stream to the users current bandwidth conditions. This means that if the bandwidth conditions are good, meaning a high throughput is provided, the media player will fetch and play content of higher quality than if the conditions were poor. The player therefore makes an estimation of the current bandwidth in order to make a decision about which quality it is going to fetch. The media itself has been divided into multiple parts, called segments, that can be fetched by the player during playback. Each segment contains a fixed amount of data, for example 4 seconds of video content, and are available in multiple qualities for the player to choose from. Figure 2.1 illustrates how media is represented as segments on the server. The length of a segment can be varied by the provider, but it is in general kept at around 4 seconds for DASH implementations 1. The player fetches new segments before the current one runs out and stash future content in a so called buffer. The buffer enables the player to continue playing already fetched segments in case the bandwidth conditions become poor, and thereby also provides the player with time to lower the quality for the future segments. 2.2 Multi-View Video Streaming The concept around multi-video streaming is that the user is able to swap between different videos during playback. This can for example be swapping between cameras at a concert or when watching surveillance footage. This requires the server to not only store one segmented video, but one for each available video. This also means that the player has to keep track of the videos that are not being played in order to be ready to switch to another when the user wants to. If the player has not prefetched the content in advance, the switch will be slow and 1 Segment lengths are discussed and evaluated by Bitmovin: 3

11 2.3. DASH and dash.js segment Kbit/s segment Kbit/s... segment Kbit/s Quality segment Kbit/s segment Kbit/s segment Kbit/s segment Kbit/s segment Kbit/s segment Kbit/s Time Figure 2.1: Illustration of the segmentation of media most likely result in a stall since the player has to gather the required segment in order to start the playback of the new video. An approach to improve the time it takes to switch between different videos is to prefetch the non-played videos during playback[1]. This enables the player to quickly swap between them, since it already got the segment stored in its cache. The downside of this approach is that a lot of fetched data will not be played and therefore puts unnecessary load on the network. This might lead to cases when the current video can not be fetched at a high quality because the prefetching takes up the bandwidth that would be required to do so. This behavior can however be minimized by only allowing the player to fetch lower quality content for the non-played videos, and therefore prioritize the currently played one. The fetching for the non-played videos can also be prioritized by letting it fetch sightly higher quality content from the videos that the user is likely to swap to, for example cameras that are geographically nearby the current one. 2.3 DASH and dash.js DASH 2 is one of the current leading standards for adaptive streaming, and is based upon fetching data over the HTTP-protocol. Dash.js is an implementation of the standard and provides a fully featured player for both fetching and playing adaptive streaming-based content. When the player is initialized, it first fetches a manifest (called a MPD-file 3 ), that specifies all needed information about the content that it will be playing. The manifest provides a XMLformatted list of available video- and audio content, different quality levels for the content, as well as where to find the segments on the server. The paths to the segments can be specified as a template with variables that is inserted by the player in order to construct a path to the desired segment. An element that goes with the path-template is a base-url, that provides the initial part of the path string like the domain. $RepresentationID$ i-$Number$.m4s is an example of a template with the variables RepresentationID and Number, that are replaced by the player to construct a valid URL for a segment. RepresentationID is replaced with the string that represents the desired quality for the segment, while Number specifies which of the segments that the player wants to fetch. But, it is also possible, if one desire, to specify absolute paths for every segment. Figure 2.2 illustrates how a manifest can be constructed to provide one video- and one audio file, both containing one quality each. In the manifest there are groups called AdaptionSets that specifies a group or set of content. This can be a video file and its qualities, an audio file or a text file that is used for subtitles. During the time of writing, dash.js supports hassle free switching between different audio tracks. This can for example be used for swapping between tracks that contains different languages. Dash.js does provide a similar functionality for switching between videotracks. But in its standard form it does not download the initialization file 4, for the video that 2 DASH Industry Forum s website: 3 MPD format for DASH: 4 Initialization file: File that contains information required to initialize playback of a corresponding video file 4

12 2.3. DASH and dash.js <MPD> <BaseURL>...</BaseURL> <Period> <AdaptionSet mimetype="video/mp4"> <SegmentTemplate media="$representationid$ i-$number$.m4s"/> <Representation id="v6_257" bandwidth="43"/> </AdaptionSet> <AdaptionSet mimetype="audio/mp4"> <SegmentTemplate media="$representationid$ i-$number$.m4s"/> <Representation id="v4_258" bandwidth="138"/> </AdaptionSet> </Period> </MPD> Figure 2.2: Example of a manifest, with some of the required properties specified the user switched to, which makes the video unplayable. In its standard form, it also lacks the ability to prefetch non-played videos. dash.js and Track Switching An important note on dash.js in the concept of multi-video is the track switch. By default dash.js already implements a strategy to avoid stalls by delaying track switches. It does this by always finish playing the currently played segment, and sometimes more segments, before it performs the track switch. This means that the player delays the actual track switch to be able to download the new segments before the currently played track ends. This conclusion is based both on first hand experience, but also from answers regarding this delay from DASH personel 5. While this provides a certain stall avoidance in a no-prefetch implementation, it also reduces the impact of a prefetch approach. This is an important aspect to the evaluation of the results and will be discussed further later in the thesis. 5 Issue and answer from DASH Github regarding switching delay: Forum/dash.js/issues/1639 5

13 3 Related Works Multi-view video streaming is a topic that gets more and more interesting as technology and especially network capabilities increases. There are various approaches to multi-view video streaming. Earlier research primary discusses the encoding of multi-view videos, MVC, to improve the rate-distortion performance [16], [6] or encode multiple streams into a single stream [4]. Distributed multi-view video coding (DMVC) is another technique in which inter-camera communication is avoided and the large computing complexity is moved from encoder to decoder to improve the coding performance [7]. To further improve the compression of DMVC to compete with traditional MVC, depth-based DMVC has been studied by Qing et al. [15]. In the context of depth-based images, 3D multi-view video is another interesting topic gaining a lot of attention. The network load of such heavy data traffic is addressed and reduced through synthesized viewpoints by Lin et al. [13]. Such implementations, DMVC and synthesized viewpoints, does however transfer much of the load onto the client. Especially since the study regarding synthesized views had mobile devices in mind, this will most likely have a significant impact on battery and processing power for such a device. Alternatives to further improve the performance of 3D multi-view video in regards to data traffic is examined by studying adaptive 3D multi-view video streaming over P2P networks to maximize link utilization [14]. Other areas that utilize encoding techniques to improve multi-view video performance is 36-video and Internet of Things (IoT). Propositions on how to improve the architecture on multi-view video in a 36-video and IoT environment has been presented by Cen et al. [3]. While the encoding can greatly contribute to the efficiency of a multi-view video in terms of bandwidth and quality, our study aims to investigate the impact of a HAS-based prefetch implementation. Research by Kito et al. [1], investigates progressive download of multiview video with the aim of reducing stall time and video traffic through different policies compared to previous schemes. Through prefetching, the stalling is reduced and with probability assumptions, certain cameras are prioritzed, resulting in a reduced video traffic compared to unintelligent prefetching of all cameras. Personalization for multi-cast video is discussed and presented in Carlsson et al. [2]. Research presented by Carlsson et al. [1] further studies the potential of different policies for various prefetching schemes with the OSMF media player. Through probability and weighting of stalls the QoE is evaluated based on stalls and playback quality. Another area 6

14 that can utilize prefetching to prevent stalls is branched video, where the user can select their own path through a video [12], [11]. Since high variations in quality and stalling has a major impact on the QoE [8], [9], [5], the importance of seamless track switching in a multi-view video environment is very significant. Similar to these papers, analysis of prefetching policies to minimize stall time while maintaining a competitive quality is the goal of this paper, but by using the DASH framework. With the ambition of DASH to become the standard framework for media players, we think it is interesting to evaluate the current functionality and options provided by dash.js to implement an adaptive multi-view video stream. At the start of this project, no known studies had been made on a HAS-based multi-view video with dash.js. However, a recent publication [17] discusses the avoidance of stall delay with the DASH framework in a multi-view video environment using parallel streaming and server push. However, while the study is DASH-based, it has a system built up around it with custom buffer controllers and more. In contrast to this, our study aims to put more weight on dash.js in its current state to evaluate the compatibility of a prefetching multi-view video implementation within dash.js. 7

15 4 System Design & Implementation This chapter explains our implementation of the multi-video media player within dash.js both as an overview and a more in-depth description of our changes and addition of code. A policy for prioritizing prefetch bandwidth to adjacent tracks of the playback track and a proof of concept implementation utilizing dash.js s delay is also presented. It is worth noting that at the time of writing dash.js consists of about 4 lines of code, which makes it practically impossible for us to have a deeper understanding of all the used mechanics. Because of this fact some descriptions of how dash.js handles certain things are described based on our first hand experience of the player and might thereby differ from how the mechanics actually works. 4.1 Implementation Overview The implementation of a dash.js-based multi-video player began with configuring the already available track-switching functionality to be able to swap video-tracks during playback. It also required the switch to make a request for the corresponding initialization-file. This enabled the player to swap between two or more videos, but without pre-fetching the content. When the player was able to switch tracks, a cache was created in JavaScript in order to cache all HTTP-responses that the player received. This enabled the player to use the cache instead of sending a request if it already had sent one at a previous point in time. The cache was constructed around a key-value store, where the key-field that represented the request- URL and a value-field containing the HTTP-response. The player could then insert, check if data exist and extract data from the cache by using public methods. The cache was then tested by swapping between two tracks and observing if the new track had to fetch data that were already prefetched. For the test we setup a website that contained a dash.js-based video element hosted on a web server, written in Go, which served the site containing the actual player and the media content. Once the cache was tested prefetch implementation was developed. The implementation built upon the idea that once the player fetched a segment for the main video, it should also round-robin fetch the segment with the same number from all other videos, but at a lower quality [1]. The result could then be evaluated by observing if the player was making a HTTP-request or if it was using the cache when a switch of tracks was made. 8

16 4.2. Testbed and Evaluation Framework 4.2 Testbed and Evaluation Framework Conventional 12 Mbit/s Used 4 Mbit/s 4 Mbit/s 4 Mbit/s Figure 4.1: Throttle techniques Firefox Web server Dash.js SegmentCache.js Debug.js Test script Figure 4.2: Testbed HTTP TCP In order to perform the required tests, a testbed was developed for creating a consistent and easy to use environment for the data gathering. Firefox 53. was used as the browser for playing and gathering data from the dash.js player. The web server that was used to deliver the segments to dash.js was based on the primary web server library for the programming language Go. To provide reliable results, tests were made with two different throttling approaches in order to evaluate possible deviations. Go-based Throttling The majority of tests were made with the Go-based throttling. To control the bandwidth for fragment transfers, the requested file was read and sent at the currently selected bandwidth divided by the number of active transmissions. This means that neither headers or the requests sent by the browser were affected by the throttled bandwidth. The divided transfer rate ensures equality in terms of speed of data transfers. The reason for why this implementation was used, is that it was the one that gave us the most consistent results with the least technical issues. A more conventional solution would be to create a virtual link between the server and the user and then set the transfer rate for the link. The throttling that was used is illustrated in Figure 4.1, together with an example of a conventional solution, a user (left) and a web server (right). The Go-implementation only throttled the transfer of the segments, which almost removes all effects created by transfers of HTML- and JavaScript files in the page initialization-phase. Dummynet-based Throttling To simulate a more realistic network scenario, an alternative and more conventional throttling method was made by storing segments on a virtual machine from the client machine. The server machine used a basic Go-based web server to provide segments to the client. The link between client machine and server machine was then throttled with Dummynet. This provides the ability to compare the data from a more realistic implementation to the rather theoretical Go-based throttling. Worth to mention is that the round-trip time (RTT) was not set for any of the throttling implementations. 9

17 4.3. Detailed System Changes Test Video Information To perform our multi-video test a custom video was made. While it would be possible to simply switch between the same video, we wanted to be able to clearly see which video was actually playing. The open project video Big Bucks Bunny 1 was downloaded and then hardcoded Table 4.1: Video table with text stating camera numbers. These videos were then encoded, split in 4 second segments and formatted for DASH with Resolution Bitrate the FFmpeg 2 and MP4Box 3. The manifest generated was then 64x modified by inserting all videos into the same manifest as adaptationsets, 128x which is mentioned earlier in chapter 2. The amount 192x of qualities and which bitrates there were encoded with are 192x shown in Table x Execution and Data Gathering All tests were performed using a scripted solution in order to eliminate possible human errors from the results. The automated solution started the test by opening a new instance of Firefox, with disabled disk-cache and enabled remote control capabilities. Firefox was controlled over a TCP-connection that was created by an add-on. It then set the desired bandwidth for the web server, by sending an HTTP-request, and then requested Firefox to navigate the site in order to start the data gathering. During the data gathering the script changed both bandwidth conditions and cameras, in order to simulate a possible user scenario. The data was collected by a JavaScript based logger. All data were sampled two times a second and saved in arrays for later use. After a test had been finished all data was collected by the script, which is later plotted and analyzed using Gnuplot. The testbed used is illustrated in Figure 4.2. All the performed tests were 12 seconds long and the data was then shortened to 1s, in order to minimize the effect of the remote control latency when the data was collected, and to be able to present clear plots. Figure 4.3 illustrates how the test was performed during the first 1 seconds. The c n s indicates the camera switches (dash.js automatically picks c 1 at the beginning) and bottom line presents the bandwidth switches. c₁ c₃ c₁ c₂ c₅ 24Mbit/s 8Mbit/s 3Mbit/s 12Mbit/s 24Mbit/s 4Mbit/s Figure 4.3: Timeline of tests for default bandwidth profile 4.3 Detailed System Changes A more precise description of the changes made, where they were made, and the tools that were utilized. Table 4.2 describes the notations that are being used through the thesis. Switching tracks The first thing that required an incursion in the code of dash.js was finding a way to change tracks. To do this, the different video-tracks were placed in separate adaptationsets in the manifest. Dash.js already detected and saved all adaptationsets within 1 Big Bucks Bunny official site: 2 FFmpeg official website: 3 GPAC official website, which provides MP4Box: 1

18 4.3. Detailed System Changes Table 4.2: Notion table Q i B B play B pre B adj B nonadj B le f t α β γ δ i k Choice of quality where i= equals lowest quality Total current bandwidth available Bandwidth given to playback track Bandwidth given to prefetching tracks Bandwidth given to prefetching tracks indexed adjacent to index of playback track Bandwidth given to prefetching tracks indexed non-adjacent to index of playback track Remaining bandwidth after adjacent tracks have chosen quality Fraction of total bandwidth given to playback track Fraction of total bandwidth given to prefetching tracks Fraction of prefetching bandwidth given to adjacent tracks of playback track playback track Fraction of prefetching bandwidth given to non-adjacent tracks of playback track Current playback track Number of tracks Figure 4.4: Flowchart of the track switch the manifest in variables easily accessed from the method where the track switch was made. Dash.js also had a method to switch tracks since it supports switching of audio-tracks. To utilize this functionality for video-tracks, the configuration for track switching was changed from TRACK_SWITCH_MODE_NEVER_REPLACE to TRACK_SWITCH_MODE_ALWAYS_REPLACE for video. On top of that we had to manually request the video-tracks initialization-file since this was not supported in the track switch. The flow chart in Figure 4.4 shows how the switch is made. Prefetching and caching With the track switching working, a cache was implemented. Since dash.js encapsulates its content quite heavily the easiest way to cache data and access it without tunneling through several classes was to make an external cache. This also minimized the amount of incursions to the dash.js code. The cache simply saved the data and the request URL that was made to download it, where the request URL works as a key to retrieve the data. To cache the correct data at the correct time the prefetching and caching was made in the XHRLoader. This is where HTTP-requests are made and sent to retrieve the corresponding data. The prefetching is handled by a centralized process that activates when- 11

19 4.3. Detailed System Changes ever the player made a HTTP request of segments through the XHRLoader for the playback videotrack. Custom request URLs for the other video-tracks included in the manifest are made and sent in a round-robin fashion, and the received data are saved in the cache. To make the player use the cache, an if-statement is constructed and executes whenever a request for the playback track is made. This statement checks if the request URL exists in the cache, and if it does the segment is extracted from the cache instead of making the HTTP-request. Since the cache in its current state fills up the RAM, we also empty the cache during playback. The segments that are thrown away during the cleanup process are segments which index has been passed by the playback index. In order to utilize as much of the available bandwidth as possible, the prefetch functionality delays outgoing requests based on an estimation of how long time it would take to download the previous requested segment. The result of this is that prefeching requests Active camera Prefetch 1 Prefetch 2 are sent in a serial fashion relative to each other, while Prefetch 3 remaining parallel to the active camera request. Figure 4.5 illustrates the theoretical behavior of the delayed Time requests. It is worth to mention that the actual behavior might differ slightly, for example, if one request Figure 4.5: Theoretical behavior of turns out to be slower than the estimated time there will be some overlap. delay Another thing that is cached is the last known quality for a track. Whenever we prefetch and cache a track the quality of the latest segment is stored for every track. This is because when a track switch is made, the player should initially use the quality that have been prefetched, in order to make use of the prefetching. Without manually setting the quality upon a track switch, dash.js often chooses a much higher quality than the one that has been cached. This is because dash.js aims not to fluctuate between qualities too quickly and to provide the user with the highest quality possible. Because of that, since a higher quality is chosen for playback than for the prefetch, the player does not want to go as low as it is required in order for the prefetch to be utilized. Unless the bandwidth is very high, this results in a stall and render the prefetching useless. Bandwidth assignment The assignment of bandwidth for the playback track and the prefetching is done inside the AbrController 4. A method called getqualityforbitrate retrieves the quality for the next segment based on the currently calculated bandwidth. Here the bandwidth given to the playback track is scaled by a chosen factor α, as follows: B play = B α. The remaining bandwidth to be used for the prefetch is determined by factor β, according to: B pre = B β, where α + β = 1. The prefetch bandwidth is stored in an array, and whenever a prefetch request is made, a bandwidth value is retrieved by extracting the most recently calculated value from the bandwidth array. Adjacent Camera Policy On top of the default assignment of bandwidth between playback and remaining videos, a policy to prioritize adjacent "cameras", in our case videos whose adaptationset is indexed adjacent to the playback one, were made. This was done by scaling the bandwidth a second time, giving the majority of the estimated prefetch bandwidth to the two cameras adjacent to the playback one. This assignment is made inside the XHRLoader where the prefetch is done. Instead of just giving all prefetch tracks the same share of the total prefetch bandwidth, a 4 AbrController: Adaptive bitrate controller. Part of dash that handles the quality adaptation 12

20 4.3. Detailed System Changes Figure 4.6: Policy illustration check is made on their index relative to the playback index, and adjacent ones are given the majority of it for their requests. So similar to how the bandwidth is split between playback and prefetch, adjacent cameras are given a part of the prefetch bandwidth based on γ. This means that the adjacent tracks will have a bandwidth according : B adj = B pre γ. The player will then choose maximum available quality for those two tracks based on B adj, resulting in a leftover bandwidth that will be passed to the remaining tracks determined by the bandwidth of the chosen quality Q i for adjacent tracks according to: B le f t = B adj (2 Q i ), where 2 Q i+1 ą B adj. The leftover bandwidth after the adjacent cameras have made their requests are then added to the remaining bandwidth from the initial prefetch scaling between adjacent and non-adjacent tracks, and finally given to the remaining tracks based on B pre, factor δ and B le f t as follows: B nonadj = B pre δ + B le f t, where γ + δ = 1 The policy is illustrated in Figure 4.6. In other words, the policy considers the probability of switching to adjacent tracks are higher than other tracks. Proof of Concept: On Switch Prefetch The final implementation is a proof of concept based upon dash.js way of performing a track switch, described earlier in chapter 2. Since dash.js does not switch track instantly in an attempt to avoid stalling, and thereby reducing the impact of a prefetch implementation, the On Switch Prefetch concept takes advantage of this to further ensure a stall free experience of a high quality video. While dash.js already delays the switch, it does not attempt to guarantee a switch free of stalls. It simply delays the switch a couple of seconds and hopes to complete the download of the new track in time. On Switch Prefetch takes into consideration the delay time and available bandwidth in order to predict when dash.js will make the switch to prefetch the correct segment, and also calculate which quality can be chosen in order to complete the prefetch in time while maintaining as high quality as possible. The concept is illustrated in Figure 4.8. When the user choose to switch track, the player will calculate the time left of the currently played segment and assign this time and the length of the playback buffer to the prefetch. The reason for this is explained further in the Discussion chapter, but it can be seen as a safety factor to avoid retries in terms of prefetch downloads due to time limitations or miscalculations. For a more detailed illustration of logic a flowchart is presented in Figure 4.7. Based on the calculated delay time and dash.js bandwidth estimation, the player chooses a quality of the prefetch that can be downloaded in time. Once the download is complete, the player checks if the download was completed in time. If the prefetched segment with index number j is the same index as the currently played ones index, or older, it means the download took 13

21 4.3. Detailed System Changes Figure 4.7: Proof of concept detailed illustration Figure 4.8: Proof of concept graphic illustration too long, and the delay is continued while a retry is made. If the currently played segments index number m is less than the segment index of the prefetch, the player continues to the next condition. If the currently played segment is the segment before the prefetched segment, j 1, it means it is time to try to switch track. However, if the currently played segment is more than one segment ahead of the prefetched segment, m ă j 1, the time until m = j 1 is estimated and a switch delay is set to this time. Finally, when the download is complete and the segment index m is equal to j 1, we check if the remaining time T current of the currently played segment is greater than the chosen threshold T threshold. Since dash.js does not change track if it considers the remaining time of the currently played segment too short, a threshold time of 2 seconds have been implemented to make sure dash.js actually switches track on the correct segment. If T current ą T threshold, the switch is initiated and the new track begin by playing the prefetched segment. Evaluation of Delay, Buffer Level and Segment Duration As our implementation describes, a vital part of the proof of concept is to determine when the switch actually occur. In order to further evaluate how dash.js decides when to perform the switch we have investigated the delay in respect to the buffer level at the switch call. We also measured how much time there is left of the playing fragment when a switch call is made. This is investigated to see if there is a correlation between remaining segment time and the observed delay for the switch. The switch delay is measured as the time between when a switch is requested and when the player actually render the new track. 14

22 4.3. Detailed System Changes Correlation Between Buffer Level and Delay Our hypothesis is that there is a direct correlation between the amount of segments in the buffer and the delay that occurs at a switch. That is, if there is much content in the buffer the switch will be longer than if there was a smaller amount of content. Correlation Between Segment Time Left and Delay The hypothesis for the correlation between the time left of the segment that is played when a request for a switch is made and the delay is that the more content there is left the smaller the delay will be. The reason for this is that if there is only a small amount of content left in the segment the player won t be able to perform the switch fast enough to avoid a stall. 15

23 5 Evaluation Methodology In this chapter we present the evaluation methodology that was used. 5.1 Testbed and Evaluation Framework As we used two different throttling solutions, they were evaluated and compared in order to give a deeper understanding of the results. This was made because, as mentioned before, the Go-based throttling was highly theoretical, while Dummynet offers a more realistic solution to the problem. 5.2 Baseline Policies The evaluation of the whole implementation was performed by doing tests for three implementations. These three were no prefetching at all, the default prefetching where the prefetch bandwidth was distributed equally between all prefetch tracks and prefetching but prioritizing adjacent tracks. The bandwidth and track change was systematically adjusted by a scripted solution, in order to remove errors caused by human interaction. 5.3 Bandwidth Profiles and Test Scenarios To analyze the policies for different bandwidth levels, tests were made for two different bandwidth profiles. These profiles are referred to as default bandwidth profile and low bandwidth profile. The default bandwidth profile is the one represented in Figure 4.3. The low bandwidth profile is the default bandwidth profiles times.75, meaning it has the same proportions but overall lower bandwidth. Both of these tests were made with the Go-based throttling. In order to analyze the implementation in a more realistic scenario, all policies were also tested with the default bandwidth profile in the real world environment scenario, meaning the link between client and server, throttled with Dummynet. To present the ability of the adjacent camera policy, a best case scenario with the default bandwidth profile was also made for the adjacent camera policy implementation. These tests were made using the Go-based throttling. The best case scenario only performed switches 16

24 5.3. Bandwidth Profiles and Test Scenarios at the same rate as the default scenario, but only to adjacent cameras. The order of the camera switches was to start at camera 1 and to perform switches backwards in a round-robin fashion. Finally, tests for the proof of concept, on switch prefetch, implementation were made. These tests were made with the Go-based throttling and the default bandwidth profile. Since this is a proof of concept, the results will not be taken into consideration with the rest for comparison. Instead they are presented alone merely as an indication of the concept in its current state, to get an idea of the current performance and possible improvements. The tests to evaluate the correlation between delay, buffer level and segment durations were made manually with no prefetching active since we were interested in the delay between switch initialization and the render of the new video. The test was made by simply starting a counter when the switch was initiated where the buffer level and segment time left was retrieved and then stopping the counter when the video actually rendered the new track. By doing so an approximate delay time and its corresponding buffer and segment duration values can be evaluated. It is worth noting that while a manual counter might sometimes result in a small delay between actual stop time and our reaction, it is usual quite clear when this happens since it will always increase the delay slightly. 17

25 6 Results This chapter presents the results that were created based on the performed tests. The Gobased results are averaged from 4 iterations if nothing else is stated. The Dummnynet-based real world scenario results are averaged from 26, 24 and 22 tests for the no-prefetch, basic prefetch and the policy respectively. The variations in numbers is due to some faulty tests that had to be removed. The Go-based results are not as thoroughly tested due to their theoretical nature, which is why the Dummynet-based tests are prioritized. 6.1 Example Run Figures 6.1 illustrates how the buffer occupancy changed during a typical playback session. Through these graphs an indication of the characteristics of the different implementations are shown Buffer Occupancy Mean Buffer Occupancy Mean Buffer Occupancy Mean Seconds Seconds Seconds Time (a) No prefetch Time (b) Basic prefetch Time (c) Policy-based prefetch Figure 6.1: Buffer occupancy of the three implementations Before going deeper into the results, it is worth mentioning that, due to how dash.js switch tracks, switching without prefetched segments is not punished as hard as it could be. As mentioned in the chaper 2, when a track switch is made, dash.js still finishes the currently played segment as part of its strategy to avoid stalling. Because of this a track switch without prefetched content can still do well if the bandwidth is good enough, since it can get up to 4 seconds of time to download a segment of the switched track. 18

26 6.2. Results Based on Means 5 Quality Mean 5 Quality Mean 5 Quality Mean Quality index 3 2 Quality index 3 2 Quality index Time (a) No-prefetch Time (b) Prefetch Time (c) Policy Figure 6.2: Quality occupation for the three implementations Figures 6.2 shows how the playback quality changed during three typical tests. It is important to note that time spent stalling is not shown in these graphs. When a stall occurs the graph will continue to display the last played quality. The reason for this is because the player still tries to play that quality, but it has run out of buffer. 6.2 Results Based on Means In this section we present results based on means from multiple runs. Buffer Occupancy Figure 6.3 illustrates how the different implementations performed during the different cases. These results are derived from the average of multiple runs for each implementation. The implementation using basic prefetching performed the best, in terms of buffer occupancy, during the low bandwidth case. No-prefetch had the highest buffer occupancy for the default and real world scenario. For the real world scenario, even though the bandwidth used was the same as in the default case, the no-prefetch implementation increased significantly. The prefetch implementations also performed better, however not as significant as the noprefetch. During the best case scenario the policy was significantly better when compared to the default case. But it still performed slightly worse than the implementation without prefetching from the real world scenario. Table 6.1 show the buffer deviation average over multiple tests for each implementation. The non-prefetch implementation had the highest deviation in all cases, except the low-bandwidth profile, where the policy prefetch had a slightly higher deviation. Buffer occupancy [s] No Prefetch Basic Prefetch Policy-based Prefetch Default Low bandwidth Real world scenario Best case Figure 6.3: Average buffer occupancy 19

27 6.2. Results Based on Means Table 6.1: Buffer standard deviation Case No Prefetch Basic Prefetch Policy-based Prefetch Default 3.47 s 3.35 s 2.3 s Low bandwidth 2.9 s 2.63 s 3.6 s Real world scenario 3.82 s 1.87 s 2.14 s Best case s In all cases except the low bandwidth profile, patterns can be seen between high average buffer levels and high buffer level standard deviations. Especially for the non-prefetch case, Figure 6.1 indicates that this might be due to unstable buffer levels with low dips or stalls combined with high tops. Because of this, high buffer average might not necessarily be regarded as good if it is combined with high standard deviations. Stalls The total time stalling is presented in Table 6.2 for each implementation. The results indicate that the policy spent the least time stalling in all cases except the real world scenario, where the basic prefetch did best. While the difference is only about 5 ms, it is still an interesting change considering it has the same bandwidth profile as the default case. Worth noting is that the stall time is approximate. This means that a dip to zero that might not last more than an instant is calculated as.5 seconds of stall. On top of this, buffer levels below.5 seconds is also counted as a stall. Such an assumption had to be made since dash.js debug buffer calculator never estimated the buffer as. Despite long stalls which clearly was due to an empty buffer, it always returned values of -.5 seconds of video in the buffer. This can give a slight variation from the true stall time. However, despite this flaw, the table still clearly indicates that the non-prefetch has the most stalls, especially for the default bandwidth profile. For the lower bandwidth profile the stall times are more even across all implementations, indicating that the bandwidth is often so low it renders the prefetching policies unable to prefetch, resulting in stalls. Table 6.2: Stall time Case No Prefetch Basic Prefetch Policy-based Prefetch Default 2.5 s.5 s. s Low bandwidth 4.88 s 4.13 s 3.13 s Real world scenario 3.87 s 1.13 s 1.64 s Best case - -. s Quality As the results presented in Table 6.3 and Figure 6.4 indicates, the non-prefetch also had the lowest quality deviation on top of the highest quality mean. This is not too surprising since it almost always attempts to play top quality because it has the bandwidth for it, but which sometimes results in stalls following a track switch. The prefetch implementations on the other hand has a lower bandwidth for the playback track, resulting in a lower mean. Prefetching also means that, upon a track switch, they play at least one of the prefetched segments if it has any, which in turn is of a lower quality. Because of this the quality deviates more and the mean is lower, but stalling is avoided or reduced. 2