On the Feasibility of DASH Streaming in the Cloud

Transcription

1 On the Feasibility of DASH Streaming in the Cloud Cong Wang and Michael Zink Department of Electrical and Computer Engineering University of Massachusetts, Amherst, MA 13 {cwang, ABSTRACT As shown in recent studies, video streaming is by far the biggest category of backbone Internet traffic in the US. As a measure to reduce the cost of highly overprovisioned physical infrastructures while remaining the quality of video services, many streaming service providers started to use cloud services where physical resources can be dynamically allocated based on current demand. This paper characterizes the performance of Dynamic Adaptive Streaming over HTTP (DASH), a new MPEG standard on adaptive streaming, in the cloud. We seek to answer the following questions that are critical to content providers that are hosting video in clouds: Which data center is the best to host videos? Does geographical distance matter? What type of instance is best suitable depending on different needs? How to efficiently solve the trade-off between performance and cost? The measurement methods and results presented in this paper can be easily expanded into other VoD services, and they allow us to i) characterize DASH behavior when streaming from the cloud; ii) identify the key factors that influence the DASH performance; and iii) suggest improvements for related services. Categories and Subject Descriptors C.4 [Performance of Systems]: Reliability, availability, and serviceability General Terms Performance, measurement Keywords Cloud computing, HTTP adaptive streaming, video-ondemand, quality of experience 1. INTRODUCTION Video Streaming in the Internet has recently been gaining tremendous popularity. In 211, YouTube had more than Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. NOSSDAV 14, March 19-21, 214, Singapore, Singapore Copyright 214 ACM /14/3...$ trillion views or around 14 views for every person on Earth [8]. According to [4], Netflix video sessions count for over 3% of Internet downstream traffic. The same report also shows that Netflix, YouTube, and Hulu together contribute for more than 5% of the Internet downstream traffic in North America. In addition, many traditional broadcast channels (e.g., NBC, ABC, CBS, etc.) now provide part of their programmes online on the web. Following this trend, the network and content providers have to supply a large amount of physical resources in order to meet the high demand of video services. As with many other services, resources are provided such that peak requests can be handled without impact on the quality of the provided VoD service; on the contrary, analyses of VoD services have shown that the overall number of requests can vary significantly over time. For example, in [21] we have shown that, YouTube requests follow a diurnal pattern, and peak demand lasts only for a relatively short period during a 24-hour interval. This can lead to significant overprovisioning of resources. As an alternative solution, cloud services, such as Amazon Web Services (AWS), provide resources that can be quickly deployed and released according to customer needs. This approach has been widely used for web and VoD services, such as Netflix, GitHub and Dropbox, since the content providers do not have to maintain expensive physical infrastructures. Instead, they only need to pay for the usage of the allocated cloud resources. Another trend in video streaming with increasing attention is Dynamic Adaptive Streaming over HTTP (DASH) [19, 18]. In this case, the quality of a video, along with resulting streaming bandwidth, can be adapted based on bottleneck bandwidth, server capacity, and client resources. Unlike traditional progressive streaming, DASH divides the videos into segments with each segment representing a portion of the video. In the progressive streaming approach, the client sends video request to the server, then establishes a continuous TCP connection for transmitting the whole video. In contrast, the DASH client periodically probes the available bandwidth and client buffer condition, then decides the next requested segment quality level. DASH provides high resilience to network variations and smooth playback experience, however, it requires much more interactions between the client and server and may increase the overhead. Typically, for a 6-minute video, in the case of YouTube progressive streaming, the client only needs to send 1 HTTP GET request containing requested video information. However, the YouTube adaptive streaming

2 (bitrates), while each quality layer is chopped into segments. The DASH client downloads the MPD in order to retrieve a complete list of components, including video, audio and subtitles. The DASH client then determines the preferred set of segments based on its capability (e.g., resolution), user preference (e.g., language), and network conditions (e.g., bandwidth or buffer status). And finally, the DASH client downloads and plays out the desired content. DASH videos are usually encoded with different segment lengths. Commercial approaches use segment lengths ranging from 2 seconds per fragment (e.g., Microsoft Smooth Streaming [5]) to 1 seconds per fragment (e.g., Apple HLS [2]). Figure 1: DASH Flow Diagram client sends 141 HTTP GET request, while a VLC-DASH client with segment length of 2 sends over 16 HTTP GET requests, depending on the frequency of quality switchings. Due to the two trends above, and currently there is no public information available regarding the performance of DASH-based streaming in the cloud, our goal is to provide an extensive evaluation of the performance of DASH video streaming from the cloud. In this paper, we investigate commercial (Amazon and Rackspace) and research (ExoGENI) clouds and stream from various network environments; also we evaluate the scalability of cloud VM nodes by running large scale distributed streaming experiments over PlanetLab [6]. Our findings include: i) the geographical location of cloud data centers is not the dominant factor for VoD streaming performance. E.g., the EC2 west (California) data center shows much better performance than EC2 east Virginia when serving clients located on the east coast (Massachusetts); ii) the video quality decreases dramatically when the number of parallel streaming sessions exceeds a certain threshold, and we show the threshold for each type of cloud instance; iii) Amazon EC2 cloud instances provide up to 7% higher throughput than Rackspace, however, it usually supports less parallel client connections. The remainder of this paper is outlined as follows. Section 2 provides a brief introduction on the operations of DASH. In Section 3, we describe the measured cloud services and our measurement setup. Section 4 presents the results from these measurements. In Section 5, we give a brief overview of related work. Section 6 concludes the paper and points out future works. 2. DYNAMIC ADAPTIVE STREAMING OVER HTTP (DASH) In the section, we provide a brief overview of the functionality of DASH, the recently developed MPEG adaptive streaming standard. DASH consists of two components: A Media Presentation Description (MPD) manifest file which contains the description of the available content, their URL addresses, available bit rates or qualities, segment lengths, and other characteristics; and video segments which contain the actual multimedia bit streams in the form of single or multiple files based on the encoder [13]. Figure 1 depicts the streaming flow process in DASH. In this case, a video is encoded into different quality layers 3. METHODOLOGY In this section, we give a brief overview on the cloud services that are used in our experiments, then we describe the metrics for evaluating the performance of DASH-based streaming in the cloud. 3.1 Measured Cloud Services Our measurement study was performed on two commercial cloud services: the AWS Elastic Compute Cloud (EC2) [1], Rackspace Cloud [7], and a research cloud service: ExoGENI [3]. Both EC2 and Rackspace provide resizable cloud computing capacity to execute applications on demand. The cost of commercial cloud services depends on the type of resources used and the duration of the usage, as well as additional factors such as the amount of I/O performed and the amount of storage used. In addition to regular cloud server hosting, the commercial clouds also offers services including cloud storage, CDN, cloud-based website hosting, etc. ExoGENI cloud is an open-source cloud platform, which allows users to programmatically manage a controllable, shared substrate on a reservation basis. The ExoGENI cloud uses a slice-based architecture which gives experimenters more flexibility than commercial clouds, since it allows them to create customized network topology for a compute cluster. 3.2 Experiment Setup We evaluate the DASH performance using instrumentalized VLC player version 2.1. equipped with DASH plugin [17]. For each of the cloud VMs, an Apache HTTP server is installed along with the MPD files, as well as the DASH segments of a video. We used the Big Buck Bunny animated video (about 1 minutes in length), which is divided into 2-second segments and encoded with 17 layers of bitrates varying from 1 to 1, kbps. Since the VLC media player cannot currently support H264/MP4 videos with dynamic resolution, we use video encoded at a constant resolution of p and a 24 fps frame rate. Our choice of 2-second segments is intended to provide high flexibility for quick adaption to bandwidth fluctuations. Meanwhile, we use a maximum buffer size of 3 seconds, to help the client smooth over temporary disruptions of connectivity. The videos are requested from three different data center locations of each cloud, nine origins in total. The nodes are located in Amazon EC2 cloud data centers in CA, OR and VA, Rackspace cloud data centers in IL, TX and VA, and ExoGENI cloud racks in NC, TX and MA, respectively. The type of cloud servers we used are shown in Table 1. Our three sets of experiments are described in detail in Section 4.

3 Cloud vcpus Memory (GB) Price ($/hr) EC Rackspace E-GENI Table 1: Summary of cloud server setup. 3.3 Measurement Metrics To best characterize the behavior of DASH streaming in the cloud, besides the traditional throughput measurements, we utilize additional evaluation metrics. In this section, we briefly introduce these metrics Bitrate The bitrate expresses the objective video quality that is actually received by the client. If there are enough segments (more than 3% of total buffer length, or 9 seconds) in the buffer, the player will choose to request the highest rate of those offered in the MPD file that is smaller than the bandwidth measured in the previous segment downloading process. Otherwise, the client player will request the lowest possible bitrate from the MPD file. Usually a higher bitrate means that a higher quality segment has been received by the client Video presentation quality Another video quality metric we considered is the presentation quality of video playback. Here we employ the metric proposed by Zink et al. [2], which provides a simple yet efficient way to determine the video quality from the viewer s perspective. The authors believe that two factors reduce the user experience: the video quality degradation and the frequency of quality switches. As shown in a comparison study, the spectrum reflects perceived quality better than PSNR. The overall spectrum of quality adaptive video v can be represented as: s(v) = T t=1 z t(h t 1 T i=1 T ( z jh j)) 2 (1) z j=1 i where the two terms h t and z t are defined as: h t: the number of layers in time slot t, t = 1,..., T ; z t: indication of a step in time slot t, z t {, 1}, t = 1,..., T. Usually, the higher the spectrum is, the more fluctuations occur while playing the video, thus it reflects lower video playback quality. The benefit of this metric is its ability to reflect the subjective quality observed by viewers, while it can be calculated very quickly Video initiation time In order to provide users with a smooth playback experience, media players usually buffer the initial part of the video before they actually start displaying the content. Most players, including VLC, which we use for the experiments presented in this paper, set a certain buffer level threshold. As soon as the buffer accumulates more than the threshold, the video playback starts. This initial waiting time (i.e., until the buffer reaches a certain level) depends on several factors, such as server load and network bandwidth. We observe this metric aiming to analyze whether the initial waiting time increases under certain load conditions for the video hosting VMs in the cloud. 4. EXPERIMENTAL RESULTS In this section, we present our experiment results. We start by characterizing the influence of geographical location of the data centers on DASH streaming. Second, we examine the scalability of cloud instances by performing parallel streaming sessions from a single VM to globally distributed clients. Finally, we look into the performance of different types of cloud VM instances, and the number of concurrent sessions that can be supported by these instance types. 4.1 DASH Performance with Cloud Services In this section, we characterize the DASH cloud performance by running single server, single client video streaming experiments from each of the cloud data centers described in Section 3. We stream the same video from each of the nine cloud sources, plus a reference video server located inside the UMass Amherst campus network. This set of experiments is repeated once per hour for a total duration of 24 hours. To avoid the variation of a single access network impairing the results, we repeat the above measurements from four different access networks that are commonly used by endusers: the UMass campus WiFi network, Amherst home Comcast cable Internet, town of Amherst public wireless Internet, and UMass campus WiMAX network. Since WiMAX network performance is sensitive to signal strength variation, we set up the experiment at a fixed point with ideal signal strength as mentioned in [11]. Average Playback Bitrate (kbps) Amherst Cloud Region Campus Wifi Campus WiMAX Home Wifi Public Wifi Figure 2: DASH average playback bitrate Figure 2 shows the average bitrate (out of 24 measurements) for each location, the error bars represent the standard deviation. For each data center location, the bitrates vary based on the access network conditions. Yet the bitrates still show very similar trends among different data center locations. The campus server shows the best performance among all the locations, while Amazon EC2 Virginia has the lowest bitrate. This demonstrates that DASH streaming can be improved when streaming from a server really close by, e.g., within the same LAN or region of the client. Yet beyond that, the performance is not strongly correlated to the geographical distance. Our measurement shows EC2 California actually performs better than EC2

4 Spectrum Amherst Cloud Region Campus Wifi Campus WiMAX Home Wifi Public Wifi Figure 3: DASH spectrum comparison Virginia. The performance difference among data centers is consistent with the results shown in [1]. We conjecture that EC2 Virginia s lower performance in this case is because it is the most heavily used one among Amazon s data centers, handling more than 8% of the total traffic generated by EC2. Here the ExoGENI research cloud shows comparable performance to the commercial clouds. Especially, EG-MA provides one of the highest bitrate and lowest spectra among the cloud nodes. In addition to the bitrates shown in Figure 2, the spectrum expresses the perceived quality for the viewers. The results in Figure 3 show a similar trend between the spectrum and the bitrate. The two figures above indicate that low bitrates are often correlated with more quality changes (high spectrum values) while high bitrate cases result in a better perceived quality. Yet exceptions exist: in the campus WiFi case, the bitrate is almost identical for EC2 OR and Rackspace TX. However, EC2 OR s spectrum is 3 units larger than that of Rackspace TX. This might be caused by a higher number of quality changes in the case of EC2 OR. 4.2 Scalability of DASH Cloud Streaming In this section, we examine the scalability of DASH streaming by requesting concurrent streaming sessions (clients) from each of the cloud VM. We make use of 6 geographically distributed PlanetLab nodes as streaming clients. This is to make sure that the local access network on the client side will not become the bottleneck. Starting with 2 parallel sessions, we increase the number of sessions to 1,8 by increments of 5. Using this method, we aim to observe how a single cloud VM performs when large amounts of video requests are getting in at the same time, and to determine the maximum number of parallel requests that can be served. In the case that more than 6 sessions are needed, we start multiple downloading sessions on each of the PlanetLab nodes, resulting in a maximum of 3 parallel sessions being active on each PlanetLab node. Since the behavior of a DASH streaming session is very similar to regular HTTP downloading, for simplicity, we run HTTP downloading of the video segments as the contention traffic from PlanetLab nodes, while the measurement is performed by running actual VLC streaming from a node connected to our campus Wifi network. We show the bitrate variation observed at the VLC client with increasing number of concurrent sessions in Figure 4. At the beginning, when the amount of contention traffic goes up, the bitrate slightly decreases yet overall remains stable. However, after a certain point, the bitrate drops dramatically, then quickly reaches, which means the video does not start playing at all. The initial buffering time (as percentage of the total video length of 1 minutes) is shown in Figure 5. For all cloud regions, the time starts from below 5% (3 seconds) and remains constant. It quickly increases to over 25% (15 seconds) after certain number of parallel sessions. As the number of concurrent sessions increases, less cloud instances are able to start playing the video. Therefore, for parallel sessions higher than 1 some bars are not shown. Both Figure 4 and Figure 5 show the same trend for each of the data centers: there exists a threshold for concurrent DASH connections, exceeding the threshold will cause low video quality and long buffering time. Therefore, it is useful to consider migrating load to either more cloud nodes or larger VMs early before the load actually reaches the threshold. In our measurements, the EC2 instances deliver higher bitrate when the number of concurrent sessions is low. However, their thresholds are lower than Rackspace s when both are using small instances. (Further comparisons of different types of cloud instances are presented in Section 4.3.) The ExoGENI cloud is for research purpose at no cost, and provides better scalability performance than commercial cloud when using comparable instance size, therefore, it offers a good platform for initially testing new cloud based streaming mechanisms. However, ExoGENI only allows the reservation of resources for short time periods, thus it is not suitable for hosting commercial, long running or large scale video services. Overall, we show that hosting DASH streaming servers in clouds is feasible from the perspective of scalability. Since in this section, the instances we have been using are relatively small, this represents the lower bound of cloud server performance. If larger scale video hosting is needed, the content providers can easily deploy higher capacity servers or increase the number of servers. Average Playback Bitrate (kbps) 1e+7 9e+6 8e+6 7e+6 6e+6 5e+6 4e+6 3e+6 2e+6 1e Number of Sessions Figure 4: The bitrate variation with number of planetlab sessions

5 Initiation Percentage of Whole Video Bandwidth (Mbps) AWS-EC2 Rackspace micro small medium large xlarge 2xlarge Number of Sessions Instance Type Figure 5: DASH video playback initiation time 4.3 Cloud Instance Capabilities Both EC2 and Rackspace offer customers the flexibility to choose different cloud instances at different prices. In this section, we investigate the DASH streaming performance on different types of cloud instances. We choose the EC2 CA and Rackspace TX data centers, since they perform better than others in the former experiment (as shown in Section 4.1). A brief summary of capabilities and prices of different instances is shown in Table 2. Amazon EC2 offers 7 types of general purpose instances varying from t1.micro to m3.2xlarge. We excluded the m3.xlarge instance because it is similar to m1.xlarge in the sense of vcpus and memory. Similarly, Rackspace provides 7 standard instance types, and we selected 6 of them that are comparable to the EC2 instances. For the micro instance, both EC2 and Rackspace share the same hourly price; other than that, the price of Rackspace instances is about twice as high as that of EC2. In return, Rackspace usually provides more vcpus, which might potentially influence the performance of applications when serving a large number of clients in parallel. In this section, we only compare between Amazon and Rackspace, because ExoGENI does not provide instance sizes that are comparable to all of those in the commercial clouds. Using the same method as mentioned in Section 4.2, we start our analysis by running 1 PlanetLab node downloads concurrently, then gradually increase the number of parallel downloading sessions, until the video on the VLC client fails to start. From this measurement, we aim to determine the maximum number of requests that can be handled by a single cloud VM type. Cloud Size vcpus Memory Price ($/hr) (GB) EC2/RS micro 1/1.6/.5.2/.2 EC2/RS small 1/1 1.7/2.6/.12 EC2/RS medium 1/2 3.7/4.12/.24 EC2/RS large 2/4 7.5/8.24/.48 EC2/RS xlarge 4/6 15/15.48/.9 EC2/RS 2xlarge 4/8 3/3.9/1.2 Table 2: Summary of EC2 and Rackspace instances. Figure 6 shows the maximum bandwidth provided by each type of cloud instance. As can be observed, EC2 has up to 7% higher peak bandwidth than Rackspace. This might be the cause for EC2-CA showing a better performance than Figure 6: Maximum bandwidth of each type of cloud instance Number of Sessions AWS-EC2 Rackspace micro small medium large Instance Type xlarge 2xlarge Figure 7: Parallel sessions supported by each type of cloud instance Spectrum AWS-EC2 Rackspace micro small medium large Instance Type xlarge 2xlarge Figure 8: The spectrum condition when instance running near maximum number of parallel sessions Rackspace-TX in the experiment presented in Section 4.1. Interestingly, Figure 7 shows that Rackspace can support 3% more parallel sessions than EC2. Our conjecture is that Rackspace provides more vcpus for the same instance type, which can better handle large amounts of HTTP requests at the same time. This observation is consistent with the results presented in [12], which shows that most of the

6 Rackspace instances perform better than Amazon instances when processing parallel tasks. Figure 8 shows the spectrum for the VLC player session at the point when each type of instance is running close to the maximum number of parallel sessions. This figure demonstrates that, while larger instances can support more parallel sessions and a higher cumulative bandwidth, the quality of the individual streams reduces. Between an EC2 micro and 2xlarge instance the spectrum increases significantly from 3,2 to 3,9. Thus, in the case when server operates near the threshold, it might be more beneficial from a viewer s perspective to stream from a large set of small instances instead of a single large one. We propose to further investigate our claim in future experiments by comparing the results of streaming from multiple small instances in parallel. 5. RELATED WORK There has been extensive works on the performance evaluation of both Amazon and Rackspace cloud services. [14] provides a benchmark to compare cloud services in the sense of CPU, memory, network, etc; while [1] presents results from network measurements focusing on multiple cloud services provided by AWS, such as EC2, S3 and CloudFront. Works such as [15] present studies on cloud VM starting time and other metrics. On the DASH side, many works have been focusing on the DASH algorithm efficiency and encoding schemes, such as [13] and [9]; while others, such as [16], provides evaluation of DASH in different environment settings. However, these works are mostly done in a relatively small scale environment. To the best of our knowledge, our work is the first one that systematically investigates the effectiveness of DASH adaptive streaming in the cloud based on real distributed system measurements. 6. CONCLUSIONS In this paper, we evaluate the effectiveness of hosting adaptive streaming video servers in the cloud. Through a measurement study, we demonstrate the cloud resource and network performance of DASH streaming from the cloud. We characterize the DASH performance in nine data centers of three cloud service providers. In particular, we study the impact of geographical distance on DASH streaming, as well as the threshold for the number of concurrent sessions that can be hosted by a single cloud VM node; also we look into the methods of choosing best suitable cloud resources based on these thresholds. In future work, we intend to extend our work to the performance comparison among different DASH implementations, e.g., YouTube adaptive streaming and DASH-JS, the Javascript DASH library. Also we intend to further investigate the issues we identified in this paper, such as adaptive video delivery that is tolerant to network instability and the trade-off between single large cloud instance and multiple small ones. Acknowledgement This work is funded by and NSF grant CNS Any opinions, findings, conclusions or recommendations expressed in this material are the authors and do not necessarily reflect the views of the National Science Foundation. 7. REFERENCES [1] Amazon s Elastic Compute Cloud. [2] Apple HTTP Live Streaming. [3] ExoGENI. [4] Global internet phenomena report. sandvine.com/news/global_broadband_trends.asp. [5] Microsoft Smooth Streaming. [6] PlanetLab. [7] Rackspace Cloud. [8] YouTube Statistics. [9] S. Akhshabi, A. C. Begen, and C. Dovrolis. An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP. In MMSys, pages , 211. [1] I. Bermudez, S. Traverso, M. Mellia, and M. Munafo. Exploring the cloud from passive measurements: The Amazon AWS case. In INFOCOM, pages , 213. [11] F. Fund, C. Wang, T. Korakis, M. Zink, and S. Panwar. GENI WiMAX performance: Evaluation and comparison of two campus testbeds. In Proceedings of the GENI Research and Educational Experiment Workshop, 213. [12] S. Kalpit, S. S., and R. S. Cloud performance benchmark series. Cloud Commons Online, 211. [13] S. Lederer, C. Müller, and C. Timmerer. Dynamic Adaptive Streaming over HTTP Dataset. In MMSys, pages 89 94, 212. [14] A. Li, X. Yang, S. Kandula, and M. Zhang. CloudCmp: Comparing public cloud providers. In IMC, pages 1 14, 21. [15] M. Mao and M. Humphrey. A performance study on the VM startup time in the cloud. In Cloud Computing (CLOUD), pages , 212. [16] C. Mueller, S. Lederer, and C. Timmerer. An Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular Environments. In MoVid, February 212. [17] C. Müller and C. Timmerer. A VLC media player plugin enabling dynamic adaptive streaming over HTTP. In MMSys, pages , 211. [18] T. Stockhammer. Dynamic Adaptive Streaming over HTTP : Standards and Design Principles. In MMSys, pages , 211. [19] C. Timmerer and C. Müller. HTTP Streaming of MPEG Media. In Streaming Day, September 21. [2] M. Zink, J. Schmitt, and R. Steinmetz. Layer-encoded video in scalable adaptive streaming. IEEE Transactions on Multimedia, 7(1):75 84, 25. [21] M. Zink, K. Suh, Yu, and J. Kurose. Characteristics of YouTube Network Traffic at a Campus Network - Measurements, Models, and Implications. Elsevier Computer Networks, 29.