Rate-Distortion Optimized Video Peer-to-Peer Multicast Streaming

Transcription

1 Eric Setton Rate-Distortion Optimized Video Peer-to-Peer Multicast Streaming Invited Paper Jeonghun Noh Bernd Girod Information Systems Laboratory, Department of Electrical Engineering Stanford University, Stanford, CA , USA ABSTRACT We study peer-to-peer multicast streaming, where a source distributes real-time video to a large population of hosts by making use of their forwarding capacity rather than relying on dedicated media servers. Hosts which may disconnect at any time, therefore a robust control protocol is needed to maintain connectivity among peers. This work presents a new peer-to-peer multicast protocol and analyzes the gains that video coding and prioritized packet scheduling at the application layer can bring to the overall streaming performance. A rate-distortion model which predicts endto-end video quality in throughput limited environments is presented and used to determine the over-provisioning necessary to avoid self-inflicted congestion. The video stream transmitted by the source contains H.264 SP and SI frames, which are used to adaptively stop error propagation due to packet loss. Distortion-optimized retransmission requests are issued by receiving hosts to recover the most important missing packets while limiting the induced congestion. Experiments for several hundred hosts simulated in NS-2 illustrate the benefits of our system. We achieve typical end-to-end delays of 1 sec, and a stable video quality with less than 2.5% of frames lost to playout interruptions. Categories and Subject Descriptors C.2 [Computer-communication networks]: Distributed systems. General Terms Design, performance. Keywords Peer-to-peer, video streaming, multicast. This work was supported, in part, by a gift from Hewlett- Packard Laboratories, Palo Alto, CA. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. P2PMMS 05, November 11, 2005, Singapore. Copyright 2005 ACM /05/ $ INTRODUCTION As IP multicast is not universally supported, distribution of media streams in the public Internet to a large audience ( multicasting ) is typically realized by a large number of unicast connections. If the maximum number of streams of an individual media server (typically between a few hundred and a few thousand) is exceeded, additional server capacity must be provided by a suitable content-delivery infrastructure, e.g., in the form of a network of replication servers. Peer-to-peer (P2P) multicasting is an elegant alternative in which each end-host may act as a potential server for other clients. This avoids dedicated replication servers altogether. The approach is self-scaling, as the number of peer servers and peer clients increases at the same rate, hence it avoids the bottleneck of a central server (or dedicated replication server). The approach, in principle, would allow a highly dynamic support of changing multicast demand at very low cost. The major challenge, however, is the complete lack of performance guarantees in the P2P network. Peer nodes might be turned off or disconnected at any time without prior notice, while other nodes join or re-join. Such a highly unreliable network fabric poses a major difficulty for media streaming. A recent study based on statistics collected over the Internet reveals that although the uplink throughput of peers is a limiting factor, there is enough bandwidth for P2P streaming, even on a large scale [21]. The stability of the system which is necessary to provide a satisfactory user experience largely depends on the design of the protocol. To gain robustness and possibly aggregate data rate, path diversity should be attained by distributing streams across a sufficiently large number of complementary multicast trees. Individual nodes are important nodes near the top in some multicast tree, but near the bottom and less important in others, thus avoiding single points of failure in the node dependency graph. Although the control protocol is essential to provide efficient means of building and maintaining multiple multicast trees, it needs to be combined with efficient multimedia coding and streaming solutions at the application layer. Stateof-the art compression which achieves better rate-distortion performance alleviates the bandwidth requirements, while error-resilient streaming techniques may improve the received media quality. In this paper, we present a P2P multicast protocol and show the benefits of video coding, adaptive streaming and 39

2 optimized packet scheduling for such a system. The purpose of this work is to make the following contributions: the design and a analysis of a new distributed P2P multicast protocol, targeted for low-latency streaming, a video distortion model analyzing the streaming performance in a throughput-limited P2P network, an adaptive video streaming technique, suitable for a P2P network, to mitigate error-propagation through the use of H.264 SP and SI picture types, a distortion-optimized retransmission scheduler which maximizes video quality while limiting the impact on congestion. Although the gains reported in this paper for streaming with SP and SI frames and for the retransmission scheduler are shown for our implementation of a P2P multicast control protocol, we believe the results are more general and could be applied to most implementations of video P2P streaming. In the next section, we describe the control protocol which builds and maintains multiple multicast trees to broadcast a video stream from a source to a set of peers. The performance of the control protocol is assessed through experiments carried out over a simulated network in NS-2 [6]. In Sec. 4, we make use of a video distortion model, previously proposed in [20], to predict the received video quality when a video stream is sent from multiple throughputlimited senders to a receiver at different rates. The model is used to determine the amount of over-provisioning of network resources required to limit the congestion created between peers. The video stream multicast over the P2P network contains the new picture types switching-p (SP) and switching-i (SI), introduced in the latest video coding standard H.264 [12]. These switching picture types can be sent adaptively to stop error propagation in the case of transmission errors. In Sec. 5, we characterize the bit rate savings and performance improvement achieved by using SP and SI frames compared to traditional video transmission based on I and P frames. In the last part of the paper, we focus on retransmission scheduling from a receiving host to its parents. Different from network-level multicast, the incorporation of application level retransmission requests into P2P multicast is possible without feedback implosion, since the fan-out of each individual node is small. We describe in Sec.6, how to schedule retransmissions, from the receiver, to maximize its decoded video quality while limiting the incurred network congestion. 2. RELATED WORK We address the problem of reducing the cost of large-scale live media delivery and focus on IP networks such as the Internet. Commercial solutions deployed today mostly rely on server overlays which act as Content Delivery Networks (CDN). When a user wishes to access multimedia content, he is re-directed towards one of the servers of the CDN. This server is usually located closer to the user and has enough available bandwidth to support the media streaming session. Such networks have been deployed, e.g., by Akamai or Cisco Systems. The design of these large scale systems is a problem of itself which has been studied, notably in [13] and [7]. Live media multicast over P2P networks proposes to shift the burden of the media delivery from a dedicated infrastructure to the users. As P2P networks do not require any special servers or routers, the cost of such solutions is appealing. However, they only offer an attractive alternative if their robustness can overcome the dynamic behavior of the peers used for forwarding. Some functioning solutions based on P2P multicast are already available. Coolstreaming which offers Chinese and American cable channels reports an ever-growing user base [1, 23]. Other implementations exist such as PPLive [5], which multicasts mostly sports channels, and ESM [3] which has been used to broadcast conferences in the scientific community. Although these implementations are very exciting advances, they all suffer from long startup delays usually on the order of 3-5 minutes, instability, and their streaming quality is not yet a threat to standard definition TV. This motivates our work which places an emphasis on the role of congestion in P2P video streaming and strives for stability and low startup latencies of a few seconds, comparable to cable or satellite television systems. In most P2P multicast protocol proposed so far peers build and maintain one or several multicast trees along which video content is distributed [8, 10, 17, 22]. The construction of the trees usually proceeds in a distributed fashion, which allows the protocol to scale without overwhelming the source. The control protocol we propose is similar to these previous systems. Our protocol establishes multiple multicast trees as proposed in [11, 17]. In these systems, multiple trees are used to mitigate the impact of peer disconnection. Through the use of multiple description coding or of forward error correction (FEC) they also maintain acceptable video quality even when a node is not connected to all the trees. In our system, we also make use of optimized encoding and application layer solutions. The overall performance of P2P video multicast systems can be greatly improved by using efficient single-description coding techniques and achieving the required robustness through over-provisioning and distortion-optimized scheduling of transmissions and retransmissions. 3. P2P MULTICAST PROTOCOL The control protocol enables a source to distribute a video stream to a population of peers via P2P multicast. The video source peer and other peers are connected via multiple trees which are constructed dynamically by the protocol. The source is the root of all trees and the trees are built and maintained mostly independently. The branches of each tree connect a host to its descendants. These links are virtual tunnels which hide the underlying physical network topology. The video stream is distributed evenly over the different trees. Hence, peers need to join each of the multiple trees to decode and play out the video successfully. Our simulations are based on a moderate size network of a few hundred nodes, which resembles a large private intranet or a campus network. An example is shown in Fig. 1. We make the following simplifying assumptions: the control and transmission protocol is implemented over the UDP/IP protocol stack and we ignore any Network Address Translator or firewall issue which may limit connectivity; peers are synchronized and have heterogeneous but fixed upload bandwidth which they have measured and know accurately. Although these problems need to be addressed for a real In- 40

3 and reduce multiple tree failures due to a common parent, different parents are chosen as often as possible. Once the selection of candidates for each tree is done, attachment requests are sent out and each tree will operate mostly independently. Figure 1: Example of network topology used for simulation. The setup used in the experiments is an extended version of this example. ternet implementation (see e.g. [10]), they are not directly relevant to the scope of this paper. Our control protocol is completely distributed, except for an approximate list of connected peers maintained by the source. Besides accepting new peers, the protocol maintains the multicast trees when peers leave or are disconnected during the session. A peer may leave ungracefully due to network shutdown or system failure. To stay connected, peers need to monitor the state of their ancestors and may decide to reconnect if they detect traffic interruptions. 3.1 Joining Each joining host discovers the address of the source of the video stream by a directory service such as a website and contacts this peer to obtain a list of hosts randomly chosen among connected members. The joining peer contacts all the members of the list and waits for replies. It then determines a candidate parent for each tree. If the latter has enough available bandwidth at the time of the request, it accepts the host and starts forwarding video packets. This process is called 6-way handshaking and consists of one message exchange with the source and 2 message exchanges with the candidate parent Partial list of connected peers To reduce control overhead, the source adjusts the size of the list sent to joining peers according to the current group size and the number of multicast trees. At the beginning of the session, the group size is small and the list size increases linearly. When the group size reaches a certain point, the list size increases logarithmically. For multiple trees, the source sends a larger list to allow a joining peer to exploit path diversity Parent selection algorithm Parents which have enough throughput to support an additional peer are selected based on their proximity to the source. Several other criteria can also be considered in this process. For example, the amount of available throughput, the round-trip time (RTT) or geographical proximity. Similarly to the results reported in [21], we found that the proximity to the source gave good performance as will be explained in more detail below. To make use of path diversity Periodic message exchange Once a peer is connected, it will inform its parents of its presence by transmitting periodic hello messages. These messages are also used to propagate topology information such as the subtree size of a peer. Reception of a hello message generates an immediate response intended to confirm the parent s presence. 3.2 Node disconnection Ungraceful leaves occur when a peer leaves the group without notice which may cause disconnection of the peer s descendants from the group. When a host leaves, it stops forwarding video packets and is unresponsive to probing. Once a peer detects a parent disconnection it will try to reconnect to the tree by choosing one of its other parents. If this fast recovery mechanism fails, the peer has to contact the source to get a new list of candidate parents. While the host reconnects, retransmission requests are issued over the other multicast trees to recover missing video packets. We will explain this retransmission procedure in detail in Sec. 6. A list of extra links to other potential parents can also be maintained by each peer. This list is initialized during the joining process. After a host joins the different multicast trees, it keeps the list of remaining available hosts to construct a pool of extra links. If the number of hosts in the pool falls below a certain threshold the list is updated using a gossip algorithm. Extra links are not reserved resources but do however indicate which hosts are available for potential reconnection. The performance of this other rejoin procedure is not presented in this paper but shows that the procedure should be lightweight in order to avoid creating additional control overhead. We observed that the time threshold necessary to detect parent disconnections may be large (approximately 1-2 seconds), as it is important to avoid false detections, and since losses will be compensated by retransmissions requested during reconnection. Detecting the disconnection of a child is less influential to the overall performance as it only results in a temporary waste of the parent s network resources. Since the penalty of a false child leave detection is high, a longer time interval is used. When a child leave is detected, parents will remove it from their forwarding table. 3.3 Loop avoidance Once a peer looses its connection to one of the trees, it will try to rejoin this tree and will send out new attachment requests. During this process it may try to attach to one of its descendants. This will create a loop which will eventually starve the whole subtree. To prevent this kind of event, each peer keeps the list of its ancestors, which are peers in the path from the peer to the source. Attachment requests issued by an ancestor are rejected. Note that this happens independently over each tree and that the hierarchy between peers can and should be different over different trees. This loop avoidance mechanism does not require global knowledge of the tree but only of a small subset of its nodes, hence additional memory and processing power are negligible. 41

4 3.4 Simulation setup To evaluate the performance of the protocol we carry out experiments over a network simulated in NS-2 [6]. Figure 1 illustrates the star-shape topology of the network supporting the hosts. Simulations are run over a similar topology with 1000 nodes, 750 of which are placed at the extremity of the network. The actual number of peers participating in each simulation is 300, placed evenly among the edge-nodes. The backbone links are sufficiently provisioned so that congestion only occurs on the links connecting the peers to the network. The latency of each link is 5 ms, and the diameter of the network is 10 hops. Losses are only due to congestion and overflowing queues, and transmission errors due to the presence of ISP boundaries or potential wireless last-hop links are ignored. The bandwidth of the peers reflects today s available ADSL network access technology. The bandwidth distribution is given in Tab. 1. The degree indicates the number of children a parent transmitting a video stream encoded at 220 kbps can potentially support. This calculation includes a bit-rate reserve of 33% to account for instantaneous rate fluctuations, retransmissions and control overhead. We refer to this bit-rate reserve as overprovisioning throughout this paper. The overprovisioning factor will be justified in more detail in Sec. 4. The distribution is similar to the findings reported in [21]. Note that more than half of the peers do not have enough bandwidth to forward the video stream. This makes them free-riders in a system with only 1 distribution tree to forward data. When more trees are used, the video stream is divided into several smaller sub-streams and these peers may forward data to one or several peers. Table 1: Distribution of peer bandwidth. The degree is computed according to a video stream encoded at 220 kbps with an over-provisioning factor of 33%. Downlink Uplink Degree Percentage 512 kbps 256 kbps 0 56% 3 Mbps 384 kbps 1 21% 1.5 Mbps 896 kbps 3 9% 20 Mbps 2 Mbps 6 3% 20 Mbps 5 Mbps 17 11% In the experiments, the dynamic behavior of peers is modelled as follows. A flash crowd is simulated by letting all the peers request the video during the first minute of the video session. During the remaining 14 minutes, peers join and leave the session ungracefully, following a random Poisson process. Peers remain on for 4.5 minutes and off for 30 seconds, on average. The 10s video sequence Mother and Daughter is transmitted from the source to the peers. It is looped enough times to cover the whole session. The video stream is encoded with H.264 [12] at a constant quality and the encoding rate is approximately 220 kbps. We use the freely available version of the codec JM 9.2 maintained by the Video Coding Expert Group (VCEG)[4]. The encoding structure will be described in detail in Sec.5. Each video frame is packetized into UDP packets. Frames exceeding the maximum transmission unit size are fragmented before packetization. When the simulation begins, the source peer multicasts the video by sending UDP packets over the different multicast trees in round-robin fashion. 3.5 Protocol evaluation We begin by analyzing the performance of the protocol connection and re-connection process, then compare the results obtained for control overhead and video quality over different numbers of multicast trees. Cumulative distribution function minimum hop minimum rtt Join time (s) Cumulative distribution function minimum hop minimum rtt Rejoin time (s) Figure 2: Distribution of the time needed to join the multicast trees (left) and to rejoin a tree after a parent disconnection (right). Figure 2 shows the cumulative distribution function (cdf) of the join time, i.e. the time necessary for a node to join a multicast tree; the cdf of the rejoin time is also plotted. This is the time a node takes to rejoin a tree once the disconnection of its parent has been detected. Results are shown for two different parent selection algorithms, the first one is based on the proximity to the source and denoted by minimum hop, the second is based on RTT. The joining procedure takes an average of 0.63 second, parent leaves are detected in less than 2 seconds and it takes an additional 0.70 second to rejoin, on average. In terms of latency, both algorithms show little difference. However, this does not translate to equivalent overall performance. The minimum hop parent selection leads to more stable trees which translates into higher video quality over time. In our experiments, 80% more rejoin events were observed when the minimum RTT method was chosen. This justifies the use of minimum hop as our parent selection algorithm. In Fig. 3, the control traffic overhead and the average video quality is shown when video is multicast over different number of trees. The percentage of overhead obviously depends on the encoding rate of the video. The video quality is measured by collecting the mean square error (MSE) between the luminance of the decoded video signal and the original video signal. The MSE is translated into Peak Signal to Noise Ratio (PSNR), and represented in db. This metric will be used throughout the paper. A difference in PSNR exceeding 1 db is considered visually significant. In Fig. 3, PSNR is averaged both over time and over the different peers. As illustrated, as long as 4 multiple trees or less are maintained, the overhead stays between 2 and 4.5% of the total 42

5 Percentage of control overhead 8% 7% 6% 5% 4% 3% 2% 1% 0% Number of trees Average quality Control overhead Figure 3: Percentage of control protocol overhead and average video quality for different numbers of multicast trees. traffic observed on the downlink of the peers. When more trees are used, the control traffic increases, this is due a higher frequency of parent leave generating more traffic exchange between the peers. When only 1 tree is maintained, the control traffic is higher than expected. In this case, a parent leave cannot be accommodated by one of the other remaining parents, as it happens when more trees are used. This creates additional control traffic. When video is distributed over a larger number of multiple trees, the effect of an ungraceful leave is less important as children have several parents which they can use for rejoining. On the other hand, maintaining more trees increases the probability of a parent disconnection and requires more control traffic. In Fig. 3, this tradeoff is shown in terms of the average video quality, as a function of the number of multiple multicast trees maintained by the protocol. In this environment, the optimal tradeoff between robustness and congestion is obtained when 2 multiple trees are used, and the performance observed for 3 and 4 trees is close to the optimal. Depending on the rate of the video, on whether or not retransmissions are requested and on the dynamic peer behavior, this optimal could occur for a different number of trees. However, we believe a small number of distribution trees is enough to guarantee good performance. In most of the experiments presented in the following video will be transmitted over 4 multiple trees. 4. VIDEO DISTORTION MODEL For live video streaming applications, video packets are transmitted over the network and need to meet a playout deadline. Decoded video quality at the receiver is therefore affected by two factors: distortion introduced by the encoder, denoted by D enc, and distortion due to packet loss or late arrivals, denoted by D loss. Assuming an additive relation of these two independent factors, a video distortion model was derived in [20]. The decoded video distortion, PSNR (db) D dec, is given by: D dec = D enc + D loss, (1) D enc = D 0 + θ/(r R 0), (2) D loss = κ(p r +(1 P r)e (C R)T/L ), (3) n C = C i, (4) R = i=1 n R i (5) i=1 In (2), R is the total rate of the video stream, and the parameters D 0, θ and R 0 are estimated from empirical ratedistortion curves via regression techniques. The second distortion term, D loss, depends linearly on the packet loss rate. The scaling factor κ indicates the sensitivity of the stream to losses. It depends on the encoding structure and on the amount of motion present in the sequence. The other factor reflects the combined rate of random losses and late arrivals. P r is the random packet loss rate and T is the time within which each packet should reach the receiver (typically a few hundred milliseconds). C is the aggregate available throughput of the network paths over which the video is transmitted and L is the average packet size. In (4), the throughput, C, isexpressedasthesumofthe available throughput between the receiver and its parents on the n different multicast trees. Likewise, in (5), R i is the fraction of the video stream transmitted over the i th multicast tree. Typically, the same throughput will be reserved for a receiver on each tree and the video will be divided equally among the trees: R i = R/n and C i = C/n. This model reflects the impact of the rate on video distortion. At lower rates, reconstructed video quality is limited by coarse quantization, whereas at high rates, more packets are delayed beyond their playout deadline due to network congestion. For live video steaming in a bandwidth-limited environment, we therefore expect to achieve maximum decoded quality for some intermediate rate. As an illustration, we collect the decoded video quality of a sequences transmitted to a peer using different numbers of multicast trees. The parents of the peer each have a total available uplink throughput of 660 kbps. A fourth of this capacity (165 kbps) is reserved by each parent to the peer to carry the video traffic, the rest being used to serve other peers which share the same parent. Figure 4 illustrates the decoded video quality for the sequence Mother and Daughter encoded at different rates. The fitted model is shown along with experimental points. In this experiment the playout deadline is 500 ms. The decrease in quality due to congestion is illustrated by the bell shape of the curve representing decoded video quality. In this experiment the decrease occurs when the rate of the video reaches approximately 85% of the available throughput. For P2P streaming, it is essential to limit the amount of self-inflicted congestion created by the media streams. Indeed, as there might be a large number of intermediate end-hosts separating a peer from the source, any increase in network congestion, may be reflected multiplicatively in the total end-to-end delay. Combined with physical link latency this delay may cause some packets to miss their playout deadline, resulting in decoding errors and a decrease in decoded quality. Hence, for fine tuning, congestion may be reduced by the following methods: increasing the amount of 43

6 43 42 PSNR (db) tree 2 trees 3 trees Rate kbps Figure 5: SI frames share the instant refresh properties of I frames but are only sent after a frame is lost. Figure 4: Streaming performance for the sequence Mother and Daughter transmitted to a receiver over a different number of multicast trees. The experimental results are shown together with the model, represented by a solid line. The parameters of the model are D 0 =0.49, θ = 1222, R 0 =10.39kbps, κ =185, T =0.5s and L =1.5kbit. over-provisioning by reserving a larger capacity C i on each of the multicast trees between a parent and a child; decreasing the encoding rate of the video R. When video is pre-encoded, as is often the case, it is not always possible to reduce the streaming rate. Thus, it may be necessary to employ more over-provisioning, this in turn may decrease the number of hosts supported by the P2P system. The model can be used to determine the right amount of overprovisioning. For example, given the rate of the video, the number of trees, and the playout deadline, we can determine the throughput necessary on each tree to keep D loss under a given threshold. For video encoding rates of 100 kbps or more, and playout deadlines of no less than 300 ms, C i =1.33 R i is enough to avoid any noticeable congestion. 5. ERROR-RESILIENT STREAMING To avoid interruptions in a streaming application, decoding errors due to packet loss or to late arrivals are concealed by freezing the previous frame. Because of the predictive nature of compressed video, this decoding error propagates to subsequent frames until an independently encoded frame (e.g. an I frame) is received. In this section, we describe how error propagation can be stopped adaptively between peers by using SP and SI pictures. 5.1 SP and SI frames SP and SI pictures types are part of the latest video coding standard H.264 [12]. These new types of predictively/intra coded pictures were initially proposed in 2001 by Karczewicz and Kurceren, as a solution for error resilience, bitstream switching and random access [15, 16]. The main advantage of SP and SI pictures is that they can be used interchangeably in a video stream without creating any decoding error or drift. This leads to interesting applications, such as stopping error propagation adaptively by refreseshing a pre- Figure 6: GOP structures used for streaming with SP and SI frames and for periodic I frame insertion. diction chain, as depicted in Fig. 5. In such a scheme, a pair of SP and SI pictures are periodically generated during the encoding of a sequence. For streaming, the sender may choose which picture of the pair to transmit. Because of their larger size, SI frames should, ideally, only be sent when their instant refresh properties are needed. Therefore, SI frames are transmitted instead of SP frames only when a decoding error at the receiver is detected, to stop error propagation. This differs from traditional video streaming where large I frames are transmitted periodically whether decoding errors occur or not. 5.2 Compression efficiency The video encoding structure shown at the top of Fig. 6 was chosen for the streaming experiments presented in this paper. The number of frames in a Groups of Pictures (GOP) is 16, with one SP frame (and its corresponding SI frame) per GOP and 3 B frames between P frames. This ensures good error resilience properties and allows to easily scale down the frame rate by 2 or even 4 if needed. Links to video sequences compressed with this coding structure at different rates, as well as rate-distortion preambles characterizing the size and quality of the frames are made publicly available [2] 1. We also evaluated the performance of traditional streaming with periodic I frames. The structure of the GOP used in this case is similar and is illustrated at the bottom of Fig. 6. As the size of SP frames is less than that of I frames, for limited loss rates, the bit rate savings obtained by not transmitting unnecessary I frames may reach up to 25%. This is illustrated in Fig. 7, which shows the compression efficiency of GOPs containing SP or SI frames compared to GOPs with periodic I frames. When loss rates are high, the rate-distortion performance decreases and may be worse than streaming with I frames, as the size of SI frames exceeds that of I frames. 1 The preambles also indicate the distortion values obtained by concealing a frame with any other frame of the stream, allowing to simulate realistically video streaming without the overhead of encoding and decoding. 44

7 PSNR (db) periodic I frames 37 periodic SP frames periodic SI frames Rate (Kbps) Figure 7: Rate-distortion performance with periodic I frame, SP frame or SI frame insertion, for Mother and Daughter. 5.3 Extension to the P2P case In Sec. 5.1, we explained how a sender possessing a pair of corresponding SP and SI pictures could achieve better ratedistortion performance by choosing which one to transmit adaptively. We propose to extend this technique to P2P streaming where a sender has not necessarily received both the SP and the SI pictures from its ancestors. The complete details of the encoding/decoding process of SP and SI frames are beyond the scope of this paper. They can be found in [16]. One interesting aspect of the decoding of SP frames is that it allows the corresponding SI picture to be regenerated with very limited additional complexity 2. As a consequence, in the P2P streaming scenario, each peer which receives and decodes an SP frame correctly may also create the corresponding SI frame. Therefore, it can subsequently use the resulting SP and SI picture pair adaptively to either reduce the bit-rate or stop the error propagation its descendant might be experiencing. This technique allows the adaptive streaming to take place not only between the source and its direct descendants but also further down in the tree, as long as SP frames are received and decoded correctly by peers. As larger video frames usually span several transport layer packets, different parts of SP and/or SI frames will usually be transmitted by different parents, when different multicast trees are maintained. As all the packets of one of the pictures are needed to continue decoding, it is necessary for these different peers to coordinate and choose to transmit the same picture to their child. This is done by signalling from the child to the parents the periods when error propagation occurs. During these periods SI frames will be transmitted. 2 This requires the use of an additional entropy encoder. Note, however, that the reverse is not true: the SP frame cannot be recovered from the SI frame without a complete video encoder. 5.4 Experimental results We analyze the benefits of SP and SI frames for video streaming with low latency over the network described in Sec. 3. The video stream is encoded at approximately 220 kbps and the maximum tolerated delay between the time a frame is available at the source and the time it is played by the peers is 2 seconds. As a comparison, results are also collected for the same sequence of events for a video stream with periodic I frames, also encoded at the same rate. For the bandwidth distribution indicated in Tab. 1, the dynamic behavior of the hosts and this tight delay constraint, the two video streams represent the highest supportable video quality for the two different encoding structures considered. Cumulative distribution function periodic I frames SP/SI frames PSNR (db) Figure 8: Video quality performance for the 300 peers for different encoding structures. Figure 8 shows the cdf of the peers decoded video quality. The average quality for the SP and SI frames is 40.3 db, 0.8 db higher than for streaming with periodic I frames. The distributions show that 99% of the hosts benefit from this novel encoding structure. Furthermore, for more than 25% of the hosts the performance gain exceeds 1 db. This gain is due to the better rate-distortion-congestion performance of the video stream containing SP/SI frames. As the number of node disconnections is limited, most of the time, SP frames are transmitted in lieu of SI frames. This reduces the overall bit-rate and allows for a higher sustainable quality. Moreover, instantaneous bit-rate fluctuation are reduced when SP frames are used instead of I frames. This results in less congestion and better end-to-end delay performance. More detailed experimental results describing similar effects were reported in our prior work [19]. 6. OPTIMIZED RETRANSMISSIONS The dynamic behavior of peers is one of the most challenging aspects of peer-to-peer live streaming. In particular, a peer must continuously monitor the state of its parent(s) and immediately regain connectivity via another peer when a leave is detected. As analyzed in Sec. 3, this process can take a few seconds, a time during which none of the packets transmitted over the affected tree will reach the peer. The corresponding loss statistics are obviously time-variant as there might be some long periods during which a host 45

8 is fully connected and experiences no loss, and other times when a large portion of packets are missing. Hence, FEC or multiple description coding which seek to protect a user for a specific loss level may not be appropriate as they will create superfluous redundancy most of the time and might be overwhelmed when losses occur. However, as reconnections are not instantaneous, in order to maintain high decoded quality, missing portions of the video need to be recovered while the peer is rejoining. In this section, we describe how a peer can mitigate the video quality degradation during the rejoining process by requesting retransmissions to its other parents. Retransmission requests will place an additional burden on the uplink of peers already forwarding a portion of the video to 1 or possibly many children. This increase in congestion is more important when there are few multicast trees as a larger portion of the video will be requested from fewer parents. Regardless of the number of multicast trees, optimal quality will be achieved when enough packets are requested to maintain high video quality while avoiding a too severe increase in terms of end-to-end delay. Our goal is to determine an optimal retransmission schedule for missing packets of a video stream. This schedule would indicate which packets of the stream will be requested to maximize the decoded video quality at the receiver while limiting the congestion created on the network. 6.1 Congestion-distortion optimized scheduler In [18], we presented and analyzed the performance of a sender-driven congestion-distortion-optimized scheduler (Co- DiO) which determines how to minimize video distortion for a given level of congestion. The benefits of using congestion and distortion as metrics rather than rate and distortion, as for example in [9], are two-fold. End-to-end delay (i.e. congestion) is inherently adaptive to time-varying network conditions. In addition, it reflects better the impact of a user operating on a bandwidth-limited network. To find an optimized schedule, CoDiO selects the most important packets in terms of video distortion reduction, and requests them in an order which minimizes the congestion created on the network. For example, I or SI frames are requested in priority whereas B frames might not be retransmitted at all. In addition, CoDiO avoids requesting packets in large bursts as this has the worse effect on the queuing delay. In the following, we describe how to extend CoDiO to the P2P receiver-driven scenario. We present a low-complexity CoDiO scheduler which performs retransmission scheduling. We show how to select the most important packets for retransmission, how to limit congestion and analyze the performance of the CoDiO retransmission scheduler. 6.2 Computing the importance of packets After detecting a parent disconnection, a peer can determine a list of missing packets and iteratively select the most important ones to request. This choice should depend on the time at which packets are due, and on the contribution of each packet to the overall video quality. CoDiO proceeds by discarding packets already past their due time and by computing an approximate sensitivity of the video distortion to the remaining packets. Given a set of received frames an approximate video distortion may be computed in the following manner. Assuming copy error concealment is used, where an undecodable frame is replaced with the nearest correctly decoded frame, and given the encoding structure of the video, we can find which frames will be shown as a function of time. Let D(s, c(s)) denote the approximate distortion resulting from showing frame c(s) instead of frame s. We assume the table D, pre-computed offline for a generic video sequence, is available at each peer. Each display outcome is associated with the appropriate pre-computed distortion value and the resulting approximate video quality is computed over several frames: D = D(s, c(s)) (6) s It is then straightforward to determine, the sensitivity of the average distortion to a single frame, and this sensitivity is extended to each of the packets composing the frame. Retransmission requests are sent out following the order of importance. The difficulty resides in determining which packets will or will not be received before their playout deadline. Ideally, probabilities for these events should be computed, based on delay distributions, and the resulting expected distortion could be computed, as described in [14], by combining these different probabilities. In our scheduler, we use a much simpler techniques and only distinguish lost packets and received packets. We consider all the parents of a peer will forward successfully the packets that are transmitted on their respective multicast tree. The packets sent on the tree(s) of the disconnected parent(s) are considered missing except if the peer has requested them from another parent within a short time interval (we arbitrarily chose 200ms in our experiments). This classification reduces the number of possible decoding states to one, thus we know which frames will be displayed over the horizon considered. 6.3 Limiting congestion In the scenario considered, the available transmission rate between a sending peer and a receiving peer depends on several factors, such as the uplink throughput of the sending peer, the number of other hosts served by this peer and the rate of the video stream. Given these parameters, the available throughput C i may be computed and the average endto-end delay over a certain time horizon can be estimated for a given retransmission schedule. However, precise estimation would require modelling the delay distribution of the path between sender and receiver. As a low-complexity alternative, we limit the number of unacknowledged retransmission requests from a peer to each of its parents. As an unacknowledged retransmission request represents a packet being transmitted or processed between the two peers, these packets contribute to end-to-end delay hence to congestion. The tradeoff between the rate of the retransmissions and the amount of congestion created on the bottleneck links can be set by determining the optimal number of unacknowledged packets tolerated between a sender and a receiver This optimization is carried out experimentally in the following. 6.4 Experimental results Influence on video quality We first analyze the influence of retransmissions on the video quality. Specifically, we would like to know to what ex- 46

9 40 100% PSNR(dB) % 80% Connected hosts % 26 Time (second) % PSNR(dB) 38 90% % 26 Time (second) Connected hosts Figure 9: Video quality (solid line) and percentage of fully connected peers (dashed line) as a function of time for 299 peers when a host close to the source leaves. Results are shown in the absence of retransmission (top) and when the maximum number of unacknowledged retransmissions on each tree is 4. tent retransmissions mitigate the quality degradation which occurs when the parent of a peer leaves. To maximize the effect of the disconnection, in the network scenario specified in Sec. 3, we let the 300 peers join and disable disconnections. When a steady state is reached, we disconnect a host close to the source in one of the trees. In this experiment, 4 multicast trees are used to transmit the video and the maximum number of unacknowledged retransmissions requests on each tree is 2. The performance is shown in Fig. 9, as a function of time, in terms of the video quality. The average video quality is taken over all 299 peers. Performance is also shown when no retransmissions are allowed. The host disconnection occurs at time 18 s, and takes about 1.5 s to be detected. This can be deduced from the fact that connectivity is reported directly by the peers. As illustrated in Fig. 9, about 30% of the hosts are affected by the disconnection. At time 21.5 s, all the affected peers have recovered. Similar behavior is observed with and without retransmissions. However, the video quality during the rejoin time is very different in both cases. When the CoDiO retransmission scheduler is used, the video quality remains almost constant over time, a large majority of the missing frames are recovered. In the other case, quality drops by approximately 2-3 db during the reconnection time Influence on congestion We analyze the impact of retransmissions on congestion and show how to determine the optimal tradeoff between congestion and the number of retransmission requests. Tab. (2) indicates the average decoded video quality for 300 peers operating following the scenario described in Sec. 3. Video is transmitted over four multicast trees. The numbers reported in the figure show the gain on the total average quality which can be achieved through the use of retransmis- Table 2: Average decoded video quality for different numbers of retransmissions. Maximum unacknowledged retransmissions per tree PSNR db db db db db sions for this scenario. As illustrated, two simultaneous retransmissions requests per tree allow to gain approximately 0.6 db. Although this gain is modest, we stress that the impact of visual quality discussed in the previous section might be much larger. The small performance gap is due to the fact that during the 15 minutes event, the number of disconnections affecting each peer might not be very large. When more than two simultaneous retransmissions are allowed, the performance degrades slightly for 3 retransmissions and more severely for 8 retransmissions. This indicates congestion is disrupting the video transmissions. Please note that the optimal number of simultaneous retransmissions increases with the number of multicast trees. 7. CONCLUSIONS Live P2P multicast streaming is constrained by the dynamic behavior of hosts and by limited uplink throughput. Dynamic control protocols are needed to support heterogeneous peers and react rapidly to node disconnections. 47

10 We present a new multicast control protocol which builds and maintains multiple trees to transmit live video to a large population of peers, and demonstrate how video encoding, streaming and scheduling techniques developed recently further enhance the performance of the system. A ratedistortion model is proposed to analyze the tradeoff between self-inflicted congestion and video quality and is used to determine the amount of over-provisioning necessary when low latency is required. H.264 SP and SI frames are incorporated into the video stream to provide adaptive error-resiliency capability and achieve bit-rate saving gains of up to 25% compared to traditional video streams. Last, retransmissions of missing packets are requested in a congestion-distortionoptimized fashion which selects the most important packets in terms of video quality while limiting the effect on endto-end delay. For simulations with up to 300 peers, achieve typical end-to-end delays of 1 sec, and a stable video quality with less than 2.5% of frames lost to playout interruptions. 8. REFERENCES [1] Coolstreaming. seen on Aug [2] Encoded sequences with SP/SI frames. esetton/sequences.htm seen on Aug [3] ESM. seen on Aug [4] H.264/AVC Reference Software. seen on Aug [5] PPLive. seen on Aug [6] The Network Simulator - ns-2. seen on Aug [7] J. Apostolopoulos, T. Wong, W. Tan, and S. Wee. On multiple description streaming with content delivery networks. Proceedings Infocom, New York, USA, 3: , June [8] S. Banerjee, B. Bhattacharjee, and C. Kommareddy. Scalable application layer multicast. Proceedings ACM Sigcomm, Pittsburgh, USA, pages , Aug [9] P. Chou and Z. Miao. Rate-distortion optimized streaming of packetized media. Microsoft Research Technical Report MSR-TR , Feb [10] Y. Chu, A. Ganjam, T. Ng, S. Rao, K. Sripanidkulchai, J. Zhan, and H. Zhang. Early experience with an internet broadcast system based on overlay multicast. Proceedings of USENIX 04, page , June [11] Y. Chu, S. Gao, and H. Zhang. A case for end system multicast. Proceedings ACM Sigmetrics, Santa Clara, USA, June [12] ITU-T and ISO/IEC JTC 1. Advanced Video Coding for Generic Audiovisual services, ITU-T Recommendation H ISO/IEC (AVC), [13] J. Jannotti, D. Gifford, K. Johnson, M. Kaashoek, and J. O. Jr. Overcast: Reliable multicasting with an overlay network. USENIX Symposium on Operation Systems Design and Implementation, San Diego, USA, Oct [14] M. Kalman, P. Ramanathan, and B. Girod. Rate-distortion optimized streaming with multiple deadlines. Proc. International Conference on Image Processing, Barcelona, Spain, Sept [15] M. Karczewicz and R. Kurceren. A Proposal for SP-Frames. Video Coding Experts Group Meeting,, Doc. VCEG-L-27, Eibsee, Germany, Jan [16] M. Karczewicz and R. Kurceren. The SP- and SI-frames design for H.264/AVC. IEEE Trans. CSVT, 13(7): , July [17] V. N. Padmanabhan, H. J. Wang, and P. A. Chou. Resilient peer-to-peer streaming. IEEE International Conference on Network Protocols, Atlanta, USA, Nov [18] E. Setton and B. Girod. Congestion-Distortion Optimized Scheduling of Video. Multimedia Signal Processing Workshop (MMSP), Siena, Italy, pages , Oct [19] E. Setton and B. Girod. Video streaming with SP and SI frames. Proc. Visual Communications and Image Processing, Beijing, China, July [20] E. Setton, X. Zhu, and B. Girod. Minimizing distortion for multipath video streaming over ad hoc networks. International Conference on Image Processing, Singapore, pages , Oct [21] K. Sripanidkulchai, A. Ganjam, B. Maggs, and H. Zhang. The feasibility of supporting largescale live streaming applications with dynamic application endpoints. Proceedings SIGCOMM 04, Portland, USA, Aug [22] D. Tran, K. Hua, and T. Do. Zigzag: An efficient peer-to-peer scheme for media streaming. Proceedings Infocom, San Francisco, USA, 2: , Mar [23] X. Zhang, J. Liu, B. Li, and T.-S. P. Yum. Donet/coolstreaming: A data-driven overlay network for live media streaming. Proceedings IEEE Infocom, Miami, USA, Feb