Synchronized Multicast Media Streaming employing Server-Client Coordinated Adaptive Playout and Error Control

Transcription

1 Synchronzed Multcast Meda Streamng employng Server-Clent Coordnated Adaptve Playout and Error Control Jnyong Jo and JongWon Km Networked Meda Lab., Department of Informaton and Communcatons, Kwang-Ju Insttute of Scence and Technology (K-JIST), Gwangju, , Korea. Abstract A new nter-clent synchronzaton framework employng a server-clent coordnated adaptve playout and error control toward one-to-many (.e., multcast) meda streamng s dscussed n ths paper. The proposed adaptve playout mechansm controls the playout speed of audo and vdeo by adoptng the tme-scale modfcaton of audo. Based on the overall synchronzaton status as well as the buffer occupancy level, the playout speed of each clent s manpulated wthn a perceptually tolerable range. Addtonally, the server mplctly helps ncreasng the tme avalable for retransmsson whle the clents perform an nteractve error recovery wth the assstance of adaptve playout control. By coordnatng the playout speed of each clent, the nter-clent synchronzaton wth respect to the target presentaton tme s smoothly acheved. The cumulatve feedback for retransmsson asssted by both playout speed manpulaton at clents and mplct help of a streamng server s also conducted to ncrease the relablty. Furthermore, RTCP-compatble sgnalng between the server and group-clents s performed, where the exchange of controllng message s restrcted. The network-smulator based smulatons show that the proposed framework can reduce the playout dscontnuty wthout degradng the meda qualty, and thus mtgate the clent heterogenety. Key words: Multcast meda streamng, nter-clent synchronzaton, source specfc multcast, and adaptve playout control. Emal address: {jny92, jongwon}@kjst.ac.kr (Jnyong Jo and JongWon Km). URL: (Jnyong Jo and JongWon Km). Preprnt submtted to Computer Networks 17 July 2002

2 1 Introducton Wth the enormous growth of the Internet, more and more applcatons are dstrbutng meda contents to worldwde users. The average sze of meda contents transferred over the Internet are exponentally ncreasng everyday. It can be partly supported by the upcomng broadband network nfrastructure. However, to provde broadband meda contents relably and scalably, an effcent utlzaton and proper management of lmted network bandwdth become ncreasngly mportant. As a vable opton to save the precous bandwdth, multcast provdes mmense merts n desgnng multmeda applcatons. Especally for contnuous streamng meda (e.g., audo and vdeo), t allow us to support a large set of clents smultaneously. The major problem wth multcast s that, due to the manageablty and scalablty problems, the ubqutous deployment of IP multcast on the Internet s expected only after several years to come [1]. Realzng an effectve meda streamng over the IP multcast faces lots of challenges. It requres feasble sgnalng/transport protocols, stable multcastenabled networks, and network-adaptve applcatons to dstrbute the broadband meda n an acceptable qualty [2]. For the network sde, source specfc multcast (SSM) model [3] allevates several complcatons (e.g., by restrctng applcaton scenaro) of conventonal any source multcast (ASM), especally for one-to-many meda streamng. For the protocol sde, real-tme transport protocol par (RTP/RTCP [4]) can provde nteroperable/montored real-tme transport channel over the IP networks. RTP/RTCP n tself does not guarantee qualty of servce (QoS) to streamng meda applcatons but acts as a helper. Streamng meda applcatons at the server and clents are responsble to adaptvely take charge of network congeston/error/delay varatons and system resource lmtatons. To overcome the network varatons, streamng meda applcatons are deployng error controls as well as rate (e.g., congeston and flow) controls. Relable transmsson of RTP/UDP packets needs error controls based on automatc repeat request (ARQ), forward error correcton (FEC), or ther hybrd. In the context of relable multcast, buldng blocks are proposed based on the NACK (negatve acknowledgement), tree-structured ACK, and FEC schemes [5]. It s also mportant to deploy an effectve congeston control scheme to cope wth the network congeston whle achevng the TCP-frendlness [6, 7]. The playout of streamng meda needs to be synchronzed accordng to the assocated tmng nformaton (e.g., tmestamp per each packet). The goal of meda synchronzaton s to reconstruct the tmng relaton of meda contents at the clents. The notceable dsrupton (due to buffer underflow or system overload) and/or dscrepancy (due to playout tmng skews between dfferent meda objects) n the playout sgnfcantly degrades the playback qualty. 2

3 Presentaton tme dfference Clent Clent Server Group coordnaton Clent Inter-clent synchronzaton Network fluctuaton Rate, Loss, Delay(jtter) System dynamc Momentary CPU overload and etc Applcaton loss and Network loss Adaptve playout control Clent Clent Clent Network loss SSM-enabled IP network Error recovery Fg. 1. Multcast streamng problem scope and related ssues. These ntra-clent meda synchronzaton ncludng the lp synchronzaton has been consdered as an mportant ssue. To keep the synchronzaton, streamng meda applcatons have to deal wth the network delay jtter and loss that can cause the playout dsrupton. They also need to keep the smooth playout regardless of the exceptonal events at the clent system. In addton, we can expect to see dfferent clents are playng dfferent porton of meda as shown n Fg. 1 (denoted as presentaton tme dfference). Ths stuaton s happenng snce each clent s connected to the streamng server through paths of dfferent bandwdth, loss, and delay. The dfference n the capablty of clent system s another reason. Thus, we need to address the playback synchronzaton ssue n both ntra- and nter-clent aspects whle properly controllng the buffers at the clents. That s, an adaptve playout s requred to mantan the playout synchronzed despte of the network fluctuatons and system lmtatons [8 10]. Note that, compared to the multcast rate and error control ssues, multcast playback synchronzaton has been rarely dscussed. To cope wth the above challenges and provde hgh-qualty meda streamng, an adaptve multcast streamng framework has to be establshed. Thus, n order to reduce the playback dscontnuty and mtgate the heterogenety, we propose to establsh a synchronzed multcast streamng framework, where the synchronzed playout of all clents s adaptvely managed. Note that synchronzaton ssue becomes more mportant as the meda streamng goes broadband and hgh-qualty. In the proposed synchronzed streamng framework, each clent s requred to keep synchronzaton despte of the exceptonal system events as well as the network fluctuatons. To assst each clent for ths challenge, we propose to adaptvely control the playback speed of playout. By extendng the audo/speech adaptve playout wth tme-scale modfcaton n [11, 12] to audo/vdeo, the playback speed of clent can be vared. That s, based on the combned buffer (.e., network and applcaton buffers) oc- 3

4 cupancy level, the player at the clent s adaptvely expandng or contractng the playout wthn the range that t does not hurt the vewers perceptually. Thus, we can proactvely conduct the adaptve playback not only to reduce the playback dscontnuty but also to guarantee hgh-qualty playback wth flexble error controls 1. In summary, the proposed streamng framework conssts of 1) local playback adaptaton (guded by a playback factor) based on the combned buffer occupancy wth the error recovery support, 2) uncast RTCP feedback on the presentaton tme as well as the channel status, 3) nter-clent synchronzaton wth the ad from the server, and 4) cumulatve NACK-based error recovery wth the assstance of adaptve playout control. More specfcally, each clent locally controls the playback speed to prevent buffer overflow/underflow (subsequently to prevent playback dscontnuty) and to assst the delayconstraned retransmsson attempt f allowed. Ths local adaptaton s then revewed at the server by aggregatng clent feedbacks and the server wll ssue target presentaton tme to coordnate and synchronze all the clents. Cumulatve NACK-based error recovery s also asssted by the adaptve playback to secure enough tme for request and reply of retransmsson. Furthermore, to reduce unnecessary feedbacks, whether to request retransmsson or not s selectvely determned. In ths paper, we are nterested n verfyng and evaluatng the effectveness of adaptve playout for the proposed framework. We choose to evaluate the proposed framework by conductng smple yet extensve network smulatons. Results show that the proposed framework can reduce the playback dscontnuty wthout degradng the meda qualty whle enhancng the nter-clent synchronzaton by mtgatng the clent heterogenety. At the same tme, t can assst the retransmsson-based error recovery wth the adaptve playout and reduce bandwdth through the selectve retransmsson and cumulatve feedbacks. The rest of ths paper s organzed as follows. the proposed one-to-many meda streamng framework s ntroduced n Secton 2. In Secton 3, the core components of the framework are explaned subsequently n the order of the adaptve playout wth the audo tme-scale modfcaton, the local adaptaton for the buffer, the playout for the ntra-clent synchronzaton, the server-aded nterclent synchronzaton wth feedback, the adaptve playout control for the error recovery, and the coordnaton of adaptve playout controls. Secton 4 shows the smulaton setup and the verfcaton results of the proposed framework. After revewng related works n Secton 5, we conclude the paper n Secton 6. 1 We are restrctng our attenton only to the adaptve playout and error control, settng asde the ntegraton of congeston control to future works. 4

5 2 Synchronzed Multcast Meda Streamng Framework Multcast has been extensvely nvestgated to tackle the network bandwdth challenge. However, the global deployment of multcast s hardly accomplshed at present because of the nvolved complextes and lmted QoS capabltes. In ths paper, we focus on a pragmatc soluton to enhance feasblty of multcast meda streamng servce. That s, we attempt to propose the synchronzed multcast meda streamng framework that can mtgate the deployment complexty and enhance the servce qualty. The proposed framework wll adopt the SSM model [13, 14], because t offers an off-the-shelf soluton to overcome the deployment complextes of IP multcast. Furthermore, the RTP/RTCP over the SSM envronment s adopted to adjust the protocols to the SSM [15]. Server audo vdeo Multcast RTP Multcast RTCP Uncast Feedback Frames played Retransmsson buffer SSM-enabled IP network ARQ Clent 1 Network loss Clent 2. Clent ARQ Clent n compostor decoder (AD) demux Clent n Applcaton loss... VD scheduler montor tmer controller Fg. 2. The proposed synchronzed multcast meda streamng framework. Fg. 2 llustrates the overall framework of the proposed synchronzed multcast meda streamng, where the adaptve playout control for audo/vdeo streams (e.g., MPEG-2 and MPEG-4) s deployed wth the meda delvery through the SSM-enabled IP network. Multplexed audo/vdeo stream (e.g., MPEG-2 program stream (PS) [16]) s transmtted to all the clents joned through a PIM-SM (protocol ndependent multcast - sparse mode) routng. It s assumed that a sngle-layer-encoded stream 2 s delvered to moderate number of clents (e.g., around 100) wth recever buffers. Under ths knd of stuaton, one can expect clents to lose synchronzaton easly and play slghtly dfferent porton of the stream compared to the other clents. Note that we are assumng the lghtly coupled synchronzaton, where the server and clents need to be 2 For the sake of smplcty, nether layered encodng nor multcast congeston (or flow) control s ntegrated at ths stage. 5

6 synchronzed wthn an allowed range 3. Note also that ths s n comparson to the tghtly synchronzed playout employng the clock synchronzaton by the network tme protocol (NTP) [17] 4. The desred nter-clent synchronzaton s accomplshed n ths work by performng the playout adaptaton wth the audo tme-scale modfcaton [11, 18]. The synchronzed playout s asssted wth the feedback-based streamng control, whch also can encompass the rate and error controls. The smple retransmsson-based error recovery can be enhanced wth the explct help of playout control, whch tres to expand the tme avalable for retransmsson by slowng down the playback temporarly. Server can also help the error recovery mplctly by relaxng the target presentaton tme to that of the slowest clent to secure the tme for retransmsson. The clent can not do a multcast feedback, snce the neghborng routers for each clent are confgured by IGMPv3 (Internet Group Management Protocol, verson 3) [19] protocol to flter a source n the SSM. Instead, each clent uses a customzed uncast RTCP recever report (RR) whle the server multcasts a RTCP sender report (SR) to delver a control message [15]. The most mportant component of the proposed framework s the adaptve playout control, that helps us to mnmze the playout dscontnuty and to cope wth the network fluctuatons and system lmtatons. Prevous efforts on the adaptve playout control have focused only on the meda synchronzaton based on the presentaton tmng and they are confned to the uncast case. We, however, propose to deploy the adaptve playout mechansm n multcast envronment and explot ts flexblty advantage. That s, we explot the gan of adaptve playout for the nter-clent synchronzaton. The nter-clent synchronzaton coordnates the presentaton tmng dfferences among the clents n a multcast group. Inter-clent synchronzaton s necessary to provde hghqualty meda streamng servce wth the farness among sesson partcpants. Synchronzed playout of partcpatng clents forms the support foundaton on whch a server easly deploy ts network adaptaton mechansms (e.g., ncludng flow and congeston controls). That s, the necessary server-clent nteracton for the rate, error, and synchronzaton can be better accomplshed wth the adaptve playout of each clent. Also, feedback-based scheme wth RTCP-compatble sgnalng s proposed to acheve the synchronzaton wth smplcty. An effort on accommodatng the RTP/RTCP to the SSM envronment has been made n [15], whch generally deals the RTCP extensons for the 3 The allowed range depends on the latency requrement of applcaton and the capabltes of clents. 4 Wth the tme synchronzaton and gven presentaton tme, the nconsstency of the playout can be controlled. However, t may result n the ncrease of playback dscontnuty, whch becomes more evdent when the performance of clents are not homogeneous and subject to dverse network fluctuatons. 6

7 SSM. We extend t to RTCP-compatble sgnalng for the nter-clent synchronzaton. We also propose to recover losses based on cumulatve NACK-based retransmsson request. Although the retransmsson-based error recovery has shortcomngs lke long repar latency and lmted multcast scalablty, t stll remans as an attractve canddate for the error recovery because of ts smplcty and effcency. In the multcast envronment, retransmsson request/reply may ncrease network traffcs drastcally unless multcast transmsson s combned wth the rght feedback mploson control. Issues even become worse f the exchange of controllng message s restrcted as n the SSM envronment. To solve the scalablty problems of retransmsson-based error recovery for the SSM wth uncast feedback [15], we propose to adopt selectve retransmsson request smlar to [20 22] wth cumulatve NACKs to save feedback bandwdth. Furthermore, server-clent coordnated playout control helps to salvage the tme for retransmsson. Note that we ntentonally smplfed the multcast model and restrcted our attenton only to one-to-many streamng applcatons n ths paper. Thus, the proposed multcast meda streamng framework can mtgate the deployment complexty and provde the enhanced streamng qualty. 3 Adaptve Playout and Error Control 3.1 Adaptve Playout wth Audo Tme-scale Modfcaton To provde hgh-qualty vdeo streamng and ease the clent adaptaton to the network fluctuatons and the system lmtatons, the clent-based adaptve playout s adopted. As shown n the magnfed part of Fg. 2, the multplexed stream s stacked at the recever buffer of each clent to wat the decodng and playback. The stream s de-multplexed nto the respectve audo and vdeo decodng buffers, where the presentaton tmestamp (PTS) located at the header of MPEG-2 PS s recorded nto a scheduler. It also gets the current buffer occupancy level from a buffer montor. Based on the tmng and bufferng status, the scheduler controls the adaptve playout by expandng/contractng the playout. The buffer montor checks the sequence number of ncomng packets and determnes the loss from the network - t performs gap-based loss detecton. The assocated tmer of the controller provdes the requred tmng nformaton and controls the retransmsson. The man role of the adaptve playout control s to reduce the dscontnuty ncurred by packet over-/underflows and momentary CPU overload. An effectve playout adaptaton allows us to avod excessve packet droppngs at the applcaton, conceal the network fluctuatons and thus mnmze the degradaton of 7

8 perceptual meda qualty. Wth the audo tme-scale modfcaton [11, 18], we can sgnfcantly enhance the adaptaton capablty of the clent at the cost of ncreased computaton. To perform the adaptve playout wth the tme-scale modfcaton, we need to know the allowed rato wthn whch a player can manpulate the playback speed wthout beng detected by the user s percepton. Even though the allowed playout varaton dffers based on the type of audo (ncludng the slence), we assume that playout varaton up to 25% s usually unnotceable 5. In ths paper, we defne the playback factor ( p ) to specfy ths rato. Orgnal P 1 P 2 P 0 P 1 P 1 Waveform smlarty Waveform smlarty P 2 P 2 P2 P 1 Realgnment Realgnment P 1 P 2 Tme scale expanson Tme scale contracton Tme scale modfed P 1 P 2 Fg. 3. Packet based SOLA n tme-scale modfcaton. Wth the tme-scale modfcaton based on the Waveform Smlarty OverLap- Add (WSOLA) algorthm [18], a player can vary the playout speed. In Fg. 3, the packet verson of modfed WSOLA [11] for the tme-scale expanson and contracton s llustrated. For the tme-scale expanson, a template segment whch s longer than one ptch perod from the orgnal waveform s sampled and then smlar segment s searched based on the waveform smlarty. The found segments followed by the rest of the samples n the orgnal packet s overlapped wth the template segment and added to the tme-scale modfed packet. The tme-scale contracton s performed n a smlar way except that t narrows searchng regon wthn a packet dstance to compress the output waveform. 3.2 Local Playout Adaptaton Under the proposed framework, each clent s performng the playout adaptaton locally. The local playout s heavly ted wth the buffer control. It adapts to control the combned buffer occupancy 6 guded by the playback factor. 5 The playback speed should be changed wth cauton to keep the playout consstent and smooth. 6 The sze of recever network buffer s assumed to be much larger than the summaton of decoder and composton buffers. If ths assumpton becomes nvald, we need 8

9 p c P + s s P s P p p b S b H b R t b L b Fg. 4. Desred operaton of the proposed adaptve playout control. In general, the recever buffer sze b s of each clent s typcally constraned by the hardware complexty and the applcaton requrement on the delay and loss. When dealng wth number of clents, t wll be smple and much easer to coordnate the playbacks of whole clent wth larger sze buffers. However, when t comes to the hardware cost, smaller sze buffers wth elegant buffer control are preferred. In ths paper, we fx the buffer sze of all clents as small as possble but large enough to servce the worst case clent wth the largest round trp tme (RTT) and loss. Then, we denote the current buffer occupancy level of clent at tme t as b (t) and specfy two fxed thresholds for the recever buffer as shown n Fg. 4. They are lower lmt (b L ) and hgher lmt (b H ) thresholds. When b (t) falls under b L or exceeds b H, there s hgh rsk of buffer underflow or overflow, respectvely. We ntroduce another mplct threshold, namely retransmsson lmt (b R ), to gude the retransmsson-based error recovery. Wth ths, the controller determnes whether t request the retransmsson of lost packet or not. We wll cover ths threshold later n the error recovery part. b b S b H b R pc = Ps ( 1+ p) pc = Ps ( 1 p) b L 0 t ε c 0.5 ε t ε t εc 0.5ε t / p t εc 0.5ε t / p Fg. 5. Key roles of local playout adaptaton (the effect of network/system fluctuatons s gnored for llustraton purpose only). to consder the vrtual combned buffer that comprses all three buffers together. 9

10 As dscussed above, the local playout adaptaton attempts to manage b (t) between b L and b H. By helpng the buffer-level control wthn the range as shown n Fg. 5, the adaptve playout reduces the rsk of buffer underflow/overflows. By keepng the playback speed of clent at t, p c(t) wthn acceptable range P s (1 p ) P s (1 + p ) (.e., 1 ± p of ts normal playback speed P s ), we can prevent exceptonal events that may result n unnecessary packet dscard. Thus, wth the adaptve playout based on the tme-scale modfcaton, we can manage the buffer level more flexbly. 3.3 Adaptve Playout for Intra-clent Synchronzaton Another key role of local playout adaptaton at each clent s towards the synchronzed playout (ntra-clent synchronzaton). If there exsts notceable dscrepancy n the audo and ts correspondng vdeo (or temporary dsrupton of playout), t dsturbs the audence a lot. In ths paper, we assume that globally synchronzed tme reference s not avalable (.e., lghtly coupled synchronzaton). Thus, each clent s responsble to manage the synchronzaton based on ts own tme reference, namely local vrtual tme. The tme reference s guded by the PTS delvered wth meda packets. We also focus on the ntra-meda aspect of ntra-clent synchronzaton. Note that the ntra-clent synchronzaton ssue s closely ted wth the nter-clent synchronzaton to be explaned later. T, V c c T ( n) s SR( n ) SR( n + 1) T ( n + 1) c T c V c ε t ε c t Fg. 6. Server-aded clent playout adaptaton for both types of synchronzaton. Fg. 6 depcts the proposed playout operaton to mantan the playback close to a target presentaton tme. Intally, the vrtual presentaton tme v c(t) starts concurrently wth the start of each playout at t start and the tmer records the current presentaton tme of clent, T c(t start ). Note that each meda packet contans the PTS. Usng T c(t start ) as the reference, we can estmate the vrtual presentaton tme at t start + t. Each clent then compares ts current presentaton tme T c(t) to the vrtual one v c(t). The target range s controlled by ɛ t, whch s centered around the v c(t). Here, we denote by ɛ c the amount of playout tme dscrepancy wth respect to the v c(t). That s, ɛ c means the tme 10

11 dfference between the vrtual (v c(t)) and actual (T c(t)) presentaton tme. When the ɛ c falls n the target range,.e., ɛ c < 0.5 ɛ t, the playout adaptaton s not necessary. When ɛ c becomes greater than ɛ t /2 or less than ɛ t /2, each clent attempts to adjust the current presentaton tme and consequently move the buffer level. Each player ether fasten or loosen ts playback speed to compensate the tme dscrepancy ( ɛ c 0.5 ɛ t ) untl the T c(t) goes nto the target range. 3.4 Server-aded Inter-clent Synchronzaton wth Feedback n T c 2 T c 1 T c Clent 1 Server T s Adaptve Ctrl Group Sync Clent 2 Multcast RTP Multcast RTCP Uncast Feedback T s T s Adaptve Ctrl Group Sync. Clent n Adaptve Ctrl Group Sync Fg. 7. Server-clent sgnallng for nter-clent synchronzaton. Fg. 7 depcts the proposed operaton of nter-clent synchronzaton to mantan the playback of all clents close to a target presentaton tme. Remember that, n ths paper, we are talkng about lghtly coupled synchronzaton, where the server and clents need to be synchronzed wthn an allowed range. Thus, feedback-based synchronzaton s adopted to accomplsh the requred serverclent synchronzaton. RTCP-compatble sgnalng between the server and clents s performed to exchange controllng messages. Note that the exchange of controllng message needs to be restrcted based on the RTCP bandwdth rule and, due to the employed SSM, each feedback s uncasted. The RTCP RR nterval s determned based on the bandwdth of standard RTP and RTCP applcaton-defned (APP) packet, whch s customzed to delver the necessary tmng nformaton for the synchronzaton. When the server receves the compound RTCP RR packets, t gets the current presentaton tme of each clent (T c(t)) and estmates the RT T 7. They are aggregated to predct the 7 RT T of each clent s approxmated at the sender utlzng the uncasted RTCP RR reports. RT T = t A RR t DLSR t LSR, where t A RR s the arrval tme of the RR packet at the server, t DLSR s the delay elapsed from recevng last SR at the clent and t LSR s the tme that has been coped from the server SR packet at the recever. 11

12 target presentaton tme, T s. Each clent s then notfed about ths target presentaton tme when t receves the RTCP SR from the server. To determne the target T s from the aggregated feedbacks from all the clents, the server needs to set an arbtraton polcy. It s mportant to adjust the farness or performance of all clents n terms of synchronzaton 8. For example, we can choose a smple averagng for the arbtraton and thus T s s calculated by averagng all T c s. Or we can use other polces such as medan, mnmum, or maxmum. The detaled calculaton for T s s as follows. Even though we are talkng about gatherng feedbacks from all clents as a response to the server SR, the feedback from each clent s returnng at arbtrary tme pont (or maybe lost durng feedback). To compensate ths, we mantan the delay d s for each clent at the server,.e., the delay after the feedback recepton tll the server SR. Also the feedback from each clent takes dfferent RT T (to be precse, half of RT T ) to reach the server. Fnally, the target presentaton tme T s s calculated as T s = F arbtraton {T c, d s, RT T = 1,..., N c }, (1) where F arbtraton mples the selected arbtraton polcy, stands for clent, N c s an actve group sze, d s s the processng delay from the last RR to the current SR, respectvely. Wth the calculated T s, the server multcasts the compound RTCP packets contanng T s to all clents and each clent adapts ts playback speed. The prevous Fg. 6 also llustrates the detaled behavor of the server-aded clent adaptaton wth respect to the presentaton tme. Note that the proposed server-aded clent adaptaton s tghtly coupled wth the local playout adaptaton. At ths stage, we are usng a smple approach to coordnate local and server-aded adaptatons. Smply speakng, we are just replacng local vrtual tme wth the gude from the server. Note however that t s very mportant to make the swtch among the local and server-aded adaptatons smooth so that the playout at each clent s consstent. The detaled procedure s as follows. Whle each clent tres to adapt the local presentaton tme T c(t) to ts vrtual presentaton tme v c(t), t receves the target presentaton tme T s (n) from the server. Upon the recepton, each clent goes through transent adaptve playout perod by smply resettng ts reference presentaton tme of vrtual tmer to the server-reported T s (n). After the reset, the player goes nto the adaptaton state to re-adjust the presentaton tme wthn the allowed target range. 8 Note that ths ssue s tghtly connected to the farness of multcast congeston control. We also assume that clents wth badly broken lnks are compelled to leave the multcast group so that t does not bas the arbtraton. 12

13 3.5 Adaptve Playout Control for Error Recovery Retransmsson-based error recovery s not drectly applcable to the applcatons wth strct delay constrant. The adaptve playout control however can reduce the repar latency effectvely. That s, we can extend the role of local playout adaptaton n order to secure the tme requred for retransmsson request/reply. It enhances the chance of error recovery even n the crcumstances that RTT s relatvely small. Proposed cumulatve NACK-based error recovery scheme s llustrated n Fg. 8. It s no use to request the retransmsson f the buffer level b (t) s less than the retransmsson lmt b R. b n br + Ac ( t) (1 + p) b b R R ( 1+ p) Up to n feedback cumulaton s possble Retransmsson request s mpossble t b R effectve br p b (t) A c b R ( 1+ p) b (t) t Fg. 8. Error recovery wth the help of adaptve playout control. Precse estmaton of one-way delay from each clent to the server s very mportant n settng the b R effectvely. However, wth the SSM and RTP/RTCP envronment, t s rather complex to measure the RTT at the clents wthout the clock synchronzaton. By slowng down the playout speed, the adaptve playout can secure suffcent tme for retransmsson. Thus, the threshold for successful retransmsson-based error recovery - the effectve b R - can be lowered to b R (1+ p) b R (1+ p ) b (t) b H.. It thus enlarges the possble range of retransmsson to Furthermore, the cumulatve NACK can save the feedback bandwdth. When clents detect a packet loss based on a sequence gap, t checks the buffer occupancy level b (t) to determne whether t needs to request retransmsson rght away or t can hold the request for a whle to process multple requests together. From Fg. 8, t s natural that b (t) must be greater than b R to merge multple requests together when there s no adaptve playout. To merge feedbacks of n consecutve packets n the btmask feld of packet header, we need the buffer level margn of A n c (t). That s, b (t) has to be mantaned over b R + A n c (t). Lkewse, wth the adaptve playout, the buffer margn to enable the cumulatve feedback becomes (1 + p ) b (t) b R. When b (t) les between b R b (1+ p) R + A n c (t), we can hold the feedbacks by A c = mn[(1 + p ) b (t) 13

14 b R, A n c (t)] by slowng down the playout speed. Based on the buffer occupancy, the selectve decson for retransmsson s made. That s, f b (t) s greater than the effectve b R, the request becomes possble. After makng the decson, each clent feeds the cumulatve NACK back to the server usng the uncast feedback channel. When the server receves cumulatve NACKs from the clents, t checks the requests to suppress duplcate retransmsson requests. For example, f there are more than two NACKs requestng the same packet wthn a RTT, late requests are suppressed at the server. The requested packet whch s unsuppressed by the server s then multcasted back to all clents. 3.6 Coordnaton of Adaptve Playout Controls Inter-clent synchronzaton SYNC Local adaptaton b t) b L ( OR b( t) bh Receve T s Set T s as a reference pont RISK b t) > b L ( AND b ( t) < bh NORMAL Loss AND br b ( t) (1 + ) p No Loss OR br {Loss AND > b ( t) } (1 + ) p 0 ARQ RISK b L SYNC NORMAL RISK b H b S ARQ Error recovery Fg. 9. Proposed coordnaton among the adaptve playout states. Fg. 9 descrbes the proposed operaton of adaptve playout control for the local adaptaton, the server-aded adaptaton, and the cumulatve NACKbased error recovery. The coordnaton s performed among the coordnaton states: NORMAL, RISK, SYNC, and ARQ. A coordnaton state transts to the other states based on the buffer occupancy, the losses from network and system, and/or the server-reported T s. When the coordnaton state s under the NORMAL (.e., t falls nto the magnfed porton of Fg. 5), t tres to adjust the current presentaton tme T c(t) to the vrtual presentaton tme v c(t). The coordnaton state goes nto the RISK when the applcaton s at the hgh rsk of buffer under-/overflows,.e., b (t) b L or b (t) b H. Snce the RISK has the hghest prorty for attenton to prevent the playback dscontnuty, the player cannot escape from the RISK whle the buffer occupancy level b (t) s below b L or above b H. The ARQ state defnes the assstance of adaptve playout control for the error recovery. Thus, the player does not go nto the 14

15 b R (1+ p ) state whle b (t) s less than or greater than b R + A n c (t) even though packets are lost. The nter-clent synchronzaton s tghtly coupled wth the adaptve playout control whch s performed on the NORMAL state. That s, the player goes nto SYNC state when t receves the server-reported T s. In the SYNC state, the vrtual presentaton tme vc(t) s re-adjusted to the sender-reported T s n the RTCP SR packet. The actual adjustment of Tc(t) to the sender-reported T s s acheved when the player goes nto the NORMAL state after escapng from the SYNC state. 4 Smulaton Results 4.1 Smulaton Setup %/120ms %/200ms 31 Streamng Server Sesson Member loss(%)/rtt(ms) %/350ms %/350ms %/350ms %/400ms %/50ms 5%/50ms %/50ms %/500ms 8%/500ms %/450ms %/250ms %/250ms 5%/200ms %/150ms Fg. 10. Smulaton topology. Fg. 10 provdes the detaled smulaton topology by the network smulator (NS2) [23]. The SSM s provded by the PIM-SM multcast routng and each clent feeds back ts status through an uncast RTCP RR. There exsts one server and 16 clents wll jon to the group from arbtrary locatons. Table 1. Vdeo traffc model. I frame(kbytes) B frame(kbytes) P frame(kbytes) Average frame sze Standard devaton We smulate the MPEG-2 audo/vdeo streamng scenaro at average 5 Mbps wth 30 f/s frame rate. Ths frame rate leads us to the temporal granularty 15

16 of 33 ms. The adopted group of pcture (GOP) structure s for 15 frames consstng of one I-frame and 3 P-frames (.e., IBBPBBPBBPBBPBB) and the bt rates of each frame s tabulated n Table 1. At ths stage, the decodng and composton delay s gnored for the sake of smplcty. The tmng accuracy of playng a frame s less than 10 ms n the smulatons. All clents jonng the multcast group have the same recever buffer sze (default sze: 1 sec, unless specfed else) and perform the same playout adaptaton. Followng thresholds are utlzed for the recever buffers. Lower lmt b L b s set to R from the bottom, where the retransmsson lmt b (1+ p ) R s based on the estmated RTT. Hgher lmt b H has margn equal to 100 ms from the top. We set the default playback factor ( p ) to 0.25 (25%), whch s somewhat an aggressve choce for the tme-scale modfcaton. For the T s arbtraton polcy, selecton of mnmum value s employed as a default polcy. Target synchronzaton dscrepancy ɛ t wth respect to the vrtual presentaton tme s set to 100 ms. Note also that the playout adaptaton needs to be done n terms of vdeo frame. The presentaton tme dscrepancy s thus converted to the number of vdeo frames Nf (based on the vdeo frame rate R f ). Thus, we are gettng the number of frames to be adaptvely played out wth the tme-scale modfcaton, Np,.e., Np = Nf/ p. For example, f we want to compensate 2 frames, then we needs to do the playout for 20 frames wth p = 0.1. In summary, the player of each clent has to perform the playout adaptaton up to Np/R f sec as long as the buffer level s properly controlled. The smulaton ntates just after 0 sec by startng the multcast streamng from the server. Each clent jons to the group wth a gap of 11 sec (fxed). After jonng and startng the playback, all the clents are subject to a randomzed, momentary CPU overload. Currently the CPU overload s happenng for less than 50 ms at full strength (.e., no other computaton s possble) and t s randomly occurrng wth a 10 sec exponental dstrbuton. For the network, varous combnaton of TCP and CBR traffcs are traversng the whole network topology and causng network fluctuatons between the server and clents. We select ndependent loss only on the recever lnks as a loss scenaro. The ndvdual losses are occurred wth an unform dstrbuton and several clents experence burst losses wth 50 to 100 ms exponentally dstrbuted burst length. The server has one second retransmsson buffer. The average loss fracton s 6 %, some clents suffer from maxmum loss at 13 % and some clents even do not experence any loss (0%). For the cumulatve NACK-based error recovery, up to 16 NACKs are merged. Wth all these setups, we are usng several measures for the synchronzed streamng performance and two for the performance of cumulatve NACK-based error recovery. They are 1) accumulated playout dscontnuty per each clent durng streamng, 2) the maxmum dfference of playout among clents, 3) the playout speed varaton of a clent, 4) the consumed feedback bandwdth and 5) the success probablty of loss recovery wth retransmsson. 16

17 4.2 Verfcaton Results To verfy the performance of the proposed approach, three dfferent playout cases are compared. No playout adaptaton means the case where the adaptve playout control for both local adaptaton and error recovery s not performed and adaptve playout cases are dvded nto Local adaptaton and Server-aded adaptaton. In both local and server-aded adaptaton cases, the player request retransmsson wth the help of playout control. The request s performed ndvdually ( ndvdual request ) or cumulatvely ( cumulatve request ). T me d f f er ence( s) No playout adaptaton Local adaptaton wth ndvdual request Local adaptaton wth cumulatve request Server-aded adaptaton wth cumulatve request Server-aded adaptaton wth ndvdual request Fg. 11. Maxmum playout tme dfferences between the leadng and tralng clents. Dscontnuty(s) No playout adaptaton Local adaptaton wth ndvdual request Local adaptaton wth cumulatve request Server-aded adaptaton wth ndvdual request Server-aded adaptaton wth cumulatve request Tme(s) Fg. 12. Accumulated playout dscontnuty averaged per each clent. 17

18 Frst, the maxmum playout tme dfferences between the leadng and tralng clents at each tme t s shown n Fg. 11. Wthout the adaptve playout, the tme dfference ncreases up to maxmum buffer occupancy level,.e., b s. Packet overflows or ntentonal flushng s useful n reducng the gap, but t pays the playout dscontnuty. Check the accumulated dscontnuty per each case n Fg. 12. It llustrates 0.8 sec dscontnuty s occurrng to each clent on the average wthout the playout adaptaton. In comparson, the proposed playout adaptaton can bound the gap more effectvely. If the player adopts local adaptaton only, there s no way to restore the nter-clent synchronzaton snce there s no gudance. Eventually t s subject to flushng of packets n the buffer. But wth the help of server coordnaton, we can mantan and the gap throughout the whole playout. The accumulated playout dscontnuty n Fg. 12 shows that the proposed playout adaptaton wth the server-aded synchronzaton effcently reduces the playout dscontnuty to half of No playout adaptaton. Probablty No playout adaptaton Local adaptaton wth cumulatve request Local adaptaton wth ndvdual request Server-aded adaptaton wth ndvdual request 0.2 Server-aded adaptaton wth cumulatve request Frame rate(f/s) Fg. 13. Playout speed varaton (a clent). Fg. 13 depcts the playout speed varaton of a clent n the group. It s no doubt that the server-aded adaptaton experences more playout speed varaton than the other cases at the expense of reduced dscontnuty. The maxmum buffer occupancy dfferences between the leadng and tralng clents n the group s llustrated n Fg. 14. Wth the adaptve playout, the buffer level dfferences between the group members are sgnfcantly reduced and server-aded adaptaton mantans slghtly smaller dfference over the local adaptaton. Snce the server selects the mnmum value as T s under the default arbtraton polcy, t uses the presentaton tme of the slowest clent n the group as the target reference presentaton tme. The adaptaton to T s at the clents makes the dstrbuton of buffer levels more concentrated. 18

19 Buffer occupancy dfference(s) Local adaptaton wth ndvdual request No playout adaptaton Local adaptaton wth cumulatve request Server-aded adaptaton wth ndvdual request Server-aded adaptaton wth cumulatve request Tme(s) Fg. 14. Maxmum buffer occupancy dfferences between the leadng and tralng clents. Table 2. Impact of T s arbtraton polcy on the performance. T s selecton Dscontnuty(s) Playout speed varaton(f/s) Mnmum Medan Maxmum Average The approprate selecton on T s, arbtraton polcy, mpacts the overall performance as shown n Table 2. The result shows that T s derved from mnmum arbtraton polcy (.e., mn(tc 1 + d 1 s + RT T 1,,, T N c c + d N c s + RT T N c ) s preferable to the others wth respect to playback dscontnuty. That s, f we choose T s accordng to the slowest clent n the group, t ncreases the tme avalable for retransmsson for all the other clents. Ths n turn contrbutes to the enhanced error recovery and hence results n the reduced dscontnuty compared to the other arbtraton polces. The averaged T s selecton also reduces the playout dscontnuty and speed varaton compared to the other polces. Fg. 15 llustrates the consumed bandwdth of RTCP RR and retransmsson requests used for feedbacks to the server. Note that we assume the compounded RTCP RR wth 120 bytes and the retransmsson request packet (regardless of feedback cumulaton) of wth 12 bytes are used to feedback QoS nformaton and retransmsson request. Furthermore, three cases compared here are exposed to the same network losses but may experence dfferent applcaton losses based on the scenaro. We can observe the bandwdth ef- 19

20 Bandwdth(b/s) Local adaptaton wth ndvdual request No playout adaptaton Server-aded adaptaton wth ndvdual request Local adaptaton wth cumulatve request 5000 Server-aded adaptaton wth cumulatve request Tme(s) Fg. 15. Requred feedback bandwdth for retransmsson request and recever report. Pr obab l t y 0.8 Server-aded adaptaton wth ndvdual request 0.7 No playout adaptaton Local adaptaton wth ndvdual request Local adaptaton wth cumulatve request Server-aded adaptaton wth cumulatve request Fg. 16. Average repar probablty at tme t n a group. fcency of cumulatve NACK for retransmsson request n Fg. 15. The cumulatve NACK-based requests show the savng of the feedback bandwdth to a half compared wth the ndvdual requests. Fg. 16 llustrates the averaged repar probablty of lost packets n each clent. It s confrmed that the coordnaton of the server and clents,.e., server-aded adaptaton, ncreases the repar probablty. Wthout the adaptve playout, the repar probablty s hghly fluctuatng over the tme. However, we note that the repar performance of the adaptve playout wth the cumulatve requests s worse than that of No playout adaptaton. Ths result partly mples that the choce on the cumulaton (.e., n = 16) s rather aggressve snce t makes the tmely recovery to fal from tme to tme. The lack of the precson n convertng buffer 20

21 level to tme quantty and the feedback packet loss (snce t can affect up to n retransmsson) are partly responsble for ths, too. 5 Related Works Streamng meda applcatons have to deal wth network fluctuatons (delay jtter and loss) that can cause the dsrupton n the playout. They also need to keep the playout synchronzed regardless of the exceptonal events at the clent systems. Wth the synchronzaton, we typcally refer to the reconstructon of temporal relaton between dfferent meda objects. There are several types of synchronzaton such as ntra-meda, nter-meda, and nter-clent synchronzaton. Intra-meda synchronzaton addresses the temporal relatons among consecutve meda contents (e.g., packets) of a sngle stream. Inter-meda synchronzaton defnes the temporal relatonshps between dfferent types of meda objects (e.g., lp synchronzaton for vdeo and audo). Controllng the skew,.e., the tme dfference between meda streams, s a key ssue to be solved. The permssable skew level s tghtly coupled wth the human percepton. For ntra-clent synchronzaton, several reactve controls lke dscardng, skppng, shortenng and extenson of output duraton, and vrtual tme contracton and expanson have been developed [24]. The playout adaptaton s thus requred to mantan the playout qualty consderng the network fluctuatons and system lmtatons [8 10]. Prevous works on the adaptve playout control mostly focus only on the ntra-clent synchronzaton ssues. Multmeda player wth an applcaton-level CPU scheduler adapts the playout speed based on the buffer occupancy level, whch s lnked to the system loads [8]. It s however focused on the system capablty varaton than the network fluctuatons. An adaptve stream synchronzaton protocols [9] s used to control the synchronzed playout based on the buffer fullness. It supports the noton of master and slave streams to coordnate nter-clent synchronzaton. However, t only covers the sgnalng for synchronzaton, and the specfc mechansms to make streams synchronzed are kept open. As a promsng soluton canddate, the adaptve playout wth the audo tme-scale modfcaton [11, 12] s proposed to synchronze the clents wthout hurtng the playout qualty (.e., wthout any dsrupton). They allow us to perform the playout adaptaton to gracefully conceal the network delay jtters and packet losses n the Internet. Especally, [12] addresses the adaptve meda playout to relax the delay constrant of retransmsson as well as mnmze the buffer underflow. Based on both channel condton and buffer fullness, t consders delay and loss reslency wth the help of adaptve playout control. The nter-clent synchronzaton s rarely dscussed yet. It s mportant to acheve harmonzed playout among the clents whle mantanng farness among 21

22 multcast group members. It should be done by effectvely mtgatng the network and system heterogenety of the clents. [9] deals wth the nter-clent synchronzaton as well as the ntra-clent one by propagatng the adapt messages. [25] and [26] ntroduce nter-clent synchronzaton for lve and stored meda n multcast envronments, respectvely. In [25], a synchronzaton agent s used to exchange control packets. To adjust the presentaton tme, the delay (expressed n levels) at each clent s reported. However, t lacks n consderng the propagaton delay and the feedback mploson due to short-term fluctuatons n the network. For the stored meda, [26] utlzes a multcast group dssemnatng control packets from a master to slave destnatons. It requres message exchanges among sesson members, whch s mpossble n the SSM envronment. Also, other works [27 29] consder the farness among group members. However, they dffer from the proposed work snce we are focusng on the role of adaptve playout to the server-aded nter-clent synchronzaton under the multcast streamng envronment. The error control drectly affects meda qualty, too. Snce the Internet does not guarantee the QoS, a relable transmsson needs error controls ncludng the ARQ, the FEC, and ther hybrd. Each of them has ts own mert and demert, and works well n the uncast envronment whle remanng too challengng n the multcast envronment. Especally, the error recovery by retransmsson s appealng due to ts smplcty and effcency. However, the requred feedback bandwdth and repar latency make the problem dffcult. In the uncast envronment, works n the IETF (Internet Engneerng Task Force) are standardzng selectve retransmsson mechansms as a part of RTP/RTCP [20 22]. They focus on modfyng the retransmsson-based error recovery. For example, [22] ntroduces a desred RTCP format for the selectve retransmsson over a wreless lnk. Cumulatve NACK approach n [30] s utlzng multple multcast channels to cope wth the feedback mploson, where a TCP s wndow-lke scheme s used to merge the retransmsson request. Note agan that we are focusng the mpact of adaptve playout to the multcast error recovery n ths paper. 6 Conclusons In ths paper, we propose a framework for the synchronzed multcast meda streamng wth the adaptve playout and error control, whch s sutable for one-to-many meda streamng wth an mproved flexblty. The adaptve playout controls the playout speed of audo and vdeo by adoptng the tme-scale modfcaton of audo based on the buffer occupancy. The man role of the adaptve playout control s to reduce the dscontnuty ncurred by packet overflow/underflows and momentary CPU overload. Furthermore, RTCP-compatble sgnalng between the server and clents s observed, where 22

23 the exchange of controllng message s restrcted. By employng the serverclent coordnated adaptve playout control wth feedback for the presentaton tme synchronzaton, t vares the playback speed wthn the perceptual lmt and adapts the presentaton tme of each clent to help the nter-clent synchronzaton. In addton, each clent performs retransmsson-based error recovery wth the assstance of playout control to compensate for long repar latency. The retransmsson request wth NACK-based cumulaton can save the allocated bandwdth and prevent the feedback mploson. We maxmze the role of adaptve playout control to help securng the tme avalable for retransmsson. Addtonally, the server mplctly asssts the error recovery whle the clents perform nteractve error recoveres. The network-smulator based smulatons show that the proposed framework 1) reduces the playout dscontnuty caused by the network fluctuatons and the system lmtatons wthout degradng the meda qualty, 2) saves the feedback bandwdth nduced by retransmsson request, and 3) mtgates the clent heterogenety by employng the sever-clent coordnated adaptve playout and error control. Thus, the potental of the proposed multcast streamng framework to enhance the multcast streamng qualty s well verfed. References [1] K.C. Almeroth, The evoluton of multcast: from the MBone to nterdoman multcast to Internet2 deployment, IEEE Network, vol. 14, pp , Jan [2] C. Dot, B.N. Levne, B. Lyles, H. Kassem, and D. Balensefe, Deployment ssues for the IP multcast servce and archtecture, IEEE Network, vol. 14, pp , Jan [3] H. Holbrook and B. Can, Source-specfc multcast for IP, Internet Draft draft-etf-ssm-arch-00.txt, IETF, Nov [4] A. Schulzrnne, S. Casner, Frederderck, and V. Jacobson, RTP: A transport protocol for real-tme applcatons, IETF RFC 1889, Jan [5] V.O.K. L and Z. Zhang, Internet multcast routng and transport control protocols, Proceedngs of the IEEE, vol. 90, pp , Mar [6] S. Floyd and et. al., Equaton-based congeston control for uncast applcatons, n Proc. ACM SIGCOMM, Aug. 2000, pp [7] J. Wdmer and M. Handley, Extendng equaton-based congeston control to multcast applcatons, n Proc. ACM SIGCOMM, Aug [8] J. Walpole and et. al., A player for adaptve MPEG vdeo streamng over the Internet, n Proc. of the SPIE 26th Appled Imagery Pattern Recognton Workshop (AIPR), Oct