OPTIMIZED NESTED PROTECTION FOR VIDEO REGION OF INTEREST WITH RAPTOR CODES

Transcription

1 OPTIMIZED NESTED PROTETION OR VIDEO REGION O INTEREST WITH RAPTOR ODES Zhengy Luo 2 L Song 23 Shbao Zheng 2 and Nam Lng 3 Shangha Dgtal Meda Processng and Transmsson Key Lab 2 Shangha Jao Tong Unversty Shangha hna 3 Santa lara Unversty Santa lara USA ASTRAT Due to the best effort feature of many exstng transmsson channels vdeo streams often suffer from nevtable transmsson errors. In ths paper we propose a scheme of robust vdeo transmsson based on the state-of-the-art Raptor codes whose applcatons are n full swng now. And consderng Regon of Interest (ROI) often draws much attenton n mages the scheme adopts a nested protecton framework to show partaltes to ROI areas for better protecton. Dfferent from many exstng Raptor codes based UEP methods our scheme s developed based on the easy-to-use standardzed Raptor codes. Expermental results show that sgnfcant robustness can be obtaned for the vdeo streams especally for the ROI areas. Index Terms UEP Raptor codes vdeo ROI. INTRODUTION Wth the development of multmeda technologes applcatons va vdeo transmsson have played an mportant role n modern lfe. However exstng transmsson channels such as the Internet and wreless networks usually provde only best effort servces. Therefore ways of robust vdeo transmsson are always desred by many applcatons. As far as the transmsson level s concerned a common way of robust vdeo transmsson s Automatc Repeat Request (ARQ) whch makes the recever send retransmsson requests to the sender f vdeo packets are lost. Obvously ARQ necesstates feedback channels and ntroduces extra delay and s unrealstc for many applcatons such as the broadcastng servces. esdes ARQ another way of robust transmsson s orward Error orrecton (E) whch provdes robustness based on channel codes. Dfferent from ARQ E requres no feedback channels but redundancy symbols to combat the transmsson errors and thus s qute popular n many applcatons such as lve vdeo streamng. requently Reed-Solomon (RS) codes are used as the E codes to provde robust vdeo transmsson. ut they have hgh computatonal complexty and are practcal only at small block lengths. Recently Raptor codes [] a new knd of fountan channel codes [2] have been developed. ompared to conventonal channel codes e.g. RS codes Raptor codes have low computatonal complexty and are stll practcal at large block lengths. And systematc Raptor codes have been standardzed and adopted as E codes n latest multmeda standards [3][4]. Now n vew of the superorty of Raptor codes applcatons of Raptor codes n the multmeda feld are n full swng [][6]. Recently Raptor codes have also been utlzed to provde robust vdeo transmsson va Unequal Error Protecton (UEP). Vukobratovc et al. [7] proposed an UEP method for scalable vdeo multcast based on expandng wndow fountan codes whch ncorporated a wndow selecton procedure n fountan codng. atald et al. [8] appled UEP based on sldng wndow Raptor codes to scalable vdeo broadcastng where Raptor codes were mproved by vrtually extendng the block length wth sldng wndows. oth of these methods adapt to UEP frameworks by modfyng the structure of Raptor codes and may ncur extra dffcultes n practce compared wth the standardzed Raptor codes. In ths paper we take a dfferent routne and propose a content based UEP optmzaton scheme based on the standardzed systematc Raptor codes. Ths scheme provdes dfferent degree of protecton to packets of dfferent types at the Group of Pctures (GOP) level and adopts a nested protecton framework consderng that Regon of Interest (ROI) often draws much attenton and should be protected more than others. ompared wth Hellge et al. [9] here packet classfcaton s ntroduced and the expected dstorton mpacts are mnmzed for better protecton. Expermental results show that our scheme can provde sgnfcant robustness for vdeo transmsson especally for the ROI areas. The remander of ths paper s organzed as follows. Secton 2 ntroduces Raptor codes brefly. Secton 3 descrbes the adopted protecton framework. Secton 4 formulates the UEP problem. Expermental results valdatng the effectveness of the scheme are shown n Secton. And Secton 6 draws the concluson. 2. RAPTOR ODES

2 g.. Illustraton of the encodng of Raptor codes. ountan codes [2] are a class of channel codes whose encoders can generate as many encodng symbols as necessary from a gven number of source symbols just as fountans spray water drops. At the decoder sde the source symbols can be recovered as long as enough encodng symbols are receved. The frst knd of practcal fountan codes are Luby Transform (LT) codes [] whose encodng can be dvded nto two steps. rst the degree d of an encodng symbol s chosen from a degree dstrbuton e.g. the Robust Solton Dstrbuton []. Then d dstnct source symbols are chosen unformly and randomly and exclusve-ored to produce the encodng symbol. Raptor codes are a new knd of fountan codes whch generally buld upon LT codes and LDP codes []. The encodng of Raptor codes can be llustrated n gure where the source symbols are frst pre-encoded nto the ntermedate symbols whch are then LT encoded nto the fnal encodng symbols. At the decoder sde the source symbols can be recovered from any subset of encodng symbols only slghtly more n number than the number of source symbols. Raptor codes have the advantage of lnear tme encodng and decodng and especally beneft large block lengths. Now n lght of the preference of systematc codes systematc Raptor codes have been developed and adopted n standards lke 3GPP [3]. And due to the good performance Raptor codes have become an excellent opton for robust multmeda transmsson [][6]. 3. ADOPTED PROTETION RAMEWORK It s well known that dfferent vdeo packets such as ntra (I) predctve (P) or b-drectonal () packets have dfferent dstorton mpacts and UEP for dfferent packet types usually can provde enhanced robustness for vdeo transmsson. Hence we may desgn the Raptor codes based UEP framework n accordance wth packet types at the GOP level at frst. urther research on Human Vsual System (HVS) reveals that people generally pay more attenton to the ROI areas n mages whch motvates us to desgn the protecton framework n a nested manner so that ROI areas have better error reslence. ased on the above two key ponts we frstly classfy the vdeo packets of a GOP nto N = 3 classes - class for I packets class 2 for P packets and class 3 for packets and then further dvde each class nto subclass for g. 2. Illustraton of the adopted protecton framework. g. 3. Illustraton of the Raptor encodng and decodng procedures of each class. ROI packets and subclass for background packets. Our scheme processes each class of packets separately but n smlar procedures. So the nested protecton framework s descrbed wth one representatve class of packets n the followng. In the context of Raptor protecton here every vdeo packet s regarded as a Raptor symbol. The nested framework nvolves nner and outer Raptor encodng where the former protects part of packets whle the latter protects all packets. As llustrated n gure 2 each class s composed of subclass and subclass whch contan respectvely ROI packets and background packets of class. As ROI packets often have hgher dstorton mpacts than background packets we frst perform the nner Raptor encodng whch takes the ROI packets of class as source packets and produces the nner repar packets. Snce systematc Raptor codes are adopted n our framework after nner encodng the ROI packets from subclass reman and we can contnue to perform the outer Raptor encodng whch takes both the ROI and the background packets as source packets and produces the outer repar packets. In ths way ROI packets can be recovered by both the nner and the outer repar packets whle background packets can be recovered only by outer repar packets. The Raptor encodng and decodng procedures of each class can be shown n gure 3. Note that at the recever sde frstly the nner Raptor decodng s performed for the ROI packets. If the nner Raptor decodng succeeds the

3 background packets may be recovered more easly as the outer Raptor decodng has more encodng symbols whch have been partly recovered by the nner decodng. ut even f the nner Raptor decodng fals ROI packets may stll be recovered by the outer Raptor decodng. Therefore the nner and outer Raptor encodng gves ROI packets a hgher probablty of recovery than background packets. And the nested protecton framework not only enables robust vdeo transmsson but also provdes hgher prortes to ROI packets. 4. PROLEM ORMULATION AND OPTIMIZATION Our scheme provdes a robust vdeo protecton framework but t stll has to be properly confgured for hgher effcency. To ths end we formulate the protecton based on the above nested framework as a constraned optmzaton problem and determne ts optmal confguraton n terms of the source and channel condtons. or descrpton purposes we frst clarfy some adopted notatons. At the random packet loss rate p l the probablty that m out of n packets are receved s m n m b n l l m ( ) ( ) P n m = p p. () Assume the standardzed systematc Raptor codes n 3GPP [3] are used n ths paper. Pf ( m k ) represents the falure probablty of the standardzed Raptor code wth k source symbols f m symbols are receved and can be modeled by [2] m< k Pf ( m k) =. (2) m k.8.67 m k As llustrated n gure 2 let k and k denote respectvely the number of ROI packets and background packets of class and let r and r denote respectvely the number of the nner and outer repar packets of class. Assume m a out of k ROI packets m b out of r nner repar packets and m c out of k + r background and outer repar packets are receved. As the ROI packets fal to recover when both the nner and outer Raptor decodng fals the probablty that a ROI packet of subclass gets lost fnally can be approxmated by k r k + r b( a) b( b) ma = mb= = Pb( k + r ) Pf ( ma+ mb k ) P P k m P r m ma k f ( a c ) ma k P m + m k + k (3) where ma k ma k s the probablty that the vdeo packet s not ncluded n the m a packets due to systematc Raptor codes. The background packets fal to recover when the outer Raptor decodng fals. As more encodng symbols wll be avalable for the outer Raptor decodng f the nner Raptor decodng succeeds the probablty that a background packet of subclass gets lost fnally can be approxmated by where k r k + r b( a) b( b) a = b= c= ( ){ b c f ( a b ) f ( c ) f ( a b ) m m m P P k m P r m P k + r m P m + m k P k + m k + k + P m + m k Pf ( ma+ k + k )} k + r k + r k + r k + r (4) s the probablty that the vdeo packet s not ncluded n the m c packets. D j denote the dstorton mpact due to the loss Let ( ) of the j th packet of subclass and D ( ) j the dstorton mpact due to the loss of the j th packet of subclass. Assume the dstorton mpacts of vdeo packets do not correlate wth each other for smplcty. Then the expected dstorton mpacts of all the vdeo packets due to packet loss can be approxmated by N k k E [ D] = P D ( j) + P D ( j) = j= j= () N k k = P D ( j) + P D ( j) = j= j= where packets of N = 3 (I P and ) classes are all consdered. or hgher vdeo qualty we can try to mnmze the expected dstorton mpacts by optmzng the dstrbuton of repar packets. Specfcally suppose R repar packets are avalable at the gven E rato then the problem can be formulated as r r r r = argmn E D { N N} [ ] ( ) r r rn rn N r + r R = r ( N) (6) st.. r ( N) r + r r+ + r + ( N ) k + k k+ + k+ where the last constrant s the descendng prorty constrant accountng for the dfferences of frame types.

4 3. 4 x 8 3 Subclass Subclass Subclass 2 Subclass 2 Subclass 3 Subclass 3 g. 4. ROI areas of Raven and Sherff. Through the penalty method the problem can be translated nto mnmzng * N E[ D] = E[ D] + λ max ( r + r ) R = (7) N + λ+ max ( k + k )( r+ + r+ ) ( k+ + k+ )( r + r ) = where all λ are penalty coeffcents and solved by the heurstc algorthms such as the smulated annealng algorthm used n ths paper.. EXPERIMENTS The proposed scheme was evaluated wth the representatve :2: sequences Raven and Sherff. Snce the scheme s confgured and appled GOP by GOP here we present the full parameter confguraton and expermental results for the frst GOP of the test sequences whose ROI areas are ndcated n gure 4 and are supposed not to change n the GOP. It should be noted that our scheme does not depend on the settng of ROI areas and the above settng s used here only to facltate comparson n the followng. In fact the scheme can be appled freely wth other settng of ROI areas as needed. We used the JM 4.2 software [3] to obtan H.264/AV vdeo packets of bytes where lexble Macroblock Orderng (MO) type 2 was enabled for ROI encodng. GOPs of IPP structure ncludng frames were consdered. QP=28 and AA was adopted durng compresson. It was supposed that vdeo packets were lost ndependently and % E rato ( R N ( k k ) R + + = % ) was allowed to mtgate = transmsson errors. In the experments the error concealment algorthm for I frames was weghted ntra nterpolaton [4] and that for P and frames was drect copy. The dstorton mpacts of vdeo packets were measured by Sum of Squared Errors (SSE). Let M W and H denote respectvely the frame number n a GOP the frame wdth and the frame heght. Then the dstorton mpact of a packet can be expressed as M W H D= f( mxy ) f( mxy ) 2 (8) m= x= y= SSE SSE Packet Number 2. x Packet Number where f ( mxy ) and f ( ) Subclass Subclass Subclass 2 Subclass 2 Subclass 3 Subclass 3 g.. lassfcaton and dstorton mpacts of the vdeo packets for the frst GOP of Raven and Sherff. mxy are respectvely the pxels of the correctly decoded sequence and the decoded sequence wth that packet error concealed. The classfcaton results along wth the dstorton mpacts of the vdeo packets for the test sequences are shown n gure. It can be seen that the dstorton mpact has qute a hgh order of value so the penalty coeffcents were set to n the penalty method. ased on such classfcaton dstrbuton of the repar packets was optmzed by the smulated annealng algorthm. At dfferent packet loss rates after a certan number of teratons the obtaned confguraton parameters of the proposed scheme are shown n Table where the number of source packets for the test sequences s also shown. As mentoned before Raptor codes have much superorty over conventonal channel codes. In addton other man Raptor codes based UEP methods nvolve more or less modfyng the standardzed Raptor codes whle our scheme bulds upon the easy-to-use standardzed Raptor codes. Therefore based on the confguraton parameters the proposed scheme was compared wth the standardzed Raptor codes based Equal Error Protecton (EEP) case whch was mplemented by protectng the vdeo packets

5 Table. onfguraton parameters for the frst GOP of the test sequences Sequence Raven Sherff Packet Loss Parameter Rate k r k r k 2 r 2 k 2 r 2 k 3 r 3 k 3 r equally n a block consderng the good performance of Raptor codes at large block lengths. At dfferent packet loss rates the measured PSNR for the ROI areas and the whole sequences s shown n gure 6 where all the reported PSNR s averaged over smulatons. It can be seen that compared wth EEP obvous gans can be obtaned for both the ROI areas and the whole sequences. Ths s because repar packets are dstrbuted accordng to the dstorton mpacts of vdeo packets n our scheme. What s more we can also see that ROI areas may have hgher PSNR mprovement than the whole sequences. Ths s because we adopt a nested protecton framework whch gves hgher prortes of error protecton and repar packet dstrbuton to ROI areas. The reason why Raven does not have as much ROI mprovement as Sherff s that ts ROI areas account for most of the mage areas and the resultng vdeo packets. ut ths can be mproved by adjustng the sze of the ROI areas. gure 7 shows the PSNR comparson for every frame under a same loss pattern of the packet loss rate.. gure 8 and gure 9 show respectvely part of the 4th reconstructed frame of Raven and the 6th reconstructed frame of Sherff n ths case. We can also see that our scheme performs better. 6. ONLUSION In ths paper we desgn a scheme of robust vdeo transmsson based on the standardzed Raptor codes. We adopt a nested protecton framework so that ROI packets can have hgher prortes of error protecton durng transmsson. Expermental results show that sgnfcant robustness can be obtaned for the vdeo streams especally for the ROI areas. 7. AKNOWLEDEMENT Ths work was supported by the natonal 863 Program (22AA73) the chna MIIT Program (2ZX34-3) the NS grants ( and 69273) STSM (2DZ22726) and the project. 8. REERENES [] A. Shokrollah Raptor codes IEEE Transactons on Informaton Theory vol. 2 no. 6 pp. -67 June 26. [2] D. J.. MacKay ountan codes IEE Proceedngs - ommuncatons vol. 2 no. 6 pp December. [3] 3GPP Multmeda broadcast/multcast servce (MMS); protocols and codecs 3GPP TS v.. March 2. [4] M. Luby A. Shokrollah M. Watson and T. Stockhammer Raptor forward error correcton scheme for object delvery R 3 October 27. [] T. Stockhammer and A.. egen Response for MMT EE #9: applcaton layer E n the IET MPEG M46 October 2. [6] T. Stockhammer IET standards on E MPEG M2272 November 2. [7] D. Vukobratovc V. Stankovc D. Sejdnovc L. Stankovc and Z. Xong Scalable vdeo multcast usng expandng wndow fountan codes IEEE Transactons on Multmeda vol. no. 6 pp October 29. [8] P. atald M. Grangetto T. Tllo E. Magl and G. Olmo Sldng-wndow Raptor codes for effcent scalable wreless vdeo broadcastng wth unequal loss

6 PSNR (d) 2 Sequence (Proposed) Sequence (EEP) Sequence Improvement PSNR (d) 2 rame (Proposed) rame (EEP) rame Improvement Packet Loss Rate rame Number 4 3 PSNR (d) 2 Sequence (Proposed) Sequence (EEP) Sequence Improvement PSNR (d) 2 rame (Proposed) rame (EEP) rame Improvement Packet Loss Rate g. 6. PSNR comparson for the frst GOP of Raven and Sherff under random packet loss. 2 rame Number g. 7. PSNR comparson for the frst GOP of Raven and Sherff under a same loss pattern of the packet loss rate.. protecton IEEE Transactons on Image Processng vol. 9 no. 6 pp June 2. [9]. Hellge D. Gomez-arquero T. Scherl and T. Wegand Layer-aware forward error correcton for moble broadcast of layered meda IEEE Transactons on Multmeda vol. 3 no. 3 pp. -62 June 2. [] M. Luby LT codes Proceedngs of the IEEE Symposum on oundatons of omputer Scence pp Vancouver November 22. [] R. G. Gallager Low-Densty Party-heck odes. ambrdge Massachusetts: MIT Press 963. [2] M. Luby T. Gasba T. Stockhammer and M. Watson Relable multmeda download delvery n cellular broadcast networks IEEE Transactons on roadcastng vol. 3 no. pp March 27. [3] JM 4.2 reference software Avalable: [4] Y.-K. Wang M. M. Hannuksela V. Varsa A. Hourunranta and M. Gabbouj The error concealment feature n the H.26L test model Proceedngs of the IEEE Internatonal onference on Image Processng pp. II-729-II-732 Rochester New York September 22. g. 8. Part of the 4th reconstructed frame of Raven for EEP and our scheme under a same loss pattern of the packet loss rate.. g. 9. Part of the 6th reconstructed frame of Sherff for EEP and our scheme under a same loss pattern of the packet loss rate..