A WAVELET CODEC FOR INTERLACED VIDEO

Transcription

1 A WAVELET CODEC FOR INTERLACED VIDEO L.M. Me, H.R. Wu and D.M. Tan School of Electrcal and Computer Engneerng, RMIT Unversty, Vctora 3000, Australa Tel: Fax: E-mal: Abstract a Dscrete Wavelet Transform (DWT) based codec for nterlaced vdeo s presented. Most DWT vdeo coders n lterature are targeted toward low or very low btrates, where nterlacng s not consdered at all. However, for vdeo codng at md range btrates, especally for the TV sgnals whch contan the so-called nterlacng artfacts, the feld mode s requred to avod the unnecessary hgh frequency dstorton and offer a more flexble soluton to moton estmaton and compensaton. Theore a frame/feld mode adaptve swtchng algorthm s proposed for both ntra- and nter-frame coder, respectvely, to mprove the Rate vs. Dstorton (R-D) performance. Some prelmnary results are reported here. Ths codng framework can be a good backbone for the applcaton of Human Vson System (HVS) model based Perceptual Dstorton Measure (PDM) for better perceved vdeo qualty. Index Terms Wavelet, Vdeo, Interlaced, Perceptual Codng I. INTRODUCTION The DWT based mage/vdeo codng has been an extensve research topc for the past decade and a half [1]. Although current nternatonal and ndustry vdeo compresson standards all adopt the Dscrete Cosne Transform (DCT) based technques, DWT based codng has contnued to receve wdespread attenton n lterature due to the followng reasons: 1) the block based DCT coders have nevtable blockng artfacts at low and very low btrates, whch s most unpleasant to vew [2]; 2) DWT based mage/vdeo coders can offer genune graceful scalablty n qualty, spatal, and temporal domans wthout sacrfcng the codng effcency through caul packng of the bt stream. In comparson, DCT based coders can only offer layered scalablty and the enhanced layer wll ncur some overheads, leadng to poorer compresson. The scalablty property s very useful for web based applcatons under dfferent or varyng bandwdth; 3) DWT can effectvely retan the property n both tme and frequency doman. As a result t better adapts to the characterstcs of the Human Vsual System (HVS) than DCT. Most exstng DWT based vdeo coders operate at low to very low bt rates and are ntended for vdeoconferencng and moble telephony applcatons. These applcatons generally codes dgtal vdeos n the frame mode. However, there are other applcatons, such as dgtal TV, where sgnals n each pcture frame conssts of two felds. If the vdeo coder only adopts frame mode to code these TV pctures, the so-called nterlacng artfacts wll consume unnecessary bts for retanng hgh spatal frequency nformaton. A DWT vdeo codec s theore proposed n ths paper to address ths ssue. The detals of the coder are explaned n Secton II followed by the results and analyss n Secton III. The concluson and dscusson are gven n Secton IV. II. THE PROPOSED CODER Both the Moton Compensaton (MC) based Two Dmensonal (2D) DWT coder and Three Dmensonal (3D) DWT vdeo coders have been ntensvely nvestgated [1, 3-15]. Each approach has ts own advantages and dsadvantages dependng on the requrements of applcatons. However, most DWT based vdeo coders n lterature dd not am at the codng of TV sgnals whch contan nterlacng artfacts. Snce mproved R-D performance s the man goal n ths work, the tradtonal MC-2D nterlaced approach s adopted here as llustrated n Fg. 1. The MC-2D nterlaced coder s a further development of [16]. The three dagrams n Fg. 1 depct the system, ntra-frame coder, and nter-frame coder, respectvely. The DWT s performed on the nput frame or ndvdual feld dependng on whether t s n frame or feld mode. The choce of codng mode s due to the followng fact: although the feld mode may mprove the compresson effcency of the data, t may also mpede the overall compresson due to the header of the second pcture feld. The allocaton of bts s a tradeoff between hgher

2 compresson rato of the data and the overhead. When the frame mode s chosen, the whole pcture s coded exactly as n [16] wth some mprovement on MC. Otherwse, the even and odd felds are coded separately. The ntra-coder bascally uses the JPEG2000 Core codng engne wth mnor modfcaton [11]. For the nter-frame pctures, the Moton Compensated (MC) predcton s performed n the transform doman, and the resdue produced by the DPCM along the temporal drecton s sent to the EBCOT codng engne. Ths framework provdes flexblty and potental for more advanced MC technques and allows applcatons of a Perceptual Dstorton Measure (PDM) based on the Human Vson System (HVS) to mprove the subjectve vdeo qualty. The DWT based vdeo coders elmnate blockng artfacts that are commonly assocated wth block based DCT vdeo coders, especally at low bt rates. Furthermore, the mult-resoluton structure of the wavelet transform generally provdes a better adaptaton to the HVS than DCT. (a) System Before a detal dscusson of the ME/MC, a revew of the concept of Wavelet Block (WB) s presented. A three level DWT analyss operaton n Mallat decomposton s appled to the whole frame or ndvdual feld dependng on the selected mode. Each ndvdual subband s denoted as (l, o) where l s the frequency level and o s the orentaton of the subband. Level 0 has only one subband but all the other levels have three subbands each wth dfferent orentaton. These orentatons are represented as HL (horzontal-hgh and vertcal-low), LH (horzontal-low and vertcal-hgh) and HH (horzontal-hgh and vertcal-hgh), respectvely. For any coeffcent block wth the sze of 2 2 from subband (0, 0), there are three sblng coeffcent block n the three neghborng subbands (1, HL), (1, LH) and (1, HH), respectvely. In the next DWT level, there are three correspondng coeffcent blocks n (2, HL), (2, LH) and (2, HH), respectvely. Snce a dyadc DWT s used, the sze of the coeffcent block at each subsequent hgher resoluton level s ncreased by a factor of 2. Theore, at level 2 and 3, the sze of the coeffcent block s 4 4 and 8 8, respectvely. These related coeffcent blocks are grouped together nto a WB wth a sze 16 by 16. The formng procedure of WB s llustrated as n Fg. 2. Intra-frame Coder (c) Fg. 1 Inter-frame Coder System Block Dagrams

3 (a) Components of a wavelet block Illustraton of wavelet block arrays these three predctors works as the predcton for the MV of the current WB. Then the dfference between the moton nformaton of the current WB and ts predcton s calculated and sent to Adaptve Arthmetc Coder (AAC) for lossless codng. When the current pcture s coded n the frame mode, the WBs n the erence frame s compared wth the current WB wthn search wndow. When the current pcture s coded n the pcture mode, the ME, MP and MC are much more complex as lsted bellow: Fg. 2 Formng of wavelet blocks Moton Estmaton (ME) and Predcton (MP) can then be performed n the DWT doman based on block matchng algorthm. The Sum of Absolute Dfference (SAD) s used as the measure to compare the dfference between the current WB, whch s the k th wavelet block, wth the erence WB as n Eq. (1). 3 x + M /2 1 y + N /2 1, k, k ( ) (, ) (, ) k cur MAD dx dy = HL x y = 1 x = x y = y, k, k ( ) dx dy ( ) ( ) HL ( x +, y + ) + LH ( x, y ) HL ( cur 2 2 dx dy ( ) ( ) x +, y + ) + HH ( x, y ) HH ( x + cur 2 2 x + M /2 3 1 y + N / , k 3, k dx dy y } { LL 2 2 x y, + ) + (, ) cur 3 3 x = x y = y 3 3, k 3 3, k dx dy LL ( x +, y + ) where ( x, y ) s the ntal pont of the th-level subbands ( ) n the k th wavelet block. ( ) ( ) HL, LH and HH are th-level HL, LH and HH subbands of the erence frame, respectvely. LL stands for the horzontal low- vertcal low subband n the lowest resoluton. ( ) ( HL, ) ( ) LH, HH and LL has smlar defnton except cur cur cur cur that they are from the current WB. ( dx, dy ) s the moton vector measured at the orgnal spatal resoluton. In our software mplementaton, the MV s calculated at the lowest resoluton. M and N are the dmensons of the WB. In ths work, they are both 16. The WB n the erence frame/feld wth the smallest SAD s chosen as the matchng WB for MC. For the moton vector (MV) of each WB, we use that of ts left neghbor, the above neghbor and the above rght neghbor as the predctors. The medan of the moton nformaton from } { (1) 1) For the top feld n the current pcture, the erence feld could be the top feld and the bottom feld n the erence pcture, respectvely, and 2) For the bottom feld n the current pcture, the erence feld could be the top feld or the bottom feld n the erence pcture, also t could be the reconstructed top feld n the current pcture. The search wndow for MV s [ 4, + 4] n the LL band. Theore the value scope for Dfferental Moton Vector (DMV) s [ 8, + 8]. As a result, the alphabet to be coded conssts of 17 symbols. A lst of the mappng between the DMV values and symbols s provded n Table 1. Wtten et al s AAC algorthm has been adopted [17] for ts clear separaton between the codng engne and adaptve model. DMV value symbol Table 1 Mappng of DMV values and symbols After Moton Predcton (MP), the resdues between the wavelet coeffcents of the current frame/feld and the predcted values are sent to the nter-frame codng engne. Strctly speakng, the contexts of the nter-frame codng engne should be redesgned smply because the statstcs pattern of the resdues s qute dfferent from the ntra-frame data. Nevertheless, the same context desgn as ntra-frame coder s kept for demonstraton purpose. The man focus of ths paper s centered on the effects of adaptve frame/feld

4 mechansm. III. PRELIMINARY EXPERIMENTAL RESULT Prelmnary expermental results are presented here to demonstrate the mprovement of the reconstructed vdeo qualty wth the frame/feld swtch mode for the ntra-frame coder. The software s wrtten n C++ and runnng on a Wndows XP system. The computer s equpped wth Pentum D Dual Core 830 CPU and 1G memory. The coder arranges the pctures n I-frame followed by P-frame, and so on (IPIPIP ). In order to have a better understandng of the above fgures and PSNR values, we present the dynamc PSNR results of pure I-frame codng n Fg. 4. It can be seen that the change of the PSNR values of pure I-frame codng s more stable than hybrd codng. As result, vewers wll see obvous temporal fluctuatons from the hybrd codng results. Ths s one open problem of DWT based vdeo coders that need to be addressed. Btrate PSNR (db) (Mbt/s) Football Baloon and Pops Fram e Feld Adaptv e Fram e Feld Adap tve Table 2 PSNR values of dfferent codng modes (a) Football coded at 1.5Mbt/s Btrate PSNR values (db) (Mbt/s) Football Baloon and Pops MPEG-4 Proposed MPEG-4 Proposed Table 3 PSNR comparsons wth MPEG-4 Two vdeo sequences, Football and Baloon&pops, wth NTSC format are used for demonstraton. Each of them conssts of 260 frames. The blue dashed lnes and the dash-dot magenta lnes represent for the PSNR curves of frame mode and feld mode, respectvely. The sold red lnes depct the PSNR curves of adaptve mode whch appear to provde the best and most stable pcture qualty. A comparson of the PSNR performances between the proposed coder and the benchmark s also provded n Table 3. It s obvous the proposed coder outperform MPEG-4 v.2 wth respect to PSNR value under same bt rate. (c) Football coded at 3Mbt/s Football coded at 5Mbt/s

5 (d) Baloon & Pops coded at 1.5Mbt/s Football coded at 3Mbt/s (e) Baloon & Pops coded at 3Mbt/s (c) Football coded at 5Mbt/s (f) Baloon & Pops coded at 5Mbt/s (d) Baloon & Pops coded at 1.5Mbt/s Fg. 3 Dynamc PSNR performance of proposed coder (e) Baloon & Pops coded at 3Mbt/s (a) Football coded at 1.5Mbt/s

6 Fg. 4 (f) Baloon & Pops coded at 5Mbt/s Dynamc PSNR values of pure I-frame codng IV. CONCLUSION AND DISCUSSION A wavelet codec for nterlaced vdeo s presented n ths paper. Prelmnary results show that the reconstructed qualty of the vdeo s mproved by adoptng the feld codng mode. Dfferent felds and frame are used durng ME for the mnmum predcton error. The proposed coder obvously outperforms MPEG-4 v.2 wth respect to PSNR value. However, the temporal fluctuaton problem s also evdent as n other DWT based vdeo coders. Ths s to be addressed n the future. Currently the same set of contexts for the Bnary Arthmetc Codng (BAC) s used for both ntra- and nter-frame codng. The dstncton of context helps the entropy coder to partton the nformaton nto relatvely more statonary sets. Here the desgn of the context follows that of JPEG2000 [11]. The contexts desgn s hghly dependent on the statstcal behavor of the nformaton to be coded. For ntra-frame nformaton, t s natural to nhert the contexts of JPEG2000. However for nter-frame nformaton, a new set of contexts needs to be derved n order to acheve better compresson effcency. Further work wll nclude the converson to perceptual vdeo coder by replacng MSE wth HVS dstorton metrc and performance comparson wth H.264. The performance of the above proposed coder can be further mproved by replace MSE n the rate dstorton stage of the bt-plane coder wth Human Vsual System (HVS) based Perceptual Dstorton Measure. Also, from Fgure 3, persstent varatons n the PSNR values between adjacent frames typcally translates to temporal fluctuatons. Ths type of dstorton s a problem for both DCT and DWT based vdeo coders whch may be overcome through perceptual codng. References [3] Y.-Q. Zhang and S. Zafar, "Moton-Compensated Wavelet Transform Codng for Color Vdeo Compresson," IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol. 2, pp , [4] S. Zafar and Y.-Q. Zhang, "Multscale Vdeo Representaton Usng Multresoluton Moton Compensaton and Wavelet Decomposton," IEEE Journal on Selected Areas n Communcatons, vol. 11, pp , [5] D.G. Sampson, E. A. B. D. Slva, and M. Ghanbar, "Low Btrate Vdeo Codng Usng Wavelet Vector Quantsaton," IEEE Proceedng of Image Sgnal Processngs, vol. 142, pp , [6] D. Taubman and A. Zakhor, "A Common Framework for Rate and Dstorton Based Scalng of Hghly Scalable Compressed Vdeo," IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol. 6, pp , [7] S.A. Martucc, I. Sodagar, T. Chang, and Y.-Q. Zhang, "A Zero Tree Wavelet Vdeo Coder," IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol. 7, pp , [8] J.Y. Tham, S. Ranganath, and A.A. Kassm, "Hghly Scalable Wavelet -Based Vdeo Codec for Very Low Bt-Rate Envronment," IEEE Journal on Selected Areas n Communcatons, vol. 16, pp , [9] B.-J. Km, Z. Xong, and W.A. Pearlman, "Low Bt-rate Scalable Vdeo Codng wth 3-D Set Parttonng n Herarchcal Trees (3-D SPIHT)," IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol. 10, pp , [10] G. Hesng, D. Marpe, H.L. Cycon, and A.P. Petukhov, "Wavelet-based Very Low Bt-rate Vdeo Codng Usng Image Warpng and Overlapped Block Moton Compensaton," IEE Proceedngs of Vson, Image and Sgnal Processng, vol. 148, pp , [11] D.S. Taubman and M.W. Marcelln, JPEG2000: Image Compresson Fundamentals, Standards and Practce: Kluwer Academc Publshers, [12] Y.-H. Yu and C.-J. Tsa, "A Model-based Rate Allocaton Mechansm for Wavelet-based Embedded Image and Vdeo Codng," presented at IEEE Internatonal Symposum on Crcuts and Systems, [13] Z. Gao and Y.F. Zheng, "An Innovatve Temporal Wavelet Flterng Scheme for Vdeo Codng," presented at IEEE Internatonal Conference on Image Processng, [14] D.S. Turaga, M. v. d. Schaar, and B.P. Popescu, "Complexty Scalable Moton Compensated Wavelet Vdeo Encodng," IEEE Transactons on Crcuts and Systems for Vdeo Technology, vol. 15, pp , [15] A. Secker and D. Taubman, "Hghly Scalable Vdeo Compresson wth Scalable Moton Codng," IEEE Transactons on Image Processng, vol. 13, pp , [16] L.M. Me, H.R. Wu, and D.M. Tan, "A New DWT/MC/DPCM Vdeo Compresson Framework Based on EBCOT," presented at SPIE VCIP 2005, Vsual Communcatons and Image Processng, Bejng Chna, [17] I.H. Wtten, R.M. Neal, and J.G. Cleary, "Arthmetc Codng for Data Compresson," Communcatons of The ACM, vol. 30, pp , [1] A.S. Lews and G. Knowles, "Vdeo Compresson Usng 3D Wavelet Transforms," IEEE Electroncs Letters, vol. 26, pp , [2] H.R. Wu and K.R. Rao, Dgtal Vdeo Image Qualty and Perceptual Codng: CRC Press, 2005.