Compression of Concentric Mosaic Scenery with Alignment and 3D Wavelet Transform Lin Luo a, Yunnan Wu a, Jin Li b, Ya-Qin Zhang b,

Transcription

1 Compression of Conentri Mosi Senery with Alignment n 3D Wvelet Trnsform Lin Luo, Yunnn Wu, Jin Li, Y-Qin Zhng, University of Siene & Tehnology of Chin Hefei, Anhui Provine, , P.R.Chin Mirosoft Reserh Chin 5F Reserh, Sigm Ctr, 49 Zhihun Ro, Hiin, Beijing, , P.R.Chin Emil: {jinl, yzhng}@mirosoft.om ABSTRACT As new sene representtion sheme, the onentri mosi offers quik wy to pture n moel relisti 3D environment. This is hieve y shooting lot of photos of the sene. Novel views n e renere y pthing vertil slits of the pture shots. The t mount in the onentri mosi is huge. In this work, we ompress the onentri mosi imge rry with 3D wvelet sheme. The propose sheme first ligns the mosi imges, n then pplies 3D wvelet trnsform on the ligne mosi imge rry. After tht, the wvelet oeffiients in eh sun re split into ues, where eh of the ues is enoe inepenently with n emee lok oer. Vrious ue itstrems re then ssemle to form the finl ompresse itstrem. Experimentl result shows tht the propose 3D wvelet oer hieves goo ompression performne. Keywors: Imge se renering, onentri mosi, ompression, 3D wvelet, emee oing, rteistortion optimiztion. 1. INTRODUCTION Imge se renering istinguishes itself s promising tool in the onstnt pursuit of omputer generte photo-relism. Using plenopti funtion[1], imge se renering moels 3D sene y reoring the light ry t every spe lotion n pointing towr every possile iretion. The plenopti funtion of full 3D sene is of 5 imensions, with 3 imensions for the viewing position n nother 2 imensions for the pointing iretion. Lightfiel[2] n Lumigrph[3] re two exmples of the 4D plenopti funtions, whih pture the omplete pperne of 3D sene/ojet if the viewer/ojet n e oune in ox. Shum n He[4] propose COnentri Mosi (COM), 3D plenopti funtion restriting viewer movement on plne. The COM hs the ility to esily onstrut relisti 3D wlkthrough y rotting single mer t the en of em, with the mer pointing outwr n shooting imges s the em rottes. When novel view is renere, we just split the view into vertil ry slits n serh for eh slit its ounterprt in existing shot imges. The tehnology is terme onentri mosi euse the t struture is stk of mosi imges long ifferent riuses. Though esy to rete 3D wlkthrough, the mount of t ssoite with the onentri mosi is huge. As n exmple, onentri mosi of 240 pixels in height, 1350 pixels in irulr (horizontl resolution), n 320 pixels in rius (epth resolution, whih provies the funtion of 3D view) oupies totl of 297 meg ytes (MB). In [4], vetor quntiztion ws use to ompress the COM sene with ompression rtio of 12:1. The ompresse t is 25MB, whih is still fr too lrge onsiering tht it will tke one hour to ownlo COM sene over 56kps moem onnetion. We my ompress eh iniviul shot of COM with high performne still imge oer, suh s JPEG or JPEG 2000[8]. Beuse COM sene onsists of multiple highly orrelte shots, this my not e very effiient. An lterntive pproh is to tret COM sene s vieo n ompress it with vieo oer, suh s MPEG[11]. The pproh oes not tke the rnom ess requirement into onsiertion, n thus is not prtil for COM renering. Moreover, though the PSNR LUO 1/12

2 performne of the MPEG oer my e stisftory, the result COM sene my not e of high qulity in the renering stge, euse vieo sequene where MPEG is optimize is plye ontinuously, while the COM sene is viewe sttilly. Consequently, highly effiient ompression is inispensle for the pplition of onentri mosi. In this work, we ompress the imge rry of the onentri mosi with 3D wvelet trnsform. The lgorithm is seprte into four funtionl loks: the lignment, 3D wvelet trnsform, slr quntizer & emee lok oer, n itstrem ssemler. First, the imge rry is ligne so tht the mosi imges looks s similr s possile. The ligne imges re then eompose y 3D lifting opertion, whih reues the require memory for the forwr n inverse trnsform of the huge COM imge rry. Vrious 3D wvelet pkets re lso investigte in this stge. After tht, the wvelet oeffiients re ut into ues, n eh ue is then ompresse inepenently into n emee itstrem. Finlly, glol rte-istortion optimizer is use to ssemle the itstrem. There o exist severl 3D wvelet oing lgorithms for vieo[6][7]. The propose lgorithm iffers from prior shemes y proviing more memory effiient lifting implementtion n lok oing struture so tht the COM sene n e esily esse lolly n isplye with ifferent resolution in the renering stge. The pper is orgnize s follows. The t struture of the onentri mosi n its renering sheme re introue in Setion 2. In setion 3-6, we explin in etils the funtions of the lignment, 3D wvelet, quntiztion & entropy oing, n itstrem ssemler. Experimentl results re shown in Setion 7. A onlusion is given in Setion THE CONCENTRIC MOSIAC SCENE A onentri mosi (COM) sene is otine y mounting single mer t the en of em, n shooting imges t regulr intervls s the em rottes, s shown in Figure 1. Let the mer shots tken uring the rottion e enote s (n,w,h), where n inexes the mer shot, w inexes the horizontl position within shot, n h inexes the vertil position. Let N e the totl numer of mer shots, W n H e the horizontl n vertil resolution of eh mer shot, respetively. In the COM sene, the originl imge t re rerrnge into mosi imges, where the mosi imge F(w)={f(w,n,h) n,h} onsists of vertil slits t position w of ll mer shots. Imge F(w) n e onsiere s tken y virtul slit mer rotting long irle o-entere with the originl em with rius r=rsinθ, where R is the rius of the rotting em, r is the equivlent rius of the slit mer, n θ is the ngle etween ry w n the mer norml. Sine imges F(w) re set of onentri mosi imges with ifferent rius, the sene representtion is terme onentri mosi. θ Cmer Bem r R Figure 1 Illustrtion of the onentri mosi LUO 2/12

3 Let the horizontl fiel of view of the mer e FOV, the COM representtion n rener ny view pointing to ny iretion within the irle R sin(fov/2), s shown in Figure 2. Let (p,β) e the lotion of new view point in polr oorintes, let the view e seprte into set of vertil slits, with one slit pointing to the iretion of α. With elementry geometry, it woul not e iffiult to show tht the viewing slit is equivlent to slit on the onentri mosi irle with rius p sin(β α) n ngulr oorinte (π/2+α). The proess is shown in Figure 2. By olleting multiple ry slits of the new view in the ove wy, the renere imge n e shown. (p,β) π/2+α α Figure 2 Renering of the onentri mosi. Thus, y reoring ll the mosi imges F(w), we equivlently pture the ense 3D wlkthrough view (the COM sene) within the irle R sin(fov/2). A single mosi imge F(W/2) provies the enter pnorm of the sene whih enles the viewer to rotte t the enter of mer trk, n it is the rest of the mosi imges tht supply the itionl informtion require y the 3D wlkthrough. The originl t of the COM sene is threeimensionl, with strong orreltion mong ifferent mosi imges. We therefore evelope 3D wvelet sheme for the ompression of the COM sene. The four iniviul funtion loks of the COM sene ompression - the lignment, 3D wvelet trnsform, slr quntizer & emee lok oer, n the itstrem ssemler will e esrie in etils in the following setions. 3. ALIGNMENT The first step in our propose ompression frmework is to lign the mosi imges so tht orreltion mong ifferent mosi imges is mximize. The lignment n e performe impliitly, onsiering only the pturing proess; lterntively, it my e performe expliitly, y rottionlly mthing two jent mosi imges. The impliit mth tkes into onsiertion tht prllel ry slits of frwy ojet ppers similr. Thus the mosi imge is ligne: g(w,n,h)=f(w,n- (w),h) where ll rys g(*,n,h) point to the sme iretion. The lignment ftor (n) n e lulte s: N 2( w W / 2) FOV ( w) = rtn tn (1) 2π W 2 Alterntively, we my expliitly mth two onseutive mosi imges n reue the men solute error (MAE) or the men squre error (MSE) etween the two rottionlly shifte mosi imges: N H 1 MAE( w) = f ( w, n ( w), h) f ( w 1, n ( w 1), h) (2) N H n= 1 h= 1 N H 1 2 MSE( w) = [ f ( w, n ( w), h) f ( w 1, n ( w 1), h) ] (3) N H n= 1 h= 1 The onseutive isplement (w)- (w-1) tht minimizes the mthing MAE or MSE is the reltive lignment ftor etween mosi imges. By setting the lignment of speifi mosi imge to zero, e.g., tht of the 0 th LUO 3/12

4 mosi imge (0)=0, we my erive the solute lignment ftors of other imges. Experiments hve een performe n results show tht the ompression performne of expliit lignment outperforms tht of impliit lignment. Therefore, we use the expliit lignment in the following. The lignment ftor (n) is reore in the ompresse itstrem. 4. 3D WAVELET TRANSFORM In the next step, 3D seprle wvelet trnsform is pplie on the onentri mosi (COM) imge rry to eorrelte the imges in ll three imensions, n to ompt the energy of the imge rry into few lrge oeffiients. In ition to energy omption, the multi-resolution struture provie y the 3D wvelet trnsform my lso e use to ess reue resolution mosi imge rry in the renering, in se tht there is not enough nwith or omputtion power to ess the full resolution COM sene, or the isply resolution of the lient evie is low. The entire COM sene is too lrge to e loe into memory simultneously to perform the 3D wvelet trnsform. For the ske of memory sving n omputtionl simpliity, 3D lifting sheme with frme/line uffer hs een implemente. A smple one-imension i-orthogonl 9-7 lifting wvelet is illustrte in Figure 3. Corresponing to the three imensions, there re three ifferent lifting units the frme lifting, the line lifting n the horizontl lifting. The originl t re fe into the lifting unit one element t time, where single element is one mosi imge for the frme lifting, one line for the line lifting, or single pixel for the horizontl lifting. The 4 stge forwr lifting, shown in the left of Figure 3, n e formulte elow: y1(2i + 1) = x(2i + 1) + [ x(2i) + x(2i + 2) ] y2(2i) = x(2i) + [ y1(2i 1) + y1(2i + 1) ], (4) h( i) = y3(2i + 1) = y1(2i + 1) + [ y2(2i) + y2(2i + 2) ] l( i) = y4(2i) = y2(2i) + [ y3(2i 1) + y3(2i + 1) ] where x(i) is the originl t, y s (i) is the sth stge lifting, h(i) n l(i) re the high n low pss oeffiients, respetively. The oeffiients of the 9-7 iorthogonl lifting re =-1.586, =-0.052, =0.883 n = The elementry opertion of lifting is Y=(L+R)*+X, whih is epite in the right of Figure 3. Sine the elementry opertion n e esily inverse s X=Y-(L+R)*, the inverse of the lifting n e esily erive, s shown in the mile Figure 3. Trnsform Inverse Trnsform L Buffere oeffiient. x0 x1 x2 x3 x4 x5 x6 x7 x8 L0 H0 L1 H1 L2 H2 L3 H3 L X Y R Y = ( L + R ) * + X X = ( L + R ) * ( - ) + Y Figure 3 One imensionl forwr n inverse lifting opertion LUO 4/12

5 The elementry opertion of the lifting onsists of 2 itions n 1 multiplition opertion. The verge omputtion lo is thus 4 itions n 2 multiplitions per oeffiient. In ontrst, the verge omputtion of the tritionl onvolution implementtion of the sme 9-7 iorthogonl wvelet is 8 itions n 4.5 multiplitions, whih more thn oules the omputtion lo. The originl t X is no longer neee one the oeffiient Y is lulte; therefore, the result Y my e store t the sme memory unit tht hols X. Suh inple lultion my e use to reue the memory require for the lifting opertion. In ft, in the 9-7 iorthogonl lifting, only 6 elements nee to e uffere. Shown in the irle of Figure 3, three elements re intermeite results of the lifting, n the other three re originl oeffiients. For every two input t points, four elementry lifting opertions re performe n one low pss n one high pss oeffiients re output. The require memory uffer to perform the 3D lifting is thus 6 frmes for the frme lifting, n 6 lines for the line lifting. A single sle 3D lifting is illustrte in Figure 4, where frme lifting is performe first, then line lifting, n finlly horizontl lifting. Fn F0 F1F2 F3 Line lifting: uffersize = 6 lines Frme lifting uffersize = 6 frmes Horizontl lifting Figure 4 Single sle three-imension lifting. y z y z y z x x x () () () Figure 5 Multiple sle 3D lifting, () two-level mllt in ll iretions, () two-level x eomposition + two-level (y,z) mllt, () two-level z eomposition + two-level (x,y) mllt. In the COM sene oing, more thn one single sle lifting opertion is usully performe, thus some of the result wvelet suns my e further eompose. We my hoose to eompose only the low pss n of the frme, line n horizontl lifting, suh eomposition eing lle the mllt eomposition. A two-level full mllt eomposition is epite in Figure 5(), where the lifting long the x, y n z xes is the frme, line n horizontl lifting, respetively. Alterntively, we my first perform wvelet eomposition long one xis, n then eompose the plne spnne y the other two xes. For exmple, we my first perform two-level eomposition long the x xis, n then two-level full mllt eomposition on the (y, z) plne; the result is LUO 5/12

6 shown in Figure 5(). Another lifting onfigurtion shown in Figure 5() first performs two-level z xis eomposition, n then two-level full mllt eomposition on the (x, y) plne. All suh eompositions re forms of wvelet pket, n the opte wvelet pket struture will e reore in the ompresse COM itstrem. 5. SCALAR QUANTIZATION AND EMBEDDED BLOCK CODING The wvelet trnsforme oeffiients re hoppe into ues, whih re ompresse y slr quntizer n emee lok oer. The ompresse lok itstrems re first uffere, n then ssemle y rteistortion optimize ssemler fter ll loks hve een enoe. Even though the quntiztion n entropy oing re performe on lok-y-lok sis, the wvelet trnsform opertes on the entire onentri mosi (COM) t, therefore, no expliit loking rtift is visile in the eoe COM sene. The lok oing struture selete for the COM sene ompression hs the following vntges: 1. Benefit of lol sttistil vritions. The sttistil property my not e homogenous ross the entire COM t set. Sine eh lok of oeffiients is proesse n enoe inepenently, the enoer my tune to lol sttistil property, n thus improve oing performne. The vrition of the sttistis ross the COM sene my lso e use in the itstrem ssemler, n its my e istriute in rte-istortion optimize fshion ross the COM sene. 2. Esy rnom ess. With the lok quntiztion n entropy oing, we my rnomly ess portion of the COM sene without eoing the entire t set. From the esse region require y the renering unit, we my erive the relte loks using the wvelet sis n wvelet eomposition sheme. Only the itstrems of the esse loks re eoe. 3. Low memory requirements. The lok struture lso wives the nee to uffer the entire volume of COM oeffiients. Only K frmes of oeffiients nee to e uffere for eh sun, where K is the size of the lok in frme iretion. One K frmes of oeffiients rrive, they re hoppe into loks, quntize n entropy enoe. We still uffer the ompresse itstrems of the oeffiients, however, they re muh smller. The eoer sie requires even less memory sine the ompresse itstrem is only prtilly esse n eoing is only performe on those loks relte to rener urrent view. The quntizer is simple slr quntizer with step size Q n e zone 2Q. The forwr n inverse quntizer n e formulrize s: w/ Q w > 0 ( q + 0.5) Q q > 0 =, (5) q = 0 w 0 w = 0 q = 0 w/ Q w < 0 ( q 0.5) Q q < 0 where w n w re the originl n reonstrute oeffiients, q is the quntizer output, x n x re the floor n eiling funtions, respetively. Sine the itstrem ssemler is use, the quntiztion step size Q no longer ontrols the finl oing qulity. We my simply hoose smll onstnt quntiztion step size Q, suh s Q=1.0 to ensure tht the COM sene is ompresse with suffiient qulity efore ssemling. Mny implementtions of the entropy enoer re fesile. Due to the use of the itstrem ssemler, the entropy oer must hve the emeing property, i.e., the ompresse lok itstrem n e trunte t lter stge with goo ompression performne t suh reue itrte. Three ifferent emee entropy oers re implemente with ifferent omplexity vs. performne treoff. They re ll itplne oers. Let the lok uner onsiertion e enote y C, let (i,l) e the lth most signifint it of the solute vlue of oeffiient i. Let B l e the lth it plne, whih onsists of ll its t the sme signifine level l. Let L e the totl numer of LUO 6/12

7 itplnes, where it is stisfie tht for ll oeffiients x in C, x <2 L, n there is t lest one oeffiient x so tht x >=2 L-1. All three oers enoe the lok itplne y itplne, n ll of them iterte from the most signifint itplne L-1 to the lest signifint itplne 0. If the ompresse itstrem is trunte lter, t lest the most signifint itplnes of ll oeffiients re essile, n therefore, the lok n still e eoe with goo qulity. For eh oeffiient x i t ertin itplne k, if ll its in prior itplnes re 0, i.e., (i,l)=0 for ll l>k, the oeffiient is lle insignifint, n signifint vise vers. For eh itplne, the its of insignifint oeffiients re enoe in signifine ientifition moe, while the its of signifint oeffiients re enoe in refinement moe. The it in the refinement moe ppers uniformly s 0 or 1, n hene leves less room for ompression. Due to the energy omption property of wvelet trnsform, the it in the signifine ientifition moe skews highly towr 0, therefore, it is the tsk of signifine ientifition to lote the oeffiient whih turns signifint in urrent itplne, i.e., those its tht stisfy (i,k)=1 n (i,l)=0 for ll l>k. The propose oers iffer primrily in how the signifine ientifition is performe. During the emee oing, the oing rte R(l) n istortion D(l) of the lok re reore t the en of eh itplne. The rte R(l) n e esily erive from urrent enoing itstrem length. The istortion D(l) my e lulte y mesuring the ifferene etween the originl n reonstrute oeffiients t urrent stge. A look-up tle s the one presente in [8] my e use to spee up the istortion lultion. Alterntively, we my estimte the istortion D(l) with tehnology presente in the rte-istortion optimize emee oer (RDE)[10]. Clulting istortion D(l) is of ourse more urte n is enefiil for the oing performne, however, the estimtion of the istortion D(l) is muh fster n introues less overhe. The reore rteistortion performne of the lok will e use in the itstrem ssemler. The etils of the three implemente lok entropy oer re esrie elow: 5.1. TREE CODER In the tree oer, the insignifint oeffiients re groupe y ot-tree, while the signifint oeffiients re split into iniviul pixels. Tree oing is effiient euse lrge re of insignifint its re groupe together n represente with single 0. As the oer itertes from the most signifint itplne to the lest signifint itplne, the ot-tree of insignifint oeffiients grully split n the lotions of the signifint oeffiients re ientifie. The proeure of the tree oer is s follows: Step 1. Initiliztion Four lists re estlishe, whih re the list of insignifint sets (LIS), the list of nite sets (LCS), the list of insignifint pixels (LIP), n the list of signifint pixels (LSP). LIS is onsiste of sets where ll oeffiients re insignifint; LCS is onsiste of sets where ll oeffiients re insignifint ut t lest one oeffiient will e signifint in this itplne; LIP is onsiste of single insignifint oeffiients; n LSP is onsiste of single signifint oeffiients. Elements in the four lists my move oring to Figure 6. Elements in LCS my split n move to LIS, LSP, LIP or form nother element of LCS. However, LIS element n only move to LCS; LIP element n only move the LSP; n one n element moves to LSP, it stys there. Figure 6 Stte trnsition of the tree oer LUO 7/12

8 Step 2. Bitplne oing. In eh itplne oing, we first exmine sets in LIS one y one. If t lest one oeffiient eomes signifint in this itplne, 1 is enoe n the set moves to LCS. Otherwise, 0 is enoe n the set remins in LIS. Then the sets in LCS re exmine. For set in LCS, sine it is gurntee tht t lest one oeffiient will eome signifint in this itplne, the set splits long the three xes into 8 su-sets: 1,, 8. If set is of size 2x2x2, 8 hil pixels re generte. For eh suset/pixel i, sttus it s i is estlishe whih is 0 if the suset/pixel is insignifint, n is 1 otherwise. The 8-it sttus string s={s 1,...,s 8 } is Huffmn enoe. The suset/pixel i is ple into LCS/LSP if the sttus it is 1, n into LIS/LIP if the sttus it is 0. If pixel i moves into LSP, its sign is enoe. After ll sets in LCS hve een proesse, ll oeffiients in LIP in the eginning of urrent itplne oing re exmine. If the oeffiient eomes signifint, 1 is enoe followe y the sign of the oeffiient, n the oeffiient moves to LSP. Otherwise, 0 is enoe. Finlly, ll oeffiients in LSP in the eginning of urrent itplne oing re refine. Although oneptully the tree oer enoes the lok itplne y itplne, in tul implementtion, tree is uilt for the oeffiient lok n ll susequent exmintion is performe on tht tree. The omputtionl omplexity of the tree oer is thus the lowest mong the three GOLOMB-RICE (GR) CODER The GR oer ientifies the signifint oeffiient with the ptive inry Golom-Rie (ABGR) oer, whih is simplifie run-length oer. More etils of the ABGR oer n e referre to [9]. For etter ompression performne, we lssify the insignifint oeffiients further into two tegories, i.e., the preite signifint its where t lest one of the 26 nerest neighors re signifint, n the preite insignifint its where ll the 26 neighors re insignifint. In eh itplne oing, we first enoe the preite signifint its, then the preite insignifint its, n finlly the refinement its. Suh oing orer is empirilly etermine y the rte-istortion property of ifferent it sets. We thus sn the oeffiient lok three times, eh time with serpentine snning orer tht lterntes etween left-to-right n right-to-left visiting of row of oeffiients. The preite signifint n insignifint its re enoe y inepenent ABGR oers with seprte prmeters. The refinement its re not enoe n re just sent iretly to the lok itstrem. The ABGR oer is use to enoe the run to next oeffiient tht will eome signifint in preite signifint or insignifint sn. If the run is greter thn or equl to 2 m, 0 is enoe to represent zero run of 2 m, while the rest of the run is further exmine. Otherwise, it 1 is enoe followe y m its of the inry representtion of the run length n the sign of the signifint oeffiient. In essene, the ABGR oer is Huffmn oer tht ssigns 1 it for run greter thn or equl to 2 m, n m+1 its for run smller thn 2 m. m is n ptive prmeter in the ABGR oer, n is etermine y the stte trnsition tle speifie in [12] CONTEXT-BASED ARITHMETIC CODER The ontext-se rithmeti oer resemles tht of the GR oer. The lok is still enoe y itplnes, n eh itplne is still snne three times, s preite signifine, preite insignifine n refinement. However, in the preite signifint n preite insignifint sn, the insignifint it is enoe with n rithmeti oer using ontext erive from the 26 signifint sttuses of the neighors of urrent oeffiient. The 26 signifint sttuses re groupe into tegories so tht the numer of ontext is reue to voi ontext ilution. The tehnique ers resemlne to the one use in JPEG 2000[8]. The oer yiels the est ompression performne, however, it is lso omputtionlly most expensive sine ontext lultion is reltively time onsuming LUO 8/12

9 6. RATE-DISTORTION OPTIMIZED BITSTREAM ASSEMBLER After ll the loks of oeffiients hve een entropy enoe, itstrem ssemler is use to optimlly llote the its mong ifferent loks. We hve otine the rte-istortion urves of iniviul loks uring the emee oing stge. The lok istortion is further multiplie y the energy weight of the sun: DWeighte = w D Originl (6) where the sun weight w is the gin (energy) of the lifting synthesis funtion. For the i-orthogonl 9-7 filter, the gin for the low pss filter is w L =1.299, n the gin for the high pss filter is w H = For wvelet sun with n low pss lifting n m high pss lifting, the energy weight of the sun n e lulte s w = w n w m L H, no mtter whether the lifting is performe long the frme, line or horizontl xes. The rte-istortion theory inites tht optiml oing performne n e hieve if ll loks operte on the sme rte-istortion urve. The funtionlity of the itstrem ssemler is thus to fin the ommon rteistortion slope of ll loks, n lulte the inlue its for eh lok. The two funtionlities re performe elow: 1. Fin the truntion it rte for eh lok Given rte-istortion slope, the oing rte of eh lok is etermine y the portion of the itstrem with operting rte-istortion slope greter thn the given slope. In essene, we fin the tngent on the rteistortion urve of the lok tht is equl to the given rte-istortion slope, n the operting it rte t this tngent point is the oing itrte for this lok. 2. Fining the optiml ommon rte-istortion slope The optiml ommon rte-istortion slope is foun through i-setion metho. We first set two rteistortion slope threshol λ min n λ mx, where the optiml rte-istortion slope is enlose in the intervl (λ min, λ mx ). A urrent threshol λ=(λ min +λ mx )/2 is lulte. Given urrent threshol λ, eh lok is exmine n the operting itrte is lulte. Current oing rte is the sum of the oing rte of ll loks. Depening on whether urrent oing rte is lrger or smller thn the esire rte, the lower or upper limit of the threshol is upte oringly. The serh stops fter numer of itertions, or s urrent oing rte is lose to the esire rte with ertin perentge. During the serh of the optiml oing rte, only the rte-istortion funtion of eh lok is exmine, no lok nees to e re-enoe. Therefore, the serh n e performe pretty fst. D 1 D 2 R 1 R 2 D 3 r 1 r 2 D 4... r 3 R 3 Figure 7 Illustrtion of the itstrem ssemling r 4 R LUO 9/12

10 The proess is illustrte in Figure 7, where the sene onsists of 4 loks. The weighte rte-istortion urve of eh lok is lulte uring the emee oing stge. We serh for n optiml rte-istortion slope whih is tngent with the rte-istortion urves of ll loks. The lok itstrem is then trunte t the tngent point, n the trunte itstrem segments of ifferent loks re ssemle together to form the ompresse itstrem. The rte-istortion slope is juste so tht output itrte is equl or lose to the esire itrte. 7 EXPERIMENTAL RESULTS The performne of the onentri mosi (COM) oing with 3D wvelet is emonstrte with experimentl results. We lso investigte the effiieny of vrious 3D wvelet pket trnsforms n lok entropy oing shemes. One test t set is the COM sene Loy (Figure 8), whih is shot with 1350 frmes t resolution 320x240. Forming the mosi imges, the Loy sene omprises of 320 mosi imges of resolution 1350x240. Another test t set is the sene Kis (Figure 9), whih omprises of 352 mosi imges of resolution 1462x288. All senes re in the YUV olor spe, with U n V omponents susmple y ftor of 2 in oth the horizontl n vertil iretion. The Loy sene is ompresse t 0.2 pp (it per pixel) n 0.4pp, respetively. The Kis sene hs more etils, n is thus ompresse t 0.4pp n 0.6pp, respetively. The ojetive pek signl-to-noise rtio (PSNR) is mesure etween the originl COM sene n the eompresse sene: PSNR = 10log10, (7) 1 2 [ f ( w, n, h) f '( w, n, h) ] N W H where f(w,n,h) n f (w,n,h) re the originl n reonstrute COM sene, respetively. The PSNR of the Y, U n V omponents re ll presente in the experiment, though it is the PSNR of the Y omponent tht mtters most, s roun 90% of the itstrem is forme y the ompresse itstrem of the Y omponent. In the first experiment, ifferent 3D wvelet pket eompositions re investigte. Let lifting long the x, y n z xes orrespon to the frme, line n horizontl lifting, respetively. Five wvelet pket eomposition strutures re evlute: Struture A: 4-level mllt in ll 3 iretions (frme, line n horizontl), Struture B: 4-level eomposition long y xis, followe y 3-level (x, z) mllt eomposition, Struture C: 4-level eomposition long y xis, followe y 4-level (x, z) mllt eomposition, Struture D: 5-level eomposition long z xis, followe y 4-level (x, y) mllt eomposition, Struture E: 5-level eomposition long x xis, followe y 4-level (y, z) mllt eomposition. The wvelet oeffiients within eh sun re hoppe into loks n re enoe y the tree oer with lok size set to 16x16x16. The test t set is the Kis sene ompresse t 0.6 pp. The results re liste in Tle 1. We oserve from Tle 1 tht full mllt eomposition my not yiel n optiml ompression result. By further eomposing some ns long the y (line) xis with struture C, ompression performne improves y out 0.5B. Compring wvelet pket struture C n D, it is oserve tht further eomposing existing wvelet suns n improve ompression performne slightly. However, we o oserve tht the gin hieve y further wvelet eomposing eyon 4-level is rther limite. We my hoose to first eomposing long the z (frme) or x (horizontl) xis with struture D n E, however, the performne is inferior to the wvelet pket struture C. Therefore, in the following experiment, the wvelet pket struture C is use, i.e., first 4-level line lifting is pplie, followe y 4-level mllt lifting in oth the frme n horizontl iretion LUO 10/12

11 Tle 1 COM sene ompression with ifferent wvelet pket eomposition strutures. Wvelet Pket Struture A B C D E PSNR: Y (B) PSNR: U (B) PSNR: V (B) In the seon experiment, we investigte the performne of vrious lok entropy oers with lok size 16x16x16. We lso ompre the performne of 3D COM sene oing with tht of MPEG-2, whih trets the entire COM sene s vieo n ompresse it with MPEG-2 oe ownloe from In MPEG-2, only the first imge is enoe s I frme, the rest imges re enoe s P frmes. The MPEG-2 is use here s enhmrk, however we o note tht rnom ess using MPEG-2 is not so strightforwr, mking it not suitle for rel time renering. Moreover, though the PSNR performne of the MPEG oer is stisftory, there is no gurntee of the qulity of the renere COM sene, s PSNR of the MPEG oe vieo usully flututes long the sequene. The results re shown in Tle 2. It is oserve tht the performne of the tree oer is very lose to tht of the Golom-Rie oer. Note tht the omputtion omplexity of the tree oer is lower thn the Golom-Rie oer, while the memory requirement of the tree oer is slightly higher, espeilly t high itrte. Sine the lok enoer opertes only on 3D lok of size 16x16x16, the memory onsume y the tree oer is limite. We thus fvor the tree oer over the Golom-Rie oer. The more omplite rithmeti oer improves the ompression performne y 0.5B. Compring MPEG-2 with 3D wvelet COM sene ompression y rithmeti oer, we oserve tht the 3D wvelet oer loses y 0.3B on verge. The performne is still omprle to tht of MPEG-2. We my improve the performne of the 3D wvelet COM oer with further tuning. Aitionlly, the resolution slility offere y the 3D wvelet n the qulity slility offere y the lok emee oer mkes the 3D wvelet COM sene oe ttrtive in numer of environments, suh s Internet streming n rowsing. Tle 2. 3D onentri mosi sene ompression results. LOBBY (0.4pp) LOBBY (0.2pp) KIDS (0.4pp) Y: 32.2 Y: 30.1 U: 38.7 U: 36.6 V: 38.1 V: 36.7 MPEG-2 (B) Y: 34.8 U: 39.9 V: D wvelet + tree oer (B) Y: 34.5 U: 41.6 V: D wvelet + Golom-Rie oer (B) 3D wvelet + rithmeti oer (B) Y: 34.4 U: 41.5 V: 41.2 Y: 35.0 U: 41.9 V: 41.5 Y: 31.4 U: 40.1 V: 39.7 Y: 31.3 U: 40.0 V: 39.7 Y: 31.9 U: 40.3 V: 39.9 Y: 29.0 U: 35.8 V: 36.6 Y: 29.0 U: 35.7 V: 36.5 Y: 29.4 U: 36.5 V: 37.2 KIDS (0.6pp) Y: 31.9 U: 38.0 V: 38.1 Y: 31.1 U: 37.3 V: 38.0 Y: 31.0 U: 37.0 V: 37.9 Y: 31.5 U: 38.0 V: CONCLUSION A new pproh for the ompression of onentri mosi (COM) senery using 3D wvelet trnsform is presente in this pper. The propose lgorithm onsists of 4 funtionl loks: lignment, lifting wvelet eomposition, slr quntizer & lok entropy oer, n itstrem ssemler. The ompression performne of the 3D wvelet COM oe is omprle to tht of MPEG-2. Moreover, the 3D wvelet oe offers the resolution n qulity slility whih my e very useful in the Internet environment LUO 11/12

12 REFERENCES [1] L. MMilln n G. Bishop, Plenopti moeling: n imge-se renering system, Computer Grphis Proeeings, Annul Conferene series (SIGGRAPH 95), pp , Aug [2] M. Levoy n P. Hnrhn, Light fiel renering, Computer Grphis Proeeings, Annul Conferene series (SIGGRAPH 96), pp , New Orlens, Aug [3] S. J. Gortler, R. Grzeszzuk, R. Szeliski, n M. F. Cohen, The lumigrph, Computer Grphis Proeeings, Annul Conferene series (SIGGRAPH 96), pp , New Orlens, Aug [4] H.-Y. Shum n L.-W. He. Renering with onentri mosis, Computer Grphis Proeeings, Annul Conferene series (SIGGRAPH'99), pp , Los Angeles, Aug [5] A. Si, "A new fst n effiient imge oe se on set prtitioning in hierrhil trees", IEEE Trns. on Ciruit n System for Vieo Tehnology, Vol. 6, No. 3, pp , Jun [6] A. Wng, Z. Xiong, P. A. Chou, n S. Mehrotr, ``3D wvelet oing of vieo with glol motion ompenstion,'' Dt Compression Conferene (DCC'99), Snowir, UT, Mr [7] D. Tumn n A. Zkhor, Multirte 3-D sun oing of vieo, IEEE Trns. on Imge Proessing, Vol. 3, No. 5, pp , Sept [8] Verifition moel -ho, JPEG 2000 verifition moel 5.0, ISO/IEC JTC1/SC29/WG1/N1420, Ot [9] E. Orentlih, M. Weinerger, n G. Seroussi, A low-omplexity moeling pproh for emee oing of wvelet oeffiients, Dt Compression Conferene (DCC 98), Slt Lke City, Uth, Mr [10] J. Li n S. Lei, An emee still imge oer with rte-istortion optimiztion, IEEE Trns. On Imge Proessing, Vol. 8, No. 7, pp , Jul [11] J. L. Mithell, W. B. Penneker, C. E. Fogg, n D. J. LeGll, MPEG vieo ompression stnr. New York: Chpmn n Hll, [12] JPEG-LS AHG, JPEG-LS prt 2 CD version 1.0, ISO/IEC JTC1/SC29/WG1N1557, Mui, Hwii, De Figure 8 Conentri mosi sene Loy. Figure 9 Conentri mosi sene Kis LUO 12/12