Motion Compensated Compression of Computer Animation Frames. are combined or manipulated and run through the compression

Transcription

1 Moton Compensated Compresson of Computer Anmaton Frames Bran K Guenter y, Hee Cheol Yun, and Russell M Mersereau z Abstract Ths paper presents a new lossless compresson algorthm for computer anmaton mage sequences The algorthm uses transformaton nformaton avalable n the anmaton scrpt and oatng pont depth and object number nformaton stored at each pxel to perform hghly accurate moton predcton wth very low computaton The geometrc data, e, the depth and object number, s very ecently compressed usng moton predcton and a new technque called drecton codng, typcally to 1 to 2 bts per pxel The geometrc data s also useful n z-buer mage compostng and ths new compresson algorthm oers a very low storage overhead method for savng the nformaton needed for z- buer mage compostng The overall compresson rato of the new algorthm, ncludng the geometrc data overhead, s compared to conventonal spatal lnear predcton compresson and s shown to be consstently better, by a factor of 14 or more, even wth large frame-to-frame moton CR Categores: I42[compresson(codng)]exact codng Addtonal keywords: compresson,computer anmaton,computer graphcs, moton predcton 1 Introducton Wth the ncreasng popularty and fallng cost of computer anmaton comes a new problem: storng the enormous data les whch even short computer anmaton sequences requre Fve mnutes of NTSC resoluton computer anmaton takes up approxmately 85 ggabytes of storage; lm resoluton takes many tmes more Ths amount of data cannot be economcally stored on lne n hgh speed secondary storage devces Anmaton mage les are typcally stored o-lne on removable meda An alternatve to usng o-lne storage s to compress the mage data and store t on-lne Ths s very desrable for sequence edtng and mage manpulaton, for example For hgh mage qualty only lossless compresson s acceptable; mages can then be exactly reconstructed from ther compressed representaton Errors do not accumulate f mages Ths work was supported by the Natonal Scence Foundaton under grant MIP y Computer Anmaton Laboratory, GVU center, College of Computng, Georga Insttute of Technology, Atlanta GA 30332, E-mal: branguenter@ccgatechedu z gtal Sgnal Processng Labratory, School of Electrcal Engneerng, Georga Insttute of Technology, Atlanta GA 30332, E-mal: yun@eedspgatechedu, rmm@eedspgatechedu are combned or manpulated and run through the compresson decompresson cycle several tmes Much more nformaton s avalable to a compresson algorthm for computer anmaton than s the case for lve acton vdeo However, surprsngly lttle work has been done on explotng the nformaton n a computer anmaton scrpt to mprove mage compresson ecency Prevous work such as that descrbed n [1] and [5] s actually mage based compresson although the applcaton s to computer anmaton The lossless compresson algorthm for computer anmaton to be descrbed n ths paper combnes elements of both moton predcton and spatal lnear predcton compresson technques, usng each when most approprate The new compresson algorthm uses transformaton nformaton n the anmaton scrpt to perform essentally perfect mage space moton predcton wth very low computaton Ths s a major advantage of the new algorthm because moton predcton wth subpxel accuracy based only the nformaton present n the mage sequence s computatonally expensve [6][3] Poor qualty moton predcton ncreases the moton predcton error whch reduces the maxmum achevable compresson rato One of the most eectve lossless mage compresson technques s PCM followed by entropy codng [4][8] For typcal lve acton vdeo sequences the best compresson achevable usng ths method s usually less than 2 to 1 [7] The best computer generated mages are nearly ndstngushable from real mages so we can expect that good synthetc mages wll not compress any better than lve acton vdeo For scenes wth farly rapd camera and object moton the compresson rato we have acheved wth our new moton predcton compresson s approxmately 15 tmes that of spatal lnear predcton compresson technques - about 3 to 1 compresson wth the new technque as opposed to 2 to 1 compresson wth PCM As camera and object moton decrease the compresson rato of the new technque steadly ncreases whle spatal predcton compresson remans constant at roughly 2 to 1 Extra geometrc nformaton, the object number and the depth at each pxel, s stored n each frame to perform moton predcton The geometrc nformaton s compressed very ecently n our new algorthm, typcally to 1 or 2 bts per pxel For z-buer compostng applcatons [2] ths s another advantage of the new algorthm, because the depth nformaton needed for z-buer compostng s stored n very lttle space We assume the anmaton scrpt contans a homogeneous matrx transformaton for every object n every frame The matrx transforms the object from the model space coordnate frame nto the screen space coordnate frame The transformaton matrces are stored n an auxlary le along wth the compressed mage data and consttute part of the overhead of the new compresson algorthm Ths lmts the current mplementaton to rgd body moton but ths s not an ntrnsc lmtaton of the algorthm Non-rgd body moton can be accommodated by storng approprate transformaton nformaton, such as free form deformaton

2 mesh ponts for example [9], n the anmaton scrpt The current mplementaton assumes that objects are represented as polygonal surfaces Algorthms exst for convertng many derent surface representatons to approxmatng polygonal surfaces Many commercal mage synthess programs perform ths converson nternally so the lmtaton to a polygonal representaton s not unduly restrctve The geometrc data has specal propertes we explot to mprove compresson As a consequence the coder s splt nto two parts: a geometrcal data coder and a color data coder General notaton used throughout the paper s presented n Secton 2 Secton 3 of the paper presents block dagrams of the algorthm Secton 4 descrbes the geometrcal data codng algorthm Secton 5 descrbes the color data codng algorthm Anmaton test results are presented n secton 6 and conclusons and suggestons for further research are presented n secton 7 T N Z C T G GEOMETRICAL ATA ENCOER ( N, Z ) T N Z T -1 N -1 Z -1 T -n N -n COLOR ATA ENCOER ( C ) GEOMETRICAL ATA ECOER ( N, Z ) Z -n T N Ż T -1 N -1 Ż -1 T -n N -n Ż -n T G I T N Ż 2 Notatons and ata structure I COLOR ATA ECOER ( C ) C In ths paper the frame number, whch s used to dentfy the specc frame, s expressed as a superscrpt A subscrpt represents the object number when t s expressed as a sngle value and the spatal locaton when t s expressed as a par of values If the subscrpts are omtted, that symbol represents the whole set of the correspondng data for that frame The data structure of a frame s dvded nto two parts The rst part s the set of 44 homogeneous matrces for all the objects f Tj ; j = 0::N o? 1 g by whch the pont n the model object space s transformed to the screen space The other part s the 2-dmensonal array of the data Pm;n; m = 0::N x? 1; n = 0::Ny? 1, where No represents the number of objects and N x and N y represent the number of pxels n each drecton Each pxel datum Pm;n s composed of the object number Nm;n, depth Zm;n and colors Cm;n of the pxel at the spatal locaton (m; n) For example, the pont n the -th frame (m; n; Zm;n) T s transformed to the pont (x j ; y j ; z j ) T n the j-th frame as follows: 0 T j k (T k)?1 m n Z m;n C B A x j y j z j 1 1 C A (1) where k = N m;n As mentoned above, the symbols wthout the subscrpts represent the whole set of data for the frame For example T stands for the set of matrces and N, Z, C represent the whole two dmensonal array, also called a eld, contanng the object number, depth and color values of the -th frame, respectvely The object number N and the depth Z are collectvely called the geometrcal data eld The color eld C represents the R,G,B color elds, but sometmes can be used for one specc color eld The object number and color values are represented as ntegers, but the depth s a real number In our mplementaton, each of the RGB color values s usually represented by 8 bts/pxel (256 levels), and the depth s double precson oatng pont Snce the compresson ecency of the geometrcal data s hghly dependent on the accuracy of the calculaton, the double precson representaton s preferred The requred number of bts for the object number depends on the total number of objects Fgure 1: n-th Order Frame Encoder and ecoder 3 System Block dagrams The heart of the codng scheme uses a lnear predctve codng algorthm (PCM) [4] Snce there exsts substantal correlaton between successve frames n computer anmaton as well as n real-lfe vdeo, good compresson gan can be acheved by these predctve schemes Snce both the object number and depth elds are needed to compute the moton trajectory of each pxel and these are encoded together nto one data stream G, the whole system s dvded nto a geometrcal data codng block for N ; Z and a color data codng block for C as shown n Fgs 1 The object number N and color data C are coded losslessly, but the depth Z s allowed to contan error wthn a speced lmt to acheve a hgh compresson gan, because Z requres a relatvely larger number of bts (64 bts/pxel for a double precson representaton) than N ; C The PCM system requres storage for several frames determned by the order of the predctor The geometrcal data codng block stores the object number elds N and the depth elds Z of prevous frames Snce only the decoded values are avalable for the depth eld n the decoder, the geometrcal data encoder uses the decoded depth eld Z nstead of the orgnal depth eld Z for correct reconstructon from the encoded data The stored geometrcal data N ; Z are provded to the color data codng block to predct the color data n other frames by moton predcton 4 Geometrcal ata Codng Fgs 2 show the block dagrams of the encoder and decoder for geometrcal data The prncple behnd the geometrcal data codng s that the geometrcal data of the current frame s predcted from several prevous frames whch are compared to the current orgnal frame pxel-by-pxel Each pxel Pm;n s classed as matched f Nm;n, Zm;n of the current frame are the same as N m;n, Z m;n of the predcted frame, and unmatched otherwse Snce the object number and depth for the matched pxels can be recovered from the

3 T N Z FRAME COMPARATOR N Z M IRECTION ENCOER N Ż M TRANSMIT ENCOER T G NON-AJACENT POLYGON PIXEL SQUARE PLANAR PIXEL SQUARE object 2 N Z n-th ORER FRAME PREICTOR object 1 T G RECEIVE ECOER N Z M T -n N -n Z -n FRAME SYNTHESIZER N Z T -1 N -1 Ż -1 T -n N -n n-th ORER FRAME PREICTOR Z -n T N Ż N ^ IRECTION N Z ^ N M Ż ECOER Z T -1 N -1 Z -1 Fgure 2: Geometrcal ata Encoder and ecoder predcted frame n the decoder, the only nformaton that needs to be transmtted are the matchng status eld M whch records whether or not each pxel s matched, and the complete geometrcal data for the unmatched pxels Unmatched pxels occur manly n recently uncovered regons whch cannot be predcted from prevous frames, or from hghly curved regons that are dcult to predct For the unmatched regons the geometrcal data can be coded eectvely by explotng the spatal correlatons between pxels, because pxels whch belong to the same planar polygon satsfy the same plane equaton An algorthm called drecton codng s proposed and descrbed later Wth drecton codng, the unmatched pxels are classed nto drecton matched pxels and totally unmatched pxels Ths matchng nformaton replaces Mm;n at the unmatched pxel and ths moded matchng status eld s M After the drecton codng, snce the majorty of the frame s matched, the entropy of M s very small Thus M can be compressed eectvely by entropy codng or run-length codng and the orgnal geometrcal data Nm;n, Zm;n are transmtted n uncompressed form only for the totally unmatched pxels At the receve decoder, the matchng status eld M m;n, and the geometrcal data Nm;n,Z m;n for totally unmatched pxels are obtaned For the matched pxels whch can be dented by the M, the object number Nm;n and the depth Z m;n are coped from the predcted frame Then for the unmatched pxels the Nm;n, Z m;n are recovered by the drecton decoder and the complete recovered frame data s fed nto the frame predctor for the next frame predcton 41 Frame Predctor The set of four pxels P m;n, P m;n+1, P m+1;n+1, P m+1;n s dened as a pxel square S m;n Each pxel square can be classed nto one of three categores The rst s the plane pxel square where all four corner pxels come from the same planar polygon and make a planar square The second s the adjacent polygon pxel square where four pxels are from the T N Z T - th FRAME AJACENT POLYGON PIXEL SQUARE j - th FRAME Fgure 3: Transform of Pxel Squares between Frames two adjacent polygons that share an edge that ntersects the pxel square The thrd s a non-adjacent polygon pxel square where the polygon boundary s across the pxel square but the polygons do not share an edge These three cases are llustrated n Fg 3 The planar pxel square can be easly transformed by Eq 1 and rendered nto other frames For the adjacent polygon pxel square, f the plane equatons of the two polygons can be obtaned by explotng the neghborng pxel squares, then the pxel square can be parttoned nto two polygons and each polygon can be transformed and rendered onto other frames n the same way There s not enough nformaton to make correct parttons for the non-adjacent polygon pxel squares, then these cannot be used to predct other frames The pxel square S m;n can be dened as a planar pxel square f the four pxels satsfy the followng plane condtons N m;n = N m;n+1 = N m+1;n+1 = N m+1;n (2) jz m;n + Z m+1;n+1? Z m;n+1? Z m+1;n j < (3) In Eq 3 the nequalty s used to deal wth the error due to the lmted precson of computaton and the small number s determned as the allowable error for the depth of the pxels whch are from the same planar polygon In some cases, the above two condtons are not enough to determne whether the pxel square s planar or not For example, the pxels lyng across the boundary of two separate polygons whch are parallel to each other mght satsfy these two condtons There are several ways of reducng the possblty of an ncorrect classcaton of a planar pxel square One way s to add the followng condtons, whch test the relatons between the depths of the surroundng pxels If the followng plane condtons are satsed wth regard to at least one corner of the pxel square, S m;n can be consdered to be a plane pxel square j2z l;k? Z l?1;k? Z l+1;k j < j2z l;k? Z l;k?1? Z l;k+1 j < for l = m; m + 1 and k = n; n + 1 For the non-planar pxel square whch does not satsfy the above plane condtons, f some two pxel squares around t are planar and f the ntersecton of those two planes are found to be across the pxel square by solvng the plane equatons of those two planes, then ths pxel square s an adjacent polygon pxel square that can be dvded nto two polygons and transformed nto other frames One way to nd the two plane pxel squares s to test the above plane (4)

4 condtons for each par of pxel squares whch are on the opposte sdes of the current pxel square The geometrcal data for most of a frame can be computed by transformng all the planar and adjacent polygon pxel squares of the prevous frame nto the current frame and renderng the transformed polygons on the frame buer of geometrcal data usng the z-buer algorthm Snce the current frame does not change much from the prevous frame, the transformed polygon of one pxel square s small and covers only a few pxels Under ths assumpton, there are several eectve technques for geometrcal data renderng One smple method s to nd the boundng box of the transformed pxel square and test whether each pxel pont nsde the boundng box s nsde the polygon or not For each nsde pxel pont, the depth of that pont can be computed from the plane equaton of the transformed polygon for the z-buer renderng process The back-face removal step mght be appled before renderng There s one specal case where the vewpont and object are not movng In ths case the transform matrces of the current and prevous frames are the same and the whole transform matrx Tk j (T k)?1 wll be the dentty matrx n Eq 1 For these pxels, the above complcated steps are not necessary and the only thng to do s smply to apply the depth of the prevous frame to the z-buer algorthm at the same pxel locaton All these pxels are classed as matched pxels ue to occluson some parts of the current frame may not be predctable from the prevous frame The percentage of predctable pxels can be ncreased by usng hgher order predcton In the n-th order case, the prevous n frames are transformed and rendered on the same frame buers of object number and depth The frame predctor used n both the encoder and decoder have dentcal frame buers for a object number and depth for hgher order predcton The encoder and decoder should store the same frame data n both predctors so that the predcted frames n both blocks wll be the same 42 Frame Comparator The frame comparator compares the nput frame data to the predcted frame data on a pxel-by-pxel bass and records the result n the matchng status eld M If a pxel P m;n of the current frame and P m;n of the predcted frame satsfy the followng condtons, then the pxel s consdered to be predctable from the prevous frames and sad to be a matched pxel N m;n = N m;n (5) jz m;n? Z m;nj < (6) In Eq 6, an nequalty s used for the same reason as n Eq 3 If a pxel Pm;n s found to be matched, Mm;n s set to 1 and otherwse set to 0 Because of the smlarty between the adjacent frames, most of M wll be 1 For matched pxels, the depth Zm;n of the nput frame s replaced by the predcted depth Z m;n to guarantee the consstency of data between the encoder and decoder, because only Z m;n wll be avalable n the decoder By Eq 6 the accuracy of the new depth Z m;n s guaranteed to be wthn Ths moded depth eld s Z and wll be gven to the drecton coder along wth N and M P ( M, N, Z ) P ( M, N, Z ) P ( M, N, Z ) EPTH MATCHING TEST MATCHING IRECTIONS Fgure 4: etermnaton of recton Matchng 43 recton Encoder Unmatched regons are usually from recently uncovered regons or hghly curved regons that cannot be predcted Snce the pxels n those regons mght come from some plane polygons or mght be on the extenson of a plane from a surroundng matched regons, a spatal predcton technque explotng the plane relatonshp between neghborng pxels can be used to code these pxels One such method s to nd the matchng drecton n whch two neghbor pxels lyng on a straght lne match the object number and depth of the current pxel and record the drecton value by overwrtng the M m;n whch was orgnally zero Snce only the reconstructed data are avalable n the decoder, those two pxels should be pxels that have been already coded Therefore t should be checked whether the matchng status values of those two pxels s stll zero The matchng condtons for a drecton at P 3 n Fg 4 are descrbed as follows : M 1 6= 0 and M 2 6= 0 (7) N 1 = N 2 = N 3 (8) jz 3? Z3j < (9) where Z3 s the spatally predcted depth of the pxel P 3 n the speced drecton as n Eq 10 Z 3 = 2Z 2? Z 1 (10) These condtons are tested for eght drectons from drecton 1 to 8 as n Fg 4 The rst matched drecton becomes the matchng drecton of the pxel and the correspondng drecton value, whch s dened as = + 1 n Fg 4, s assgned to Mm;n whch was orgnally zero The depth of the current pxel s replaced by the predcted value n the matched drecton as n Eq 10 for consstency of the data n the encoder and decoder If there s no matchng drecton, the number 10 s assgned, correspondng to a totally unmatched pxel After drecton codng, the frame data wll be: M m;n = Z m;n = ( 1 matched 2::9 drecton matched 10 totally unmatched 8 < : Z m;n matched Z m;n drecton matched totally unmatched Z m;n (11) (12) N remans unchanged because the object number s losslessly coded Fg 5 shows an example of drecton codng where the regon nsde the polygon s orgnally unmatched The encodng s performed from left to rght and from bottom to top

5 POLYGON BOUNARY K -n K -1 K n - th ORER COLOR FRAME PREICTOR C L FRAME SUBTRACTOR and PCM ENCOER TRANSMIT ENCOER I MATCHE PIXEL IRECTION MATCHE PIXEL TOTALLY UNMATCHE PIXEL C K -n K -1 K n - th ORER COLOR FRAME PREICTOR C L FRAME AER and PCM ECOER C Fgure 5: Example of recton Encodng The unmatched regon nsde the dashed polygon s coded by drecton codng Codng s performed from left to rght and from bottom to top and the arrows represent the drecton of matchng Ths llustrates that the number of totally unmatched pxels s very small and the matchng drectons are mostly 2 because drecton 2 s the rst test drecton n Fg 4 Snce the geometrcal data for the matched and drecton matched pxels are predctable, the data to be transmtted are the matchng status eld M whch has very low entropy and the fnm;n; Zm;ng's for a few totally unmatched pxels 44 Frame Syntheszer Snce the matched pxels can be dented from the matchng status eld M decoded n the receve decoder, the frame syntheszer can recover the geometrcal data for all the matched pxels by copyng the data from the predcted frame Then the geometrcal data ^N m;n, ^Z m;n for the matched pxels and totally unmatched pxels are correctly recovered whereas those for drecton matched pxels reman undetermned: ^N m;n = ^Z m;n = 45 recton ecoder ( N m;n = Nm;n f M m;n = 10 N m;n = Nm;n undetermned f M m;n = 1 f M m;n = 2::9 ( Z m;n = Zm;n f M m;n = 10 Z m;n undetermned f M m;n = 1 f M m;n = 2::9 (13) (14) For the drecton matched pxels, the object number N m;n s recovered by copyng the object number of the pxel whch s located n the matchng drecton and the depth Z m;n becomes the predcted value Z m;n from the two pxels located n the matchng drecton as n Eq 10 N m;n are recovered correctly for all the pxels and the depth eld Z s the same as Z n Eq 12 The accuraces of Z m;n for the matched pxel and Z m;n for the drecton matched pxel are guaranteed by Eq 6 and Eq 9, respectvely These decoded data are fed nto the predctor and are the same for both the encoder and decoder to make the same predctons as n Fg 2 I RECEIVE ECOER Fgure 6: Color ata Encoder and ecoder, where K = (T N Z ) 5 Color ata Codng Snce the (T; N; Z)'s from the geometrcal data codng block can be used to compute the locatons of a current frame pxel on the prevous frames and the prevous color data frames are stored, the color of the pxel can be predcted by estmatng the color at the transformed locatons n the prevous frames The error mage between the orgnal and the predcted frame, also called the resdual mage R, has a relatvely low entropy compared to the orgnal The pxels are classed nto two classes, matched and unmatched pxels, based on whether the locatons n the old frames are traceable or not Snce ths resdual mage stll has some spatal correlaton, spatal lnear predcton codng (PCM) can be appled to reduce the entropy Ths PCM coded resdual mage s called a derental resdual and entropy codng usng Human codng or arthmetc codng, can compress the derental resdual losslessly In decoder after the resdual R s recovered by PCM decodng, the orgnal color data Cm;n s obtaned as the sum of the predcted value C m;n and the resdual R m;n for the matched pxel The color Cm;n of the unmatched pxel s the same as resdual R m;n 51 Color Frame Predctor A pxel of one frame can be mapped to another frame by the transformaton n Eq 1 and f the transformed pont satses the followng condton, t s consdered to be the same pont as the current pxel and the pxel s sad to be matched If the pxel P m;n s transformed nto the j-th frame and the transformed pont s nsde the pxel square S j p;q, the matchng condton s as follows: Z j mn Zj trans Zj max (15) where the Z j trans s the depth of the transformed pont and Z j mn, Zj max are the mnmum and maxmum depth of the four corner pxels of S j p;q, respectvely Snce several prevous frames of geometrcal and color data are avalable to deal wth the occluson problem, the transformed pont n the nearest frame whch satses the above condton s used to predct the color data of the current pxel

6 L records the matchng status values for color data whch are zeros for the unmatched and ones for the matched pxels Snce generally the transformed pont s not the pxel pont, an nterpolaton s necessary for the computaton of the color 52 Subtractor and PCM Encoder Colors of matched regons n the resdual R are resdual values generated by sutractng the predctced colors from the orgnal colors whereas the colors n the unmatched regons reman unchanged as follows: R C m;n = m;n f L m;n = 0 Cm;n? C m;n f L (16) m;n = 1 Unmatched regons are mostly from recently uncovered regons and the entropy of such unmatched regons can be reduced by spatal lnear predctve codng (PCM) The resdual mage n the matched regon has low entropy caused by changes of llumnaton, by the movement of objects, vewponts or lght sources between frames Snce ths knd of error has relatvely slow spatal varaton, PCM can be eectvely appled also to the matched regons of the resdual mage These two steps, frame subtracton and PCM, can be mplemented wth a combned operaton as n Fg 6 Snce the matched and unmatched regons have derent knd of data as explaned above, each regon should be coded ndependently Usually 2- PCM uses three left and lower neghborng pxels Cp;q to predct the current pxel C m;n If the current pxel s an unmatched pxel, C p;q should be the orgnal color C p;q If the current pxel s a matched pxel, C p;q should be the resdual value Snce the resdual value s not avalable for an unmatched pxel, the C p;q s set to zero when L p;q = 0 as follows: C p;q = C p;q f L m;n = 0 L p;q(c p;q? C p;q ) f L m;n = 1 (17) The spatally predcted value C m;n s the nteger part of a lnear combnaton of those as follows: C m;n = nt( C m?1;n + C m?1;n?1 + C m;n?1 ) (18) In ths paper, the predcton coecents,, are selected to be 075,-05,075 respectvely Then the derental resdual mage s obtaned by subtractng the spatally predcted value C m;n from the resdual value as follows: m;n C = m;n? C m;n f L m;n = 0 Cm;n? C m;n? C m;n f L (19) m;n = 1 The derental resdual, whch has very small entropy, can be compressed losslessly by an entropy codng technque 53 Adder and PCM decoder The L m;n and C m;n n the predctor of the decoder are the same as those n the encoder The orgnal color eld C s recovered by the combned step of PCM decodng and frame addton as follows: Cm;n = m;n + C m;n f L m;n = 0 m;n + C m;n + C m;n f L (20) m;n = 1 where the spatal predcton C s the same as n the encoder The recovered color C s fed back to the predctor for the predcton of the next frames 6 Test Results 31 anmaton frames were generated to test the proposed compresson algorthm Each frame s composed of 7 objects and varous knds of textures were mapped by sold texture mappng Through the whole 31 frames the ball bounces back and forth between the two wood blocks In the rst 11 frames the vewpont does not change and n the next 10 frames the vew pont moves approxmately 5 degrees/frame urng the last 10 frames zoomng s performed Plate 2,3, and 4 show the frames for above three cases In the case of Plate 2, snce most of the objects are not movng and the vew pont s xed, all regons except the rollng ball are matched regons Pxel ponts n the current frame are transformed exactly to the same pxel ponts of other frames, then there are no errors due to blnear color nterpolaton and snce even the colors on the edges of statonary objects are predctable, the resdual wll have extremely small entropy whch results n very hgh compresson gan Plate 3 s a more general case n whch both an object and the vewpont are movngthere are manly three knds of resdual errors Frst, the recently uncovered regons are the major error regons whch can be compressed only by PCM Second, n matched regons the changes n llumnatons on the object surfaces causes resdual errors Usually llumnaton changes are due to the changes of specular re- ecton whch vares wth the movement of the vew pont or the object tself Generally these errors change slowly and can be lowered by PCM Thrd, snce the object boundares are often unmatched regons whch do not satsfy Eq 15, the resdual errors are large there And even n the case where the pxels on the object boundary are matched, relatvely large errors due to the color nterpolaton occur because the color data of these pxels were generated by antalasng In Plate 4, snce wth zoomng there are no recently uncovered regons and the spatal frequency s decreasng, the entropy of the resdual sgnal wll be smaller than that n Plate 3 The second order compresson algorthm was mplemented and tested on ths test sequence Fg 7 shows the entropes of the orgnal pctures and the derental resdual mages by 2- PCM and moton predcton algorthm Table 1 llustrates the entropes of several frames Plates 5 and 6 show the derental resdual mages of the RE component by lnear predctve codng (PCM) and the new moton predcton, respectvely As explaned above, the major errors n Plate 6 are on recently uncovered regons and along the object boundares whch are often unmatched regons The bggest derences between PCM and moton predcton occurred on the wood texture whch has relatvely hgher spatal frequency than any other regons In these hgh spatal frequency regons, moton predcton shows much hgher performance than spatal lnear predctve codng Through the whole 31 frames, the entropy of the orgnal frame s around 207 bts/pxel In the rst 11 frames where the vewpont s xed, only about 1 bts/pxel s requred for the moton predcton technque, except for the rst frame whch cannot be moton predcted and s coded only by PCM Ths contrasts wth PCM whch needs around 114 bts/pxel The necessary bts/pxel for the next 10 frames n whch both the vewpont and object are movng s around 83 bts/pxel whch s about 3 bts/pxel gan over PCM

7 frame 5 frame 15 frame 25 N ORIGINAL N P CM N GEOM N RGB N MOT ION Table 1: Entropes of orgnal mage and resduals by PCM and Moton Predcton (unt : bts/pxel) BITS/PIXEL BITS/PIXEL ORIGINAL ENTROPY SPATIAL LINEAR PREICTION (PCM) MOTION PREICTION FRAME NUMBER Fgure 7: Entropes of the Bouncng Ball Sequence The data for moton predcton are the sum of compressed geometrcal data and entropy of resdual In the next 10 frames, all entropes are decreasng wth zoomng as expected above, but the moton predcton algorthm stll outperforms PCM by about 35 bts/pxel 7 Concluson The moton predcton compresson algorthm for computer anmaton mage sequences presented here consstently outperforms spatal lnear predcton Ths s true even though the new algorthm encodes double precson depth and nteger object number nformaton for a total of 96 bts/pxel ncludng RGB data whle the spatal compresson algorthm does not The depth and object number nformaton are useful n ther own rght for performng z-buer mage compostng The compresson rato acheved by the new algorthm was 14 tmes or more greater than that acheved wth spatal lnear predcton even for scenes wth rapd changes n camera vew pont and substantal changes n object occluson relatonshps The overall compresson rato acheved wth the new algorthm for mage sequences wth sgncant object and vewpont moton was approxmately 3 to 1 There s stll scope for mprovement n the algorthm We have notced sgncant remanng correlaton n the resdual mages Long strngs of zero resdual values are punctuated by short bursts of plus and mnus one resdual values Addtonally the geometrc overhead can be sgncantly reduced by storng the polygonal surface data and re-renderng t wth the z- buer algorthm, a process whch should be no more tme consumng than re-renderng pxel squares as n the current mplementaton Ths makes the auxlary data le more complex but for anmaton sequences more than a few seconds long the geometrc data overhead should drop to 1 bt/pxel or less We are currently mplementng ths extenson The current mplementaton of the algorthm s lmted to objects represented as polygonal surfaces As dscussed n the ntroducton ths s not an undue restrcton snce ecent algorthms exst for approxmatng many derent surface types wth polygonal surfaces However t would be an nterestng research project to extend the geometrc coder to other surface types so that an exact, rather than an approxmatng polygonal, surface representaton could be used Ths extenson would requre modfyng both the moton predcton stage and the drecton codng stage to compute surface equatons drectly from the depth nformaton stored n the mage 8 Acknowledgments The authors acknowledge help from Ragnar Jonsson, Stephen A Martucc, Wayne Wooten and Lonne Harvel References [1] enber, Mchael J and Turner, Paul M A erental Compler for Computer Anmaton Proceedngs of SIGGRAPH '86 (allas, Taxas, August 18{22,1986) In Computer Graphcs 20,4 (August 1986), 21{27 [2] u, Tom Compostng 3- Rendered Images Proceedngs of SIGGRAPH '85 (San Francsco, Calforna, July 22{26,1985) In Computer Graphcs 19, 3(July 1985), 41{44 [3] Jan, Anl K Fundamentals of gtal Image Processng Prentce-Hall, Englewood Cls, New Jersey, 1989 [4] Jayant, Nuggehally S and Noll, Peter gtal Codng of Waveforms Prentce-Hall, Englewood Cls, New Jersey, 1984 [5] Jones, Stephen C and Moorhead II, Robert J Hardware-specc Image Compresson Technques for the Anmaton of CF data Proceedngs of SPIE - Internatonal Socety for Optcal Engneerng, 1668 (San Jose, Calforna, February 10{11,1992), 141{146 [6] Lm, Jae S Two-mensonal Sgnal and Image Processng, Prentce-Hall, Englewood Cls, New Jersey, 1990 [7] Martucc, Stephen A Reversble Compresson of HTV Images Usng Medan Adaptve Predcton and Arthmetc Codng Proceedngs of IEEE ISCAS, 2(New Orleans, Lousana, May 1{3,1990),1310{1313 [8] Melnychuck, Paul W and Rabban, Majd Survey of Lossless Image Codng Technques Proceedngs of SPIE - Internatonal Socety for Optcal Engneerng, 1075 (Los Angeles, Calforna, January 17{20, 1989), 92{100 [9] Sederberg, Thomas W and Parry, Scott R Free-Form eformaton of Sold Geometrc Models Proceedngs of SIGGRAPH '86 (allas, Taxas,August 18{22,1986) In Computer Graphcs 20,4 (August 1986), 151{160 [10] Wtten, Ian H, Neal, Radford M, and Cleary, John G Arthmetc Codng for ata Compresson In Communcatons of the ACM 30,6 (June 1987), 520{540

8 Plate 2: Frame 0,9 Plate 3: Frame 10,19 Plate 1: Frame 5 of the test sequence Plate 4: Frame 20,29 Plate 5: The derental mage of frame 5 by PCM ( ) Plate 6: The derental resdual mage of frame 5 by moton compensaton ( )