High Quality Normal Map Compression

Transcription

1 Graphic Hardware (2006) M. Olano, P. Sluallek (Editor) High Quality Noral Map Copreion Jacob Munkberg Toa Akenine-Möller Jacob Strö 2 Lund Univerity 2 Ericon Reearch Abtract Noral apping i a widely ued technique in real-tie graphic, but o far little reearch ha focued on copreing noral ap. Therefore, we preent everal iple technique that iprove the quality of ATI Dc noral ap copreion algorith. We ue varying point ditribution, rotation, and differential encoding. On average, thi iprove the peak-ignal-to-noie-ratio by db, which i clearly viible in rendered iage. Our algorith alo allow u to better handle lowly varying noral, which often occur in real-world noral ap. We alo decribe the decoding proce in detail. Categorie and Subject Decriptor (according to ACM CCS): I..7 [Coputer Graphic]: Texture. Introduction Bup apping [Bli7] i a widepread technique which add the illuion of detail to geoetrical object in an inexpenive way. More pecifically, a texture, called a bup ap or noral ap, i ued at each pixel to perturb the urface noral. A coon approach to generate noral ap i to tart with a high polygon count odel and create a low coplexity odel uing oe geoetrical iplification algorith (ee, for exaple, Cohen et al work [COM9]). The difference between thee two odel i then baked into a noral ap. For real-tie rendering, the noral ap i applied to the low coplexity odel, giving it a ore detailed appearance. Thee technique are heavily ued in recent gae. A poible diadvantage i that the aving in tranfor and rendering due to the lower vertex count i tranlated into an increae in bandwidth uage of texture (noral ap). A traditional technique to alleviate thi proble i loy texture copreion (TC), which wa introduced in 996 [BAC96,KSKS96,TK96]. TC developed priarily for color can alo be applied to noral ap [Gre04], but the quality can be higher if pecialized algorith are developed. One uch technique, called Dc, ha been propoed by ATI [ATI05]. However, little effort ha been pent on developing new algorith for noral ap copreion. One proble with Dc i that it cannot handle lowly varying noral ap well. Thi i illutrated in Figure 0. In thi paper, we develop everal variation and extenion of Dc that perfor uch better on average, and handle lowly varying data particularly well. We preent viual proof howing that our noral apping algorith give higher quality rendering, and we alo how that the peak-ignal-to-noie ratio (PSNR) i iproved. 2. Previou Work The firt exaple of noral copreion in graphic that we know of i decribed in the context of geoetry copreion [Dee95], i.e., it wa not targeted toward noral ap copreion. Deering preent a ethod for copreing urface noral, arguing that about 00,000 noral ditributed over the unit phere would give ufficient quality. Thee noral can be repreented by a ingle 7-bit index, and by exploring yetrie on the phere, only a /4 of the phere need to be repreented. A regular grid in the angular pace of one uch patch i ued a aple ditribution. Nearby noral are encoded differentially. With thee technique he anage to copre a noral to about 2 bit. However, the decopreion tep include a nuber of trigonoetric operation and i quite cotly copared to the chee decribed below. 2.. Dc Noral Copreion Next, we will review ATI noral ap copreion chee called Dc [ATI05]. A far a we know, thi i the only forat dedicated to thi purpoe alone.

2 Y Z n=(x,y,z) X Y (x,y) X Figure : Dc elect a rectangle in the xy-plane (left), and place point uniforly over thi rectangle (in thi figure, only 4 4 point were placed to ake the illutration clearer). Thee point can be een a a palette of xy pair, and each texel in a 4 4 tile can elect one of thee pair. To the right, one uch (x,y)-point ha been ued to generate a noral, n = (x,y,z). Thi i done by requiring that we ue unit noral. In the ajority of cae today, bup apping i perfored in local tangent pace, (X,Y, Z), of each rendering priitive (e.g. a triangle). Since the length of the noral i not of interet, Dc ue unit noral, and hence it uffice to copre the x- and y-coponent. The third coponent i obtained through noralization: z = x 2 y 2, () and thi coputation can either be done in the pixel hader, or by pecial purpoe hardware. The x- and y-coponent are copreed independently uing a variant of STC/DXTC [INH99]. A block of 4 4 texel (a.k.a. a tile) i copreed into 2 bit, i.e., at eight bit per pixel (bpp). The x-coordinate are encoded in the following way. Two eight-bit value, x tart and x top, repreenting an interval encloing the x-value in the tile, are found. Each texel can elect fro eight different x-value: x k = x tart + k(x top x tart)/7, k = 0...7, which are thu pread uniforly over the interval. Thi require three bit per texel. To encode the x-value of a tile, we need 2 bit for x tart and x top, and 6 bit for the per-pixel indice. Thi u up to 64 bit. The y-coponent are encoded in the ae way, and the total cot per tile i 2 bit. An illutration of Dc i hown in Figure.. Iproved Noral Copreion In the following three ubection, we preent three iple general technique for iproving the quality of the Dc noral copreion chee. Thee are cobined into a ingle copreion forat in Section 4, while keeping a bit budget of bit per pixel (bpp). Copared to Dc, the extra cot i a ore expenive decopreion phae (Section 4.). Firt, however, we will explain how we can incorporate three new ode into Dc. It te fro the fact that wapping the value x tart and x top will produce exactly the ae Figure 2: By rotating the coordinate frae, we can often find a uch tighter bounding box. Thi will iprove the encoding preciion. recontruction level x 0...x 7, albeit in the revered order. Since thee two repreentation are equivalent, it i poible to ignal one extra bit, b: If x tart < x top, then b 0, ele b. The ae trick i ued in DXT to ignal whether a block i RGB or RGBA, and we call thi trick the ordering technique. In Dc, the ordering technique can be ued on both x and y, and hence two extra bit can be ued... Rotation Copreion When the ajor axi of a inial box around the (x,y) point of a tile do not coincide with either the x- or the y- axi, the quality of Dc decreae. By rotating the coordinate frae, a uch tighter fit can be obtained, and the extra torage cot i only an angle per block. Figure 2 illutrate thi cenario. For exaple, uing a ingle extra bit, one can elect to ue an angle in the et {0,π/4}, and two bit increae the et to {0,π/,π/4,π/}. Note that the tandard Dc cae i included, thu, thi technique can only achieve reult equivalent to or better than Dc. A een in Figure, the peak-ignal-to-noie-ratio (PSNR) iprove with ore than a decibel on average, already with a et of three angle. Viual reult are hown in Section Variable Point Ditribution Norally, the Dc technique place the aple point uniforly in a grid over the axi-aligned box defined by (x in,y in ) and (x ax,y ax), where x in = in(x tart,x top), x ax = ax(x tart,x top), and ditto for y in and y ax. However, other ditribution ay allow for better copreion. A iple way of altering the aple ditribution i to ue different ditribution depending on the apect ratio of the box. For exaple, if the box i ore than twice a wide a it i high, then it could be beneficial to ue a 6 4- ditribution rather than the tandard -ditribution. See Figure 4. No extra bit are needed to ignal thi, ince the point ditribution i autoatically triggered by the apect ratio, a = yax yin x ax x in, of the box. For Dc, the per-texel indice are encoded in ix bit (+ bit for an (x,y) pair). However, if the apect ratio trigger, ay, the ditribution 2 2, we

3 40.0 b a d c x PSNR (db) Figure 5: The x-axi i hown with quantized value arked with bold vertical line. Left: a i the deired interval, but the allet interval repreentable in Dc i b. Right: With value on both ide of a quantized value, the allet interval in Dc that cover the deired interval c i d, twice the ize of the allet repreentable interval b Figure : The average PSNR for a et of 20 noral ap a a function of the nuber of angle in the copreor. Angle count repreent no rotation, 2 repreent the two angle {0, π 2 } and generally, for an angle count a, the et of angle i {0, 2a π π(a ),..., 2a }. 2 x 2 4 x 6 x 6 x 4 2 x 2 Figure 4: Different point ditribution are triggered autoatically dependent on the apect ratio, a = yax yin x ax x in, of the bounding box. apect ratio (a = yax yin x ax x in ) ditribution (d x d y) a < / 2 2 / a < /2 6 4 /2 a 2 2 < a 4 6 a > 2 2 Table : The bounding box apect ratio autoatically elect a point ditribution. iply ove two bit, It hould be noted that thi approach cannot guarantee higher quality in all cae. We have teted thi technique on a et of 20 noral ap, with iproved PSNR value on all ap. The bound for electing a ditribution were choen epirically and are preented in Table. The ditribution 64 and 64 did not iprove the quality, and are not ued in our copreor... Differential Encoding One of the cae where it i eay to detect copreion artifact i in area that have a light curvature, for exaple, on a car hood. The oothne of the urface ake it eay for the viewer to predict what the iage hould look like, which i not a iple for a rough urface. Copreing uch low varying noral with Dc poe two proble. Firt, the allet repreentative interval i too wide. Since the quantized reolution i only eight bit, an interval of /255 of the range ight be to coare for repreenting nearly contant noral (ee Figure 5a and b). Second, the allet interval cannot be placed accurately enough, a the interval liit ut coincide with the quantization tep. Thu, if value of a block are preent on both ide of a quantized tep (Figure 5c), the allet interval covering all value will be at leat twice the iniu interval (Figure 5d). In thi ection, we will preent a technique to ake the preciion higher in order to olve thee to proble. Our idea i to ue the 2 bit that are norally ued for toring x tart, x top, y tart and y top in a different way, with an encoding that i pecialiced for repreenting all interval accurately. However, we ut be able to flag thi ode of encoding, o oe bit are irretrievably lot. Uing a iple apping technique decribed in the next paragraph, we can exploit 0 bit for a differential ode that handle lowly varying noral. In thi ode, we ue eleven bit each to encode x in and y in uing. (eight bit for the integer part and three bit for the fractional part), and we pend four bit each on two delta value, x and y, uing 2.2 bit. x ax i calculated a x ax = x in + x, and ditto for y ax. Due to the differential coding, we call thi ode the differential ode, and it addree both proble identified above: the allet repreentable interval i now four tie aller, and ince the preciion of the location of the interval ( fractional bit) i twice that of the allet length (2 fractional bit), we can handle value on both ide of a border a in Figure 5c without doubling the interval. In the following, we will preent a general ethod ueful when exploiting the ordering technique (ee beginning of Section ). Aue that we have detected a pecial ode ignaled by x tart x top. Unfortunately, we cannot et the bit of x tart and x top arbitrarily, ince x top ut be le than or equal to x tart. We thu want to olve the proble of exploiting a axiu nuber of the ixteen bit occupied by x tart and x top, while preerving x tart x top. Thi

4 x tart x top x x x 7 x Table 2: By irroring the poition for nuber, 6, 7, 24, 25 and 26, it i poible to fit the nuber 0 through without uing poition where x tart < x top (arked with black). can be olved by a iple apping, illutrated in Table 2, where x tart and x top are -bit value intead of -bit value for iplicity. Here, we have entered the nuber 0 through into the table, while avoiding the black boxe where x tart < x top. The nuber are entered row-by-row, except for the nuber which would have fallen in the forbidden poition, naely nuber, 6, 7, 24, 25 and 26. The poition for thee nuber are therefore irrored both in the vertical and horizontal direction relative to the center of the table. A can be een, we have tored 2 nuber, and we can therefore extract five bit. Thi i the axiu nuber of bit we can obtain ince roughly half the value are arked with black. Decoding thi 5-bit nuber i epecially iple for the upper half (row 0 through ) uing value = (x top << ) OR x tart, where << repreent a left hift and OR i the bit-wie logical OR operator. For the lower half (row 4 through 7), we have to irror x tart and x top firt to (7 x tart) and (7 x top), which i the ae a inverting their bit, and we can ue value = (NOT(x top) << ) OR NOT(x tart), where NOT( ) denote bit-wie inverion. For eight bit x- value, we hift with intead of, and we can tore 5 bit in value. Encoding i traightforward we ue the lower part of value for x tart and the upper part for x top, and invert both if x top > x tart according to the peudocode below: xtart = value AND 0xff xtop = (value» ) AND 0x7f if xtop > xtart xtart = NOT(xtart) xtop = NOT(xtop) end where NOT operate on all eight bit. 4. Propoed Schee In thi ection, we will cobine the three technique decribed above into a forat that fit in an bpp budget. The foundation for our cobined ode i Dc, but we exploit redundancy in it encoding to allow for ore ode. Next, we will decribe how thee two extra bit can be ued to iprove the quality of Dc ubtantially. We allow two rotation and liit the differential ode to tile where both the x- and the y-coponent can be encoded differentially. Altogether, we have four different ode: I) the tandard Dc ode, II) a rotation with 0 degree, III) a rotation with 60 degree, and IV) a differential ode, encoded with bit. A een in Figure, uing three angle give a ignificant iproveent in quality. It would be poible to add yet another angle, but that ode i ore wiely pent on the differential ode in ter of PSNR. The variable point ditribution i applied to all ode except the differential one where it did not increae quality. Table 4 how the quality contribution that each technique add on a tet erie. The uage of each ode i further illutrated in Figure 6, howing how often the different ode are ued for each tet iage. All ode are ued quite frequently, which indicate a balanced algorith. Note that ode I differ lightly fro Dc in that it ue variable point ditribution. Alternatively, it i poible to avoid uing variable point ditribution in ode I. Thi would ean that exiting Dc hardware deign could be reued to decode thi ode. Maybe ore iportant, it would allow exiting Dc texture to be trancoded to our new forat without lo, by wapping x tart and x top if x tart > x top (and perforing bit-wie NOT on the per-pixel indice to reflect the inverted ordering). However, thi backward copatibility would coe at a cot: On the tet iage of Section 5, the average PSNR for thi alternative olution i about. db lower than the propoed chee. 4.. Decoding The decoding of a block i perfored a follow: ode X Y bit vpd I: rot 0 x tart < x top y tart < y top + ye II: rot 0 x tart x top y tart < y top + ye III: rot 60 x tart < x top y tart y top + ye IV: diff x tart x top y tart y top.+2.2 no Table : The encoding ode for the cobined noral copreor. vpd indicate variable point ditribution. ode PSNR (db) Dc 6.4 Dc + Point Ditr. 7.5 Dc + Point Ditr. + Rot. Dc + Point Ditr. + Rot + Diff 9.4 Table 4: The average PSNR for the noral ap preented in Figure.

5 00% 0% 60% 40% ode, claping the value to the interval [,] can alo be neceary.. Finally, the z coordinate i calculated a z = x 2 y 2. The decopreed noral for the pixel i (x,y,z ). The lat two tep can be perfored in the pixel hader. 20% 0% diff rot 60 rot 0 rot 0 bupy car dot dot2 dot dot4 lupy etal NoralMap Figure 6: The frequencie of the different algorith for the iage ued in the tet. onetile. Firt, x tart, x top, y tart and y top are teted to ee which ode the block belong to, according to Table. For intance, if x tart < x top and y tart y top, then ode III i elected. 2. The next tep i to calculate x in and x ax. For ode I through III, thi i iply done uing x in = in(x tart,x top) and x ax = ax(x tart,x top), and likewie for y in and y ax. All reulting nuber will be between 0 and 255. For ode IV, the 5-bit value i firt calculated fro x tart and x top a decribed in Section.. Then, the firt eleven bit of value are ued to decode x in in forat., i.e., with eight bit for the integer part and three for the fractional part, reulting in a nuber between 0 and The lat four bit of value are decoded a an offet, x, in fixed-point forat 2.2, reulting in a nuber between 0 and x ax i finally calculated a x in + x. Siilar coputation are perfored for y in and y ax.. The apect ratio a = yax yin x ax x in i coputed, and a point ditribution i elected according to Table. Denote the ditribution d x d y. For ode IV, the ditribution i alway. 4. The recontruction level are calculated uing x k = x in + k d x (xax x in), k = 0,...,d x, and likewie for y k. 5. The pixel indice are now ued to deterine which recontruction level to ue. For intance, a value of 00 bin elect recontruction level x 2 for x. The y-value i obtained analogouly. 6. For ode ( II and III, we will alo rotate the coordinate x ) ( ) ( ) x co(φ) in(φ) uing y = M, where M = i y in(φ) co(φ) a rotation atrix and φ i π/6 or π/. See Section 4.2 for an efficient ipleentation. For ode I and IV, we jut ue x = x and y = y. 7. Diviion by 255, and reapping to [,] follow: x = 2x /255 and y = 2y /255. In the differential turtle voronoi lowmap bulge0 ultib ulge tar boxe toru kin barrel AV E R AG E 4.2. Efficient Rotation In thi ection, we ugget a hardware-friendly rotation. For ode II and III of our algorith, the decopreor need to rotate a two-dienional point by -0 and -60 degree. In the following, we develop an inexpenive, approxiate rotation for 0. The cae with 60 ue the ae contant, but at different location in the atrice, o thi i oitted fro our decription. The atrix for rotating 0 degree i: ( ) ( ) co( π/6) in( π/6) M= =. (2) in( π/6) co( π/6) The 0.5-ter above are not expenive to ipleent, but ultiplication by / i. To that end, we ugget that the hardware-friendly atrix M i ued intead: M M = ( ) = ( ), () where ultiplication by 0.75 can be ipleented a a hift by three and a ubtraction. Note that M i not an orthogonal atrix, i.e., M M T I. Therefore, we ephaize that we cannot ue M T during copreion, becaue it alo hold that MM T I. Intead, we ut ue the invere of M during copreion: M = ( ) ( ) (4) If M i ued to tranfor a rectangle, the reult will be different fro the rectangle obtained by uing M = M T. In fact, when uing M, the rectangle will get a light kew due to the fact that the tranfor i not orthogonal. However, the average PSNR for all our tet iage wa only reduced by 0.0 db on average, which i not ignificant. See Figure 7 for a poible hardware ipleentation. 5. Reult To evaluate the viual quality of our copreor, we have teted everal noral ap, taken fro the et in Figure, in a real-tie hader developent application, in order to iic a typical uer cenario. We have alo rendered iage uing a high-end renderer, with aniotropic ipap filtering, HDR environent apping and creen pace anti-aliaing. When copreing with Dc, we perfor exhautive earch for the bae value in the x- and y-direction eparately, to enure that our Dc copreor i near-optial. A full exhautive earch over x and y iultaneouly wa too cotly. In Figure 0, we how viual reult obtained uing a noral ap with lowly varying noral. The pixel hader ipleented iple environent apping in order to better

6 x tart x top < < AND XOR n 5 xor value 5 differential decoding unit x in. 2.2 x in - << << << << + * x ax 5 7 a < < < < a x >> >> + - rotation unit 0.75x n 0.5x neg neg clap to [0, 255] x y tart y top x pixel indice 4 y pixel indice 4 pixel index bit elector pixel index bit elector 5 xor value 5 x pixel index y pixel index y in y in. - + * y ax 7 a 5 point ditribution unit a a right hift tep right hift tep y >> >> y 0.5y n clap to [0, 255] y Figure 7: A hardware decopreor unit for our noral ap copreion algorith. To the left, 2 bit of data are hown, and thee are ued to decode one of the 6 noral in a 4 4 tile. A can be een, our three technique have been clearly arked. The reaining part i baically Dc (except that Dc only divide by 7). how the quality. A can be een, our technique provide uperior reult copared to ATI Dc technique. For thi particular ap, we have oberved an increae of 0 db in PSNR copared to Dc. Figure illutrate a tet with a typical gae noral ap [Gre04] with harp edge. Our algorith handle any difficult tile better due to the flexibility offered by the extra rotation and variable point ditribution. We rendered the iage in Figure 0 and uing an NVIDIA GeForce FX 600 graphic card. In the tet, we ue RGBfp6 texture, which are upported by the GPU. Another viual tet i hown in Figure 2, which wa rendered uing a high-quality offline renderer. In addition to obtaining viual reult, we alo ued the ean quare error (MSE), which i coputed a a uation over all noral in the iage: MSE = w h ( ˆx x) 2 + (ŷ y) 2 + (ẑ z) 2, (5) where w and h are the width and the height of the iage, x [,] i the x-coponent of the uncopreed noral and ˆx [, ] i the correponding copreed x-coponent, and iilar for y and z. For noral value, we ue the Peak Signal to Noie Ratio (PSNR): PSNR = 0log 0 ( MSE ), (6) where the noinator i one, ince the peak ignal for a noral of unit length will alway be equal to one, by contruction. PSNR value for all iage teted, for Dc and our cobined algorith are preented in Figure 9, with iproved value on all ap. The average iproveent i about db. We ee large difference on lowly varying ap and ap with harp egde. 6. Concluion We have deigned three new technique which can be ued in conjunction to the Dc noral copreion forat. A hown in our paper, the cobination of thee handle any of Dc weaknee uch better. Our technique are cobined into a chee that till fit into a bit budget of bpp and require only all addition to a hardware decopreor. The new forat i ore flexible, with Dc a a ubet, and we have obtained better reult on all noral ap teted, both viually and in the PSNR error eaure. For a erie of 20 noral ap, the average PSNR increaed with db. Acknowledgent We acknowledge upport fro the Swedih Foundation for Strategic Reearch and Vetenkaprådet.

7 Munkberg, Akenine-Möller, and Strö / High Quality Noral Map Copreion Figure 0: A grid cube-ap environent i ued for thee iage. The noral ap i a very lowly varying ap () fro Figure. Left: noral ap copreed with ATI Dc technique. Middle: rendered uing original noral ap. Right: noral ap copreed with our algorith Our algorith Dc a. Bupy b. Car c. dot d. dot2 PSNR (db) i. noralap j. onetile k. turtle l. voronoi. lowmap n. bulge o. ultibulge p. tar kin barrel toru tar boxe bulge0 ultib ulge lowmap turtle voronoi onetile etal NoralMap dot4 lupy 20 dot 0 dot2 h. etal car g. lupy dot f. dot4 bupy e. dot Figure 9: Thi chart how the PSNR value for the iage in Figure for Dc and our algorith. Our algorith i the cobined algorith, uing a tandard Dc ode, rotation (0 and 60 degree), a differential ode and variable point ditribution. [COM9] C OHEN J., O LANO M., M ANOCHA D.: Appearance-preerving iplification. In Proceeding of SIGGRAPH (99), ACM Pre, pp q. boxe r. toru. kin t. barrel Figure : The et of noral ap ued for evaluating our copreion algorith., n, o, p, q, and r are 2 bit/channel ap, all other ap are bit/channel. Reference [ATI05] ATI: Radeon X00: Dc White Paper. Tech. rep., [BAC96] B EERS A., AGRAWALA M., C HADDA N.: Rendering fro Copreed Texture. In Proceeding of SIGGRAPH (996), pp [Bli7] B LINN J.: Siulation of Wrinkled Surface. In Proceeding of SIGGRAPH (97), pp c The Eurographic Aociation [Dee95] D EERING M.: Geoetry Copreion. In Proceeding of SIGGRAPH (995), ACM Pre, pp. 20. [Gre04] G REEN S.: Bup Map Copreion. Tech. rep., NVIDIA, [INH99] I OURCHA K., NAYAK K., H ONG Z.: Syte and Method for Fixed-Rate Block-baed Iage Copreion with Inferred Pixel Value. In US Patent 5,956,4 (999). [KSKS96] K NITTEL G., S CHILLING A., K UGLER A., S TRASSER W.: Hardware for Superior Texture Perforance. Coputer & Graphic, 20, 4 (July 996), [TK96] T ORBORG J., K AJIYA J.: Talian: Coodity Real-tie D Graphic for the PC. In Proceeding of SIGGRAPH (996), pp

8 Figure : A typical gae noral ap (t), rendered in a real-tie hader developent application, with a cube reflection ap. Left: noral ap copreed with ATI Dc technique. Middle: rendered uing original noral ap. Right: noral ap copreed with our technique. Figure 2: The noral ap (k), rendered in a high-end off-line renderer, with HDR environent apping, texture filtering and advanced anti-aliaing. Left: Dc. Middle: uncopreed ap. Right: our algorith. A can be een in the iage, Dc how ore "wobbling" artifact, and oe feature even diappear. Our new algorith how higher quality, even though oe artifact reain.