THE PULL-PUSH ALGORITHM REVISITED

Transcription

1 THE PULL-PUSH ALGORITHM REVISITED Improvements, Computaton of Pont Denstes, and GPU Implementaton Martn Kraus Computer Graphcs & Vsualzaton Group, Technsche Unverstät München, Boltzmannstraße 3, Garchng, Germany Keywords: Abstract: pyramd algorthm, mage processng, real-tme renderng, scattered data, meshless data, nterpolaton, GPU The pull-push algorthm s a well-nown and very effcent pyramd algorthm for the nterpolaton of scattered data wth many applcatons n computer graphcs. However, the orgnal algorthm s not very well suted for an mplementaton on GPUs (graphcs processng unts). In ths wor, several mprovements of the algorthm are presented to overcome ths lmtaton, and mportant detals of the algorthm are clarfed, n partcular the mportance of the correct normalzaton of the employed flters. Moreover, we present an extenson for a very effcent estmate of the local densty of sample ponts. INTRODUCTION Interpolaton of scattered data ponts s a classcal problem n computer graphcs wth many applcatons, e.g., npantng, reconstructon of smooth functons from meshless pont data, mage postprocessng effects, etc. (Gortler et al., 996; Dror et al., 2003). As recent GPU mplementatons of pyramd algorthms for scattered data nterpolaton acheve real-tme performances, they are also suted for real-tme renderng and real-tme vdeo processng (Lefebvre et al., 2005; Strengert et al., 2006; Guennebaud et al., 2007; Kraus and Strengert, 2007a). However, there are partcularly challengng renderng algorthms, whch requre the nterpolaton of scattered pxels for multple mages per frame. For example, the renderng of depth-of-feld effects by Kraus and Strengert (Kraus and Strengert, 2007a) requres two full-screen nterpolaton passes for each submage wth to 20 submages per frame (dependng on the strength of the effect). Thus, even a hgh-performance GPU mplementaton of scattered data nterpolaton can consttute a serous performance bottlenec for some renderng algorthms. Therefore, ths wor s concerned wth mprovements of GPU mplementatons of the very effcent pullpush algorthm by Gortler et al. (Gortler et al., 996). The next secton brefly summarzes related wor whle Secton 3 presents our algorthmc mprovements of the pull-push algorthm and compares them to the orgnal algorthm. Secton 4 presents a varant of the algorthm to compute the local densty of sample ponts, and n Secton 5, the GPU mplementatons of the algorthms and ther performance s dscussed. 2 RELATED WORK The orgnal pull-push algorthm by Gortler et al. (Gortler et al., 996) s based on wor by Burt (Burt, 98; Burt, 988) and Mtchell (Mtchell, 987) and was used to nterpolate between many samplng ponts n a four-dmensonal doman n the Lumgraph system. A varant of the pull-push algorthm was employed, for example, by Dror et al. (Dror et al., 2003) for an ntal reconstructon of mssng parts of mages. Lefebvre et al. (Lefebvre et al., 2005) used a GPU-based mplementaton for npantng gaps n texture maps. Strengert et al. (Strengert et al., 2006) proposed a more effcent GPU mplementaton, whch s of lnear tme complexty n the number of pxels. The algorthm by Strengert et al. was employed by Kraus and Strengert (Kraus and Strengert, 2007a) to dsocclude pxels n ther depth-of-feld al-

2 x 2 0 x 0 x x 0 0 x 0 x 0 2 x 0 3 x n 0 x n 0 x n x 2 2 n 2... x 2 n 2 x 2 n... x 0 2 n 4 x0 2 n 3 x0 2 n 2 x0 2 n Fgure : Illustraton of the pxels x r of an mage pyramd for a one-dmensonal nput mage x 0 of sze 2 n. gorthm. Guennebaud et al. (Guennebaud et al., 2007) employed a GPU mplementaton of the pull-push algorthm to reconstruct soft shadows from a reduced number of sample ponts; however, they dd not descrbe any detals of ther mplementaton. 3 REVISION OF THE PULL-PUSH ALGORITHM 3. Pull The orgnal pull phase by Gortler et al. (Gortler et al., 996) computes an mage pyramd of data values x and postve weghts w usng a pull flter h. The nput mage determnes the sze of the fnest pyramd level r = 0 whle the sze of pyramd level r+ s half of the sze of pyramd level r n each dmenson. For clarty we descrbe the algorthm for one-dmensonal mages and restrct ther szes to powers of two. Data values of pyramd level r are denoted by x r and ther weghts by w r where r equals 0 for the orgnal (fnest) resoluton and pxels are ndexed by. Ths structure of a one-dmensonal mage pyramd s llustrated n Fgure. If the pull flter s specfed by a dscrete sequence h of postve numbers, Gortler et al. suggest to compute the pxels of the next coarser mage level by: w r+ x r+ = = h 2 mn(w r,) () w r+ h 2 mn(w r,)xr (2) To reduce the number of clamp operatons,.e., mn functons, we propose to ntroduce clamped weghts ŵ r and weghted data values x r, n partcular for the nput data on level r = 0: ŵ 0 def = mn ( w 0, ), x 0 def = ŵ 0 x 0 (3) The computaton of coarser mage levels can now be performed wth just one clamp operaton and basc flter operatons: w r+ = h 2 ŵ r (4) ŵ r+ = mn ( w r+, ) (5) x r+ = ŵr+ w r+ h 2 x r (6) Ths formulaton s more suted for an effcent GPU mplementaton as dscussed n Secton 5. It should be noted that the normalzaton of h s crucal for the qualty of the resultng nterpolaton. As shown n Secton 3.3, a normalzaton of the maxmum to provdes good results n most cases whle a normalzaton of the ntegral of the flter h to leads to sgnfcantly worse results. Our new formulaton encapsulates the nonlnear part of the pull phase n a sngle clamp operaton n Equaton 5, whch can be replaced easly by a contnuous operaton. For example, one could use a flter h wth a normalzaton such that ts ntegral s and acheve an effect smlar to the nonlnear clamp operaton by employng the followng equaton nstead of Equaton 5: ŵ r+ = ( w r+ ) γ (7) Here, γ s a parameter that controls the range of nfluence of data ponts: larger values of γ tend to result n larger regons around solated data ponts where the nterpolated values are close to the values specfed for the data ponts; examples are gven n Secton Push After the mage pyramd has been computed, a sequence of push operatons recomputes all levels from coarse to fne wth a push flter h. Frst, two temporary values are computed: ω r = h 2 mn ( w r+, ) (8) χ r = ω r h 2 mn ( w r+, ) x r+ (9) Note that Gortler et al. use the symbol tw nstead of ω and tx nstead of χ.

3 0 x 0 x 0 x (a) (b) (c) Fgure 2: (a) Interpolaton of a set of scattered data (gray bars) by the orgnal algorthm by Gortler et al. (Gortler et al., 996) wth h = { 3,,, 3} (sold lne) and h = { 8, 3 8, 8 3, 8} (dashed lne). (b) Interpolaton by the w/o clamp algorthm for γ = 4 (sold lne) and γ = 6 (dashed lne). For comparson, the result of the orgnal algorthm s also ncluded (dotted lne). (c) Same as (b) for nput data of weght 32 nstead of. These temporary values are blended wth the prevously computed pyramd mage. However, the equatons provded by Gortler et al. lac clampng operatons; the corrected equatons are: x r = χ r ( mn(w r,))+mn(wr,)xr (0) w r = mn(ω r,)( mn(w r,)) +mn(w r,) () These equatons can also be smplfed by employng clamped weghts ŵ r and weghted data values xr as ntroduced n the prevous secton. For the push phase we can addtonally assume a normalzaton of the push flter h such that no further clamp operatons are necessary. Therefore, Equatons 8 can be rewrtten as: ˆω r = h 2 ŵ r+ (2) χ r = ˆω r h 2 x r+ (3) x r = χ r ( ŵ r )+ xr (4) ŵ r = ˆω r ( ŵ r )+ŵr (5) x r = ŵ r x r (6) These flter operatons can be mplemented very effcently (see Secton 5); however, the lnear nterpolaton wth weghts ŵ r and ŵr cannot be mplemented by standard blendng on GPUs due to the multplcaton n Equaton 6. Fortunately, these equatons can be further smplfed by a new approxmaton, whch wors very well for most nput data and flters h and h. As noted above, the pull flter h should be scaled such that the weghts ŵ r qucly approach n coarser pyramd levels. Durng the push phase these weghts are further ncreased, thus, they are usually equal to or close to n all pyramd levels. Ths motvates a varaton of the blendng such that all new weghts ŵ r are (mplctly) set to. Thus, Equatons 2 6 are sgnfcantly smplfed to only two equatons: χ r = h 2 x r+ (7) x r = χ r ( ŵ r )+ xr (8) Note that we start wth x r+ nstead of x r+ on the coarsest level, whch can be easly computed n the last step of the pull phase by avodng the multplcaton wth ŵ r+ n Equaton 6. For all but the frst teraton, x r+ n Equaton 7 refers to the pyramd level that was most recently computed accordng to Equaton 8 n the push phase, whle ŵ r and xr n Equaton 8 refer to values computed n the pull phase. Our new formulaton allows for both an effcent GPU mplementaton of the flterng operatons by texture flterng as well as an effcent GPU mplementaton of the lnear nterpolaton by frame buffer blendng as dscussed n Secton Comparson of Varants In ths secton, we compare the orgnal algorthm by Gortler et al. (Gortler et al., 996) wth the two varants proposed n Sectons 3. and Orgnal Algorthm For the purpose of ths comparson, the orgnal algorthm s descrbed by Equatons 3 6 for the pull phase and Equatons 8 for the push phase. A one-dmensonal pull flter h s used, whch s defned by the sequence { 3,,, 3},.e., the maxmum { s normalzed to. The push flter h s defned by 4, 4 3, 3 4, } 4. These flters correspond to quadratc B- splne flters (Catmull and Clar, 978) and are partcularly well suted for GPU mplementatons (Kraus and Strengert, 2007b).

4 (a) (b) (c) Fgure 3: (a) A two-dmensonal mage of sze (b) Mased varant of the mage n (a): the weght of blac pxels s 0, the weght of all others s. (c) Densty of the pxels wth weght computed by the algorthm presented n Secton 4. (a) (b) (c) Fgure 4: Images reconstructed from Fgure 3b usng (a) the orgnal algorthm, (b) the w/o clamp varant, (c) the w/o weght varant. Fgure 2a depcts a set of scattered nput pxels of weght n a one-dmensonal mage of dmenson 256 and the data nterpolaton computed wth the orgnal algorthm by Gortler et al. as descrbed above. For comparson, the results wth a dfferently normalzed pull flter h specfed by { 8, 3 8, 3 8, 8}, s also ncluded. Wth nput weghts less than or equal to, ths normalzaton guarantees that all weghts stay less than or are equal to n the pull phase, therefore, the clamp operatons are unnecessary. However, the latter pull flter does not provde a smooth nterpolaton of the nput ponts. In fact, t appears to be extremely dffcult f not mpossble to desgn approprate pull flters for the orgnal algorthm that do not requre clampng operatons but stll result n smooth nterpolatons. Thus, n contrast to the dscusson by Gortler et al., the clampng s not only necessary to lmt the nfluence of clusters of many nput ponts but t s also crucal for a smooth nterpolaton n general Varant w/o clamp In order avod the nondfferentable clampng and to smplfy the desgn of pull flters, we propose a frst varant of Gortler et al. s algorthm, named w/o clamp, whch replaces the clampng specfed n Equaton 5 by the dfferentable Equaton 7. For ths varant of the algorthm, the ntegral of the pull flter h has to be normalzed to n order to guarantee that weghts do not exceed. We choose { 8, 3 8, 8 3, } 8 for the examples depcted n Fgure 2b and 2c. The parameter γ n Equaton 7 was set to 4 and 6 n order to show ts nfluence. Whle the nput pxels n Fgure 2b are the same as n Fgure 2a, the nput weghts n Fgure 2c are only 32 nstead of n Fgure 2b. The dotted curve depcted n Fgure 2c suggests that t s dffcult to obtan a smooth curve approxmatng sample ponts wth weghts less than usng the orgnal algorthm. The stuaton s qute smlar to an unsutable normalza-

5 ton of h as the clampng operaton s neffectve for several levels n both cases. On the other hand, alternatve nonlnear flters such as mplemented n our varant w/o clamp, can provde smoother curves for nput ponts wth weghts less than as llustrated by the dashed curve n Fgure 2c Varant w/o weght The second varant s named w/o weght and mplements Equatons 3 6 for the pull phase; however, the multplcaton wth ŵ r+ n Equaton 6 s spped for the coarsest level n order to compute x r+ nstead of x r+. For the push phase, Equatons 7 and 8 are employed. In our experments, ths varant and the orgnal algorthm computed very smlar results although the w/o weght varant employs a consderably smplfed push phase, whch s better suted for an effcent mplementaton on GPUs Two-Dmensonal Examples Fgure 4 presents the two-dmensonal results for the nput depcted n Fgure 3b. Vsually there are no dfferences between the result of the orgnal algorthm employed for Fgure 4a and the results for the two new varants of the algorthm shown n Fgure 4b and Fgure 4c. 4 COMPUTATION OF POINT DENSITIES It s often useful to compute a local densty of scattered ponts, for example n order to decde where to place addtonal samplng ponts. Here we present a computaton of the local densty of nput ponts by a varant of the presented pull-push algorthm. We assume that all weghts of nput pxels w 0 are equal to and all weghts of unspecfed pxels are 0. In ths case, the pyramd mages w r computed n the pull phase are n fact approxmatons of the densty of nput ponts f the ntegral of the pull flter h s normalzed to. However, t s crucal to choose an approprate pyramd level r. In partcular, r should be chosen locally. To ths end, we base the computaton of the pont densty on those values w r where r s approprate for the specfc poston. Ths can be acheved by ntroducng new weghts p r for the nterpolaton of denstes w r : the weght pr determnes the nfluence of w r analogously to the role w r played for x r n the orgnal pull-push algorthm. We determne actual values for p r n dependency of the average number n r of nput ponts at pxel n p n Fgure 5: Two examples for the functon p(n) to determne weghts dependng on the average number n of nput ponts per pxel. The sold curve s sutable for onedmensonal mages whle the dotted curve s sutable for two-dmensonal mages. the pyramd mage at level r: n r def = 2 r w r (9) If the average number of nput ponts per pxel n r s less than, p r should be 0 because the correspondng w r s very lely to overestmate the local densty n the neghborhood of a sngle, solated nput pont. On the other hand, f n r s very large, p r should also be 0 as local varatons of the densty of nput ponts are not reflected by the correspondng w r f too many nput ponts are ncluded n the average. In between these extremes, p r s set accordng to a functon p( n), whch s applcaton-specfc as even very fne local varatons are of nterest n some cases for example when estmatng the dstance to the nearest nput pont whle other applcatons n partcular for hgh-dmensonal mages requre larger values of n r to relably estmate the densty of nput ponts. Fgure 5 llustrates two examples for the functon p( n). Havng computed denstes w r and ther weghts p r n the pull phase, we can then apply a push phase, whch reconstructs a smooth functon for w 0 on the fnest pyramd level analogously to reconstructng a smooth functon x 0 from data values x r and weghts w r. Next we descrbe ths algorthm for the GPUfrendly varant of the pull-push algorthm presented n Secton 3.2 (called w/o weght n Secton 3.3). The algorthm starts wth an nput mage w 0, where the w 0 s are ether 0 or. The pull phase mplements the followng equatons: w r+ = h 2 w r (20) n r+ = 2 r+ w r+ (2) p r+ = p ( n r+ ) (22) As mentoned above, the ntegral of the pull flter h has to be normalzed to. The push flter h has to be n

6 normalzed analogously and s used n the push phase, whch recomputes the mage pyramd from coarse to fne levels usng these equatons: χ r = h 2 w r+ (23) w r = χ r ( p r )+ pr wr (24) The fnest pyramd mage w r s the resultng estmate of the local densty of ponts. Fgure 3c depcts the resultng densty for the nput shown n Fgure 3b. The two-dmensonal generalzatons of the same pull and push flters were employed and the dotted curve n Fgure 5 was used for p( n). 6 CONCLUSIONS As shown n ths wor, the proposed algorthms for the nterpolaton of scattered data ponts and the computaton of local denstes of scattered ponts are well suted for very effcent GPU mplementatons, whch could be used for real-tme renderng or realtme mage and vdeo processng. The qualty of the results computed by the proposed algorthms depends on the choce of the pull and push flters, the nonlnear functon n Equaton 7, and the functon p( n) n Equaton 22. Determnng sutable flters and functons for partcular applcatons s part of future wor. 5 GPU IMPLEMENTATION Whle there are very effcent GPU mplementatons of lnear flterng (Kraus and Strengert, 2007b), any addtonal functons n the summaton (as n Equaton ) or addtonal nonconstant factors (as n Equaton 2) defeat an effcent GPU mplementaton. Ths problem s solved by our mprovements presented n Secton 3, n partcular Equatons 4, 6, and 7. For multdmensonal data (e.g., RGB mages) the flterng operatons n Equatons 6 and 7 can be performed n parallel for up to 3 components on GPUs. Moreover, the flterng of weghts n Equaton 4 can be performed n parallel n the A component of an RGBA texture mage n order to mplement Equaton 8 by alpha-blendng. Note that only ŵ r and x r have to be stored n the pull phase, whle the push phase only stores new values for x r n place of the values x r, whch were computed n the pull phase. Whle Gortler et al. (Gortler et al., 996) clam that Equatons 8 can be mplemented by standard blendng, ths s unfortunately no longer true f the computaton s based on weghted varables x r for more effcent flterng. Ths problem s solved by the approxmaton presented n Secton 3.2, n partcular Equatons 7 and 8. The two-dmensonal flters employed n Secton 3.3 requre only four texture looups per pxel of the coarser level n the pull phase and one texture looup per pxel of the fner level n the push phase f the flterng s mplemented effcently (Kraus and Strengert, 2007b). As GPU mplementatons of the proposed algorthms are bandwdth-lmted, the arthmetc operatons do not sgnfcantly affect the performance. Our mplementaton requres 0 ms for a bt floatng-pont RGBA mage on a NVIDIA Quadro FX 5800 graphcs board. REFERENCES Burt, P. J. (98). Fast Flter Transforms for Image Processng. Computer Graphcs and Image Processng, 6:20 5. Burt, P. J. (988). Moment mages, polynomal ft flters, and the problem of surface nterpolaton. In Proceedngs of Computer Vson and Pattern Recognton, pages Catmull, E. and Clar, J. (978). Recursvely Generated B-Splne Surfaces on Arbtrary Topologcal Meshes. Computer Aded Desgn, 0(6): Dror, I., Cohen-Or, D., and Yeshurun, H. (2003). Fragment-Based Image Completon. ACM Transactons on Graphcs, 22(3): Gortler, S. J., Grzeszczu, R., Szels, R., and Cohen, M. F. (996). The Lumgraph. In SIGGRAPH 96: Proceedngs of the 23rd Annual Conference on Computer Graphcs and Interactve Technques, pages Guennebaud, G., Barthe, L., and Pauln, M. (2007). Hgh- Qualty Adaptve Soft Shadow Mappng. Computer Graphcs forum (Proceedngs Eurographcs 2007), 26(3): Kraus, M. and Strengert, M. (2007a). Depth-of-Feld Renderng by Pyramdal Image Processng. Computer Graphcs Forum (Conference Issue), 26(3): Kraus, M. and Strengert, M. (2007b). Pyramd Flters Based on Blnear Interpolaton. In Proceedngs GRAPP 2007 (Volume GM/R), pages Lefebvre, S., Hornus, S., and Neyret, F. (2005). Octree Textures on the GPU. In Pharr, M., edtor, GPU Gems 2, pages Mtchell, D. P. (987). Generatng Antalased Images at Low Samplng Denstes. In Proceedngs of SIG- GRAPH 87, pages Strengert, M., Kraus, M., and Ertl, T. (2006). Pyramd Methods n GPU-Based Image Processng. In Proceedngs Vson, Modelng, and Vsualzaton 2006, pages