Hierarchical Stochastic Motion Blur Rasterization

Transcription

1 Hierarchical Sochasic Moion Blur Raserizaion Jacob Munkberg Perik Clarberg Jon Hasselgren Rober Toh Masamichi Sugihara Tomas Akenine-Möller, Inel Corporaion Lund Universiy Absrac We presen a hierarchical raversal algorihm for sochasic raserizaion of moion blur, which efficienly reduces he number of inside ess needed o resolve spaio-emporal visibiliy. Our mehod is based on novel ile agains moving primiive ess ha also provide emporal bounds for he overlap. The algorihm works enirely in homogeneous coordinaes, suppors MSAA, faciliaes efficien hierarchical spaio-emporal occlusion culling, and handles ypical game workloads wih widely varying riangle sizes. Furhermore, we use high-qualiy sampling paerns based on digial nes, and presen a novel reordering ha allows efficien procedural generaion wih good ani-aliasing properies. Finally, we evaluae a se of hierarchical moion blur raserizaion algorihms in erms of boh deph buffer bandwidh, shading efficiency, and arihmeic compleiy. CR Caegories: I.. [Compuer Graphics]: Hardware archiecure Graphics processors I.. [Compuer Graphics]: Picure/Image Generaion Anialiasing I..7 [Compuer Graphics]: Three-Dimensional Graphics and Realism Hidden line/surface removal Keywords: sochasic raserizaion, moion blur, hierarchical raversal, occlusion culling Inroducion A he hear of every rendering engine here is some form of visibiliy compuaions. A more advanced algorihm allows effecs such as moion blur and deph of field o be rendered by a more elaborae camera model. Deph of field helps o direc he viewer s aenion, and moion blur reduces emporal aliasing, so ha lower frame raes can be used. Boh hese effecs are also highly desired in he field of real-ime graphics. While an incredible amoun of research and engineering effor has been spen on perfecing and fine-uning he algorihms and he corresponding hardware unis for raserizing saic riangles [Fuchs e al. 99; Pineda 9; Olano and Greer 997; McCormack and McNamara ; McCool e al. ], he same is far from rue for raserizaion of moion-blurred geomery. However, here has been increased research aciviy in his field [Cook e al. 97; Akenine- Möller e al. 7; Faahalian e al. 9; McGuire e al. ; Brunhaver e al. ], bu much remains o be done before he relaive efficiency of raserizing riangles wih blur effecs is close o ha of saic riangle raserizaion. To be able o add correc moion blur o curren and fuure games, one of our goals is o suppor efficien rendering of moion blur wih mied sizes of he riangles, i.e., boh large riangles and smaller riangles, generaed by, e.g., essellaion. To ha end, we presen wha we believe is he firs hierarchical raserizaion algorihm for moion-blurred riangles. We srive for an algorihm ha eends curren real-ime GPU pipelines, while reaining many of is imporan feaures, such as per-ile occlusion culling, mied riangle sizes, shading afer visibiliy, and mulisampling ani-aliasing (MSAA). Our conribuions are: A hierarchical algorihm for moion blurred riangle raserizaion, including a low-cos ile vs moving riangle overlap es ha reurns a conservaive ime inerval of overlap. Modificaion of an eising hardware-friendly sampling paern for use in moion blur raserizaion wih high-qualiy anialiasing. We presen an efficien algorihm for compuing he samples wihin a ime inerval on he fly. Deailed performance evaluaion of several differen moion blur raserizaion algorihms in erms of arihmeic inensiy, memory bandwidh usage, and shading efficiency. We hope ha our new algorihms will advance he field of moion blur raserizaion so ha in he near fuure, fied-funcion raserizaion unis will have suppor for such effecs. Relaed Work Efficien rendering of moion blur has been a long-sanding problem in compuer graphics. Eising soluions ofen rely on approimae pos-processing based mehods or sochasic ray racing [Cook e al. 9]. We will no go ino deail on hese mehods, and insead focus on raserizaion-based mehods for correc moion blur ha can be inegraed ino fuure hardware GPU pipelines. A brue-force echnique is o draw he scene a N differen imes and average he resul using accumulaion buffering [Korein and Badler 9; Haeberli and Akeley 99]. The resuling srobing arifacs can be replaced by noise by using sochasic raserizaion [Cook e al. 97]. Here, a bounding bo around he blurred riangle is raversed, and all samples are esed agains he primiive displaced according o he samples imes. This becomes inefficien when he bounding bo is large compared o he primiive. The screen space area of he raversed region can be reduced using eiher an oriened bounding bo (OBB) in D homogeneous space [Akenine-Möller e al. 7], or he conve hull in screen space [McGuire e al. ]. Eising (wo-dimensional) hierarchical raserizaion mehods can be leveraged o efficienly raverse hese bounds, bu all emporal samples sill have o be esed. This becomes epensive wih large moion. In conras, our algorihm derives emporal bounds per ile o cull samples. Faahalian e al. [9] improve he siuaion for sochasic micropolygon raserizaion by pariioning he ime domain ino muliple inervals (iniially proposed by Piar), or by using inerleaved sampling [Keller and Heidrich ] wih a fied number of sample imes. Boh mehods raserize he primiive independenly for each ime/inerval, which generaes samples in an incoheren order, i.e., sparse in screen space. For a REYES pipeline wih shading a he

2 y y y where a ile is a recangular block of piels: BBOX = Compue moving riangle bounding bo for each ile in BBOX [hierarchical raversal] TIME = Compue ime inerval of overlap Occlusion culling of ile in TIME 5 for each sample in ile in TIME Tes sample agains primiive Our algorihm Conve hull Inerleaved Sampling Figure : The hree-dimensional sampling space, (, y, ), raversed wih differen mehods for sochasic moion blur raserizaion. Sample-in-riangle inside ess are performed for all samples wihin he red regions. Our algorihm, based on novel hierarchical ile ess wih emporal overlap compuaions, significanly reduces he amoun of inside ess compared o using he conve hull in screen space [McGuire e al. ]. Wih inerleaved sampling [Keller and Heidrich ] he samples are resriced o a fied number of pre-defined imes. vere level, his is no a problem, bu applied o a graphics pipeline wih shading a he fragmen level, i makes reusing shading over muliple samples (MSAA) difficul. Per-ile occlusion culling also becomes subsanially more epensive. Furhermore, each riangle has o be seup muliple imes. In conras, our algorihm uses a coheren screen space raversal order, which faciliaes MSAA and efficien occlusion culling. Figure shows he spaio-emporal coverage of each algorihm. Alhough a hard problem, analyical deerminaion of visibiliy has been eplored. Mos recenly, Gribel e al. [] presened a mehod for analyical moion blur raserizaion where he samples emporal overlaps wih a moving primiive are analyically deermined and sored in linked liss per piel. Our work is similar in ha we analyically deermine conservaive ime bounds, bu we do his for enire iles of piels and he generaed samples are sored in a radiional muli-sampled render arge. The use of a iled raversal wih emporal bounds allows us o quickly rejec samples. Hierarchical occlusion culling is criical for achieving good performance in modern GPUs by early deermining if a ile is enirely occluded (z ma-culling) [Morein ] or enirely visible (z minculling) [Akenine-Möller and Sröm ]. However, moion blur makes culling using a radiional hierarchical z-buffer [Greene e al. 99] less efficien. By soring muliple emporal deph values (zslice) [Akenine-Möller e al. 7], or a full emporal pyramid of deph values (z-pyramid) [Boulos e al. ] per node, efficiency can be improved. Our raserizaion order, i.e., one ile a he ime, ogeher wih conservaive emporal bounds, makes he use of hese occlusion culling echniques efficien and sraighforward. Overview Our hierarchical moion blur raversal algorihm works enirely in wo-dimensional homogeneous space (DH) o robusly handle moving riangles crossing he z = plane. The firs sep is conservaive backface culling [Munkberg and Akenine-Möller ] and emporal view frusum culling. Each riangle vere moves along a line in DH, and by finding he inersecion of hese hree lines wih each frusum plane, we obain he ime inerval, [ s, e], when he moving riangle is inside he view frusum. The raversal algorihm for a riangle can be summarized as follows, To compue a screen space bounding bo around he moving riangle, we bound he screen space projecions of he si verices (he riangle verices a s and e) if he moving riangle is enirely in fron of he z = plane, and rever o he conservaive bounding approach presened by McGuire e al. [] oherwise. In Secion, we inroduce he ile vs moving riangle ess (line ), which form he necessary basis for our hierarchical raversal algorihm. The oupu for a cerain ile is eiher rivial rejec, or a conservaive ime inerval where overlap possibly occurs. The compuaion of per-ile ime bounds grealy reduces he number of emporal samples ha are esed for fas moving primiives, as large subses of he spaio-emporal samples wihin a ile can be discarded. I also makes hierarchical occlusion culling simple and efficien. For each ile, we only es he primiive agains occlusion informaion in he relevan ime inerval (line ). As wih all sochasic mehods, he saisical disribuion of he sample poins has a large impac on he resul. Sochasic raserizaion has he addiional consrains ha he samples mus be consisen from primiive o primiive (oherwise cracks may appear), and eremely fas o generae as he sampling akes place in he inner loop of he raserizer. We have chosen o base our samples on binary (, m, s)-nes [Niederreier 99] for heir eensive sraificaion properies. Secion 5 inroduces a remapping of a known paern o provide a emporally ordered sequence ha is eremely inepensive o compue in hardware. Las, we discuss emporal filering for high-qualiy shading of he generaed samples in Secion, followed by implemenaion deails and a horough evaluaion in Secions 7 and, respecively. Tile Tess wih Temporal Bounds I is well-known ha efficien raserizaion of saic geomery can be obained by hierarchical esing of a ile of piels agains a riangle [McCormack and McNamara ]. This is done by overlap esing he bounding bo of he riangle agains he ile, and also esing each riangle edge agains he ile [Akenine-Möller and Aila 5]. We will eend his o moving geomery, where he bounding bo becomes a moving bo, and he riangle edges sweep hrough space. More specifically, we derive igh bounds for he overlap beween a screen space ile and a riangle wih linear per-vere moion in hree dimensions. Each vere, p i, moves from he posiion q i a =, o r i a =, ha is: p i() = ( )q i + r i. All compuaions are performed in D homogeneous coordinaes, wih a vere defined as p = (p, p y, p w). The main idea is o find a conservaive ime inerval, ˆ o = [ o, o], in which he moving riangle overlaps he ile. Per-sample ess are hen done only for samples whose imes belong o he ime inerval. In he following, we firs describe how a ile is esed agains a moving bo, and hen how a ile is esed agains a moving riangle edge.. Frusum Plane Moving AABB Overlap We creae a moving AABB in DH by bounding he riangle a = and = and inerpolaing beween he wo AABBs. This is an approimaion o he rue swep bounding bo, bu i is guaraneed

3 ile frusum plane ile frusum plane e(,y,) e(,y,) e(,y,) = ile w = origin ile screen space y = = origin screen space Figure : A moving riangle is enclosed by an AABB in DH wih linear per-vere moion. The lef figure shows he w plane, wih indicaors when he moving AABB eners (green do) and eis (red do) he ile frusum. The righ illusraion shows he screen space view of his eample. o be conservaive a all imes. Based on a ile on screen, we hen seup four frusum planes ha are aligned o he sides of he ile. Each frusum plane, π i, passes hrough he origin and is defined by is plane equaion n i p =, where n i is he plane s normal. A poin p is ouside he plane if n i p >. If a poin is inside all planes, hen i is inside he frusum. For saic geomery, i is sufficien o es he corner of an enclosing AABB ha is farhes in he negaive direcion (n-vere) relaive o π i [Greene 99], in order o deermine if he bo is enirely in he posiive half-space. The sign bis of he plane s normal, n i, direcly decides which corner is he n-vere. We noe ha he same holds for linearly moving bounding boes, as he orienaions of he frusum planes remain consan. Figure shows an eample of a moving riangle, whose moving AABB inersecs wih wo ile frusum planes. The poin of inersecion in ime beween he moving n-vere and a plane π i is given by: n i (( )q n + r n) = = n i q n n i (q n r n), () where ( )q n + r n is he moving n-vere for π i. Le d = n i q n and d = n i r n. The emporal overlap, ˆ i, beween he AABB and he plane π i is given by: if d, d > boh ouside [ma(, ), ] else if d ˆ i = > q n ouside [, min(, )] else if d > r n ouside [, ] oherwise boh inside, where is compued using Equaion. The emporal overlap beween all he ile planes and he moving AABB is given by ˆ bo = ˆ i i, where we can es for fine-grained rivial rejecion afer each ieraion of he loop over he four frusum planes, π i.. Moving Triangle Edge Tess For riangles wih linear vere moion in hree dimensions, each riangle edge sweeps ou a bilinear pach. The corresponding imedependen edge funcions are quadraic in. To deermine if a screen space ile overlaps he swep riangle, we evaluae he riangle s hree edge equaions for he four corners of he ile and check if any corner is inside all hree edges. In ha case, we again deermine a ime inerval in which he riangle conservaively overlaps he ile o reduce he number of per-sample inside ess. The edge equaion for a riangle wih linear per-vere moion can be wrien as follows [Akenine-Möller e al. 7]: () e(, y, ) = n() s = (f + g + h) s, () Figure : Edge equaions as funcions of for a specific (, y) locaion. We are ineresed in finding he ime inervals where e < (highlighed in urquoise). (,c) e,y() = a +b+c (,a+b+c) (,b+c) Figure : The lower bound of a quadraic polynomial from Equaion is bounded by a linear approimaion around =. where s = (, y, ) is a sample posiion in screen space. For a given s, we have a maimum of wo roos o e(, y, ) =, and up o wo ime inervals per edge, ˆ i = [, ], where he sample is inside (e < ). Some eamples are given in Figure. Handling near-linear edge moion in an efficien and robus way is eremely imporan, because ofen a large porion of he riangles in a scene will have close o linear moion. In hese cases, direcly finding he roos of e = involves a division wih a very small quadraic coefficien, which may lead o numerical insabiliy. Therefore, we have devised a robus es ha performs well when he edge equaions are near-linear, and ha is increasingly conservaive when he quadraic erm grows. We bound he quadraic edge funcion s projecion wihin a screen space ile using lines wih consan slopes. This linearizaion of he overlap es grealy reduces he compuaions needed. The edge funcion (Equaion ) is linearized according o: e(, y, ) = n() s o s + γ, s S, () where S is a region in screen space, e.g., he bounding bo of he swep riangle. Wih n()=f +g+h, we rewrie he edge funcion for a cerain screen space posiion, s, as [Gribel e al. ]: n() s = f s + g s + h s = a + b + c, (5) where a = f s, b = g s and c = h s. I can be shown ha his curve is included in he riangle given by he poins (, c), (, b+c) and (, a + b + c) as seen in Figure. We search for a lower linear bound of he curve s slope, which is given by: min(b, a + b) = min(g s, (f + g) s). () A conservaive minimal slope, γ, for all s S, is given by: γ = min(g s, (f + g) s). (7) s S If we se o = n() = h, we have obained a linearized version of he edge equaion according o Equaion. This linear represenaion is conservaive even if he edge funcion has a large quadraic erm. Noe ha γ can be compued in he riangle seup using he moving riangle s screen space AABB as S. For more accurae bounding, γ can be recompued on a coarse ile level, using he ile eens as S. Given he linearizaion, he ile vs moving edge es is considerably simplified. By looking a he signs of he y-componens of o, we

4 Figure 5: Eample of a (,, )-ne in base. The five figures illusrae all elemenary inervals wih area b m = over he uni square, where each one has eacly b = = samples. We presen a mehod for procedural consrucion of hree-dimensional (, m, s)-nes wih properies argeed a moion blur raserizaion. only need o es one ile corner, s. A conservaive ime for he inersecion of he riangle edge and he ile is given by: o s + γ = = o s γ. () Noe ha o can be precompued, so he ime of overlap for a ile γ only coss MADD per edge. Depending on he sign of γ, he ile s emporal overlap, ˆ k, wih edge e k is defined as: ˆ k = { [ma(, ), ] if γ <, [, min(, )] oherwise, where is compued according o Equaion. Once all hree riangle edges have been esed, he emporal overlap beween he ile and he swep riangle is given by ˆ edges = k ˆ k, where we can es for fine-grained rivial rejecion afer each ieraion of he loop over edges (e k ). The final inerval is he inersecion of he inervals from boh he moving bo es (Secion.) and he moving edge es, i.e., ˆ o = ˆ bo ˆ edges. Given ˆ o, we firs perform spaio-emporal occlusion culling, and for he surviving iles and ime inervals, we proceed wih individual sample-in-riangle inside ess. The following secion describes he compuaion of our sampling posiions, (, y, ). 5 Sampling In a moion blur raserizer, each piel is associaed wih a number of fied (, y, ) samples. The samples mus be he same from riangle o riangle o ge correc visibiliy, bu vary randomly from piel o piel o reduce emporal and spaial aliasing. Sochasic sampling inroduces noise, and i is well-known ha sample poins wih good saisical properies, e.g., large minimum disance, provide a good balance beween noise and aliasing. For us, i is also desirable ha he samples projec o a good disribuion in (, y) for high-qualiy ani-aliasing of saic primiives. Our applicaion imposes a number of furher consrains. Firs, sampling needs o be fas and use minimal sorage, as i is performed a he core of he raserizer. Second, each piel should have he same number of samples o simplify hardware design. Addiionally, since our ile ess compue he emporal overlap, ˆ o, i is imporan o be able o quickly find he relevan samples for a ile, i.e., he samples should be ordered in. These requiremens severely resric our opions. For eample, Poisson disk poins are no guaraneed o projec o a good disribuion in wo dimensions, and i may be hard o guaranee a fied number of samples. For hese reasons, we have chosen o work wih sampling disribuions ha are realizaions of digial (, m, s)-nes [Niederreier 99]. Alhough ofen used for quasi-mone Carlo inegraion in offline rendering [Kollig and Keller ], we believe samples based on digial nes are ideal also for moion blur raserizaion due o heir eensive sraificaion properies and ease of consrucion. Ne, we will give a brief inroducion (see Niederreier s work [99] for more deails), and inroduce a novel variaion of a (9) C C C C C C Figure : The righ hree images show eamples of our generaor marices for m=. The firs wo componens are given by shifed and refleced Sierpiňski riangles. These wo marices generae he same poins as C and C, bu permued ino an order ha beer suis our purposes (ordered in ). known mehod for generaing hree-dimensional samples wih good properies. Our samples are ordered in and have a good spaial disribuion when projeced o screen space. Definiion A se of b m s-dimensional poins j=( () j,..., (s) j ) is a (, m, s)-ne in base b if every elemenary inerval of volume b m conains eacly b poins, where b and m are inegers. The elemenary inervals are discree subinervals of space: E = s i= [ ai b, ai + ) [, ) s, () l i b l i where l i and a i < b l i are inegers. The volume consrain gives s i= li = m. An eample in wo dimensions is shown in Figure 5. In our case, s= and we work eclusively wih binary numbers (b=) for efficiency reasons. Lower value (referred o as qualiy ) gives beer sraificaion, i.e., fewer poins per sraum. Hence, we are ineresed in (, m, )-nes in base, which have m poins and eacly one poin per elemenary inerval. This propery ensures ha samples near in ime are spaially far apar, and vice versa, which is imporan o minimize noise. A digial (, m, s)-ne can be defined using a se of generaor marices C,..., C s over a finie field F q, where q is prime [Niederreier 99]. F q consiss of elemens numbered {,..., q }, and all arihmeic operaions are performed modulo q. Since we work in base, q = and he C i marices are binary m m marices. The i h componen of he j h poin is given by: (i) j = (,..., m) C i d(j). d m (j) [, ), () where d k (j) are he bis of j, j {,..., m }, wih d being he leas significan bi. The mari-vecor produc C i (d (j) d m (j)) T is performed in F. Our Mehod Three-dimensional digial nes wih good D projecions are no very well eplored. Grünschloß and Keller [9] propose one mehod based on reordering of he Sobol sequence, where he firs wo dimensions are he Larcher-Pillichshammer (LP) poins [Kollig and Keller ], which have a good spaial disribuion. The firs componen is sequenially ordered, i.e., () j = j. Unforunaely, we have observed ha he projecion ono he oher m wo dimensions is no as well-disribued, ehibiing a srucure of diagonal lines. See Figure 7 (lef). This leads o inferior spaial ani-aliasing, as our algorihm uses he ordered dimension as ime. To address his, we propose a permued consrucion ha is ordered in, while sill projecing o he LP-poins in he remaining wo dimensions, as shown in Figure 7 (righ). Our samples are given by

5 Figure 7: The original samples [Grünschloß and Keller 9] are ordered in he firs componen. The projecion ono he oher wo dimensions are shown on he lef for m=. Afer our permuaion, he non-ime dimensions projec o he Larcher-Pillichshammer poins (righ), which give beer spaial ani-aliasing. he generaor marices (see Appendi A for deails): (( ) ) m C m + l = mod, () m + k k,l= (( ) ) m ( ) C m l = mod and C = k.... k,l= These binary marices are visualized in Figure. The figure compares our marices o he original marices of Grünschloß and Keller, denoed C, C, C. Noe ha our modified marices compue he same se of poins as before, bu he poins are generaed in a differen order. The order is imporan, as our inpu is a sequenial inde in ime, and he remaining wo dimensions are used as he spaial sample posiion. We wan he projecion o screen space o be as good as possible for high-qualiy spaial ani-aliasing. This is especially imporan for saic and slowly moving primiives, where he user ges pleny of ime o sudy he qualiy. Noe ha reordering he poins by permuing he marices is no he same as jus assigning he dimensions differenly. Alhough he marices look deerring, hey make an efficien procedural compuaion of samples possible. Addiion equals XOR in F, so enire columns of he mari-vecor produc in Equaion can be added using single XOR operaions. Addiionally, we omi he lefmos vecor muliplicaion and view he resul as he digis of a fied-poin represenaion. The following C-funcion compues he y-coordinaes of he poin wih sequenial inde j, i.e., sample ime = j/ m [, ), for any m <. All coordinaes are inegers in {,..., m }. void GeXY(uin j, cons uin m, uin&, uin& y) { = y = ; uin c =, c = << (m-); for (j <<= -m; j!= ; j <<= ) { 5 if (j & u<<) { // Add mari columns (XOR) ˆ= c >> ; 7 y ˆ= c; } 9 c ˆ= c << ; // Updae mari columns c ˆ= c >> ; } } The algorihm eamines j one bi a he ime, saring a is high bi m, and adds up columns of C and C. The marices are compued on he fly, using only bi shifs and XOR operaions. In hardware, he above algorihm can be implemened using a very small number of gaes. During raserizaion, we generae samples for ˆ o a he fines hierarchical level in he raversal. For eample, wih piel iles a samples/piel, we have 5 samples, so m = and he samples y-coordinaes are inerpreed as. bis fied-poin numbers (i.e., he op wo bi gives he piel posiion, and he lower si bis he sub-piel placemen). Throughou he paper, we also quanize ime o discree values (see Secion 7), alhough his is no a necessiy. samples/piel samples/piel Sraified Random Digial ne Figure : Comparison beween sraified random sampling (lef) and sampling based on digial nes (righ), a wo differen sampling raes. The regulariy in he digial ne based paern gives a noiceably smooher appearance, while avoiding obvious aliasing. To avoid a repeaing sample paern, we apply random digi scrambling [Kollig and Keller ] o he generaed y-coordinaes, as i gives good resuls a an eremely low cos. Concepually, he sampling domain is hierarchically spli in half along each spaial dimension, and he wo halves randomly permued. The same permuaions are applied o all samples wihin a ile. In base, his operaion can be performed by a biwise XOR beween he ycoordinaes and wo independen random bi vecors. We compue he random vecors based on a hash of he ile posiion, which ensures consisency from frame o frame. The regular srucure of he scrambled digial ne gives low noise wihou any obvious aliasing arifacs. Noe ha random digi scrambling also largely preserves he properies of he projeced samples (Figure 7), which is an imporan aspec. To increase he randomness, a sub-piel jiering can be applied, bu we have no found ha o be necessary. Figure shows he rendering qualiy compared o radiional sraified random sampling, where y and have been independenly sraified per piel. Discussion An alernaive o using procedurally generaed samples is o sore a lookup able of samples, ordered in. This allows for more fleibiliy, bu incurs an addiional hardware cos. Inspired by Grünschloß and Keller [9], we have eperimened wih randomized permuaion-based search for (, m, )-nes wih larger minimum poin disance han he above consrucion, bu he resuls of his has been lef for fuure work. Shading A core feaure of our algorihm is ha we visi a paricular piel a mos once for a cerain riangle. This is similar o McGuire e al. s work [], bu differen from inerleaved and inerval raserizaion [Keller and Heidrich ; Faahalian e al. 9], where a piel may be visied many imes for he same riangle. Using our raversal order, i is herefore possible o use mulisampling ani-aliasing (MSAA) sraegies wih emporal filering [Loviscach 5; McGuire e al. ], which is no feasible for inerleaved and inerval raserizaion since hey slice he ime dimension. As will be seen in Secion, mulisampling can give a considerable reducion in shader cos, which is a big advanage when implemening he raversal algorihm in an eising GPU pipeline. I should be noed ha decoupled shading soluions [Ragan-Kelley e al. ;

6 McGuire e al. Loviscach Saic Small moion Large moion Figure 9: Comparison beween he filer kernel approimaions proposed by McGuire e al. (op) and Loviscach (boom). The resuls are rendered on an NVIDIA GeForce 9 GTX wih anisoropic filering. Noe he severe aliasing in he op row. This is due o over-emphasizing he moion conribuion, and he approimaion of he screen space filer kernel as a fied (, )-vecor. Burns e al. ] show more promise o furher reduce he shading cos. However, he mulisampling approach can be implemened on curren hardware wih no, or very small, modificaions as shown by McGuire e al [], which makes i a good firs sep owards a graphics hardware soluion fully supporing moion blur. Alhough our shading sysem is very similar o previous work, we describe i briefly since being able o use mulisampling is currenly an imporan feaure of our algorihm. We base our emporal shader filering on he work of Loviscach [5] and McGuire e al. []. The basic idea is o use anisoropic eure filering o inegrae eures over he moion fooprin by modifying he derivaives. By inegraing over ime in he shader, i is possible o sample he shader only once per piel, and wrie he resul o all covered samples. We assume ha he only varying shader inpus are he barycenric coordinaes u(, y, ), v(, y, ), and disregard from eplici shader inpu variables represening he sample ime. For eure filering, we need o esimae he eure fooprin. The inegraion domain is given by a fourh order raional funcion in, bu we choose o make he same approimaions as Loviscach [5], and use his approach for perurbing he screen space eure gradien aes o accoun for he emporal derivaive. McGuire e al. [] presen an approimaion where he screen space gradiens use a fied ais in eure space. However, his mehod suffers from severe aliasing for some view direcions, as can be seen in Figure 9. We compue u, u y u and by finie differences (same for he par- ial derivaives of v). For each quad of piels wih a leas one sample covered, we evaluae he shader a five poins: each of he four piel ceners a =.5 for he quad, in order o compue and y derivaives using finie differences, and one addiional poin for one of he samples a = o compue a per-quad approimaion of he emporal derivaive. For a more fine-grained emporal derivaive, one can shade he enire quad a wo disinc imes and compue per-piel emporal derivaives, or shade on a per sample/piel basis, which would shade four poins o compue perpiel u, u u, and derivaives (and he parial derivaives of v). y We use he quad approimaion in all our ess. The shading approach inegraes very well ino eising pinhole camera rendering sysems wih MSAA suppor wih relaively modes modificaions o he hardware. I should be noed ha our approach for compuing derivaives may lead o shading samples ha lie ouside he riangle. This can cause shading arifacs, bu we have no found hem o be significan in our es scenes. McGuire e al. [] used a differen approach and picked he las covered ime sample as he shading sample, bu reprojeced o =. This has oher implicaions such as overblurring due o oo large filer kernels. Working ou a sraegy for correc shader filering and derivaive compuaions wih sochasic sampling is an ineresing problem ha deserves a horough evaluaion. However, we leave ha for fuure work. Wih inerleaved raserizaion [Keller and Heidrich ; Faahalian e al. 9] in a pipeline wih shading afer visibiliy, here are no adjacen screen space samples wih he same sample ime. Wihou inroducing large hardware changes (e.g., a shader cache) he spaial derivaives can be compued in wo ways. One opion is o compue derivaives per sample by eecuing he shader a hree spaial posiions. The oher opion is o compue derivaives using finie differences from four nearby samples wih he same ime, while aking he perurbed sample posiions ino accoun. This mehod produces coarser derivaives, as he samples wih he same ime are ypically separaed by wo or more piels. In eiher case, as we shade a all sample imes, he emporal derivaives are less imporan and can be ignored wihou much loss in image qualiy. We chose o use he firs alernaive for he saisics presened in his paper, as he qualiy more closely maches ha of our derivaives. Furhermore, coarse derivaives will degrade he qualiy even for saic scenes compared o curren graphics API specificaions, which we wan o avoid. 7 Implemenaion Hierarchical Raserizers To evaluae our algorihm, we have implemened four raserizers called CONVEX, OUR, INTERVAL and HIERAR- CHICAL INTERLEAVE in a sofware raserizaion pipeline ha can eecue DX races, bu lacks a general shading sysem. In order o evaluae shading and sampling qualiy, we have also designed a GPU raserizer which performs sochasic raserizaion in a piel shader, similar o McGuire e al. s work []. CONVEX (based on McGuire e al. []) raverses all iles wihin an AABB overlapping he screen space conve hull (CH) of he moving riangle. The original paper uses a wo-sep algorihm ha riangulaes he conve hull and raserizes sochasically in he piel shader (o run on curren GPUs). In our sofware implemenaion, each ile is esed agains he CH edges, and if i overlaps, all spaioemporal samples wihin he ile are esed agains he riangle. Boh approaches perform hierarchical raserizaion, bu he laer may be more efficien in hardware, as a ile is visied only once. OUR algorihm is similar, bu uses he ile ess wih emporal bounds inroduced in Secion insead of he CH edges. The emporal bounds significanly reduce he number of samples ha mus be esed. IN- TERVAL is based on Piar s moion blur algorihm (described by Faahalian e al. [9]). The main difference compared o CONVEX and OUR is he ieraion order, where INTERVAL has an ouer loop over sample imes insead of over iles. This is he only algorihm ha canno be easily eended o hierarchical raserizaion. We sill use a iled raversal approach for he sake of hierarchical deph culling, bu we have no es for rivially rejecing a non-overlapping ile. HI- ERARCHICAL INTERLEAVE is a hierarchical DH version of Faahalian e al. s [9] inerleaved raserizer, where he riangle is raserized wih an inerleaved sampling paern. Like INTERVAL, his algorihm has he ouer loop over sample imes raher han iles. Noe ha for mied riangle sizes, adding a hierarchical es o Faahalian e al. s [9] inerleaved raserizer improves performance a lo, which is o be epeced since he arge of he original paper was micropolygon rendering.

7 O UR Tiles Bound ri over [, ] Swep ri o MSAA Ordered in Ieraion order Screen space bbo Tile es Occl. & sample es Shading Sampling paern I NTERVAL Sample imes Bound ri over [i, i+k ] None = [i, i+k ] Supersampling Ordered in H IER.I NTERLEAVE Sample imes Bound ri a i Tri edges a i = i i Supersampling Inerleaved Scene SoneGian SoneGian Heaven SubD Triangles.M.M.M.M Resoluion Moion Camera ranslaion Camera roaion Camera ranslaion Keyframed animaion SoneGian C ONVEX Tiles BBo of CH CH edges [, ] MSAA Arbirary SoneGian Ieraion order Screen space bbo Tile es Occl. & sample es Shading Sampling paern Heaven Table : Algorihm comparison. CH denoes he screen space conve hull of he moving riangle. For I NTERVAL, we divide he N unique sample imes in N/k inervals. All algorihms perform backface culling [Munkberg and AkenineMo ller ] and view-frusum culling (including emporal bounds). In all ess, we use he same high-qualiy inerleaved sampling paerns, described in Secion 5, wih fied imes. The reason for his is ha he edge funcions can be pre-compued for imes in he riangle seup and reused over he riangle, which gives subsanially reduced coss for our es scenes. The inside es, which uses DH edge equaions, is herefore idenical for all four algorihms. The raversal sraegies are summarized in Table. Using a sample paern wih a fied se of sample imes, here is an amoun of moion where a fas moving riangle has no spaial overlap beween adjacen imes, i and i+. Figure illusraes his. In his case, here is lile emporal coherence o eploi, i.e., each ile has only one or a few covered samples. Therefore, we propose a fallback o H IERARCHICAL I NTERLEAVE raversal whenever he individual bounding boes no longer overlap. We use a simple heurisic o decide when his occurs. The dimensions of he bounding boes a = and = are w h and w h, respecively, and he swep bounding bo ws hs. If we raserize a N unique imes, H IERARCHICAL I NTERLEAVE raversal is chosen whenever min(w, w ) < ws /N or min(h, h ) < hs /N. We use his fallback in all our measuremens of O UR, C ONVEX, and I NTERVAL. I is primarily acivaed for very small, fas moving riangles. A mulisampling, we use hree hierarchical levels wih,, and piel iles for all algorihms ecep H IERARCHICAL I NTERLEAVE (which uses, and piel iles). Deph culling is performed on he coarses and fines levels, and rivial rejec or ime overlap es are performed on he wo finer levels. These configuraions were deermined by eensive evaluaion of boh arihmeic cos and bandwidh usage. We use coarser ile levels for H IERARCHICAL I NTERLEAVE since he ile ess are eecued for each sample ime i. For H IERARCHICAL I NTERLEAVE, a single ile es a piel iles culls a mos samples wih = i. In conras, up o 5 samples wih [, ] can be culled for piel iles wih our ile ess. Frame 5 Frame Frame 95 SubD Figure : Two moving riangles are raserized a four fied imes for illusraive purposes. Wih lile moion, he individual bounding boes (red) overlap, which makes a screen space raversal in he swep bbo (green) preferable. A higher moion, he bounding boes separae and H IERARCHICAL I NTERLEAVE raversal is more efficien. Moion: Figure : Our es scenes wih wo frames from he SoneGian demo (couresy of BiSquid), one from he Heaven demo (couresy of Unigine Corp.), and he SubD animaion from Microsof DX SDK (June ). Moion blur has been added o all scenes. A radiional z-ma buffer sores one conservaive z-ma value for each screen space ile. We use a ime-dependen z-ma buffer [Akenine-Mo ller e al. 7; Boulos e al. ], which conains muliple emporal deph values per ile for increased culling efficiency in he presence of moion blur. For beer efficiency, our algorihm uses he ile s emporal overlap, o (Secion ), o avoid performing occlusion queries for ime inervals when he riangle does no overlap. Our simulaor uses a fully associaive cache backing he deph and z-ma buffers wih byes cache lines. In he following, we denoe he number of differen sample imes represened in one cache line as s. A corresponding z-ma value hen represen s imes over a screen space ile whose spaial eens is proporional o /s o make i fi in he cache line. A coarse z-ma buffer is eiher consruced for each of he imes, and hence s =, or for a group of consecuive imes (represening a smaller screen space area), where s >. In all cases, we have one z-ma value for each cache line of sample dephs. Time-Dependen Occlusion Culling The emporal coherence may be furher eploied by consrucing a emporal hierarchy [Boulos e al. ]. We eplored his wih a wo-level spaio-emporal cache-backed hierarchy, bu were no able o reduce bandwidh usage. This is parly due o he cache line grouping of values large chunks of geomery mus be successfully culled in order o avoid reading z-ma daa and parly due o he added cos of keeping anoher level of z-ma daa in he cache. Also noe ha our ess are no using any deph compression. However, a wo-level emporal hierarchy decreased he arihmeic cos by around 5 %, and we use i for all algorihms. Resuls Our es scenes are presened in Figure and include various ypes of moion, riangles sizes, and geomery disribuions. All resuls were generaed using our own simulaion framework de-

8 Bandwidh (GB)..5. Heaven kb cache SoneGian kb cache Heaven, moion SoneGian, moion.75.5 Our s= Hierarchical Inerleave s= Our s= Hierarchical Inerleave s=.5..5 Our s= Hierarchical Inerleave s= Our s= Hierarchical Inerleave s= Moion Amoun.5 Moion Amoun 5 Cache size (kb).5 5 Cache size (kb) Figure : Bandwidh usage for z and z-ma a samples per piel. Lef: varying amoun of moion wih a kb deph cache. A higher emporal z-ma resoluion (lower s) scales beer wih increasing moion, bu has a higher consan cos. s = is a suiable choice for OUR, as he crossover occurs a ereme moion (ouside he graph). Righ: varying cache size wih fied amoun of moion. The minimum cache requiremen for our algorihm is o accommodae all N ime layers and he z-ma sorage. Our algorihm wih s = scales well o decenly small cache sizes. CONVEX is no shown as i behaves similar o our algorihm, wih he only difference ha more z-ma queries are performed. scribed in Secion 7 wih samples per piel. We presen resuls for he deph buffer bandwidh, he number of shader eecuions, and he number of arihmeic operaions required for raserizaion. In mos chars, we have also included a sandard hierarchical raserizer wihou moion blur. This is referred o as STATIC. Deph Buffer Bandwidh Reducing memory bandwidh usage is incredibly imporan, and herefore, we sar wih a sudy on deph buffer bandwidh usage. Figure (lef) shows he deph buffer memory bandwidh usage from cache misses (including boh sample dephs and z-ma values) when he number of sample imes per cache line, s, is varied. Grouping more imes ino a cache line increases he penaly of larger moion, bu lowers he bandwidh usage for pars of he scene moving slowly. For our algorihm, s = is preferable for a wide range of moion. We use his number for all measuremens for OUR, CONVEX, and INTERVAL since hey have similar access paerns. Noe, however, ha our algorihm performs slighly fewer hierarchical occlusion queries han CONVEX since we use he resuls from he ile es o avoid esing some ime inervals. More aggressive emporal grouping (s = ) only pays off for very small moion, while no emporal grouping (s = ) is more efficien for ereme moion. HIERARCHICAL INTERLEAVE scales differenly, and he benefi for grouping sample imes ino he same cache line is low even for saic scenes. This is an effec of he raversal order, where he riangle is fully raversed for one sample ime before coninuing wih he ne. For larger riangles, his may lead o evicion of he firs cache line before saring raversal for he ne sample ime. Therefore, we use he s = framebuffer layou for he HIERARCHICAL INTERLEAVE algorihm. To deermine a suiable cache size, we ran he bandwidh measuremens wih various cache sizes as presened in Figure (righ). Our algorihm needs a minimum cache size of kb (and in he case of s =, i needs kb) o avoid a % miss rae. However, above his minimum, i scales beer han HIERARCHICAL INTERLEAVE raversal for decreasing cache sizes. In fac, HIERARCHICAL INTERLEAVE does no level ou unil he cache becomes very large (kb MB). For he remaining comparisons, we use a kb cache for all algorihms. This corresponds o approimaely 5 fully covered piels worh of daa, no couning z-ma sorage. As shown in Figure and (op row), he deph buffer bandwidh (including boh sample dephs and z-ma values) o eernal memory is relaively consan for he HIERARCHICAL INTERLEAVE raversal order wih increasing moion, while i is increasing somewha for OUR, CONVEX, and INTERVAL. Our algorihm becomes less efficien han HIERARCHICAL INTERLEAVE a some poin. However, ereme moion is needed o reach he break-even poin, and our algorihm consisenly ouperforms he compeing algorihms ecep for he difficul SoneGian scene wih ereme moion (> ). In Figure, we see ha our algorihm uses less bandwidh han HIER- ARCHICAL INTERLEAVE for he SubD animaion. In fac, in mos frames, OUR uses only abou 5% of ha of HIERARCHICAL INTER- LEAVE, even hough he scene is animaed a a low frame rae ( fps) and conains frames wih relaively large moion. Our algorihm also uses slighly less bandwidh han CONVEX since our ile ess wih emporal bounds enable more efficien occlusion culling. This is paricularly noiceable in he frames wih larges moion. The deph buffer bandwidh of INTERVAL is similar o our algorihm bu wih one imporan difference. Since INTERVAL does no have a rivial rejec es we need o perform deph culling for all iles overlapping he riangle bounding bo. This is no significan for small riangles or for scenes wih only moion in he - or y-direcions. However, for SubD which conains large sliver riangles, his leads o many unnecessary deph culling queries and increases bandwidh significanly. Shading Efficiency Figure shows he number of shader eecuions per frame for he SubD animaion. For OUR and CONVEX, we use mulisampling, while HIERARCHICAL INTERLEAVE raversal has o resor o supersampling due o he raversal order (i.e., for each ime, here is only one sample per ile in he inerleaved sampling paern). For INTERVAL, we use ime inervals. This implies ha wihin each ime inerval, here is one sample per piel in he inerleaved sampling paern. As can be seen, he benefi of mulisampling is very high for SubD. Also, INTERVAL is more efficien han HIERARCHICAL INTERLEAVE since shading is compued once per inerval ( ) raher han per sample ime ( ), and quad-fragmen shading and finie differences can be used. A ereme moion relaive o he primiive size, we effecively rever o supersampling. This happens a > moion for he Sone- Gian scene, which is highly essellaed (see he middle row in Figure ). I is quesionable wheher his ereme moion will be usable for real-ime rendering (> fps). In all cases, he shading overhead compared o STATIC is ofen quie high. Given hese observaions, we conclude ha beer long erm soluions for shading may be cache or objec-space based approaches, as discussed in Secion. However, we believe ha a mulisampling approach for moion blur is likely o be adoped by hardware vendors as an inermediae sep owards a pipeline wih decoupled shading. Raserizaion Cos We have insrumened our code wih cos esimaions for he criical sages of he raserizaion algorihm: riangle seup, ile es, sample es, and inerpolaion seup. We only accoun for he cos of he more comple operaions such as ADD, MUL, RCP, ec., and disregard from he cos of bi-widdling and conrol logic. Therefore, our resuls should no be seen as absolue coss for an implemenaion, bu raher demonsrae he general

9 Deph BW (GB) Shader E. (M).5 Our Hier. Inerleave. Conve Inerval.9 Saic.. Frame Figure : Deph buffer bandwidh for he SubD animaion. 5 9 Our Hier. Inerleave Conve Inerval Saic Frame Figure : Number of shader eecuions for SubD. rend of he algorihms and how hey relae. We inenionally do no use sample es efficiency [Faahalian e al. 9] as an efficiency measure, since i only includes a par of he raserizaion cos. For eample, sample es efficiency would benefi of using as small iles as possible, bu in pracice, he bes radeoff is found wih medium sized iles where he sum of he ile es cos and sample es cos is minimized. In Figure 5, we show a breakdown of he coss for he differen sages of he raserizer for he SubD animaion. HIERARCHICAL INTERLEAVE has a significan riangle seup cos, due o ha each riangle is bounded a N = discree imes. Also, he inerpolaion seup is more epensive, as he shading is supersampled. Addiionally, he sample es work is increased due o a larger screen space ile size. We have eperimened wih finer ile sizes, bu ha resuled in a subsanial increase in he ile es cos, making he overall cos increase. INTERVAL performs quie poorly for his es scene. The reason for his is ha he model conains large sliver riangles and since he INTERVAL raserizer lacks a hierarchical ileoverlap es, i performs many unnecessary sample ess. For CON- VEX, he cos of sample esing dominaes, and varies widely. Our algorihm has more epensive per-ile ess, bu hey manage o cull a larger par of subsequen work. The oal arihmeic cos using our wo ile ess (moving bo and moving edge) is presened in Table. We also measured he efficiency of our linearized edge es compared o direcly solving he quadraic edge equaion per ile. On he enire SubD animaion (he scene wih mos comple moion), he linearized es resuls in % more inside es, bu reduces he oal arihmeic cos by % hanks o less epensive per-ile compuaions. On he Heaven scene, he linearized es resuls in % more inside es, bu a oal cos reducion of %. Saic BBo Bo Edge Bo+Edge SubD.5(5).().(.9).7(.5) 9 Heaven () 7() 7() () 9 Table : Efficiency of our ile ess in erms of average oal arihmeic cos per frame (maimum in parenhesis). The combinaion of he wo ess gives a cos efficien and robus ile es. In Figure (boom row), we show how he four differen moion blur algorihms scale wih increased moion. SoneGian and Heaven boh have moion blur from a camera ranslaion and show similar rends despie a large difference in essellaion. For modes moion (< ), our algorihm has abou half he raversal cos of HIERARCHICAL INTERLEAVE, and has roughly he same cos a ereme moion. INTERVAL scales similar o our algorihm bu has lower overall performance. CONVEX is efficien for small moion, bu he arihmeic cos grows very quickly for larger moion. SoneGian is a highly essellaed frame wih a camera roaion, so ha each riangle ges similarly long moion rails. There are abou a million riangles in he wo fron-mos spiders alone. This is a wors-case scenario for a screen space hierarchical raversal, as a large fracion of he scene geomery is near piel-sized. Here, HIERARCHICAL INTERLEAVE handles ereme moion robusly a a significan cos (around Gops per frame). Our approach is compeiive up o abou moion. A ereme moion levels, nearly all riangles are compleely separaed (see Figure ), so here is no gain in using a screen space raversal algorihm. 9 Conclusions This paper has described he firs efficien algorihm for hierarchical raserizaion of moion blur. The algorihm builds on novel ile ess ha compue he emporal overlap beween a screen space ile and a moving primiive. We have shown ha hese emporal bounds are imporan o reduce he volume of esed samples and enable efficien hierarchical occlusion culling. We have furher devised a high qualiy sampling mehod ha uses he emporal bounds o quickly generae samples wih good saisical properies. Our mehod is based on reordering of a known hree-dimensional digial (, m, s)-ne o beer fi he requiremens for moion blur raserizaion. Finally, we have provided eensive measuremens of deph buffer bandwidh usage, arihmeic inensiy, and shader efficiency for MSAA on modern comple workloads, which we have no seen in oher sudies. In his paper, we have aken one sep owards efficien moion blurred rendering on graphics processors. Our focus has been on a raher non-inrusive change o curren GPUs. One area ha needs more work is efficien shading, which makes he ne naural sep o add a decoupled shading cache [Ragan-Kelley e al. ; Burns e al. ]. This is lef for fuure work a his poin. In addiion, i would be ineresing o ransform he algorihms ino using efficien fied-poin mah robusly. We hope ha our work will help drive a coninued ineres in sochasic raserizaion as a realisic mehod o achieve high-qualiy moion blur in fuure graphics pipelines. Acknowledgemens Thanks o Tobias Persson from BiSquid for leing us use he SoneGian demo, and o Denis Shergin from Unigine for leing us use images from Heaven.. Tomas Akenine-Möller is a Royal Swedish Academy of Sciences Research Fellow suppored by a gran from he Knu and Alice Wallenberg Foundaion. In addiion, we acknowledge suppor from he Swedish Foundaion for sraegic research. References AKENINE-MÖLLER, T., AND AILA, T. 5. Conservaive and Tiled Raserizaion Using a Modified Triangle Se-Up. Journal of Graphics Tools,,,. AKENINE-MÖLLER, T., AND STRÖM, J.. Graphics for he Masses: A Hardware Raserizaion Archiecure for Mobile Phones. ACM Transacions on Graphics,,,. AKENINE-MÖLLER, T., MUNKBERG, J., AND HASSELGREN, J. 7. Sochasic Raserizaion using Time-Coninuous Triangles. In Graphics Hardware, 7. BOULOS, S., LUONG, E., FATAHALIAN, K., MORETON, H., AND HANRAHAN, P.. Space-Time Hierarchical Occlu-