X-Ray Renderng CSE 564: Vsualzaton Other Renderng Technques: Ponts, Splats, Shear-Warp Estmate ray ntegral va dscrete raycastng: mage plane nterpolaton kernel h pxel p Klaus Mueller pont v Complete dscrete ray ntegral: p = v h( X ( s ) X ( v )) k k Stony Brook Unversty Computer Scence Center for Vsual Computng s k sample s k = v h( X ( sk ) X ( v )) Reversng the order of and k: p = v h( X ( s ) X ( v )) k k 2 X-Ray Renderng Estmate the ray ntegral va pont proecton: pont v nterpolaton kernel h mage plane pxel p Footprnt (splat) of pont v : ext h ( r ) = h( r, s) ds ext. 2. X-Ray Pont Splattng Example: proectng a volume of two ponts 3. 4. rasterze footprnt ray r Compute contnuous ray ntegral at p : = v p h( r ) 3 rasterze footprnt add footprnts 4
X-Ray Pont Splattng Pont Proecton Re-orderng was frst recognzed by Hanson and Wecksung for 2D CT (Hanson 85) Later ndependently dscovered by Westover for 3D volume renderng (Westover 89) Facltates computaton of the true ray ntegral not ust a dscrete Remann sum (raycastng) Pre-ntegrated footprnt s stored nto a table Need a kernel functon for whch mappngs nto the footprnt table can be defned for any orentaton The Gaussan s such a functon 5 Each pont s represented by a 3D Gaussan G V : G V T 0.5( x v ) V ( x v ) e 0.5 v = 2 π G V V G V s an ellpsod to facltate more general grds It s a sphere for cubc grds A vewng matrx M transforms G V nto G MVM : G G ( u Mv T ) MVM M MVM = T (Heckbert 89, Zwcker 0) G MVM Mv +T 6 Pont Proecton Blendng Proecton P of G MVM s screen ellpse P(G MVM ) Fnd v s screen proecton P(M v + T) Fnd lnear mappng of P(G MVM ) nto footprnt table Rasterze footprnt table under P(G MVM ) at P(V v ) P(M v + T) P(G MVM ) mappng ellpse crcle Footprnt table (crcular Gaussan) 7 Note: Gaussan kernels do not blend perfectly A small rpple always remans: Typcal range: (0.99845,.00249) (assumng a functon of unty) The wder the Gaussans, the smaller the rpple In practce, a radus = 2.0 n volume space works well (gven the approprate Gaussan) See (Crawfs and Max, Vs 93) for an optmzed kernel 8
Complexty Splattng seemngly reduces the nterpolaton complexty by one dmenson: Raycastng: nterpolaton of samples n 3D Splattng: rasterzaton of footprnts n 2D But Consder magnfcaton = Raycastng: Complexty Commonly uses trlnear nterpolaton Requres 8 ponts to calculate one ray sample pont Total complexty: O(8 n 3 ) Splattng: Uses Gaussan kernel of radus=2 Footprnt rasterzaton touches 6 pxels Total complexty: O(6 n 3 ) 9 0 Complexty Storage Complexty Does ths mean that raycastng s more effcent than splattng? It depends. Spatally ntrcate obects are good canddates for pont-based renderng (splattng) But the smplcty of splattng has advantages even for less favorable obects Generally, only need to store relevant ponts Non-ar ponts, masked-out ponts, ROI-ponts Provdes easy space-leapng for rregular obects Storage schemes (n ncreasng order of spatal coherence): Lst of ponts, sorted by value (fast so-contourng) RLE lst of ponts (fast transformatons and sparse) Octree wth herarchcal bns of ponts RLE: #E #Fv v 2 v 3 v 4 #E #Fv v 2 #E #Fv v 2 v 3 #E #Fv v 2 v 3 2
Renderng Alasng RLE lst facltates fast ncremental arthmetc for pont proecton n software Texture mappng hardware can also be used Texture map footprnt onto a square polygon Set GL blendng functons, etc. Warp polygon accordng to pont s screen space ellpse Algn the warped polygon wth the screen Proect polygon to the screen 3 In perspectve or at low magnfcatons, some volume portons may be sampled below Nyqust screen oversampled volume grd Ray grd samplng rate > volume grd samplng rate no alasng undersampled volume grd Ray grd samplng rate volume grd samplng rate alasng 4 Alasng Ant-Alasng Effects of alasng Adapt kernel bandwdth for proper ant-alasng Amounts to a stretch of the 3D kernel screen checkerboard tunnel terran 5 constant splat kernel sze (Swan 97, Mueller 98) lnearly ncrease splat kernel sze lnearly decrease splat ampltude 6
Ant-Alasng Convenently done n perspectve (ray-) space Ant-Alasng Compute the Gaussan ellpsod n ray space Calculate the Jacoban J of the local perspectve dstorton (vares for each pont) Compute the ray space ellpsod G JMV usng J screen camera space screen ray space 7 camera space ray space (Zwcker 0) G G MV JMV = = J M G G T MVM T JMVJ ( u Mv ( x x k ) T ) center of Gaussan n ray space generalzed Gaussan ellpsod n camera space 8 Ant-Alasng - Results Compostng - Raycastng Reconstructon followed by compostng nterpolaton kernel h mage plane pxel p Compostng: alased ant-alased 9 Reconstructon of c(s k ), α(s k ) c = C( s k α = α( s ) α( s k k ) ( α) + c ) ( α) + α 20
Compostng - Splattng Compostng Reconstructon not separable from compostng pont v C(v ), α(v ) ext ext nterpolaton kernel h ray r h ( r ) = h( r, s) ds c = C( v α = α( v ) h ( r ) α( v mage plane pxel p ) h ( r ) ( α) + α ) h ( r ) ( α) + c compostng of pre-ntegrated kernel 2 Two strateges devsed by Westover (Westover 89, 90) Composte every pont: Shown n prevous slde Fast and smple Leads to sparklng n anmated vewng Axs-algned sheet-buffers: Add splats wthn sheets most parallel to mage plane Composte these sheets n depth-order Leads to poppng artfacts n anmated vewng 22 Axs-Algned Sheet-Buffers Image-Algned Sheet-Buffers volume slces mage plane at 30 z Swtch compostng axs at 45 y x Poppng occurs: volume slces mage plane at 70 Elmnates poppng Slcng slab cuts kernels nto sectons Kernel sectons are added nto sheet-buffer Sheet-buffers are composted bnary cube 44.7 45.2 23 sheet buffer mage plane compostng buffer bnary cube (Mueller 98) 24
Image-Algned Sheet-Buffers Footprnt mappng as usual Requres multple footprnt rasterzatons per pont Pre-Classfed Splattng normal axs-algned mage-algned blurred close-ups 25 26 Pre-Classfed Splattng One Soluton: Edge Splats Orgnal edge Edge splats (Huang 98) replace normal splat by specal edge splat Sampled edge Classfcaton and shadng Splatted wth Gaussan kernel Reconstructon: blurred edge mage Shortcomngs: pre-processng requred problems wth dscontnutes mcro-edges are hard to resolve 27 28
Pre-Classfed Renderng Post-Classfed Renderng (Mueller 99) Renderng Loop Renderng Loop Color and opacty volume Splat nto sheet-buffer Composte sheet-buffer Splat nto sheet-buffer Classfy and shade Composte sheet-buffer Classfy and shade Advance sheet-buffer Advance sheet-buffer Raw densty volume Vewng parameters Image Raw densty volume Vewng parameters Image 29 Note: ths can only be done wth mage-algned sheet buffers 30 Post-Classfed Splattng Post-Classfed Splattng Orgnal edge Sampled edge Splatted wth Gaussan kernel pre-shaded post-shaded, central dfference post-shaded, gradent splats Reconstructon: blurred edge Classfcaton: crsp edge mage 3 Sheet buffers current current+ current current- current 32
Post-Classfed Splattng Perspectve Pre-Warp Revsted pre-shaded Camera space post-shaded Ray space 33 34 EWA Volume Splattng Analyss Formally separates volume space (pre-) flterng from screen space (post-) flterng Warped reconstructon kernel Low-pass flter Resamplng flter Mnfcaton Proecton Low-Pass Flter W Convoluton x k r Volume EWA (Ellptcal Weghted Average) ρ k volume resamplng flter r 0 Magnfcaton Zwcker et al. 35 36
Effects From Volume To Surface Reconstructon flter only EWA flter Iso-surface of a volumetrc pont-based obect s represented by a hull of Gaussan kernels Flattenng the ponts n drecton of the surface normals yelds a more exact representaton In the lmt get a surface composed of 2D Gaussans (aka surface ponts or surfels) Flattened ponts Low-pass flter only EWA flter Volumetrc ponts 37 Iso-contour 38 Volumetrc vs. Surface Ponts Volumetrc ponts: Most often on a regular lattce Represent both boundary and nteror Overlappng ponts reconstruct volumetrc obect Dfferent so-surfaces, shapes, and compostons can be produced va transfer functons on the fly Surface ponts: Irregular dstrbuton (on the surface) Usually located only on boundares Algorthms: Surfaces Surfels and Surface Splattng (Zwcker et al.) QSplat (Rusnkewcz et al.) Man pont ponts can represent fne detal better than trangles 39 40
Prmtves: Surfels and QSplats Algorthms: Volumes Traverse pont herarchy from top to bottom For each block of ponts, fnd level where the local resoluton of the proected pont set matches the screen resoluton (oversample for better qualty) Splat the selected ponts nto the z-buffer Blend vsble ponts, fll holes, shade hole reconstr. flter Rymon-Lpnsk et al., Trcoche et al. (Vs 04) Vega Hguera (Vs 05) Emphass: so-surface renderng (obvously) fast data traversal and pont localzaton (use span-trangle) How about mxng surface ponts and volume ponts? from Zwcker 0 4 42 Alternatve Grds Alternatve Grds cubc Cartesan (CC) hexagonal grd (3.4% less samples) spatal 3 doman 2 / Fourer transform replca Fourer transform man spectrum frequency doman 43 Body-centered cartesan (BCC) grd: Reduces # of requred pont samples to 70.3% cubc body-centered 6 / 2 4D BCC grd requres only 50% of the equvalent 4D cubc grd samples 2 2 2 44
Alternatve Grds Alternatve Grds Notes: BCC grds assume sphercally bandlmted sgnal Under that assumpton compresson s lossless Renderng (Theussl 0): All usual pont renderng methods are applcable Need to shft slces by / 2 Turbulent Jet 4D CC 99 tme steps (68M) Relevant voxels: 9.4M Turbulent Jet 4D BCC 38 tme steps (87M) Relevant voxels: 7.4M 3D extracted: 27k 3D extracted: 07k Sze RLE lst: 46k Sze RLE lst: 46k CC BCC BCC 45 Render tme:.23s Render tme:.0s (7%) (Neophytou 02) 46 Alternatve Grds Detal Modelng Anmatons of tme-varyng datasets: turbulent et Footprnts do not have to serve nterpolaton alone (va the pre-ntegrated kernel functon) They can be used to add addtonal detal or nformaton between the sample ponts The Gaussan footprnt provdes the blendng turbulent flow vector feld splat 47 (Crawfs/Max 93) 48
M vew = M shear M warp M vew = M shear M warp shear 49 50 M vew = M shear M warp M vew = M shear M warp baseplane mage shear warp shear warp screen mage 5 52
Cheap blnear nterpolaton wthn slce Keep baseplane resoluton at volume resoluton Allows re-use of pre-computed blnear weghts Each voxel stores ndex nto reflectance map For each slce: Compute blnear weghts For each voxel square: Shade voxels usng the reflectance cube Interpolate shaded voxels and composte result 53 54 RLE encodng of voxel and pxel runs skp over transparent voxels quckly skp over opaque pxels quckly RLE encodng of voxel and pxel runs skp over transparent voxels quckly skp over opaque pxel quckly transparent opaque transparent opaque work 55 56
RLE encodng of voxel and pxel runs skp over transparent voxels quckly skp over opaque pxel quckly RLE encodng of voxel and pxel runs skp over transparent voxels quckly skp over opaque pxel quckly transparent opaque transparent opaque skp skp 57 58 RLE encodng of voxel and pxel runs skp over transparent voxels quckly skp over opaque pxel quckly RLE encodng of voxel and pxel runs skp over transparent voxels quckly skp over opaque pxel quckly transparent opaque transparent opaque work work skp work 59 60
Warpng the baseplane mage nto screen mage s a cheap 2D operaton warp Sources of Speed Cheap blnear nterpolaton Use of pre-computed (per slce) blnear weghts Fxed number of rays Zoomng s deferred to 2D warp step Very effcent occluson cullng 6 62 Drawbacks Renderng Results Ray sample dstance s vew dependent Slce artfacts may appear at oblque vew angles For zooms the baseplane mage s magnfed Causes blur 3 Interpolaton of shaded voxels (pre-shadng) Also causes blur 3 axs-algned RLE lsts are used Memory consumpton s trpled Poppng may occur on lst change 2 Standard (0.30s) Engne (28 3 ) nterm. slce (0.40s) + post-class (2.9s) + matched sampl. (3.3s) 63 64
Engne (enlarged detal) Renderng Results standard one ntermedate slce + matched samplng + post-class 65