NVIDIA Case Studies: OptiX & Image Space Photon Mapping David Luebke NVIDIA Research Beyond Programmable Shading 0
How Far Beyond? The continuum Beyond Programmable Shading Just programmable shading: DX, OGL Beyond Programmable Shading 1
How Far Beyond? The continuum Beyond Programmable Shading Just programmable shading: DX, OGL Pure compute-based graphics: CUDA, OptiX Beyond Programmable Shading 2
Ray Tracing SIGGRAPH 2008 Demo SIGGRAPH 2009 Product: OptiX Beyond Programmable Shading 3
The OptiX Engine A General Purpose Ray Tracing API Rendering, baking, collision detection, A.I. queries, etc. Modern shader-centric, stateless and bindless design Is not a renderer but can implement many types of renderers Highly Programmable Shading with arbitrary ray payloads Ray generation/framebuffer operations (cameras, data unpacking, etc.) Programmable intersection (triangles, NURBS, implicits ) Easy to Program Write single ray code (no exposed ray packets) No need to rewrite shaders to target different hardware Beyond Programmable Shading 4
Programmable Operations Hit program Miss program Intersection program Ray generation program Beyond Programmable Shading 5
Programmable Operations Rasterization Fragment Vertex Geometry Ray Tracing Closest Hit Any Hit Intersection Selector Ray Generation Miss Exception Beyond Programmable Shading 6
Shading in OptiX Interconnection of shaders defines the outcome Whitted ray tracing, cook, path tracing, photon mapping Or collision detection, sound propagation, Shading language is based on C/C++ for CUDA No new language to learn Powerful language features available immediately Pointers Templates Overloading Default arguments Classes (no virtual functions) Adds a powerful object model designed for ray tracing Caveat: still need to use it responsibly to get perf Beyond Programmable Shading 7
How Far Beyond? The continuum Beyond Programmable Shading Interesting middle ground! Just programmable shading: DX, OGL Pure compute-based graphics: CUDA, OptiX Beyond Programmable Shading 8
How Far Beyond? Image Space Photon Mapping Interesting middle ground! Just programmable shading: DX, OGL Pure compute-based graphics: CUDA, OptiX Beyond Programmable Shading 9
Hardware-Accelerated Global Illumination by Image Space Photon Mapping Morgan McGuire Williams College David Luebke NVIDIA Corporation ACM Hardware-accelerated SIGGRAPH / EuroGraphics global illumination High-Performance by image space Graphics photon mapping 2009
Goal: Dynamic Global Illumination a 1920x1080, 20fps Beyond Programmable Shading 11
Multi-Bounce Video Beyond Programmable Shading 12
Beyond Programmable Shading 13
Photon Mapping Time (seconds) e.g., [Jensen 01, Purcell et al. 03, Zhou et al. 08] 1 st Bounce 2 nd 3 rd 4 th Build Tree Last Bounce ( Estimate Radiance ) Image Space Photon Mapping (milliseconds) GPU Bounce Map CPU Trace GPU Photon Volumes Data Transfer Beyond Programmable Shading 14
System CPU Trace 1 st Bounce: Bounce Map Direct + Shadows Last Bounce: Photon Volumes Hardware-accelerated global illumination by image space photon mapping
Bounce Map Position Normal Material Parameters (BSDF) Outgoing Power Outgoing Direction Refractive Index, A Priori Differential Probability Beyond Programmable Shading 16
(Trace Rays) Beyond Programmable Shading 17
(Direct Illumination) Beyond Programmable Shading 18
Radiance Estimate Traditional photon mapping: gather Per pixel k-nn search in k-d tree World-space (3D) Image-space photon mapping: scatter Per photon Hardware rasterization using photon volumes Image space (2D) Beyond Programmable Shading 19
Photon Volumes photon Invoke an illumination contribution on all pixels for which a photon might be a valid estimate of incident radiance Not virtual point lights (a.k.a. instant radiosity) Not 2D splatting Hardware-accelerated global illumination by image space photon mapping
Hardware-accelerated global illumination by image space photon mapping 21
Hardware-accelerated global illumination by image space photon mapping 22
Hardware-accelerated global illumination by image space photon mapping 23
Results Beyond Programmable Shading 24
Direct Hardware-accelerated global illumination by image space photon mapping
Hardware-accelerated global illumination by image space photon mapping
Indirect Hardware-accelerated global illumination by image space photon mapping
Direct + Indirect Hardware-accelerated global illumination by image space photon mapping
Direct + Ambient Hardware-accelerated global illumination by image space photon mapping
Direct + Indirect Hardware-accelerated global illumination by image space photon mapping
Performance @ 1920 x 1080, 30 lights Exact PLS: 21 fps 31 fps 23 fps Conservative PLS: 15 fps 5 fps No PLS: 2 fps Hardware-accelerated global illumination by image space photon mapping
Subsampling Joint-bilateral upsampling on normal and depth Subsample incident radiance, not pixel color (no diffuse surface assumption) Beyond Programmable Shading 32
Diffuse Interreflection Video Beyond Programmable Shading 33
Videos Hardware-accelerated global illumination by image space photon mapping 34
Caustics Video Beyond Programmable Shading 35
Hardware-accelerated global illumination by image space photon mapping
ISPM Properties General physically-based method Caustics, color bleeding, contact shadows, etc. As fast as single-phenomenon GPU algorithms (but lower quality for that phenomenon) 1M poly scenes 1920x1080 @ 30 fps Simple implementation ~600 statements in GLSL and C++ Hardware-accelerated global illumination by image space photon mapping
Theme Exploit the strengths of graphics pipeline, without being fettered to it Rasterizer, ROP, Z Cull, Z compression: exploit single center of projection ROP, texture: massive blending capability ( hidden FLOPS) NVIDIA GPUs: CUDA + Acceleration Engines like OptiX Beyond Programmable Shading 38
Thank You! Source code: ISPM: G3D: http://graphics.cs.williams.edu http://g3d-cpp.sf.net Onos and NS Theme from Natural Selection 2 by Unknown Worlds Entertainment Nexus 6 from Tremulous Bunny from The Stanford 3D Scanning Repository Ironworks from Quake Live by id Software Sponza Atrium by Marko Dabrovic Special thanks to Qi Mo, Evan Hart, Kefei Lei, Daniel Fast, Eric Enderton, and anonymous HPG09 reviewers Beyond Programmable Shading 39
Backup Slides (from HPG presentation) Beyond Programmable Shading 40
Hardware-accelerated global illumination by image space photon mapping 41
Hardware-accelerated global illumination by image space photon mapping 42
Hardware-accelerated global illumination by image space photon mapping 43
Radiance Estimate RT r = ho) Hardware-accelerated global illumination by image space photon mapping
Assumptions Point light Pinhole camera Limitations Clipping at near plane annoying, but ignorable/avoidable 4x more expensive than direct illumination (heavy fill consumption) Consistent, but biased (like photon mapping) Hardware-accelerated global illumination by image space photon mapping
Future Work Tracing on GPU Fill rate Shaping photon volume Game-specific optimizations and analysis The Cornell box is harder to render than Natural Selection 2! Hardware-accelerated global illumination by image space photon mapping 46
Performance Details Hardware-accelerated global illumination by image space photon mapping 47
Trace-bound 139k polys 36k photons 16 fps Trace: 25 ms Hardware-accelerated global illumination by image space photon mapping
Fill-bound 176k polys 8 objects 11 lights 100k photons 156 Mpix fill--80x overdraw! 10 fps Hardware-accelerated global illumination by image space photon mapping