Traditional Image Generation. Reflectance Fields. The Light Field. The Light Field. The Light Field. The Light Field

Transcription

1 Traditional Image Generation Course 10 Realistic Materials in Computer Graphics Surfaces + BRDFs Reflectance Fields USC Institute for Creative Technologies Volumetric scattering, density fields, phase functions The Light Field The Light Field Levoy & Hanrahan. Light Field Rendering. SIGGRAPH 96 Gortler, Grzeszczuk, Szeliski, Cohen. The Lumigraph. SIGGRAPH 96 Images captured from all directions Allows realistic images from arbitrary viewpoint No geometry, no material properties The Light Field The Light Field virtual camera 1

2 The Light Field The Light Field virtual camera virtual camera The Light Field The Light Field virtual camera Assumes free space no occluders L r (u r, v r, θ r, φ r ) The Light Field How Can Illumination be Quantified? Levoy, SIGGRAPH 96 Photorealistic renderings Illumination cannot be varied 2

3 How Can Illumination be Quantified? Radiant Light Field Answer: As a light field! L r (u r, v r, θ r, φ r ) Incident Light Field Light Transport L i (u i, v i, θ i, φ i ) L i (u i, v i, θ i, φ i ) Light Transport Superposition Principle L i (u i, v i, θ i, φ i ) L r (u r, v r, θ r, φ r ) 3

4 Superposition Principle Superposition Principle Superposition Principle Superposition Principle Impulse Light Field Adding Impulse Light Fields r 1 L 1 r 1 L 1 4

5 Adding Impulse Light Fields Impulse Light Field L 1 + r 1 L 2 L 1 + r 1 L 2 + L 3 r 2 r 2 r 3 Impulse Light Field Linearity of Light Transport L 1 + r 1 L 2 + L L(r 1 ) r 1 L 1 r 2 r 3 Linearity of Light Transport The Reflectance Field L(r 1 ) 2 r 1 L 1 2 5

6 Reflectance Field Rendering Reflectance Fields Are Powerful virtual camera light source L r (u r,v r,θ r,φ r ) = R(u i,v i,θ i,φ i ; u r,v r,θ r,φ r ) L i (u i,v i,θ i,φ i ) dω da Contrast with Traditional Models Measuring a Reflectance Field Complex geometry with simple reflectance becomes very simple geometry (sphere) with complex reflectance (8D reflectance field) Ray tracing and visibility calculations are not needed, rendering is simple convolution Use a laser or projector to send in every ray Use a camera array to record all of the resulting radiant light fields r 8 Dimensions! Assume a resolution of 100 samples for each dimension = samples (10 million gigabytes) Too much data Lower dimensional variants are more practical, and can still give considerable control over lighting 4D Slice of Reflectance Field 6

7 4D Slice of the 8D Reflectance Field Light Stage 1 Single View Directional Illumination Reflectance Field Debevec, Hawkins, Tchou, Duiker, Sarokin, Sagar. Acquiring the Reflectance Field of a Human Face. SIGGRAPH Lighting a 4D Reflectance Field Relighting Results 7

8 Reflectance functions Limitations Limited resolution Fixed viewpoint Spatially uniform lighting Static object Limitations Reflectance functions Limited resolution Fixed viewpoint Spatially uniform lighting Static object 8

9 Highly Specular Objects Capturing High Resolution Reflectance Functions Hawkins, Einarsson, Debevec. A Dual Light Stage. EGSR Traditional light stage lights Dual light stage camera with fisheye lens camera laser and galvanometer object object hollow diffuse gray sphere 9

10 Dual Light Stage Renderings Helmholtz Reciprocity at SIGGRAPH 2005 Sen, Chen, Garg, Marschner, Horowitz, Levoy, Lensch. Dual Photography. SIGGRAPH

11 Limitations Limited resolution Fixed viewpoint Spatially uniform lighting Static object Varying the Viewpoint Full light field capture for every direction of illumination > 6D Reflectance Field Directional Illumination Reflectance Field Varying the Viewpoint Problem: With no geometry, an enormous amount of data is needed to avoid ghosting artifacts. Solution: Capture as many viewpoints as is feasible, then use a geometric model to help with view interpolation. 4D Slice of Reflectance Field Light Stage 1 Moving the Viewpoint Light Stage 1 Moving the Viewpoint Debevec, Hawkins, Tchou, Duiker, Sarokin, Sagar. Acquiring the Reflectance Field of a Human Face. SIGGRAPH Debevec, Hawkins, Tchou, Duiker, Sarokin, Sagar. Acquiring the Reflectance Field of a Human Face. SIGGRAPH

12 Reflectance functions Original RF Transforming a Reflectance Function Specular Component => Torrance- Sparrow microfacet distribution Subsurface Component Shifted and Scaled Specular Surface Normal Estimate Final RF Comparison RF Point-Source Comparison Original Image Novel Viewpoint 6D Reflectance Fields Matusik, Pfister, Ngan, Beardsley, Ziegler, McMillan. Image-Based 3D Photography with Opacity Hulls. SIGGRAPH D Reflectance Fields on Opacity Hull Geometry Rotating arc of lights Static arc of cameras Rotating platform Two plasma monitors Captures medium resolution 6D reflectance field + rough geometry + opacity 12

13 Obects Integrated Into Environments Opacity Hull Relighting Example Kaleidoscopic Reflectometry Han & Perlin. Measuring Bidirectional Texture Reflectance with a Kaleidoscope. SIGGRAPH Kaleidoscopic Reflectometry Kaleidoscopic Reflectometry 13

14 Limitations Limited resolution Fixed viewpoint Spatially uniform lighting Static object Spatially Varying Incident Light Capturing Spatially Varying Incident Light Unger, Wenger, Hawkins, Gardner, and Debevec. Capturing and Rendering with Incident Light Fields. EGSR 2003 Relighting with Incident Light Fields Masselus, Peers, Dutre, Willems. Relighting with 4D Incident Light Fields. SIGGRAPH

15 Relighting with Incident Light Fields Scanning Patterns Captures a basis for illumination by any incident light field, but from a fixed viewpoint 6D reflectance field, but different than the directional illumination reflectance field Single view reflectance field Derived Pulse Response Relighting with Incident Light Fields Relit with synthetic incident light fields Relighting with Incident Light Fields Spatially Varying Illumination at SIGGRAPH 2005 Sen, Chen, Garg, Marschner, Horowitz, Levoy, Lensch. Dual Photography. SIGGRAPH Relit with synthetic incident light fields Jones, Gardner, Bolas, McDowall, Debevec. Performance Geometry Capture for Spatially Varying Relighting. SIGGRAPH

16 Limitations Animatable Facial Reflectance Fields Limited resolution Fixed viewpoint Spatially uniform lighting Static object Hawkins, Wenger, Tchou, Gardner, Goransson, Debevec. Animatable Facial Reflectance Fields. EGSR Light Stage 2 Light Stage 2 Captured Expressions Six Viewpoints 16

17 Animatable Model Results Registering all expressions and views gives a morphable model across expression, viewpoint, and lighting Rough geometric model for view interpolation Raw data size is 60 expressions x 6 views x 480 lights x 1 MB per image = 160 GB Expression Morphing Expression morphing Performance Driven Reflectance Fields Performance Driven Reflectance Fields 17

18 Performance Driven Reflectance Fields Real Time Reflectance Field Capture Wenger, Gardner, Tchou, Unger, Hawkins, Debevec. Performance Relighting and Reflectance Transformation with Time-Multiplexed Illumination. SIGGRAPH 2005 Conclusion Questions? Reflectance field capture can provide highly realistic relightable image based models of real scenes and objects Capturing and rendering with reflectance fields becomes challenging at high dimension Relatively new research area, plenty of future work 18