Light Field 2015 Spring
Recall: Light is Electromagnetic radiation (EMR) moving along rays in space R(l) is EMR, measured in units of power (watts) l is wavelength Useful things: Light travels in straight lines In vacuum, radiance emitted = radiance arriving i.e. there is no transmission loss
Plenoptic Function The body of the air is full of an infinite number of radiant pyramids caused by the objects located in it Leonardo Da Vinci Pencil of rays: The set of light rays passing through any point in space Plenoptic function Plenus : complete or full Optic 3
The Plenoptic Function Figure by Leonard McMillan Q: What is the set of all things that we can ever see? A: The Plenoptic Function (Adelson & Bergen) Let s start with a stationary person and try to parameterize everything that he can see
Grayscale snapshot is intensity of light P(q,f) Seen from a single view point At a single time Averaged over the wavelengths of the visible spectrum (can also do P(x,y), but spherical coordinate are nicer)
Color snapshot is intensity of light Seen from a single view point At a single time As a function of wavelength P(q,f,l)
A movie is intensity of light Seen from a single view point Over time As a function of wavelength P(q,f,l,t)
Holographic movie is intensity of light Seen from ANY viewpoint Over time As a function of wavelength P(q,f,l,t,V X,V Y,V Z )
The Plenoptic Function P(q,f,l,t,V X,V Y,V Z ) Can reconstruct every possible view, at every moment, from every position, at every wavelength Contains every photograph, every movie, everything that anyone has ever seen.
Geometric Components of the pencil of ray lights ANGULAR COORDINATES I E = (Ex, Ey, Ez) Viewpoint V. ( v, v) P( v, v, Ex, Ey, Ez, t, l) Direction of the ray light passin trough the Viewpoint CARTESIAN COORDINATES I P( x, y, Ex, Ey, Ez, t, l) Comonly used in machine vision 10
Sampling Plenoptic Function (top view)
Ray Let s don t worry about time and color: P(q,f,V X,V Y,V Z ) 5D 3D position 2D direction Slide by Rick Szeliski and Michael Cohen
How can we use this? Lighting No Change in Radiance Surface Camera
Ray Reuse Infinite line Assume light is constant (vacuum) 4D 2D direction 2D position non-dispersive medium Slide by Rick Szeliski and Michael Cohen
Only need plenoptic surface
Synthesizing novel views Slide by Rick Szeliski and Michael Cohen
Lumigraph / Lightfield Outside convex space Empty Stuff 4D Slide by Rick Szeliski and Michael Cohen
Lumigraph - Organization 2D position 2D direction q s Slide by Rick Szeliski and Michael Cohen
Lumigraph - Organization 2D position 2D position s u 2 plane parameterization Slide by Rick Szeliski and Michael Cohen
Lumigraph - Organization 2D position 2D position t s,t s,t u,v v 2 plane parameterization u,v s u Slide by Rick Szeliski and Michael Cohen
Lumigraph - Organization Hold s,t constant Let u,v vary An image s,t u,v Slide by Rick Szeliski and Michael Cohen
Lumigraph / Lightfield
Lumigraph - Capture Idea 1 Move camera carefully over s,t plane Gantry see Lightfield paper s,t u,v Slide by Rick Szeliski and Michael Cohen
Lumigraph - Capture Idea 2 Move camera anywhere Rebinning see Lumigraph paper s,t u,v Slide by Rick Szeliski and Michael Cohen
Lumigraph - Rendering For each output pixel determine s,t,u,v either use closest discrete RGB interpolate near values s u Slide by Rick Szeliski and Michael Cohen
Lumigraph - Rendering Nearest closest s closest u draw it Blend 16 nearest quadrilinear interpolation s u Slide by Rick Szeliski and Michael Cohen
Stanford multi-camera array 640 480 pixels 30 fps 128 cameras synchronized timing continuous streaming flexible arrangement
2D: Image What is an image? All rays through a point Panorama? Slide by Rick Szeliski and Michael Cohen
Image Image plane 2D position
Spherical Panorama See also: 2003 New Years Eve http://www.panoramas.dk/fullscreen3/f1.html All light rays through a point form a ponorama Totally captured in a 2D array -- P(q,f) Where is the geometry???
Other ways to sample Plenoptic Function Moving in time: Spatio-temporal volume: P(q,f,t) Useful to study temporal changes Long an interest of artists: Claude Monet, Haystacks studies
Space-time images Other ways to slice the plenoptic function t y x
What is a BRDF? Must know something about light and how it interacts with matter When light interacts with matter: Complicated light-matter dynamic occurs Dependent on characteristics of both the light and the matter Example, sandpaper vs. a mirror
What is a BRDF? Typical light-matter interaction scenario: Reflected Light Incoming Light Internal Reflection Transmitted Light Absorption Scattering and Emission 3 types of interaction: transmission, reflection, and absorbtion Light incident at surface = reflected + absorbed + transmitted BRDF describes how much light is reflected
What is a BRDF? BRDF is a function of incoming light direction and outgoing view direction light L θ L N θ V V observer surface In 3D, a direction D can be represented in spherical coordinates (q D, f D ) A BRDF is a 4D function: BRDF( q L, f L, q V, f V )
Background Bidirectional Reflectance Distribution Function
BRDF Bidirectional Reflectance Distribution Function ρ(θ i,f i ; θ o, f o )
BRDF Bidirectional Reflectance Distribution Function ρ(θ i,f i ; θ o, f o ) Isotropic material Invariant when material is rotated BRDF is 3D
BRDF Models Phenomenological Phong [75] Blinn-Phong [77] Ward [92] Lafortune et al. [97] Ashikhmin et al. [00] Physical Cook-Torrance [81] He et al. [91] Lafortune [97] Cook-Torrance [81]