L1a: Introduction to Light Fields

Size: px

Start display at page:

Download "L1a: Introduction to Light Fields"

Cornelia Green
5 years ago
Views:

1 L1a: Introduction to Light Fields 2018 IEEE SPS Summer School on Light Field Data Representation, Interpretation, and Compression Donald G. Dansereau, May 2018

2 Schedule 2

3 Outline Lecture 1a: Introduction to Light Fields Intro History Parameterizations Visualizations Lecture 1b: Cameras, Sampling, & Calibration Lecture 1c: Basic Processing Hands-on: Writing a renderer, handling light fields in matlab 3

Resources Exercise handouts are available at:

vision/lf2018/ These slides will also be up there

to datasets, forums, tools: https://github.

4 Resources Exercise handouts are available at: These slides will also be up there soon Light Field Resources page on GitHub with links to datasets, forums, tools: Light field toolbox, sampling pattern explorer, LF Synth: 4

5 What is a light field? 5

6 1) Represent scenes as a surface of light We can represent complex scenes inside a volume by describing the light rays passing through a surface surrounding the volume [Image c/o Ihrke et al 2016] 6

7 2) Represent light as a scalar field Monochrome Photo: ℒ( u, v ) Plenoptic Function ℒ(.) Position (3) Direction (2) Time (1) Wavelength (1) Phase (1) Polarization (1..3) [Adelson and Bergen 1991] 7

8 3) 4D is a sweet spot for rays Minimum for describing position & direction 1 less than the obvious 5 Assumes non-participating medium Restricted to outside a volume 8

9 A Versatile Representation Regular, densely sampled light Emulate virtual cameras a meta-camera Fundamental structure capturing complex behaviours 9

10 History How old are these ideas? 10

11 Welcome to Lights Fields, Google 2018 Video: Google s Welcome to Light Fields Demo 11

12 1996 The Light Field Representation 12

13 1996 The Lumigraph 13

14 1996 Nintento 64 Deep blue beats Garry Kasparov MP3 patented IPv6 introduced Tom's Hardware starts IMDB KDE JDK 1.0 The Internet Archive The wheel mouse hits mainstream Pentium II 486DX! 14

15 1996 Video: Levoy et al Light Field Rendering - Siggraph '96 video 15

16 1992 Depth from Epipolar Planes 16

17 1991 The Plenoptic Function 17

18 1985/1987 Epipolar Plane Images 18

19 1981/1986 Epipolar Plane Images M. Yamamoto, "Motion analysis using the visualized locus method," untranslated Japanese articles,

20 1908 Light Field Capture and Display 20

Instrumentenkd, vol. 20, no. 47, p. 2, 1900.

21 1900 Hartmann Mask for Astronomy J. Hartmann, Bemerkungen uber den bau und die justirung von spektrographen, Z. Instrumentenkd, vol. 20, no. 47, p. 2, Evolved into Shack-Hartmann sensors for adaptive optics R. V. Shack and B. C. Platt, Production and use of a lenticular Hartmann screen, Journal of the Optical Society of America, vol. 61, no. 5, p. 656,

22 1642 A Light Field Camera Obscura Mario Bettini "Apiaria universae philosophiae mathematicae",

23 Parameterizations 23

24 2-Plane Parameterization (2pp) 4D Image ℒ( s, t, u, v ) 24

25 Planar vs Planar/Spherical Two-Plane Position + Direction 25

26 Spherical [Todt 2007] [Dansereau 2017] Camera-centered R = focal distance 26

27 Surface [images c/o Wood et al, UWashington, 27

28 Absolute vs Relative 2pp Absolute 2pp: U,V relative to fixed point Relative 2pp: u,v relative to s,t 28

29 A Caution on Terminology Angular? Spatial? Not universal. In doubt assume scene-centric s,t,u,v not universal, careful of order! Mix of continuous (m) / sampled (index) 29

30 Visualizations How many ways can you slice a 4D function into 2D slices? 30

31 Visualizations LF c/o Stanford Light Field Archive 31

32 2D images in u,v Each image: ℒ( u,v ) s,t fixed For relative 2pp these are perspective images 32

33 2D images in s,t Each image: ℒ( s,t ) u,v fixed For relative 2pp these are orthographic images 33

34 Still perspective image ℒ( u, v ) s,t fixed 34

35 Animated perspective image ℒ( u, v ) s,t animated Video: panning around the s,t plane 35

36 Epipolar Slices (aka Phase Space, EPIs) ℒ( s,u ) t,v fixed ℒ( t,v ) s,u fixed More on these later... 36

37 Points to Ponder We saw slices s,t; u,v; s,u; and t,v. What about the other combinations? What should we call a 3D subset L(s,u,v)? Or a 2D subset L(s,v)? When might each of these arise? What sorts of tasks might be simpler in s,t slices than in u,v slices? Simpler in s,u or t,v slices? With what LF parameterizations could you represent all four walls, ceiling, and floor of a room? How about all views surrounding an object? 37

Computational Photography

Computational Photography Matthias Zwicker University of Bern Fall 2010 Today Light fields Introduction Light fields Signal processing analysis Light field cameras Application Introduction Pinhole camera