Computational light transport

Transcription

1 Computational light transport , , Computational Photography Fall 2018, Lecture 19

2 Course announcements Homework 5 has been posted. - Due on Friday November 9 th. Friday s office hours will be held by Alankar. Great talk this Thursday: Eric Fossum, inventor of the CMOS sensor, will talk about quantum (i.e., photon-counting) CMOS sensors.

3 Overview of today s lecture Direct and global illumination. Direct-global separation using high-frequency illumination. Light transport probing. Direct-global separation using diagonal probing (coaxial case). Direct-global separation using epipolar probing (non-coaxial case). Energy-efficient epipolar imaging.

4 Slide credits These slides were directly adapted from: Shree Nayar (Columbia). Matthew O Toole (CMU). Supreeth Achar (Google, formerly CMU).

5 Direct and global illumination

6 Direct and Global Illumination participating medium surface source D B camera A E translucent surface P C A : Direct B : Interrelection C : Subsurface D : Volumetric E : Diffusion

7 Direct and Global Components: Interreflections source surface j i camera L[ c, i] = L [ c, i] L [ c, i] d + radiance direct global g L [ c, i] = g A[ i, j] L[ i, j] P BRDF and geometry

8 Direct-global separation using highfrequency illumination

9 High Frequency Illumination Pattern source surface i L + camera [ c, i] = L [ c, i] L [ c, i] d + g fraction of activated source elements

10 High Frequency Illumination Pattern source surface i L + camera [ c, i] = L [ c, i] L [ c, i] d + - L [ c, i] = ( 1 ) L [ c, i] g g fraction of activated source elements

11 Separation from Two Images = 1 : 2 L d = L max L min, L g = 2Lmin direct global

12 Other Global Effects: Subsurface Scattering source translucent surface j i camera

13 Other Global Effects: Volumetric Scattering participating medium source j surface i camera

14 Diffuse Interreflections Specular Interreflections Diffusion Volumetric Scattering Subsurface Scattering

15 Scene Direct Global

16 V-Grooves: Diffuse Interreflections concave convex Psychophysics: Gilchrist 79, Bloj et al. 04 Direct Global

17 Real World Examples: Can You Guess the Images?

18 Eggs: Diffuse Interreflections Direct Global

19 Wooden Blocks: Specular Interreflections Direct Global

20 Novel Images

21 Photometric Stereo: The Pseudo Shape Actual Shape ( =0.95) Pseudo Shape Nayar et al., 1991

22 Photometric Stereo using Direct Images Source 1 Source 2 Source 3 Bowl Shape Global Direct

23 Variants of Separation Method Coded Structured Light Shifted Sinusoids Shadow of Line Occluder Shadow of Mesh Occluders

24 Building Corner 3D from Shadows: Bouguet and Perona 99 Stick Shadow L d = L max L min, L g = L min direct global

25 Building Corner Direct Global

26 Shower Curtain: Diffuser Mesh Shadow L d = L max L min, L g = L min direct global

27 Shower Curtain: Diffuser Direct Global

28 Kitchen Sink: Volumetric Scattering Volumetric Scattering: Chandrasekar 50, Ishimaru 78 Direct Global

29 Direct Global Peppers: Subsurface Scattering

30 Real or Fake? F R F R R F Direct Global

31 Direct Global Tea Rose Leaf Leaf Anatomy: Purves et al. 03

32 Translucent Rubber Balls Direct Global

33 Resolution Marble: When BSSRDF becomes BRDF Scene Direct Global Subsurface Measurements: Jensen et al. 01, Goesele et al. 04

34 Hand Skin: Hanrahan and Krueger 93, Uchida 96, Haro 01, Jensen et al. 01, Igarashi et al. 05, Weyrich et al. 05 Direct Global

35 Hands Afric. Amer. Female Chinese Male Spanish Male Afric. Amer. Female Chinese Male Spanish Male Afric. Amer. Female Chinese Male Spanish Male Direct Global

36 Separation from a Single Image

37 Face Direct Global Sum

38 Blonde Hair Hair Scattering: Stamm et al. 77, Bustard and Smith 91, Lu et al. 00 Marschner et al. 03 Direct Global

39 Direct Global Pebbles: 3D Texture

40 Pink Carnation Spectral Bleeding: Funt et al. 91 Direct Global

41 = + = + = + = + = + = + = + = + = + = +

42 Mirror Ball: Failure Case Direct Global

43 Light transport probing (see part 2)

44 Direct-global separation using epipolar probing (non-coaxial case)

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69 Energy-efficient epipolar imaging

70 Energy-efficient transport parsing

71 Regular Imaging Light Source Sensor

72 Regular Imaging Light Source Sensor

73 Regular Imaging Light Source Sensor

74 Regular Imaging Light Source Sensor

75 Epipolar Imaging Epipolar Plane Light Source Sensor & Mask

76 Epipolar Imaging Light Source Sensor & Mask

77 Epipolar Imaging Light Source Complete Image

78 Epipolar Imaging Light Source Sensor & Mask

79 Epipolar Imaging Light Source Sensor & Mask

80 Energy-efficient transport parsing

81 all paths planar (mostly direct) non-planar (always indirect) a great deal of indirect transport occurs in many common LOS scenes

82 all paths planar (mostly direct) non-planar (always indirect)

83 References Basic reading: Nayar et al., Fast separation of direct and global components of a scene using high frequency illumination, SIGGRAPH The paper on separation of direct and global illumination using high-frequency illumination. O Toole et al., Primal-dual coding to probe light transport, SIGGRAPH O Toole et al., 3d shape and indirect appearance by structured light transport, CVPR These two papers introduce the concepts of light transport probing and epipolar probing, as well as explain how to use primal-dual coding to achieve them. O Toole et al., Homogeneous codes for energy-efficient illumination and imaging, SIGGRAPH This paper shows how to efficiently implement epipolar imaging with a simple projector and camera. Additional reading: Seitz et al., A theory of inverse light transport, ICCV This early paper shows a way to exactly decompose light transport by number of bounces, under certain assumptions for the imaged scene. Chandraker et al., On the duality of forward and inverse light transport, PAMI Reddy et al., Frequency-space decomposition and acquisition of light transport under spatially varying illumination, ECCV These two papers have additional analysis about the relationship between direct and global illumination and illumination frequency.