Inverse Light Transport (and next Separation of Global and Direct Illumination)

Transcription

1 Inverse ight Transport (and next Separation of Global and Direct Illumination) CS434 Daniel G. Aliaga Department of Computer Science Purdue University

2 Inverse ight Transport ight Transport Model transfer of light from source (e.g., light/projector) to destination (e.g., eye/camera) modulated by scene light scene camera Inverse ight Transport Given a photograph of an unknown scene, compute (or decompose) the light into the needed source(s) light scene camera

3

4 Topics A Theory of Inverse ight Transport Seitz et al., ICCV 2005 Radiometric Compensation and Inverse ight Transport Wetzstein et al., PG 2007 Work at Purdue

5 Topics A Theory of Inverse ight Transport Seitz et al., ICCV 2005 Radiometric Compensation and Inverse ight Transport Wetzstein et al., PG 2007 Work at Purdue

6 Theory of Inverse ight Transport Given a photo, decompose it into a sum of n-bounce images Each bounce image records the light that bounces n times before reaching the camera Formulated for ambertian scenes

7 Theory of Inverse ight Transport I 2 I I I... 3 I n I = direct illumination image I i = indirect illumination image, for i 2 by removing the I i s the photographs are converted into a form more amenable to existing graphics/vision processing algorithms

8 Formulation Outward light field from x to point y is ( x, y) ( x, y) 2,3,... ( x, y) Recall Rendering Equation (or synthetic lighttransport equation): I( x, x') g( x, x') ( x, x') ( x, x', x'') I( x', x'') dx'' s

9 Formulation Outward light field from x to point y is ( x, y) ( x, y) Rewrite Rendering Equation as 2,3,... ( x, y) ( x, y) ( x, y) A( x', x, y) ( x', x) dx' x' A( x', x, y) is the proportion of irradiance from x to x that gets transported to y

10 Formulation ( x, y) ( x, y) A( x', x, y) ( x', x) dx' x' [ i] [ i] A[ i, j] [ j] j for small facets i,j A?

11 Formulation ( x, y) ( x, y) A( x', x, y) ( x', x) dx' x' [ i] [ i] A[ i, j] [ j] j for small facets i,j A ( I A) (well-known) maps a light field containing only direct light to a light field having indirect light

12 Cancellation Operator ( I A) C I A ( C ) or C which means C cancels the interreflections in

13 Cancellation Operator C So what is all the light except for the direct illumination? C So now the previous first bounce indirect light is effectively now the direct illumination component What is the due to the second bounce of light? C ( C )

14 So in general, Cancellation Operator n C C n C ( I C n ) n where n defines the light field due to the n-th bounce of light, and n n

15 Computing C For ambertian scenes, it turns C T T where T is a light-impulse response matrix similar to that used for dual photography of a diffuse scene (i.e., it is a diagonal matrix) where T is a matrix of the reciprocals of the diagonal elements of T -

16 Examples

17 Examples

18 Topics A Theory of Inverse ight Transport Seitz et al., ICCV 2005 Radiometric Compensation and Inverse ight Transport Wetzstein et al., PG 2007 Work at Purdue (Aliaga et al. TOG 202)

19 Radiometric Compensation and Inverse ight Transport Single projector case (Wetzstein et al. 2007) Theoretically simple, just invert the light transport: C TP P T C P T C 0 Computation can be expense Bimber spatially decomposes T and uses GPU Success of spatial decomposition is scene dependent

20 Example

21 Example

22 Radiometric Compensation and Inverse ight Transport Multiple projector case More complicated Need to constrain solution and is computationally much more challenging

23 Use IT to alter appearance Alter the appearance of the object s surface

24 Single Projector Appearance Editing

25 Multi-Projector Appearance Editing Partially overlapping projectors

26 Multi-Projector Appearance Editing Fully superimposed projectors

27 Multi-Projector Appearance Editing Use higher resolution camera to capture projector pixel interaction

28 Overlapping Projector Interaction

29 Overlapping Projector Interaction proj.

30 Overlapping Projector Interaction proj. proj. 2

31 Overlapping Projector Interaction proj. proj. 2 proj. 3

32 Overlapping Projector Interaction proj. proj. 2 proj. 3

33 Overlapping Projector Interaction Model metapixels and their interaction within and across projectors proj. proj. 2 proj. 3 contribution from 3 projector pixels

34 Challenges Efficiently model proj-proj-cam pixel interactions

35 Challenges Constrain solution to produce valid projection values

36 Challenges Constrain solution to produce valid projection values

37 Examples