Reconstructing Reflective and Transparent Surfaces from Epipolar Plane Images

Transcription

1 Sven Wanner and Bastian Goldlücke Heidelberg Collaboratory for Image Processing

2 1. Light Fields and Epipolar Plane Images 1. Light Fields and Epipolar Plane Images

3 Light Fields as a dense sampling of a scene on a regular planar grid 1. Light Fields and Epipolar Plane Images

4 Light Fields as a dense sampling of a scene on a regular planar grid Camera movement Epipolar Planes as linear mapping of 3D points in space 1. Light Fields and Epipolar Plane Images

5 2. Orientation vs. Matching 2. Orientation vs. Matching

6 Matching means Searching slow local minima discrete search space 2. Orientation vs. Matching

7 Analysing the Orientation fast more robust continous disparity space 2. Orientation vs. Matching

8 Analysing the Orientation fast more robust continous disparity space 2. Orientation vs. Matching

9 Analysing the Orientation fast more robust continous disparity space 2. Orientation vs. Matching

10 Analysing the Orientation fast more robust continous disparity space 2. Orientation vs. Matching

11 Analysing the Orientation fast more robust continous disparity space 2. Orientation vs. Matching

12 3. Single Orientation Analysis 3. Single Orientation Analysis

13 Orientation estimation using the structure tensor: The direction of the local level lines can be computed via a decomposition of the structure tensor which can be solved analytically: Eigenvector: 3. Single Orientation Analysis

14 view of input light field 3. Single Orientation Analysis ground truth

15 multiview stereo [1] [1] T. Pock, D. Cremers, H. Bischof, and A. Chambolle. Global solutions of variational models with convex regularization. SIAM Journal on Imaging Sciences, Single Orientation Analysis orientation analysis [2] [2] S. Wanner, B. Goldlücke: Variational Light Field Analysis for Disparity Estimation and Super-Resolution. IEEE TPAMI (2013)

16 3. Single Orientation Analysis

17 4. What happens on Mirrors and Transparencies? 4. What happens on Mirrors and Transparencies

18 M : mirror m : point on mirror p : mirrored point p' : virtual point 4. What happens on Mirrors and Transparencies

19 M : mirror m : point on mirror p : mirrored point p' : virtual point 4. What happens on Mirrors and Transparencies

20 5. Double Orientation Analysis 5. Double Orientation Analysis

21 v Observed colors: base color : reflection color : 5. Double Orientation Analysis

22 To recap the model for single orientation: A region of an image has a orientation if and only if for all This orientation is given by the Eigenvector corresponding to the smaller Eigenvalue of the structure tensor But this model fails if 5. Double Orientation Analysis

23 In this case the two orientations, needs to satisfy the condition: and 5. Double Orientation Analysis

24 In this case the two orientations, needs to satisfy the condition: and Which can be solved by analysing the Eigensystem of the second order structure Tensor [1]: [1] Mühlich, Matthias, and Til Aach. "A theory of multiple orientation estimation." Computer Vision ECCV Springer Berlin Heidelberg, Double Orientation Analysis

25 In analogy to the Eigenvector decomposition of the 2D structure tensor, a decomposition of results in an Eigenvector. [1] Mühlich, Matthias, and Til Aach. "A theory of multiple orientation estimation." Computer Vision ECCV Springer Berlin Heidelberg, Double Orientation Analysis

26 In analogy to the Eigenvector decomposition of the 2D structure tensor, a decomposition of results in an Eigenvector The two disparities are then equal to the Eigenvalues. of the matrix: A= [1] Mühlich, Matthias, and Til Aach. "A theory of multiple orientation estimation." Computer Vision ECCV Springer Berlin Heidelberg, Double Orientation Analysis

27 6. Results 6. Results

28 6. Results

29 6. Results

30 Input light field 6. Results single orientation

31 object channel 6. Results reflection channel

32 single orientation object channel 6. Results reflection channel

33 single orientation object channel 6. Results reflection channel

34 single orientation 6. Results

35 transparent channel 6. Results object channel

36 7. Conclusion Geometry reconstruction in light fields, using orientation analysis instead of matching Transparencies and Reflections maps double orientation pattern on epipolar planes. Patterns can be analyzed efficiently by decomposing a second order structure tensor. We provide a benchmark database containing simulated and real world light fields Conclusion