Step-by-Step Model Buidling

Transcription

1 Step-by-Step Model Buidling

2 Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure and camera motion

3 Review Feature selection Feature selection Feature correspondence Camera Calibration Epipolar Rectification Dense Correspondence Euclidean Reconstruction Sparse Structure and motion Texture mapping 3-D Model

4 Review Feature selection Feature selection Feature correspondence Projective Reconstruction Partial Scene Knowledge Partial Motion Knowledge Partial Calibration Knowledge Camera Self-Calibration Epipolar Rectification Dense Correspondence Euclidean Reconstruction Texture mapping 3-D Model

5 Examples

6 Feature Selection Compute Image Gradient Compute Feature Quality The image cannot be displayed. Your measure for each pixel Search for local maxima Feature Quality Function Local maxima of feature quality function

7 Feature Tracking Translational motion model Closed form solution 1. Build an image pyramid 2. Start from coarsest level 3. Estimate the displacement at the coarsest level 4. Iterate until finest level

8 Coarse to fine feature tracking compute 2. warp the window in the second image by 3. update the displacement 4. go to finer level 5. At the finest level repeat for several iterations The image cannot be displayed. Your

9 Optical Flow Integrate around over image patch Solve

10 Affine feature tracking Contrast change Intensity offset

11 Tracked Features

12 Wide baseline matching Point features detected by Harris Corner detector

13 Wide baseline Feature Matching 1. Select the features in two views 2. For each feature in the first view 3. Find the feature in the second view that maximizes 4. Normalized cross-correlation measure Select the candidate with the similarity above selected threshold

14 More correspondences and Robust matching Select set of putative correspondences Repeat 1. Select at random a set of 8 successful matches 2. Compute fundamental matrix 3. Determine the subset of inliers, compute distance to epipolar line The ima ge cann 4. Count the number of points in the consensus set

15 RANSAC in action Inliers Outliers

16 Epipolar Geometry Epipolar geometry in two views Refined epipolar geometry using nonlinear estimation of F

17 Two view initialization calibrated Recover epipolar geometry Compute (Euclidean) projection matrices and 3-D struct. uncalibrated unknown Compute (Projective) projection matrices and 3-D struct.

18 Nonlinear Refinement Euclidean Bundle adjustment Initial estimates of are available Final refinement, nonlinear minimization with respect to all unknowns

19 Example - Euclidean multi-view reconstruction

20 Example Original sequence Tracked Features

21 Recovered model

22 Euclidean Reconstruction

23 Epipolar rectification Make the epipolar lines parallel Dense correspondences along image scanlines Computation of warping homographies 1. Map the epipole to infinity Translate the image center to the origin Rotate around z-axis for the epipole lie on the x-axis Transform the epipole from x-axis to infinity The imag e The imag e 2. Find a matching transformation is compatible with the epipolar geometry is chosen to minimize overall disparity T h e

24 Epipolar rectification Rectified Image Pair

25 Epipolar rectification Rectified Image Pair

26 Dense Matching Establish dense correspondences along scan-lines Standard stereo configuration Constraints to guide the search 1. ordering constraint 2. disparity constraint limit on disparity 3. uniqueness constraint each point has a unique match in the second view

27 Dense Matching

28 Dense Reconstruction

29 Texture mapping, hole filling

30 Texture mapping

31 Steps + = Images! Points: Structure from Motion Points! More points: Multiple View Stereo Points! Meshes: Model Fitting Meshes! Models: Texture Mapping Images! Models: Image-based Modeling =

32 Bundle adjustment Theory: The Levenberg Marquardt algorithm Practice: The Ceres-Solver from Google

33 Multiple View Stereo State-of-the-art: PMVS: Accurate, Dense, and Robust Multi-View Stereopsis, Y Furukawa and J Ponce, 2 Benchmark: A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithm SM Seitz, B Curless, J Diebel, D Scharstein, R Szeliski Baseline: Multi-view stereo revisited. M Goesele, B Curless, SM Seitz

34 How to get the intrinsic parameters? Auto-calibration Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters, M Pollefeys, R Koch and L Van Gool, Grid Search to look for the solution with minimal reprojection error for f=min_f:max_f do everything, then obtain reprojection error after bundle adjustment Optimize for this value in bundle adjustment Camera Calibration (with checkerboard) EXIF of JPEG file recorded from digital camera Read the code of Bundler to understand how to convert EXIF into focal length value

35 Real World Applications Streetview Reconstruction and Recognition Photo Tourism Microsoft Photosynth 2d3, boujor (Matchmovers) and movies Robotics: SLAM Simultaneous Localization And Mapping