Vision par ordinateur

Transcription

1 Epipolar geometry π Vision par ordinateur Underlying structure in set of matches for rigid scenes l T 1 l 2 C1 m1 l1 e1 M L2 L1 e2 Géométrie épipolaire Fundamental matrix (x rank 2 matrix) m2 C2 l2 Frédéric Devernay Avec des transparents de Marc Pollefeys Canonical representation: 1. Computable from corresponding points 2. Simplifies matching. Allows to detect wrong matches. Related to calibration Epipolar geometry x1 C1 l1 ΠP M L2 L1 The projective reconstruction theorem If a set of point correspondences in two views determine the fundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent allows reconstruction from pair of uncalibrated images! l T 1 l 2 e1 e2 x2 l2 Fundamental matrix (x rank 2 matrix) C2 Properties of the fundamental matrix Computation of F Linear (8-point) Minimal (7-point) Robust (RANSAC) Non-linear refinement (MLE, ) Practical approach

2 Epipolar geometry: basic equation the NOT normalized 8-point algorithm separate known from unknown (data) (unknowns) (linear) ~10000 ~10000 ~100 ~10000 ~10000 ~100 ~100 ~100 1 Orders of magnitude difference! between column of data matrix least-squares yields poor results the normalized 8-point algorithm the singularity constraint Transform image to ~[-1,1]x[-1,1] SVD from linearly computed F matrix (rank ) (0,00) (700,00) (-1,1) (1,1) (0,0) Compute closest rank-2 approximation (0,0) (700,0) (-1,-1) (1,-1) normalized least squares yields good results (Hartley, PAMI 97) F vs. F' the minimum case 7 point correspondences one parameter family of solutions but F 1 +λf 2 not automatically rank 2

3 the minimum case impose rank 2 σ (obtain 1 or solutions) F 7pts F F 1 F 2 Robust estimation What if set of matches contains gross outliers? (to keep things simple let s consider line fitting first) (cubic equation) Compute possible λ as eigenvalues of (only real solutions are potential solutions) Minimal solution for calibrated cameras: -point RANSAC (RANdom Sampling Consensus) Objective Robust fit of model to data set S which contains outliers Algorithm (i) Randomly select a sample of s data points from S and instantiate the model from this subset. (ii) Determine the set of data points S i which are within a distance threshold t of the model. The set S i is the consensus set of samples and defines the inliers of S. (iii) If the subset of S i is greater than some threshold T, reestimate the model using all the points in S i and terminate (iv) If the size of S i is less than T, select a new subset and repeat the above. (v) After N trials the largest consensus set S i is selected, and the model is re-estimated using all the points in the subset S i Distance threshold Choose t so probability for inlier is α (e.g. 0.9) Often empirically Zero-mean Gaussian noise σ then follows distribution with m=codimension of model Codimension 1 2 (dimension+codimension=dimension space) Model line,f H,P T t 2.8σ 2.99σ σ 2 How many samples? Choose N so that, with probability p, at least one random sample is free from outliers. e.g. p=0.99 Acceptable consensus set? Typically, terminate when inlier ratio reaches expected ratio of inliers s % 2 10% proportion of outliers e 20% 2% 0% % % Note: Assumes that inliers allow to identify other inliers

4 Adaptively determining the number of samples e is often unknown a priori, so pick worst case, i.e. 0, and adapt if more inliers are found, e.g. 80% would yield e=0.2 N=, sample_count =0 While N >sample_count repeat Choose a sample and count the number of inliers Set e=1-(number of inliers)/(total number of points) Recompute N from e Increment the sample_count by 1 Terminate Other robust algorithms RANSAC maximizes number of inliers LMedS minimizes median error Not recommended: case deletion, iterative least-squares, etc. Non-linear refinment Geometric distance Gold standard Symmetric epipolar distance Gold standard Gold standard Alternative, minimal parametrization (with a=1) Maximum Likelihood Estimation (= least-squares for Gaussian noise) Initialize: normalized 8-point, (P,P ) from F, reconstruct X i Parameterize: Minimize cost using Levenberg-Marquardt (preferably sparse LM, e.g. see H&Z) (overparametrized) (note (x,y,1) and (x,y,1) are epipoles) problems: a=0 pick largest of a,b,c,d to fix to 1 epipole at infinity pick largest of x,y,w and of x,y,w xx=6 parametrizations! reparametrize at every iteration, to be sure

5 Symmetric epipolar error Recommendations: 1. Do not use unnormalized algorithms 2. Quick and easy to implement: 8-point normalized. Better: enforce rank-2 constraint during minimization. Best: Maximum Likelihood Estimation (minimal parameterization, sparse implementation) Residual error: (for all points!)

6 Automatic computation of F Step 1. Extract features Step 2. Compute a set of potential matches Step. do Step.1 select minimal sample (i.e. 7 matches) Step.2 compute solution(s) for F Step. determine inliers (verify hypothesis) until Γ(#inliers,#samples)<9% (generate hypothesis) Two-view geometry Step. Compute F based on all inliers Step. Look for additional matches Step 6. Refine F based on all correct matches #inliers #samples 90% 80% 1 70% 60% 106 0% 82 geometric relations between two views is fully described by recovered x matrix F