Multiple View Geometry. Frank Dellaert

Transcription

1 Multiple View Geometry Frank Dellaert

2 Outline Intro Camera Review Stereo triangulation Geometry of 2 views Essential Matrix Fundamental Matrix Estimating E/F from point-matches

3 Why Consider Multiple Views? P X P' x x' Answer: To extract 3D structure via triangulation.

4 Stereo Rig Top View Matches on Scanlines Convenient when searching for correspondences.

5 Feature Matching!

6 Real World Challenges Bad News: Good correspondences are hard to find Good news: Geometry constrains possible correspondences. 4 DOF between x and x'; only 3 DOF in X. Constraint is manifest in the Fundamental matrix F can be calculated either from camera matrices or a set of good correspondences.

7 Geometry of 2 views? What if we do not know R,t? Caveat: My exposition follows book conventions but more intuitive (IMHO) Different from Hartley & Zisserman! F&P use [R T -R T t] camera matrices H&Z uses [R t]

8 Epipolar Geometry Where can p appear? P p t C C M =[R T -R T t] M=[I 0]? p

9 Image of Camera Center epipole M =[R T -R T t] M=[I 0]

10 Example:Cameras Point at Each Other Top View Epipolar Lines

11 Epipoles Camera Center C in first view: [ ] t 1 e = I 0 " $ # Origin C in second view: % ' = t & # e'= [ R T "R T t] 0 & % ( = "R T t $ 1'

12 Image of Camera Ray? epipole M =[R T -R T t] M=[I 0]

13 Point at infinity Given p, what is corresponding point at infinity [x 0]? Answer for any camera M =[A a]: # p'= [ A a] " x & % ( = Ax ) x = A *1 p' $ 0' A -1 = Infinite homography In our case M =[R T -R T t]: x = Rp'

14 Sidebar: Infinite Homographies Homography between image plane plane at infinity Navigation by the stars: Image of stars = function of rotation R only! Traveling on a sphere rotates viewer

15 Essential Matrix

16 Epipolar Line Calculation 1) Point 1 = epipole e=t 2) Point 2 = point at infinity [ ] # Rp' p " = I 0 3) Epipolar line = join of points 1 and 2 $ & % 0 l = t " Rp' ' ) = Rp' (

17 P Epipolar Lines P p C e e=t C M =[R T -R T t] M=[I 0] p l = t! p =Rp Rp'

18 Epipolar lines e=t l = t! Rp' p =Rp

19 Epipolar Plane P p l' l p C e e=t C M =[R T -R T t] M=[I 0]

20 Essential Matrix mapping from p to l l = t " Rp'= [ t] R # p'= E # p' " E = 3*3 matrix Because p is on l, we have p T Ep'= 0

21 E s Degrees of Freedom R,t = 6 DOF However, scale ambiguity! = 5 DOF

22 Fundamental Matrix FUNDAMENTAL

23 P Uncalibrated Case P p p =A -1 p p [ ]! e A 1 p' C e =a e=-a -1 a C M =K [R [A a] -1 K [R T -R -R T t] t] M=K[I 0] M=[I 0] l = "

24 Uncalibrated Case, Forsyth & Ponce Version Fundamental Matrix (Faugeras and Luong, 1992)

25 Fundamental Matrix mapping from p to l l = e " A #1 p'= [ e] A #1 $ p'= F $ p' " F = 3*3 matrix Because p is on l, we have p T Fp'= 0

26 Properties of the Fundamental Matrix Fp is the epipolar line associated with p. F T p is the epipolar line associated with p. F T e=0 and Fe =0. F is singular.

27 The Eight-Point Algorithm (Longuet-Higgins, 1981) Minimize: under the constraint 2 F =1.

28 Non-Linear Least-Squares Approach (Luong et al., 1993) Minimize with respect to the coefficients of F, using an appropriate rank-2 parameterization.

29 The Normalized Eight-Point Algorithm (Hartley, 1995) Center the image data at the origin, and scale it so the mean squared distance between the origin and the data points is 2 pixels: q i = T p i q i = T p i. Use the eight-point algorithm to compute F from the points q i and q i. Enforce the rank-2 constraint. Output T -1 F T.

30 RANSAC Frank Dellaert Slides all my own this time

31 Motivation Estimating motion models Typically: points in two images Candidates: Translation Homography Fundamental matrix

32 Mosaicking: Homography

33 Fundamental Matrix

34 Omnidirectional example Images by Branislav Micusik, Tomas Pajdla, cmp.felk.cvut.cz/ demos/fishepip/

35 Simpler Example Fitting a straight line

36 Discard Outliers No point with d>t RANSAC: RANdom SAmple Consensus Fischler & Bolles 1981 Copes with a large proportion of outliers

37 Main Idea Select 2 points at random Fit a line Support = number of inliers Line with most inliers wins

38 Why will this work?

39 Best Line has most support More support -> better fit

40 In General Fit a more general model Sample = minimal subset Translation:? Homography? Fundamental Matrix?

41 RANSAC Objective: Robust fit of a model to data S Algorithm Randomly select s points Instantiate a model Get consensus set Si If Si >T, terminate and return model Repeat for N trials, return model with max Si

42 Distance Threshold Requires noise distribution Gaussian noise with σ Chi-squared distribution with DOF m 95% cumulative: Line, F: m=1, t=3.84 σ 2 Translation, homography: m=2, t=5.99\ σ 2 I.e. -> 95% prob that d<t is inlier

43 How many samples? We want: at least one sample with all inliers Can t guarantee: probability p E.g. p =0.99

44 Calculate N If w = proportion of inliers = 1-etha P(sample with all inliers)=w s P(sample with an outlier)=1-w s P(N samples an outlier)=(1-w s )^N We want P(N samples an outlier)<1-p (1-w s )^N<1-p N>log(1-p)/log(1-w s )

45 Example P=0.99 s=2, etha=5% => N=2 s=2, etha=50% => N=17 s=4, etha=5% => N=3 s=4, etha=50% => N=72 s=8, etha=5% => N=5 s=8, etha=50% => N=1177

46 Remarks N = f(etha), not the number of points N increases steeply with s

47 Threshold T Remember: terminate if Si >T Rule of thumb: T #inliers So, T=(1-etha)n

48 Adaptive N When etha is unknown? Start with etha=50%, N=inf Repeat: Sample s, fit model -> update etha as outliers /n -> set N=f(etha,s,p) Terminate when N samples seen