Epipolar Geometry Prof. D. Stricker. With slides from A. Zisserman, S. Lazebnik, Seitz

Transcription

1 Epipolar Geometry Prof. D. Stricker With slides from A. Zisserman, S. Lazebnik, Seitz 1

2 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two views The geometry of two cameras Definition of the fundamental matrix F

3 Recall: The projective plane Why do we need homogeneous coordinates? represent points at infinity, homographies, perspective projection, multi-view relationships What is the geometric intuition? a point in the image is a ray in projective space -y (0,0,0) -z x (sx,sy,s) (x,y,1) image plane Each point (x,y) on the plane is represented by a ray (sx,sy,s) all points on the ray are equivalent: (x, y, 1) (sx, sy, s)

4 Recall: 2D projective Geomety Projective Transformation: linear transformation that keeps lines. Projective Space: an extension of the Euclidian space where two lines always meet. Homogeneous coordinates in P 2 x = x/1 R 2 y = y/1 (x,y) = (x,y,1) = (kx,ky,k) k 0 i.e. Position Euclidian Coordinates (x,y,0) = (x/0,y/0,0) = (,,0) Point at Infinity i.e. Direction

5 Projective lines What does a line in the image correspond to in projective space? A line is a plane of rays through origin all rays (x,y,z) satisfying: ax + by + cz = 0 in vector notation : 0 = [ a b c] x y z A line is also represented as a homogeneous 3-vector l l T p

6 Point and line duality A line l is a homogeneous 3-vector It is to every point (ray) p on the line: l T p=0 l p 1 p 2 l 2 l 1 p What is the line l spanned by rays p 1 and p 2? l is to p 1 and p 2 l = p 1 p 2 l is the plane normal What is the intersection of two lines l 1 and l 2? p is to l 1 and l 2 p = l 1 l 2 Points and lines are dual in projective space given any formula, can switch the meanings of points and lines to get another formula

7 Example: Computing vanishing points (from lines) v q 1 q 2 p 2 p 1 Intersect p 1 q 1 with p 2 q 2 In practice: least squares version Better to use more than two lines and compute the closest point of intersection See notes by Bob Collins for one good way of doing this:

8 Intersection of parallel lines Intersections of parallel lines l = ( ) ( ) T a, b, c T and l'= a, b, c' l l' = ( b, a,0) T Skew matrix of l ( b,-a) ( ) a,b tangent vector normal direction Example x = 1 x = 2

9 Outline 1. Short introduction: points and lines 2. Two views geometry: Epipolar geometry Relation point/line in two views The geometry of two cameras Definition of the fundamental matrix F

10 Stereo head Camera on a mobile vehicle 10

11 Pentagon example left image right image range map 11

12 Scenarios The two images can arise from A stereo rig consisting of two cameras or the two images are acquired simultaneously A single moving camera (static scene) the two images are acquired sequentially The two scenarios are geometrically equivalent

13 The objective Given two images of a scene acquired by known cameras compute the 3D position of the scene (structure recovery) Basic principle: triangulate from corresponding image points Determine 3D point at intersection of two back-projected rays

14 Corresponding points are images of the same scene point Triangulation C C / The back-projected points generate rays which intersect at the 3D scene point 14

15 An algorithm for stereo reconstruction 1. For each point in the first image determine the corresponding point in the second image (this is a search problem) 2. For each pair of matched points determine the 3D point by triangulation (this is an estimation problem)

16 The correspondence problem Given a point x in one image find the corresponding point in the other image This appears to be a 2D search problem, but it is reduced to a 1D search by the epipolar constraint

17 General outline of 3D reconstruction 1. Epipolar geometry TODAY the geometry of two cameras reduces the correspondence problem to a line search 2. Stereo correspondence algorithms 3. Triangulation

18 Notation The two cameras are P and P /, and a 3D point X is imaged as X P P / x x / C C / Warning for equations involving homogeneous quantities = means equal up to scale

19 Epipolar geometry Given an image point in one view, where is the corresponding point in the other view?? epipolar line C epipole C / baseline A point in one view generates an epipolar line in the other view The corresponding point lies on this line

20 Epipolar line Epipolar constraint Reduces correspondence problem to 1D search along an epipolar line

21 Epipolar geometry Epipolar geometry is a consequence of the coplanarity of the camera centres and scene point X x x / C C / The camera centres, corresponding points and scene point lie in a single plane, known as the epipolar plane

22 Nomenclature left epipolar line x e X C C / e / l / right epipolar line x / The epipolar line l / is the image of the ray through x The epipole e is the point of intersection of the line joining the camera centres with the image plane " this line is the baseline for a stereo rig, and " the translation vector for a moving camera The epipole is the image of the centre of the other camera: e = PC /, e / = P / C

23 The epipolar pencil X e e / baseline As the position of the 3D point X varies, the epipolar planes rotate about the baseline. This family of planes is known as an epipolar pencil. All epipolar lines intersect at the epipole. (a pencil is a one parameter family)

24 The epipolar pencil X e e / baseline As the position of the 3D point X varies, the epipolar planes rotate about the baseline. This family of planes is known as an epipolar pencil. All epipolar lines intersect at the epipole. (a pencil is a one parameter family) Epipolar geometry depends only on the relative pose (position and orientation) and internal parameters of the two cameras, i.e. the position of the camera centres and image planes. It does not depend on the scene structure (3D points external to the camera).

25 Epipolar geometry example I: converging cameras e e / Note, epipolar lines are in general not parallel

26 Epipolar geometry example II: parallel cameras

27 Algebraic representation of epipolar geometry We know that the epipolar geometry defines a mapping x l / point in first image epipolar line in second image

28 Derivation of the algebraic expression Outline P Step 1: for a point x in the first image back project a ray with camera P P / Step 2: choose two points on the ray and project into the second image with camera P / Step 3: compute the line through the two image points using the relation l / = p x q

29 choose camera matrices internal calibration rotation translation from world to camera coordinate frame first camera world coordinate frame aligned with first camera second camera 29

30 Step 1: for a point x in the first image back project a ray with camera P A point x back projects to a ray where Z is the point s depth, since satisfies 30

31 P / Step 2: choose two points on the ray and project into the second image with camera P / Consider two points on the ray Z = 0 is the camera centre Z = is the point at infinity Project these two points into the second view 31

32 Step 3: compute the line through the two image points using the relation l / = p x q Compute the line through the points Using the identity F is the fundamental matrix F 32

33 The fundamental matrix F F is the unique 3x3 rank 2 matrix that satisfies x T Fx=0 for all x x (i)epipolar lines: l =Fx & l=f T x (ii)epipoles: on all epipolar lines, thus e T Fx=0, x e T F=0, similarly Fe=0 (iii)f has 7 d.o.f., i.e. 3x3-1(homogeneous)-1(rank2) (iv)f is a correlation, projective mapping from a point x to a line l =Fx (not a proper correlation, i.e. not invertible)

34 Example I: compute the fundamental matrix for a parallel camera stereo rig X Y Z f f λ = t x /f (but we are in homogeneous space) reduces to y = y /, i.e. raster correspondence (horizontal scan-lines) 34

35 X Y f Z f Geometric interpretation? 35

36 Example II: compute F for a forward translating camera f X Y Z f λ = t z /f (but we are in homogeneous space) 36

37 X Y Z f f first image second image 37

38 Summary: Properties of the Fundamental matrix

39 THANK YOU!