Multiple View Geometry in Computer Vision

Transcription

1 Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1

2 More on Single View Geometry Lecture 11 2

3 In Chapter 5 we introduced projection matrix (which is a 3 4 homogeneous matrix) as the model for the action of a camera on points. In this lecture, we will examine the action of the projection matrix on other 3D entities such as planes, lines, conics and quadrics. Further, we will develop their forward and back-projection properties. 3

4 Action of projective camera on planes Let X π = (x, y, 0, 1) T be a point on the xy-plane Π on the world coordinate system (see figure below). Suppose the camera center C is not on Π. Prospective image of points on xy-plane 4

5 Then the image x of this point X π is given by x = P X π = [ p 1 p 2 p 3 p 4 ] x y 0 1 = [ p 1 p 2 p 4 ] where p i is the i th column of P. If we denote the point (x, y, 1) T by x π, and the 3 3 matrix [ ] p 1 p 2 p 4 by H, then we have x = H x π. x y 1 5

6 So the map between points x π = (x, y, 1) T on plane Π and their image x is a general planer homography H with rank 3. Hence we have: The most general transformation that can occur between a scene plane and an image plane under prospective imaging is a plane projective transformation. Similarly, affine camera leads to affine transformation. 6

7 Action of projective camera on lines Forward Projection of lines. A line in 3-space projects to a line in the image plane. The line in 3-space and the camera center define a plane, and the image is the intersection of this plane with the image plane. 7

8 The image line l is the intersection of the planes 8

9 Let A and B be two points in 3-space, and a and b be their images under P. Then X(µ) = A + µ B is a point on the line joining the points A and B. The image of X(µ) under P is given by x(µ) = P(A + µ B) = PA + µ PB = a + µ b which is a point on the line joining a and b. lines are mapped to lines. Hence 9

10 Under the mapping P lines in 3-space are mapped to lines in image plane. A y c µb PA z c p f C x c Lines are mapped to lines. 10

11 Back-projection of lines. The back-projection of a line in image space is the set of points in 3-space. We will show that this set of point in 3-space is a plane defined by the camera center and the image line. π L L y c z c p f C Line L back-projects to plane π L 11 x c

12 Result 7.1. The set of points in space mapping to a line l via the camera matrix P is the plane P T l. Proof: Let x = P X. The point x lies on the line l x T l = 0. The point PX lies on the line l (P X) T l = 0, which is X T (P T l) = 0, an equation in P 3. Hence the 4-vector P T l is a plane in P 3. Thus lines l are back-projected to planes P T l. 12

13 The plane π defined by the intersection of the line L and the camera center C is the pull-back of the line l on the image plane. The pull-back of the plane π 13

14 Some facts about matrix [l] Given a 3-vector l = (l 1, l 2, l 3 ) T, the skew-symmetric matrix [l] of l is defined as [l] = a b = [a] b. det([l] ) = 0. 0 l 3 l 2 l 3 0 l 1 l 2 l 1 0. The l is the right as well as left null space of [l]. Any skew-symmetric matrix can be written as [a] for some vector a. 14

15 Result 7.2. Under the camera mapping P, a line in 3-space represented as a Plücker matrix L is imaged as the line l where [l] = PLP T. Proof: Let A and B be two points in 3-space, and a and b be their images under P. The Plücker matrix L for the line through the points A and B in 3-space is given by L = A B T B A T. 15

16 Consider the matrix M = P L P T. Then the matrix M = P L P T = P ( A B T B A T) P T = P A B T P T P B A T P T = a (P B) T b (P A) T = a b T b a T is 3 3 and antisymmetric, with null-space a b. Therefore M = [a b]. 16

17 Since the line l in the image plane is given by l = a b, we have M = [a b] = [l], where [l] = 0 l 3 l 2 l 3 0 l 1 l 2 l 1 0 and l = (l 1, l 2, l 3 ) T. 17

18 Action of projective camera on conics Back-projection of conics. A conic C back-projects to a cone. The cone vertex, which is the camera center, is the the null-vector of the quadric matrix C. Recall that a cone is a degenerate quadric, that is a homogeneous 4 4 matrix with rank less than 4. 18

19 C y c z c p f V x c A conic C back-projects to a cone. 19

20 Result 7.3. Under the camera P the conic C backprojects to the cone Q co = P T C P. Proof: Let x = P X. The point x lies on the conic C x T C x = 0. The point PX lies on C (P X) T C P X = 0, which is X T (P T C P) X = 0, an equation in P 3. Hence the 4 4 matrix P T C P is a quadric in P 3. Thus conics C are back-projected to quartics P T C P. 20

21 Note that the camera center C is the vertex of the degenerate quadric since Q co C = (P T C P) C = P T C (P C) = 0. Example 1. Suppose P = K [I 0]. Then the conic C back-projects to the cone quadric Q co = P T C P = KT 0 T C K [I 0] = KT CK 0 0 T 0. The matrix Q co has rank 3. Its null-vector is the camera center C = (0, 0, 0, 1) T. 21

22 Images of smooth surfaces The contour generator Γ is the set of points X on the smooth surface S at which rays are tangent to the surface. The corresponding image apparent contour γ is the set of points x which are the image of X, that is γ is the image of Γ. 22

23 The apparent contour is also called the outline or profile. (a) parallel projection (b) central projection Contour generator and apparent contour. 23

24 Action of projective camera on quadrics A quadric is a smooth surface and so its outline curve is given by points where the back-projected rays are tangent to the quadric surface as shown below. A cone of rays for a quadric. 24

25 If the quadric is a sphere, then the cone of rays between the camera center and quadric is right-circular. That is the contour generator is a circle. 25

26 Forward projection of quadrics. Since the outline or apparent contour arises from tangency, the dual of the quadric, Q, is important since it defines the tangent planes to the quadric Q. Recall that the dual quadric are equations on planes: The tangent planes Π to the point quadric Q satisfy Π T Q Π = 0. 26

27 Result 7.4. Under the camera matrix P the apparent contour of the quadric Q is the conic C given by Proof: C = P Q P T. Observe that lines l tangent to a conic outline satisfy l T C l = 0. These lines l back-project to planes Π = P T l. The planes Π = P T l are tangent to the quadric Q. 27

28 Hence they satisfy Π T Q Π = 0. Thus for each line l we have 0 = Π T Q Π = (P T l) T Q P T l = l T C l where C = P Q P T. Since this is true for all lines l tangent to C the outline of Q is the conic C. 28

29 A dual quadric Q in world implies a dual conic C in image plane. C* Q* y c z c p f V x c A dual quadric Q a dual conic C 29

30 Result 7.5. The cone with vertex V and tangent to the quadric Q is the degenerate quadric given by Q co = (V T QV (QV)(QV) T. Q Q co y c z c p f V x c A quadric Q a cone Q co 30

31 Importance of camera center An object in 3-space and camera center define a set of rays, and an image is obtained by intersecting these rays with a plane. The set of rays is called the cone of rays. The cone of rays with vertex at the camera center. 31

32 If the cone of rays is intercepted by two planes, then two images, I and I, are related by a perspective map. This means that images obtained with the same camera center may be mapped to one another by a plane projective transformation. 32

33 Consider two camera P = KR [ I C ] and P = K R [ I C ] ( ) with common center C. Here C is the coordinates of C in world frame. From ( ), we have P = K R [ I C ] = K R (KR) 1 P. The images of a 3-space point X by two camera is related as x = P X = K R (KR) 1 PX = K R (KR) 1 x. 33

34 Therefore, the corresponding image points x and x are related by a planner homography as x = H x, where H = K R (K R) 1, a 3 3 matrix. 34

35 An increase in focal length produces a displacement of the image plane along the principal axis. The image effect is a simple magnification. If x and x are images of a point X before and after zooming, then x = K [ I 0 ] X and x = K [ I 0 ] X, where K = f 0 p x 0 f p y and K = f 0 p x 0 f p y

36 It is easy to see that x = K [ I 0 ] X = K K 1 x = H x, where H := K K 1. Hence K K 1 = f 0 p x 0 f p y f 0 p x f 0 1 f p y f = f f 0 p x ( 0 f f p y ( f f 1 f f ) ). 36

37 Letting k = f f, we have K K 1 = k 0 p x (1 k) 0 k p y (1 k) = k I (1 k) x 0 0 T 1, where x 0 = (p x, p y ) T is the inhomogeneous coordinates of the principal point. Hence K = k I (1 k) x 0 0 T 1 A x 0 0 T 1 = K k I 0 T 0 T 1. 37

38 Moving the image plane Hence we have the following: The effect of zooming by a factor k is to multiply the calibration matrix K on the right by diag(k, k, 1). 38

39 END 39