Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1
Projective 3D Geometry (Back to Chapter 2) Lecture 6 2
Singular Value Decomposition Given a square matrix A, the singular value decomposition factors A into three matrices U, D and V such that A = U D V T. In this decomposition U and V are two orthogonal matrices and D is a diagonal matrix with non-negative entries. (A matrix U is orthogonal if U T = U 1.) 3
Singular Value Decomposition To solve the matrix equation A x = 0, we use the svd function from matlab by calling [U, D, V] = svd(a) The solution x is the last column of the matrix V. 4
Projective 3D geometry In Chapter 2, we will study: Points, lines, planes and quadrics Transformations The absolute conic and absolute dual conic 5
2D Homogeneous Coordinates of Points 2D homogeneous coordinates extend the projective space from 1D to 2D. In 2D projective space, P 2 : - From 2D world space (x, y) make - 2D homogeneous coordinates (x 1, x 2, x 3 ) T - 2D projective image space (x, y ) T = ( x1 x, x ) T 2 3 x 3 6
3D Homogeneous Coordinates of Points 3D homogeneous coordinates extend the projective space from 2D to 3D: In 3D projective space, P 3 : - From 3D world space (x, y, z) make - 3D homogeneous coordinates (x 1, x 2, x 3, x 4 ) T - 3D projective image space (x, y, z ) T = ( x1 x, x 2 4 x, x ) T 3 4 x 4 7
Advantages of Homogeneous Coordinates 3D homogeneous coordinates - Unifies points and planes - (but lines are messy) - Puts perspective projection into matrix form - No divide-by-zero, points at infinity are defined. 8
Unifies points and planes (but lines are messy) Puts perspective projection into matrix form 3D Homogeneous Coordinates No divide-by-zero, points at infinity defined in R 3, write point x as x y z y (x,y,z) x z But in P 3, write same point x as: where: x = x 1 / x 4, y = x 2 / x 4, z = x 3 / x 4, x 1 x 2 x 3 x 4 x 4 = anything non-zero! (but usually defaults to 1) R 3 P 3 9
P 2 homog. coords: 2D projective map (2D point 3D ray is correct) P 3 homog. coords. 3D projective map ( But 3D point 3D ray is WRONG!) A very common mistake (x,y) (x 1, x 2, x 3 ) (x,y,z) (x 1, x 2, x 3, x 4 ) x 2 x 2 x 3 x 1 x 1 P 2 P 3 A Point in P 2 corresponds to a ray in R 3 10
Projective Transformations Definition. A projective transformation on P 3 is a linear transformation on homogeneous 4-vectors represented by a non-singular 4 4 matrix H: x 1 x 2 x 3 x 4 = h 11 h 12 h 13 h 14 h 21 h 22 h 23 h 24 h 31 h 32 h 33 h 34 h 41 h 42 h 43 h 44 x 1 x 2 x 3 x 4. 11
The matrix equation on the previous page can be written more briefly as x = H x where H is a 4 4 non-singular matrix. The projective transformation H is a collineation, that is, H maps lines on to lines. This matrix H is homogeneous, and has 15 degrees of freedom. 12
Plane Point duality The equation of a plane in R 3 is given by π 1 x + π 2 y + π 3 z + π 4 = 0. This above equation is unaffected by multiplication by a non-zero scalar k since k π 1 x + k π 2 y + k π 3 z + k π 4 = 0. Hence a plane Π in homogeneous coordinates can be written as Π = (π 1, π 2, π 3, π 4 ) T. In P 3, a plane has 3 degrees of freedom. 13
Homogenizing the equation π 1 x + π 2 y + π 3 z + π 4 = 0 by replacing x x 1 x 4, y x 2 x 4, z x 3 x 4, we obtain π 1 x 1 + π 2 x 2 + π 3 x 3 + π 4 x 4 = 0. In matrix form, this equation can be written as Π T x = 0 or x T Π = 0. These equations say that the point x is on the plane Π or x is incident with the plane Π. 14
The Euclidean representation of the plane is given by n x + d = 0 where n = (π 1, π 2, π 3 ) T, x = (x 1, x 2, x 3 ) T, x 4 = 1, π 4 = d In P 3, there is a duality between points and planes. 15
Planes from points It is known that the join of three points uniquely defines a plane. To find the plane Π through the three given points x 1, x 2 and x 3 stack the points and find Π by solving the following matrix equation: x T 1 x T 2 x T 3 Π = x 11 x 12 x 13 x 14 x 21 x 22 x 23 x 24 x 31 x 32 x 33 x 34 Π = A Π = 0. 16
If x 1, x 2 and x 3 are linearly independent, then the 3 4 matrix A has a rank of 3 and hence Π can be determined uniquely up to scale. If the matrix A has a rank of 2, then the points are collinear, and define a pencil of planes with the line of Lines in P 3 : Awkward collinear points as axis. etrically, Lines are: Pencil of planes rsection of 2 (or more) planes, axis or pencil of planes, ar combo of 2 points, p 1 + A(p 2 -p 1 ) 17
Let M = [X, X 1, X 2, X 3 ] be a 4 4 matrix compose of a general point X and three points X 1, X 2, X 3 which define the plane Π. When the point X lies on the plane Π, the determinant of M becomes zero, that is det M = x 1 D 234 x 2 D 134 + x 3 D 124 x 4 D 123 = 0, where D jkl is the determinant formed from the jkl rows of the 4 3 matrix [X 1, X 2, X 3 ]. 18
Hence the homogeneous coordinates of the plane Π can be read off from det M = x 1 D 234 x 2 D 134 + x 3 D 124 x 4 D 123 = 0 and we have Π = (D 234, D 134, D 124, D 123 ) T. This is the solution vector (that is, the null-space) of the equation A Π = 0 we have discussed earlier. 19
Planes from points It is well known that three distinct planes intersect in a unique point. To find the point of intersection x between three planes Π 1, Π 2 and Π 3 stack the planes and find x by solving the matrix equation Π T 1 Π T 2 Π T 3 x = π 11 π 12 π 13 π 14 π 21 π 22 π 23 π 24 π 31 π 32 π 33 π 34 x = B x = 0. 20
Let N = [Π, Π 1, Π 2, Π 3 ] be a 4 4 matrix compose of a general plane Π and three planes Π 1, Π 2, Π 3 which define the point x. When the plane Π is incident on the point x, the determinant of N becomes zero, that is det N = π 1 d 234 π 2 d 134 + π 3 d 124 π 4 d 123 = 0, where d jkl is the determinant formed from the jkl rows of the 4 3 matrix [Π 1, Π 2, Π 3 ]. 21
Hence the homogeneous coordinates of the point x can be read off from det N = π 1 d 234 π 2 d 134 + π 3 d 124 π 4 d 123 = 0 and we have x = (d 234, d 134, d 124, d 123 ) T. This is the solution vector (that is, the null-space) of the equation B x = 0 we have discussed earlier. 22
Projective Transformation of Planes Under the point transformation x = H x a plane Π transforms as Π = H T Π where H T denotes the transpose of inverse of the matrix H. 23
Lines in P 3 A line is defined by the join of two points or intersection of two (or more) planes. A line in P 3 has 4 degrees of freedom. (x 1, 0, x 3, 1) T (x 1, x 2, 0, 1) T The point of intersection on each plane is specified by two parameters, producing a total of 4 degrees of freedom for the line. 24
Lines are very awkward to represent in P 3 since an object with 4 degrees of freedom would be a homogeneous 5-vector. The problem is that a homogeneous 5-vector cannot easily be used in mathematical expression together with the 4-vectors representing points and planes. To overcome this problem a number of representations for lines in P 3 have been proposed. 25
Representation of Lines We examine three representations of lines using Span: Null Space of matrix W Plücker Matrix Plücker Line Coordinates 26
Lines: (Point-Point) Span W This representation is build on the geometric notion that a line is a pencil of collinear points, and is defined by any two of these points. Recall that a point x is on a plane Π if and only if x T Π = 0. 27
If two given (non-coincident) points A and B in P 3 intersect with a pencil of planes on a line, then A T B T Π = 0. Define a line in P 3 as that intersection. That is, stack A T and B T to make a 2 4 matrix W, W = A T B T = a 1 a 2 a 3 a 4 b 1 b 2 b 3 b 4 28
Then the line joining the points A and B is represented by the span of the row space of the 2 4 matrix W. Example 1. The line x-axis is represented by the span of the matrix where (0, 0, 0, 1) T W = 0 0 0 1 1 0 0 0 denotes the origin and (1, 0, 0, 0) T denotes the point at infinity in the x-direction. The span of W yields a pencil of collinear points. 29
3 Lines: (Plane-Plane) Span W This representation is build on the geometric notion that a line is the axis of a pencil of planes, and is defined by the intersection of any two planes from Lines in P 3 : Awkward the pencil. etrically, Lines are: Pencil of planes rsection of 2 (or more) planes, axis or pencil of planes, ar combo of 2 points, p 1 + A(p 2 -p 1 ) 30
If two given planes Π 1 and Π 1 in P 3 intersect at a pencil of points on a line, then x T Π 1 Π 2 = 0. Define a line in P 3 as that intersection. That is, stack Π T 1 and ΠT 1 to make a 2 4 matrix W, W = Π T 1 Π T 2 = π 11 π 12 π 13 π 14 π 21 π 22 π 23 π 24. 31
Then the line obtained by the intersection of two planes Π 1 and Π 2 is represented by the span of the row space of the 2 4 matrix W. Example 2. The line x-axis is represented by the span of the matrix W = 0 0 1 0 0 1 0 0 where (0, 0, 1, 0) T denotes the xy-plane and (0, 1, 0, 0) T denotes the xz-plane. The span of W yields an axis of a pencil of planes. 32
Lines: Plücker Matrix The line joining the two points A and B is represented by a 4 4 skew-symmetric homogeneous matrix L given by L = A B T B A T. L has rank 2 L has 4 degrees of freedom L is a generalization of l = x y L is independent of the choice of A and B Transformation: L = H L H T 33
Example 3. Let A = (0, 0, 0, 1) T be the origin, and B = (1, 0, 0, 0) T be the point at infinity (or ideal point) in the x-direction. The the line x-axis can be represented using Plücker matrix as L = 0 0 0 1 1 ( ) 1 0 0 0 0 0 0 0 0 0 1 ( ) 0 0 0 1 = 0 0 0 0 0 0 0 0 1 0 0 0. 34
Lines: Dual Plücker Matrix A dual Plücker matrix representation L is obtained for a line formed by the intersection of two planes Π 1 and Π 2 L = Π 1 Π T 2 Π 2 Π T 1 and has similar properties to L. Under the point transformation x = H x, the matrix L transforms as L = H T L H 1. 35
The matrix L can be obtained directly from L by a simple rewrite rule: l 12 : l 13 : l 14 : l 23 : l 42 : l 34 = l 34 : l 42 : l 23 : l 14 : l 13 : l 12 The correspondence rule is very simple: the indices of the dual and original component always include all the numbers {1, 2, 3, 4}, so if the original if ij then the dual is those numbers of {1, 2, 3, 4} which are not ij. For example 12 34. 36
Example 4. Given Plücker matrix representation of the line x-axis, L, we can easily find L : L = 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 L = 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 Here l 14 = 1 = l 23 and l 41 = 1 = l 32. 37
Plane defined by a line and a point The plane defined by the join of the point x and the line L is given by Π = L x. L x = 0 if and only if x is on the line L. 38
Point defined by a line and a plane The point defined by the intersection of the line L with the plane Π is given by x = L Π. L Π = 0 if and only if L is on the plane Π. 39
Example 5. The intersection of the x-axis with the plane x = 1 (in R 3 ) is given by x = L Π = 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 1 = 1 0 0 1 which is the point (1, 0, 0) T in R 3. 40
Lines: Plücker Line Coordinates Given the Plücker matrix L for a line, that is L = l 11 l 12 l 13 l 14 l 21 l 22 l 23 l 24 l 31 l 32 l 33 l 34 l 41 l 42 l 43 l 44 the Plücker line coordinates for the line L is a set of elements of the 4 4 skew-symmetric matrix L, namely L = { l 12, l 13, l 14, l 23, l 42, l 34 }. 41
L is a homogeneous 6-vector, and therefore is an element of P 5. If det L = 0, then the Plücker coordinates satisfy the equation l 12 l 34 + l 13 l 42 + l 14 l 23 = 0. A 6-vector L only corresponds to a line in P 3 if satisfies the above equation. The geometric interpretation of this constraint is that the lines of P 3 define a surface in P 5 known as the Klein quadric. 42