CSE 252B: Computer Vision II

Transcription

1 CSE 5B: Computer Vision II Lecturer: Ben Ochoa Scribes: San Nguyen, Ben Reineman, Arturo Flores LECTURE Guest Lecture.. Brief bio Ben Ochoa received the B.S., M.S., and Ph.D. degrees in electrical engineering from the University of California, San Diego, in 999, 3, and 7, respectively. Since 8, he has been the vice president of research and development at d3. For more information, see ~bochoa/... Review of linear algebra... Matrix Rank The rank of a matrix is the number of linearly independent rows or columns of the matrix. Given a matrix A R m n, its rank r min(m, n) is calculated as the number of nonzero singular values of A. Note that rank(a) = implies A is the matrix. Department of Computer Science and Engineering, University of California, San Diego. April 3, 9

2 SERGE BELONGIE, CSE 5B: COMPUTER VISION II... Singular Value Decomposition (SVD) The SVD of A R m n is defined as (.) A = UDV T, where U R m m, D = diag ( σ, σ,..., σ min(m,n) ) R m n, and V R n n. U and V are orthogonal matrices, and σ, σ,..., σ min(m,n) are called the singular values of A. Typically, and assumed in this lecture, the singular values are ordered in descending order...3. Null Spaces of A Right null space Left null space AX = Y A = where A R m n. Here, X R n (n r) is called the (right) null space of A and Y R (m r) m is called the left null space of A, where r = rank(a). The right and left null spaces of a matrix A can be calculated using SVD as follows. From A = UDV T, the right null space X = null(a) is the matrix composed of the columns of V that correspond to singular values that are equal to zero. Similarly, the left null space Y of A is the matrix composed of the rows of U T that correspond to singular values equal to zero. For example, this can be used to compute the epipoles from the fundamental matrix F. The SVD of F = UDV T where F, U, D, V R 3 3 and rank(f) =. For the epipole in first image (e ) we have (.) Fe =, so e is the last column of V. For the epipole in second image (e ) we have (.3) (.4) F T e = e T F = T, hence e T is the last row of U T (or e is the last column of U)..3. Configurations of some geometric primitives.3.. Configurations of points and lines in D Consider the homogeneous coordinates of a point x = (x, y, w) T and line l = (a, b, c) T. If x lies on l, then (.5) (.6) x T l = l T x =. Also, recall that homogeneous coordinates are only determined up to scale.

3 (.7) A = LECTURE. GUEST LECTURE D Points. Given a set of n points, we can arrange them into a single matrix as x T x T. x nt Rn 3. The rank and null space of A determine the configuration of the set of points. The following table shows the different possible configurations and the corresponding rank and null space of A. Illustration Configuration rank(a) Columns of null(a) general position 3 empty... collinear line that points lie on coincident two lines intersecting at points.3... D Lines. Similarly, given a set of n lines, we can arrange them into a single matrix as l T l T (.8) A =. Rn 3. l nt The rank and null space of A determine the configuration of the set of lines. The following table shows the different possible configurations and the corresponding rank and null space of A. Illustration Configuration rank(a) Columns of null(a) general position 3 empty pencils of lines point at intersection of lines coincident two points on lines.3.. Configurations of points and planes in 3D Consider a point X = (X, Y, Z, T) T and plane P = (a, b, c, d) T. If X lies on P then (.9) (.) X T P = P T X =

4 4 SERGE BELONGIE, CSE 5B: COMPUTER VISION II D Points. Given a set of n points, we can arrange them into a single matrix as X T X T (.) A =. Rn 4. X nt The rank and null space of A determine the configuration of the set of points. The following table shows the different possible configurations and the corresponding rank and null space of A. Illustration Configuration rank(a) Columns of null(a) See figure general position 4 empty See figure coplanar 3 plane that points lie on See figure 3 collinear two planes intersecting at line that points lie on See figure 4 coincident three planes intersecting at points D Planes. Similarly, given a set of n planes, we can arrange them into a single matrix as P T P T (.) A =. P nt Rn 4. The rank and null space of A determine the configuration of the set of planes. The following table shows the different possible configurations and the corresponding rank and null space of A. Illustration Configuration rank(a) Columns of null(a) See figure 5 general position 4 empty See figure 4 bundle of planes 3 point at intersection of planes See figure 3 sheaf of planes two points on 3D line at intersection of planes See figure coincident three points on planes.4. An application of the left null space The (right) null space has many applications in computer vision, but do not overlook the left null space. For example, consider two non-coincident D lines l and l. The point x at the intersection of l and l can be calculated

5 LECTURE. GUEST LECTURE 5 using the cross product by x = l l. Alternatively, x can be calculated by taking the null space of a matrix whose rows are the lines, ] l T (.3) x =. l T However, this equation can also be used to determine two lines l and l that intersect at x. The two lines are simply the rows of the left null space of x. As shown previously, the left null space of x can be calculated using SVD. x = UDV T, where U = u u u 3 ] R 3 3, D = (σ,, ) T, and V T is a scalar. As such, the left null space of x is u u 3 ] T = l l ] T..4.. Householder matrix Although SVD can be used to calculate the left null space of a vector x R n, a much less computationally expensive method involves determining an orthogonal matrix H R n n such that (.4) Hx = ± x. Such a matrix H is called a Householder matrix and is calculated by (.5) H = I vvt v T v where (.6) v = x + sign (x ) x e. Here e = (,,,..., ) T, not the epipole in the first image, and x is a general vector, not a D point in homogeneous coordinates. (.7) (.8) a T x] Hx = ] x = ± x. ± x ], where a T is the first row of H and x] is the matrix containing the remaining rows of H. From this, we see that a T x = ± x and (.9) x] x =.

6 6 SERGE BELONGIE, CSE 5B: COMPUTER VISION II That is, x] is the left null space of x, where x] is the matrix H with its first row omitted..4.. Points, lines, and planes that lie on and pass through The left null space of points and lines in D and points and planes in 3D provides some useful results Two lines through a D point. (.) (.) x] x = ] l T x = l T.4... Two points on a D line. (.) (.3) l] l = ] x T x T l = Three planes through a 3D point. (.4) (.5) X] X = P T P T X = P 3T Three points on a 3D plane. (.6) (.7) P] P = X T X T P = X 3T.5. Direct linear transformation algorithm Using the methods from this lecture, we can reformulate some of the equations that describe different mappings in computer vision. This is useful when applying the direct linear transformation (DLT) algorithm to estimate these mappings.

7 .5.. Planar homography LECTURE. GUEST LECTURE 7 The following notation will be used for the planar homography matrix H. (.8) (.9) H = H = h h h 3 h h h 3 h 3 h 3 h 33 h T h T h 3T where h it = (h i, h i, h i3 ), i.e., h it is the ith row of H. Further, let us denote (.3) (.3) (.3).5... D points. (.33) (.34) (.35) (.36) (.37) (.38) (.39) (.4) (.4), h = (h, h, h 3, h, h, h 3, h 3, h 3, h 33 ) T h h = h h 3 h = vec(h T ). l i T l i T a i b i c i a i b i c i a i h T x i + bi h T x i + c h3t x i a i h T x i + bi h T x i + ci h 3T x i a i x it b i x it c i x it a i x it b i x it c i x it x i Hx i ] x i Hx i = ] h T h T x i h 3T = ] h T x i h T x i = h 3T x i ] h h h 3 l i T x it l i T x it ( l i T l i T ( x i ] x it ] = = ] h = ) h = ] x it )h =

8 8 SERGE BELONGIE, CSE 5B: COMPUTER VISION II The last equation is each of the two rows in design matrix used for estimating h D lines. (.4) (.43) (.44) (.45) l j H T l j l j H T l j l j ] H T l j = ( l jt ] l j ) h = D points and D lines. Sets of corresponding points and lines can be combined to estimate the planar homography. (.46) ( ) x i ] x it ( l jt l j ] ) h =.5.. 3D projective transformation The equations for points and hyperplanes in D scale to 3D. In 3D, H R 4 4 and h = vec(h T ) R 6. Each point of plane correspondence generates three rows in the design matrix D points. (.47) (.48) (.49) D planes. (.5) (.5) (.5) (.53) X i HX i ] X i HX i = ( X ] ) i X it h = P j H T P j P j H T P j P j ] H T P j = ( P jt P j ] ) h = D points and 3D planes. ( X ] ) i X it (.54) ( P jt ] P j ) h =

9 .6. Additional figures LECTURE. GUEST LECTURE 9 These figures are illustration for the various configurations of 3D points and planes. Some of them were generated using the Householder matrix method. For example, 3D coincident points or bundle of planes (figure 4) was generated using the following matlab code: % random 3d point (homogeneous coordinates ) x = randn (3,); ] ; v = x + sign (x ()) norm(x ) ; ; ; ] ; h = eye(4) (v v )/( v v ) ; plot3 (x (), x (), x (3),. r, MarkerSize,4); hold on ; drawplane (h (, : ) ) ; drawplane (h ( 3, : ) ) ; drawplane (h ( 4, : ) ) ; where drawplane is the following function function drawplane ( p) % p is a b c d ] where % a x + b y + c z + d = defines the plane x= 5:.5:5; X,Y] = meshgrid (x ) ; a=p (); b=p (); c=p (3); d=p (4); Z=( d a X b Y)/ c ; mesh(x,y,z) ; shading f l a t ;

10 SERGE BELONGIE, CSE 5B: COMPUTER VISION II Figure. 3D points in general position Figure. 3D points in coplanar position, also coincident planes.

11 LECTURE. GUEST LECTURE Figure 3. 3D points in collinear position, also a sheaf of planes. 5 5 Figure 4. 3D coincident points, also a bundle of planes.

12 SERGE BELONGIE, CSE 5B: COMPUTER VISION II Figure 5. Planes in general position