Calculation of the fundamental matrix

Transcription

1 D reconstruction alculation of the fundamental matrix Given a set of point correspondences x i x i we wish to reconstruct the world points X i and the cameras P, P such that x i = PX i, x i = P X i, i Without any additional information it is possible to 1 Estimate the fundamental matrix from point correspondences alculate camera pairs from the fundamental matrix Estimate world coordinates X i corresponding to each point pair x i x i The reconstructed cameras and points will be unique up to a projective homography of P Depending on which information we have about the world andor the cameras (point coordinates, orthogonal line pairs, parallel line pairs, calibrated cameras, etc) we may either perform a stratified reconstruction first projectively, then affinely, the metrically, or directly perform a metric reconstruction The defining equation for the fundamental matrix is x Fx = for each pair of corresponding points x x Given enough corresponding points we may calculated F Each point pair produces one equation linear in the elements in F If x = (x, y, 1) and x = (x, y, 1) then x xf 11 +x yf 1 +x f 1 +y xf 1 +y yf +y f +xf 1 +yf +f =, that may be written as [ x x x y x y x y y y x y 1 where f is the 9-vector with elements F row-wise ] f =, p 1 alculation of the fundamental matrixf Given n corresponding points we get a linear equation system on the form Af = x 1x 1 x 1y 1 x 1 y 1x 1 y 1y 1 y 1 x 1 y 1 1 x nx n x ny n x n y nx n y ny n y n x n y n 1 5 f = The equation is homogenous, ie f can only be detered up to scale In order for a homogenous solution to exist, the rank of A cannot be larger than 8 In that case a solution f exists in the null-space of A A solution may always be detered by solving f Af with solution f = v 9 where A = UDV st f = 1, Minimal correspondence the -point algorit If the matrix A is constructed from n = correspondences, A will have a two-dimensional null-space Let the vectors f 1 and f be basis vectors for the null-space of A Then each vector f = αf 1 + (1 α)f is a solution of Af = the corresponding F-matrices are F = αf 1 + (1 α)f The condition det(f) = leads to the equation det(αf 1 + (1 α)f ) =, that is a third order equation in α with one or three real roots, ie one or three solutions are possible p

2 The -point algorithm, solutions The 8-point algorithm The easiest way to calculate the fundamental matrix is to use n 8 points and 1 alculate F that imizes the algebraic error, ie solves f Af st f = 1 Find the closest matrix F of rank, ie solve If A = UDV is the singular value decomposition of A, then the solution of problem 1 is f = v 9 If F = UDV is the singular value decomposition of F, where then D = diag(r,s, t),r s t, F F F F st rank(f ) =, where F is the Frobenius norm F = U diag(r,s, )V is a solution to problem p 5 Non rank-deficientfmatrix If F has full rank it will have an empty null-space, ie not have any point that is on all lines, ie no epipole rank(f) = rank(f) = The normalized 8-point algorithm In order for the 8-point algorithm to work in practice, it has to be normalized The algorithm then becomes: 1 Detere a transformationtand T such that ˆx i = Tx i and ˆx i = Tx i has center of gravity at the origin and a mean squared distance of from the origin alculate a fundamental matrix ˆF corresponding to the point pairs ˆx i ˆx i : (a) alculate ˆF from the solution ˆf that solves ˆf st ˆf = 1, where  is calculated from the point pairs ˆx h ˆx i (b) alculate ˆF that solves ˆF ˆF ˆF F st rank(ˆf ) = p alculatef = T ˆF T as the fundamental matrix corresponding to the original point pairs x i x i

3 OptimalF OptimalF In order to detere an optimal F, the reprojection error has to be imized This may be achieved through eg the following problem formulation: where u m d(x i, ˆx i ) + d(x i, ˆx i) i=1 st ˆx i Fˆx i =, i = 1,,m F F = 1, det(f) =, u = [f 1,,f 9, ˆx 1x, ˆx 1y,, ˆx mx, ˆx my, ˆx 1x, ˆx 1y,, ˆx mx, ˆx my] Given an algorithm to solve we get f(u) = ˆx 1 x 1 ˆx m x m ˆx 1 x 1 ˆx m x m 1 u f(u)t Wf(u) st c(u) = and c(u) = 5 ˆx 1 Fˆx 1 ˆx mfˆx m F F 1 det(f) 5 A starting approximation of F can be calculated by the normalized 8-point algorithm The initial estimates of the line points ˆx i and ˆx i are the corresponding measured points x i and x i p 9 OptimalF, normal example OptimalF, normal example res rms=85 line rms=8e 1 Starting approximation x 1 norm constraint=e 1 x 1 det constraint= 5e 1 One iteration 1 1 Solution Iteration sequence p 11

4 Automatic calculation off Example By using RANSA we may estimate F automatically Given n preliary point matches, a probability p and a distance threshold t: Draw point pairs randomly alculate a fundamental matrix from the point pairs We have 1 or solutions For each of the solutions alculate the distance d i between each point pair and the corresponding epipolar lines alculate the number of point pairs k i that are inliers, ie with distance d i < t If k i > k best or k i = k best and σ(d i ) < σ best then best = i, ɛ = k best (1 n), N max = i = i + 1 Repeat until i N max log 1 p log(1 (1 ɛ) ) Then refine the best solution by imizing the reprojection error Add correspondences that now satisfies d i < t Optionally repeat the previous step Suggested matches After calculation of F with RANSA After manual cleaning p 1 lculation of D coordinates, homogenous solution alculation of D coordinates, homogenous solutio Given a corresponding point pair x x and two camerasp,p, calculate the corresponding D point X such that x = PX and x = P X As previously with homography calculations we may dehomogenize the image points in eg image 1 and get x (PX) = or The equation systemax = is overdetered in the sense that X has degrees of freedom, but we have independent equations If the points x and x do not correspond exactly, X = will be the only solution x(p X) (p 1 X) = y(p X) (p X) = x(p X) y(p 1 X) = x Of these, only two are linearly independent If we remove the third and combine with the point in image we get AX =, where A = xp p 1 yp p x p p 1 y p p 5 l = F x l = F x e e image 1 image We thus have to solve AX = approximately, either via SVD A = UDV = A, X = v or by specifying X = (X, Y, Z, 1) p 15

5 alculation of D coordinates, Euclidian solution Minimization of the reprojection error We may also choose to imize the Euclidian distance in R If camera 1 has center in and we know another point Ũ on the line U = P+ x, then a point X on the line satisfies The optimal solution is obtained if we imize the reprojection error, ie solves X Similarly in image X = + α(ũ ) X = + α (Ũ ), where U = P + x d(x, ˆx,ˆx,X ˆx) + d(x, ˆx ) st ˆx = PX ˆx = P X x d d e e If the lines do not intersect there is no point X that satisfies both equations However, then we may imize the distance to the lines α,α, X # "Ũ α " I Ũ α 5 I X This method is called forward intersection in the photogrammetric community # which exactly corresponds to d(x, ˆx,ˆx ˆx) + d(x, ˆx ) st ˆx Fˆx = If we have detered F by imizing the reprojection error, the corresponding points have already been detered and the SVD solution of AX = gives the nullspace solution p 1 Example Starting approximate from the normalized 8-point algorithm, via the essential matrix and forward intersection One iteration Solution p 19