The reconstruction problem Both intrinsic and extr insic parameters are known: we can solve the reconstruction problem unambiguously by triangulation. Only the intrinsic parameters are known: we can solve the reconstruction problem but only up to an unknown scaling factor. Neither the extr insic nor the intrinsic parameters are available: we can solve the reconstruction problem but only up to an unknown, global projective transformation. Reconstruction by triangulation Assumptions and problem statement Both the extrinsic and intrinsic camera parameters are known. Compute the location of the 3D points from their projections p l and p r What is the solution? -The point P lies at the intersection of the two rays from O l through p l and from O r through p r.
-2- Practical difficulties -The two rays will not intersect exactly in space because of errors in the location of the corresponding features. -Find the point that is closest to both rays. -This is the midpoint P of line segment being perpendicular to both rays. Parametric line representation (review) -The parametric respresentation of the line passing through P 1 and P 2 is given by: P(t) = P 1 + t(p 2 P 1 ) or x(t) = x 1 + t(x 2 x 1 ) and y(t) = y 1 + t(y 2 y 1 ) -The direction of the line is given bythe vector t(p 2 P 1 ) t=1 t>1 t=0 0<t<1 P 2 t<0 P 1
-3- Computing P -Parametric representation of the line passing through O l and p l : O l + a(p l O l ) = ap l -Parametric representation of the line passing through O r and p r : Or L + b(pr L Or L ) = T + br T p r since Or L = R T Or R + T = T and pr L = R T pr R + T (pr R = p r,, pl L = p l ) -Suppose the endpoints P 1 and P 2 are given by P 1 = a 0 p l and P 2 = T + b 0 R T p r -The parametric equation of the line passing through P 1 and P 2 is given by: P 1 + c(p 2 P 1 ) -The desired point P (midpoint) is computed for c = 1/2
-4- Computing a 0 and b 0 -Consider the vector w orthogonal to both l and r is given by: w = (p l O l )x(pr L Or L ) = p l xr T p r (since pr R = R( pr L Or L )) -The line s going through P 1 with direction w is given by: a 0 p l + cw = a 0 p l + c(p l xr T p r ) -The lines s and r intersect at P 2 : a 0 p l + c 0 ( p l xr T p r ) = T + b 0 R T p r (assume s passes through P 2 for c = c 0 ) -Wecan obtain a 0 and b 0 by solving the following system of equations: a 0 p l b 0 R T p r + c 0 ( p l xr T p r ) = T
-5- Reconstruction up to a scale factor Assumptions and problem statement Only the intrinsic camera parameters are known. We have established n 8correspondences to compute E Compute the location of the 3D points from their projections p l and p r. Comments: We cannot recover the true scale of the viewed scene since we do not know the T baselile T (recall: Z = f d ). Reconstruction is unique only up to an unknown scaling factor. This factor can be determined if we know the distance between two points in the scene. Estimate E -Wecan estimate E using the 8-point algorithm. -The solution is unique up to an unknown scale factor. Recover T E T E = S T R T RS = S T S = T y 2 + Tz 2 T y T x T z T x T x T y Tz 2 + T x 2 T z T y T x T z T y T z T 2 x + T 2 y note: Tr(E T E) = 2 T 2 or T = Tr(E T E)/2 To simplify the recovery of T,let s divide E by T : 1 ˆT 2 x ˆT x ˆT y ˆT x ˆT z Ê T Ê = ˆT y ˆT x 1 ˆT 2 y ˆT y ˆT z where ˆT = T / T ˆT z ˆT x ˆT z ˆT y 1 ˆT 2 z (we can compute ˆT much easier from Ê T Ê)
-6- Recover R It can be shown that R i = w i + w j xw k where w i = Ê i x ˆT Ambiguity in ( ˆT, R) There is a twofold ambiguity in the sign of Ê and ˆT There will be four different estimates for ( ˆT, R) 3D reconstruction Let s compute the z coordinate of each point in the left camera frame: p l = f l P l Z l and p r = f r P r Z r = f r R(P l ˆT ) R T 3 (P l ˆT ) (since P r = R(P l T )) The first component x r of p r is: x r = f r R T 1 (P l ˆT ) R T 3 (P l ˆT ) Substiting P l = p l Z l f l into the above equation we get: Compute X l and Y l from P l = p l Z l f l Z l = f l ( f r R 1 x r R 3 ) T ˆT ( f r R 1 x r R 3 ) T p l Compute (X r, Y r, Z r )from P r = R(P l T )
-7- Algorithm Input: a set of corresponding points p l and p r 1. Estimate E 2. Recover ˆT 3. Recover R 4. Reconstruct Z l and Z r 5. If the signs of Z l and Z r are: 5.1 both negative for some point, change the sign of ˆT and goto step 4 5.2 one negative, one positive, for some point, change the sign of each entry of Ê and goto step 3. 5.3 both positive for all points, exit. Comment: the algorithm should not go through more than 4 iterations (why?)
-8- Reconstruction up to a projective transformation -Wewill not discuss this case (more complicated) -Look in the book if interested (pp. 166-170)