Stereo Observation Models

Transcription

1 Stereo Observation Models Gabe Sibley June 16, 2003 Abstract This technical report describes general stereo vision triangulation and linearized error modeling. 0.1 Standard Model Equations If the relative position and orientation of two cameras are known, then the 3D position (X, Y, Z) of a point P, can be reconstructed from the perspective projection of P on the image planes of two cameras. The simplest case arise when the optical axes of two cameras are parallel, and the translation of the right camera is only along the X axis. Consider the geometry in figure 1, known as the standard model, or fronto-parallel configuration. If C 1 Figure 1: Standard model of stereo triangulation for axis aligned cameras. and C 2 are two pinhole cameras with parallel optical axes, f is the focal length of both cameras, and b is the distance between the two lens centers, then the equations of stereo triangulation are P = P x P y = (b f)/(x 1 x 2 ) x 1 P x /f (1) P z y 1 P x /f These basic equations form the intuition behind stereo triangulation. It is common to choose the 3D world reference system to be the left camera reference system. 0.2 Non-Parallel Camera Stereo Triangulation Let X 1 and X 2 be the 2D projections of a 3D point P in two cameras. Stereo triangulation is the process of calculating the position of P given X 1 and X 2.

2 Figure 2 shows a geometric model - the CAHV model used by NASA-JPL [2]- describing this imaging process with two cameras that are in the same global coordinate frame. Figure 2: Two camera geometry for imaging a 3D point P. This is the linear camera model known as CAHV. C is the 3D pinhole center of focus. A is a unit vector normal to the sensor plane (but not necessarily the same as the lens optical axis). H is known as the horizontal information vector and V is the vertical information vector. H and V need not be unit vectors and are not necessarily perpendicular to A. (h c, v c ) specifies the image sensor center in pixels. Units of pixels are related to global units via the scale factors h s and v s. In this paper unless otherwise stated image sensor locations, X = (x, y), will be given in global units. Given a 3D point, P, the geometery of the CAHV model tells us where in the image plane P will project to: x = y = (P C) H (P C) A (P C) V (P C) A (2) Inversely, given a pixel location, X = (x, y), we can use the camera models to calculate a 3D ray r such that all points on r project to x - i.e. X projects out into 3D along the unit ray r. There are many different cammera models one can use, such as Tsai s, the JPL CAHV* camera models, Brown s original etc. The history of camera models is interesting and important - indeed stereo triangulation is only as good as the camera models it relies on [1]. In general, all camera models can be used to define unit rays of projection.

3 We find the ray formula by rearranging (2) and noting that, (P C) (xa H) = 0 (P C) (ya V ) = 0 In other words (xa H) and (ya V ) are perpendicular to (P C), so we can get r from their cross product: Evaluating (3) at X 1 = (x 1, y 1 ) and X 2 = (x 2, y 2 ) gives unit rays r 1 and r 2. The distance along these rays from P to the sensor planes is λ 1 = P 1 c 1, λ 2 = P 2 c 2. Ideally the rays (c 1 + λ 1 r 1 ) and (c 2 + λ 2 r 2 ) will intersect at P. However, as shown in Figure 3, due to errors both in the model of the imaging process and in the sensor itself, these rays will often only come close to intersecting. If P 1 and P 2 are the closest points on r 1 and r 2 then P 2 P 1 will be perpendicular to both r 1 and r 2. Thus, Solving for λ 1 and λ 2 r = (xa H) (ya V ) (3) P 1 = λ 1 r 1 + c 1 P 2 = λ 2 r 2 + c 2 Figure 3: Calculate P as the midpoint of P 1 P 2, since (c 1 + λ 1 r 1 ) and (c 2 + λ 2 r 2 ) do not intersect (P 2 P 1 ) r 1 = (c 2 c 1 + r 2 λ 2 r 1 λ 1 ) r 1 = 0 (P 2 P 1 ) r 2 = (c 2 c 1 + r 2 λ 2 r 1 λ 1 ) r 2 = 0 λ 1 = b r1 (b r2)(r1 r2) 1 (r 1 r 2) 2 Finally the observation function is simply: λ 2 = (r 1 r 2 )λ 1 b r 2 (4) f(x 1, y 1, x 2, y 2 ) = P = (P 1 + P 2 )/2 = (c 1 + c 2 + λ 1 r 1 + λ 2 r 2 )/2 (5) Modeling Error Uncertainty in image location is modeled as Gaussian. Standard error propagation techniques (i.e. linearization) can be used to approximate the 3D covariance around P: [ ] Σ1 0 Σ 3D = JΣ x J T, Σ x = (6) 0 Σ 2

4 where Σ 1 and Σ 2 are pixel covariance matrices for each image and J is the Jacobian of f(x). Note that when correlation techniques (such as Sum Squared Difference - SSD) are used to find the point correspondences between images, one can estimate Σ 1 and Σ 2 based on the curve of the correlation surface that these methods define. Alternatively, it is also common to assume that pixel variance is uncorrelated (no off-diagonal elements in Σ 1 and Σ 2 ) and to use half pixel standard deviations for the diagonal elements. To compute the 3D covariance we need the Jacobian of (5): where J = f(x) (x 1, y 1, x 2, y 2 ) = (λ 1 r 1 + r 1 λ 1 + λ 2 r 2 + r 2 λ 2 )/2 (7) [ λ 1 = b r1 (b r 2)(r 1 r 2) (x) = u(x)v(x) v(x)u(x) v(x) 2 λ 2 = (r2 T r 1 + r1 T r 2)λ 1 + (r1 Tr (8) 2) λ 1 b T r 2 1 (r 1 r 2) 2 ] where u(x), v(x), u(x) and v(x) are: u(x) = [b r 1 (b r 2 )(r 1 r 2 )], u(x) = [b T r 1 (b T r 2 )(r 1 r 2 ) (b r 2 )(r T 2 r 1 + r T 1 r 2 )] v(x) = [1 (r 1 r 2 ) 2 ], v(x) = [ 2(r 1 r 2 )(r T 2 r 1 + r T 1 r 2)] since for vectors r 1 r 2 = r T 1 r 2 and r 1 r 2 = r 2 r 1. Also, note that λ 1 and λ 2 are scalars, λ 1 and λ 2 are [1 4] matrices, and the gradients of the ray equations are the [3 4] matrices: r 1 = r 1x r 1y r 1z r 1x r 1y r 1z, r 2 = r 2x r 2y r 2z r 2x r 2y r 2z (9) Recall that for the cross product d dt [r 1(t) r 2 (t)] = r 1 (t) dr2 dt + r 2 (t) dr1 Thus, A y V z A z V y r = (xa H) A + (ya V ) A = A z V x A x V z A x V y A y V x Finally, using (3), (4), (8) and (9) we can compute (7), the Jacobian. References [1] T.A. Clarke and J.G. Fryer. The development of camera calibration methods and models. Photogrammetric Record, 16(91):51 66, April dt.

5 [2] Y. Yakimovsky and R. Cunningham. A system for extracting threedimensional measurements from a stereo pair of tv cameras. In Computer Graphics and Image Processing, volume 7, pages , 1978.