3D Geometr and Camera Calibration
3D Coordinate Sstems Right-handed vs. left-handed
2D Coordinate Sstems ais up vs. ais down Origin at center vs. corner Will often write (u, v) for image coordinates v u v u v u
3D Geometr Basics 3D points = column vectors Transformations = pre-multiplied matrices = p = i h g f e d c b a Tp
Rotation Rotation about the ais R cosθ = sin θ sin θ cosθ Rotation about, aes similar (cclicall permute,, )
Arbitrar Rotation An rotation is a composition of rotations about,, and Composition of transformations = matri multiplication (watch the order!) Result: orthonormal matri Each row, column has unit length Dot product of rows or columns = Inverse of matri = transpose
Arbitrar Rotation Rotate around,, then : sin θ cosθ R = sin θ sin θ cosθ cosθ sin θ + sin θ cosθ cosθ cosθ cosθ + sin θ cosθ sin θ sin θ sin θ sin θ sin θ + cosθ cosθ cosθ sin θ cosθ + cosθ cosθ sin θ cosθ sin θ Don t do this! It s probabl bugg! Compute simple matrices and multipl them
Scale Scale in,, : S = s s s
Shear Shear parallel to plane: σ = σ σ
Translation Can translation be represented b multipling b a 3 3 matri? No. Proof: A : A =
Homogeneous Coordinates Add a fourth dimension to each point: To get real (3D) coordinates, divide b w: w w w w w
Translation in Homogeneous Coordinates After divide b w, this is just a translation b (t, t, t ) + + + = w w t w t w t w t t t
Perspective Projection What does 4 th row of matri do? After divide, = w
Perspective Projection This is projection onto the = plane (,,) (/,/,) (,,) = Add scaling, etc. pinhole camera model
Putting It All Together: A Camera Model Translate to image center Then perform homogeneous divide, and get (u,v) coords Scale to piel sie T img S pi Perspective projection P cam R cam Camera orientation T cam Camera location 3D point (homogeneous coords)
Putting It All Together: A Camera Model Intrinsics Etrinsics T img S pi P cam R cam T cam
Putting It All Together: A Camera Model Camera coordinates Normalied device coordinates Ee coordinates Image coordinates Piel coordinates T img S pi P cam R cam T cam World coordinates
More General Camera Model Multipl all these matrices together Don t care about after transformation Scale ambiguit free parameters + + + + + + + + + + + + l k j i h g f e l k j i d c b a l k j i h g f e d c b a homogeneous divide
Radial Distortion Radial distortion can not be represented b matri u img c u + u * img ( ) * 2 * 2 + κ ( u + v ) img img v img c v + v * img ( ) * 2 * 2 + κ ( u + v ) img img (c u, c v ) is image center, u * img= u img c u, v * img= v img c v, κ is first-order radial distortion coefficient
Camera Calibration Determining values for camera parameters Necessar for an algorithm that requires 3D 2D mapping Method used depends on: What data is available Intrinsics onl vs. etrinsics onl vs. both Form of camera model
Camera Calibration Eample Given: 3D 2D correspondences General perspective camera model (-parameter, no radial distortion) Write equations: a i e i + b + j + + f j + c + k + g + k + d + l + h + l = = u v
Camera Calibration Eample Linear equation Overconstrained (more equations than unknowns) Underconstrained (rank deficient matri an multiple of a solution, including, is also a solution) = l c b a v v v v u u u u v v v v u u u u 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
Camera Calibration Eample Standard linear least squares methods for A= will give the solution = Instead, look for a solution with = That is, minimie A 2 subject to 2 =
Camera Calibration Eample Minimie A 2 subject to 2 = A 2 = (A) T (A) = ( T A T )(A) = T (A T A) Epand in terms of eigenvectors of A T A: = µ e + µ 2 e 2 + T (A T A) = λ µ 2 +λ 2 µ 22 + 2 = µ 2 +µ 22 +
Camera Calibration Eample To minimie λ µ 2 +λ 2 µ 22 + subject to µ 2 +µ 22 + = set µ min = and all other µ i = Thus, least squares solution is eigenvector of A T A corresponding to minimum (nonero) eigenvalue
Camera Calibration Eample 2 Incorporating additional constraints into camera model No shear, no scale (rigid-bod motion) Square piels etc. These impose nonlinear constraints on camera parameters
Camera Calibration Eample 2 Option : solve for general perspective model, then find closest solution that satisfies constraints Option 2: constrained nonlinear least squares Usuall gradient descent techniques Common implementations available (e.g. Matlab optimiation toolbo)
Camera Calibration Eample 3 Incorporating radial distortion Option : Find distortion first (straight lines in calibration target) Warp image to eliminate distortion Run (simpler) perspective calibration Option 2: nonlinear least squares
Camera Calibration Eample 4 What if 3D points are not known? Structure from motion problem! As we saw, can often be solved since # of knowns > # of unknowns After Thanksgiving
Multi-Camera Geometr Epipolar geometr relationship between observed positions of points in multiple cameras Assume: 2 cameras Known intrinsics and etrinsics
Epipolar Geometr P p p 2 C C 2
Epipolar Geometr P l 2 p p 2 C C 2
Epipolar Geometr P Epipolar line l 2 p p 2 C C 2 Epipoles
Epipolar Geometr Goal: derive equation for l 2 Observation: P, C, C 2 determine a plane P l 2 p p 2 C C 2
Epipolar Geometr Work in coordinate frame of C Normal of plane is T Rp 2, where T is relative translation, R is relative rotation P l 2 p p 2 C C 2
Epipolar Geometr p is perpendicular to this normal: p (T Rp 2 ) = P l 2 p p 2 C C 2
Epipolar Geometr Write cross product as matri multiplication P C C 2 p 2 p l 2 = =, * * T T T T T T T T T
Epipolar Geometr p T * R p 2 = p T E p 2 = E is the essential matri P l 2 p p 2 C C 2
Essential Matri E depends onl on camera geometr Given E, can derive equation for line l 2 P l 2 p p 2 C C 2
Fundamental Matri Can define fundamental matri F analogousl, operating on piel coordinates instead of camera coordinates u T F u 2 = Advantage: can sometimes estimate F without knowing camera calibration