3D Geometry and Camera Calibration

Transcription

1 3D Geometr and Camera Calibration

2 3D Coordinate Sstems Right-handed vs. left-handed

3 2D Coordinate Sstems ais up vs. ais down Origin at center vs. corner Will often write (u, v) for image coordinates v u v u v u

4 3D Geometr Basics 3D points = column vectors Transformations = pre-multiplied matrices = p = i h g f e d c b a Tp

5 Rotation Rotation about the ais R cosθ = sin θ sin θ cosθ Rotation about, aes similar (cclicall permute,, )

6 Arbitrar Rotation An rotation is a composition of rotations about,, and Composition of transformations = matri multiplication (watch the order!) Result: orthonormal matri Each row, column has unit length Dot product of rows or columns = Inverse of matri = transpose

7 Arbitrar Rotation Rotate around,, then : sin θ cosθ R = sin θ sin θ cosθ cosθ sin θ + sin θ cosθ cosθ cosθ cosθ + sin θ cosθ sin θ sin θ sin θ sin θ sin θ + cosθ cosθ cosθ sin θ cosθ + cosθ cosθ sin θ cosθ sin θ Don t do this! It s probabl bugg! Compute simple matrices and multipl them

8 Scale Scale in,, : S = s s s

9 Shear Shear parallel to plane: σ = σ σ

10 Translation Can translation be represented b multipling b a 3 3 matri? No. Proof: A : A =

11 Homogeneous Coordinates Add a fourth dimension to each point: To get real (3D) coordinates, divide b w: w w w w w

12 Translation in Homogeneous Coordinates After divide b w, this is just a translation b (t, t, t ) = w w t w t w t w t t t

13 Perspective Projection What does 4 th row of matri do? After divide, = w

14 Perspective Projection This is projection onto the = plane (,,) (/,/,) (,,) = Add scaling, etc. pinhole camera model

15 Putting It All Together: A Camera Model Translate to image center Then perform homogeneous divide, and get (u,v) coords Scale to piel sie T img S pi Perspective projection P cam R cam Camera orientation T cam Camera location 3D point (homogeneous coords)

16 Putting It All Together: A Camera Model Intrinsics Etrinsics T img S pi P cam R cam T cam

17 Putting It All Together: A Camera Model Camera coordinates Normalied device coordinates Ee coordinates Image coordinates Piel coordinates T img S pi P cam R cam T cam World coordinates

18 More General Camera Model Multipl all these matrices together Don t care about after transformation Scale ambiguit free parameters l k j i h g f e l k j i d c b a l k j i h g f e d c b a homogeneous divide

19 Radial Distortion Radial distortion can not be represented b matri u img c u + u * img ( ) * 2 * 2 + κ ( u + v ) img img v img c v + v * img ( ) * 2 * 2 + κ ( u + v ) img img (c u, c v ) is image center, u * img= u img c u, v * img= v img c v, κ is first-order radial distortion coefficient

20 Camera Calibration Determining values for camera parameters Necessar for an algorithm that requires 3D 2D mapping Method used depends on: What data is available Intrinsics onl vs. etrinsics onl vs. both Form of camera model

21 Camera Calibration Eample Given: 3D 2D correspondences General perspective camera model (-parameter, no radial distortion) Write equations: a i e i + b + j + + f j + c + k + g + k + d + l + h + l = = u v

22 Camera Calibration Eample Linear equation Overconstrained (more equations than unknowns) Underconstrained (rank deficient matri an multiple of a solution, including, is also a solution) = l c b a v v v v u u u u v v v v u u u u

23 Camera Calibration Eample Standard linear least squares methods for A= will give the solution = Instead, look for a solution with = That is, minimie A 2 subject to 2 =

24 Camera Calibration Eample Minimie A 2 subject to 2 = A 2 = (A) T (A) = ( T A T )(A) = T (A T A) Epand in terms of eigenvectors of A T A: = µ e + µ 2 e 2 + T (A T A) = λ µ 2 +λ 2 µ = µ 2 +µ 22 +

25 Camera Calibration Eample To minimie λ µ 2 +λ 2 µ 22 + subject to µ 2 +µ 22 + = set µ min = and all other µ i = Thus, least squares solution is eigenvector of A T A corresponding to minimum (nonero) eigenvalue

26 Camera Calibration Eample 2 Incorporating additional constraints into camera model No shear, no scale (rigid-bod motion) Square piels etc. These impose nonlinear constraints on camera parameters

27 Camera Calibration Eample 2 Option : solve for general perspective model, then find closest solution that satisfies constraints Option 2: constrained nonlinear least squares Usuall gradient descent techniques Common implementations available (e.g. Matlab optimiation toolbo)

28 Camera Calibration Eample 3 Incorporating radial distortion Option : Find distortion first (straight lines in calibration target) Warp image to eliminate distortion Run (simpler) perspective calibration Option 2: nonlinear least squares

29 Camera Calibration Eample 4 What if 3D points are not known? Structure from motion problem! As we saw, can often be solved since # of knowns > # of unknowns After Thanksgiving

30 Multi-Camera Geometr Epipolar geometr relationship between observed positions of points in multiple cameras Assume: 2 cameras Known intrinsics and etrinsics

31 Epipolar Geometr P p p 2 C C 2

32 Epipolar Geometr P l 2 p p 2 C C 2

33 Epipolar Geometr P Epipolar line l 2 p p 2 C C 2 Epipoles

34 Epipolar Geometr Goal: derive equation for l 2 Observation: P, C, C 2 determine a plane P l 2 p p 2 C C 2

35 Epipolar Geometr Work in coordinate frame of C Normal of plane is T Rp 2, where T is relative translation, R is relative rotation P l 2 p p 2 C C 2

36 Epipolar Geometr p is perpendicular to this normal: p (T Rp 2 ) = P l 2 p p 2 C C 2

37 Epipolar Geometr Write cross product as matri multiplication P C C 2 p 2 p l 2 = =, * * T T T T T T T T T

38 Epipolar Geometr p T * R p 2 = p T E p 2 = E is the essential matri P l 2 p p 2 C C 2

39 Essential Matri E depends onl on camera geometr Given E, can derive equation for line l 2 P l 2 p p 2 C C 2

40 Fundamental Matri Can define fundamental matri F analogousl, operating on piel coordinates instead of camera coordinates u T F u 2 = Advantage: can sometimes estimate F without knowing camera calibration