Augmented Reality II - Camera Calibration - Gudrun Klinker May 11, 2004

Transcription

1 Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24

2 Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5, pp. 39-6)

3 Camera Calibration 3D-to-2D Projection: x = PX x y = p p 2 p 3 p 4 p 2 p 22 p 23 p 24 p 3 p 32 p 33 p 34 È Î X Y Z

4 Outline Finite camera model (Definition of camera parameters) Projective camera model (Properties of the projection matrix P, relationship to camera properties) Models of cameras at infinity Computation of P

5 Finite Camera Model

6 Finite Camera Model Pinhole camera geometry Central projection using homogeneous coordinates Principal point offset Camera rotation and translation CCD cameras Skew parameter

7 Pinhole Camera Geometry Projection of 3D-Points onto a plane principal Center of projection: C cam point f origin of Euclidean principal image plane plane coordinate system C cam Image plane at distance z = f (focal length) X Y Æ f X Z f Z Y Z principal ray

8 Central Projection Using Homogeneous Coordinates X f X Y Z Æ f Y Z Z X fx È f fy = f Y Z Z Î C cam principal plane f principal point image plane principal ray x = PX P = diag( f, f,) [ I O]

9 Principal Point Offset Principal point at (p x, p y ) rel. to image origin X fx + Zp x È f p x fy + Zp y = f p y Y Z Z Î x = K[ I O]X cam È f p x K = f p y Î camera calibration matrix 3 degrees of freedom camera coordinate frame C cam image origin principal point

10 Camera Rotation and Translation inhomogeneous (3x3): X cam = R( X - C ) = RX - RC homogeneous (4x4): È x cam = R -R C X Î 3D-to-2D (3x4): x = KR[ I -C ]X camera coordinate frame world coordinate frame 9 degrees of freedom; internal parameters K,external parameters R, C R,C C cam

11 Camera Rotation and Translation - Alternative Representation - inhomogeneous (3x3): t = -RC X cam = RX + t homogeneous (4x4): È x cam = R t X Î all-together (3x4): x = K[ R t]x camera coordinate frame world coordinate frame 9 degrees of freedom; internal parameters K,external parameters R, t R,t C cam

12 Example C = È R z (3 ) = Î x cam = [ R z -R z C ]X È = X Î 2. camera coordinate frame C cam R,C world coordinate frame

13 CCD Cameras non-square pixels scale factors m x, m y, aspect ratio m x /m y È K = Î m x m y È f Î f p x p y È a x x = a y y Î all-together (3x4): x = KR[ I t]x degrees of freedom

14 Skew Parameter (Finite Projective Camera) skew parameter s È a x s x K = a y y Î all-together: x = KR[ I t]x degrees of freedom

15 Finite Camera Model - Summary - Pinhole camera geometry (f) Central projection Principal point offset (p x, p y ) Camera rotation and translation (R, t) CCD cameras (m x, m y ) --> (a x, a y, x, y ) Skew parameter (s) È a x s x x = a y y [ R t]x Î

16 Projective Camera Model

17 Projective Camera Model Given some matrix P Notations for P: È p p 2 p 3 p 4 P = p 2 p 22 p 23 p 24 Î p 3 p 32 p 33 p 34 È = Î P T P 2T P 3T [ ] [ ] = p p 2 p 3 p 4 = M p 4

18 Camera Center P is 3x4 matrix, rank = 3 -dimensional null-space PC = [ M p 4 ]C = Ray through the camera center X(l) = la + (- l)c x = PX(l) = lpa + (- l)pc = lpa C camera center

19 Camera Center -dimensional null-space PC = [ M p 4 ]C = Ray through the camera center Hom. representation of camera center KR[ I -C ]C = C = C T = -M- p 4 C camera center

20 Example PC = 2 È = Î 2. = [ R z -R z C ]C = [ M p 4 ]C = C = -M- p 4 È C = Î = 2-2 camera coordinate frame C cam R,C world coordinate frame

21 Summary - Camera Center - M -M - p 4 camera center

22 Column Vectors of P P = [ p p 2 p 3 p ] 4 p,p 2,p 3 : vanishing points of x-, y-, z-axis, (e.g.: direction of x-axis, D) p = PD = P p 4 : image of world origin (,, ) T p world coordinate frame camera center p (,,)

23 Example È Î = È Î = È Î = È Î = p 4 p 2 p

24 Summary - Column Space - distance w of world origin from camera ctr image of x-axis image of y-axis image of z-axis image of world origin

25 È P = Î P T P 2T P 3T P 3T : principal plane x PX = y i.e., (line at infinity) Row Vectors of P P T, P 2T : axis planes PX = y w i.e., (plane P ) P 3T X = P T X =

26 P 3T : principal plane È x X = y Î 2 ( 2)X = axis plane P T Example z = -2 È X = y Î 2 w ( )X =.866x +.5y =.732 y = 2( x)

27 Summary - Row Space - normal to plane through y-axis and camera center normal to plane through x-axis and camera center normal to principal plane through camera center principal ray

28 Principal Point Principal axis: ray from C, perp. to principal plane P 3 Normal to P 3 :(p 3,p 32,p 33,) T (on plane at infinity) Principal point: intersection of principal axis with image plane Projection of (p 3,p 32,p 33,) T onto the image plane x = P(p 3,p 32,p 33,) T = Mm 3 (with P = [M p 4 ]) C cam principal plane f principal point image plane principal ray

29 Example principal point P = È Î x = Mm 3 = È Î =

30 Summary - Principal Ray - image of world origin principal ray direction distance w of world origin from camera center

31 Depth of Points Perpendicular distance w of a point X from the principal plane (along the principal ray) w = P 3T X = P 3T (X - C) (since PC= for the camera center) = m 3T ( X - C ) Dot product of the ray from the camera center to X with the principal ray direction

32 Cameras at Infinity

33 Cinematography Move camera backwards along the optical axis (with increasing z) Rescale picture by increasing f x' = f x z dx'= f( x z - x z + ) = f( x z 2 +z ) small z and f: perspective large z and f: weak perspective

34 Cameras at Infinity Moving camera along the principal ray: r T -r T C r 2T -r 2T C r 3T d..t = -r 3T C distance w of world origin from camera center

35 Cameras at Infinity Zooming (by factor k = d t /d ): r T -r T C d t /d K r 2T -r 2T C r 3T d /d t T P, affine Camera d distance w of initial world origin from camera center

36 Cameras at Infinity - Weak Perspective - The action of a weak perspective camera is equivalent to orthographic projection onto a plane at Z=d, followed by a perspective projection from the plane. x f d

37 Cameras at Infinity - Affine Approximation - The effect of the affine approximation P to the true camera matrix P is to move the image of a point X radially towards or away from the principal point x by a factor equal to. + D d x f d

38 Cameras at Infinity The distance between the true perspective image position and the position obtained using the affine camera approximation is small, if the depth relief is small compared to the average depth (d ) the distance of the point from the principal ray is small

39 Parallel Projection versus Perspective Projection

40 Reminder Central Projection Using Homogeneous Coordinates X f X Y Z Æ f Y Z Z X fx È f fy = f Y Z Z Î C cam principal plane f principal point image plane principal ray x = PX P = diag( f, f,) [ I O]

41 Reminder Principal Point Offset Principal point at (p x, p y ) rel. to image origin X fx + Zp x È f p x fy + Zp y = f p y Y Z Z Î x = K[ I O]X cam È f p x K = f p y Î camera calibration matrix 3 degrees of freedom camera coordinate frame C cam image origin principal point

42 Cameras at Infinity - Parallel Projection - parallel projection matrix: È Î calibration matrix È K 2x2 Î T principal point is not defined

43 Hierarchy of Affine Cameras

44 Hierarchy of Affine Cameras orthographic projection with Euclidean transformations (5 DOF) x y = È Î x y z x y = R t T È Î È Î x y z = r T t r 2T t 2 T È Î x y z

45 Hierarchy of Affine Cameras scaled orthographic projection (6 DOF) weak perspective projection (7 DOF) x y = k k È Î r T t r 2T t 2 T È Î x y z = r T t r 2T t 2 T k È Î x y z x y = a x a y È Î r T t r 2T t 2 T È Î x y z

46 Hierarchy of Affine Cameras affine camera (8 DOF) x y = a x s a y È Î r T t r 2T t 2 T È Î x y z P A = m m 2 m 3 t m 2 m 22 m 23 t 2 È Î