Augmented Reality II - Camera Calibration - Gudrun Klinker May, 24
Literature Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2. (Section 5, pp. 39-6)
Camera Calibration 3D-to-2D Projection: x = PX x y = p p 2 p 3 p 4 p 2 p 22 p 23 p 24 p 3 p 32 p 33 p 34 È Î X Y Z
Outline Finite camera model (Definition of camera parameters) Projective camera model (Properties of the projection matrix P, relationship to camera properties) Models of cameras at infinity Computation of P
Finite Camera Model
Finite Camera Model Pinhole camera geometry Central projection using homogeneous coordinates Principal point offset Camera rotation and translation CCD cameras Skew parameter
Pinhole Camera Geometry Projection of 3D-Points onto a plane principal Center of projection: C cam point f origin of Euclidean principal image plane plane coordinate system C cam Image plane at distance z = f (focal length) X Y Æ f X Z f Z Y Z principal ray
Central Projection Using Homogeneous Coordinates X f X Y Z Æ f Y Z Z X fx È f fy = f Y Z Z Î C cam principal plane f principal point image plane principal ray x = PX P = diag( f, f,) [ I O]
Principal Point Offset Principal point at (p x, p y ) rel. to image origin X fx + Zp x È f p x fy + Zp y = f p y Y Z Z Î x = K[ I O]X cam È f p x K = f p y Î camera calibration matrix 3 degrees of freedom camera coordinate frame C cam image origin principal point
Camera Rotation and Translation inhomogeneous (3x3): X cam = R( X - C ) = RX - RC homogeneous (4x4): È x cam = R -R C X Î 3D-to-2D (3x4): x = KR[ I -C ]X camera coordinate frame world coordinate frame 9 degrees of freedom; internal parameters K,external parameters R, C R,C C cam
Camera Rotation and Translation - Alternative Representation - inhomogeneous (3x3): t = -RC X cam = RX + t homogeneous (4x4): È x cam = R t X Î all-together (3x4): x = K[ R t]x camera coordinate frame world coordinate frame 9 degrees of freedom; internal parameters K,external parameters R, t R,t C cam
Example C = 2.. -2. È.866.5. R z (3 ) = -.5.866. Î x cam = [ R z -R z C ]X È.866.5 -.732 = -.5.866. X Î 2. camera coordinate frame C cam R,C world coordinate frame
CCD Cameras non-square pixels scale factors m x, m y, aspect ratio m x /m y È K = Î m x m y È f Î f p x p y È a x x = a y y Î all-together (3x4): x = KR[ I t]x degrees of freedom
Skew Parameter (Finite Projective Camera) skew parameter s È a x s x K = a y y Î all-together: x = KR[ I t]x degrees of freedom
Finite Camera Model - Summary - Pinhole camera geometry (f) Central projection Principal point offset (p x, p y ) Camera rotation and translation (R, t) CCD cameras (m x, m y ) --> (a x, a y, x, y ) Skew parameter (s) È a x s x x = a y y [ R t]x Î
Projective Camera Model
Projective Camera Model Given some matrix P Notations for P: È p p 2 p 3 p 4 P = p 2 p 22 p 23 p 24 Î p 3 p 32 p 33 p 34 È = Î P T P 2T P 3T [ ] [ ] = p p 2 p 3 p 4 = M p 4
Camera Center P is 3x4 matrix, rank = 3 -dimensional null-space PC = [ M p 4 ]C = Ray through the camera center X(l) = la + (- l)c x = PX(l) = lpa + (- l)pc = lpa C camera center
Camera Center -dimensional null-space PC = [ M p 4 ]C = Ray through the camera center Hom. representation of camera center KR[ I -C ]C = C = C T = -M- p 4 C camera center
Example PC = 2 È.866.5 -.732.732 -.732 -.5.866. = -2 -. +. Î 2. = -2. + 2. [ R z -R z C ]C = [ M p 4 ]C = C = -M- p 4 È.866 -.5 -.732 2 C = -.5.866 Î = 2-2 camera coordinate frame C cam R,C world coordinate frame
Summary - Camera Center - M -M - p 4 camera center
Column Vectors of P P = [ p p 2 p 3 p ] 4 p,p 2,p 3 : vanishing points of x-, y-, z-axis, (e.g.: direction of x-axis, D) p = PD = P p 4 : image of world origin (,, ) T p world coordinate frame camera center p (,,)
Example.866.5 -.732 -.5.866. 2. È Î =.866 -.5.866.5 -.732 -.5.866. 2. È Î =.5.866.866.5 -.732 -.5.866. 2. È Î =.866.5 -.732 -.5.866. 2. È Î = -.732. 2. p 4 p 2 p
Summary - Column Space - distance w of world origin from camera ctr image of x-axis image of y-axis image of z-axis image of world origin
È P = Î P T P 2T P 3T P 3T : principal plane x PX = y i.e., (line at infinity) Row Vectors of P P T, P 2T : axis planes PX = y w i.e., (plane P ) P 3T X = P T X =
P 3T : principal plane È.866.5 -.732 x -.5.866 X = y Î 2 ( 2)X = axis plane P T Example z = -2 È.866.5 -.732 -.5.866 X = y Î 2 w (.866.5 -.732)X =.866x +.5y =.732 y = 2(.732 -.866x)
Summary - Row Space - normal to plane through y-axis and camera center normal to plane through x-axis and camera center normal to principal plane through camera center principal ray
Principal Point Principal axis: ray from C, perp. to principal plane P 3 Normal to P 3 :(p 3,p 32,p 33,) T (on plane at infinity) Principal point: intersection of principal axis with image plane Projection of (p 3,p 32,p 33,) T onto the image plane x = P(p 3,p 32,p 33,) T = Mm 3 (with P = [M p 4 ]) C cam principal plane f principal point image plane principal ray
Example principal point P =.866.5 -.732 -.5.866 2 È Î x = Mm 3 =.866.5 -.5.866 È Î =
Summary - Principal Ray - image of world origin principal ray direction distance w of world origin from camera center
Depth of Points Perpendicular distance w of a point X from the principal plane (along the principal ray) w = P 3T X = P 3T (X - C) (since PC= for the camera center) = m 3T ( X - C ) Dot product of the ray from the camera center to X with the principal ray direction
Cameras at Infinity
Cinematography Move camera backwards along the optical axis (with increasing z) Rescale picture by increasing f x' = f x z dx'= f( x z - x z + ) = f( x z 2 +z ) small z and f: perspective large z and f: weak perspective
Cameras at Infinity Moving camera along the principal ray: r T -r T C r 2T -r 2T C r 3T d..t = -r 3T C distance w of world origin from camera center
Cameras at Infinity Zooming (by factor k = d t /d ): r T -r T C d t /d K r 2T -r 2T C r 3T d /d t T P, affine Camera d distance w of initial world origin from camera center
Cameras at Infinity - Weak Perspective - The action of a weak perspective camera is equivalent to orthographic projection onto a plane at Z=d, followed by a perspective projection from the plane. x f d
Cameras at Infinity - Affine Approximation - The effect of the affine approximation P to the true camera matrix P is to move the image of a point X radially towards or away from the principal point x by a factor equal to. + D d x f d
Cameras at Infinity The distance between the true perspective image position and the position obtained using the affine camera approximation is small, if the depth relief is small compared to the average depth (d ) the distance of the point from the principal ray is small
Parallel Projection versus Perspective Projection
Reminder Central Projection Using Homogeneous Coordinates X f X Y Z Æ f Y Z Z X fx È f fy = f Y Z Z Î C cam principal plane f principal point image plane principal ray x = PX P = diag( f, f,) [ I O]
Reminder Principal Point Offset Principal point at (p x, p y ) rel. to image origin X fx + Zp x È f p x fy + Zp y = f p y Y Z Z Î x = K[ I O]X cam È f p x K = f p y Î camera calibration matrix 3 degrees of freedom camera coordinate frame C cam image origin principal point
Cameras at Infinity - Parallel Projection - parallel projection matrix: È Î calibration matrix È K 2x2 Î T principal point is not defined
Hierarchy of Affine Cameras
Hierarchy of Affine Cameras orthographic projection with Euclidean transformations (5 DOF) x y = È Î x y z x y = R t T È Î È Î x y z = r T t r 2T t 2 T È Î x y z
Hierarchy of Affine Cameras scaled orthographic projection (6 DOF) weak perspective projection (7 DOF) x y = k k È Î r T t r 2T t 2 T È Î x y z = r T t r 2T t 2 T k È Î x y z x y = a x a y È Î r T t r 2T t 2 T È Î x y z
Hierarchy of Affine Cameras affine camera (8 DOF) x y = a x s a y È Î r T t r 2T t 2 T È Î x y z P A = m m 2 m 3 t m 2 m 22 m 23 t 2 È Î