Lecture 5.3 Camera calibration. Thomas Opsahl

Lecture 5.3 Camera calibration Thomas Opsahl

Introduction The image u u x X W v C z C World frame x C y C z C = 1 For finite projective cameras, the correspondence between points in the world and points in the image can be described by the simple model f u s c u P = 0 f v c v R t 0 0 1 This camera model is typically not good enough for accurate geometrical computations based on images 2

Introduction Since light enters the camera through a lens instead of a pinhole, most cameras suffer from some kind of distortion Z y x This kind of distortion can be modeled and compensated for X Y Z No radial distortion x A radial distortion model can look like this x = x 1 + κ 1 x 2 + y 2 + κ 2 x 2 + y 2 2 y = y 1 + κ 1 x 2 + y 2 + κ 2 x 2 + y 2 2 X Y y Barrel distortion x where κ 1 and κ 2 are the radial distortion parameters X Y Z y Pincushion distortion 3

Introduction Image from a non-projective camera Does not satisfy u = K R t X When we calibrate a camera we typically estimate the camera calibration matrix K together with distortion parameters A model K R t, using such an estimated K, does not in general describe the correspondence between points in the world and points in the image Instead it describes the correspondence between points in the world and points in a undistorted image an image where the distortion effects have been removed Equivalent projective-camera image Satisfies u = K R t X Images: http://www.robots.ox.ac.uk/~vgg/hzbook/ 4

Undistortion So earlier when we estimated homographies between overlapping images, we should really have been working with undistorted images! How to undistort? Matlab [undist_img,neworigin] = undistortimage(img,cameraparams); undistortedpoints = undistortpoints(points,cameraparams); OpenCV cv::undistort(img, undist_img, P, distcoeffs); cv::undistortpoints(pts,undist_pts,p,distcoeffs); The effect of undistortion is that we get an image or a set of points that satisfy the perspective camera model u = PX much better than the original image or points So we can continue working with the simple model 5

Camera calibration Camera calibration is the process of estimating the matrix K together with any distortion parameter that we use to describe radial/tangential distortion We have seen how K can be found directly from the camera matrix P The estimation of distortion parameters can be baked into this One of the most common calibration algorithms was proposed by Zhegyou Zhang in the paper A Flexible New Technique for Camera Calibration in 2000 OpenCV: calibratecamera Matlab: Camera calibration app This calibration algorithm makes use of multiple images of a asymmetric chessboard 6

Zhang s method in short Z W W X W C x C y C z C Y W Zhang s method requires that the calibration object is planar Then the 3D-2D relationship is described by a homography ØX ø Øuø X X Y Ø ø Ø ø v = K[ r r r t] = K[ r r t ] Y = H Y 0 º 1ß º 1ß º 1ß º 1 ß 1 2 3 1 2 7

Zhang s method in short This observation puts 2 constraints on the intrinsic parameters due to the fact that R is orthonormal r r = 0 h K K h = 0 T T - T -1 1 2 1 2 T T T -T -1 T -T -1 1 1 = 2 2 1 K K 1 = 2K K 2 r r r r h h h h Where H = h 1 T h 2 T T h 3 and K T = K T 1 = K 1 T So for each 3D-2D correspondence we get 2 constraints on the matrix B = K T K 1 Next step is to isolate the unknown parameters in order to compute B 8

Zhang s method in short B can be computed directly from K Ø 1 s vs 0 - u0f ø y - 2 2 2 fu fu fv fu fv Øb11 b12 b13 ø 2 T 1 s s 1 s( vs 0 - u0f - - y ) v0 B= K K = b21 b22 b 23 = - + - 2 2 2 2 2 2 2 fu fv fu fv fv fu fv f v º b31 b32 b33 ß 2 s 2 vs 0 - u0fy ( vs 0 -u0fy) v ( vs 0 -u0f ) 0 y v 0 - + + 1 2 2 2 2 2 2 2 º fu fv fu fv fv fu fv fv ß We see that B is symmetric, so that we can represent it by the parameter vector b = b 11 b 12 b 22 b 13 b 23 b 33 T 9

Zhang s method in short If we denote Ø hh i1 j1 ø hh i1 j2 + hh i2 j1 hh i2 j2 T T vij = then hi Bhj = vb ij hh i3 j1+ hh i1 j3 hh i3 j2 + hh i2 j3 hh º i3 j3 ß Thus we have T T T r1 r2 = 0 h1 Bh2 = 0 Ø v ø 0 12 Ø ø T T T T T T b = 0 r1 r1 = r2 r2 h1 Bh1 = h2 Bh2 º v11 - v22 ß º ß 10

Zhang s method in short Given N images of the planar calibration object we stack the equations to get a homogeneous system of linear equations which can be solved by SVD when N 3 From the estimated b we can recover all the intrinsic parameters The distortion coefficients are then estimated solving a linear least-squares problem Finally all parameters are refined iteratively More details in Zhang s paper http://research.microsoft.com/en-us/um/people/zhang/papers/tr98-71.pdf 11

Camera calibration in practice OpenCV Camera calibration tutorial We ll test it out in the lab Matlab App: Camera Calibrator This opens a new window 12

Camera calibration in practice Add Images and specify the size of the chessboard squares Chessboard detection starts 13

Camera calibration in practice Add Images and specify the size of the chessboard squares Chessboard detection starts Inspect the correctness of detection and choose what to estimate 2/3 radial distortion coeffs? Tangential distortion? Skew? Calibrate 14

Summary Calibration Zhangs method using planar 3D-points Matlab app DLT method: Est P, decompose into K R t Undistortion For geometrical computations we work on undistorted images/feature points Additional reading Szeliski: 6.3 Optional reading A flexible new technique for camera calibration, by Z. Zhang 17