Professor Willia Hoff Dept of Electrical Engineering &Coputer Science http://inside.ines.edu/~whoff/ 1
Caera Calibration 2
Caera Calibration Needed for ost achine vision and photograetry tasks (object recognition, pose estiation, ). Calibration eans estiating extrinsic (external) and intrinsic (internal) paraeters Extrinsic paraeters position and orientation (pose) of caera Intrinsic paraeters Focal length Pixel size Distortion coefficients Iage center In this lecture we will concentrate on finding the intrinsic paraeters 3
Exaple Use of Calibration Given a point in an iage, what ray eanating fro the caera center corresponds to that pixel? Or, given a 3D point in the scene, what pixel does it project to? p We know how to do this if the caera is a pinhole caera: ray x = f X/Z, y = f Y/Z By intersecting the ray with the ground plane, you can estiate the position of the point P 4
Calibration The use of a calibration pattern or set of arkers is one of the ore reliable ways to estiate a caera s intrinsic paraeters A planar target is often used (it is easy to ake), and you take ultiple iages at different poses You can ove the target in a controlled way (where the relative poses of the target are known) or just ove it in an uncontrolled way (where you have to solve for the poses) It is best if the calibration object spans as uch of the iage as possible The strategy is to first solve for all calibration paraeters except lens distortion by assuing there is no distortion Then perfor a final nonlinear optiization which includes solving for lens distortion 5
Exaple Calibration Patterns Geoetry of target is known Pose of target is not known Iages fro Sebastian Thrun 6
Siple Perspective Projection A 3x4 caera projection atrix M projects 3D points onto 2D iage points This atrix odels: rotation and translation focal length ratio of pixel height and width iage center 2D iage point (x,y) X u Y v M Z w 1 x u / w, y v / w 3D point in world coords It doesn t odel lens distortion M K M ext 3x3 intrinsic paraeter atrix 3x4 extrinsic paraeter atrix, contains world to caera pose 7
Perspective projection u v w 11 21 31 12 22 32 13 23 33 14 24 34 X Y Z 1 If the target is planar, we can use Z=0 for all points on the target So equation siplifies to u v w 11 21 31 12 22 32 14 24 34 X Y 1 This 3x3 atrix is a hoography that aps the planar target s calibration points (X, Y, 0) into iage coordinates (x,y), where x=u/w, y=v/w 8
Perspective projection of a plane (continued) We can solve for the 9 unknowns (11..34) by observing a set of known points on a calibration planar target, then do least squares fitting This is called the direct linear transforation (DLT) To do this, write: x u w 11 12 14, 31 X X 32 Y Y 34 y v w 21 31 X X 22 32 Y Y 14 34 Then 11 21 X X 12 22 Y Y 14 14 x y 31X 32Y 34 0 X Y 0 31 32 34 9
Direct Linear Transforation We collect all the unknowns ( ij ) into a vector (9x1), so A=0 A = 0 is a hoogeneous syste of equations You can only solve for up to an unknown scale factor The solution is the eigenvector corresponding to the zero eigenvalue of A T A You can also find this using Singular Value Decoposition (SVD) Recall that we can take the SVD of A; ie., A = U D V T And x is the colun of V corresponding to the zero singular value of A (Since the coluns are ordered, this is the rightost colun of V) We can repeat for other poses 10
Extracting paraeters We next extract the intrinsic and extrinsic caera paraeters fro M, where M = K M ext The intrinsic paraeters (in K) are fx, fy, cx, cy The extrinsic paraeters (in M ext ) are r11, r12,, r33, tx, ty, tz The procedure requires soe linear algebra but is fairly straightforward. It is described clearly in: Zhang, Z. (2000). A flexible new technique for caera calibration. IEEE Trans on Pattern Analysis &Machine Intel, 22(11):1330 1334. The final result is that we have solved for all intrinsic and extrinsic paraeters except lens distortion 11
Lens Distortion Lens distortion - projected points do not follow the siple pinhole caera forula Most coon is barrel distortion and pin-cushion distortion Points are displaced radially inward (barrel) or outward (pincushion) fro correct position Tangential distortions are also possible There are other types of lens aberrations that we won t consider Spherical, coa, astigatis (these blur the point) Chroatic aberrations (color affects focal length) 12
Radial Distortion Exaples wideangle (barrel) telephoto (pincushion) http://aluni.edia.it.edu/~sbeck/results/dis tortion/distortion.htl Iage fro Sebastian Thrun 13
Caera Model We use the odel of Heikkil and Silven at the University of Oulu in Finland Also in the Matlab caera calibration toolbox by Strobl, et al 1 Usual convention of x increasing left to right, y increasing top to botto the top left pixel is the origin 1 http://www.vision.caltech.edu/bouguetj/calib_doc/ 14
2 2 2 6 5 4 2 2 1 distance fro center, where r ) (1 y x r r k r k r k y x y x d d Model of Radial Distortion 15
dx 2k k3 2 2 3xy k4 r 2x 2 2 r 2y 2k xy 4 Model of Tangential Distortion 16
17
Coplete Projection Model A 3D point is projected onto the iage plane (x,y) using the pinhole odel Then the point is distorted using x y Then copute pixel location x (1 y Note: f(1) and f(2) are focal lengths expressed in units of horizontal and vertical pixels. Both coponents are usually very siilar. The ratio f(2)/f(1), often called "aspect ratio", is different fro 1 if the pixels in the CCD array are not square d d 2 4 k1r k2r k5r x y p p f f 1 2 x y d d 6 ) dx cc cc 1 2 There are a total of 9 paraeters to be calibrated f1,f2, cc1, cc2, k1..k5 18
Solving for Paraeters Typically an initial guess for pose and focal length is found, ignoring lens distortions Then non-linear optiization to find all paraeters (such as iterative least squares) this iniizes iage residual errors fro Heikkilä and Silvén 19
How Many Iages are Needed? Unknowns: There are 9 intrinsic paraeters 6 additional paraeters for pose of each viewpoint Given: K iages with M points each Each point yields two equations So 9+6K unknowns and 2KM equations Need 2KM > 9+6K Exaple: 3 viewpoints => 27 unknowns Need at least 5 corners per iage Of course, ore are better 20
Matlab Caera Calibration Toolbox See http://www.vision.caltech.edu/bouguetj/calib_doc/ Requires anual initial picking of corners in iages Software finds pose of target in each iage; and intrinsic paraeters Can rectify entire iage This will undo lens distortion Creates an iage as if it were taken with a pinhole caera Or, use the noralize function fro the toolbox to undistort a single point 21
Minor fix to code If you are having eory probles, try this In file apply_distortion., coent out these lines % aa = (2*k(3)*x(2,:)+6*k(4)*x(1,:))'*ones(1,3); % bb = (2*k(3)*x(1,:)+2*k(4)*x(2,:))'*ones(1,3); % cc = (6*k(3)*x(2,:)+2*k(4)*x(1,:))'*ones(1,3); Code doesn t use these variables If you don t coent the out, it ay run out of eory on large iages 22