Homogeneous Coordinates Overall scaling is NOT important. CSED44:Introduction to Computer Vision (207F) Lecture8: Camera Models Bohyung Han CSE, POSTECH bhhan@postech.ac.kr (",, ) ()", ), )) ) 0 It is useful in projective geometry for Basic representation of geometric entities: points, lines, planes, etc. Representation of point and line at infinity Transformations Others 2 Cross Product Binary operation on two vectors Perpendicular to both of the two vectors The magnitude of the product equals the area of a parallelogram with the vectors for sides, -, / [, - ], /, -, / 7 8 9 6-5 - 4-6 / 5 / 4 / 0 4-5 - 4-0 6-5 - 6-0 6 / 5 / 4 / 5-4 - 5 / 4 / 7 6-4 - 6 / 4 / 8+ 6-5 - 6 / 5 / 9 5-4 / 7 + 6 / 4-8 + 6-5 / 9 5 / 4-7 6-4 / 8 6 / 5-9 Representation of Line and Point in 2D Homogeneous representation of line Homogeneous representation of point Point in 2D: (",, ) Special cases Ideal point (or point at infinity) Point that two parallel lines meet Does not corresponds to any finite point in R /. Line at infinity: line that Ideal points meet Ideal point is on line at infinity. 6" + 5 + 4 0 (6, 5, 4) < >?@A (",, 0) B C (0,0,) < >?@A B C 0 (", ): direction of the lines 3 4
Relationship between Lines and Points Lines vs. points A point < (",, ) lies on the line B (6, 5, 4) 2D Transformation Schematic view of 2D planar transformation < B B < 0 < F B B F < 0 Intersection of two lines B (6, 5, 4) and B (6, 5, 4 ) < B < B 0 < B B Line through two points < (",, ) and < (",, ) B < B < 0 B < < Duality of point and line: Points and lines can be swapped. 5 6 Hierarchy of 2D Transformation 2D Transformation Translation: < H < + I < H J I K < Euclidean (rigid): < H L< + I < H L I K < Similarity: < H ML< + I < H ML I K < Affine: < H N< < H 6 -- 6 -/ 6 -O 6 /- 6 // 6 /O 0 0 < Projective: < H P< < H 6 -- 6 -/ 6 -O 6 /- 6 // 6 /O < 6 O- 6 O/ 6 OO 7 8
Hierarchy of 2D Transformation (Class ) Isometry (distance-preserving map between metric spaces) " H cos T sin T W X sin T cos T W Y 0 0 Preserve Euclidean distances and angles 3 degrees of freedom: translation and rotation < H P Z < < H L I K " cos T sin T < L sin T cos T Hierarchy of Transformations (Class 2) Similarity transformation Isometry composed with an isotropic scaling " H 4 degrees of freedom: W X, W Y, T, M Preserve the shape (equi-form) invariants: angle, parallel line, ratio of length M cos T M sin T W X M sin T M cos T W Y 0 0 < H P [ < < H ML I K < " 9 0 Hierarchy of Transformations (Class 3) Affine transform Non-singular linear transformation followed by a translation " 6 -- 6 -/ W X 6 /- 6 // W Y 0 0 Include non-isotropic scaling 6 degrees of freedom Invariants: parallel lines, ratio of length of parallel line segments, ratio of areas < H P \ < < H N I K < N ]^_ ]_` _^_ ^ a - 0 L T L b ^L(b) 0 a / " Hierarchy of Transformations (Class 4) Projective transformation < H P f < < H N I g h < General non-singular linear transformation of homogeneous coordinates An invertible mapping such that three points lie on the same line if and only if the images of the points do. 8 degrees of freedom Invariants Cross ratio of four collinear points: < - < / < O < c /( < - < e < / < c ) Concurrency: multiple lines meeting at a single point Collinearity Parallel lines are no more parallel in projective transformation. Ideal point (point at infinity) -> finite point ex) vanishing point 2
3D Transformations 3D to 2D Projections Transformation Preservation Matrix Orthographic projection Translation Euclidean Volume J I K L I K < /> 0 0 0 0 0 0 0 0 0 Similarity Angles Absolute conic ML I K Perspective projection Affine Projective Parallelism Volume ratio Plain at infinity Straight lines Intersection Tangency of surfaces N I K N g I h < /> 0 0 0 0 0 0 0 0 0 3 4 Pinhole Camera Basic Pinhole Model What is pinhole camera? An abstract camera model: box with a small hole and translucent plate One ray would pass through each point in the plane. Inverted image is observed in. Virtual image: plane in front of pinhole with same distance to the image plane Y C camera center f X x y x p principal point X Z principal axis Mapping from Euclidean 3-space R O to Euclidean 2-space R i ",, j l" j, l j 5 6
Projection with Homogeneous Coordinates Central projection using homogenous coordinates Principal point offset Image Plane vs. Sensor Plane C l /> j /> l Sensor plane 0,0 " />, /> W X, W Y l, l Image plane principal point Point in Point in camera coordinate system l l l 0 0 0 0 l 0 0 0 0 0 < /> m m diag l, l, [J K] l + W X l + W Y l 0 W X 0 0 l W Y 0 0 0 0 < /> s J K <t O> s l 0 W X 0 l W Y 0 0 Camera calibration matrix 7 8 Camera Rotation and Translation Other Projective Cameras 9 Camera rotation and translation Camera coordinate <t O> L ( u ) u World coordinate <t O> <t O> L Lu K < /> s J K <t O> sl J u 9 DOF: 3 for s, 3 for L, 3 for u Internal (intrinsic) camera parameters: s External (extrinsic) camera parameters: L and u m sl J u s[l I] camera matrix In non-homogeneous coordinate system where I Lu 20 CCD cameras Having non-square pixels 0 DOF Finite projective camera Add skew parameter: DOF s is homogeneous. sl is non-singular. sl can be decomposed by QR factorization General projective camera Arbitrary rank-3 3x4 matrix No non-singularity restriction on the left hand 3x3 submatrix DOF s s ) X 0 " w 0 ) Y w 0 0 ) X M " w 0 ) Y w 0 0 m sl u m ) X lx X ) Y lx Y x X, x Y : # of pixels in unit distance 6 -- 6 -/ 6 -O 6 -c 6 /- 6 // 6 /O 6 /c 6 O- 6 O/ 6 OO
Camera Calibration Finding extrinsic and intrinsic parameters of a camera Camera Calibration Finding intrinsic and extrinsic camera parameters Intrinsic parameters: s Extrinsic parameters: L and u 3D to 2D correspondences m sl J u < /> m < /> m K < /> ",, y < /> m K y K " y " K 3x2 rank-2 matrix z - z / z O DOF K 2 22 Solution Camera Calibration Nz K N N - N / N We need at least 5.5 correspondences to solve the linear system. Note that m is a homogeneous matrix. When N is an x2 matrix, z is -dimensional null-space. When N is larger, it is an over-determined problem. z argmin z Nz subject to z z is the unit singular vector of N corresponding to the least singular value. Nz ]_ z _ z Ä _ z min Nz min Ä subject to Ä solution: Ä 0 0 z _Ä Camera Calibration After finding camera matrix We got 6 -- 6 -/ 6 -O 6 -c m 6 /- 6 // 6 /O 6 /c 6 O- 6 O/ 6 OO 6 Oc Recall m sl J u [sl slu] l M W X s 0 l W Y L: orthonormal matrix 0 0 Camera parameters can be obtained by QR (RQ) factorization and solving a simple linear system 23 24
Lens Distortions Linear camera model Image point and optical center are collinear. Not realistic in the real (non-pinhole) lenses Radial distortion Non-linear error as the lens is cheaper More significant as the focal length of the lens decrease Lens Distortions Radial distortion Note: Lens distortion takes place during the initial projection of the world onto the Ideal projection "Ü />, Ü />, < /> s J K <t O> " /> É(Ñ ) "Ü /> /> Ü /> Actual projection Distortion factor Correction: using (low-order) polynomials "Å " Ç + É Ñ " " Ç Å Ç + É Ñ Ç É Ñ + Ö - Ñ + Ö / Ñ / + Ö O Ñ O + Ñ " " / Ç + / Ç ", : measured " Ç, Ç : distortion center "Å, Å : corrected 25 barrel distortion pincushion distortion 26 27