Computer Vision: Lecture 3 Carl Olsson 2019-01-29 Carl Olsson Computer Vision: Lecture 3 2019-01-29 1 / 28
Todays Lecture Camera Calibration The inner parameters - K. Projective vs. Euclidean Reconstruction. Finding the camera matrix. DLT - Direct Linear Transformation. Normalization of uncalibrated cameras. Radial distortion Carl Olsson Computer Vision: Lecture 3 2019-01-29 2 / 28
The Inner Parameters -K The matrix K is the upper triangular matrix: γf sf x 0 K = 0 f y 0. 0 0 1 f - focal length γ - aspect ratio s - skew (x 0, y 0 ) - principal points Carl Olsson Computer Vision: Lecture 3 2019-01-29 3 / 28
The Inner Parameters -K The focal length f fx fy 1 = f 0 0 0 f 0 0 0 1 Re scales the images (e.g. meters pixels). x y 1 Carl Olsson Computer Vision: Lecture 3 2019-01-29 4 / 28
The Inner Parameters -K The principal point (x 0, y 0 ) fx + x 0 fy + y 0 = 1 f 0 x 0 0 f y 0 0 0 1 Re centers the image. Typically transforms the point (0, 0, 1) to the middle of the image. x y 1 480 0 0 640 Carl Olsson Computer Vision: Lecture 3 2019-01-29 5 / 28
The Inner Parameters -K Aspect ratio γfx + x 0 fy + y 0 1 = γf 0 x 0 0 f y 0 0 0 1 Pixels are not always squares but can be rectangular. In such cases the scaling in the x-direction should be different from the y-direction. x y 1 Skew γf sf x 0 0 f y 0 0 0 1 Corrects for tilted pixels. Typically zero. Carl Olsson Computer Vision: Lecture 3 2019-01-29 6 / 28
Projective vs. Euclidean Reconstruction Calibrated Cameras A camera P = K [R t], where the inner parameters K are known is called calibrated. If we change coordinates in the image using x = K 1 x, we get a so called normalized (calibrated) camera x = K 1 K [R t] X = [R t] X. Carl Olsson Computer Vision: Lecture 3 2019-01-29 7 / 28
Projective vs. Euclidean Reconstruction Projective The reconstruction is determined up to a projective transformation. If λx = PX, then for any projective transformation X = H 1 X we have λx = PHH 1 X = PH X. PH is also a valid camera. Carl Olsson Computer Vision: Lecture 3 2019-01-29 8 / 28
Projective vs. Euclidean Reconstruction Euclidean The reconstruction is determined up to a similarity transformation. If λx = [R t]x, then for any similarity transformation we have X = [ λ Q v x = [R t] 1 s 0 s [ sq v 0 1 ] 1 X ] X = [RQ Since RQ is a rotation this is a normalized camera. Rv + t s ] X. Carl Olsson Computer Vision: Lecture 3 2019-01-29 9 / 28
Projective vs. Euclidean Reconstruction Projective Euclidean Arch of triumph, Paris. The reconstructions have exactly the same reprojection error. But the projective coordinate system makes things look strange. Carl Olsson Computer Vision: Lecture 3 2019-01-29 10 / 28
Projective vs. Euclidean Reconstruction Demo. Carl Olsson Computer Vision: Lecture 3 2019-01-29 11 / 28
Finding K See lecture notes. Carl Olsson Computer Vision: Lecture 3 2019-01-29 12 / 28
RQ-factorization Theorem If A is an n n matrix then there is an orthogonal matrix Q and a right triangular matrix R such that A = RQ. (If A is invertible and the diagonal elements are chosen the be positive, then the factorization is unique.) Note: In our case we will use K for the triangular matrix and R for the rotation. Carl Olsson Computer Vision: Lecture 3 2019-01-29 13 / 28
Finding K See lecture notes. Carl Olsson Computer Vision: Lecture 3 2019-01-29 14 / 28
Direct Linear Transformation - DLT Finding the camera matrix Use images of a known object to eliminate the projective ambiguity. If X i are 3d-points of a known object, and x i corresponding projections we have λ 1 x 1 = PX 1 λ 2 x 2 = PX 2. λ N x N = PX N. There are 3N equations and 11 + N unknowns. We need 3N 11 + N N 6 points to solve the problem. Carl Olsson Computer Vision: Lecture 3 2019-01-29 15 / 28
Direct Linear Transformation - DLT Matrix Formulation P = p T 1 p T 2 p T 3 where p i are the rows of P The first equality is X T 1 p 1 λ 1 x 1 = 0 X T 1 p 2 λ 1 y 1 = 0 X T 1 p 3 λ 1 = 0, where x 1 = (x 1, y 1, 1). In matrix form X T 1 0 0 x 1 0 X T 1 0 y 1 0 0 X T 1 1 λ 1 Carl Olsson Computer Vision: Lecture 3 2019-01-29 16 / 28 p 1 p 2 p 3 = 0 0 0
Direct Linear Transformation - DLT Matrix Formulation More equations: X T 1 0 0 x 1 0 0... 0 X T 1 0 y 1 0 0... 0 0 X T 1 1 0 0... X T 2 0 0 0 x 2 0... 0 X T 2 0 0 y 2 0... 0 0 X T 2 0 1 0... X T 3 0 0 0 0 x 3... 0 X T 3 0 0 0 y 3... 0 0 X T 3 0 0 1............ } {{ } =M p 1 p 2 p 3 λ 1 λ 2 λ 3 }. {{ } =v = 0 Carl Olsson Computer Vision: Lecture 3 2019-01-29 17 / 28
Direct Linear Transformation - DLT Homogeneous Least Squares See lecture notes... Carl Olsson Computer Vision: Lecture 3 2019-01-29 18 / 28
Singular values decomposition Theorem Each m n matrix M (with real coefficients) can be factorized into M = USV T, where U and V are orthogonal (m m and n n respectively), S = [ diag(σ1, σ 2,..., σ r ) 0 0 0 σ 1 σ 2... σ r > 0 and r is the rank of the matrix. ], Carl Olsson Computer Vision: Lecture 3 2019-01-29 19 / 28
Direct Linear Transformation - DLT Homogeneous Least Squares See lecture notes... Carl Olsson Computer Vision: Lecture 3 2019-01-29 20 / 28
Direct Linear Transformation - DLT Improving the Numerics (Normalization of uncalibrated cameras) The matrix contains entries x i, y j and ones. Since x i and y i can be about a thousand, the numerics are often greatly improved by translating the coordinates such that their center of mass is zero and then rescaling the coordinates to be roughly 1. Change coordinates according to s 0 s x x = 0 s sȳ x. 0 0 1 Solve the homogeneous linear least squares system and transform back to the original coordinate system. Similar transformations for the 3D-points X i may also improve the results. Carl Olsson Computer Vision: Lecture 3 2019-01-29 21 / 28
Pose estimation using DLT 3D points measured using scanning arm. Carl Olsson Computer Vision: Lecture 3 2019-01-29 22 / 28
Pose estimation using DLT 14 points used for computing the camera matrix. Carl Olsson Computer Vision: Lecture 3 2019-01-29 23 / 28
Pose estimation using DLT 14 points used for computing the camera matrix. Carl Olsson Computer Vision: Lecture 3 2019-01-29 24 / 28
Texturing the chair Project the rest of the points into the image. Carl Olsson Computer Vision: Lecture 3 2019-01-29 25 / 28
Texturing the chair Form triangles. Use the texture from the image. Carl Olsson Computer Vision: Lecture 3 2019-01-29 26 / 28
Textured chair Carl Olsson Computer Vision: Lecture 3 2019-01-29 27 / 28
Radial Distortion Not modeled by the K -matrix. Cannot be removed by a projective mapping since lines are not mapped onto lines (see Szeliski). Carl Olsson Computer Vision: Lecture 3 2019-01-29 28 / 28