Camera model and calibration

Transcription

1 Camera model and calibration Karel Zimmermann, (some images taken from Tomáš Svoboda s) Czech Technical University in Prague, Center for Machine Perception Last update: March 13, 2017

2 Pinhole camera principle 2/ camera

3 Pinhole camera principle 3/ camera

4 Pinhole camera principle 4/ camera

5 Camera Obscura room-sized 5/26 Used by the art department at the UNC at Chapel Hill obscura

6 1D Pinhole camera projects 2D to 1D 6/26 image plane 1 f C z optical ais 1 Z 1 Z 2 Z 3 1 f = 1 Z 1 1 = f 1 Z 1

7 Distant objects are smaller 7/26 image plane 1 2 f 2 1 C z Z 1 Z 2 Z 3 optical ais 1 = 2 1 2

8 1D Pinhole camera projects 2D to 1D 8/26 image plane 2 3 f 23 C z optical ais Z 2 Z = 3

9 1D Pinhole camera projects 2D to 1D 9/26 image plane image plane f C f z Z 1 Z 2 optical ais We move image plane in front of C to get rid of ( ) sign. Z 3 = f Z

10 10/26 How does the 3D world project to the 2D image plane?

11 A 3D point in a world coordinate system 11/26 z [0, 0, 0] y C

12 A pinhole camera observes a scene 12/26 z [0, 0, 0] y C

13 Point projects to the image plane, point 13/26 z [0, 0, 0] y C

14 Scene projection 14/26 z [0, 0, 0] y C

15 Scene projection 15/26 z [0, 0, 0] y C

16 We remember that: 3D 2D Projection [ y ] = f Z f Y Z 16/26 [ Z Zy ] = [ f fy ] λ λy λ = f fy Z C

17 3D 2D Projection Homogeneous coordinates: = [ y 1] and = [ Y Z 1] yield λ λy λ = f fy Z 17/26 λ = f 0 f but... λ [1 1] [3 1] = K [3 3] [ I 0 ] [4 1] C 4 for the notation conventions, see the talk notes

18 Transform into the camera coordinate system Rotate the vector: z 18/26 e = R w( e w C w ) Use homogeneous coordinates to get a matri equation w [0, 0, 0] = [ R w R wc w 0 1 ] w cam y Translation of the world in the camera C w t = R wc w Rotation of the world in the camera C R = R w

19 Camera matri 19/26 z t and R are called Eternal parameters of the camera. w The matri K is called Internal parameters of the camera. [0, 0, 0] λ = K = cam y = K [ R t ] w = C w = P w We omit the world inde and write simply λ = P C

20 Estimation of camera parameters camera calibration 20/26 The goal: estimate the 3 4 camera projection matri P from calibration object. Assume: known correspondence between 2D camera coordinates [u, v] and 3D point [ Y Z] with known coordinates

21 Estimation of camera parameters camera calibration 21/26 λu λv λ = p 1 p 2 p 3 Y Z 1 λu λ = p 1 λv p and 3 λ = p 2 p 3 Re-arrange and assume λ 0 to get set of homegeneous equations u p 3 p 1 = 0 v p 3 p 2 = 0

22 Estimation of camera parameters camera calibration 22/26 u p 3 p 1 = 0 v p 3 p 2 = 0 Re-shuffle into a matri form: [ 0 u } 0 {{ v } A [2 12] ] p 1 p 2 p 3 }{{} p [12 1] = 0 [2 1] A correspondece u i i forms two homogeneous equations. P has 12 parameters but scale does not matter. We need at least 6 2D 3D pairs to get a solution. We constitute A [ 12 12] data matri and solve p = argmin Ap subject to p = 1 which is a constrained LSQ problem. p minimizes algebraic error

23 Solution of constrained LSQ problem We solve p = argmin Ap subject to p = 1 by Lagrange function 23/26 L(p, λ) = Ap + λ(1 p ) = = p A Ap + λ(1 p p)

24 Solution of constrained LSQ problem We solve p = argmin Ap subject to p = 1 by Lagrange function 23/26 L(p, λ) = Ap + λ(1 p ) = = p A Ap + λ(1 p p) Critical points: L(p, λ) p L(p, λ) λ = 2A Ap 2λp = 0 = 1 p p = 0

25 Solution of constrained LSQ problem We solve p = argmin Ap subject to p = 1 by Lagrange function 23/26 L(p, λ) = Ap + λ(1 p ) = = p A Ap + λ(1 p p) Critical points: L(p, λ) p L(p, λ) λ = 2A Ap 2λp = 0 = 1 p p = 0 First equation is characteristic equation (A A λi)p = 0, every eigen-vector p of A A with unit length is critical point.

26 Solution of constrained LSQ problem We solve p = argmin Ap subject to p = 1 by Lagrange function 23/26 L(p, λ) = Ap + λ(1 p ) = = p A Ap + λ(1 p p) Critical points: L(p, λ) p L(p, λ) λ = 2A Ap 2λp = 0 = 1 p p = 0 First equation is characteristic equation (A A λi)p = 0, every eigen-vector p of A A with unit length is critical point. Since cost function Ap in these eigen-vectors is equal to their eigen-values Ap = p A Ap = p λp = λp p = λ p = λ. the solution is the eigen-vector of A A with the smallest eigen-value.

27 Decomposition of P into the calibration parameters 24/26 P = [ KR Kt ] and C = R 1 t We know that R should be 3 3 orthonormal, and K upper triangular. P = P./norm(P(3,1:3)); [K,R] = rq(p(:,1:3)); t = inv(k)*p(:,4); C = -R *t;

28 References The book [2] is the ultimate reference. It is a must read for anyone wanting use cameras for 3D computing. Details about matri decompositions used throughout the lecture can be found at [1] [1] Gene H. Golub and Charles F. Van Loan. Matri Computation. Johns Hopkins Studies in the Mathematical Sciences. Johns Hopkins University Press, Baltimore, USA, 3rd edition, [2] Richard Hartley and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge University, Cambridge, 2nd edition, /26

29 End 26/26

30

31

32

33

34 image plane 1 f 1 C z Z 1 Z 2 Z 3 optical ais 1 f = 1 Z 1 1 = f 1 Z 1

35 image plane 1 2 f 2 1 C z optical ais Z 1 Z 2 1 = Z 3

36 image plane 2 3 f 23 C z optical ais Z 2 Z = 3

37 image plane image plane f C f z Z 1 Z 2 optical ais Z 3

38 C z [0, 0, 0] y

39 C z [0, 0, 0] y

40 C z [0, 0, 0] y

41 C z [0, 0, 0] y

42 C z [0, 0, 0] y

43 C

44 C

45 C z w [0, 0, 0] y cam C w

46 C z w [0, 0, 0] y cam C w

47