(Geometric) Camera Calibration

Transcription

1 (Geoetric) Caera Calibration CS635 Spring 217 Daniel G. Aliaga Departent of Coputer Science Purdue University

2 Caera Calibration Caeras and CCDs Aberrations Perspective Projection Calibration

3 Caeras First photograph due to Niepce (1826) nent/firstphotograph/process/#top

4 Digital Caera vs. Fil Caera Charge-Coupled Device (CCD) Iage plane is a CCD array instead of fil CCD arrays are typically ¼ or ½ inch in size CCD arrays have a pixel resolution (e.g., 64x48, 192x18, 1MP) CCD Caeras have a axiu frae rate, usually deterined by the hardware and bandwidth Nuber of CCDs 3: each CCD captures only R, G, or B wavelengths 1: the single CCD captures RGB siultaneously, reducing the resolution by 1/3 (kinda) Video Interlaced: only half of the horizontal lines of pixels are present in each frae Progressive scan: each frae has a full-set of pixels

5 The siplest 1-CCD caera in town iage plane CCD array object

6 Exposures iage plane aperture CCD array object effective shutter (e.g., often just turn CCD on/off)

7 Exposures An exposure is when the CCD is exposed to the scene, typically for a brief aount of tie and with a particular set of caera paraeters The characteristics of an exposure are deterined by ultiple factors, in particular: Caera aperture Deterines aount of light that shines onto CCD Caera shutter speed Deterines tie during which aperture is open and light shines on CCD

8 Caera Calibration Digital Caeras and CCDs Aberrations Perspective Projection Calibration

9 Aberrations A real lens syste does not produce a perfect iage Aberrations are caused by iperfect anufacturing and by our approxiate odels Lenses typically have a spherical surface Aspherical lenses would better copensate for refraction but are ore difficult to anufacture Typically 1 st order approxiations are used Reeber sin = - 3 /3! + 5 /5! - Thus, thin-lens equations only valid iff sin

10 Aberrations Most coon aberrations: Spherical aberration Coa Astigatis Curvature of field Chroatic aberration Distortion

11 Spherical Aberration Deteriorates axial iage

12 Coa Deteriorates off-axial bundles of rays

13 Astigatis and Curvature of Field Produces ultiple (two) iages of a single object point

14 Chroatic Aberration Caused by wavelength dependent refraction Apochroatic lenses (e.g., RGB) can help

15 Distortion Radial (and tangential) iage distortions Exaple radial distortions

16 Radial Distortion before after

17 Radial Distortion (x, y) pixel before distortion correction (x, y ) pixel after distortion correction Let r = (x 2 + y 2 ) -1 Then x = x(1 - r/r) y = y(1 - r/r) where r = k r + k 1 r 3 + k 2 r 5 + Finally, x = x(1 k k 1 r 2 k 2 r 4 - ) y = y(1 k k 1 r 2 k 2 r 4 - )

18 Caera Calibration Digital Caeras and CCDs Aberrations Perspective Projection Calibration

19 Perspective Projection Z iage plane f (X, Y, Z) (x, y) object x = f X Z y = f Y Z

20 Perspective Projection eye/viewpoint f Z z (x, y) y? iage plane (X, Y, Z) Y optical axis y = Y f Z y = f Y Z & x = f X Z

21 Caera Calibration Digital Caeras and CCDs Aberrations Perspective Projection Calibration

22 Tsai s Caera Calibration A widely used caera odel to calibrate conventional caeras based on a pinhole caera Reference A Versatile Caera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Caeras and Lenses, Roger Y. Tsai, IEEE Journal of Robotics and Autoation, Vol. 3, No. 4, August 1987

23 Zhang s Caera Calibration Another widely used caera odel to calibrate conventional caeras based on a pinhole caera Many ipleentations are floating around! Reference A Flexible New Technique for Caera Calibration, Zhengyou Zhang, IEEE Trans. on PAMI, 22(11): , 2

24 Bouguet s Caera Calibration Another widely used caera odel to calibrate conventional caeras based on a pinhole caera Many ipleentations are floating around! Reference:

25 Calibration Goal Deterine the intrinsic and extrinsic paraeters of a caera (with lens) 25

26 Caera Paraeters Intrinsic/Internal Focal length Principal point (center) Pixel size (Distortion coefficients) Extrinsic/External Rotation Translation f p x, p y s x, s y k 1,...,, t, t, t x y z 26

27 Focal Length 1 1 Z Y X f f Z fy fx Z fy Z fx / / y x

28 Focal Length 1 1 Z Y X f f Z fy fx Z fy Z fx / / y x

29 Principal Point 1 1 Z Y X p f p f Z Zp fy Zp fx y x y x y x

30 CCD Caera: Pixel Size 1 1 y x y x p f p f s s K 1 y y x x p p K (intrinsic) calibration atrix

31 Translation & Rotation x ca x ca R( X C) RX RC t x ca (extrinsic) calibration atrix P R X t Y 1 Z 1 R R t R R 3x3 rotation atrices t x t y t T z translation vector

32 Calibration Task points in space physical arrangeent (calibration pad) observation (caera with initial paraeters) calibration result (caera with calibrated paraeters) Given X i x i K P What is??

33 Caera Calibration: Conics Conic is degree 2 curve on a plane: or ax 2 + bxy + cy 2 + dx + ey + f = x y a b/2 b/2 c x y + d e x y + f =

34 Caera Calibration: Conics Conic is degree 2 curve on a plane: ax 2 + bxy + cy 2 + dx + ey + f = or x T Cx = C = a b/2 d/2 b/2 c e/2 d/2 e/2 f, x = x y 1

35 Caera Calibration: Conics A point transforation on an iage is x = Hx (for hoography H) A conic transforation on an iage is C = H T CH 1

36 Caera Calibration: Absolute Conic The Absolute Conic Ω is invariant under Euclidean transforations and critical to caera calibration (e.g., like oon following you on straight road)

37 Angle between two rays d 1 θ x 1 x 2 d 2 x i = Kd i cos θ = d 1 T d 2 = K 1 x T 1 (K 1 x 2 ) d 1 d 2 K 1 x 1 K 1 x 2 x T 1 K T K 1 x 2 = x T 1 K T K 1 x 1 x T 2 K T K 1 x 2

38 Absolute Conic Given point on Ω called x = d T T, its iage on a general caera is x = KRd Recall x = Hx Thus iage of the conic is ω = H T CH 1 = KR T C KR 1 T 1 = KK or ω = K T K 1

39 Angle between two rays d 1 θ x 1 x 2 d 2 x i = Kd i cos θ = x 1 T ωx 2 x 1 T ωx 1 x 2 T ωx 2

40 Siple Calibration Device Observe these 3 planes, foring 3 hoographies Each H = h 1 h 2 h 3 gives constraints h 1 T ωh 2 = and h 1 T wh 1 = h 2 T h 2 Conic ω is deterined fro 5 or ore such equations, up to a scale Copute K fro ω = KK T 1 using Cholesky factorization, for exaple

41 Zhang s Caera Calibration 1. Detect corners 2. Estiate atrix P 3. Recover instrinsic/extrinsic paraeters 4. Refine: bundle adjustent

42 Typical Forulation MX x ca 1 ' ' ' Z Y X w y x ' / ' ' / ' w y w x y x Let M = KP ) )/( ( 3 1 X X x ) )/( ( 3 2 X X y

43 A Linear Forulation for observations ( 3 ) 1 i i X x n i n hoogeneous linear equations and 12 unknowns (coefficients of M) ) )/( ( 3 1 X X x ) )/( ( 3 2 X X y ( 3 ) 2 i i X y Thus, given n 6 can solve for M; naely Q T n n T T T n n T T n T T T T T T X y Xn X x X X y X X x X Q

44 A Linear Forulation Goal: in Q subject to = 1 Recall: noral equation Ax = b A T Ax = A T b Solution: so solve Q T Q =, e.g., use eigenvector of Q T Q associated with the sallest eigenvalue. Use to ake atrix M.

45 Decoposing M into Caera Paraeters M = ρ A b = K R t α γ u K = β v 1 (often γ = π/2 which eans no skew))

46 Decoposing M into Caera M = ρ A b = K R t α γ u K = β v 1 B = KR and b = Kt Let A = BB T = KR A = Paraeters (often γ = which eans no skew)) (so B is first 3x3 of M) KR T = KK T α 2 + γ u u v + cβ u u v + cα α 2 2 v + v v u v 1

47 Decoposing M into Caera Paraeters A = α 2 + γ u u v + cβ u u v + cα α 2 2 v + v v u v 1 = k u k c u k c k v v u v 1 (assues square pixels and equal focal length in x and y)

48 Decoposing M into Caera Paraeters A = k u k c u k c k v v u v 1 u = A 13 v = A 23 β = k v v 2 γ = k c u v β α = k u u 2 γ 2 K = R = K 1 B t = K 1 b α γ u β v 1

49 Zhang s Caera Calibration 1. Detect corners 2. Estiate atrix P 3. Recover instrinsic/extrinsic paraeters 4. Refine: bundle adjustent

50 Bundle Adjustent Given initial guesses, use nonlinear least squares to refine/copute the calibration paraeters Siple but good convergence depends on accuracy of initial guess

51 Bundle Adjustent ) )/( ( 3 1 X X x ) )/( ( 3 2 X X y Recall ij j i j i ij j i j i ij X X y X X x n E ) ( ) ( 1 Goal is E

52 Bundle Adjustent Option A: Define M as a atrix of 11 unknowns (i.e., 34 1 ) And solve for ij Can be ade very efficient, especially for sparse atrices Option B: M Define as function of intrinsic and extrinsic paraeters so that it is recoputed during each loop of the optiization