Geometry of image formation

Transcription

1 Geometr of image formation Tomáš Svoboda, ech Technical Universit in Prague, enter for Machine Perception Last update: November 0, 2008 Talk Outline Pinhole model amera parameters Estimation of the parameters amera calibration Motivation 2/47 parallel lines window sies image units distance from the camera What will we learn 3/47 how does the 3D world project to 2D image plane? how is a camera modeled? how can we estimate the camera model?

2 Pinhole camera 4/47 camera amera Obscura 5/ obscura amera Obscura room-sied 6/47 Used b the art department at the UN at hapel Hill obscura

3 D Pinhole camera 7/47 D Pinhole camera projects 2D to D 8/47 image plane f optical ais Z Z 2 Z 3 f = Z = f Z Problems with perspective I 9/47 image plane 2 f 2 Z Z 2 Z 3 optical ais = 2 2

4 Problems with perspective II 0/47 image plane 2 3 f 23 optical ais Z 2 Z = 3 Get rid of the ( ) sign /47 image plane image plane 2 3 f f Z Z 2 optical ais Z 3 2/47 How does the 3D world project to the 2D image plane?

5 3/47 A 3D point in a world coordinate sstem [0, 0, 0 4/47 A pinhole camera observes a scene [0, 0, 0 5/47 Point projects to the image plane, point [0, 0, 0

6 Scene projection 6/47 [0, 0, 0 Scene projection 7/47 [0, 0, 0 3D Scene projection observations 8/47 3D lines project to 2D lines but the angles change, parallel lines are no more parallel. area ratios change, note the front and backside of the house

7 9/47 Put the sketches into equations 3D 2D Projection We remember that: = [ f Z, fy Z [ [ f fy Z f 0 f 0 0 Use the homegeneous coordinates 4 [ 20/47 λ [ [3 = K [3 3 [ I 0 [4 but... 4 for the notation conventions, see the talk notes... we need the in camera coordinate sstem Rotate the vector: 2/47 = R( w w ) R is a 3 3 rotation matri. The point coordinates are now in the camera frame. Use homogeneous coordinates to get a matri equation [ = [ R Rw 0 [ w cam w w [0, 0, 0 The camera center w is often replaced b the translation vector t = R w

8 Eternal (etrinsic parameters) The translation vector t and the rotation matri R are called Eternal parameters of the camera. K [ I 0 [ w [0, 0, 0 22/47 λ = K [ R t [ w cam amera parameters (so far): f, R, t Is it all? What can we model? w What is the geometr good for? 23/47 video: Zoom out vs. motion awa from scene How would ou characterie the difference? Would ou guess the motion tpe? What is the geometr good for? 24/47 video: Zoom out vs. motion awa from scene

9 25/47 Enough geometr 5, look at real images 5 just for a moment From geometr to piels and back again 26/ Problems with piels 27/

10 Is this a stright line? 28/ Problems with piels 29/ What are we looking at? 30/

11 Did ou recognie it? 3/ Piel images revisited 32/ There are no negative coordinates. Where is the principal point? Lines are not lines an more. Piels, considered independentl, do not carr much information. Piel coordinate sstem Assume normalied geometrical coordinates = [,, 33/47 u = m u ( ) + u 0 v = m v + v 0 where m u, m v are sies of the piels and [u 0, v 0 are coordinates of the principal point. u v [u 0, v 0

12 Put piels and geometr together From 3D to image coordinates: λ λ λ From normalied coordinates to piels: Put them together: λ u v = = u v f f = fm u 0 u 0 0 fm v v [ R t [4 m u 0 u 0 0 m v v [ R t 34/47 Finall: u K [ R t Introducing a 3 4 camera projection matri P: u P Non-linear distortion Several models eist. Less standardied than the linear model. We will consider a simple on-parameter radial distortion. n denote the linear image coordinates, d the distorted ones. 35/47 d = ( + κr 2 ) n where κ is the distortion parameter, and r 2 = 2 n + 2 n is the distance from the principal point. Observable are the distorted piel coordinates u d = K d Assume that we know κ. How to get the lines back? Undoing Radial Distortion 36/47 From piels to distorted image coordinates: d = K u d From distorted to linear image coordinates: n = d +κr 2 Where is the problem? r 2 = 2 n + 2 n. We have unknowns on both sides of the equation. Iterative solution:. initialie n = d 2. r 2 = 2 n + 2 n 3. compute n = d +κr 2 4. go to 2. (and repeat few times) And back to piels u n = K n

13 Undoing Radial Distortion 37/47 video Estimation of camera parameters camera calibration The goal: estimate the 3 4 camera projection matri P and possibl the parameters of the non-linear distortion κ from images. Assume a known projection [u, v of a 3D point with known coordinates λu P λv = P Y 2 λ P Z 3 λu λ = P λv P and 3 λ = P 2 P 3 Re-arrange and assume 6 λ 0 to get set of homegeneous equations u P 3 P = 0 v P 3 P 2 = 0 38/47 6 see some notes about λ = 0 in the talk notes Estimation of the P matri 39/47 Re-shuffle into a matri form: u P 3 P = 0 v P 3 P 2 = 0 [ 0 u } 0 {{ v } A [2 2 P P 2 P 3 } {{ } p [2 = 0 [2 A correspondece u i i forms two homogeneous equations. P has 2 parameters but scale does not matter. We need at least 6 2D 3D pairs to get a solution. We constitute A [ 2 2 data matri and solve p = argmin Ap subject to p = which is a constrained LSQ problem. p minimies algebraic error

14 Decomposition of P into the calibration parameters 40/47 P = [ KR Kt and = R t We know that R should be 3 3 orthonormal, and K upper triangular. P = P./norm(P(3,:3)); [K,R = rq(p(:,:3)); t = inv(k)*p(:,4); = -R *t; See the slide notes for more details. An eample of a calibration object 4/47 2D projections localied 42/ R R R R R R2 R3 R R R2 R3 R R R2 R3 R R R2 R R R R R R R R R6 3.4 R R6 2.0 R R R R R R G G2 G3 G G G6 0.4 G7 R 0.2 R2 R3 R4 R R G G G G G G G 0.3 R R R R R R R G G G G G G G G G R R R R R R R G3 0.53G R R 4 G 0.34G5 R4 R G2 G6 R3 0.33G G R2 7 R 5.27 R7 0.39G R R G5 R G6 R3 R G R

15 Reprojection for linear model 43/ Reprojection for full model 44/ Reprojection errors comparison between full and linear model 45/ sorted 2D reprojection errors full model linear model 4 piels

16 References The book [2 is the ultimate reference. It is a must read for anone wanting use cameras for 3D computing. Details about matri decompositions used throughout the lecture can be found at [ [ Gene H. Golub and harles F. Van Loan. Matri omputation. Johns Hopkins Studies in the Mathematical Sciences. Johns Hopkins Universit Press, Baltimore, USA, 3rd edition, 996. [2 Richard Hartle and Andrew Zisserman. Multiple view geometr in computer vision. ambridge Universit, ambridge, 2nd edition, /47 End 47/47