Geometry of a single camera. Odilon Redon, Cyclops, 1914

Transcription

1 Geometr o a single camera Odilon Redon, Cclops, 94

2 Our goal: Recover o 3D structure Recover o structure rom one image is inherentl ambiguous???

3 Single-view ambiguit

4 Single-view ambiguit Rashad Alakbarov shadow sculptures

5 Anamorphic perspective H. Holbein he Younger, he Ambassadors, 533

6 Ames Room

7 Our goal: Recover o 3D structure When certain assumptions In general, we need hold, we can recover multi-view geometr structure rom a single view Image source But irst, we need to understand the geometr o a single camera

8 Review: Pinhole camera model world coordinate sstem Normalized (camera) coordinate sstem: camera center is at the origin, the principal ais is the z-ais, and aes o the image plane are parallel to and aes o the world Camera calibration: iguring out transormation rom world coordinate sstem to image coordinate sstem

9 ) /, / ( ),, ( Z Y Z Z Y! Z Y Z Y Z Y! Review: Pinhole camera model P

10 Principal point p p Principal point (p): point where principal ais intersects the image plane Normalized coordinate sstem: origin o the image is at the principal point Image coordinate sstem: origin is in the corner

11 ) /, / ( ),, ( p Z Y p Z Z Y + +! + + Z Z p Y Z p Z Y! Principal point oset We want the principal point to map to (p, p ) instead o (,) p p Z Y p p

12 + + Z Y p p Z Zp Y Zp Principal point oset p p K calibration matri [ ] P K I principal point: ), ( p p p p

13 p p m m K β α β α Piel coordinates m piels per meter in horizontal direction, m piels per meter in vertical direction Piel size: m m piels/m m piels

14 Camera rotation and translation In general, the camera coordinate rame will be related to the world coordinate rame b a rotation and a translation Conversion rom world to camera coordinate sstem (in non-homogeneous coordinates): ~ cam ( ) R ~ C ~ coords. o point in camera rame coords. o a point in world rame coords. o camera center in world rame

15 Camera rotation and translation ~ cam ( ) R ~ C ~ cam ~ cam R RC ~ ~ R RC ~ [ ] K[ R ] K I RC ~ cam P K[ R t], t RC ~

16 Camera parameters Intrinsic parameters Principal point coordinates Focal length Piel magniication actors Skew (non-rectangular piels) Radial distortion p p m m β α β α K [ ] t P K R

17 Camera parameters P K[ R t] Intrinsic parameters Principal point coordinates Focal length Piel magniication actors Skew (non-rectangular piels) Radial distortion Etrinsic parameters Rotation and translation relative to world coordinate sstem P [ ~ ] K R RC What is the projection o the camera center? PC ~ ~ C [ RC] K R coords. o camera center in world rame he camera center is the null space o the projection matri!

18 Camera calibration Z Y λ λ λ [ ] t K R Source: D. Hoiem

19 Camera calibration Given n points with known 3D coordinates i and known image projections i, estimate the camera parameters i i P?

20 i P i λ Camera calibration: Linear method i i P 3 i i i i i P P P 3 P P P i i i i i i i i i i wo linearl independent equations

21 Camera calibration: Linear method P has degrees o reedom One D/3D correspondence gives us two linearl independent equations 6 correspondences needed or a minimal solution Homogeneous least squares: ind p minimizing Ap Solution given b eigenvector o A A with smallest eigenvalue p A 3 P P P n n n n n n!!!

22 Camera calibration: Linear method Note: or coplanar points that satis Π, we will get degenerate solutions (Π,,), (,Π,), or (,,Π) Ap 3 P P P n n n n n n!!!

23 Camera calibration: Linear vs. nonlinear Linear calibration is eas to ormulate and solve, but it doesn t directl tell us the camera parameters λ λ λ Y Z vs. K[ R t] In practice, non-linear methods are preerred Write down objective unction in terms o intrinsic and etrinsic parameters Deine error as sum o squared distances between measured D points and estimated projections o 3D points Minimize error using Newton s method or other non-linear optimization Can model radial distortion and impose constraints such as known ocal length and orthogonalit

24 A taste o multi-view geometr: riangulation Given projections o a 3D point in two or more images (with known camera matrices), ind the coordinates o the point

25 riangulation Given projections o a 3D point in two or more images (with known camera matrices), ind the coordinates o the point? O O

26 riangulation We want to intersect the two visual ras corresponding to and, but because o noise and numerical errors, the don t meet eactl? O O

27 riangulation: Geometric approach Find shortest segment connecting the two viewing ras and let be the midpoint o that segment O O

28 riangulation: Nonlinear approach Find that minimizes d (,P ) + d (,P )? P P O O

29 riangulation: Linear approach b a b a ] [ z z z b b b a a a a a a P P λ λ P P ]P [ ]P [ Cross product as matri multiplication:

30 riangulation: Linear approach λ λ P P P P [ [ ]P ]P wo independent equations each in terms o three unknown entries o

31 Structure rom motion Figure credit: Noah Snavel Camera Camera R,t R,t Camera 3 R 3,t 3 Given D point correspondences between multiple images, compute the camera parameters and the 3D points

32 Structure rom motion? Camera Camera R,t R,t Camera 3 R 3,t 3 Structure: Given known cameras and projections o the same 3D point in two or more images, compute the 3D coordinates o that point

33 Structure rom motion Camera? Camera R,t R,t?? Camera 3 R 3,t 3 Motion: Given a set o known 3D points seen b a camera, compute the camera parameters

34 Useul reerence