Image Formation I Chapter 2 (R. Szelisky)

Transcription

1 Image Formation I Chapter 2 (R. Selisky) Guido Gerig CS 632 Spring 22 cknowledgements: Slides used from Prof. Trevor Darrell, ( Some slides modified from Marc Pollefeys, UNC Chapel Hill. Other slides and illustrations from J. Ponce, addendum to course book.

2 GEOMETRIC CMER MODELS The Intrinsic Parameters of a Camera The Extrinsic Parameters of a Camera The General Form of the Perspective Projection Equat Line Geometry Reading: Chapter 2.

3 Camera model Relation between pixels and rays in space?

4 Camera obscura + lens The camera obscura (Latin for 'dark room') is an optical device that projects an image of its surroundings on a screen (source Wikipedia).

5 Physical parameters of image Geometric Type of projection Camera pose Photometric Type, direction, intensity of light reaching sensor Surfaces reflectance properties Optical Sensor s lens type focal length, field of view, aperture Sensor sampling, etc. formation

6 Physical parameters of image formation Geometric Type of projection Camera pose Optical Sensor s lens type focal length, field of view, aperture Photometric Type, direction, intensity of light reaching sensor Surfaces reflectance properties Sensor sampling, etc.

7 Perspective and art Use of correct perspective projection indicated in st century.c. frescoes Skill resurfaces in Renaissance: artists develop systematic methods to determine perspective projection (around 48 55) Raphael Durer, 525 K. Grauman

8 Perspective projection equations 3d world mapped to 2d projection in image plane Image plane Focal length Camera frame Optical axis Scene / world points Scene point Image coordinates Forsyth and Ponce

9 ffine projection models: Weak perspective projection x' mx y' my where m f ' is the magnification. When the scene relief is small compared to its distance from the Camera, m can be taken constant: weak perspective projection.

10 ffine projection models: Orthographic projection x' y' x y When the camera is at a (roughly constant) distance from the scene, take m=.

11 Homogeneous coordinates Is this a linear transformation? no division by is nonlinear Trick: add one more coordinate: homogeneous image coordinates homogeneous scene coordinates Converting from homogeneous coordinates Slide by Steve Seit

12 Perspective Projection Matrix divide by the third coordinate to convert back to nonhomogeneous coordinates Projection is a matrix multiplication using homogeneous coordinates: ' / ' / f y x y x f ) ', ' ( y f x f Slide by Steve Seit Complete mapping from world points to image pixel positions?

13 Points at infinity, vanishing points Points from infinity represent rays into camera which are close to the optical axis. Image source: wikipedia

14 Perspective projection & calibration Perspective equations so far in terms of camera s reference frame. Camera s intrinsic and extrinsic parameters needed to calibrate geometry. Camera frame K. Grauman

15 The CCD camera

16 Perspective projection & calibration World frame Extrinsic: Camera frame World frame Camera frame Intrinsic: Image coordinates relative to camera Pixel coordinates 2D point (3x) = Camera to pixel coord. trans. matrix (3x3) Perspective projection matrix (3x4) World to camera coord. trans. matrix (4x4) 3D point (4x) K. Grauman

17 Intrinsic parameters: from idealied world coordinates to pixel values Forsyth&Ponce Perspective projection u v f f x y W. Freeman

18 Intrinsic parameters ut pixels are in some arbitrary spatial units u v x y W. Freeman

19 Intrinsic parameters Maybe pixels are not square u v x y W. Freeman

20 Intrinsic parameters We don t know the origin of our camera pixel coordinates u v x y u v W. Freeman

21 Intrinsic parameters ) sin( ) cot( v y v u y x u May be skew between camera pixel axes v u v u v u v u u v v ) cot( ) cos( ) sin( W. Freeman

22 p p C (K) Intrinsic parameters, homogeneous coordinates ) sin( ) cot( v y v u y x u ) sin( ) cot( y x v u v u Using homogenous coordinates, we can write this as: or: In camera based coords In pixels W. Freeman

23 Perspective projection & calibration World frame Extrinsic: Camera frame World frame Camera frame Intrinsic: Image coordinates relative to camera Pixel coordinates 2D point (3x) = Camera to pixel coord. trans. matrix (3x3) Perspective projection matrix (3x4) World to camera coord. trans. matrix (4x4) 3D point (4x) K. Grauman

24 Coordinate Changes: Pure Translations O P = O O + O P, P = P + O

25 Coordinate Changes: Pure Rotations ),, ( R k j i k k k j k i j k j j j i i k i j i i k j i T T T k j i

26 Coordinate Changes: Rotations about the k xis R cos sin sin cos

27 rotation matrix is characteried by the following properties: Its inverse is equal to its transpose, and its determinant is equal to. Or equivalently: Its rows (or columns) form a right-handed orthonormal coordinate system.

28 Coordinate Changes: Pure Rotations P R P y x y x OP k j i k j i

29 Coordinate Changes: Rigid Transformations P R P O

30 lock Matrix Multiplication What is? Homogeneous Representation of Rigid Transformations P R T O P R P O T P

31 Extrinsic parameters: translation and rotation of camera frame t p R p C W W C W C Non homogeneous coordinates Homogeneous coordinates p t R p W C W C W C W. Freeman

32 Combining extrinsic and intrinsic calibration parameters, in homogeneous coordinates Forsyth&Ponce p t R K p W C W C W,, p p C K p M p W Intrinsic Extrinsic p t R p W C W C W C World coordinates Camera coordinates pixels W. Freeman

33 Other ways to write the same equation W y W x W T T T p p p m m m v u p M p W P m P m v P m P m u pixel coordinates world coordinates Conversion back from homogeneous coordinates leads to (note that = m^t_3*p) : W. Freeman

34 Extrinsic Parameters

35 Explicit Form of the Projection Matrix Note: M is only defined up to scale in this setting!!

36 Calibration target Find the position, u i and v i, in pixels, of each calibration object feature point. CL.html