Image Formation I Chapter 1 (Forsyth&Ponce) Cameras Guido Gerig CS 632 Spring 213 cknowledgements: Slides used from Prof. Trevor Darrell, (http://www.eecs.berkeley.edu/~trevor/cs28.html) Some slides modified from Marc Pollefeys, UNC Chapel Hill. Other slides and illustrations from J. Ponce, addendum to course book.
GEOMETRIC CMER MODELS The Intrinsic Parameters of a Camera The Extrinsic Parameters of a Camera The General Form of the Perspective Projection Equation Line Geometry Reading: Chapter 1.
Images are two-dimensional patterns of brightness values. Figure from US Navy Manual of asic Optics and Optical Instruments, prepared by ureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969. They are formed by the projection of 3D objects.
nimal eye: a looonnng time ago. Photographic camera: Niepce, 1816. Pinhole perspective projection: runelleschi, XV th Century. Camera obscura: XVI th Century.
Camera model Relation between pixels and rays in space?
Camera obscura + lens The camera obscura (Latin for 'dark room') is an optical device that projects an image of its surroundings on a screen (source Wikipedia).
Limits for pinhole cameras
Physical parameters of image Geometric Type of projection Camera pose Photometric Type, direction, intensity of light reaching sensor Surfaces reflectance properties Optical Sensor s lens type focal length, field of view, aperture Sensor sampling, etc. formation
Physical parameters of image formation Geometric Type of projection Camera pose Optical Sensor s lens type focal length, field of view, aperture Photometric Type, direction, intensity of light reaching sensor Surfaces reflectance properties Sensor sampling, etc.
Perspective and art Use of correct perspective projection indicated in 1 st century.c. frescoes Skill resurfaces in Renaissance: artists develop systematic methods to determine perspective projection (around 148-1515) Raphael Durer, 1525 K. Grauman
Perspective projection equations 3d world mapped to 2d projection in image plane Image plane Focal length Camera frame Optical axis Scene / world points Scene point Image coordinates Forsyth and Ponce
ffine projection models: Weak perspective projection x' mx y' my where m f ' is the magnification. When the scene relief is small compared to its distance from the Camera, m can be taken constant: weak perspective projection.
ffine projection models: Orthographic projection x' y' x y When the camera is at a (roughly constant) distance from the scene, take m=1.
Homogeneous coordinates Is this a linear transformation? no division by is nonlinear Trick: add one more coordinate: homogeneous image coordinates homogeneous scene coordinates Converting from homogeneous coordinates Slide by Steve Seit
Perspective Projection Matrix divide by the third coordinate to convert back to nonhomogeneous coordinates Projection is a matrix multiplication using homogeneous coordinates: ' / 1 ' 1/ 1 1 f y x y x f ), ( ) ', ' ( ' ' y x y f x f Slide by Steve Seit Complete mapping from world points to image pixel positions?
Points at infinity, vanishing points Points from infinity represent rays into camera which are close to the optical axis. Image source: wikipedia
Perspective projection & calibration Perspective equations so far in terms of camera s reference frame. Camera s intrinsic and extrinsic parameters needed to calibrate geometry. Camera frame K. Grauman
The CCD camera
Perspective projection & calibration World frame Extrinsic: Camera frame World frame Camera frame Intrinsic: Image coordinates relative to camera Pixel coordinates 2D point (3x1) = Camera to pixel coord. trans. matrix (3x3) Perspective projection matrix (3x4) World to camera coord. trans. matrix (4x4) 3D point (4x1) K. Grauman
Intrinsic parameters: from idealied world coordinates to pixel values Forsyth&Ponce Perspective projection: Worls point and pixels in camera coordinates x y ' ' f f x y W. Freeman
Intrinsic parameters Pixel dimensions: 1/k*1/k k: cells/mm, units [mm -1 ] ut pixels are in some arbitrary spatial units, which can be described by #pixels per mm. x u, with f * k y v, with f * k represents magnification W. Freeman
Intrinsic parameters Pixel dimensions: 1/k*1/l k,l: cells/mm, units [mm -1 ] Maybe pixels are not square and have different horiontal and vertical dimensions. (u,v): pixel numbers, x,y,): World point in camera coordinates. x u, with f * k y v, with f * l, represent magnifications W. Freeman
Intrinsic parameters We don t know the origin of our camera pixel coordinates: (u,v) represent intersection of optical axis with image plane: (u, v): image center in pixel coordinates. u v x y u v W. Freeman
Intrinsic parameters v v May be skew between camera pixel axes due to manufacturing errors and eventually line-by-line readouts. u v x vsin( ) sin( ) u u cos( ) v cot( ) y v v v v sin( ) y u u cot( ) v u u W. Freeman
p p C (K) 1 Intrinsic parameters, homogeneous coordinates ) sin( ) cot( v y v u y x u 1 1 ) sin( ) cot( 1 1 y x v u v u Using homogenous coordinates, we can write this as: or: World point in camera-based coordinates In pixels W. Freeman
Perspective projection & calibration World frame Extrinsic: Camera frame World frame Camera frame Intrinsic: Image coordinates relative to camera Pixel coordinates 2D point (3x1) = Camera to pixel coord. trans. matrix (3x3) Perspective projection matrix (3x4) World to camera coord. trans. matrix (4x4) 3D point (4x1) K. Grauman
Coordinate Changes: Pure Translations O P = O O + O P, P = P + O
Coordinate Changes: Pure Rotations ),, (......... R k j i k k k j k i j k j j j i i k i j i i k j i T T T k j i
Coordinate Changes: Rotations about the k xis R cos sin sin cos 1
rotation matrix is characteried by the following properties: Its inverse is equal to its transpose, and its determinant is equal to 1. Or equivalently: Its rows (or columns) form a right-handed orthonormal coordinate system.
Coordinate Changes: Pure Rotations P R P y x y x OP k j i k j i
Coordinate Changes: Rigid Transformations P R P O
lock Matrix Multiplication 11 21 12 22 11 21 12 22 What is? 11 21 11 11 12 22 21 21 11 21 12 12 12 22 22 22 Homogeneous Representation of Rigid Transformations P 1 R T O 1 P 1 R P 1 O T P 1
Extrinsic parameters: translation and rotation of camera frame t p R p C W W C W C Non-homogeneous coordinates Homogeneous coordinates p t R p W C W C W C 1 W. Freeman
Combining extrinsic and intrinsic calibration parameters, in homogeneous coordinates Forsyth&Ponce p t R K p W C W C W 1,, 1 p p C K 1 p M p W 1 Intrinsic Extrinsic p t R p W C W C W C 1 World coordinates Camera coordinates pixels W. Freeman pixels (u,v,1) World coordinates (x,y,,1)
Other ways to write the same equation 1......... 1 1 3 2 1 W y W x W T T T p p p m m m v u p M p W 1 pixel coordinates world coordinates Conversion back from homogeneous coordinates leads to (note that = m T 3*P) : P m P m v P m P m u 3 2 3 1 W. Freeman
Extrinsic Parameters
Explicit Form of the Projection Matrix Note: M is only defined up to scale in this setting!!
Calibration target Find the position, u i and v i, in pixels, of each calibration object feature point. http://www.kinetic.bc.ca/compvision/opti-cl.html