Geometric Model of Camera

Transcription

1 Geometric Model of Camera Dr. Gerhard Roth COMP 42A Winter 25 Version 2

2 Similar Triangles 2

3 Geometric Model of Camera Perspective projection P(X,Y,Z) p(,) f X Z f Y Z 3

4 Parallel lines aren t 4 Figure b David Forsth

5 Lengths can t be trusted... 5

6 Coordinate Transformation 2D p p p p p cos sin sin cos ' ' ' P ' '' ' '' T T T p p T T p p p cos sin sin cos ' ' '' Rotation and Translation 6

7 Homogeneous Coordinates Go one dimensional higher: z z is an arbitrar non-zero scalar, usuall e choose / / / / / From homogeneous coordinates to Cartesian coordinates: 7

8 2D Transformation ith Homogeneous Coordinates 2D coordinate transformation using homogeneous coordinates: T T p p p cos sin sin cos '' cos sin sin cos '' '' p p T T p p 2D coordinate transformation: 8

9 Homogeneous coordinates (In 2d) To points are equal if and onl if: / = / and / = / =: points at infinit useful for projections and curve draing Homogenize = divide b. Homogenized point eample: = = / = 644/7 = = / = 78/7 =

10 Wh use homogeneous co-ordinates We require a composition (sequence of) rotations, translations and projections Even the projection is a matri multiplication Each of these can be described b matri operations using homogeneous coordinates Composing them together, appling them one after the other just matri multiplication The final operation, hich takes a 3d point and produces a 2d image is one big matri multiplication

11 Translations ith homogeneous t t t t t t

12 Scaling ith homogeneous s s s s s s 2

13 Rotation ith homogeneous co-ord s cos sin sin cos cos sin sin cos cos sin sin cos 3

14 3D Rotation about X-Ais Representation in homogeneous co-ordinates

15 3D Rotation about Y-Ais Representation in homogeneous co-ordinates 5

16 3D Rotation about Z-Ais Representation in homogeneous co-ordinates 6

17 3D Rotation Matri Euler Angles Rotate around each coordinate ais: R R Rz R ( ) cos sin sin cos R 2 ( ) cos sin sin cos sin R3( ) sin cos cos Combine the three rotations: R RRRz Can describe an 3d rotation b a sequence of rotations about the three ais So R = RRRz or RRzR or RRRz or RRzR or RzRR or RzRR 7

18 3d Rotation Matri When ou specif the values for each ais ou musts also specif order of operations Different orders have different angle values Succeeding rotations are about an alread modified set of three ais Remember matri multiplication is not commutative AB is not same as BA A rotation matri has 9 elements but e need onl three numbers to specif a 3d rotation uniquel! 8

19 3d Rotation Matri - Eamples 9

20 3D Rotation Rodriguez Rotations also described b ais and angle Rotation ais (2 parameters) and amount of rotation (angle parameter) Can convert beteen Euler Angles and Rodriguez representation 2

21 Degrees of freedom This is the number of parameters necessar to generate all possible instances of a given geometric object A 2d rotation has one degree of freedom Varing one parameter (angle) ill generate ever possible 2d rotation What are degrees of freedom of a 3d rotation It is a 3 b 3 matri ith nine numbers so is it 9? No, the anser is three (alas!) B varing three parameters e can generate ever possible 3d rotation matri (for ever representation!) Wh is it 3 and not 9? 2

22 Rotation Matrices Both 2d and 3d rotation matrices have to characteristics The are orthogonal (also called orthonormal) R T R I R T R Their determinant is Matri belo is orthogonal but not a rotation matri because the determinate is not this is a reflection matri 22

23 Rotation Matrices Ros and columns of a rotation matri are unit vectors Ever ro is orthogonal to ever other ros Their dot product is zero Ever column is orthogonal to ever other column Their dot product is zero These etra constraints mean that the entries in the rotation matri are not independent This reduces the degrees of freedom 23

24 3d Rotation Matri - Inversions Remember that cos(-t) = cos(t) And that sin(-t) = -sin(t) Also for rotation matrices hich e can see R T R I R T R 24

25 Homogeneous co-ordinates Transformation transform a point in an n dimensional space to another n dim point Transformations are scale, rotations, translations, etc. You can represent all these b multiplication b one appropriate matri using homogeneous co-ordinates Projection transform a point in an n dimensional space to an m dim point For projection m is normall less than n Perspective projection is a projection (3d to 2d) From the 3d orld to a 2d point in the image You can also represent a projection as matri multiplication ith one appropriate matri and homogeneous co-ordinates 25

26 Four Coordinate Frames im piel frame Z P Y Yc p im im camera frame optical center Xc Zc image plane frame principal point orld frame X Camera model: p transforma tion matri im P 26

27 Perspective Projection Z X f Z Y f These are nonlinear. Z Y X f f v u Using homogenous coordinate, e have a linear relation: u / v / 27

28 World to Camera Coordinate T RX X c T R c c c Z Y X Z Y X Transformation beteen the camera and orld coordinates. Here e rotate, then translate, to go from orld to camera co-ordinates hich is opposite of book, but is simpler and is the a in hich OpenCV routines do it: After R, and T e have converted from orld to camera frame. In the camera frame the z ais is along the optical center. 28

29 Camera Coordinates to Image Coordinates ( o ) s s im, s : ( o ) s im piel sizes in millimeters per piel (o,o) im im im im / s / s o o Camera co-ordinates, and are in millimetres Image co-ordinates im, im, are in piels Center of projection o, o is in piels Sign change because horizontal and vertical ais of the image and camera frame have opposite directions. 29

30 Image and Camera frames No e look from the camera outard and image origin is the top left piel (,) Zcamera Xcamera (im, im) (o,o) Ycamera Ximage Yimage 3

31 Put All Together World to Piel / / / / / / / / 3 2 c c c Z Y X T R o s f o s f Z Y X T R f f o s o s Z Y X f f o s o s v u o s o s Z Y X T R K / 3 im 3 im 2 / From camera to piel Add projection Add Rotation And Translation 3

32 Perspective Projection Matri Projection is a matri multiplication using homogeneous coordinates f f z f f z In practice: lots of coordinate transformations ( f z, f z ) divide b the third coordinate 2D point (3) = Camera to piel coord. trans. matri (33) Perspective projection matri (34) World to camera coord. trans. matri (44) 3D point (4) 32

33 Camera Parameters Etrinsic parameters define the location and orientation of the camera reference frame ith respect to a orld reference frame Depend on the eternal orld, so the are etrinsic Intrinsic parameters link piel co-ordinates in the image ith the corresponding coordinates in the camera reference frame An intrinsic characteristic of the camera Image co-ordinates are in piels Camera co-ordinates are in millimetres In formulas that do conversions the units must match! 33

34 Intrinsic Camera Parameters K f / s f / s o o K is a 33 upper triangular matri, called the Camera Calibration Matri. There are five intrinsic parameters: (a) The piel sizes in and directions s, s in millimeters/piel (b) The focal length f in millimeters (c) The principal point (o,o) in piels, hich is the point here the optic ais intersects the image plane. 34 (d) The units of f/s and f/s are in piels, h is this so?

35 Camera intrinsic parameters Can rite three of these parameters differentl b letting f/s = f and f/s = f Then intrinsic parameters are o,o,f,f The units of these parameters are piels! In practice piels are square (s = s) so that means f should equal f for most cameras Hoever, ever eplicit camera calibration process (using calibration objects) introduces some small errors These calibration errors make f not eactl equal to f So in OpenCV the intrinsic camera parameters are the four folloing o,o,f,f Hoever f is usuall ver close to f and if this is not the case then there is a problem 35

36 Etrinsic Parameters and Proj. Matri [R T] defines the etrinsic parameters. The 34 matri M = K[R T] is called the projection matri. It takes 3d points in the orld co-ordinate sstem and maps them to the appropriate image co-ordinates in piels 3 2 im Z Y X M Z Y X T R K p 36

37 Create a complete projection matri Camera located at T c =[T, T, T z ] t =[,2,3] t Camera aimed along z-ais,, aes parallel to orld aes (R=I) T I I I z I I I z I z I z I zz T c = = 2 R= = T T z 3 S= Camera has focal length f = f = image=6448, assume u =32, v =24 Calculate complete 34 projection matri P P 2 P 3 P 4 P 2 P 22 P 23 P = -2 = P 3 P 32 P 33 P

38 Using the projection matri - eample P P 2 P 3 P 4 P 2 P 22 P 23 P 24 P 3 P 32 P 33 P 34 = Where ould the point (2,5,2) project to in the image? Projection matri u v = P P 2 P 3 P 4 P 2 P 22 P 23 P 24 P 3 P 32 P 33 P 34 X Y Z = u v = u=u / = 644/7 = v=v / = 78/7 = 46.5 World point (2,5,2) project to piel With co-ordinates of (379,47) 38

39 Using the projection matri - eample P P 2 P 3 P 4 P 2 P 22 P 23 P 24 P 3 P 32 P 33 P 34 = Where ould the point (,2,2) project to in the image? Projection matri u v = P P 2 P 3 P 4 P 2 P 22 P 23 P 24 P 3 P 32 P 33 P 34 X Y Z = u v = u=u / = 544/7 = 32 v=v / = 48/7 = 24 39

40 Effect of change in focal length Small f is ide angle, large f is telescopic 4

41 Orthographic Projection X I m X Y M X Y Z C c Z Y 4

42 Orthographic Projection X I O 3 C Z O O2 Y 42

43 Orthographic Projection Special case of perspective projection Distance from center of projection to image plane is infinite Image World Also called parallel projection What s the projection matri? 43 Slide b Steve Seitz

44 Weak Perspective Model Assume the relative distance beteen an to points in an object along the principal ais is much smaller (/2 th at most) than the Z average distance of the object. Then the camera projection can be approimated as: f X Z f Z X f Y Z f Z Y This is the eak-perspective camera model. Sometimes called scaled orthograph. 44

45 Weak Perspective Projection To step projection: X I C Z c Z Y plane image plane average depth 45

46 Weak Perspective 46

47 Impact of different projections Perspective projections Parallel lines in orld are not parallel in the image Object projection gets smaller ith distance from camera Weak perspective projection Parallel lines in the orld are parallel in the image Object projection gets smaller ith distance from camera Orthographic projection Parallel lines in the orld are parallel in the image Object projection is unchanged ith distance from camera 47

48 Ordinar Perspective Parallel lines in orld - not parallel in image 48

49 Weak Perspective Parallel lines in orld parallel in image 49

50 Image distortion due to optics Radial distortion hich depends on radius r, distance of each point from center of image r 2 = (- o) 2 +(- o) 2 Correction uses three parameters, k,k 2,k 3 5

51 Radial Distortion Error is proportional to distance of piel from the camera center (the radius of the point) Barrel too far, Pincushion too close 5

52 Correcting Radial Distortions 52

53 Tangential Distortion (O, O) center of projection) is not the center of image Also causes a more comple distortion of the image 53

54 Tangential Distortion Lens not eactl parallel to the image plane Correction uses to parameter p, p 2 Both tpes of distortion are removed (image is un-distorted) and onl then does standard calibration matri K appl to the image Camera calibration computes both K and these five distortion parameters 54

55 Ho to find the camera parameters K Can use the EXIF tag for an digital image Has focal length f in millimeters but not the piel size But ou can get the piel size from the camera manual There are onl a finite number of different piels sizes because the number of sensing element sizes is limited If there is not a lot of image distortion due to optics then this approach is sufficient (this is onl a linear calibration) Can perform eplicit camera calibration Put a calibration pattern in front of the camera Take a number of different pictures of this pattern No run the calibration algorithm (different tpes) Result is intrinsic camera parameters (linear and non-linear) and the etrinsic camera parameters of all the images 55

56 Summar Questions Wh do e use homogeneous co-ordinates for image projection? Write the projection equation in a linear form using a matri ith homogeneous co-ordinates. What are the units of image co-ordinates and camera co-ordinates? Ho do ou convert beteen them? Wh do the names etrinsic, and intrinsic parameters of a lens make sense? What are the units of f/s, and f/s? Implications? What are characteristics of perspective, eak perspective and orthographic projection? What are parameters of models for non-linear lens characteristics? 56