Image Formation. 2. Camera Geometry. Focal Length, Field Of View. Pinhole Camera Model. Computer Vision. Zoltan Kato

Transcription

1 Image Formation 2. amera Geometr omuter Vision oltan Kato htt:// seged.hu/~kato/ 3D Scene Surace Light (Energ) Source inhole Lens Imaging lane World Otics Sensor Signal amera: Sec & ose 2D Image Slide adoted rom higang hu omuter Vision - S I676 2 inhole amera Model Otical Ais inhole lens in-hole is the basis or most grahics and vision Derived rom hsical construction o earlameras Mathematics is ver straightorward 3D World rojected to 2D Image Image inverted, sie reduced Image is a 2D lane: No direct deth inormation ersective rojection called the ocal length o the lens Image lane Focal Length, Field O View onsider case with object on the otical ais: viewoint Otical ais: the direction o imaging Image lane: a lane erendicular to the otical ais enter o rojection (inhole), ocal oint, viewoint, nodal oint Focal length: distance rom ocal oint to the image lane FOV : Field o View viewing angles in horiontal and vertical directions Increasing will enlarge igures, but decrease FOV Image lane Out o view Slide adoted rom higang hu omuter Vision - S I676 3 Slide adoted rom higang hu omuter Vision - S I676 4

2 Equivalent Geometr onsider case with object on the otical ais: More convenienith uright image: ersective rojection omute the image coordinates o in terms o the camera coordinates o. (, ) X ( X,, ) rojection lane Equivalent mathematicall Slide adoted rom higang hu omuter Vision - S I676 5 Origin o camera at center o rojection ais along otical ais Image lane at ; X and Slide adoted rom higang hu omuter Vision - S I676 6 inhole camera model Reverse rojection Given a center o rojection and image coordinates o a oint, it is not ossible to recover the 3D deth o the oint rom a single image. (X,,) can be anwhere along this line (,) X a X X X a X All oints on this line have image coordinates (,). In general, at least two images o the same oint taken rom two dierent locations are required to recover deth. 7 Slide adoted rom higang hu omuter Vision - S I676 8

3 Stereo Geometr l (X,,) entral rojection Ras Vergence Angle Object oint Deth obtained b triangulation orresondence roblem: l and r must corresond to the let and right rojections o, resectivel. Slide adoted rom higang hu omuter Vision - S I676 9 r inhole camera image straight line sie arallelism/angle shae shae o lanes deth Amsterdam : what do ou see in this icture? Slide adoted rom higang hu omuter Vision - S I676 0 hoto b Robert Kosara, robert@kosara.net htt:// inhole camera image inhole camera image Amsterdam Amsterdam straight line sie arallelism/angle shae shae o lanes deth hoto b Robert Kosara, robert@kosara.net htt:// straight line sie arallelism/angle shae shae o lanes deth hoto b Robert Kosara, robert@kosara.net htt:// Slide adoted rom higang hu omuter Vision - S I676 Slide adoted rom higang hu omuter Vision - S I676 2

4 inhole camera image Amsterdam hoto b Robert Kosara, robert@kosara.net htt:// straight line sie arallelism/angle shae shae o lanes deth Slide adoted rom higang hu omuter Vision - S I676 3 inhole camera image Amsterdam hoto b Robert Kosara, robert@kosara.net htt:// inhole camera image Amsterdam straight line sie arallelism/angle shae shae o lanes arallel to image deth Slide adoted rom higang hu omuter Vision - S I676 5 straight line sie arallelism/angle shae shae o lanes deth Slide adoted rom higang hu omuter Vision - S I676 4 inhole camera image straight line sie arallelism/angle shae shae o lanes arallel to image Deth? stereo motion sie structure Amsterdam: what do ou see? - We see satial shaes rather than individual iels - Knowledge: to-down vision belongs to human - Stereo &Motion most successul in 3D V & alication - ou can see it but ou don't know how Slide adoted rom higang hu omuter Vision - S I676 6 hoto b Robert Kosara, robert@kosara.net htt:// hoto b Robert Kosara, robert@kosara.net htt://

5 amera arameters im Image rame ( im, im ) im D oordinate Sstems Frame coordinates ( im, im ) iels Image coordinates (,) amera coordinates (X,,) World coordinates (X w, w, w ) amera arameters Intrinsic arameters (o the camera and the rame grabber): link the rame coordinates o an image oinith its corresonding camera coordinates Etrinsic arameters: deine the location and orientation o the camera coordinate sstem with resect to the world coordinate sstem ose / amera X X w w Object / World w w Intrinsic arameters () X From rame to image Image center Directions o aes iel sie Intrinsic arameters Sie: (s,s ) im X X + o s a + o (o,o ): rincial oint (image center) (s,s ): eective sie o the iel : ocal length (,,) o (0,0) o iel ( im, im ) im s o o X Slide adoted rom higang hu omuter Vision - S I676 7 Slide adoted rom higang hu omuter Vision - S I676 8 Intrinsic arameters (2) amera rotation and translation (, ) d d k,k2 ( + k r ( + k r Lens Distortions 2 2 (d, d) + k 2 + k 2 r r 4 4 ) ) Modeled as simle radial distortions r 2 d 2 +d 2 (d, d) distorted oints k, k2: distortion coeicients A model with k2 0 is still accurate or a D sensor o with ~5 iels distortion on the outer boundar im ( im, im ) im From World to amera R w + T T Etrinsic arameters A 3D translation vector, T, describing the relative locations o the origins o the two coordinate sstems A 33 rotation matri, R, an orthogonal matri that brings the corresonding aes o the two sstems onto each other X X w T T T R R, i. e. RR R R I w w w Slide adoted rom higang hu omuter Vision - S I676 9 Slide adoted rom higang hu omuter Vision - S I676 20

6 Etrinsic arameters: [R T] X cam R 0 X X ~ cam R ~ X R R X 0 K[ R T] T R ~ R X ~ [ 0] ( - ~ ) K I X KR[ I ~ cam ]X ~ rojection center in world coordinate rame. 2 Finite rojective camera K s KR[ I - ~ ] γ arctan(/s) degree o reedom (5+3+3) non-singular decomose in K,R,? [ M 4 ] ~ [ K, R] RQ( M) M 4 ~ rojection center in world coordinate rame. {inite cameras}{ 43 det M 0} I rank 3, but rank M<3, then camera is at ininit s skew or D/MOS, alwas s0 22 amera matri decomosition olumn vectors Finding the camera center 0 Algebraicall, ma be obtained as: (use SVD to ind null-sace) X det( [ 2,3, 4] ) det( [,3, 4] ) det( [,, ]) W det( [,, ]) Finding the camera orientation and internal arameters [ ] [ ] Image oints corresonding to X,, directions and origin ( 4 ) M KR (use RQ decomosition ~QR) (i onl QR, invert) ( Q R ) - R - - Q 23 24

7 Row vectors 0 T 2T 3T X Aine cameras 0 w T 2T 3T X note:, 2 deendent on image rearametriation Weak ersective rojection Weak ersective rojection Average deth is much larger than the relative distance between an two scene oints measured along the otical ais (, ) X (X,,) α r r 0 t t / k T 2T α 2 (7 degrees o reedom) 0 A sequence o two transormations Orthograhic rojection : arallel ras Isotroic scaling : / Linear Model reserve angles and shaes X Slide adoted rom higang hu omuter Vision - S I

8 Aine camera α r r 0 t t / k T s T A α 2 m m 0 m m 0 m m 0 t t 2 3 A A [ 3 3 aine] 0 0 0[ 4 4 aine] (8do). Aine cameracamera with rincial lane coinciding with the lan at ininit Π 2. Aine camera mas arallel lines to arallel lines 3. No center o rojection, but direction o rojection A D0 (oint on Π ) 29 Slides adoted rom: S 395/495-25: Sring 2004 IBMR: R 3 R 2 and 3 2 : The rojective amera Matri Jack Tumblin jet@cs.northwestern.edu ameras Revisited ameras Revisited lent o Terminolog: Image lane or Focal lane Focal Distance amera enter rincial oint rincial Ais rincial lane amera oords (,, ) Image oords (,, ) lent o Terminolog: Image lane or Focal lane Focal Distance amera enter rincial oint rincial Ais rincial lane amera oords (,, ) Image oords (,, )

9 ameras Revisited ameras Revisited lent o Terminolog: Image lane or Focal lane Focal Distance amera enter rincial oint rincial Ais rincial lane amera oords (,, ) Image oords (,, ) lent o Terminolog: Image lane or Focal lane Focal Distance amera enter rincial oint rincial Ais rincial lane amera oords (,, ) Image oords (,, ) ameras Revisited ameras Revisited lent o Terminolog: Image lane or Focal lane Focal Distance amera enter rincial oint rincial Ais rincial lane amera oords (,, ) Image oords (,, ) lent o Terminolog: Image lane or Focal lane Focal Distance amera enter rincial oint rincial Ais rincial lane amera oords (,, ) Image oords (,, )

10 ameras Revisited ameras Revisited lent o Terminolog: Image lane or Focal lane Focal Distance amera enter rincial oint rincial Ais rincial lane amera oords (,, ) Image oords (,, ) [0,0,0] lent o Terminolog: Image lane or Focal lane Focal Distance amera enter rincial oint rincial Ais rincial lane amera oords (,, ) Image oords (,, ) ameras Revisited Basic amera 0 : 3 2 (oror camera R 3 ) Goal: Formalie rojective 3D 2D maing Homogeneous coords handles ininities well: rojective cameras ( convergent ee ras) Aine cameras (arallel ee ras) omosed, controlled as matri roduct Recall Euclidian R 3 R 2 : / / (Much Better: 3 2 ) Basic amera 0 is a 34 matri: α s 0 0 α K (33 submatri) [K 0] 0 0 X Non-square iels? change scaling (α, α ) arallelogram iels? set nonero skew s K matri: : (internal) camera calib.. matri

11 omlete amera Matri K matri: : internal camera calib.. matri R T T matri: : eternal camera calib.. matri T matri: Translate world to cam. origin R matri: 3D rotate world to it cam. aes ombine: write ( 0 R T) T) X X as X X Inut: X 3D World Sace X (world sace) Outut: 2D amera Image (camera sace) The ieces o amera Matri X X, or w w olumns o matri image o world-sace aes:, 2, 3 image o,, ais vanishing oints Direction D [ 0 0 0] T oint on 3 s s ais, at iniinit D st column o. Reeat or and aes. 4 image o the world-sace origin t. roo: let D [0 0 0 ]T direction to world origin D 4 th column o image o origin t., or The ieces o amera Matri X X, or w w Rows o matri: camera lanes in world sace row T image -ais lane row 2 2T image -ais lane row 3T camera s s rincial lane, or T 2T 3T The ieces o amera Matri X X, or w w Rows o matri: lanes in world sace row world lane whose image is 0 row 2 2 world lane whose image is 0 row 3 3 camera s s rincial lane, or T 2T 3T Wh? Recall that in 3, oint X is on lane π i and onl i πt X X 0, so

12 The ieces o amera Matri X X, or w w Rows o matri: lanes in world sace row world lane whose image is 0 row 2 2 world lane whose image is 0 row 3 3 camera s s rincial lane, or T 2T 3T The ieces o amera Matri X X, or w w, or Rows o matri: lanes in world sace row image s s 0 lane row 2 2 image s s 0 lane areul! Shiting the image origin b, shits the 0,0 lanes! 2T 3T T The ieces o amera Matri The ieces o amera Matri X X, or w w, or Rows o matri: lanes in world sace row image s s 0 lane row 2 2 image s s 0 lane row 3 3 camera s s rincial lane T 2T 3T X X, or w w, or Rows o matri: lanes in world sace row image -ais lane row 2 2 image -ais lane row 3 3 camera s s rincial lane rinci.. lane 3 [ ] T its normal direction: [ ] T Wh is it normal? It s s the world-sace 3 direction o the ais T 2T 3T rincial lane 3

13 The ieces o amera Matri X X, or w w M rincial Ais Vector (( c ) in world sace:, or Normal o rincial lane: m 3 [ Scaling Ambiguous direction!! +/- m 3? Solution: use det(m) m 3 as ront o camera rincial oint in image sace: image o (ininit oint on ais m 3 ) M m 3 0 (isserman 33 ] T isserman book renames as 0 ) m T m 2T m 3T 0 4 The ieces o amera Matri X X, or w w Where is camera in world sace? at : amera center is at camera origin( c,, )0 ~ amera transorms world-sace oint to 0 ~ But how do we ind? It is the Null Sace o : ~ 0 ~ (solve or. SVD works, but here s s an easier wa:) amera s s osition in the World: ~ -M - 4 ~ m T m 2T m 3T 4 M