Lecture 4 Single View Metrology Professor Silvio Svrese Computtionl Vision nd Geometry Lb Silvio Svrese Lecture 4-4-Jn-5
Lecture 4 Single View Metrology Review clibrtion nd 2D trnsformtions Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projective Geometry nd Trnsformtion in 2D [HZ] Chpter 3 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoeim Svrese] Chpter 2 Silvio Svrese Lecture 4-4-Jn-5
Clibrtion Problem j C u P M P p i i i i vi World ref. system In piels M K[ R T] K α α cotθ β sinθ u v o o
Clibrtion Problem j C u P M P p i i i i vi World ref. system In piels M K[ R T] unknown Need t lest 6 correspondences
Once the cmer is clibrted... Pinhole perspective projection P Line of sight p O w C M K[ R T] -Internl prmeters K re known -R, T re known but these cn only relte C to the clibrtion rig Cn I estimte P from the mesurement p from single imge? No - in generl [P cn be nywhere long the line defined by C nd p]
Recovering structure from single view Pinhole perspective projection P Line of sight p O w C unknown known Known/ Prtilly known/ unknown
Recovering structure from single view http://www.robots.o.c.uk/~vgg/projects/singleview/models/hut/hutme.wrl
Trnsformtion in 2D -Isometries -Similrities -Affinity -Projective
Trnsformtion in 2D Isometries: y H y t R y' ' e - Preserve distnce (res) - 3 DOF - Regulte motion of rigid object [Euclidens] [Eq. 4]
Trnsformtion in 2D Similrities: y H y t R s y' ' s - Preserve - rtio of lengths - ngles -4 DOF [Eq. 5]
Trnsformtion in 2D Affinities: y H y t A y' ' 22 2 2 A ) R( D ) R( ) ( R φ φ θ y s s D [Eq. 6] [Eq. 7]
Trnsformtion in 2D Affinities: y H y t A y' ' 22 2 2 A ) R( D ) R( ) ( R φ φ θ y s s D -Preserve: - Prllel lines - Rtio of res - Rtio of lengths on colliner lines - others - 6 DOF [Eq. 6] [Eq. 7]
Trnsformtion in 2D Projective: y H y b v t A y' ' p - 8 DOF - Preserve: - cross rtio of 4 colliner points - collinerity - nd few others [Eq. 8]
The cross rtio 4 2 3 2 4 3 P P P P P P P P The cross-rtio of 4 colliner points Cn permute the point ordering i i i i Z Y X P 3 4 2 2 4 3 P P P P P P P P P P 2 P 3 P 4 [Eq. 9]
Lecture 4 Single View Metrology Review clibrtion nd 2D trnsformtions Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projective Geometry nd Trnsformtion in 2D [HZ] Chpter 3 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoeim Svrese] Chpter 2 Silvio Svrese Lecture 4-4-Jn-5
Lines in 2D plne c by + + -c/b -/b c b l If [, 2 ] T l c b T 2 l y [Eq. ]
Lines in 2D plne Intersecting lines y l l l Proof l [Eq. ] l l l ( l l ) l l [Eq. 2] l l l ( l l ) l l [Eq. 3] is the intersecting point
2D Points t infinity (idel points), 3 3 2 ' c b l ' ' c b l ' ' ) b l l Let s intersect two prllel lines: In Euclidin coordintes this point is t infinity Agree with the generl ide of two lines intersecting t infinity l l ' ' 2 ' / ' / b b Eq.3]
2D Points t infinity (idel points), 3 3 2 ' c b l ' ' c b l ' ' [ ] l T b c b Note: the line l [ b c] T pss trough the idel point So does the line l since b b l l ' / ' / b b [Eq. 5]
Lines infinity l Set of idel points lies on line clled the line t infinity How does it look like? l l T 2 Indeed: A line t infinity cn be thought of the set of directions of lines in the plne ' ' ' b '' '' b ''
Projective trnsformtion of point t infinity p p H ' b v t A H? p H z y p p p b v t A ' ' ' is it point t infinity? no? p H A ' ' y p p b t A An ffine trnsformtion of point t infinity is still point t infinity [Eq. 7] [Eq. 8]
Projective trnsformtion of line (in 2D) l H l T b v t A H? l H T b t t b v t A y T is it line t infinity? no? l H T A T T T T A t A t A [Eq. 9] [Eq. 2] [Eq. 2]
Points nd plnes in 3D 3 2 Π d c b d cz by + + + y z Π T Π How bout lines in 3D? Lines hve 4 degrees of freedom - hrd to represent in 3D-spce Cn be defined s intersection of 2 plnes [Eq. 23] [Eq. 22]
Points t infinity in 3D Points where prllel lines intersect in 3D world point t infinity Prllel lines 2 3
Vnishing points The projective projection of point t infinity into the imge plne defines vnishing point. M 3 2 3 2 p p p p world p Prllel lines point t infinity direction of the line in 3D
[ ] c b c b K K I M X v v K d d direction of the line [, b, c] T d C v c b M v v d K K [Eq. 24] [Eq. 25] Proof: Vnishing points nd directions
Vnishing (horizon) line l π horizon Projective trnsformtion M l hor T H l [Eq. 26] P Imge
Are these two lines prllel or not? Recognition helps reconstruction Humns hve lernt this - Recognize the horizon line - Mesure if the 2 lines meet t the horizon - if yes, these 2 lines re // in 3D
Vnishing points nd plnes l n π l horiz C T n K l [Eq. 27] horiz
Plnes t infinity Π z y Π plne t infinity Prllel plnes intersect t infinity in common line the line t infinity A set of 2 or more lines t infinity defines the plne t infinity Π
Angle between 2 vnishing points 2 θ d v 2 d 2 v C cosθ [Eq. 28] v T ω v T v ω v v 2 T 2 ω v 2 ω (K T K) If 9 v T 2 θ ω v Sclr eqution [Eq. 29]
Projective trnsformtion of Ω Absolute conic M T Ω M ( K K T ) [Eq. 3] M K R T. It is not function of R, T 2. ω ω2 ω4 ω ω2 ω3 ω5 ω 4 ω5 ω6 ω 2 ω2 zero-skew ω ω3 3. 4. symmetric nd known up scle squre piel
Why is this useful? To clibrte the cmer To estimte the geometry of the 3D world
Lecture 4 Single View Metrology Review clibrtion Vnishing points nd line Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 3 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoeim Svrese] Chpter 2 Silvio Svrese Lecture 4-4-Jn-5
Single view clibrtion - emple v 2 θ 9 v v T ω v 2 ω (K T K) Do we hve enough constrints to estimte K? K hs 5 degrees of freedom nd Eq.29 is sclr eqution
Single view clibrtion - emple v 2 v 2 v3 ω known up to scle ω ω ω 2 4 ω ω ω 2 3 5 ω ω ω 4 5 6 [Eqs. 3] T v ω v2 T v ω v3 T v ω v3 2 ω Compute ω : 2 ω ω 3 Once ω is clculted, we get K: ω (K T K) K (Cholesky fctoriztion; HZ pg 582)
Single view reconstruction - emple l h K known T n K l horiz Scene plne orienttion in the cmer reference system Select orienttion discontinuities
Single view reconstruction - emple C Recover the structure within the cmer reference system Notice: the ctul scle of the scene is NOT recovered Recognition helps reconstruction Humns hve lernt this
Lecture 4 Single View Metrology Review clibrtion Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 3 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoeim Svrese] Chpter 2 Silvio Svrese Lecture 4-4-Jn-5
Criminisi Zissermn, 99 http://www.robots.o.c.uk/~vgg/projects/singleview/models/merton/merton.wrl
Criminisi Zissermn, 99 http://www.robots.o.c.uk/~vgg/projects/singleview/models/merton/merton.wrl
L Trinit' (426) Firenze, Snt Mri Novell; by Msccio (4-428)
L Trinit' (426) Firenze, Snt Mri Novell; by Msccio (4-428)
http://www.robots.o.c.uk/~vgg/projects/singleview/models/hut/hutme.wrl
Single view reconstruction - drwbcks Mnully select: Vnishing points nd lines; Plnr surfces; Occluding boundries; Etc..
Automtic Photo Pop-up Hoiem et l, 5
Automtic Photo Pop-up Hoiem et l, 5
Automtic Photo Pop-up Hoiem et l, 5 Softwre: http://www.cs.uiuc.edu/homes/dhoiem/projects/softwre.html
Mke3D Trining Imge Sen, Sun, Ng, 5 Prediction Depth Plne Prmeter MRF Plnr Surfce Segmenttion youtube Connectivity Co-Plnrity
Single Imge Depth Reconstruction Sen, Sun, Ng, 5 A softwre: Mke3D Convert your imge into 3d model http://mke3d.stnford.edu/ http://mke3d.cs.cornell.edu/
Coherent object detection nd scene lyout estimtion from single imge Y. Bo, M. Sun, S. Svrese, CVPR 2, BMVC 2 M. Sun Y. Bo
Net lecture: Multi-view geometry (epipolr geometry)