Lecture 4 Single View Metrology

Transcription

1 Lecture 4 Single View Metrology Professor Silvio Svrese Computtionl Vision nd Geometry Lb Silvio Svrese Lecture 4-4-Jn-5

2 Lecture 4 Single View Metrology Review clibrtion nd 2D trnsformtions Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projective Geometry nd Trnsformtion in 2D [HZ] Chpter 3 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoeim Svrese] Chpter 2 Silvio Svrese Lecture 4-4-Jn-5

3 Clibrtion Problem j C u P M P p i i i i vi World ref. system In piels M K[ R T] K α α cotθ β sinθ u v o o

4 Clibrtion Problem j C u P M P p i i i i vi World ref. system In piels M K[ R T] unknown Need t lest 6 correspondences

5 Once the cmer is clibrted... Pinhole perspective projection P Line of sight p O w C M K[ R T] -Internl prmeters K re known -R, T re known but these cn only relte C to the clibrtion rig Cn I estimte P from the mesurement p from single imge? No - in generl [P cn be nywhere long the line defined by C nd p]

6 Recovering structure from single view Pinhole perspective projection P Line of sight p O w C unknown known Known/ Prtilly known/ unknown

7 Recovering structure from single view

8 Trnsformtion in 2D -Isometries -Similrities -Affinity -Projective

9 Trnsformtion in 2D Isometries: y H y t R y' ' e - Preserve distnce (res) - 3 DOF - Regulte motion of rigid object [Euclidens] [Eq. 4]

10 Trnsformtion in 2D Similrities: y H y t R s y' ' s - Preserve - rtio of lengths - ngles -4 DOF [Eq. 5]

11 Trnsformtion in 2D Affinities: y H y t A y' ' A ) R( D ) R( ) ( R φ φ θ y s s D [Eq. 6] [Eq. 7]

12 Trnsformtion in 2D Affinities: y H y t A y' ' A ) R( D ) R( ) ( R φ φ θ y s s D -Preserve: - Prllel lines - Rtio of res - Rtio of lengths on colliner lines - others - 6 DOF [Eq. 6] [Eq. 7]

13 Trnsformtion in 2D Projective: y H y b v t A y' ' p - 8 DOF - Preserve: - cross rtio of 4 colliner points - collinerity - nd few others [Eq. 8]

14 The cross rtio P P P P P P P P The cross-rtio of 4 colliner points Cn permute the point ordering i i i i Z Y X P P P P P P P P P P P 2 P 3 P 4 [Eq. 9]

15 Lecture 4 Single View Metrology Review clibrtion nd 2D trnsformtions Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projective Geometry nd Trnsformtion in 2D [HZ] Chpter 3 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoeim Svrese] Chpter 2 Silvio Svrese Lecture 4-4-Jn-5

16 Lines in 2D plne c by + + -c/b -/b c b l If [, 2 ] T l c b T 2 l y [Eq. ]

17 Lines in 2D plne Intersecting lines y l l l Proof l [Eq. ] l l l ( l l ) l l [Eq. 2] l l l ( l l ) l l [Eq. 3] is the intersecting point

18 2D Points t infinity (idel points), ' c b l ' ' c b l ' ' ) b l l Let s intersect two prllel lines: In Euclidin coordintes this point is t infinity Agree with the generl ide of two lines intersecting t infinity l l ' ' 2 ' / ' / b b Eq.3]

19 2D Points t infinity (idel points), ' c b l ' ' c b l ' ' [ ] l T b c b Note: the line l [ b c] T pss trough the idel point So does the line l since b b l l ' / ' / b b [Eq. 5]

20 Lines infinity l Set of idel points lies on line clled the line t infinity How does it look like? l l T 2 Indeed: A line t infinity cn be thought of the set of directions of lines in the plne ' ' ' b '' '' b ''

21 Projective trnsformtion of point t infinity p p H ' b v t A H? p H z y p p p b v t A ' ' ' is it point t infinity? no? p H A ' ' y p p b t A An ffine trnsformtion of point t infinity is still point t infinity [Eq. 7] [Eq. 8]

22 Projective trnsformtion of line (in 2D) l H l T b v t A H? l H T b t t b v t A y T is it line t infinity? no? l H T A T T T T A t A t A [Eq. 9] [Eq. 2] [Eq. 2]

23 Points nd plnes in 3D 3 2 Π d c b d cz by y z Π T Π How bout lines in 3D? Lines hve 4 degrees of freedom - hrd to represent in 3D-spce Cn be defined s intersection of 2 plnes [Eq. 23] [Eq. 22]

24 Points t infinity in 3D Points where prllel lines intersect in 3D world point t infinity Prllel lines 2 3

25 Vnishing points The projective projection of point t infinity into the imge plne defines vnishing point. M p p p p world p Prllel lines point t infinity direction of the line in 3D

26 [ ] c b c b K K I M X v v K d d direction of the line [, b, c] T d C v c b M v v d K K [Eq. 24] [Eq. 25] Proof: Vnishing points nd directions

27 Vnishing (horizon) line l π horizon Projective trnsformtion M l hor T H l [Eq. 26] P Imge

28 Are these two lines prllel or not? Recognition helps reconstruction Humns hve lernt this - Recognize the horizon line - Mesure if the 2 lines meet t the horizon - if yes, these 2 lines re // in 3D

29 Vnishing points nd plnes l n π l horiz C T n K l [Eq. 27] horiz

30 Plnes t infinity Π z y Π plne t infinity Prllel plnes intersect t infinity in common line the line t infinity A set of 2 or more lines t infinity defines the plne t infinity Π

31 Angle between 2 vnishing points 2 θ d v 2 d 2 v C cosθ [Eq. 28] v T ω v T v ω v v 2 T 2 ω v 2 ω (K T K) If 9 v T 2 θ ω v Sclr eqution [Eq. 29]

32 Projective trnsformtion of Ω Absolute conic M T Ω M ( K K T ) [Eq. 3] M K R T. It is not function of R, T 2. ω ω2 ω4 ω ω2 ω3 ω5 ω 4 ω5 ω6 ω 2 ω2 zero-skew ω ω symmetric nd known up scle squre piel

33 Why is this useful? To clibrte the cmer To estimte the geometry of the 3D world

34 Lecture 4 Single View Metrology Review clibrtion Vnishing points nd line Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 3 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoeim Svrese] Chpter 2 Silvio Svrese Lecture 4-4-Jn-5

35 Single view clibrtion - emple v 2 θ 9 v v T ω v 2 ω (K T K) Do we hve enough constrints to estimte K? K hs 5 degrees of freedom nd Eq.29 is sclr eqution

36 Single view clibrtion - emple v 2 v 2 v3 ω known up to scle ω ω ω 2 4 ω ω ω ω ω ω [Eqs. 3] T v ω v2 T v ω v3 T v ω v3 2 ω Compute ω : 2 ω ω 3 Once ω is clculted, we get K: ω (K T K) K (Cholesky fctoriztion; HZ pg 582)

37 Single view reconstruction - emple l h K known T n K l horiz Scene plne orienttion in the cmer reference system Select orienttion discontinuities

38 Single view reconstruction - emple C Recover the structure within the cmer reference system Notice: the ctul scle of the scene is NOT recovered Recognition helps reconstruction Humns hve lernt this

39 Lecture 4 Single View Metrology Review clibrtion Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 3 Projective Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoeim Svrese] Chpter 2 Silvio Svrese Lecture 4-4-Jn-5

40 Criminisi Zissermn, 99

41 Criminisi Zissermn, 99

42 L Trinit' (426) Firenze, Snt Mri Novell; by Msccio (4-428)

43 L Trinit' (426) Firenze, Snt Mri Novell; by Msccio (4-428)

44

45 Single view reconstruction - drwbcks Mnully select: Vnishing points nd lines; Plnr surfces; Occluding boundries; Etc..

46 Automtic Photo Pop-up Hoiem et l, 5

47 Automtic Photo Pop-up Hoiem et l, 5

48 Automtic Photo Pop-up Hoiem et l, 5 Softwre:

49 Mke3D Trining Imge Sen, Sun, Ng, 5 Prediction Depth Plne Prmeter MRF Plnr Surfce Segmenttion youtube Connectivity Co-Plnrity

50 Single Imge Depth Reconstruction Sen, Sun, Ng, 5 A softwre: Mke3D Convert your imge into 3d model

51 Coherent object detection nd scene lyout estimtion from single imge Y. Bo, M. Sun, S. Svrese, CVPR 2, BMVC 2 M. Sun Y. Bo

52 Net lecture: Multi-view geometry (epipolr geometry)