Two View Geometry Part 2 Fundamental Matrix Computation

Transcription

1 3D Computer Visio II Two View Geometr Part Fudametal Matri Computatio Nassir Navab based o a course give at UNC b Marc Pollefes & the book Multiple View Geometr b Hartle & Zisserma November 9, 009

2 Outlie Two-View Geometr Epipolar Geometr Fudametal Matri Computatio 3D Recostructio 3D Computer Visio II - Two View Geometr

3 Remider Three Questios (i) Correspodece geometr: Give a image poit i the first image, how does this costrai the positio of the correspodig poit i the secod image? (ii) Camera geometr (motio): Give a set of correspodig image poits { i i }, i=,,, what are the cameras P ad P for the two views? (iii) Scee geometr (structure): Give correspodig image poits i i ad cameras P, P, what is the positio of (their pre-image) X i space? 3D Computer Visio II - Two View Geometr 3

4 The Fudametal Matri F Algebraic represetatio of Epipolar Geometr: l' We will see that mappig is (sigular) correlatio (i.e. projective mappig from poits to lies) represeted b the fudametal matri F. 3D Computer Visio II - Two View Geometr 4

5 Poits From Lies ad Vice-versa Itersectios of lies The itersectio of two lies l ad l' is ll' [ l] l' [ l' ] l Lie joiig two poits The lie through two poits ad ' is l ' [] ' [' ] Ati-smmetric Matri Eample [ l] 0 l 3 l 0 l l 3 l l 0 3D Computer Visio II - Two View Geometr 5

6 The Fudametal Matri F Geometric Derivatio ' H π l' e' ' e' Hπ F mappig from -D to -D famil (rak ) 3D Computer Visio II - Two View Geometr 6

7 The Fudametal Matri F Algebraic derivatio: X λ ( λ)c λp P P I l P'C P' P P Xλ F e' P' P (ote: does t work for C=C F=0) 3D Computer Visio II - Two View Geometr 7

8 The Fudametal Matri F Correspodece coditio: The fudametal matri satisfies the coditio that for a pair of correspodig poits i the two images: ' T F 0 ' T l' 0 3D Computer Visio II - Two View Geometr 8

9 The Fudametal Matri F F is the uique 33 rak matri that satisfies T F=0 for all. (i) Traspose: if F is fudametal matri for (P,P ), the F T is fudametal matri for (P,P) (ii) Epipolar lies: l =F & l=f T (iii) Epipoles: o all epipolar lies, thus e T F=0, e T F=0, similarl Fe=0 (iv) F has 7 d.o.f., i.e. 33-(homogeeous)-(rak) (v) F is a correlatio, projective mappig from a poit to a lie l =F (ot a proper correlatio, i.e. ot ivertible) 3D Computer Visio II - Two View Geometr 9

10 Epipolar Geometr l e p l T T P P l π P T l C m l M L L T l 0 e T T T P P [e] 0 e m l C

11 Epipolar Geometr Caoical represetatio: P [I 0] P' [[e' ] F e' v T λe' ]. Computable from correspodig poits. Simplifies matchig 3. Allows to detect wrog matches 4. Related to calibratio 3D Computer Visio II - Two View Geometr

12 Epipolar Geometr Basic Equatios ' T F 0 ' f ' f ' f3 ' f ' f ' f3 f3 f3 f separate kows from ukows: T ', ', ', ', ', ',,, f, f, f, f, f, f, f, f, f (data) (ukows) (liear) 3D Computer Visio II - Two View Geometr 3

13 Af 0 0 f ' ' ' ' ' ' ' ' ' ' ' ' Epipolar Geometr Basic Equatios 4 3D Computer Visio II - Two View Geometr

14 0 F e' T 0 Fe 0 detf F rak T T T T 3 V U σ V σ U V U σ V σ σ σ U F SVD from liearl computed F matri (rak 3) T T T V σ U V U σ V 0 σ σ U F' F F- F' mi Compute closest rak- approimatio The Sigularit Costrait 5 3D Computer Visio II - Two View Geometr

15 The Sigularit Costrait No sigular F Forcig sigularit usig the SVD method 3D Computer Visio II - Two View Geometr 6

16 0 f ' ' ' ' ' ' ' ' ' ' ' ' T V,0,0,...,σ σ diag A U V ] A[V...7 0, λf ) (F T i i i Oe parameter famil of solutios But F +lf ot automaticall rak The Miimum Case 7 Poit Correspodeces 7 3D Computer Visio II - Two View Geometr

17 The Miimum Case Impose Rak 3 (obtai or 3 solutios) F 7pts F F F 3 det( F λf ) a3λ aλ aλ a0 0 (cubic equatio) Oe or three solutios (ol real solutios are acceptable) 3D Computer Visio II - Two View Geometr 8

18 f f f f f f f f f The 8-Poit Algorithm 9 3D Computer Visio II - Two View Geometr

19 f f f f f f f f f ~0000 ~0000 ~0000 ~0000 ~00 ~00 ~00 ~00! Orders of magitude differece betwee colums of data matri least-squares ields poor results The Uormalized 8-Poit Algorithm 0 3D Computer Visio II - Two View Geometr

20 The Normalized 8-Poit Algorithm Trasform image to ~[-,][-,] (0,500) (700,500) (-,) (0,0) (,) (0,0) (700,0) (-,-) (,-) Least squares ields good results (Hartle, PAMI97) 3D Computer Visio II - Two View Geometr

21 Smmetric Epipolar Error i d F d T ',,F ' i i T (' F) T T ' F F F ' F i i 3D Computer Visio II - Two View Geometr 4

22 Gold Stadard Maimum Likelihood Estimatio (= least-squares for Gaussia oise) i d, ˆ d', ˆ' i i i i subject to ˆ' T Fˆ 0 Iitialize: ormalized 8-poit, (P,P ) from F, recostruct X i Parameterize: P [I 0], P' [M t], ˆ i PX, ˆ i i P' X i X i Miimize cost usig Leveberg-Marquardt (preferabl sparse LM, see book) (overparametrized) 3D Computer Visio II - Two View Geometr 5

23 Gold Stadard Alterative, miimal parametrizatio (with a=) a b a b F c d c d a' c' b' d' F 33 where F ( a b) ' ( c d) ' 33 problems: a=0 pick largest of a,b,c,d to fi (ote (,,) ad (,,) are epipoles) epipole at ifiit pick largest of,,w ad of,,w 433=36 parametrizatios! reparametrize at ever iteratio, to be sure 3D Computer Visio II - Two View Geometr 6

24 First-order Geometric Error (Sampso Error) e T JJ T e e ' T F T e e JJ T 0 e i (oe eq./poit JJ T scalar) ' T F0 0 JJ T T T ' F ' F F F T e e JJ T T T ' F ' F F F (' T F) (problem if some is located at epipole) advatage: o subsidiar variables required 3D Computer Visio II - Two View Geometr 7

25 Recommedatios Do ot use uormalized algorithms Quick ad eas to implemet: 8-poit ormalized Better: eforce rak- costrait durig miimizatio Best: Maimum Likelihood Estimatio (miimal parameterizatio, sparse implemetatio) 3D Computer Visio II - Two View Geometr 3

26 Special Case Eforce costraits for optimal results: Pure traslatio (dof): F [e' ] Plaar motio (6dof), Calibrated case (5dof) 3D Computer Visio II - Two View Geometr 33

27 The Evelope of Epipolar Lies What happes to a epipolar lie if there is oise? Mote Carlo 3D Computer Visio II - Two View Geometr =0 =5 =5 =50 34

28 Other Etities Lies give o costrait for two view geometr (but will for three ad more views) Curves ad surfaces ield some costraits related to tagec 3D Computer Visio II - Two View Geometr 35

29 Automatic Computatio of F (i) Iterest poits (ii) Putative correspodeces (iii) RANSAC (iv) No-liear re-estimatio of F (v) Guided matchig Repeat (iv) ad (v) util stable 3D Computer Visio II - Two View Geometr 36

30 Feature Poits Etract feature poits to relate images Required properties: Well-defied (i.e. eigborig poits should all be differet) Stable across views (i.e. same 3D poit should be etracted as feature for eighborig viewpoits) 3D Computer Visio II - Two View Geometr 37

31 Feature Poits (e.g. Harris & Stephes 88; Shi & Tomasi 94) Fid poits that differ as much as possible from all eighborig poits homogeeous edge corer 3D Computer Visio II - Two View Geometr 38

32 Feature Poits Select strogest features (e.g. 000/image) 3D Computer Visio II - Two View Geometr 39

33 Feature Matchig Evaluate NCC for all features with similar coordiates e.g. w w,, h h 0 0 0, 0 Keep mutual best matches Still ma wrog matches!? 3D Computer Visio II - Two View Geometr 40

34 Feature Eample Gives satisfig results for small image motios 3D Computer Visio II - Two View Geometr 4

35 Wide-baselie Matchig Requiremet to cope with larger variatios betwee images Traslatio, rotatio, scalig Foreshorteig No-diffuse reflectios Illumiatio geometric trasformatios photometric chages 3D Computer Visio II - Two View Geometr 4

36 Wide-baselie Matchig (Tutelaars ad Va Gool BMVC 000) Wide baselie matchig for two differet regio tpes 3D Computer Visio II - Two View Geometr 43

37 RANSAC Step. Etract features Step. Compute a set of potetial matches Step 3. do Step 3. select miimal sample (i.e. 7 matches) Step 3. compute solutio(s) for F Step 3.3 determie iliers util (#iliers,#samples)<95% geerate hpothesis verif hpothesis Step 4. Compute F based o all iliers Step 5. Look for additioal matches Step 6. Refie F based o all correct matches ( #iliers 7 # samples ) #matches #iliers 90% 80% 70% 60% 50% #samples D Computer Visio II - Two View Geometr 44

38 Fidig More Matches Restrict search rage to eighborhood of epipolar lie (.5 piels) Rela disparit restrictio (alog epipolar lie) 3D Computer Visio II - Two View Geometr 45

39 Degeerate Cases Degeerate cases Plaar scee Pure rotatio No uique solutio Remaiig DOF filled b oise Use simpler model (e.g. homograph) Model selectio (Torr et al., ICCV98, Kaatai, Akaike) Compare H ad F accordig to epected residual error (compesate for model compleit) 3D Computer Visio II - Two View Geometr 46

40 More Problems Absece of sufficiet features (o teture) Repeated structure ambiguit Robust matcher also fids support for wrog hpothesis Solutio: detect repetitio (Schaffalitzk ad Zisserma, BMVC 98) 3D Computer Visio II - Two View Geometr 47

41 Two-view Geometr Geometric relatios betwee two views are full described b recovered 33 matri F. 3D Computer Visio II - Two View Geometr 48

42 Image Rectificatio Simplif stereo matchig b warpig the images. Appl projective trasformatio so that epipolar lies correspod to horizotal scalies e e 0 0 He map epipole e to ifiit (,0,0) tr to miimize image distortio problem whe epipole i (or close to) the image 3D Computer Visio II - Two View Geometr 49

43 Plaar Rectificatio (stadard approach) ~ image size (calibrated) Brig two views to stadard stereo setup (moves epipole to ) (ot possible whe i/close to image) Distortio miimizatio (ucalibrated) 3D Computer Visio II - Two View Geometr 50

44 3D Computer Visio II - Two View Geometr 5

45 5

46 Disparit Map image I(,) Disparit map D(,) image I(,) (,)=(+D(,),) 3D Computer Visio II - Two View Geometr 65

47 Dowsamplig (Gaussia pramid) Disparit propagatio Hierarchical Stereo Matchig Allows faster computatio Deals with large disparit rages (Falkehage97;Va Meerberge,Vergauwe,Pollefes,VaGool IJCV 0) 3D Computer Visio II - Two View Geometr 66

48 Eample: Recostruct Image from Neighborig Images 3D Computer Visio II - Two View Geometr 67