Lecture 08 Multiple View Geometry 2

Transcription

1 Insttte of Informatcs Insttte of Neronformatcs Lectre 8 Mltple Vew Geometry Dade Scaramzza

2 Lab Exercse 6 - Today afternoon Room ETH HG E. from 3:5 to 5: Work descrpton: 8-pont algorthm Estmated poses and 3D strctre

3 3 stdent projects on Robotcs, ML, and Percepton 3

4 -Vew Geometry: Recap Depth from stereo (.e., stereo son) Assmptons: K, T and R are known. Goal: Recoer the 3D strctre from mages P =? -ew Strctre From Moton: Assmptons: none (K, T, and R are nknown). K, R,T K, R,T K, R,T Goal: Recoer smltaneosly 3D scene strctre, camera poses (p to scale), and ntrnsc parameters from two dfferent ews of the scene P =? K, R,T =? K, R,T =? K, R,T =? 4

5 Otlne Two-Vew Strctre from Moton Robst Strctre from Moton 5

6 Problem formlaton: Gen n pont correspondences between two mages, {p = (, ), p = (, )}, smltaneosly estmate the 3D ponts P, the camera relate-moton parameters (R, T), and the camera ntrnscs K, K that satsfy: R, T =? P =? C C Strctre from Moton (SFM) w w w Z Y X I K w w w Z Y X R T K 6

7 Strctre from Moton (SFM) Two arants exst: Calbrated camera(s) K, K are known Uncalbrated camera(s) K, K are nknown P =? C R, T =? 7 C

8 Let s stdy the case n whch the cameras are calbrated For conenence, let s se normalzed mage coordnates Ths, we want to fnd R, T, P that satsfy Strctre from Moton (SFM) K w w w Z Y X I w w w Z Y X R T R, T =? P =? C C 8

9 Scale Ambgty If we rescale the entre scene and camera ews by a constant factor (.e., smlarty transformaton), the projectons (n pxels) of the scene ponts n both mages reman exactly the same: 9

10 Scale Ambgty In monoclar son, t s therefore not possble to recoer the absolte scale of the scene! Stereo son? Ths, only 5 degrees of freedom are measrable: 3 parameters to descrbe the rotaton parameters for the translaton p to a scale (we can only compte the drecton of translaton bt not ts length)

11 Strctre From Moton (SFM) How many knowns and nknowns? 4n knowns: n correspondences; each one (, ) and (, ), = n 5 + 3n nknowns 5 for the moton p to a scale (3 for rotaton, for translaton) 3n = nmber of coordnates of the n 3D ponts Does a solton exst? If and only f the nmber of ndependent eqatons nmber of nknowns 4n 5 + 3n n 5 Frst attempt to dentfy the soltons by Krppa n 93 (see slde 7). E. Krppa, Zr Ermttlng enes Objektes as zwe Perspekten mt Innerer Orenterng, Stz.-Ber. Akad. Wss., Wen, Math. Natrw. Kl., Abt. IIa., 93. Englsh Translaton pls orgnal paper by Gllermo Gallego, Arx, 7

12 Cross Prodct (or Vector Prodct) Vector cross prodct takes two ectors and retrns a thrd ector that s perpendclar to both npts, wth a drecton gen by the rght-hand rle and a magntde eqal to the area of the parallelogram that the ectors span: So c s perpendclar to both a and b (whch means that the dot prodct s ) Also, recall that the cross prodct of two parallel ectors s The cross prodct between a and b can also be expressed n matrx form as the prodct between the skew-symmetrc matrx of a and a ector b c b a c b c a b a b a ] [ z y x x y x z y z b b b a a a a a a sn( ) b a c

13 Can we sole the estmaton of relate moton (R,T) ndependently of the estmaton of the strctre (3D ponts)? The next cople of sldes proe that ths s possble. Once R,T are known, the 3D ponts can be tranglated sng the tranglaton algorthms from Lectre 7 (sldes 3-36) 3

14 Eppolar Geometry P p p p eppolar plane p n p = Rp T p, p, T are coplanar: p T n p T ( ' T p ) p T ( T ( Rp)) T p [ T ] R p T eppolar constrant E [ T ] R essental matrx 4

15 Eppolar Geometry p p Normalzed mage coordnates p T E p Eppolar constrant or Longet-Hggns eqaton (98) E [ T ] R Essental matrx The Essental Matrx can be decomposed nto R and T recallng that For dstnct soltons for R and T are possble. E [ T ] R H. Chrstopher Longet-Hggns, A compter algorthm for reconstrctng a scene from two projectons, Natre, 98, PDF. 5

16 Exercse Compte the Essental matrx for the case of two rectfed stereo mages Rectfed case T T = b T = b b E = b b 6

17 How to compte the Essental Matrx? Image Image If we don t know R and T, can we estmate E from two mages? Yes, gen at least 5 correspondences 7

18 How to compte the Essental Matrx? Krppa showed n 93 that 5 mage correspondences s the mnmal case and that there can be at p to soltons. Howeer, n 988, Demazre showed that there are actally at most dstnct soltons. In 996, Phlpp proposed an terate algorthm to fnd these soltons. Only n 4, the frst effcent and non terate solton was proposed. It ses Groebner bass decomposton [Nster, CVPR 4]. The frst poplar solton ses 8 ponts and s called the 8-pont algorthm or Longet-Hggns algorthm (98). Becase of ts ease of mplementaton, t s stll sed today (e.g., NASA roers). H. Chrstopher Longet-Hggns, A compter algorthm for reconstrctng a scene from two projectons, Natre, 98, PDF. D. Nster, An Effcent Solton to the Fe-Pont Relate Pose Problem, PAMI, 4, PDF 8

19 The 8-pont algorthm The Essental matrx E s defned by p T E p Each par of pont correspondences p p T T (,,), (,,) prodes a lnear eqaton: p T E p E e e e 3 e e e 3 e e e e e e3 e e e3 e3 e3 e33 9

20 For n ponts, we can wrte The 8-pont algorthm e e e e e e e e e n n n n n n n n n n n n Q (ths matrx s known) തE (ths matrx s nknown)

21 The 8-pont algorthm Mnmal solton Q (n 9) shold hae rank 8 to hae a nqe (p to a scale) non-tral solton തE Each pont correspondence prodes ndependent eqaton Ths, 8 pont correspondences are needed Oer-determned solton n > 8 ponts A solton s to mnmze Q തE sbject to the constrant തE =. The solton s the egenector correspondng to the smallest egenale of the matrx Q T Q (becase t s the nt ector x that mnmzes Qx = x T Q T Qx). It can be soled throgh Snglar Vale Decomposton (SVD). Matlab nstrctons: [U,S,V] = sd(q); E = V(:,9); E = reshape(e,3,3)'; Q E Degenerate Confgratons The solton of the eght-pont algorthm s degenerate when the 3D ponts are coplanar. Conersely, the fe-pont algorthm works also for coplanar ponts

22 8-pont algorthm: Matlab code A few lnes of code. Go to the exercse ths afternoon to learn how to mplement t

23 8-pont algorthm: Matlab code fncton E = calbrated_eghtpont( p, p) p = p'; % 3xN ector; each colmn = [;;] p = p'; % 3xN ector; each colmn = [;;] Q = [p(:,).*p(:,),... p(:,).*p(:,),... p(:,3).*p(:,),... p(:,).*p(:,),... p(:,).*p(:,),... p(:,3).*p(:,),... p(:,).*p(:,3),... p(:,).*p(:,3),... p(:,3).*p(:,3) ] ; [U,S,V] = sd(q); Eh = V(:,9); E = reshape(eh,3,3)'; 3

24 Interpretaton of the 8-pont algorthm The 8-pont algorthm seeks to mnmze the followng algebrac error T ) Usng the defnton of dot prodct, t can be obsered that N ( p E p ഥp Eഥp = ഥp Eഥp cos(θ) We can see that ths prodct depends on the angle θ betweenഥp and the normal n = Ep to the eppolar plane. It s non zero when ഥp,ഥp, and T are not coplanar. 4

25 Extract R and T from E (ths slde wll not be asked at the exam) Snglar Vale Decomposton: Enforcng rank- constrant: set smallest snglar ale of to : T V U E 3 T V U T ˆ z y x x y x z y z t t t t t t t t t t T ˆ ˆ ˆ ˆ RK K R t K t T V U R ˆ 5

26 4 possble soltons of R and T Only one solton where ponts are n front of both cameras These two ews are flpped by 8 arond the optcal axs 6

27 Strctre from Moton (SFM) Two arants exst: Calbrated camera(s) K, K are known Uses the Essental Matrx Uncalbrated camera(s) K, K are nknown Uses the Fndamental Matrx P =? C R, T =? 7 C

28 p E p T K K E T The Fndamental Matrx Before, we assmed to know the camera ntrnsc parameters and we sed normalzed mage coordnates to get the eppolar constrant for calbrated cameras:

29 The Fndamental Matrx By sbstttng the defnton of normalzed coordnates nto the eppolar constrant, we get the eppolar constrant for ncalbrated cameras: F K E K T - -T T Fndamental Matrx K ] [ K ] [ K E K - -T - -T R T F R T E F 9

30 The 8-pont Algorthm for the Fndamental Matrx The same 8-pont algorthm to compte the essental matrx from a set of normalzed mage coordnates can also be sed to determne the Fndamental matrx: Adantage: we work drectly n pxel coordnates F T 3

31 Problem wth 8-pont algorthm f f f f f f f f f n n n n n n n n n n n n 33

32 Problem wth 8-pont algorthm ~ ~ ~ ~ ~ ~ ~ ~! Orders of magntde dfference between colmn of data matrx least-sqares yelds poor reslts Poor nmercal condtonng, whch makes reslts ery senste to nose Can be fxed by rescalng the data: Normalzed 8-pont algorthm [Hartley, PAMI 97] 34 f f f f f f f f f

33 Normalzed 8-pont algorthm (/3) Ths can be fxed sng a normalzed 8-pont algorthm [Hartley 97], whch estmates the Fndamental matrx on a set of Normalzed correspondences (wth better nmercal propertes) and then nnormalzes the reslt to obtan the fndamental matrx for the gen (nnormalzed) correspondences Idea: Transform mage coordnates so that they are n the range ~[,] [,] One way s to apply the followng rescalng and shft (,) (7,) 7 5 (-,-) (,) (,-) (,5) (7,5) (-,) (,) Hartley, In defense of the eght-pont algorthm, PAMI 97, PDF 35

34 Normalzed 8-pont algorthm (/3) In the orgnal 997 paper, Hartley proposed to rescale the two pont sets sch that the centrod of each set s and the mean standard deaton, so that the aerage pont s eqal to [,, ] T (n homogeneos coordnates). Ths can be done for eery pont as follows: where μ = (μ x, μ y ) = σ N = n p = σ (p μ) mean standard deaton of the pont set. p s the centrod and σ = σ n N = p μ s the Ths transformaton can be expressed n matrx form sng homogeneos coordnates: σ σ μ x p = p σ σ μ y Hartley, In defense of the eght-pont algorthm, PAMI 97, PDF 36

35 Normalzed 8-pont algorthm (3/3) The Normalzed 8-pont algorthm can be smmarzed n three steps:. Normalze pont correspondences: p = B p, p = B p. Estmate normalzed F wth 8-p. algorthm sng normalzed coordnates p, p 3. Compte nnormalzed F from F: F = B F B p T F p = p B F B p F = B F B 37

36 Can R, T, K, K be extracted from F? In general no: nfnte soltons exst Howeer, f the coordnates of the prncpal ponts of each camera are known and the two cameras hae the same focal length f n pxels, then R, T, f can determned nqely 38

37 Comparson between Normalzed and non-normalzed algorthm 8-pont Normalzed 8-pont Nonlnear refnement Ag. Ep. Lne Dstance.33 pxels.9 pxel.86 pxel Ag. Ep. Lne Dstance.8 pxels.85 pxel.8 pxel 39

38 Error Measres The qalty of the estmated Essental matrx can be measred sng dfferent error metrcs. The frst one s the algebrac error that s defned drectly by the Eppolar Constrant: err N ( p T E p ) Remember Slde 4 for the geometrcal nterpretaton of ths error What s the drawback wth ths error measre? Ths error wll exactly be f E s compted from jst 8 ponts (becase n ths case a non-oerdetermned solton exsts). For more than 8 ponts, t may not be (de to mage nose or otlers (oerdetermned system)). There are alternate error fnctons that can be sed to measre the qalty of the estmated Fndamental matrx: the Drectonal Error, the Eppolar Lne Dstance, or the Reprojecton Error. 4

39 Drectonal Error Sm of sqared cosnes of the angle from the eppolar plane: err = (cos(θ )) From slde 4, we obtan: cos(θ) = pt Ep p Ep P =? l E T p Eppolar plane p p n C C 4

40 Eppolar Lne Dstance Sm of Sqared Eppolar-Lne-to-pont Dstances err N d Cheaper than reprojecton error becase does not reqre pont tranglaton (p, l ) d (p, l ) P =? l F T p eppolar plane p p l Fp C C 4

41 Reprojecton Error Sm of the Sqared Reprojecton Errors err N p - π(p ) p - π (P, R, T) Comptaton s expense becase reqres pont tranglaton Howeer t s the most poplar becase more accrate P =? How to compte P? See Sldes 3-36 of Lectre 7 Reprojected pont ( P ) Reprojected pont Obsered pont Obsered pont p p ( P) C C R, T 43

42 Otlne Two-Vew Strctre from Moton Robst Strctre from Moton 44

43 Matched ponts are sally contamnated by otlers (.e., wrong mage matches) Cases of otlers are: changes n ew pont (ncldng scale) and llmnaton mage nose occlsons blr For the camera moton to be estmated accrately, otlers mst be remoed Ths s the task of Robst Estmaton Image Image 45 Dade Scaramzza Unersty of Zrch Robotcs and Percepton Grop - rpg.f.zh.ch

44 Matched ponts are sally contamnated by otlers (.e., wrong mage matches) Cases of otlers are: changes n ew pont (ncldng scale) and llmnaton mage nose occlsons blr For the camera moton to be estmated accrately, otlers mst be remoed Ths s the task of Robst Estmaton Image Image 46 Dade Scaramzza Unersty of Zrch Robotcs and Percepton Grop - rpg.f.zh.ch

45 Error at the loop closre: 6.5 m Error n orentaton: 5 deg Trajectory length: 4 m Before remong the otlers After remong the otlers Otlers can be remoed sng RANSAC [Fshler & Bolles, 98] Dade Scaramzza Unersty of Zrch Robotcs and Percepton Grop - rpg.f.zh.ch 47

46 RANSAC (RAndom SAmple Consenss) RANSAC s the standard method for model fttng n the presence of otlers (ery nosy ponts or wrong data) It can be appled to all sorts of problems where the goal s to estmate the parameters of a model from the data (e.g., camera calbraton, Strctre from Moton, DLT, PnP, P3P, Homography, etc.) Let s reew RANSAC for lne fttng and see how we can se t to do Strctre from Moton M. A.Fschler and R. C.Bolles. Random sample consenss: A paradgm for model fttng wth applcatons to mage analyss and atomated cartography. Graphcs and Image Processng,

47 49 RANSAC

48 5 RANSAC Select sample of ponts at random

49 5 RANSAC Select sample of ponts at random Calclate model parameters that ft the data n the sample

50 5 RANSAC Select sample of ponts at random Calclate model parameters that ft the data n the sample Calclate error fncton for each data pont

51 53 RANSAC Select sample of ponts at random Calclate model parameters that ft the data n the sample Calclate error fncton for each data pont Select data that spport crrent hypothess

52 54 RANSAC Select sample of ponts at random Calclate model parameters that ft the data n the sample Calclate error fncton for each data pont Select data that spport crrent hypothess Repeat

53 55 RANSAC Select sample of ponts at random Calclate model parameters that ft the data n the sample Calclate error fncton for each data pont Select data that spport crrent hypothess Repeat

54 56 RANSAC Select the set wth the maxmm nmber of nlers obtaned wthn k teratons

55 57 RANSAC How many teratons does RANSAC need? Ideally: check all possble combnatons of ponts n a dataset of N ponts. Nmber of all parwse combnatons: N(N-)/ comptatonally nfeasble f N s too large. example: ponts need to check all *999/ 5 possbltes! Do we really need to check all possbltes or can we stop RANSAC after some teratons? Checkng a sbset of combnatons s enogh f we hae a rogh estmate of the percentage of nlers n or dataset Ths can be done n a probablstc way

56 58 RANSAC How many teratons does RANSAC need? w := nmber of nlers/n N := total nmber of data ponts w : fracton of nlers n the dataset w = P(selectng an nler-pont ot of the dataset) Assmpton: the ponts necessary to estmate a lne are selected ndependently w = P(both selected ponts are nlers) -w = P(at least one of these two ponts s an otler) Let k := no. RANSAC teratons exected so far ( -w ) k = P(RANSAC neer selected two ponts that are both nlers) Let p := P(probablty of sccess) -p = ( -w ) k and therefore : k log( log( p) w )

57 59 RANSAC How many teratons does RANSAC need? The nmber of teratons k s k log( log( p) w ) knowng the fracton of nlers w, after k RANSAC teratons we wll hae a probablty p of fndng a set of ponts free of otlers Example: f we want a probablty of sccess p=99% and we know that w=5% k=6 teratons these are sgnfcantly fewer than the nmber of all possble combnatons! Notce: nmber of ponts does not nflence mnmm nmber of teratons k, only w does! In practce we only need a rogh estmate of w. More adanced arants of RANSAC estmate the fracton of nlers and adaptely pdate t at eery teraton (how?)

58 RANSAC appled to Lne Fttng. Intal: let A be a set of N ponts. repeat 3. Randomly select a sample of ponts from A 4. Ft a lne throgh the ponts 5. Compte the dstances of all other ponts to ths lne 6. Constrct the nler set (.e. cont the nmber of ponts whose dstance < d) 7. Store these nlers 8. ntl maxmm nmber of teratons k reached 9. The set wth the maxmm nmber of nlers s chosen as a solton to the problem 6

59 RANSAC appled to general model fttng. Intal: let A be a set of N ponts. repeat 3. Randomly select a sample of s ponts from A 4. Ft a model from the s ponts 5. Compte the dstances of all other ponts from ths model 6. Constrct the nler set (.e. cont the nmber of ponts whose dstance < d) 7. Store these nlers 8. ntl maxmm nmber of teratons k reached 9. The set wth the maxmm nmber of nlers s chosen as a solton to the problem k log( p) log( w s ) 6

60 The Three Key Ingredents of RANSAC In order to mplement RANSAC for Strctre From Moton (SFM), we need three key ngredents:. What s the model n SFM?. What s the mnmm nmber of ponts to estmate the model? 3. How do we compte the dstance of a pont from the model? In other words, can we defne a dstance metrc that measres how well a pont fts the model? 6

61 Answers. What s the model n SFM? The Essental Matrx (for calbrated cameras) or the Fndamental Matrx (for ncalbrated cameras) Alternately, R and T. What s the mnmm nmber of ponts to estmate the model?. We know that 5 ponts s the theoretcal mnmm nmber of ponts. Howeer, f we se the 8-pont algorthm, then 8 s the mnmm 3. How do we compte the dstance of a pont from the model?. Algebrac error ( pҧ Ep ҧ = or p Fp = ) (Slde 4). Drectonal error (Slde 4) 3. Eppolar lne dstance (Slde 4) 4. Reprojecton error (Slde 43) 63

62 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows Image Image 64

63 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows For conenence, we oerlay the featres of the second mage on the frst mage and se arrows to denote the moton ectors of the featres Image 65

64 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows For conenence, we oerlay the featres of the second mage on the frst mage and se arrows to denote the moton ectors of the featres. Randomly select 8 pont correspondences Image 66

65 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows For conenence, we oerlay the featres of the second mage on the frst mage and se arrows to denote the moton ectors of the featres. Randomly select 8 pont correspondences. Ft the model to all other ponts and cont the nlers Image 67

66 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows For conenence, we oerlay the featres of the second mage on the frst mage and se arrows to denote the moton ectors of the featres. Randomly select 8 pont correspondences. Ft the model to all other ponts and cont the nlers 3. Repeat from Image 68

67 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows For conenence, we oerlay the featres of the second mage on the frst mage and se arrows to denote the moton ectors of the featres Image 69

68 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows For conenence, we oerlay the featres of the second mage on the frst mage and se arrows to denote the moton ectors of the featres. Randomly select 8 pont correspondences Image 7

69 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows For conenence, we oerlay the featres of the second mage on the frst mage and se arrows to denote the moton ectors of the featres. Randomly select 8 pont correspondences. Ft the model to all other ponts and cont the nlers Image 7

70 Example: 8-pont RANSAC appled to SfM Let s consder the followng mage par and ts mage correspondences (e.g., Harrs, SIFT, etc.), denoted by arrows For conenence, we oerlay the featres of the second mage on the frst mage and se arrows to denote the moton ectors of the featres. Randomly select 8 pont correspondences. Ft the model to all other ponts and cont the nlers 3. Repeat from for k tmes k log( p) log( ( ) 8 ) Image 7

71 RANSAC teratons k s. s k s exponental n the nmber of ponts s necessary to estmate the model: 8-pont RANSAC Assmng p = 99%, ε = 5% (fracton of otlers) s = 8 ponts (8-pont algorthm) log( p) k s log( ( ) ) 77 teratons 5-pont RANSAC Assmng p = 99%, ε = 5% (fracton of otlers) s = 5 ponts (5-pont algorthm of Dad Nster (4)) log( p) k s log( ( ) ) 45 teratons -pont RANSAC (e.g., lne fttng) Assmng p = 99%, ε = 5% (fracton of otlers) s = ponts log( p) k s log( ( ) ) 6 teratons 73

72 RANSAC teratons k s. ε k s ncreases exponentally wth the fracton of otlers ε 74

73 RANSAC teratons As obsered, k s exponental n the nmber of ponts s necessary to estmate the model The 8-pont algorthm s extremely smple and was ery sccessfl; howeer, t reqres more than 77 teratons Becase of ths, there has been a large nterest by the research commnty n sng smaller moton parameterzatons (.e., smaller s) The frst effcent solton to the mnmal-case solton (5-pont algorthm) took almost a centry (Krppa 93 Nster 4) The 5-pont RANSAC (Nster 4) only reqres 45 teratons; howeer: The 5-pont algorthm can retrn p to soltons of E (worst case scenaro) The 8-pont algorthm only retrns a nqe solton of E Can we se less than 5 ponts? Yes, f yo se moton constrants! 75

74 Planar Moton Planar moton s descrbed by three parameters: θ, φ, ρ R cos sn sn cos T cos sn y x Let s compte the Eppolar Geometry E [ T ] R Essental matrx p T E p Eppolar constrant 76

75 Planar Moton cos sn sn cos R sn cos T Planar moton s descrbed by three parameters: θ, φ, ρ x y Let s compte the Eppolar Geometry R T E ] [ cos sn sn cos cos sn cos sn cos sn cos sn ] [ T 77

76 Planar Moton cos sn sn cos R sn cos T Planar moton s descrbed by three parameters: θ, φ, ρ x y Let s compte the Eppolar Geometry R T E ] [ cos sn cos sn cos sn cos sn ] [ T 78

77 Planar Moton Planar moton s descrbed by three parameters: θ, φ, ρ R cos sn sn cos T cos sn y x Obsere that E has DoF (θ, φ, becase ρ s the scale factor); ths, correspondences are sffcent to estmate and φ [ -Pont RANSAC, Ortn, ] sn E [ T ] R cos sn cos 79

78 Can we se less than pont correspondences? Yes, f we explot wheeled ehcles wth non-holonomc constrants 8

79 Planar & Crclar Moton (e.g., cars) Wheeled ehcles, lke cars, follow locally-planar crclar moton abot the Instantaneos Center of Rotaton (ICR) Example of Ackerman steerng prncple Locally-planar crclar moton 8

80 Planar & Crclar Moton (e.g., cars) Wheeled ehcles, lke cars, follow locally-planar crclar moton abot the Instantaneos Center of Rotaton (ICR) Example of Ackerman steerng prncple Locally-planar crclar moton φ = θ/ => only DoF (θ); ths, only pont correspondence s needed Ths s the smallest parameterzaton possble and reslts n the most effcent algorthm for remong otlers Scaramzza, -Pont-RANSAC Strctre from Moton for Vehcle-Monted Cameras by Explotng Non-holonomc Constrants, Internatonal Jornal of Compter Vson, 8

81 cos sn sn cos R sn cos T Let s compte the Eppolar Geometry p E p T Eppolar constrant R T ] [ E Essental matrx Planar & Crclar Moton (e.g., cars) 83

82 cos sn sn cos R sn cos T Let s compte the Eppolar Geometry R T E ] [ cos sn cos sn cos sn sn cos cos sn cos sn Planar & Crclar Moton (e.g., cars) 84

83 cos sn sn cos R sn cos T Let s compte the Eppolar Geometry cos sn cos sn E Planar & Crclar Moton (e.g., cars) p E p T ) ( cos ) ( sn tan 85

84 -Pont RANSAC algorthm Compte θ for eery pont correspondence tan Only teraton! The most effcent algorthm for remong otlers, p to Hz -Pont RANSAC s ONLY sed to fnd the nlers. Moton s then estmated from them n 6DOF 86

85 Comparson of RANSAC algorthms 9 8 Nmber of teratons, N pont RANSAC -pont RANSAC -pont RANSAC Fracton of otlers n the data (%) N log( p) log( ( ) s ) where we typcally se p 99% Nmb. of teratons 8-Pont RANSAC [Longet-Hggns 8] 5-Pont RANSAC [Nster 4] -Pont RANSAC [Ortn ] -Pont RANSAC [Scaramzza ] > 77 >45 >6 = 87

86 Vsal Odometry wth -Pont RANSAC Scaramzza, -Pont-RANSAC Strctre from Moton for Vehcle-Monted Cameras by Explotng Non-holonomc Constrants, Internatonal Jornal of Compter Vson, 88

87 Thngs to remember SFM from ew Calbrated and ncalbrated case Proof of Eppolar Constrant 8-pont algorthm and algebrac error Normalzed 8-pont algorthm Algebrac, drectonal, Eppolar lne dstance, Reprojecton error RANSAC and ts applcaton to SFM 8 s 5 s pont RANSAC, pros and cons Readngs: Ch. 4. of Corke book CH. 7. of Szelsk book 89

88 Understandng Check Are yo able to answer the followng qestons? What's the mnmm nmber of correspondences reqred for calbrated SFM and why? Are yo able to dere the eppolar constrant? Are yo able to defne the essental matrx? Are yo able to dere the 8-pont algorthm? How many rotaton-translaton combnatons can the essental matrx be decomposed nto? Are yo able to prode a geometrcal nterpretaton of the eppolar constrant? Are yo able to descrbe the relaton between the essental and the fndamental matrx? Why s t mportant to normalze the pont coordnates n the 8-pont algorthm? Descrbe one or more possble ways to achee ths normalzaton. Are yo able to descrbe the normalzed 8-pont algorthm? Are yo able to prode qalty metrcs for the essental matrx estmaton? Why do we need RANSAC? What s the theoretcal maxmm nmber of combnatons to explore? After how many teratons can RANSAC be stopped to garantee a gen sccess probablty? What s the trend of RANSAC s. teratons, s. the fracton of otlers, s. the nmber of ponts to estmate the model? How do we apply RANSAC to the 8-pont algorthm, DLT, P3P? How can we redce the nmber of RANSAC teratons for the SFM problem? 9