Prof. Feng Liu. Winter /24/2019

Transcription

1 Prof. Feg Lu Wter /24/209

2 Last Tme Feature detecto 2

3 Toda Feature matchg Fttg The followg sldes are largel from Prof. S. Lazebk 3

4 Wh etract features? Motvato: paorama sttchg We have two mages how do we combe them?

5 Wh etract features? Motvato: paorama sttchg We have two mages how do we combe them? Step : etract features Step 2: match features

6 Wh etract features? Motvato: paorama sttchg We have two mages how do we combe them? Step : etract features Step 2: match features Step 3: alg mages

7 Geeratg putatve correspodeces?

8 Geeratg putatve correspodeces?? = feature descrptor feature descrptor Need to compare feature descrptors of local patches surroudg terest pots

9 Dsadvatage of test vectors as descrptors Small deformatos ca affect the matchg score a lot

10 Feature descrptors Recall: covarat detectors => varat descrptors Detect regos Normalze regos Compute appearace descrptors

11 Feature descrptors: SIFT Descrptor computato: Dvde patch to 44 sub-patches Compute hstogram of gradet oretatos 8 referece agles sde each sub-patch Resultg descrptor: 448 = 28 dmesos Davd G. Lowe. "Dstctve mage features from scale-varat kepots. IJCV 60 2, pp. 9-0, 2004.

12 How to compare two such vectors? Sum of squared dffereces SSD Not varat to test chage Normalzed correlato Ivarat to affe test chage Feature descrptors u v v u 2, SSD j j j j v v u u v v u u v u 2 2,

13 Feature descrptors: SIFT Descrptor computato: Dvde patch to 44 sub-patches Compute hstogram of gradet oretatos 8 referece agles sde each sub-patch Resultg descrptor: 448 = 28 dmesos Advatage over raw vectors of pel values Gradets less sestve to llumato chage Poolg of gradets over the sub-patches acheves robustess to small shfts, but stll preserves some spatal formato Davd G. Lowe. "Dstctve mage features from scale-varat kepots. IJCV 60 2, pp. 9-0, 2004.

14 Feature Matchg Stadard approach for par-wse matchg: For each feature pot mage A Fd the feature pot wth the closest descrptor mage B

15 Feature Matchg Compare the dstace, d, to the closest feature, to the dstace, d2, to the secod closest feature Accept f d/d2 < 0.8 If the rato of dstaces s less tha a threshold, keep the feature Wh the rato test? Elmates hard-to-match repeated features Dstaces SIFT descrptor space seem to be o-uform

16 Readg Davd G. Lowe. "Dstctve mage features from scale-varat kepots. IJCV 60 2, pp. 9-0, 2004.

17 Fttg We ve leared how to detect edges, corers, blobs. Now what? We would lke to form a hgher-level, more compact represetato of the features the mage b groupg multple features accordg to a smple model 9300 Harrs Corers Pkw, Charlotte, NC

18 Fttg Choose a parametrc model to represet a set of features smple model: les smple model: crcles complcated model: car Source: K. Grauma

19 Fttg: Issues Case stud: Le detecto Nose the measured feature locatos Etraeous data: clutter outlers, multple les Mssg data: occlusos

20 Fttg: Overvew If we kow whch pots belog to the le, how do we fd the optmal le parameters? Least squares What f there are outlers? Robust fttg, RANSAC What f there are ma les? Votg methods: RANSAC, Hough trasform What f we re ot eve sure t s a le? Model selecto

21 Least squares le fttg Data:,,,, Le equato: = m + b Fd m, b to mmze Y X XB X db de T T 2 2 XB XB Y XB Y Y XB Y XB Y XB Y E b m B X Y T T T T Normal equatos: least squares soluto to XB=Y b m E 2, =m+b Y X XB X T T

22 Problem wth vertcal least squares Not rotato-varat Fals completel for vertcal les

23 Total least squares Dstace betwee pot, ad le a+b=d a 2 +b 2 =: a + b d Fd a, b, d to mmze the sum of squared perpedcular dstaces E 2 a b, d a+b=d Ut ormal: N=a, b E a b d 2

24 Total least squares Dstace betwee pot, ad le a+b=d a 2 +b 2 =: a + b d Fd a, b, d to mmze the sum of squared perpedcular dstaces d b a E 2, a+b=d d b a E 2 Ut ormal: N=a, b 2 2 UN UN b a b a E T 0 2 N U U dn de T Soluto to U T UN = 0, subject to N 2 = : egevector of U T U assocated wth the smallest egevalue least squares soluto to homogeeous lear sstem UN = d b a d E b a b a d

25 Total least squares U T U U 2 2 secod momet matr

26 Total least squares, N = a, b, U T U U 2 2 secod momet matr

27 Least squares as lkelhood mamzato Geeratve model: le pots are sampled depedetl ad corrupted b Gaussa ose the drecto perpedcular to the le u v a b a+b=d u, v ε, pot o the le ose: sampled from zero-mea Gaussa wth std. dev. σ ormal drecto

28 Least squares as lkelhood mamzato d b a d b a P d b a P ep,,,,,,,,, Lkelhood of pots gve le parameters a, b, d: d b a d b a L 2 2 2,,,,,, Log-lkelhood:, a+b=d u, v ε b a v u Geeratve model: le pots are sampled depedetl ad corrupted b Gaussa ose the drecto perpedcular to the le

29 Least squares: Robustess to ose Least squares ft to the red pots:

30 Least squares: Robustess to ose Least squares ft wth a outler: Problem: squared error heavl pealzes outlers

31 Robust estmators Geeral approach: fd model parameters θ that mmze r, θ resdual of th pot w.r.t. model parameters θ ρ robust fucto wth scale parameter σ r, ; The robust fucto ρ behaves lke squared dstace for small values of the resdual u but saturates for larger values of u

32 Choosg the scale: Just rght The effect of the outler s mmzed

33 Choosg the scale: Too small The error value s almost the same for ever pot ad the ft s ver poor

34 Choosg the scale: Too large Behaves much the same as least squares

35 Robust estmato: Detals Robust fttg s a olear optmzato problem that must be solved teratvel Least squares soluto ca be used for talzato Adaptve choce of scale F&P, Sec Davd Forsth ad Jea Poce. Computer Vso: A Moder Approach, 2 d edto, 20

36 RANSAC Robust fttg ca deal wth a few outlers what f we have ver ma? Radom sample cosesus RANSAC: Ver geeral framework for model fttg the presece of outlers Outle Choose a small subset of pots uforml at radom Ft a model to that subset Fd all remag pots that are close to the model ad reject the rest as outlers Do ths ma tmes ad choose the best model M. A. Fschler, R. C. Bolles. Radom Sample Cosesus: A Paradgm for Model Fttg wth Applcatos to Image Aalss ad Automated Cartograph. Comm. of the ACM, Vol 24, pp , 98.

37 RANSAC for le fttg eample Source: R. Raguram

38 RANSAC for le fttg eample Least-squares ft Source: R. Raguram

39 RANSAC for le fttg eample. Radoml select mmal subset of pots Source: R. Raguram

40 RANSAC for le fttg eample. Radoml select mmal subset of pots 2. Hpothesze a model Source: R. Raguram

41 RANSAC for le fttg eample. Radoml select mmal subset of pots 2. Hpothesze a model 3. Compute error fucto Source: R. Raguram

42 RANSAC for le fttg eample. Radoml select mmal subset of pots 2. Hpothesze a model 3. Compute error fucto 4. Select pots cosstet wth model Source: R. Raguram

43 RANSAC for le fttg eample. Radoml select mmal subset of pots 2. Hpothesze a model 3. Compute error fucto 4. Select pots cosstet wth model 5. Repeat hpothesze-adverf loop Source: R. Raguram

44 RANSAC for le fttg eample. Radoml select mmal subset of pots 2. Hpothesze a model 3. Compute error fucto 4. Select pots cosstet wth model 5. Repeat hpothesze-adverf loop Source: R. Raguram 44

45 RANSAC for le fttg eample Ucotamated sample. Radoml select mmal subset of pots 2. Hpothesze a model 3. Compute error fucto 4. Select pots cosstet wth model 5. Repeat hpothesze-adverf loop Source: R. Raguram 45

46 RANSAC for le fttg eample. Radoml select mmal subset of pots 2. Hpothesze a model 3. Compute error fucto 4. Select pots cosstet wth model 5. Repeat hpothesze-adverf loop Source: R. Raguram

47 RANSAC for le fttg Repeat N tmes: Draw s pots uforml at radom Ft le to these s pots Fd lers to ths le amog the remag pots.e., pots whose dstace from the le s less tha t If there are d or more lers, accept the le ad reft usg all lers

48 Choosg the parameters Ital umber of pots s Tpcall mmum umber eeded to ft the model Dstace threshold t Choose t so probablt for ler s p e.g Zero-mea Gaussa ose wth std. dev. σ: t 2 =3.84σ 2 Number of samples N Choose N so that, wth probablt p, at least oe radom sample s free from outlers e.g. p=0.99 outler rato: e Source: M. Pollefes

49 Choosg the parameters Ital umber of pots s Tpcall mmum umber eeded to ft the model Dstace threshold t Choose t so probablt for ler s p e.g Zero-mea Gaussa ose wth std. dev. σ: t 2 =3.84σ 2 Number of samples N Choose N so that, wth probablt p, at least oe radom sample s free from outlers e.g. p=0.99 outler rato: e s N e p N p/ log e log s proporto of outlers e s 5% 0% 20% 25% 30% 40% 50% Source: M. Pollefes

50 Choosg the parameters Ital umber of pots s Tpcall mmum umber eeded to ft the model Dstace threshold t Choose t so probablt for ler s p e.g Zero-mea Gaussa ose wth std. dev. σ: t 2 =3.84σ 2 Number of samples N Choose N so that, wth probablt p, at least oe radom sample s free from outlers e.g. p=0.99 outler rato: e Cosesus set sze d Should match epected ler rato Source: M. Pollefes

51 Adaptvel determg the umber of samples Iler rato e s ofte ukow a pror, so pck worst case, e.g. 50%, ad adapt f more lers are foud, e.g. 80% would eld e=0.2 Adaptve procedure: N=, sample_cout =0 Whle N >sample_cout Choose a sample ad cout the umber of lers Set e = umber of lers/total umber of pots Recompute N from e: N p/ log e log Icremet the sample_cout b s Source: M. Pollefes

52 RANSAC pros ad cos Pros Smple ad geeral Applcable to ma dfferet problems Ofte works well practce Cos Lots of parameters to tue Does t work well for low ler ratos too ma teratos, or ca fal completel Ca t alwas get a good talzato of the model based o the mmum umber of samples

53 Net Tme Hough trasform 53