Motivation. Matching, Alignment, and Registration. Components in Matching. Image Alignment

Transcription

1 Matcng, Algnent, and Regstraton CIS 554 Coputer Vson Habn Lng Motvaton ransforaton between two enttes are often requested n an vson tass Iage algnent Pont set atcng Surface regstraton Contour atcng Snons Algnent Matcng Regstraton Coponents n Matcng ransforaton (warpng) odels Lnear odel Eucldean, affne, projectve Local (near) lnear tn-plate-splne (PS) None-lnear Correspondences to estate te odels Known Unnown jont odel estaton and correspondence Estaton algorts Least squares estaton RANSAC Iteratve closest ponts (ICP) Iage Algnent Sldes source: S. Lazebn Iage algnent Iage algnent: Motvaton Panoraa sttcng Recognton of object nstances

2 Iage algnent: Callenges Iage algnent Sall degree of overlap Occluson, clutter wo broad approaces: Drect (pel-based) algnent Searc for algnent were ost pels agree Feature-based algnent Searc for algnent were etracted features agree Can be verfed usng pel-based algnent Algnent as fttng Prevous lectures: fttng a odel to features n one age M Fnd odel M tat nzes resdual(, M ) Algnent as fttng Prevous lectures: fttng a odel to features n one age M Fnd odel M tat nzes resdual(, M ) Algnent: fttng a odel to a transforaton between pars of features (atces) n two ages ' Fnd transforaton tat nzes resdual( ( ), ) Feature-based algnent outlne Feature-based algnent outlne Etract features

3 Feature-based algnent outlne Feature-based algnent outlne Etract features Copute putatve atces Etract features Copute putatve atces Loop: Hpotesze transforaton (sall group of putatve atces tat are related b ) Feature-based algnent outlne Feature-based algnent outlne Etract features Copute putatve atces Loop: Hpotesze transforaton (sall group of putatve atces tat are related b ) Verf transforaton (searc for oter atces consstent wt ) Etract features Copute putatve atces Loop: Hpotesze transforaton (sall group of putatve atces tat are related b ) Verf transforaton (searc for oter atces consstent wt ) D transforaton odels Slart (translaton, scale, rotaton) Let s start wt affne transforatons Sple fttng procedure (lnear least squares) Approates vewpont canges for rougl planar objects and rougl ortograpc caeras Can be used to ntalze fttng for ore cople odels Affne Projectve (oograp)

4 4 Fttng an affne transforaton Assue we now te correspondences, ow do we get te transforaton? ), ( ), ( 4 t t t t 4 Fttng an affne transforaton t 4 Lnear sste wt s unnowns Eac atc gves us two lnearl ndependent equatons: need at least tree to solve for te transforaton paraeters t Wat f we don t now te correspondences?? Wat f we don t now te correspondences?? Need to copare feature descrptors of local patces surroundng nterest ponts ( ) ( ) =? feature descrptor feature descrptor Feature descrptors Assung te patces are alread noralzed (.e., te local effect of te geoetrc transforaton s factored out), ow do we copute ter slart? Want nvarance to ntenst canges, nose, perceptuall nsgnfcant canges of te pel pattern Descrptor coputaton: Dvde patc nto 44 sub-patces Copute stogra of gradent orentatons (8 reference angles) nsde eac sub-patc Resultng descrptor: 448 = 8 densons Feature descrptors: SIF Davd G. Lowe. "Dstnctve age features fro scale-nvarant eponts. IJCV 6 (), pp. 9-, 4.

5 Feature atcng Generatng putatve atces: for eac patc n one age, fnd a sort lst of patces n te oter age tat could atc t based solel on appearance Readng? Davd G. Lowe. "Dstnctve age features fro scalenvarant eponts. IJCV 6 (), pp. 9-, 4. Dealng wt outlers e set of putatve atces contans a ver g percentage of outlers Heurstcs for feature-space outler rejecton Geoetrc fttng strateges: RANSAC Increental algnent Houg transfor Hasng RANSAC RANSAC loop:. Randol select a seed group of atces. Copute transforaton fro seed group. Fnd nlers to ts transforaton 4. If te nuber of nlers s suffcentl large, re-copute least-squares estate of transforaton on all of te nlers Keep te transforaton wt te largest nuber of nlers RANSAC eaple: ranslaton RANSAC eaple: ranslaton Putatve atces Select one atc, count nlers 5

6 6 RANSAC eaple: ranslaton Select translaton wt te ost nlers Proble wt RANSAC In an practcal stuatons, te percentage of outlers (ncorrect putatve atces) s often ver g (9% or above) Alternatve strateg: restrct searc space b usng strong localt constrants on seed groups and nlers Increental algnent V t t t (H t f ) Votng n te paraeter space (Houg transfor) Or, ore effcent soluton: Hasng Beond affne transforatons Hoograp: plane projectve transforaton (transforaton tang a quad to anoter arbtrar quad) Fttng a oograp Recall: oogenenous coordnates Convertng to oogenenous age coordnates Convertng fro oogenenous age coordnates Fttng a oograp Recall: oogenenous coordnates Convertng to oogenenous age coordnates Convertng fro oogenenous age coordnates Equaton for oograp: Fttng a oograp Equaton for oograp: H 9 entres, 8 degrees of freedo H H equatons, onl lnearl ndependent g (scale s arbtrar)

7 Drect lnear transfor A n n n n n n H as 8 degrees of freedo (9 paraeters, but scale s arbtrar) One atc gves us two lnearl ndependent equatons Four atces needed for a nal soluton (null space of 89 atr) More tan four: oogeneous least squares Applcaton: Panoraa sttcng Recognzng panoraas Gven contents of a caera eor card, autoatcall fgure out wc pctures go togeter and sttc te togeter nto panoraas M. Brown and D. Lowe, Recognzng Panoraas, ICCV. ttp:// Issues n algnent-based applcatons Coosng te geoetrc algnent odel radeoff between correctness and robustness (also, effcenc) Coosng te descrptor Rc ager (natural ages): g-densonal patc-based descrptors (e.g., SIF) Ipoversed ager (e.g., star felds): need to create nvarant geoetrc descrptors fro -tuples of pont-based features Strateg for fndng putatve atces Sall nuber of ages, one-te coputaton (e.g., panoraa sttcng): brute force searc Large database of odel ages, frequent queres: ndeng or asng Heurstcs for feature-space prunng of putatve atces Issues n algnent-based applcatons Coosng te geoetrc algnent odel Coosng te descrptor Strateg for fndng putatve atces Hpotess generaton strateg Relatvel large nler rato: RANSAC Sall nler rato: localt constrants, Houg transfor Hpotess verfcaton strateg Sze of consensus set, resdual tolerance depend on nler rato and epected accurac of te odel Possble refneent of geoetrc odel Dense verfcaton Regstraton wt Iteratve Closest Pont Sldes source: R. Gvl 7

8 e Proble Algn two partalloverlappng eses gven ntal guess for relatve transfor Data pes Pont sets Lne segent sets (pollnes) Iplct curves : f(,,z) = Paraetrc curves : ((u),(u),z(u)) rangle sets (eses) Iplct surfaces : s(,,z) = Paraetrc surfaces ((u,v),(u,v),z(u,v))) (, (, ))) Motvaton Sape nspecton Moton estaton Appearance analss eture Mappng racng Motvaton Range ages regstraton Algnng D Data Correspondng Pont Set Algnent Let M be a odel pont set. Let S be a scene pont set. We assue :. N M = N S.. Eac pont S correspond to M. 8

9 Algnng D Data If correct correspondences are nown, can fnd correct relatve rotaton/translaton Algnng D Data How to fnd correspondences: User nput? Feature detecton? Sgnatures? Alternatve: assue closest ponts correspond Algnng D Data How to fnd correspondences: User nput? Feature detecton? Sgnatures? Alternatve: assue closest ponts correspond Algnng D Data Converges f startng poston close enoug Fndng Matces Fndng Matces e scene sape S s algned to be n te best algnent wt te odel sape M. e dstance of eac pont s of te scene fro te odel s : d ( s, M ) n d s M d ( s, M ) n d s M M Y C ( S, M ) Y M C te closest pont operator Y te set of closest ponts to S d ( s, ) 9

10 e Algort Convergence eore Y = CP(M,S),e (rot,trans,d) S`= rot(s)+trans d` = d Int te error to Calculate correspondence Calculate algnent Appl algnent Update error If error > tresold e ICP algort alwas converges onotoncall to a local nu wt respect to te MSE dstance objectve functon. Convergence eore Correspondence error : e N N S S Algnent error: d N N S S s Rot ( s o ) rans e analss Eac teraton ncludes an steps A. Fndng te closest ponts : O(N M ) per eac pont O(N M *N S ) total. B. Calculatng te algnent: O(N S ) C. Updatng te scene: O(N S ) Optzng te Algort As te ICP algort proceeds a sequence of vectors s generated : q, q, q, q 4 q q q q q cos q q ICP Varants Varants on te followng stages of ICP ave been proposed:. Selectng saple ponts (fro one or bot eses). Matcng to ponts n te oter es. Wegtng te correspondences 4. Rejectng certan (outler) pont pars 5. Assgnng an error etrc to te current transfor 6. Mnzng te error etrc w.r.t. transforaton