Ecient Computation of the Most Probable Motion from Fuzzy. Moshe Ben-Ezra Shmuel Peleg Michael Werman. The Hebrew University of Jerusalem

Size: px

Start display at page:

Download "Ecient Computation of the Most Probable Motion from Fuzzy. Moshe Ben-Ezra Shmuel Peleg Michael Werman. The Hebrew University of Jerusalem"

Jeffrey McKenzie
5 years ago
Views:

1 Ecent Computaton of the Most Probable Moton from Fuzzy Correspondences Moshe Ben-Ezra Shmuel Peleg Mchael Werman Insttute of Computer Scence The Hebrew Unversty of Jerusalem Jerusalem, Israel Emal: fmoshe, peleg, Abstract An algorthm s presented for ndng the most probable mage moton between two mages from fuzzy pont correspondences. In fuzzy correspondence a pont n one mage s assgned to a regon n the other mage. Such a regon can be lne (aperture eect) or a probablty matrx. Nose and outlers are always present, and ponts may come from derent motons. The presented algorthm, whch uses lnear programmng, recovers the moton parameters and performs outler rejecton and moton-segmentaton at the same tme. The lnear program computes the global optmum wthout an ntal guess. 1 Introducton Many methods have been developed for moton recovery, yet the recovery of moton parameters n the presence of nose, outlers and multple-motons remans a dcult problem. Ths paper descrbes a new approach that overcomes many of the lmtaton of exstng methods ncludng errors due to outlers and to multple motons, lmted range, and the need for a good ntal guess of the moton model to avod local mnma. Secton 2 descrbes the new algorthm. Secton 3 presents several test results, and nally, Secton 4 summarzes the advantages of the new algorthm. 1.1 Prevous Work Gven a set of matched ponts between two mages (from an optcal ow or from feature matchng) a lnear parametrc mage moton (such as an ane moton) can be recovered usng a lnear pseudo nverse equaton system that mnmzes the average error (RMS, L 2 metrc). Ths RMS mnmzaton s vald only f the errors have zero mean. It wll fal n the presence of outlers and multple motons. Moreover, moton computaton that uses matched pars of ponts cannot express uncertanty drectly. In order to overcome these drawbacks several methods have been developed that utlze one or more of the followng technques: Moton segmentaton [2] - Technques for outler detecton are used, usually combned wth moton recovery by an teratve algorthm. Probablstc algorthm [1] - Algorthms that calculate the moton parameters from randomly selected pars of matched ponts untl they reach the desred accuracy. Probablty matrces [8] - A pont n one mage corresponds to a dstrbuton over locatons n the second mage. Moton recovery can be vewed as maxmzng the combned lkelhood of many local matches. When the moton conssts of pure translaton, the local moton gven by each probablty matrx matches the global moton. In ths case the most probable global moton can be recovered drectly from the local probablty matrces. When the moton s more complex, the other parameters (rotaton, scale) are recovered by extensve search over the parameter space. Drect computaton from grey level [5, 7] - Algorthms are based on the constant brghtness assumpton and the optcal ow constrant. They are usually combned wth moton segmentaton methods n an teratve manner. Global algnment of local measures [4] - Ths algorthm s a generalzaton of the drect grey level algorthms n the sense that t s not restrcted to grey level mnmzaton. The algorthm denes match-measure surface over the local match eld and uses Newton teratons to maxmze (or mnmze) the sum of the local measures.

2 2 Moton Computaton from Fuzzy Correspondence A pont P n mage I 1 s fuzzy correspondng to a group G n mage I 2 f the target of P, denoted as P, 0 s located wthn an area (or a group) n I 2 that s desgnated by G. There are varous ways of denng G. We start our dscusson wth groups that are dened as convex polygons. Fg.1.a llustrates fuzzy correspondence of four ponts. Each group G s represented by ts vertces G 1. ::Gk Pont P s mapped to the (unknown) pont P 0 where P 0 can be expressed as a lnear combnaton of vertces: G 1. ::Gk The moton computaton problem s dened as: Gven a set of pars: (P ; G ) nd the best parametrc moton that maps the source ponts P n mage I 1 nto ther target ponts P 0 n mage I 2 where P 0 combnaton of G. Subproblems are: s a lnear 1. If the vertces G k have derent weghts that represent lkelhood, nd the best moton that maxmzes the combned lkelhood. 2. If the set of pars contans outlers or multple motons, dsregard the outlers whle ndng the parametrc moton (and nd the pars that belong to the recovered moton). Ths problem that s called the moton segmentaton problem, has an nherent dculty: the parametrc moton s easly recovered f the outlers are known and the outlers are easly found f the the parametrc moton s known - solvng for both presents a dculty. A well known specal case of fuzzy correspondences s the aperture eect, or the recovery of global moton from normal ow. Fg.1.b llustrates the aperture effect problem and the normal ow for a pure translatng object. The wdth of the groups represents the uncertanty n the magntude of the normal ow and the length of the groups s the aperture eect uncertanty. The normal ow vector can be derved drectly from the mage grey levels usng the well known optcal ow constrant [3]. The target ponts resdes on the perpendcular lne to the normal ow vector n an unknown poston. Probablty matrces are also a specal case of fuzzy correspondence. A probablty matrx can be wrtten as a group f the surface of the probablty matrx s convex (whch s the common case). If the shape of the matrx s non convex the matrx should be parttoned nto convex subparts. The redundant subparts wll be dscarded as outlers by outler rejecton. (In ths case the groups wll have nner ponts and the area dened by each group s ts convex-hull). A dscrete probablty matrx can be mapped nto a pont to pont match where each group sze s of sze one, n ths case the problem becomes: selectng the maxmum lkelhood components of the matrx. (no group nterpolaton) Moton computaton from fuzzy correspondences s demonstrated n ths paper usng ane parametrc moton. 2.1 Implementaton The algorthm s mplemented by mappng the moton computaton problem nto a lnear programmng problem [6]. The lnear program tself s solved usng a standard lnear programmng algorthm. The most mportant features of the algorthm: global optmzaton and outler rejecton, are nduced by the lnear program's propertes. We rst descrbe the mappng of the geometrcal moton nto lnear programmng constrants (Secton. 2.2), then we descrbe the ndexng relaton that connects the geometrcal constrants to a selecton vector of lnear programmng varables (Secton. 2.3), nally we descrbe the lnear programmng objectve functon whch optmzes the selecton of the vertces n each group by ther weght to get the maxmum lkelhood soluton (Secton. 2.4). Fnally we wll rene the program allowng t to reject outlers and have better control on the requested moton; for example, allow only rotaton, translaton, and scale (Secton. 2.5) The nput for the mappng s a set of n pars fp ; G k g. Each par represent a mappng from pont P n mage I 1 nto the group of k () vertces G k() n mage I 2. Each group has ts own number of vertces k whch s a functon of, however n order to make the notaton more readable we wll just use a unform k for all groups. When we refer to a group as a whole - we wll use the notaton G. An optonal weght vector C can be assgned to each group. Ths weght vector represent preference of the vertces of G. 2.2 The Geometrcal Moton Constrant The ane transformaton that maps pont P = (x ; y ) n mage I 1 to the (unknown) pont P 0 = (x 0 ; ) n mage I y0 2 s gven by the followng par of constrants: x 0 = ax + by + e; y 0 = cx + dy + f Where: a; b; c; d; e; f are varables of the lnear program that are common to all n pars of these geometrcal constrants. The value of P 0 n unknown. However t s known that that P 0 s a convex combnaton of the vertces of the group G.

3 Group 5 G 1.4 Group 1 P5 G 1.3 P1 G 1.1 G 1.2 P2 Real Object moton vector P1 P5 G 4.4 G 4.3 P4 G 4.2 P4 G 4.1 P3 P1 P2 P3 Group 2 P2 Normal flow vector P2 Object P1 Normal flow vector a) Group 4 Group 3 b) Fgure 1: Fuzzy correspondence. A Pont P n mage I 1 s mapped to a group G n mage I 2. a) Ponts to groups. b) Normal ow as fuzzy data. 2.3 The Indexng relaton The relaton between the pont P 0 and ts group G s gven by the convex coecent vector S as: P 0 = kx j=1 S j Gj 0 S j 1 (1) P be dened by selecton of at most 3 vertces of G. These vertces are selected by three correspondng values of S j that are non zero. The vector S s called the selecton vector for the par fp ; G g and each element G j s the selecton values for vertex j n group G. 2.4 The Maxmal Selecton constrant Let S = [S k n] be the selecton matrx of all group vertces. S j s the selecton varable for vertex j n group, 0 S j 1. Row of the matrx S s the selecton vector for group G. S satses the selecton constrant: 8 Pk = 1. j=1 Sj Let C = [Cn] k be the weght matrx for all group vertces. C j s the predened weght of vertex j n group. For groups that represent convex probablty matrces, C wll satsfy: 0 C j Pk 1, 8 j=1 C j = 1 Let: T = nx kx =1 j=1 C j Sj (2) Then M ax(t ) over maxmzng assgnment of the selecton matrx S satses the followng propertes: (The maxmal selecton propertes). 1. If the assgnment of S s constraned only by the the selecton constrant and 8; C j 6= C l ; j 6= l then the assgnment S that maxmzes T wll be an ntegral f0,1g matrx contanng exactly n nstances of the value 1:0. Each selecton vector S wll have a sngle nstance of the value 1:0 correspondng to the maxmal member of C. Ths s true snce the maxmum of C s larger than the average of any subset of C (that has more than one member). 2. If S s constraned by the selecton constrant and by the geometrcal constrant (va the ndexng relaton) than each row S wll have at most 3 nonzero values, where one of the values corresponds to the maxmum member C. (S wll have at most two non-zero values f G s located on a lne). Ths s true snce P 0, that s located wthn the convex-hull of G s located n one of the two convex parttons created by the lne that s de- ned by the maxmal pont and any other pont of G. Notes: 1. If the values of C are not unform then there could be more non-zero values - but the objectve functon value and the recovered moton wll not change (snce the geometrcal constrants can be satsed wth no mpact on the objectve functon). 2. If the algorthm s forced to select more ponts than the maxmum pont m of some group (due to the geometrcal constrant) - t wll tend to select the other pont w geometrcally far from m snce ths wll maxmze the weght of m tself.

4 Ths behavor only ncreases the selecton of the maxmal lkelhood provded that the groups are convex shaped. 3. The Selecton matrx S plays two roles - It s the geometrcal nterpolaton values and t s also the weght selecton values. S actually selects at most three vertces and nterpolate only these weghts. (Ths stands n contrast to to nterpolaton of all ponts by ther dstance whch leads to L 2 metrc that we wsh to avod). 4. In groups havng convex shapes, the trangle de- ned by the three selected ponts (one of whch has the maxmum weght) s a contnuous lnear approxmaton to the surface near the maxmum pont - ths property enables ecent and robust soluton of surface lke optmzaton usng lnear programmng. 5. Non-convex shape groups are dealt wth by splttng them nto convex shaped groups. The outler rejecton wll dspose of the wrong parttons. In extreme cases (checker-board) each group wll be of sze one and the algorthm wll have no geometrcal nterpolaton - t wll be reduced to an L 1 selecton of ponts (whch s stll much better that L 2 due to ts outler rejecton property). 2.5 Outlers Rejecton and Transformaton Control In order to be able to reject outlers (ponts that do not agree wth the global moton that maxmzes T ). Free varables Z were added to each geometrcal constrant. The varable Z corresponds to the geometrcal error of the group G. The total number of Z varables s 2n (one for the X axs constrant and one for the Y axs constrant). In order to lmt the error we subtract P 2n =1 jz j from T, where s a parameter used to adjust the unts between the selecton vector unts (0..1) and the Z error unts (0..mage-radus), and s an optonal preference parameter for the whole group. When T reaches ts maxmal value the Z varables contans match nformaton and therefore can be used for segmentaton purposes as feedback weghts for teratve applcaton of the algorthm (elmnatng the need for threshold selectng). 2.6 The Lnear Program In order to get a lnear program, the only changes requred are the reshapng the C and S matrces nto one dmensonal vector. In order to make the namng conventon of the vectors C and S compatble wth the matrx namng conventon, double vector ndex s used: S j S j. The nal lnear program s gven by: 8 max : C t S? 2nX =1 jz j s:t: (3) S j 0 Pk j=1 S j = 1 Pk =1 S jg j :x = ax + by + e + Z Pk =1 S jg j :y = cx + dy + e + Z n+ The ane transformaton can be lmted, for example, to rotaton + scale + translaton only (no ane dstorton) by addng the constrants: a = d; c =?b. (The converson of the lnear program nto standard form s smple and requres two varables for each Z varable and two varables for each moton parameter). 3 Experments All tests have used a unform C = 1, = 0:001, = 1 whch gves equal preferences to all vertces. (Ths s a worst case scenaro - no preference nformaton exsts). 3.1 Synthetc Test Results Outler Rejecton Test In ths test selected 59 pars of ponts were selected randomly (group sze = 1) n the range (-100, 100), belongng to the ane model-1 and 41 ponts belongng to the ane model-2 ( 40 : 60 rato). The accuracy of the nput was up to 0.5 pxels, as only nteger locatons were used. The results are summarzed n Table 1. Errors were calculated as Eucldean dstances n the mage plane between the ground truth and the mage of the recovered ane transformaton. Fg. 2 shows the error of all the pont sorted by dstance. The ponts that belong to the recovered model are the rst 59 ponts. The rest of the ponts are consdered outlers. The segmentaton nto model and outlers s very clear Polygon Uncertanty and Outler Rejecton In ths test we used the same orgnal data as n the prevous test. Ths tme we gave the algorthm groups of four ponts whch are boundng rectangles of the real destnaton. The boundng rectangle vertces were selected randomly usng unform dstrbuton n the range (-3..+3) pxels. All vertces had equal weght of one. (The real locaton of each pont can be anywhere

5 Ane Model-1 Ane Model-2 Orgnal Recovered (Outlers) Ponts Mean(Error) Var(Error) mn(error) max(error) Table 1: Outlers Rejecton. (Unts: Pxels). Ane Model-1 Ane Model-2 Orgnal Recovered (Outlers) Ponts Mean(Error) Var(Error) mn(error) max(error) Table 2: Outlers and uncertanty. (Unts: Pxels). 14 Error pont (sorted by dstance) Fgure 2: Outlers. Error pont (sorted by dstance) Fgure 3: Outlers and uncertanty. wthn the boundng rectangle). The results of ths scenaro s presented n Table 2 and Fg. 3. The segmentaton s clear but the mean error of 1.9 pxels s stll to large. To solve ths, a second teraton was made gvng weght of 0 to the outlers that were dscovered by the segmentaton at rst teraton. The results of second teraton results are presented n Table 3 and Fg Images test results Two mages wth large dsplacement were selected form the \Puma" robot sequence. 14 ponts were manually selected from several locatons n the mage that have clear 3D structure (therefore they do not agree wth any sngle ane transformaton). The ane transformaton was recovered usng a pseudo-nverse (wth the exact dsplacements) and by the lnear programmng algorthm (usng group of uncertanty of one pxel at each drecton). The results are shown n Fg. 5. We can see that the pseudo-nverse algorthm reduced the average error but couldn't fully regster anythng at the scene whle the lnear programmng algorthm have fully regstered half of the ponts after Error pont (sorted by dstance) Fgure 4: 2nd teraton. a sngle teraton (Z = 0) resultng n a much better regstraton.

a) b) c) d) Fgure 5: a) Frst frame. b) Frames before regstraton. c) Pseudo-nverse regstraton. d) Lnear programmng regstraton. Ane Model-1 Ane Model-2 Orgnal 1.055-0.598 2.593 N/A 0.598 1.055 3.

4 Concludng Remarks A lnear programmng algorthm for the recovery of parametrc moton has been ntroduced. Ths algorthm has the followng advantaged over conventonal methods: 1.

6 a) b) c) d) Fgure 5: a) Frst frame. b) Frames before regstraton. c) Pseudo-nverse regstraton. d) Lnear programmng regstraton. Ane Model-1 Ane Model-2 Orgnal N/A N/A Recovered N/A Ponts 59 N/A Mean(Error) 0.47 N/A Var(Error) 0.01 N/A mn(error) 0.19 N/A max(error) 0.66 N/A Table 3: 2nd teraton. (Unts: Pxels). 4 Concludng Remarks A lnear programmng algorthm for the recovery of parametrc moton has been ntroduced. Ths algorthm has the followng advantaged over conventonal methods: 1. The algorthm utlzes fuzzy nput data that can span over large dsplacements usng L 1 metrc for optmzaton. 2. The recovery of the moton parameters as well as the outler rejecton and moton segmentaton s done smultaneously. 3. The algorthm does not requre an a-pror ntal guess of the transformaton 4. The algorthm provdes global optmzaton of the objectve functon whch s maxmal probablty. 5. The algorthm soluton can be controlled by addng more constrants (such as: pure scale, rotaton and scale, rotaton scale and translaton). vews. In Workshop on Geometrcal Modelng and Invarants for Computer Vson, [2] M. Ben-Ezra, S. Peleg, and B. Rousso. Moton segmentaton usng convergence propertes. In ARPA94, pages II:1233{1235, [3] B.K.P. Horn and B.G. Schunck. Determnng optcal ow. In MIT AI, [4] M. Iran and P. Anandan. Robust mult-sensor mage algnment. In ICCV98, pages 959{966, [5] M. Iran, B. Rousso, and S. Peleg. Detectng and trackng multple movng objects usng temporal ntegraton. In ECCV92, pages 282{287, [6] H. Karlo. Lnear Programmng. Brkhauser Verlag, Basel, Swtzerland, [7] B.D. Lucas and T. Kanade. An teratve mage regstraton technque wth an applcaton to stereo vson. In IJCAI81, pages 674{679, [8] Y. Rosenberg and M. Werman. Representng local moton as a probablty dstrbuton matrx and object trackng. In DARPA Image Undersadng Workshop, pages 153{158, References [1] Torr P.H.S. Zsserman A. and Murray D. Moton clusterng usng the trlnear constrant over three

CS 534: Computer Vision Model Fitting

CS 534: Computer Vision Model Fitting CS 534: Computer Vson Model Fttng Sprng 004 Ahmed Elgammal Dept of Computer Scence CS 534 Model Fttng - 1 Outlnes Model fttng s mportant Least-squares fttng Maxmum lkelhood estmaton MAP estmaton Robust