N-View Point Set Registration: A Comparison

Transcription

1 N-Vew Pont Set Regstraton: A Comparson S. J. Cunnngton and A. J. Centre for Vson, Speech and Sgnal Processng Unversty of Surrey, Guldford, GU2 5XH, UK fa.stoddart,s.cunnngtong@ee.surrey.ac.uk Abstract Recently 3 algorthms for regstraton of multple partally overlappng pont sets have been publshed by [], & Hlton [2] and Benjemma & []. The problem s of partcular nterest n the buldng of surface models from multple range mages taken from several vewponts. In ths paper we perform a comparson of these three algorthms wth respect to cpu tme, ease of mplementaton, accuracy and stablty. Introducton Several authors have consdered the problem of buldng complete surface models of complex objects usng range mages taken from several vews [2, 4, 5, 6]. Snce the vewponts (or object poses) are usually not known t s necessary to regster the surfaces taken from varous vews pror to fuson [8]. In the case of 2 vews the terated closest pont algorthm (ICP) [3] may be used to regster surfaces. Ths algorthm requres a soluton to the 3D pont set regstraton problem under rotaton and translaton. Several analytc solutons are avalable, see [] and references contaned theren. When more than 2 vews must be regstered the ICP algorthm may stll be used, provded that a soluton for the N-vew pont set regstraton problem s avalable. The N-vew pont set regstraton problem may be reduced to a chan of parwse problems and solved wth the 2 vew algorthm, however ths s not an optmal soluton. Informaton present n the unused overlappng vew pars should also be used for an optmal soluton. Recently 3 algorthms for regstraton of multple partally overlappng pont sets have been publshed [, 2, ]. The relatve merts have not yet been studed. In the 2 vew case a thorough evaluaton of the varous technques has been performed by Eggert et al [7]. The purpose of ths paper s smlar to the Eggert work and we present a comparson of 3 N-Vew regstraton methods. 2 N-Vew Regstraton Several analytcal solutons exst for the 2 vew pont set regstraton problem. These methods decouple the rotaton and translaton, and solve for the rotaton by computng the SVD of a (3 3) matrx [] or the egenvectors of a (4 4) matrx [9]. The translaton s then usually solved by calculatng the dstance between rotated centrods. 234 BMVC 999 do:.5244/c.3.24

2 Brtsh Machne Vson Conference 2 We wll study the methods of and Hlton (SH) [2], [] and Benjemaa and (BS) []. All the current methods for N-vew regstraton are teratve. However, Benjemaa and made a sgnfcant advance nsofar as they have been able to analytcally decouple the rotaton and translaton. 2. Problem defnton Each of the three papers s presented n a dstnct notatonal framework. We attempt to brng about some degree of consstency n our descrpton. We begn by defnng the problem followng the notaton of Benjemaa and. Benjemaa and assume that there are M pont sets each taken from a dfferent vewpont, S :::S M,whereS = fp :::p N g. The objectve s to fnd the best rgd body transforms, f :::f M, to apply to each pont set. A rgd body transform s denoted as f = fr T g,wherer denotes the rotaton and T denotes the translaton. Hence, f p = R p + T. The overlap of S wth S s denoted as O S where O = fp :::p N g. O has N ponts where each pont p s matched wth p 2 O S. Therefore N = N. also states that O = and N = for convenence n subsequent formulae. The problem may be specfed as mnmsng over the N transforms f the cost E where NX E[f :::f M ]= = = = w k f p ; f p k 2 () where the weghts w are gven. We note that the problem s undetermned up to a global transformaton appled to all pont sets. Ths can be removed by requrng that f s the dentty transform. 2.2 s [] method s by far the easest to mplement (provded that one of the 2 vew algorthms s already avalable!) It s teratve and based on the concept of mean shape. In [] ntroduced a formal framework n whch he was able to defne the averagng of shape. Each vew has correspondng ponts on the mean shape to whch t s regstered usng a standard pont set regstraton method, such as Horn et al. [9]. At the begnnng of the next teraton, the new mean shape s calculated, and agan the vews are regstered to t. Ths contnues untl convergence. consders an object represented by a k-tuple X = fx :::x k g. A rgd transform f appled on X s smply f X = ff x :::f x k g. In the real world, however, there are often mssng data and thus ncomplete k-tuples. If we have several nosy k-tuples of ponts we may wsh to compute the mean shape M = fm :::m k g. provdes a lengthy dscusson of formal ssues related to mean shape, but for our purposes t s suffcent to adopt the obvous defnton of the mean shape. Suppose we have N k-tuples X ::X N wth X j = fx j :::xj kg then the mean shape may be defned as P N j= m wj r xj r r = P N j= wj r (2) 235

3 Brtsh Machne Vson Conference 3 The weght wr j may be set to zero n the case of mssng data. At frst glance ths may appear to be a dfferent problem to our own but our problem may be converted nto a mean shape problem. Ths s done by reorderng the ponts and assocatng each measurement wth some pont r n a k-tuple by use of a mappng r( ). Hence the mean shape s defned as m r kx P M = r =P M P N = = w r= r p r( ) P M P M = =P N (3) r( ) = w Once we have a mean shape defned we can regster one vew at a tme to ths mean shape by mnmsng the resduals over f kx NX E[f ]= r= = = w r( ) k m r ; f p k 2 (4) f s solved by usng a closed-form soluton such as Horn et al. [9]. The soluton to the N-vew regstraton problem s obtaned by teratng over the two steps of computng mean shape and regsterng all vews to the mean shape. To summarse, the mean surface s frst computed. Next, the optmal transforms for each vew are then solved. The mean surface s then recomputed takng nto account the new transforms. The transforms are agan solved for, and ths process contnues untl convergence. 2.3 and Hlton and Hlton [2] use an teratve numercal method based on gradent descent. The problem s solved by analogy wth a physcal system of rgd bodes connected by sprngs. The frcton domnated equatons of moton dctate a soluton that evolves over tme to a local mnmum n potental energy. By ntegratng the equatons of moton over tme we solve the regstraton problem. To start off the process, guesses must be suppled. Durng the computaton, all transforms assocated wth each vew vary smultaneously. Once the computaton s complete, all vews are transformed so that the frst vew s transformed by the dentty transform. Each vew s assocated wth a rgd body havng an arbtrary center of mass and moment of nerta. These parameters do have consderable effect on the rate of convergence. A sensble choce for the parameters s as follows. The center of mass should be set to the centrod of a pont set and the moment of nerta s chosen as f each object were a sphere of radus equal to half the dagonal of a boundng box contanng the data. SH consder each par of correspondng ponts to be connected by a sprng of strength w makng t s possble to compute forces appled by these sprngs. The ndvdual sprng forces may be combned nto an overall force Ftot actng on the center of mass and a torque tot around the center of mass of a rgd body for each vew. In [2] t s shown how to compute these n a very effcent way. The force and torque are nserted nto a dynamcal system whch moves towards a potental mnmum. The followng frcton domnated equatons of moton are chosen. dy cm = Ftot (5) ;! = dt tot (6) resembles the mass and ; the moment of nerta, but here they represent the drag and 236

4 Brtsh Machne Vson Conference 4 rotatonal drag coeffcents.! s the angular velocty (rate of change of orentaton wth respect to tme). The system of equatons are then ntegrated by a smple qualty controlled Euler method whch can solve the dynamcal system wth adaptve step sze. It s guaranteed to converge to a local mnmum. 2.4 Benjemaa and Benjemaa and [] use a quaternon approach smlar to that of Horn et al. [9]. The general approach s as follows. Frstly the translatons are all elmnated analytcally. One vew, or pont set, s used as a reference frame by havng an dentty translaton and rotaton assocated wth t whch reman constant. Durng each teraton the rotaton for all of the other pont sets are solved. Only one pont set at a tme s allowed to be moved so that the rotaton can be solved for t, whle the others are kept fxed. Once that rotaton s solved, the rotaton for the next pont set s determned. Ths contnues untl all of the optmal rotatons have been obtaned. Ths process s contnued untl convergence Optmal Translaton Benjemaa and shows that the optmsaton of rotatons can be decoupled from the values of the translatons. The optmal translatons are then obtaned by usng a lnear combnaton of dfferences between rotated centrods. By gnorng the weght component of equaton () and expressng f n ts two components of rotaton R and translaton T, t can be wrtten as follows. E = NX = = = k R p ; R p + T ; T k 2 rewrtes the cost functon as E = E R + E t R,where E R = NX = = = k R p ; R p k 2 and goes on to show that E t R can be wrtten n a matrx form. Mnmsng E t R becomes then equvalent to the mnmsaton of Q(X),where X contans the unknowntranslatons for the vews and A & B are matrces computed from the data ponts []. Q(X) =X T AX +2X T B (7) By settng R = I and T =, the frst pont set s fxed and equaton (7) becomes, Q( X)= X T A X +2 X T B (8) where X and B are the vectors X and B deprved of ther frst element, and A s A wthout the frst row and column. notes that Q( X) s a quadratc form whch s mnmal when A X = ; B. Hence the translatons can be smply obtaned by the nverson of the matrx A. X mn = ; A ; B (9) 237

5 2.4.2 Optmal Rotaton Brtsh Machne Vson Conference 5 uses the result obtaned n equaton (9) and substtutes X mn for X n equaton (8) to obtan equaton (). Q( X mn )=; B T A ; B () E t R =2Q( X mn ) and s now expressed n terms of rotatons. goes on to show that mnmsng E s equvalent to maxmsng H. H = NX = = = R p R p + B T A ; B () uses propertes of quaternons to re-express the frst term of H as H = (_q _q ) T Q R (_q _q ) (2) = = where _q s a unt quaternon and _q the conjugate of _q. shows that the 2nd term of H, B T A ; B can be expressed as B T A ; B = (_q _q ) T Q t (_q _q ) (3) = = The reader s advsed to consult [] for more detals. Fnally the problem for each vew reduces to a problem of the form H(_q j )=2_q jt N j _q j (4) where N j = P M = 6=j QT Q j Q, and where Q s the quaternon matrx form. H(_q j ) s a quadratc form, and so the optmal unt quaternon whch maxmses ths functon s the egenvector correspondng to the hghest egenvalue of the matrx N j. 3 Results To characterse the three methods a seres of numercal experments were performed to determne the rate of convergence, accuracy, stablty, and computatonal tme requred by the methods. The reader may also wsh to consder mplementaton ssues alongsde other crtera. In our subjectve opnon havng mplemented all three algorthms the algorthm s by far the easest to mplement. The other two algorthms are both complex algorthms needng sgnfcant effort to mplement. In addton the SH algorthm has several free parameters whch are chosen heurstcally. The present mplementaton s based on a qualty controlled Euler routne whch requres some parameters to be set. In contrast the algorthm requres no parameters to be chosen other than the termnaton crteron and threshold. The same apples to Benjemaa and. 238

6 3. Creaton of Synthetc Data Sets Brtsh Machne Vson Conference 6 Synthetc data sets were generated from 3D surface models by a process ntended to emulate a multple vew range data acquston. We begn by selectng a 3D model. We choose some number of ponts at random on the surface. For each of the =::M vews a vew drecton was chosen and the subset of the total set was chosen that was vsble from the specfed vew drecton. Ths results n the pont sets S. It s n prncple possble for every pont set to overlap wth every other pont set. We specfed overlaps to take place only when a pont was smultaneously vsble from 2 vews. Thus we obtan the overlap sets (also called correspondence sets) O = O. The number of correspondence sets generated wll depend on the number of vews specfed and the characterstcs of the 3D model used. When addng a pont to a correspondence set, each vew wll have an dentcal copy of that pont. Hence the vews wthn each correspondence set are perfectly regstered. After addng nose t wll no longer be true that O = O. The next step n creaton of smulated data s the creaton of a number of vews. The vews were chosen to get the maxmum coverage of the 3D model. For each experment the same vews were used. The number of vews chosen were 2, 3, 6 and 8 whch allows a sequence of tests of ncreasng dffculty. The vews ( ) and ( ) were used n the two vew case. The three vew case used the addtonal vew ( ;), and for the sx vew case, the addtonal vews (; ), ( ) and ( ; ) were used. The eghteen vew case was generated by rotatng the sx vews around the x,y & z axes ndvdually by 45. In order to test the algorthms we need to start from some erroneous poston. We choose these postons as follows. The transform for the frst vew (vew ) s always null, and thus can be used as a reference frame. For each vew the rotaton and translaton s ncremented. The rotaton for the second vew (vew ) s and for each subsequent vew the angle s ncreased by. The rotaton axs s always ( ). The translaton for vew s(:2 :2 :2), and for subsequent vews the x, y and z components are ncremented by :2. Hence each vew has a unque rotaton and translaton assocated wth t. Fnally we add zero mean Gaussan nose wth rms to each coordnate of the synthetc measurements. [ Ths corresponds to sotropc nose wth rms p 3 when consderng the rms error on vectors.] The nose s set n terms of a percentage p of the dagonal of the boundng box of the nose free data, B as follows = pb= (5) 3.2 Quanttatve Measures Used There are two quanttatve measures that we can use to evaluate the result of regstraton. We can consder the errors between the ground truth and the estmated rotatons and translatons for each vew. For convenence we report the error for the last vew denoted and T, the former n unts of degrees. The second measure s the resduals between correspondng ponts after regstraton. 239

7 Brtsh Machne Vson Conference 7 Ths should be a weghted average over all the pont pars and s gven by e = P M P M P N w P M P M P N k f p w ; f p k 2 (6) Care should be taken to compute e from the above expresson as we have found that some mathematcally equvalent expressons are naccurate for small e due to roundoff errors. Snce we have added a known amount of nose to each component of the pont par we expect that p p e = 3 2 (7) The p 3 takes account of the 3 components (x, y, z), and the p 2 accounts for the fact that nose has been added to both ponts. 3.3 Convergence We begn by consderng data sets wth no nose added. Ths s an artfcal problem as t s possble to solve the problem by regsterng vews n a parwse manner. However ths case s very useful for determnng the rate of convergence of the algorthm. The dataset s derved from a surface model of an cosahedron wth unt radus. Frstly we examne the 2 vew case wth 5 random ponts per vew. Fgure shows the convergence of e as a functon of teraton number. We see that BS converges n step, converges n step and SH converges n 45 steps. The one step convergence of BS s expected snce n the 2 vew case t s equvalent to exstng analytc methods. As a purely numercal method the convergence of SH s as expected. The method of s somewhat faster than mght be expected n ths case, but we recall that t too contans a 2 vew analytc method wthn. For ths problem we expect e to converge to zero, we observe that all methods converge to a number n the regon of ;5, n other words the algorthms all converge to a number close to full machne precson. A more meanngful test of the algorthm s a case wth more than 2 vews. The next case we consder has 2 ponts sampled from the cosahedron and 6 vews. There were 2 overlap sets. No nose was added. The convergence s llustrated by the graphs n fgure 2. As can be seen all methods show geometrc convergence but SH and converge faster than. The results are summarsed n table. We see that all methods converge to full machne precson. The fastest method s SH Fgure : e, 5 Ponts, 2 Vews, No Nose 24

8 Brtsh Machne Vson Conference Fgure 2: e, 2 Ponts, 6 Vews, No Nose Table : 2 Ponts, 6 Vews, No Nose method teratons cpu e ;6 ;4 T ; In the next case we add nose equvalent to.5% of the dagonal of the boundng box. The results are summarsed n fgure 3 and table 2. The predcted value for e s.353 whch s consstent wth the result n the table (a) e Fgure 3: 2 Ponts, 6 Vews,.5 Nose Table 2: 2 Ponts, 6 Vews,.5 Nose (b) method teratons cpu e T An unexpected result s the overshoot of BS n the angle graph whch s not vsble n the graph of e. It does seem that s more affected by ncreasng number of vews as can be seen n a fgure of convergence for the 2 pont 8 vew case shown n fgure 4. 24

9 Brtsh Machne Vson Conference Hghly nonsphercal models Fgure 4:, 2 Ponts, 8 Vews,.5 Nose The results n the prevous secton are representatve of the overall behavour of the varous methods as appled to a dataset that comes from a regular approxmately sphercal shape. It s our belef that there are several stuatons where the behavour of the algorthm may be much worse. We have tested one such case n whch the data comes from a hghly non sphercal object. The object s generated from the prevously used cosahedron by scalng two axes by a factor of. The result s a long thn cgar shaped object. The reader wll recall that when ponts are collnear n the two vew case regstraton s not possble. We consder the 6 vew case wth 2 ponts and no nose. The results are shown n fgure 5 and table 3. It s clear that the SH method now fals completely and BS produces a sgnfcantly worse answer than Fgure 5: Degenerate - e, 2 Ponts, 6 Vews, No Nose Table 3: Degenerate - 2 Ponts, 6 Vews, No Nose method teratons cpu e T e-5.86e e e e e e As expected the angular error has become much worse ( ; ) due to the fact that we have begun to approach a degenerate case. 242

10 Brtsh Machne Vson Conference If we add nose of.% of the boundng dagonal we get the results shown n fgure 6 and table Fgure 6: Degenerate - e, 2 Ponts, 6 Vews,. Nose Table 4: Degenerate - 2 Ponts, 6 Vews,. Nose method teratons cpu e T e e e In fgure 7 we also show the behavour of under nose for 3, 6 and 8 vews. Some unusual convergence behavour s vsble for the BS method but t does make steady progress to the soluton as measured by e (a) 3 Vew (b) 6 Vew (c) 8 Vew Fgure 7: Degenerate -, 2 Ponts,. Nose 243

11 4 Concluson Brtsh Machne Vson Conference It s clear that s method s by far the easest to mplement. There are no parameters to choose. Its rate of convergence s geometrcal. It s the only method that consstently gves hgh accuracy solutons. We have also seen that s method s by far the slowest and n some applcatons we can magne the addtonal cpu tme would not be a major dsadvantage. That t s the slowest s a nevtable consequence of the fact that the other algorthms use cpu tme proportonal to the number of ponts added to the number of teratons whereas uses tme proportonal to the number of ponts multpled by the number of teratons. The BS method s harder to mplement and suffers from a slght loss of accuracy for the near degenerate case. [ Ths may be a flaw n our mplementaton. ] If speed s the most mportant crterum t s the best algorthm. The SH method has the dsadvantage of requrng addtonal parameters to be chosen. It fals n the near-degenerate case. References [] R. Benjemaa and F.. A soluton for the regstraton of multple 3d pont sets usng unt quaternons. In Ffth European Conference on Computer Vson, pages 34 5, Freburg, Germany, 998. [2] R. Bergevn, D. Laurendeau, and D. Poussart. Regsterng range vew of multpart objects. Computer Vson and Image Understandng, 6(): 6, 995. [3] P.J. Besl and N.D. McKay. A method for regstraton of 3D shapes. IEEE Trans. Pattern Analyss and Machne Intell., 4(2): , 992. [4] G. Blas and M. D. Levne. Regsterng multvew range data to create 3D computer objects. IEEE Trans. Pattern Analyss and Machne Intell., 7(8):82 824, 995. [5] Y. Chen and G. Medon. Object modellng by regstraton of multple range mages. Image and Vson Computng, (3):45 55, 992. [6] C. Dora, J. Weng, and A.K. Jan. Optmal regstraton of multple range vews. In 2th Int. Conference on Pattern Recognton, pages A569 57, Jerusalem, Israel, 994. [7] D. W. Eggert, A. Lorusso, and R. B. Fsher. Estmatng 3D rgd body transformatons: a comparson of four major algorthms. Machne Vson and Applcatons, 9:272 29, 997. [8] A. Hlton, A. J., J. Illngworth, and T. Wndeatt. Implct surface based geometrc fuson. Computer Vson and Image Understandng, 69(3):273 29, 998. [9] B. K. P. Horn, H. M. Hlden, and S. Negahdarpour. Closed form soluton of absolute orentaton usng orthonormal matrces. J. of Optcal Socety of Amerca, A5:28 35, 988. [] K. Kanatan. Analyss of 3D rotaton fttng. IEEE Trans. Pattern Analyss and Machne Intell., 6(5): , 994. [] X.. Multple regstraton and mean rgd shapes: Applcaton to the 3D case. In 6th Leeds Annual Statstcal Workshop, pages 78 85, Leeds, U.K., 996. [2] A. J. and A. Hlton. Regstraton of multple pont sets. In 3th Int. Conference on Pattern Recognton, pages B4 44, Venna, Austra,