Color Constrained ICP for Registration of Large Unstructured 3D/color Data Sets

Transcription

1 Color Constraned ICP for Regstraton of Large Unstructured 3D/color Data Sets Sébasten Druon, Mare-José Aldon, André Crosner To cte ths verson: Sébasten Druon, Mare-José Aldon, André Crosner. Color Constraned ICP for Regstraton of Large Unstructured 3D/color Data Sets. IEEE ICIA 06: Internatonal Conference on Informaton Acquston, Aug 2006, 1, pp , <lrmm > HAL Id: lrmm Submtted on 2 Feb 2007 HAL s a mult-dscplnary open access archve for the depost and dssemnaton of scentfc research documents, whether they are publshed or not. The documents may come from teachng and research nsttutons n France or abroad, or from publc or prvate research centers. L archve ouverte plurdscplnare HAL, est destnée au dépôt et à la dffuson de documents scentfques de nveau recherche, publés ou non, émanant des établssements d ensegnement et de recherche franças ou étrangers, des laboratores publcs ou prvés.

2 Color Constraned ICP for Regstraton of Large Unstructured 3D/color Data Sets. S. DRUON, M.-J. ALDON and A. CROSNIER Unversty of Montpeller2-CNRS Laboratore d Informatque, de Robotque et de Mcroélectronque de Montpeller 161 rue Ada MONTPELLIER - FRANCE Abstract In ths paper, we address the problem of par-wse regstraton of large unstructured 3D/color datasets. Our purpose s to mprove the classcal ICP (Iteratve Closest Pont) algorthm by usng color nformaton, n order to deal wth large datasets and wth objects for whch the geometrc nformaton s not sgnfcant enough. After a bref presentaton of classcal ICP (Iteratve Closest Pont) algorthm and of the research works developed to mprove ts performance, we propose a new strategy to mprove the selecton of ponts. Color nformaton s used to reduce the search space durng the matchng step. Expermental results obtaned wth real range mages show that the algorthm provdes an accurate estmaton of the rgd transformaton. 1. Introducton There s an ncreasng nterest n automatc buldng of 3D computer models of real-world objects and scenes. These models may be used for a large varety of purposes such as buldng 3D envronment maps for robot navgaton, creatng vrtual realty models through observaton of real objects, dgtzng hstorcal buldngs for restoraton plannng, or archvng hertage objects from museums or cultural organzatons. Buldng these 3D models requres three steps: the acquston of a set of range mages, the regstraton of these mages n order to place them n a common coordnate system, and a data merge step whch smplfes the algned vews to produce a smplfed surface descrpton. Two knds of technques may be used for range mage acquston. The frst category s based on actve range data (tme-of-flght, structured lght, laser trangulaton) and can provde dense range mages of the observed scene. In the second category, stereovson and dynamc vson (whch uses only one movng camera) are based on vdeo mages. They allow us to extract the texture of the scene and a mnmum 3D structure (e.g. some nterest ponts), but generally the full geometry of the scene s not avalable. Creatng an object model requres generally multple partally overlappng vews acqured at dfferent sensor locatons. The number of vews ncreases wth the sensor resoluton. It depends also on the object complexty that create more or less shadow zones, and on the object sze. So, these dfferent range mages must be regstered,.e. placed n a common coordnate system. Our research work has been carred out n the context of creaton of hgh-resoluton 3D/color models of hertage objects for museums. They nclude pantngs, sculptures, and archaeologcal fgurnes, etc. To acheve ths, we are developng an ntegrated approach to the constructon of 3D textured models that take full advantage of the complementary nature of range and vdeo data provded by some structured lght sensors. Moreover, wth ths acquston mode, range and color scannng beng acheved by the same sensor, the samplng of color and 3D data s provded n the same coordnate system (the sensor frame). In ths paper, we address the problem of par-wse regstraton of large unstructured 3D/color datasets. Our purpose s to mprove the classcal ICP (Iteratve Closest Pont) algorthm [1,2] by usng color nformaton n order to deal wth: - Large datasets (up to a few mllon ponts), - Objects for whch geometrc nformaton s not sgnfcant enough. Ths paper s organzed as follows. In secton 2 we descrbe the classcal ICP (Iteratve Closest Pont) algorthm and the research works developed to mprove ts performance. In secton 3, we propose a new method to use color nformaton as a constrant n the matchng step. The last secton presents expermental valdaton results obtaned wth real mages of three knds of scanned objects: a colored wood statue, a cylndrcal vory box wth colored patterns and an ol pantng. 2. Prevous work Regstraton s the process of brngng two data sets nto the best possble algnment by estmatng the rgd moton 1

3 parameters that transform correspondng data nto each other [3]. Data sets to be regstered may be ntensty pctures, surface data, volume data, or multmodalty data (for example textured range mages). We present a bref overvew of regstraton methods based on the Iteratve Closest Pont algorthm applcable to range mages of an object taken from dfferent vewponts Basc ICP algorthm Snce ts ntroducton [1, 2], ICP has become the most used method for regsterng 3D shapes. Ths algorthm estmates the rgd moton parameters between two 3D shapes by assumng that: - The two shapes are partally overlappng,.e. a set of N ponts n the frst shape are supposed to have correspondences n the second shape, - They are approxmately algned;.e. a rough estmate of the ntal transformaton s known. The ICP algorthm s composed of two steps: the frst one generates temporary correspondences, and the second one estmates the relatve rgd-body transformaton. Let us consder two ponts clouds { p } (cloud 1) and { p' } (cloud 2) ssued from the measurement of an object by a 3D sensor, from two dfferent vewponts. From a geometrc pont of vew, we can consder that the correspondng pont p of p',.e. ts closest pont n { p }, can be defned as: p = arg mn p j p' (1) p { p } j For every par of correspondng ponts p and p', we want to fnd the rotaton (3x3) matrx R and the translaton (3x1) vector t so that: p = Rp' + t (2) wth p = ( x, y, z) T. For convenence, we use the homogeneous coordnate transformaton T, whch has sx free moton parameters (3 angles and 3 translaton components): T ( p) = Rp + t (3) So, we wrte (2) as: p = T ( p' ) (4) Theoretcally, three pont pars ( p, p' ) are necessary to dentfy a unque transform T. However, due to the dfferent nose sources (sensor measurements, mage segmentaton, pont matchng), the transformaton estmated from three arbtrary pont pars s not generally the best one. Estmatng the best moton parameters requres usng an error mnmzaton technque such as searchng a leastsquare soluton to the over-determned system of equatons (4). For a set of N matches, the crteron to be mnmzed s: N 1 2 ε = p Tp' (5) N = 1 Ths two steps procedure must be terated to reach the convergence of the computed transformaton,. e. when the change n mean square error falls below a threshold, whch specfes the requred precson of the regstraton. So, the ICP algorthm can be summarzed as follows: Let T 0 be an ntal estmate of the transformaton Repeat for k = 1... k, or untl termnaton crtera s met Fnd a set of k max N closest pont pars n { p } and { } p', accordng to (1); Estmate the transformaton Tk that mnmzes the dstance crteron (5) Apply the transformaton T k to all ponts of { p' }. Besl and Mc Kay [1] have desgned ths method to regster a set of 3D ponts { p } to a reference 3D model M. Assumng that all the ponts p belong to M, they have shown that the ICP algorthm always converges monotoncally to the nearest local mnmum. In our p. applcaton, cloud { } p' s not a subset of cloud { } So, t s necessary to prevent false matchng wth ponts that don t belong to the overlappng regons. False matchng can occur durng the frst steps of ICP and lead to a based soluton Varants of the ICP algorthm Snce the last decade, many varants have been ntroduced to mprove the orgnal ICP approach. The reader can fnd an nterestng classfcaton and comparson of these varants n [4]. We gve here a bref overvew of the major approaches, whch try to reduce the cost of the expensve closest pont fndng, and to make regstraton more robust aganst outlers due to nose or to false parng of nonoverlappng surfaces. 2

4 In [5], all avalable ponts are used to acheve the matchng phase. In order to decrease the computaton tme, a reduced set of ponts may be selected ether n one, or n both clouds. For nstance, n [5] a random process s used to select dfferent samples of ponts at each teraton. The closest pont research strategy may also be mproved by usng a k-d tree [7] Addtonal vewpont nvarant attrbutes may be used to select only compatble ponts durng the matchng step. They can be computed ether from the range mage or from addtonal data provded by the magng sensor (ntensty, color). In [6], a vector of geometrc and photometrc attrbutes s attached to each pont. It ncludes the magntude of the prncpal curvatures computed from range data, and ntensty based attrbutes (dffuse and reflecton coeffcents). The photometrc data are also exploted n [9] and [10]. Local curvature, texture [8], shape (3D dfferental attrbutes) [6] and angle between normals [10] have also been explored. Dfferent methods have been proposed to elmnate false matches whch may prevent the estmaton algorthm to converge to the correct soluton. In [7], pont pars are elmnated when ther dstance s larger than a threshold adaptvely computed wth a statstcal method based on the dstance dstrbuton. In [5], a Least Medan of Square (LMS) estmator classfes range data nto nlers and outlers. Moreover, the dstance functon (5) can be modfed n order to reduce the effect of bad matches by ncludng addtonal terms that measure the ponts smlarty. In [11] a dstance n the color space s added to the Eucldan dstance Our approach When geometrc nformaton s not meanngful enough to acheve regstraton, color nformaton becomes the only avalable crteron for the matchng step. Our hypothess states that a colored pont should be pared wth a pont of the same color n most cases. We have seen that one way of applyng ths constrant s to use a metrc takng nto account both geometrc and color nformaton, as n [11]. The weakness of ths approach s to fnd the correct weghts for both terms snce they don t have the same nature and don t rely on the same scale. Our approach conssts n usng a prelmnary color-based clusterng of the data before performng ICP. prelmnary classfcaton of the pont clouds usng color classes ams at - Improvng the robustness of the matchng process by only allowng the parng of color compatble ponts, - Reducng the amount of data to be processed, - Resolvng cases of objects for whch the algnment problem s dffcult to solve wth a pure 3D ICP algorthm (symmetres, etc.) 3.1. Color based classfcaton of the pont clouds Because lght condtons are not controlled durng acquston, we had to choose a perceptual color representaton. Furthermore, ths representaton must be ndependent of any color calbraton snce ths nformaton s not avalable on most 3D commercal sensors. The segmentaton of each data cloud { p } and { p' } reles on the Hue component of the HSV color space, as defned n [12]. Saturaton S and ntensty V are not taken nto account snce they are really dependant on the acquston condtons. For nstance, under normal llumnaton, most color varatons of an object dgtalzed from dfferent vewponts come from varatons n shadng. These varatons generally affect the ntensty of the ponts but not ther ntrnsc color. Moreover, usng Hue as addtonal nformaton for pont selecton doesn t requre an llumnaton model as n [7] where the ntrnsc surface reflectance parameters must be computed. We consder a color space n whch the H component vares from 0 to 6. Wth each nteger value s assocated one pure basc color: Red, Yellow, Green, Cyan, Blue, and Magenta. One color class s characterzed by a tght nterval centered on one of these values. From each orgnal data cloud, we generate sx sub clouds correspondng to each color class. The remanng ponts are not taken nto account D matchng under color constrants In ths paper, we present a new approach of the ICP algorthm. The contrbuton of the work concerns the use of color nformaton to acheve the pont selecton step. A Fgure 1: Repartton of the 6 classes on the H component When saturaton s under a certan threshold, a color s more or less grey, and ts hue component doesn t seem to 3

5 be meanngful. Ths s the reason why we also apply a flter on the S component, removng all ponts wth small values of S Crtera for class selecton. Presently, the regstraton algorthm uses only 3D/color ponts belongng to one class C (for nstance to the Red class). We have defned two crtera for the selecton of ths class: The rato R C of ponts contaned nto the class C. A good class must have a reasonable number of ponts, for example, n the followng expermentatons we have: 5 % < R C < 15 % ( 6 ) The lower lmt s easy to understand. If a class s less than 5% of the ntal data cloud, we consder that t s a bad representaton of the whole populaton of data. The upper lmt s for performance purposes. The am s to reduce the computaton tme by tryng to select a class whch contans regons of nterest. The rato Nc of ponts of the class C ntally ncluded n the boundng box of data cloud 2 One dffculty wth reducng the problem of regstraton to a sub problem based on only one class s that ths class must be representatve of the overlappng regon. Snce we don t have any nformaton about ths overlappng regon, we use the ntal transformaton provded by the user n order to estmate ths regon. The class wth the hghest number of ponts n ths regon s the most lkely to allow good regstraton. To each class s assgned a score Sc based on the followng expresson: Where Sc = Nc + f ( Rc ) (7) f( Rc ) = 100 f constran (6) s respected f( Rc ) = 0 f constran (6) s volated The class wth the hghest score Sc s selected for the regstraton ICP based regstraton For the regstraton, we now consder only two sub clouds { p } and { p' } belongng to the class wth the hghest value of Sc. The algorthm for ICP regstraton makes use of an adaptve threshold Dmax nspred from [7]: - Let T 0 be an ntal estmate of the transformaton - Intalze Dmax to ( 20 x resoluton of the datasets ) - Repeat for k = 1... kmax, or untl termnaton crtera s met - For each pont n { p } - Search for ts nearest neghbor n { } - If dstance ( P, P ) < Dmax p'. Add the par to the lst of regstered pars Lk - Compute the mean and standard devaton of the dstances from the lst L k - Set Dmax to ( mean + standard devaton ) - Estmate the transformaton T k based on the regstered pars L k - Apply the transformaton k T to all ponts of { } p'. - Loop untl mean, standard devaton, and number of regstered pars are stablzed. In order to speed up the search process for the closest p' as pont, a kd-tree s computed on the cloud { } descrbed n [7]. The complexty of the regstraton algorthm s : n 1. n 2. log ( n 2 ) where n 1 and n 2 are respectvely the number of ponts of p'. { } p and { } If k max teratons are performed wthout convergence, a new regstraton s performed wth the class havng the second hghest score, and so on. 4

6 4. Expermental results wth real data Ths algorthm has been evaluated wth data from real objects acqustons. The datasets descrbed here were acqured wth a Breuckman TRITOS sensor [14] (Image:1280x1024 pxels). Ths sensor uses a non-contact 3D measurng method (frnge projecton technque) wth color capabltes, each RGB color component beng coded on 5 bts Descrpton of the data sets Three sets of data are consdered n ths paper. They have been selected because they are representatve of dfferent object classes encountered n the doman of art preservaton Results The results obtaned wth our regstraton algorthm on these 3 datasets are summarzed n table 2. For each experment, we gve the number of ponts n the selected color class for each cloud, the fnal average error n mm, the number of teratons requred to fulfll the convergence crtera, and the fnal number of pars n L k. We consder that convergence s reached when the two followng condtons are satsfed durng two successve steps of the ICP algorthm : - The number of pars remans the same, - The average error and the standard devaton varatons are less then 1e-6 mm Name of the dataset : Nature Bal ( f g 4 ) Panted wooden statue Ivore ( fg 5 ) Colored Ivory Box Walls ( fg 6 ) Ol Pantng Sze of cloud 1 ponts ponts ponts Sze of cloud ponts ponts Ponts Sensor Resoluton (µm) 180x180x6 60x60x3 60x60x3 X Y Z 3D Poor Poor Rch Informaton (cylnder) (plane) Color Poor nformaton (shny) Rch Rch Table 1 : Datasets descrpton Due to the lmted sze of ths paper, the effect of sensor errors and nose s not nvestgated here by usng synthetc mages. However, we underlne that results presented n the followng secton have been obtaned wth real nosy data: Nose on color data s essentally due to the dgtalzaton process (each color component s encoded wth only 5 bts), and to the acquston condtons (lghtng varatons) Concernng the range data qualty, Bal has a 3D complex shape, whch nvolves many shadow zones, Ivore and Walls are dffcult objects because ther shape present symmetres and any nterest pont. Name of the dataset : Bal Ivore Walls Selected Class Red Green Cyan Rato of selected ponts 5.95 % 7.06 % 9.10 % Fnal average error (mm) Iteratons Fnal number of pars n L k Table 2: Summary of the results The fnal average error s very close to the sensor resoluton chosen for each experment, whch valdates the qualty of the fnal regstraton. Runnng on a Dell Pentum GHz - 1 Gg Ram computer, wthout any optmzaton of the source code, the experments took a few mnutes for Ivore and Bal. In the case of Walls the computng tme s very mportant. Because of the poor 3D nformaton n the Z drecton., the kd-tree structure degenerates. Furthermore, the class selecton could be enhanced. We present n fgure 2, the curves showng the evoluton of mean and standard devaton of dstances resultng from regstraton for Bal expermentaton. 5

7 (mm) Dataset : Bal 5 4,5 4 3,5 3 2,5 2 1,5 1 Average Error Standard devaton whch the algnment problem s dffcult to solve wth a pure 3D ICP algorthm. Ths soluton appears as a new alternatve for 3D color objects when purely geometrc regstraton does not lead to an acceptable soluton. Our present work ams at mprovng the segmentaton process by ntroducng automatc clusterng n the process of class buldng. Moreover, further work must be done on the computaton tme, by ntroducng parallelsm. 0, Iteraton Fgure 2 : Evoluton of error n Bal experment Fgure 3 shows the evoluton of regstraton through the number of pars n Lk. Dataset : Bal pars regstered Iteraton Fgure 3 We can deduce from those results that rough regstraton s reached after only teratons. 3D vews of orgnal datasets before and after regstraton are llustrated on fgures 4, 5, 6 and 7. We can outlne that, n the case of Ivore and Walls, purely geometrc regstraton algorthms do not converge to an acceptable soluton. 5. Conclusons We have proposed a new varant of the ICP algorthm that may be effcently appled to large 3D/color pont sets by takng advantage of the color nformaton. A prelmnary segmentaton step s used to select only ponts belongng to an nterestng color class, n order to speed up the 3D matchng and to ncrease the convergence rate of the teratve process The expermental valdaton acheved wth real data shows that: - The computatonal cost of the algorthm s reduced. - The method provdes an accurate estmaton of the rgd transformaton, even n the case of objects for Next step on ths work concerns expermental valdaton of the method by comparng t wth other approaches, whenever possble. 6. References [1] P. Besl and N.D McKay, "A Method for Regstraton of 3-D Shapes", Proc. of IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 14, No. 2, pp , 1992 [2] Y. Chen and G.Medon, "Object Modellng by Regstraton of Multple Range Images", Image and Vson Computng, Vol. 10, no. 3, 1992, pp [3] S. Seeger, X. Laboureux, Feature Extracton and Regstraton, Prncples of 3D Image Analyss and Synthess, Kluwer Academc Pub, 2000, pp [4] Szymon Rusnkewcz and Mark Levoy, "Effcent Varant of the ICP Algorthm". Proc. 3DIM, Québec, Canada, May 28- June 1, 2001, pp [5] G. Blas and M. D. Levne, "Regstratng Mult-Vew Range Data to Create 3-D Computer Objects", IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 17, No. 8, 1995, pp [6] Takesh Masuda and Naokazu Yokoya, "A Robust Method for Regstraton and Segmentaton of Multple Range Images", Computer Vson And Image Understandng, Vol. 61, No. 3, May 1995, pp [7] G. Godn, D. Laurendeau, R. Bergevn, A method for the regstraton of Attrbuted Range Images, Proc. 3DIM, Québec, Canada, May 28- June 1, 2001, pp [8] Zhengyou Zhang, "Iteratve Pont Matchng for Regstraton of Free Form Curves", IJCV, vol. 13, no. 2, 1994, pp [9] Wek S., Regstraton of 3D partal surface models usng lumnance and depth nformaton, Proc. 3DIM, Ottawa, Canada, May 1997, pp [10] Kar Pull, "Multvew Regstraton for Large Data Sets", Proc. 3DIM, Ottawa, Canada, October 1999, pp [11] Andrew Johnson and Sng Bng Kang, "Regstraton and Integraton of Textured 3-D Data", Proc. 3DIM, Ottawa, Canada, May 1997, pp [12] Foley and Van Dam, Fundamentals of nteractve computer graphcs, [13] Breuckmann web ste on Trtos sensor 6

8 Fgure 4 : Bal before and after regstraton Fgure 5 : Ivore before and after regstraton 7

9 Fgure 6: Walls before and after regstraton Fgure 7: Closer vew of the surface of Walls after regstraton 8