IMPLEMENTATION OF 3D SHAPE RECONSTRUCTION FROM RANGE IMAGES FOR OBJECT DIGITAL MODELING

Transcription

1 IMPLEMENTATION OF 3D SHAPE RECONSTRUCTION FROM RANGE IMAGES FOR OBJECT DIGITAL MODELING Landecr A. Albuquerque Automaton and Control Group (GRACO), Department of Mechancal Engneerng, Faculty of Technology, Unversty of Brasla Brazl lander@unb.br José Maurco S. T. Motta Automaton and Control Group (GRACO), Department of Mechancal Engneerng, Faculty of Technology, Unversty of Brasla Brazl jmmotta@unb.br Abstract. The necessty of obtanng geometrc models n three-dmenson that represent wth precson a real world object s becomng common each day. For ths, one has to recur to methods of 3D Modelng. Three-dmenson models have applcaton n several areas, amongst whch one can cte photogrammetry, archaeology, reverse engneerng, robotc gudance, vrtual realty, medcne, cnema, game programmng, and others. A current challenge s the constructon of 3D models dgtzed wth precson enough to be used n manufacturng systems or numercal smulaton of the performance of machnes and components n operaton, such as turbnes and flows n non-crcular ducts when the geometrc model s not avalable. The reconstructon of 3D shapes of objects or scenes from range mages, also known as depth maps, s preferable than usng ntensty mages or stereoscopy. These maps represent nformaton of dstances measured from an observer (optcal sensor or camera) to the scene n a rectangular grd. Therefore, the 3D nformaton s explct and wll not need to be recovered as n the case of ntensty mages. The reconstructon process presents three stages. The frst one s samplng of the real world n depth maps. The second stage s the algnment of several vews wthn the same coordnate system, known as mage regstraton. The thrd stage s the ntegraton of the vews for the generaton of surface meshes, named mergng. The current challenges converge to searchng methods that meet wth the hghest number of desrable propertes, such as robustness to outlers, effcency of tme and space complexty and precson of results. Ths work conssts n the dscusson of dfferent methods dealng wth 3D shape reconstructon from range mages found n the lterature and n the mplementaton of the second phase of 3D reconstructon: range mage regstraton. Keywords: Image Regstraton, Range Images, Surface Reconstructon, 3D Shape 1. Introducton Commonly, n manufacture processes a 3D model of the product prototype s frst constructed n CAD (Computer Aded Desgn) software. Next the computer model s mported to CAM (Computer Aded Manufacturng) software such that the product s manufactured usng some manufacturng process as for example machnng or rapd prototypng. Reverse Engneerng works n the opposte way. The real, physcal product, already exsts and t s necessary to buld up ts computer model (Eggert et al., 1998, Cerrada et al., 1990). Three-dmenson reconstructon can be used many tmes n ths task, snce a prototype can pass through several tmes between the real and vrtual world, and vce versa, untl the desred result s attaned. The prorty of 3D Reconstructon n Reverse Engneerng s the precson of the obtaned models. The reconstructon process presents the followng stages that are descrbed shortly n ths artcle (Dora, Weng and Anl, 1997, Pull, 1997, Chen and Medon, 1992): ) data acquston from multple vewponts (for short, these mages are called vews); ) regstraton of range mages, and ) ntegraton of vews. Ths research has for focus the mplementaton of the second stage: regstraton of range mages. The method used to algn the mages n ths work s the ICP (Iteratve Closest Pont) algorthm and the models obtaned are manly used for applcatons towards reverse engneerng to CAD 3D models. 2. Modelng Based on 3D Reconstructon Three-dmenson models are an essental resource for several areas. As examples one can cte: a) Modern ndustry: the avalablty of dgtal models used for desgn and mprovement of prototypes has already become a decsve factor for productvty n companes and qualty of products. Manufacture s one of the most benefted areas (Fg. 1). b) Autonomous navgaton: modelng by reconstructon has been studed ntensvely n recent tmes as a method to get three-dmensonal envronment maps for autonomous navgaton. It s not uncommon that moble robots are equpped wth long dstance range sensors (Ladars) for the specfc task of envronment 3D mappng;

2 c) Entertanment Industry: the entertanment ndustry has been stmulatng the development of 3D modelng technques. Snce t has vrtual realty as prorty, creatures and objects to be used n flms and 3D games are commonly generated by dgtalzaton of a real model. Fgure 1: Reverse engneerng for buldng a CAD model of a turbne rotor - GRACO/UnB. In the left, a vertcal secton of a Kaplan turbne. To the center, a sngle-blade turbne mockup. To the rght, the computer model of the turbne rotor obtaned from a contact profle dgtzer. d) Art and archaeology: the reconstructon of hstorcal sculptures for study, preservaton or creaton of vrtual museums; e) Medcne: vrtual models are largely used n medcne for surgcal plannng (e. g., plastc surgeres), prosthess constructon, etc. The area of 3D Modelng s relatvely young and as hardware and software technology advances and costs become lower new applcatons are thought. Three-dmenson models are tradtonally constructed by usng two methods: the frst s related to Computer Graphcs and conssts of syntheszng the real world. Ths lterally means to pcture the three-dmenson subject as an object or a real world scene. Ths approach s not recommended to all applcatons, snce t s very laborous and t does not produce realstc results when the object has some shape complexty. The second method s named reconstructon. It conssts n reconstructng the real world from mages taken by sensors and matchng them n order to buld the model. Ths approach s tradtonally the am of research of Computer Vson. The latter process conssts of dgtzng the object from several vewponts usng preferentally 3D dgtzers (rangefnders) based on the prncple of actve sensng. These dgtzers acqure samples of the object surface n the form of 2D matrces representng dstances from the sensor to the object. These samples are commonly called depth maps or range mages. Followng the mage acquston process t s necessary to algn the mages. Each mage s acqured centered n the sensor coordnate system. The relatve moton between the sensor and the object s a rgd body transformaton recoverng drecton and magntude of these motons (translatons and rotatons). These transformatons can be appled successvely to the mage vews to brng them together n a common coordnate system. Fnally, when the maps are algned t s possble to assemble them n order buld a sngle model. General solutons for the whole process do not exst, nor for each stage of reconstructon. The technques to be employed n each stage depend on several factors that ranges from the cost of the sensorng system to object surface characterstcs. All the stages are mportant and decsvely affect the reconstructed model. However, the most mportant factor for choosng a technque s the type of model applcaton. 3. Data Acquston One of the attractons for modelng usng reconstructon s the large amount of 3D dgtzers avalable n the market today (Petrov et al., 1998) n comparson wth some decades ago. Three-dmenson shape dgtzers, also known as rangefnders, capture surfaces or even object volumes. Choosng the correct dgtzer depends on some factors: type of applcaton, object sze and fnally the cost. Currently, complete commercal packages already exst, ncludng hardware and software to construct 3D models of free shape. However, n many of the cases t s necessary to construct a rangefnder from scratch, as n the acquston setup showed n Fg. 2.

3 (a) Fgure 2: Setup for range mages acquston - GRACO/UnB: Photograph of the expermental setup, showng camera, dodes and laser lght planes on a (a) plan object and a (b) reduced turbne model. Dgtzers use dfferent sensng technques to nteract wth the scene and measure ts dstance from the sensor, such as contact devces, optcal or acoustc probes, and others. Curless (1997) conceved a taxonomy that looks forward to organzng all these technques (Fg. 3). The technques for 3D-shape data acquston normally use passve and actve optc sensors. Passve technques acqure surfaces from ntensty mages, whle actve technques acqure a surface geometry projectng energy to the object surface. Due to ths nteracton wth the object surface, actve technques are hghly affected by the object surface propertes (Curless, 1997). A very used passve optcal technque s stereopse. Ths technque nvolves two calbrated ntensty cameras. The depth nformaton s extracted by means of trangulaton. The largest problem wth ths technque s the generaton of sparse depth maps. Amongst actve optcal technques, the most popular are: Lght Structure, also known as actve laser trangulaton, and Laser Imagng (Ladar). A system of structured lght s composed by calbrated cameras and a laser projector,. e., the locaton relatve to each other s known. Thus, t s possble to obtan depth nformaton by trangulaton. For long dstances such as n applcatons lke autonomous navgaton or photogrammetry the use of ladars s recommended. The great advantage of ths type of rangefnder s ts versatlty to work embedded on a moble vehcle. Ladars are expensve systems and use the TOF (Tme of Flght) prncple to recover depth. Amongst actve dgtzers the most attractve ones use coherent llumnaton such as the low ntensty laser. They produce depth maps wth a rapd and hgh samplng tax n contrast to contact dgtzers that are precse but also slow, dsjonted and produce maps wth a low samplng tax. In applcatons such as moble robotcs ladars are the most sophstcated soluton for envronment 3D modelng. In the ndustry, rangefnders of actve optcal trangulaton are the most used. In practce, the cost of hardware of actve methods s hgher than passve methods. However, passve methods demand more complex software for reconstructon, whch results also n expensve systems. 3. Image Regstraton For the constructon of 3-D models from range mages t s hghly desrable that the entre object surface s dgtzed. As each dgtzer scan occurs n only one drecton t s mpossble that the entre object surface s dgtzed n only one pass. Therefore, more than one mage has to be taken, comng up the need to algn the set of acqured mages. In the Computer Vson communty ths problem of algnment s known as Image Regstraton. 3.1 Prevous Work Image regstraton s a problem of crucal mportance n computer vson and much research nvolves the subject. Up to date many methods had been developed and new methods are constantly beng proposed, most of tmes, amng at a soluton for a specfc applcaton. Image regstraton s commonly assorted as an optmzaton problem (Blas and Levne, 1993), snce t ams at searchng the parameters of an optmal rgd moton amongst a class of possble ones between two mages. What n general dstngushes a regstraton method from another s the form they search for ths optmal moton transformaton. (b)

4 Fgure 3. Curless Taxonomy for the 3D shape acquston methods (Curless,1997). Although t s dffcult to sort out all the range of exstng methods n the lterature, many researchers dvde the methods nto two man categores (Blas & Levne, 1993, Dora, Weng and Anl, 1997): a) the ones the data acquston process s controlled n some way, ether by means of a robotc manpulator or a revolvng platform, and use the calbraton parameters for the regstraton, and b) the ones that obtan the transformatons straght from data. However, better results can be possbly obtaned combnng the two methods. The frst regstraton technque was based on matchng of dscrete features. Ths type of approach conssts n the extracton of local mage features nvarant to rgd moton such as concal or polygons, for example. The moton estmaton s obtaned from the correspondence between features overlappng mage areas. The problem wth ths approach s related to the extracton and sortng of these features. Iteratve approaches are more recent than the prevous ones and usually search for the optmal transformaton through teratve refnng of an ntal transformaton. An example of those approaches s the ICP (Iteratve Closest Pont) algorthm, one of the most popular regstraton methods, whch wll be descrbed n more detals ahead. Approaches of ths type are fast, easy to mplement and are based on the mnmzaton of a cost functon. However, some assumptons need to be fulflled for attanng some guarantee of convergence, such as havng a good ntal estmate. Optmzaton approaches that search for the optmal transformaton n the space of transformatons also exst. Stochastc optmzaton are used wth these methods such as Smulated Annealng (Blas & Levne, 1993), or technques of robust computer vson such as RANSAC (Chen et al., 1999).

5 3.2 Parwse and Smultaneous Regstraton There are two dstnct strateges to regster mages: a) Local or Parwse Regstraton and b) Global, Smultaneous or Parallel Regstraton. The dea of parwse regstraton s to dvde the entre process nto stages. The drect strategy s to focus on two mages each tme, and to regster one relatve to the other. Next to an mage par s regstered, a new par, ncludng a range mage of the formed par prevously, s regstered n the resulted coordnate system. Ths s repeated untl all the mages are used (Nshno and Ikeuch, 2002). The advantages of the parwse regstraton are the low computatonal cost n relaton to the global regstraton. The dsadvantage s a lower accuracy n the fnal result. The errors of each stage are added to the errors of the prevous one such that n the last stages there wll be a consderable accumulaton of regstraton errors. The global regstraton of multple range mages solves the problem of error accumulaton by regsterng all vews at the same tme. When regsterng all mages at the same tme the regstraton errors are spread among them. The number of data sets on whch the local and global strateges operate and how the correspondence problem s dealt wth consst the man dfferences between the two approaches. Approaches n pars operate on two sets of ponts, wth correspondences one-to-one defned between these sets. Global regstraton, on the other hand, nvolves multple sets of ponts wth multple sets of correspondence between them (Wllams and Bennamoun, 2001). The dsadvantage of the global regstraton s the hgh computatonal cost, manly the requrement of large spent of memory. 3.3 The ICP Algorthm Introduced by Besl and McKay (1992), the algorthm s an teratve and fast approach and of easy mplementaton of the 3D data algnment problem. The ICP presents several steps to whch heurstcs can be adjusted for turnng t to be faster or more accurate, gvng rse to a famly of algorthms. Heurstcs are approxmaton methods to solve problems n polynomal tme complexty, and meta-heurstcs are general purpose methods that gve good solutons, but the optmal soluton s not assured (Vana,1998). Rusnkewcz and Levoy (2001) proposed a classfcaton system and compared dfferent ICP varants based on the sx stages of tranng dentfed n the algorthm: Selecton, Matchng, Weghtng, Rejectng, error Metrc and Mnmzng. The ICP algorthm carres out a 3D regresson explorng the redundant ponts n the mages to calculate the moton parameters and to approach the vews (Arun, Huang and Blosten, 1987, Umeyama, 1991). That means, t s assumed that the mages overlap. Another requrement s to supply an ntal regster estmaton such that the algorthm refnes t teratvely (Besl and McKay, 1992). The algorthm stops when a certan level of precson n the overlappng between mages reaches a threshold. The frst stage of the algorthm s the selecton of control ponts wthn an mage. Next these ponts are matched to ther nearest par wthn the next mage. Ths par of ponts s to be assocated to the same pont on the object surface. Ths stage s the most challengng and needs a large span of tme due to the correspondng problem complexty. The errors assocated to the cost functon to be mnmzed are dependent on the precson by whch each par of ponts was corresponded. The mnmzaton of ths cost functon s a nonlnear optmzaton problem. Dependng on the precson of the ntal regster, ths cost functon can stck n a local mnmum and results n a non-precse algnment. Next to the mnmzaton stage, the ICP recovers the 3D moton parameters between mages and reuses them n one of the mages to move them towards each other. The estmaton of these parameters can be carred out usng ether quaternons (Horn, 1987) to represent rotaton or the SVD (Sngular Value Decomposton) (Arun, Huang and Blosten, 1987). 3.4 Mathematcal Formulaton of Regstraton Consder two sets of ponts x e y, where = 1,..., N. Eq. (1) models the rgd moton between the two sets of ponts = (1) x Ry + T where R s a rotaton matrx and T s a translaton vector. In the least-square sense, the cost functon for Eq. (1) s:

6 2 = N = 1 ( ) 2 Ry + T x (2) Eq.(2) s a cost functon based on the ordnary least-square. The optmal rotaton and translaton parameters are those that transform ths cost functon n a mnmum. It s convenent to uncouple the rotaton and translaton components. If the data ponts are algned, then x e y have the same centrod, that means, x = y, e T = x R y where T and R are the optmal transformatons. To recover rotaton, the errors p e q forms the cost functon of Eq.(6). p q x x = (4) y y = (5) 2 N 2 = p Rq = 1 (6) After some manpulaton Eq.(6) yelds Eq.(7) from whch the rotaton matrx can be uncoupled nto two orthonormal bass V and U (Arun, Huang and Blosten, 1987) from the SVD (Sngular Value Decomposton) of the covarance matrx H : H = N = q p T 1 (7) T H = UDV (8) So, T R = VU (8) Eq.(8) returns a rotaton f R s orthonormal ( R T 1 = R ), and det R =1. If det R =-1, R returns a reflecton. In the ICP algorthm ths process s run teratvely untl a threshold for the rotaton matrx, R, s reached. 4. Integraton of Vews After the range mages are algned they need to be ntegrated to shape the 3D model. The Integraton stage conssts of the generaton of surface representatons from the algned data and the edton of the model. The ntegraton s the process to create a representaton of a unque surface from sampled ponts of two or more mages (Turk and Levoy, 1994). Ths process can be consdered a post-processng stage. Free-shape objects can have complex surfaces. Many parts may not be reached by the scanner such as regons wth steep curvatures or regons that cannot be reached by the sensor. That may result n flaws n the fnal model that need to be mended, and sometmes some manual nterventon s necessary to correct these problems. 5. Metodology and Expermental Results Several test runs were carred out wth the ICP algorthm wth data from a cube (wthout nose), Fg. 4, and a range mage par of a sculpture (Buddha), Fg. 5. In the tests wth the cube, a regular object wth well contrasted edge lnes, the ICP algorthm reached the fnal algnment wth few teratons. The estmated rotaton angles showed a decreasng order as the teratons proceeded n the cases where there were no false pars of matchng ponts. In the presence of false pars the convergence could not be guaranteed. The Buddha range mages used are avalable n an mage database from OSU 3D Database, (Campbell and Flynn, 1998). These mages were acqured wth 200x200 laser scannng.

7 Several tests were planned wth dverse ntal startng postons. The results showed a clear relatonshp between the speed of convergence and the number of teratons wth the proxmty the range mages were to each other, ether results from the tests wth the cube or wth the Buddha. Fgure. 4. Regstraton of two range mages of a cube from two dfferent vewponts usng the ICP algorthm. The mages were ntally rotated 45º around the z axs. The teratons stopped when the threshold of 1º was reached. Fgure 5. Renderng of a 3D model (Buddha) from a par of range mages acqured by laser scannng. The expermental results had demonstrated that the convergence of the ICP algorthm s hghly nfluenced by the presence of false pars of correspondence. The nfluence of these false pars decsvely affects the convergence due to the error metrc used to be based on ordnary least-squares (not robust). Another fact to pont out s that, even n the cases wthout nose, the matchng heurstc (n the case, the next pont) can fal. 6. Conclusons The bggest challenge of the ICP algorthm s the convergence. Convergence depends on several factors, such as the qualty of the ntal regster, methods for choosng ponts and matchng, methods for cost functon mnmzaton and methods to estmate the 3D transformatons. Speckles and occluded regons are natural n range mages acqured by laser scannng. These problems can be tackled ntroducng some heurstcs n the ICP stages turnng t to be more robust. Ths artcle presents a descrptve sequence of the man methods used for 3D reconstructon amng at reverse engneerng to produce precse 3D CAD models. The ICP algorthm was mplemented and tests were carred out n several dfferent condtons to show up the man factors that nfluence the fnal precson of the model. Expermental results showed that convergence s a key problem wth the ICP algorthm, whch depends on several factors: ntal startng regstraton pont, the method used to choose the ntal control ponts, method to mnmze the cost functon and method to obtan the 3D moton transformatons.

8 To mprove the algorthm accuracy, n future works one can adjust heurstcs to avod false correspondences or, alternatvely, a more robust estmator to the error metrc can be mplemented. 7. References Arun, K. S., Huang, T. S. and Blosten, S. D., 1987, Least-Squares Fttng of Two 3-D Pont Sets, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 9, No. 5, pp Besl, P. J. and McKay, N.D., 1992, A Method for Regstraton of 3-D Shapes, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 14, No. 2, pp Blas, G. and Levne, M. D., 1993, Regsterng Multvew Range Data to Create 3D Computer Objects, Techncal Report TR-CIM-93-96, Centre for Intellgent Machnes, McGll Unversty. Campbell, R., Flynn, P., A WWW-Accessble 3D Image and Model Database for Computer Vson Research, Emprcal Evaluaton Methods n Computer Vson, Bowyer K.W. and Phllps P.J. (eds.), IEEE Computer Socety Press, pp , Cerrada, C., Ikeuch, K., Wess, L., and Reddy, R., 1990, A 3D-Object Reconstructon System Integratng Range- Image Processng and Rapd Prototypng, Techncal Report CMU-RI-TR-90-32, Robotcs Insttute, Carnege Mellon Unversty. Chen, C. S., Hung, Y. P. and Cheng, J. B., 1999, RANSAC-based DARCES: A New Approach to Fast Automatc Regstraton of Partally Overlappng Range Images, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 21, No. 11, pp Chen, Y. and Medon, G., 1992, Object modelng by regstraton of multple range mages, Image and Vs. Computng, Vol. 10, No. 3, pp Curless, B. L., 1997, New Methods for Surface Reconstructon from Range Images, PhD Thess, Stanford Unversty. Dora, C., Weng, J. and Anl, K. J., 1997, Optmal Regstraton of Object vews usng range data, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 19, No. 10, pp Eggert, D.W., Ftzgbbon, A.W. and Fsher, R.B., 1998, Smultaneous Regstraton of Multple Range Vews for Use n Reverse Engneerng of Cad Models, Computer Vson and Image Understandng, Vol. 69, No.3, pp Horn, B. K. P., 1987, Closed-Form Soluton of Absolute Orentaton Usng Unt Quaternons, Journal of the Optcal Socety of Amerca A, Vol. 4, No. 4, pp Lorusso, A., Eggert, D. W. and Fsher, R. B., 1995, A Comparson of Four Algorthms for Estmatng 3-D Rgd Transformatons, In Brtsh Machne Vson Conference, Brmngham, England, pp Nshno, K. and Ikeuch, K., 2002, Robust Smultaneous regstraton of Multple Range Images, In: The 5th Asan Conference on Computer Vson (ACCV2002), pp Petrov, M. A. Talapov, T. Robertson, A, Lebedev, a. Zhlyaev, L. Polonsky, 1998 Optcal 3D Dgtzers: Brngng Lfe to the Vrtual World, IEEE Computer Graphcs and Applcatons, Vol. 18, No. 3, pp Pull, K., 1997, Surface Reconstructon and Dsplay from Range and Color Data, PhD Thess, Unversty of Washngton. Rusnkewcz, S. and Levoy, M., 2001, Effcent Varants of the ICP Algorthm, In Proc. Thrd Internat. Conf. 3D Dgtal Imagng and Modelng, pp Turk, G. and Levoy, M., 1994, Zppered Polygon Meshes from Range Images, In Proceedngs of SIGGRAPH, pp Umeyama, S., 1991, Least-Squares Estmaton of Transformaton Parameters Between Two Pont Patterns. IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 13, No. 4, pp Vana, G. V. R, 1998, Meta-Heurístcas e Programação Paralela em Otmzação Combnatóra, Fortaleza, EUFC, Ceará, Brasl, 250 p. Wllams, J. and Bennamoun., M., 2001, Smultaneous Regstraton of Multple Correspondng Pont Sets, Computer Vson and Image Understandng, Vol. 81, No. 1, pp