A Volumetric Approach for Interactive 3D Modeling

Transcription

1 A Volumetrc Approach for Interactve 3D Modelng Dragan Tubć Patrck Hébert Computer Vson and Systems Laboratory Laval Unversty, Ste-Foy, Québec, Canada, G1K 7P4 Dens Laurendeau E-mal: (tdragan, hebert, Abstract Range mage regstraton and surface reconstructon have been tradtonally consdered as two ndependent problems where the latter reles on the results of the former. Ths paper presents a new approach to surface recovery from range mages where these two processes are ntegrated and performed n a common volumetrc representaton. The volumetrc representaton contans both mplctly represented reconstructed surface as the sgned dstance feld and correspondng matchng nformaton n the form of the gradent of the dstance feld. Ths allows both smultaneous and ncremental regstraton where matchng complexty s lnear wth respect to the number of mages. Ths mprovement leads to ncremental modelng from range mage acquston to surface reconstructon. It s shown that the approach s tolerant to ntal regstraton errors as well as to measurement errors whle keepng the detals of the ntal range mages. The paper descrbes the formalsm of the approach. Expermental results demonstrate performance advantages, tolerance to aforementoned types of errors and, as an applcaton, flterng usng redundant range data wthout loss of sharp detals on the reconstructed surface. 1 Introducton Regstraton and ntegraton (geometrc fuson) are two man steps n 3D modelng from multple range mages. However, the complexty of exstng algorthms [12], especally regstraton algorthms, s too hgh to allow real-tme, nteractve modelng. Ths paper takes a step towards realtme modelng systems by provdng a method for ncremental regstraton and ntegraton of range mages. Smlarly to [11, 12] we merge the ntegraton and regstraton steps. Both reconstructed model and precalculated matchngs are bult ncrementally. The algorthm s of lnear complexty wth respect to both the number of mages and the number of ponts and s thus a good canddate for parallel mplementaton. Ths mprovement leads to numerous applcatons: ffl Interactve acquston. Provdng a partally reconstructed model to the user durng the acquston greatly facltates the selecton of the next best vew and assures that the acqured mages are suffcent for the reconstructon of the model. ffl Flterng. Real-tme regstraton and ntegraton of redundant range data can be used to mprove the qualty of the reconstructed model by reducng the varance of the nose whle keepng ntact the sharp detals of the surface. Ths type of the flterng s equvalent to the frame averagng n mage processng. ffl Self referencng. Regstraton of range mages can be used to reference the sensor wth respect to ts envronment. Ths applcaton s mportant n moble robotcs snce the algorthm provdes an up-to-date volumetrc model of the scene and the rgd transformaton between subsequent vews. The rest of the paper s organzed as follows: a short overvew of regstraton strateges and problems related to the complexty of regstraton, as well as an overvew of volumetrc methods are gven n secton 2. Secton 3 presents the proposed soluton and descrbes the formalsm of the approach for regstraton and ntegraton of range mages. Expermental results obtaned wth the proposed algorthm are presented n secton 4. Fnally, drectons of further research and concludng remarks are gven n secton 5.

2 2 Regstraton and Integraton of Multple Range Images There are three ways to perform the regstraton of range mages: ) regstraton of two surfaces at a tme [17], usually referred to as par-wse regstraton, ) smultaneous regstraton of all mages [9, 13, 14, 18], and ) sequental regstraton of mages to prevously regstered and merged mages [3]. The frst approach generally causes an accumulaton of the regstraton error: when a par of mages s not perfectly regstered, the regstraton error propagates to the next par. On the opposte, smultaneous regstraton does not suffer from ths problem, but regsterng a sngle mage requres a matchng to all other mages. As a consequence, the algorthm complexty grows exponentally wth the number of mages. The thrd soluton s a compromse between the two others. In ths case the regstraton error s reduced and the number of parwse matchngs s equal to the number of mages. For a more detaled revew of exstng regstraton algorthms as well as a comparson of dfferent algorthms, the reader s referred to [6, 16]. The man performance problem of regstraton s lnked to the matchng step. The smplest approach to acheve matchng between two mages s to select ponts n one mage and to project them on the trangulated surface of the second one along the drecton of the sensor [2, 9]. By dong so, matchng s determned by the relatve poston of the mages, rather then by the surface shape. Another commonly used approach s to take the closest pont as the correspondng pont [1]. A brute force matchng algorthm requres O(N 2 ) operatons, where N s the number of ponts n each mage. Usng more sophstcated approaches based on k-d trees reduces complexty to O(NlogN). Recently, a number of volumetrc approaches for ntegraton have been proposed [5, 8, 12, 15]. These algorthms use an mplct representaton of the surface n the form of a sgned dstance feld calculated on a dscrete lattce of ponts. The surface can be recovered from such a representaton by extractng the zero-set of the dstance feld, usually usng the Marchng Cubes algorthm [1]. A strong pont of the volumetrc approach s ts ablty to ncrementally buld the model by smply summng the dstance felds for ndvdual mages. The ntegraton algorthm proposed n [8] s of lnear complexty and s thus well suted for our purpose. Nevertheless, none of those volumetrc algorthms, except [12], provde the regstraton of range mages. In ths last approach the extensve use of k-d trees to match the ponts makes ths approach unsutable for ncremental regstraton. 3 Incremental Regstraton The best compromse between the complexty and the qualty of regstraton s to regster an mage to a model reconstructed from prevously regstered and merged mages. However, ths does not solve the complexty problem tself: whenever an mage s merged to the model, the number of ponts and trangles grows and so does matchng complexty. Snce the ponts cannot be projected on the surface of the model, (the model s generally not a graph surface) one must search for the correspondng pont; ths leads to complexty O(N 2 ) or O(N log N ) usng k-d trees [7]. Our soluton to ths problem and the man dea of ncremental regstraton s to buld the matchng nformaton ncrementally n the same way as the dstance feld s bult for ntegraton. Ths conssts n precomputng closest ponts n the neghbourhood of the reconstructed surface such that matchng s encoded n closest ponts of a dscrete 3D lattce. Ths reduces to precalculate closest ponts n the neghbourhood of the reconstructed surface (on a dscrete lattce of ponts), so that the matchng nformaton can be obtaned smply as the value from the closest voxel centre. To do so, we note that the drecton of the closest pont on the surface s actually gven as the drecton of the gradent of the dstance feld d (where dfferentable), and that the followng relaton s true r X d =X rd: (1) The gradent of the ntegraton (summed) feld, and thus the drecton towards the closest pont of the reconstructed model can be computed ncrementally n the same way as the dstance feld tself. Ths solves the performance problem related to the number of mages snce each mage s used only once for matchng. It s possble to compute the gradent drectly from the dstance feld but the result s naccurate snce the dstance feld s calculated only on a dscrete lattce of ponts. Therefore, the gradent s rather computed explctly on the same lattce ponts for whch the dstance feld s calculated. For the performance problem related to the number of ponts we note that the dstance feld s always calculated relatve to a sngle trangle on the surface, and that there s a connected regon n 3D space where that trangle contans the closest pont for each pont. Parttonng a surface envelope n these regons allows ndependent computaton of the feld for each trangle. Ths reduces the complexty to be proportonal to the number of trangles. Such a partton of an envelope, referred to as fundamental prsms, s ntroduced n [4]. Not all dstance felds are well suted for regstraton. Calculatng the dstance n the drecton of the sensor, as proposed n [5], results n gradent values, and therefore n matchng drectons, that are determned by the drecton of the sensor nstead of dependng on the shape of the surface. In [11, 12] the computaton of the dstance feld s based on the dstance to the closest pont and s thus very senstve to nose (see [16], fg. 8). Hlton [8] calculates the dstance 2

3 relatve to the trangles n the drecton of ther normal. He also uses normal volumes to partton the space, so that nsde such a volume the dstance s calculated relatve to a sngle trangle and may lead to dscontnuous so-surfaces over the edges of fundamental prsms. To ensure contnuty of the dstance felds, we ntroduce a new dstance defned n the secton 3.1. Due to the senstvty of the closest pont drecton to nose, matchng errors occur whenever the dstance between two surfaces s large because many ponts on one surface are attracted by outlers. To lmt ths behavour, Masuda [12] restrcts the search for the closest ponts to those ponts wthn the dstance equal to a voxel dagonal. However, ths severely lmts the maxmum acceptable ntal regstraton errors. Our soluton to ths problem s to compute the dstance feld n the drecton of fltered normals rather then towards the closest pont on the surface. The ratonale s that the normals can be fltered effcently wthout flterng range data and, by dong so, the nfluence of nose can be reduced to a very small area and s practcally nsgnfcant. 3.1 Calculatng the Dstance Feld n the Drecton of Fltered Normals The normal at each vertex of a trangulated surface s computed as the average of the normals of all trangles contanng the vertex and fltered by averagng wth normals at vertces of adjacent trangles. Ths procedure yelds normals at vertces but not at ponts located nsde trangle boundares, an nformaton that s requred n order to match ponts not only to vertces but to the trangulated surface. For those ponts we lnearly nterpolate the normals at vertces. Thus, for any pont p on a trangle the normal s obtaned as n(p) =b 1 n 1 + b 2 n 2 + b 3 n 3 ; where b 1, b 2 and b 3 are barycentrc coordnates of p and n 1 ; n 2 ; n 3 are normals at the vertces of the trangle. Usng ths defnton of the normals we defne the closest pont p c to pont p as the pont whch verfes p = p c + dn c ; (2) where n c s the normal at pont p c and d s the dstance between p and p c. The computaton of the sgned dstance feld s based on ths defnton. Interpolated normals, closest pont p c along the normal n c, and the closest pont p e found usng Eucldean dstance are llustrated n fgure 1. The dstance feld s calculated only wthn some envelope of the surface. Accordng to our defnton of the dstance, ths envelope s bounded by two so-surfaces that are obtaned by dsplacng each vertex of the orgnal trangulated mage n the drecton of the normals for some constant value (see Fgure 1). As mentoned earler, the envelope s parttoned n fundamental prsms, an example of whch s shown n fgure 2. Fundamental prsms are bounded by two so-trangles T (d mn ) and T (d max ) and by three edge-surfaces whch are blnear patches defned usng two vertces of a trangle and assocated normals (see Fgure 2). dmax p1 dmn Edge surface T(dmax) T() p2 p3 Fgure 2. Fundamental prsm. dmax dmn T(dmn) As defned n Eq. 2, the dstance for any pont located nsde a fundamental prsm wth respect to the assocated trangle (generator trangle) s computed as the soluton of the followng system: [p 1 + dn 1 p; p 2 + dn 2 p; p 3 + dn 3 p] =: (3) In Eq. 3 square brackets denote the scalar trple product, p 1 ; p 2 ; p 3 are vertces of the trangle and n 1 ; n 2 ; n 3 ther normals. If pont p s n the prsm then t s located n the trangle T (d) whose vertces are p 1 + dn 1 ; p 2 + dn 2 ; p 3 + dn 3. Barycentrc coordnates b 1, b 2 and b 3 of the pont p n ths trangle correspond to the barycentrc coordnates of the closest pont n the generator trangle. Therefore, once the dstance s known, the closest pont s obtaned as p c = b 1 p 1 + b 2 p 2 + b 3 p 3 : (4) The gradent of the dstance map s nothng but the normalzed drecton of the closest pont: rd(p) = p p c kp p c k In practce, both felds (dstance and gradent) are calculated ndependently for each fundamental prsm. To do so effcently, the boundng box s calculated for each prsm and the dstance s calculated usng equaton 3 for each lattce pont n the boundng box. Furthermore, one must verfy that the pont s located nsde the prsm. Ths reduces to verfyng that the pont s contaned n a trangle T (d) as explaned above. Snce the normals at vertces of trangles are not parallel n general, a lne passng through a vertex n the drecton of the normal at the same vertex may ntersect facng edge (5) 3

4 d max n 1 p 1 n 2 p 2 d p e p c Interpolated normals p n c T(d) d n 3 p 3 T 3 () T 3 (d max ) n 4 p 4 n 5 p 5 d mn Fundamental prsms T 3 (d mn ) Fgure 1. Volumetrc envelope. surface. Ths lmts the maxmal dstance nsde a prsm because the equaton 3 have nfntely many solutons at the pont of ntersecton. Therefore, the calculated dstance s accepted only f t s smaller than ths maxmal dstance. Note that t s always possble to choose (flter) normals so that those ntersectons occur outsde the envelope, for example by takng all normals to be parallel to sensors drectons. 3.2 Incremental Update of the Felds The sum of felds for multple mages contans both the mplct representaton of the surface as a dstance feld and ts assocated matchng nformaton n the form of a vector feld correspondng to the gradent of the dstance feld. We refer to these felds as to ntegraton felds. As mentoned above, the ntegraton felds are obtaned by averagng felds for ndvdual mages: f nt (p) =" N X N d nt (p) =" X f (p)! (p) d (p)! (p) # N =" X # N =" X! (p)! (p) # # ; (6) ; (7) where f denotes the vector (gradent) feld, d denotes the sgned dstance feld, and where! represents the confdence level for the measured ponts, usually expressed as the cosne of the angle between the drecton of the sensor and surface normal. To preserve the contnuty of the felds, the weghts! should be nterpolated n the same way as the normals, usng the barycentrc coordnates.e. w(p) =w 1 b 1 + w 2 b 2 + w 3 b 3 : (8) An example of the dstance and vector feld s shown n fgure Image Regstraton Once the ntegraton felds are computed, the regstraton of a sngle mage s performed by algnng all measured ponts p wth p +d nt (p v )f nt (p v )+hf nt (p v ); (p p v ), where hf nt (p v ); (p p v ) compensates for the dstance between the pont p and the closest voxel center p v. There are two mplementatons of the regstraton algorthm. The frst one s sequental snce each mage s regstered to the ntegraton feld and then added to t. The second one creates the ntegraton feld usng all mages and then regsters each mage ndvdually. Both algorthms are descrbed n pseudo-code below. Algorthm 1 (Incremental Regstraton). Calculate felds f 1 and d 1 for the frst fmage f nt ψ f 1 and d nt ψ d 1 ψ 2 Repeat Regster mage I usng f nt and d nt Calculate felds f and d for mage I Add felds f and d to f nt and d nt ψ +1 Untl no mages left Algorthm 2 (Smultaneous Regstraton). Repeat Intalze ntegraton felds f nt and d nt to zero For = 1:number of mages 4

5 Fgure 3. 2D slce through the ntegraton felds. The dstance feld d nt s depcted as shades of gray, whle the drectons of gradent feld d nt f nt are depcted as arrows. Calculate felds f and d for mage I Add felds f and d to f nt and d nt End For = 1:number of mages Regster mage I End Untl convergence Whle the frst algorthm s ntended for use durng the acquston, the second one s used for regstraton and ntegraton after all the mages have been acqured. 4 Results In order to assess the performance of the algorthm t s very convenent to use synthetc range mages snce both regstraton and measurement errors can be perfectly controlled. More mportantly, the poston of the mages followng regstraton can be compared to ther exact poston. For ths purpose, 12 perfectly algned and noseless range mages from a CAD model of a Beethoven statue were generated. Images were then transformed as follows: each mage was translated along each axs for a random value between and 5 voxels and was rotated around each axs (whle centered at the orgn) for a random angle between and 5 degrees. Rotaton angles and translaton vectors have unform dstrbuton. Nose added to measured ponts followed a normal dstrbuton. The assessment of the regstraton error s made by comparng the poston of each pont n the regstered model to ts exact poston. Resoluton of the synthetc mages was whle the resoluton of the 3D lattce was To provde expermental evdence supportng the clam that the flterng of normals makes the algorthm less senstve to nose, the resdual regstraton error was measured for varyng level of nose whle keepng the regstraton errors constant. The result shown n fgure 4 ndcates that the nose has a mnor mpact on the performance of the algorthm. One mght argue that the dscretzaton of the felds should result n a less accurate regstraton. Ths s true but, snce the model s reconstructed on a dscrete lattce of ponts, regstraton errors smaller then the voxel sze are nvsble. Therefore t s suffcent to reduce the regstraton errors below voxel sze. Expermental results confrm that ths s accomplshed by the proposed algorthm. Fgure 5 shows the average and the maxmum error dstrbuton for the Beethoven model before (a) and after (b) regstraton. Fnally, the nfluence of nose on the convergence speed of the algorthm s llustrated n Fgure 6 for both 12 noseless mages and 12 mages corrupted by nose of varance 5

6 Maxmal regstraton error (voxels) Average regstraton error (voxels) Nose varance (voxels) Nose varance (voxels) Fgure 4. Regstraton error vs. level of nose. 1 voxel. The dagrams on the rght sde show change of the norm of rotaton the matrx whle the plots on the left sde show the change of the translaton vector as a functon of the number of teratons. These dagrams show that the convergence of the algorthm s practcally unaffected by nose. Snce the reconstructon feld s an average of all ndvdual felds, the flterng s performed automatcally. An example of the surface reconstructed by regsterng and ntegratng a sngle mage wth 1, 1 and 5 observatons of the object from another vewpont usng the above algorthm s shown n fgure 7. The second mage covers the left sde of the frst mage. Note that averagng makes sense only f the mages are well regstered. Also note that regsterng a very large number of mages, say a few thousands, wth a regstraton algorthm whose complexty s O(M 2 ) wth respect to the number of mages, s very dffcult f not almost mpossble for current algorthms. Another problem s that a frame rate of 3 or 6 mages per second clutters dsk space rapdly. By dong the reconstructon and flterng onlne, as t s proposed n ths paper, the redundant data can be dscarded as soon as ts feld s summed n the ntegraton feld. Illustraton of reconstructon and regstraton of the syn- Number of ponts 12 Number of ponts x Error (voxels) (a) Error (voxels) (b) Fgure 5. Dstrbuton of regstraton error for 12 noseless mages of a Bethoveen statue. (a) Regstraton error before regstraton. (b) Resdual regstraton error after regstraton thetc and real data s shown n fgure 8 for the Beethoven model and the model of a rabbt from the Stanford mage repostory. The executon tme of the unoptmzed algorthm, ncludng I/O, on a 1.2GHz PC, s 2 seconds per mage contanng approxmately 1 trangles each. 5 Concluson We presented volumetrc algorthms for regstraton and ntegraton of range mages. The algorthms are of lnear complexty wth respect to the number of mages and the number of trangles. The mpacts of our approach are numerous, but perhaps, the most mportant one s on sensor desgn. Onlne regstraton and flterng by averagng allow desgn of less accurate 3D scanners wthout sophstcated referencng systems that stll can produce hgh qualty models. As a matter of fact, we beleve that the future of 3D scanners s not only n the desgn of very accurate scanners 6

7 that use accurate referencng, but also n hgh-speed scanners that provde hgh-qualty models not by the qualty of range mages but rather by ther quantty (redundancy). The most mportant drawback of the proposed approach, as well as of any volumetrc approach, s the lmted resoluton mposed by memory requrements. Representng a surface, whch s a 2D object, n a 3D volume leads to very neffcent memory usage snce most of the voxels are unused. There are several proposed solutons for ths problem such as run-length encodng [5] or octrees. However, those methods are just more sophstcated data structures and they do not reflect any geometrc propertes of the shape they represent. We beleve that more effcent and useful compresson schemes are possble; ths wll be the next step n our research. 6 Acknowledgements The authors wsh to express ther grattude to Fonds pour la Formaton de Chercheurs et l Ade à la Recherche (FCAR) and Natural Scences and Engneerng Research Councl of Canada (NSERC) supportng ths research. References [1] P. Besl and N. McKay. A method for regstraton of 3-d shapes. IEEE Transactons on Pattern Analyss and Machne Intellgence, 14(2): , February [2] G. Blas and M. Levne. Regsterng multvew range data to create 3d computer objects. IEEE Transactons on Pattern Analyss and Machne Intellgence, 17:82 824, [3] Y. Chen and G. Medon. Object modellng by regstraton of multple range mages. Internatonal Journal of Image and Vson Computng, 1(3): , Aprl [4] J. Cohen, A. Varshney, D. Manocha, G. Turk, H. Weber, P. Agarwal, F. Brooks, and W. Wrght. Smplfcaton envelopes. In SIGGRAPH 96 Conference Proceedngs, pages , August [5] B. Curless and M. Levoy. A volumetrc method for buldng complex models from range mages. In SIGGRAPH 96 Conference Proceedngs, pages , August [6] G. Dalley and P. Flynn. Range mage regstraton: A software platform and emprcal evaluaton. In Proceedngs of the Thrd Internatonal Conference on 3D Dgtal Imagng and Modelng, pages , May 21. [7] M. Greenspan and G. Godn. A nearest neghbor method for effcent cp. In Proceedngs of the Thrd Internatonal Conference on 3D Dgtal Imagng and Modelng, pages , May 21. [8] A. Hlton and J. Illngworth. Geometrc fuson for a handheld 3d sensor. Machne vson and applcatons, 12:44 51, 2. [9] O. Joknen. Area-based matchng for smultaneous regstraton of multple 3-d profle maps. Computer Vson and Image Understandng, 71(3): , September [1] W. E. Lorensen and H. E. Clne. Marchng cubes: A hgh resoluton 3D surface constructon algorthm. SIGGRAPH 87 Conference Proceedngs, 21(4): , [11] T. Masuda. A unfed approach to volumetrc regstraton and ntegraton of multple range mages. In Proceedngs of the 14th Internatonal Conference On Pattern Recognton, pages , August [12] T. Masuda. Generaton of geometrc model by regstraton and ntegraton of multple range mages. In Proceedngs of the Thrd Internatonal Conference on 3D Dgtal Imagng and Modelng, pages , May 21. [13] P. J. Neugebauer. Reconstructon of real-world objects va smultaneous regstraton and robust combnaton of multple range mages. Internatonal Journal of Shape Modelng, 3(1,2):71 9, [14] H. G. R. Bergevn, M. Soucy and D. Laurendeau. Towards a general mult-vew regstraton technque. Pattern Analyss and Machne Intellgence, 18(5):54 547, May [15] G. Roth and E. Wbowoo. An effcent volumetrc method for buldng closed trangular meshes from 3-d mage and pont data. In W. Davs, M. Mante, and V. Klassen, edtors, Graphcs Interface, pages , May [16] S. Rusnkewcz and M. Levoy. Effcent varants of the cp algorthm. In Proceedngs of the Thrd Internatonal Conference on 3D Dgtal Imagng and Modelng, pages , May 21. [17] G. Turk and M. Levoy. Zppered polygon meshes from range mages. SIGGRAPH 94 Conference Proceedngs, 26: , [18] J. Wllams and M. Bennamoun. Smultaneous regstraton of multple correspondng pont sets. Computer Vson and Image Understandng, 81(1): , January 21. 7

8 T 3 R-I Iteraton number () (a) Iteraton number () (b) T 3 R-I Iteraton number () (c) Iteraton number () (d) Fgure 6. Change of transformaton parameters as a functon of the number of teratons. Top row: Change of translaton (a) and rotaton (b) for 12 noseless mages of Beethoven. Bottom row: Change of translaton (c) and rotaton (d) for 12 mages corrupted by nose of varance 1 voxel. (a) (b) (c) Fgure 7. Example of flterng. (a) Reconstructon from a sngle mage. (b) Reconstructon from 1 regstered and averaged mages (left sde). (c) Reconstructon from 5 regstered and averaged mages (left sde). 8

9 Intal mages Regstered mages Reconstructed model Fgure 8. Examples of reconstructon and regstraton. Top row: real range data from Stanford mage repostory. Mddle row: regstraton and reconstructon usng noseless synthetc data. Bottom row: regstraton and reconstructon usng nosy mages (nose varance 1 voxel). 9