Surface Reconstruction with Higher-Order Smoothness

Transcription

1 Surface Recontruction with Higher-Order Smoothne Rongjiang Pan, Vaclav Skala 2 School of Computer Science and Technology, Shandong Univerity, Jinan, China, 25000, panrj@du.edu.cn 2 Centre of Computer Graphic and Data Viualization, Department of Computer Science and Engineering, Univerity of Wet Bohemia, Plzen, Czech Republic Abtract Thi work propoe a method to recontruct urface with higher-order moothne from noiy 3D meaurement. The recontructed urface i implicitly repreented by the zero level-et of a continuou valued embedding function. The key idea i to find a function whoe higher-order derivative are regularized and whoe gradient i bet aligned with a vector field defined by the input point et. In contrat to method baed on the firt-order variation of the function that are biaed toward the contant function and treat the extraction of the iourface without aliaing artifact a an afterthought, we impoe higher-order moothne directly on the embedding function. After olving a convex optimization problem with a multi-cale iterative cheme, a triangulated urface can be extracted uing the marching cube algorithm. We demontrated the propoed method on everal data et obtained from raw laer-canner and multi-view tereo approache. Experimental reult confirm that our approach allow u to recontruct mooth urface from point in the preence of noie, outlier, large miing part and very coare orientation information. Keyword: Surface recontruction, Higher-order moothne, Convex optimization Introduction Recontructing three-dimenional digital model from real world object i one of the major reearch topic in computer graphic a well a in computer viion. The majority of the developed geometric acquiition technique, uch a active and paive range ening, uually meaure a large number of 3D point. However, the dicrete point are not ueful for many practical application although point-baed geometry repreentation ha been propoed []. Thu, recontructing watertight urface from a et of pare point i becoming a common tep in the acquiition proce. The problem ha been reearched extenively and many technique have been developed over the pat two decade [2]-[]. However, urface recontruction remain a difficult and, in general, an ill-poed problem ince noie and outlier often contaminate the canned data. Moreover, due to inacceibility during canning and ome material propertie, there will be cae where point are miing or incomplete. To cope with mot of the deficiencie, energy-baed method, which combine the quality of fit to data with urface regularization, are particularly appropriate for robutly contructing urface from ampled point et. Recently, global optimization framework, e.g. graph-cut [2] and convex relaxation technique [3], have been applied to the urface fitting problem, where the urface are repreented implicitly by the binary-valued indicator function. The binary volume technique focu on egmenting a voxel a the interior or the exterior of the underlying hape. Once the function i computed, a triangulated urface model can be efficiently recovered uing an iourface extraction algorithm uch a marching cube [4]. Neverthele, the iourface often uffer from aliaing artifact and require pot-proceing to achieve mooth urface [5]. In thi paper, we propoe to impoe higher-order moothne directly on a continuou-valued embedding function. Moreover, intead of meauring the ditance between the urface and the given noiy data point, we wih to compute the function whoe gradient i bet aligned with an etimated coare normal field. A a reult, the urface recontruction problem i formulated a a convex optimization, whoe minimum yield higher quality urface. Computationally, the function i dicretized on a regular 3D grid and contructed by olving a large pare linear ytem uing a multi-cale iterative cheme. The paper i organized a follow. We give ome related work in the next ection. Section 3 preent our energy formulation and the implementation detail are provided in Section 4. In Section 5, we how ome experimental reult and a brief ummary i concluded in Section 6. 2 Related Work In the preence of noie and inhomogeneou ample denity, mot popular approache for urface recontruction fit continuou valued or characteritic (inide-outide) function to the input point et and then Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20

2 extract the recontructed urface a an appropriate iourface of thi function. The pioneered work by Hoppe [2] define the implicit function a the igned ditance to the tangent plane of the cloet input point. Signed ditance can alo be merged together uing an averaging proce into a volumetric grid function [3]. Since both method did not employ any urface regularization, they are prone to problem when the data contain large gap and outlier. A Markov Random Field (MRF) baed regularization i recently applied to the igned ditance field [6]. Several method define the implicit function a the weighted um of radial bai function (RBF) centered at ome point [6]-[9]. The fitting i done by olving a large linear ytem, where the trivial olution and the computational problem in practical olution ought to be overcome. An alternative approach combine everal local ditance function over an octree tructure uing a multilevel partition-of-unity [0]. The moving leat quare (MLS) method [30] locally approximate the urface with polynomial. Iue of thee method are the lack of robutne of the local approximation and the preence of puriou urface artifact. Poion urface recontruction [] compute a moothed indicator function (defined a at point inide the model and 0 at point outide) over an octree uing the Poion equation. The gradient for thi function approximate a vector field defined by the ample point. The method indirectly minimize the membrane energy of the clipped igned ditance field and thi i not alway optimal [6]. Mot of thee method are enitive to the accuracy of point orientation and varying ampling denity. To handle noiy, incomplete and uncertain data tatitical method are alo applied in the urface recontruction domain [3]. However, they uffer from the need of uer parameter and are relatively low. For robut urface fitting, dicrete graph-cut [2] or continuou convex relaxation [3] cheme have recently been ued where urface are repreented implicitly by the binary characteritic function. The method conider the problem of urface recontruction a a three-dimenional egmentation tak and employ total variation contraint. However, iourface extracted from binary egmentation of dicrete grid often exhibit aliaing artifact and require pot-proceing tep [5]. In implicit urface framework, a lack of accurate information about the urface orientation at the point ample i known to be a main challenge. The generation of the implicit function relie on a way to ditinguih between the inide and outide of the cloed urface. Variou method have been propoed to obtain orientation information, uch a etimating point normal uing local principal component analyi (PCA) [2], claifying pole of the Voronoi diagram of the input point [7], heuritically computing inide/outide contraint [8]. In the preence of noie or thin feature, the additional information i highly unreliable and often lead to an erroneou urface recontruction. Some approache try to recontruct a urface approximation from unoriented point et [9][20][2].Without orientation information, however, thee algorithm may lead to over-moothing urface [9], cannot fill large gap [20] or deal with large data et [7] [2]. 3 Our Approach 3. Problem Formulation Let S be a et of ampled data point lying on or near the urface M of an unknown three-dimenional model M. Each ample S conit of a point p and a weak etimate of global urface orientationn. We wih to contruct a continuou calar-valued function (), f p p Ω defined over a cloed and bounded domain 3 Ω R, whoe iourface i the bet fitting of the data point. A triangulated urface M % can then be recontructed by extracting the correponding iourface from the computed function f. A a mooth function f lead to a mooth iourface, we directly impoe the moothne on f o that the recontructed urface M % poee a certain degree of moothne. The idea i extenively ued in image egmentation [22][23]. 3.2 Meaure of Smoothne In order to meaure the energy or moothne of a function, we can define a norm on the olution pace: function with a mall norm are moother than thoe with a large norm. Membrane energy i ued in the Poion urface recontruction algorithm [] and contrained FEM recontruction [8], () = = Ω Ω x + y + z E f ( xyz,, ) dxdydz f ( xyz,, ) f ( xyz,, ) f ( xyz,, ) dxdydz 2 Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20

3 , where the ubcript denote differentiation and f i the Euclidean norm on L norm, which i often called total variation [22], 3 R. It i more common to ue the /2 = f xyz dxdydz= fx xyz+ fy xyz+ fz xyz dxdydz Ω Ω (2) E (,, ) (,, ) (,, ) (,, ) The method baed on ome function of the firt-order variation function, a they are the globally optimal function under thi meaure. f are biaed toward the contant To achieve higher-order moothne, we ugget uing the meaure that integrate the quared econd derivate: = f x y z + f x y z + f x y z dxdydz (3) E xx(,, ) yy(,, ) zz(,, ) Ω o that the globally minimal function are the polynomial of degree at mot three. Thu, the iourface include all planar urface and ome quadric urface. Alternatively, when the mixed derivative are included, the meaure f x y z f x y z f x y z f x y z f x y z f x y z dxdydz (4) E = xx(,, ) + yy(,, ) + zz(,, ) + 2 xy(,, ) + 2 xz(,, ) + 2 yz(,, ) Ω become rotationally invariant and biae the function toward a linear polynomial whoe iourface include all planar urface. 3.3 Data Fitting Accounting for the uncertaintie in the input data point, flux-baed functional are well jutified data fit meaure and are le enitive to the orientation error, a demontrated in [2]. Moreover, nearly all capture device can provide ome kind of point normal information. For example, direction toward the enor are often 3 known for data point. From the orientation for the data point, a vector field F : Ω R encoding the normal direction i etimated by a moothing filter. We then wih the gradient of the implicit function i bet aligned with the vector field. A reaonable data fitting meaure i the integration of the dot product between the gradient field f and the vector field F over the domain Ω : E d = < f, F > (5) Ω The more imilar between the two function in the domainω, the larger the meaure i. Uing the integration by part, we can derive the following equivalent data energy: where divf i the vector field divergence. 3.4 Energy Formulation Ed = f divf (6) Ω To obtain a global energy that can be minimized, the moothne and data energy are combined together, λ > E= λe -E (7) where the parameter 0 control how mooth the olution hould be and determine in ome ene the mallet feature that will be maintained in the recontructed urface. The functional in (7) i convex becaue each of the term i a convex one. The gradient and Laplacian in the firt one are linear operator and the econd one i linear. To make the global minima well defined, we can imply contrain the olution to lie in a fixed interval, e.g. [-,] for all p Ω. 4 Implementation We now decribe the implementation detail of the propoed approach. In order to find the minimum of the continuou problem, the function ()and f p the vector field F are dicretized on a regular 3D grid. For derivative and divergence operator, the judiciou election of the uniform pace diviion reult in imple dicrete form. d 3 Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20

4 Moreover, when a more advanced tructure i ued, e.g. the octree, area with no ample are repreented by very big cell and have low reolution in the final urface, a hown in Section Vector Field Etimation The vector field i computed uing a method imilar to the one decribed in [2]. Initially, the weak etimate of global urface orientation n at each ample S i ditributed to it eight nearet grid vertice a follow: n ( dx)( dy)( dz), n dx( dy)( dz), n ( dx) dy( dz), n ( dx)( dy) dz, n dx( dy) dz, n ( dx) dydz, n dxdy( dz), n dxdydz, where dx, dy and dz are the difference between the coordinate of point p and the mallet coordinate among the eight grid vertice divided by grid pacing h. In order to approximate a dene vector field { F( p) p Ω }, we mooth the vector field with a Gauian. For efficiency, we approximate the Gauian by the n -th convolution of a box filter with itelf: t < h Bt () = 0 otherwie F F x y Fz where h i the ize of grid cell and we chooe n = 3 in our implementation. Then, divf = + + x y z i approximated by tandard central difference where 4.2 Optimization F Fx Fy Fz T =( ). After dicretization, the correponding data energy in (6) become E d = [ f ( i, j, k) div F( i, j, k) ] (8) i, j, k and the derivative in (4) can be approximated a fxx 2 [ f ( i, j, k ) 2 f ( i, j, k ) + f ( i+, j, k )] h fyy 2 [ f ( i, j, k ) 2 f ( i, j, k ) + f ( i, j+, k )] h fzz 2 [ f ( i, j, k ) 2 f ( i, j, k ) + f ( i, j, k+ )] h fxy 2 [ f ( i+, j+, k ) f ( i+, j, k ) f ( i, j+, k ) + f ( i, j, k )] 4h (9) fxz 2 [ f ( i+, j, k+ ) f( i+, jk, ) f( i, jk, + ) + f( i, jk, )] 4h fyz 2 [ f ( i, j+, k+ ) f ( i, j+, k ) f ( i, j, k+ ) + f ( i, j, k )] 4h The total energy of the dicretized problem can be written a a quadratic form T T E= λe -E = λxax+ xb (0) d T where x = [ f(0,0,0) L f( i, j, k) L f( m, n, l )] and b = [div F(0,0,0) Ldiv F( i, j, k) L div F( m, n, l )] for the grid of reolution m n l and i = 0Lm, j = 0Ln, k = 0L l. Minimizing the quadratic form i equivalent to olving the pare linear ytem Ax = b () 2λ T 4 Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20

5 In our implementation, we ued the Gau-Seidel method for determining the olution of the linear ytem and tarted with an initialization x 0 = [0L 0] T. The iteration terminate when the change in ucceive iterate, k + k x x, reache the preciion given by the uer. 4.3 Multi-cale Solver Due to uniform pace diviion, a multi-cale olver i eay to implement. We ue three level of grid reolution. The divergence i computed at finet grid only once, and down-ampled by ummation. The reult computed on a coare grid are up-ampled a a good initialization on the next fine level. Therefore, the optimization may concentrate on a narrow band around the input point at fine level. 4.4 Mehing In order to recontruct a triangulated urface M %, it i neceary to elect an iovalue and extract the correponding iourface from the computed function f. The iovalue i elected a the average value of f at the ample poition: γ = f ( p) (2) S where f ( p ) denote the trilinear interpolation to the eight nearet grid vertice of. Finally, we extract the iourface uing an adaptation of the implementation code of Marching Cube publihed by Lewiner [24]. 5 Reult We validated our approach on a erie of experiment. The propoed method wa implemented in C++ and run on a notebook with 2.26GHz Core 2 Duo CPU and 2GB of RAM. In order to compare the behavior of the different energy model, we began with an artificial data et ued in [6]. The point are ampled randomly from a collection of primitive. Figure clearly how that the high order derivate in (3) and (4) reult in a moother urface than uing the firt-order variation. Table give the accuracy of the recontructed model hown in Figure in term of the root mean quare (RMS) of ditance from the input point to the nearet point on the urface. The moothne of the recontructed urface i etimated by computing the mean curvature and Gauian curvature uing the trimeh2 library [29]. S Figure : Surface recontruction on an artificial data et. The point are ued a input. From left to right: the reult uing the energy term in (), (2), (3) and (4). Average Maximum Average Maximum Energy model Triangle RMS Mean curvature Mean curvature Gauian curvature Gauian curvature () 49, (2) 49, (3) 5, (4) 5, Table : The energy model, the number of triangle in the recontructed model, the RMS of ditance from the input point to the nearet point on the urface, the average and maximum mean curvature, the average and maximum Gauian curvature. We alo proceed a number of range data et from the Stanford 3D Scanning Repoitory [25]. To demontrate the robutne of our method to orientation error, the regitered raw range can were treated a a collection of 3D point and a ingle orientation vector correponding to can viewing direction wa aigned to all 5 Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20

6 point in the ame can. In the following, the reult are baed on the energy term in (4). Figure 2 illutrate the effect of the parameter λ for the Bunny model. It i oberved that the parameter λ affect the fitne to the ample point and moothne of the urface. Large value of λ lead to increaed moothing. In our experiment, we got reaonable reult with λ between 0. and 0.3. Figure 2: Recontruction of the Bunny model at variou value of λ : 0. (left), 0.5 (middle),.0(right) We compared the reult of our method to the reult obtained uing Poion urface recontruction method [] and the MRF algorithm [6]. Surface recontructed with the three method for the Dragon model are hown in Figure 3. It can clearly be een that the Poion method (default parameter and depth=0) and the MRF method (type=point with approximate normal ) were unable to handle the original raw can with coare orientation etimate (one direction toward the canner per can), while our method produced an reaonable reult. (a) (b) (c) Figure 3: Comparion of three different algorithm. (a) The reult of our method. (b) The reult of the Poion urface recontruction algorithm and (c) the reult of the MRF algorithm. To tudy calability with large variation in ampling denity and ome outlier, we removed 98% of point from one-half of Armadillo and kept the outlier added by canning proce. Unlike [2] that ued non-uniform Euclidean regularization, our method wa able to handle the 50-to- difference in denity and tolerate outlier without uing any other information, a hown in Figure 4. Figure 4: Recontruction reult (right) of the Armadillo range can with 50-to- difference in denity (left). An input data et with everal large hole i given in Figure 5. The hole-filling capabilitie can be een from the recontructed urface. 6 Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20

7 Figure 5: Recontruction reult (right) of an input data et with large hole (left). The preented algorithm ha alo been teted on everal data et produced by multi-view tereo algorithm. After taking a collection of photograph with a digital camera, we etimated the camera parameter uing the tructure from motion oftware Bundler [26]. Then, a patch-baed multi-view tereo algorithm [27] can produce a et of oriented point covering the urface of the object. The oriented point are ued a input to our urface recontruction method and the recontructed urface i textured from the photograph uing a modified verion of the technique decribed in [28]. Figure 6 how a 3D recontruction of the Confuciu tatue from 40 photo. Becaue the tatue i about three meter high, ome part of the model are not imaged and the obtained point et ha everal large hole. Our method fill the hole in a plauible way. For a 30cm high tatue of the Godde of Mercy, the reult of our urface recontruction method from 45 photo i hown in Figure 7. (a) (b) (c) (d) Figure 6: Recontruction of the Confuciu tatue. (a) A photograph of the tatue. (b) The oriented point. (c) The recontructed urface and (d) a textured view. 7 Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20

8 (a) (b) (c) (d) Figure 7: Recontruction of the Godde of Mercy tatue. (a) A photograph of the tatue. (b) The oriented point. (c) The recontructed urface and (d) a textured view. 6 Concluion We have preented a method to recontruct urface with higher-order moothne from 3D meaurement. The method i robut to noie, large hole, non-uniform ampling denity and very coare orientation information. There are everal future work to purue. Becaue of dicretizing on a regular 3D grid, the problem become impractical for much fine-detailed recontruction. We intend to extend thi work by an adaptive tructure. Future work will alo include peed optimization uing parallel computing. Acknowledgment The author would like to expre their thank to Victor Lempitky and Michael Miha Kazhdan for fruitful dicuion. The author would alo like to thank Ramu R. Paulen, Noah Snavely and Yautaka Furukawa for making their algorithm publicly available. Thi work wa upported by the Key Project in the National Science & Technology Pillar Program of China (Grant No.2008BAH29B02), Shandong Natural Science Foundation of China (Grant No. ZR200FM046) and project of the Minitry of Education of the Czech Republic (No. 2C06002 and ME0060). Reference [] L. Kobbelt, M. Botch. A Survey of Point-Baed Technique in Computer Graphic. Computer & Graphic, 28(6): 80-84, [2] H. Hoppe, T. Deroe, T. Duchamp, J. McDonald, and W. Stuetzle. Surface recontruction from unorganized point. In ACM SIGGRAPH 92, page 7 78, 992. [3] B. Curle, M. Levoy. A volumetric method for building complex model from range image. In ACM SIGGRAPH 96, page , 996. [4] C. Bajaj, F. Bernardini, G. Xu. Automatic recontruction of urface and calar field from 3d can. In ACM SIGGRAPH 95, page 09 8, 995. [5] N. Amenta, M. Bern, and M. Kamvyeli. A new Voronoi-baed urface recontruction algorithm. In ACM SIGGRAPH 98, page 45-42,998. [6] J. C. Carr, R. K. Beaton, J. B. Cherrie, T. J. Mitchell, W. R. Fright, B. C. McCallum, and T. R. Evan. Recontruction and repreentation of 3d object with radial bai function. In ACM SIGGRAPH 200, paeg 67-76, 200. [7] B. S. More, T. S. Yoo, D. T. Chen, P. Rheringan, K. R. Subramanian. Interpolating implicit urface from cattered urface data uing compactly upported radial bai function. In Proceeding of the international Conference on Shape Modeling & Application, page 89-98, Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20

9 [8] Y. Ohtake, A. Belyaev, H. P. Seidel. A multi-cale approach to 3d cattered data interpolation with compactly upported bai function. In Proc. Intl. Conf. Shape Modeling 2003, page 53-6, [9] RongJiang Pan, XiangXu Meng and TaegKeun Whangbo. Hermite variational implicit urface recontruction. Science in China Serie F: Information Science, 52(2): ,2009. [0] Y. Ohtake, A. Belyaev, M. Alexa, G. Turk, and H. Seidel. Multi-level partition of unity implicit. In ACM SIGGRAPH 2003, page , [] M. Kazhdan, M. Bolitho, and H. Hoppe. Poion urface recontruction. In Proceeding of the Fourth Eurographic Sympoium on Geometry Proceing, page 6-70, [2] V. Lempitky and Y. Boykov. Global optimization for hape fitting. In CVPR 07, page: 8, [3] K. Kolev,M. Klodt, T.Brox, and D. Cremer. Continuou global optimization in multiview 3D recontruction. International Journal of Computer Viion, 84():80-96, [4] W.E. Lorenen and H.E. Cline. Marching Cube: A High Reolution 3D Surface Contruction Algorithm. In Proc. ACM SIGGRAPH 87, vol. 2, no. 4, pp , July 987. [5] V. Lempitky. Surface Extraction from Binary Volume with Higher-Order Smoothne. In IEEE Computer Viion and Pattern Recognition (CVPR), San Francico, 200. [6] Ramu R. Paulen, Jakob Andrea Bærentzen, Ramu Laren. Markov Random Field Surface Recontruction. IEEE Tranaction on Viualization and Computer Graphic, 6(4): , July/Aug [7] M. Samozino, M.Alexa, P.Alliez and M.Yvinec. Recontruction with Voronoi centered radial bai function. In Eurographic Sympoium on Geometry Proceing 2006, page 5-60, 2006 [8] A. Sharf, T. Lewiner, G.Shklarki, S. Toledo, and D. Cohen-Or. Interactive topology-aware urface recontruction. ACM Tran. Graph. 26(3):43-439, [9] A. Hornung and L. Kobbelt. Robut recontruction of watertight 3D model from non-uniformly ampled point cloud without normal information. In Proceeding of the Fourth Eurographic Sympoium on Geometry Proceing, page 4-50, [20] A. C. Jalba, B.T.M.Roerdink. Efficient urface recontruction uing generalized Coulomb potential. IEEE Tranaction on Viualization and Computer Graphic 3(6): 52-59, [2] P.Alliez, D. Cohen-Steiner, Y.Tong, and M. Debrun. Voronoi-baed variational recontruction of unoriented point et. In Proceeding of the Fifth Eurographic Sympoium on Geometry Proceing, page 39-48, [22] M. Nikolova, S. Eedoglu, and T. F. Chan. Algorithm for finding global minimizer of image egmentation and denoiing model. SIAM Journal of Applied Mathematic, 66(5): , [23] L. Grady. Random walk for image egmentation. IEEE Tran.Pattern Anal. Mach. Intell., 28(): , [24] T. Lewiner, H. Lope, A. Vieira, and G. Tavare. Efficient implementation of Marching Cube cae with topological guarantee. Journal of Graphic, Gpu, and Game Tool, 8(2):-5, [25] Stanford 3D Scanning Repoitory: [26] N. Snavely, S. M.Seitz, and R. Szeliki. Photo tourim: exploring photo collection in 3D. In ACM SIGGRAPH 2006, page ,2006 [27] Y. Furukawa, J. Ponce. Accurate, dene, and robut multi-view tereopi. In CVPR '07, page -8, [28] Ming Chuang, Linjie Luo, Benedict J. Brown, Szymon Ruinkiewicz, and Michael Kazhdan. Etimating the Laplace-Beltrami Operator by Retricting 3D Function. Proceeding of the Sympoium on Geometry Proceing. Page: , July [29] trimeh2: [30] M. Alexa, J. Behr, D. Cohen-Or, S. Fleihman, D. Levin, C.T. Silva. Point et urface. In Proc. of Viualization, page:2-28. IEEE Computer Society, 200. [3] Saleem W., Schall O., Patane' G., Belyaev A., Seidel H. On tochatic method for urface recontruction. The Viual Computer, 23 (6): Springer Berlin/Heidelberg, Surface Recontruction with higher-order moothne, The Viual Computer, on-line, ISSN , Springer, 20