Depth Buffer Based Registration of Free-form Surfaces

Depth Buffer Based Registration of Free-form Surfaces Ulf Labsik, Roman Sturm, and Günther Greiner Computer Graphics Group, University of Erlangen Am Weichselgarten 9, 91058 Tennenlohe, Germany Email: labsik,sturm,greiner @informatik.uni-erlangen.de Abstract Many algorithms have to deal with multiple representations of the same object. The process of aligning these representations is called registration. In this paper we present a new approach for the registration of free-form surfaces which is not based on finding closest points. Instead, our algorithm takes advantage of the graphics hardware and uses the depth buffer for determining the distance between the surfaces. We show the efficiency of our approach in several examples in two different application areas, the registration of range images from the same object and the nominal value comparison for spherical lenses. 1 Introduction In order to construct a polygonal mesh of a complex object range images from different viewpoints have to be taken. To merge the range images to a single mesh they have to be transformed into the same coordinate system. The process of finding the rigid transformation is called registration. The process of finding this rigid transformation is normally split into two parts. First, a rough estimation has to be found which is used as an initial value for a subsequent optimization process. The defacto standard for the registration of polygonal surfaces is the Iterated Closest Point algorithm (ICP)[1, 3, 18, 5]. In this approach closest points are specified iteratively. Between such corresponding points the least square sum of the distances is computed and minimized in each iteration step. When started with a good initial estimation of the rigid transformation the ICP algorithm converges and allows a precise registration. A main drawback of ICP algorithms is the expensive task of finding the corresponding point on one mesh for a vertex of the other mesh. To avoid the task of explicitly finding corresponding points we follow a completely different approach for the registration of polygonal surfaces. After having specified a rough transformation we use the depth buffer images of overlapping regions of the meshes for determining a distance functional which has to be minimized. By using rendered images of polygonal meshes our algorithm takes advantages of the rapidly improving graphics hardware. For a further speed up of our algorithm for the registration of highly detailed polygonal surfaces we use a multiresolution approach based on mesh decimation[6, 7, 2]. As additional benefits the new algorithm can be easily used for the registration of arbitrary surface descriptions like Bézier patches or NURBS and is easy to implement. This paper is organized as follows. We present our new registration algorithm in Section 2, followed by discussing implementation details of the algorithm in Section 3. We have tested our algorithm in two different application areas, the registration of range images and the comparison of multifocal lenses, which are described in Section 4. We close with a summary in Section 5.

2 Registration As mentioned in the introduction registration of two data sets means to find a rigid transformation which maps one data set into the coordinate system of the other data set. The rigid transformation is defined by the vector a q i q j q k t x t y t z T. In the first three elements the rotation is stored, in the last three elements are the translation vector t. For encoding the rotation we use the unit quaternion which is a four vector q q 0 q 1 q 2 q 3 T with q 0 0 and q 2 0 q 2 1 q 2 2 q 2 3 1. The 3 3 rotation matrix R which is defined by a unit quaternion can be found in [1]. The task of registration can be defined as an optimization problem. To register two polygonal meshes 1 and 2 we define an appropriate functional f a and find a minimum for the value of this functional. Using an optimization algorithm normally leads to a local minimum of the functional. This means that in order to converge to the global minimum we have to find an initial guess for a which is close enough to that. This can be done either manual or by some algorithmic approach[14, 5, 16]. In this paper we focus on the construction and the minimization of the distance functional f a. As described in Section 3 we use a manual approach for the rough initial guess. 2.1 Algorithm In order to avoid the complicated task of finding corresponding points on both meshes which is inevitable for ICP algorithms we use a completely different approach. The basic idea of our algorithm is not to use the meshes directly for the registration process but a rendered image of the meshes. This approach has two major advantages. It utilizes the fast hardware of modern graphics systems and it enables the registration of not only polygonal meshes but also other surface representations like NURBS patches. In fact, we do not use the rendered image of the meshes in the frame buffer but the depth buffer images of the meshes. In the graphics pipeline the depth buffer is used for the hidden surface removal [17]. A depth value is stored for each pixel. This depth is measured as the distance to the viewer. In Figure 1 a depth buffer image of a rendered mesh is shown. Figure 1: Depth buffer image of the head of a Roman statue of Gnaeus Pompeius Magnus. The registration algorithm works as follows. Given are two 3D surfaces 1 and 2 in arbitrary positions. The task is to find a rigid transformation a which maps 2 into the coordinate system of 1. Therefore we first specify a camera position and a viewing frustrum so that a part of 1 is in the viewport. When rendering 1 we also get the depth buffer image of this scene. Now we have to find the optimal transformation a so that the rendered depth buffer image from a 2 has got a minimal distance to the depth buffer image of 1. Instead of thinking of transforming the mesh we could also think of finding a new camera position for viewing 2 which is illustrated in Figure 2. In the depth buffer the distance from the viewer to the object is stored as one float value per pixel. To measure the distance between the depth buffer images of two meshes 1 and 2 we compute the least square sum of the differences between the depth values of corresponding

2 1 q q i q j q k t x t y t z T Figure 2: The basic idea of the algorithm is to find a transformation for the camera position so that the depth buffer images of 1 and 2 have a minimal distance. pixels which is computed as d 1 2 z i 1 z i 2 2 i (1) Now the distance functional f a can be defined as f a d 1 a 2 (2) This functional can now be optimized to find a minimum for the distance of the depth buffer images. Aspects about the optimization will be explained in the next subsection. Up to now we do the registration by using one view of the object. This may be a problem when registering complex models. The algorithm may only converge to a local optimum for the rendered viewport whereas other parts of the meshes are not registered correctly. This problem in the behavior of the algorithm can be solved by allowing multiple views for the registration. The depth buffer images are now rendered for several views of the meshes and the distance functional is computed by summing up the distance values of each view. The complete registration algorithm can be summarized by the following algorithm. find rough transformation a and specify views render all specified views of mesh M 1 and save depth buffer images repeat render all specified views of mesh M 2 and compute f a find new transformation a by Powell algorithm until convergence 2.2 Optimization In our algorithm we have to optimize a functional f a whereas a describes the rigid transformation and is a six-dimensional vector. Also we cannot compute derivatives of this functional. Therefore we use Powell s method [10] which belongs to the class of direction set methods. As mentioned before the rigid transformation a consists of three values for the translation and three values for the rotation. In our application we encode rotations as a unit quaternion which is a four vector q q 0 q 1 q 2 q 3 T with q 2 0 q 2 1 q 2 2 q 2 3 1. Therefore it is enough to choose three out of this four values for the optimization steps, the fourth value can be reconstructed for the rendering of the object. We choose the three

elements of the quaternion q with the smallest absolute values. Now Powell s method can be applied with the functional f a. To optimize the value of f a, the optimization algorithm will evaluate the functional, and after evaluation change the values of a. This is done until the algorithm converges to a local minimum of the functional. 2.3 Hierarchical Registration Even with modern accelerated graphics hardware it is still time consuming to render meshes with more than 100k triangles. In our algorithm one mesh has to be rendered for every optimization step. Therefore rendering speed is very important for the efficiency of the registration. One possibility to improve the rendering speed of a mesh is to use mesh decimation [6, 7, 2]. The number of triangles in the mesh can be reduced dramatically by a mesh decimation algorithm and this coarse mesh can be rendered much faster. This leads to an hierarchical ap- proach for the registration algorithm. In an initial step different levels of detail 0 1 n 2 2 2 are produced for the second mesh 2. The registration of the meshes 1 and 2 now starts by using the coarsest level of detail n 2. This mesh can be rendered very fast but has a certain derivation from the original mesh. Hence, the optimization algorithm will not converge to the optimal transformation a but to an approximation value. After the optimization has converged we switch to the next finer level of detail and start the optimization with the approximation from the coarser level as start value to further improve the transformation a. This is repeated until the original resolution of the mesh is reached. 3 Implementation An optimal implementation of the presented registration algorithm is depended on the used graphics hardware. Therefore we only implemented a basic version only using standard OpenGL commands which are accessible on all OpenGL implementations. The time consuming part of the algorithm is the optimization process. Here for every evaluation of the distance functional one mesh has to be rendered n times, with n being the number of the specified views. We are only interested in the depth buffer image of the object. Therefore we do not need to render the object in the color buffer. After rendering the object into the depth buffer we copy the complete depth buffer into the main memory and can compute the distance functional with the help of the depth buffer images from the first mesh. They have to be rendered and stored only once at the beginning of the optimization process because this mesh will not be moved during the whole process. In our implementation we used Open Inventor as a high level graphics library. To achieve an optimal utilization of the depth buffer depth which is defined by the OpenGL implementation, the Inventor viewer normally adapts the front and back clipping plane of the viewing frustrum. This feature is unwanted for our algorithm and would lead to incorrect results. Therefore these automated clipping planes have to be switched off. The front and back clipping planes are chosen in a way that for all views of the mesh the whole mesh can be displayed and no parts are clipped. By using advanced features of the used graphics hardware the registration algorithm may be further accelerated. The use of p-buffers on an SGI O2 should be investigated. Also interesting would be an optimized version for PC graphics hardware like the NVIDIA GeForce, where it may even be possible to use the graphics hardware for evaluating the functional f a. 4 Applications In the this section we want to show some of the results we have achieved with our registration algorithm. We want to present two applications were we have two polygonal meshes or a polygonal mesh and a NURBS surface.

4.1 Registration of laser range images Laser range scanners are widely used for the construction of polygonal meshes of real objects. By one of the range images normally only parts of the object can be seen. This leads to undesired holes in the resulting mesh. To construct one complete mesh of a complex shape it is necessary to scan the object from multiple viewing directions and combine the resulting range images to one mesh. Therefore it is necessary to align these meshes. After that techniques for combing the registered range images can be applied [15, 4]. The example object here is the head of a Roman statue. As it can be seen in Figure 3 the first cylindrical scan of the object was not able to produce vertices on the top of the head. Therefore a second scan was necessary to achieve information in this area of the model. Figure 3: Two range images of the head of a Roman statue in arbitrary position. For using the presented registration algorithm the given triangular meshes have to be aligned roughly. This has to be done manually in our application. After this is done we can start the registration algorithm. The first task is to specify one or more views of the objects which shall be used for the registration. In this views both meshes must cover the whole viewport. For this example we specified the nose and a part of the hair of the object (cf. Fig. 4). Starting the registration the algorithm converges to a minimum of the defined distance functional. The time needed for the optimization depends on the mesh complexity of the surfaces. For our examples we reduced the head meshes to 39721 and the hair mesh to 19510 triangles. Figure 4: The two views used for the registration of the meshes. Tests have shown that the time needed for the registration not only depend on the number of triangles but also on the size of the viewport. With a viewport of 466 418 pixel the registration took 56s, shrinking the viewport to 263 191 pixel only 16s were needed. We also tested the accuracy of the algorithm by registering the head mesh with itself. For this registration the second mesh was slightly rotated and translated. By using four camera positions we nearly found the exact rigid transformation. The maximum distance from one vertex of the second mesh to the first mesh was below 0.004% of the bounding box size of the object. 4.2 Comparison of multifocal lenses We also tested our registration-software with data sets of multifocal lenses. These are ophthalmical lenses with a free-form front surface used to correct presbyopia. (In order to see objects at different distances sharply, the lens of the human eye is able to alter its curvature and thus its power of refraction. The ability to accommodate, i.e. to change the shape of this lens, decreases with age, due to gradual stiffening of the lens. This defect is called presbyopia.) Multifocal lenses usually have a large far vision area with low refractive power in the upper part, and a smaller near vision area with higher power in the lower part. Moreover, the distribution of refrac-

tive power (and thus of curvature) on the multifocal lens is quite smooth and monotonic (see Figure 5). For a detailed explanation of multifocal lenses and how to design such lenses with methods of geometric modeling and computer graphics, see [13, 9, 12]. Figure 6: Triangular mesh of a multifocal lens generated by 3D scanner - color coded is the discrete curvature evaluation. Figure 5: A typical multifocal lens - color coded is the refractive power. We have to deal with two types of data sets: Our software for designing optimal multifocal lenses outputs NURBS surfaces. Lenses are then manufactured according to this NURBS description. To quality control the manufacturing process, the front surface of randomly picked lenses is measured with an highly accurate tactile 3D measuring device (Primar MX4). The output of this 3D scanner are 3D coordinates, which we connect to triangular meshes. To ease the process of manually finding a first initial guess for a, we color code the surfaces regarding to their refractive power (mean curvature). Algorithms for computing curvatures from range data can be found in [8, 11]. When using our registration-software for matching a NURBS surface, i.e. output of the lens-design software, of a multifocal lens against itself, we have seen that a very rough first manual guess for a and one view a full view of the two lenses is sufficient to achieve a very satis- fying registration result. For matching a NURBS surface (output of lens-design software) against a triangular mesh obtained through 3D scanning a real lens, which was manufactured according to the NURBS description, one full view of the two datasets is still sufficient, but we have to (manually) find a good first guess for a. None the less the results of the registration process are good enough for quality controlling the manufactured lenses for form defects. (Other defects like scratches are already detectable through the curvature visualization.) 5 Conclusion In this paper we presented an efficient algorithm for the registration of polygonal meshes as well as all other kinds of surface representations like Bézier or NURBS patches. By using the depth buffer images of the surfaces our algorithm takes advantage of the rapidly growing performance of the graphics hardware. We also showed that combined with mesh decimation techniques for an hierarchical approach our algorithm can be further accelerated.

Figure 7: Rough manual registration and view used for registration. A drawback of the presented registration algorithm is the necessity to interactively specify the views of the overlapping parts of the objects. Future work should aim to an automatic detection of such views. By further implementing a strategy for the determination of the rough initial guess for the rigid transformation the algorithm could be fully automated. Acknowledgements We would like to thank Dr. Martin Boss from the Archaeological Institute for providing the Roman statue and Matthias Teschner from the Telecommunications Laboratory for scanning it. Also we would like to thank Dieter Geus from the Institute for Quality Management and Manufacturing Metrology for measuring the multifocal lenses. References [1] P. J. Besl and N. D. McKay. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and machine Intelligence, 14(2):239 258, February 1992. [2] S. Campagna. Polygonreduktion zur effizienten Speicherung, Übertragung und Darstellung komplexer polygonaler Modelle. Herbert Utz Verlag, 1998. [3] Y. Chen and G. Medioni. Object modelling by registration of multiple range images. International Journal of Image and Vision Computing, 10(3):145 155, April 1992. [4] Brian Curless and Marc Levoy. A volumetric method for building complex models from range images. In ACM Comp. Graph. (SIGGRAPH 96 Proc.), pages 303 312, 1996. [5] J. Feldmar and N. Ayache. Rigid, affine and locally affine registration of free-form surfaces. International Journal of Computer Vision, 18(2):99 119, 1996. [6] H. Hoppe. Progressive Meshes. In ACM Comp. Graph. (SIGGRAPH 96 Proc.), pages 99 108, 1996. [7] L. Kobbelt, S. Campagna, and H.-P. Seidel. A general framework for mesh decimation. In Proceedings of the Graphics Interface conference 98, 1998. [8] P. Krsek, G. Lukács, and R. R. Martin. Algorithms for computing curvatures from range data. In T. Goodman and R. Martin, editors, The Mathematics of Surfaces VIII, pages 1 16. IMA, 1998. [9] J. Loos. Konstruktion von Flächen mit vorgegebenen Krümmungseigenschaft-

en und Anwendungen in der Augenoptik. Dissertation, Universität Erlangen- Nürnberg, 1997. [10] W. H. Press, Teukolsky S. A., Vetterling W. T., and B. P. Flannery. Numerical recipes in C. Cambridge University Press, 2nd edition, 1992. [11] C. Rössel and L. Kobbelt. Approximation and visualization of discrete curvature on triangulated surfaces. In Vision, Modeling and Visualization 99, pages 339 346, 1999. [12] R. Sturm and G. Greiner. Wavefrontbased methods for the design of progressive additive lenses. Technical Report 29, University of Erlangen-Nuremberg, 1998. [13] M. Tazeroualti. Designing a progressive lens. Curves and surfaces in geometric design, pages 467 474, 1993. [14] J. P. Thirion. New feature points based on geometric invariants for 3d image registration. International Journal of Computer Vision, 18(2):121 137, 1996. [15] Greg Turk and Marc Levoy. Zippered polygon meshes from range images. In ACM Comp. Graph. (SIGGRAPH 94 Proc.), pages 311 318, 1994. [16] Wu Wang and S. S. Iyengar. Efficient data structures for model-based 3-d object recognition and localization from range images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(10):238 249, 1992. [17] M. Woo, J. Neider, T. Davis, and D. Shreiner. OpenGL Programming Guide. Addison Wesley, third edition, 1999. [18] Zhengyou Zhang. Iterative point matching for registration of free-form curves and surfaces. International Journal of Computer Vision, 13(1):119 152, 1994.