Range Image Registration with Edge Detection in Spherical Coordinates

Range Image Registration with Edge Detection in Spherical Coordinates Olcay Sertel 1 and Cem Ünsalan2 Computer Vision Research Laboratory 1 Department of Computer Engineering 2 Department of Electrical and Electronics Engineering Yeditepe University, Istanbul 34755, Turkey unsalan@yeditepe.edu.tr Abstract. In this study, we focus on model reconstruction for 3D objects using range images. We propose a crude range image alignment method to overcome the initial estimation problem of the iterative closest point (ICP) algorithm using edge points of range images. Different from previous edge detection methods, we first obtain a function representation of the range image in spherical coordinates. This representation allows detecting smooth edges on the object surface easily by a zero crossing edge detector. We use ICP on these edges to align patches in a crude manner. Then, we apply ICP to the whole point set and obtain the final alignment. This dual operation is performed extremely fast compared to directly aligning the point sets. We also obtain the edges of the 3D object model while registering it. These edge points may be of use in 3D object recognition and classification. 1 Introduction Advances in modern range scanning technologies and integration methods allow us to obtain detailed 3D models of real world objects. These 3D models are widely used in reverse engineering to modify an existing design. They help determining the geometric integrity of manufactured parts and measuring their precise dimensions. They are also valuable tools for many computer graphics and virtual reality applications. Using either a laser scanner or a structured light based range scanner, we can obtain partial range images of an object from different viewpoints. Registering these partial range images and obtaining a final 3D model is an important problem in computer vision. One of the state-of-art algorithms for registering these range images is the iterative closest point (ICP) algorithm [1,2]. One of the main problems of ICP is the need for a reliable initial estimation to avoid convergence to a local minima. Also, ICP has a heavy computational load. Many researchers proposed variants of ICP to overcome these problems [3,4,5,6]. Jiang and Bunke [7] proposed an edge detection algorithm based on a scan line approximation method which is fairly complex. They mention that, B. Gunsel et al. (Eds.): MRSC 26, LNCS 415, pp. 745 752, 26. c Springer-Verlag Berlin Heidelberg 26

746 O. Sertel and C. Ünsalan detecting smooth edges in range images is the hardest part. Sappa et. al. [8] introduced another edge based range image registration method. Specht et. al. [9] compared edge and mesh based registration techniques. In this study, we propose a registration method based on edge detection. Our difference is the way we obtain the edge points from the range image set. Before applying edge detection, we apply a coordinate conversion (from cartesian to spherical coordinates). This conversion allows us to detect smooth edges easily. Therefore, we can detect edges from free-form objects. These edges help us in crudely registering patches of free-form objects. Besides, we also obtain the edge information of the registered 3D object which can be used for recognition. In order to explain our range image registration method, we start with our edge detection procedure. Then, we focus on applying ICP on the edge points obtained from different patches (to be registered). We test our method on nine different free-form objects and provide their registration results. We also compare our edge based registration method with ICP. Finally, we conclude the paper with analyzing our results and providing a plan for future study. 2 Edge Detection on Range Images Most of the commercial range scanners provide the point set of the 3D object in cartesian coordinates. Cartesian coordinates is not a good choice for detecting smooth edges in range images. We will provide an example to show this problem. Therefore, our edge detection method starts with changing the coordinate system. 2.1 Why Do We Need a Change in the Coordinate System? Researchers have focused on cartesian coordinate representations for detecting edges on 3D surfaces. Unfortunately, applying edge detection in cartesian coordinates do not provide acceptable results as shown in Fig. 2 (a) (to be discussed in detail next). The main reason for this poor performance is that, most edge detectors are designed for step edges in gray-scale images (it is assumed that, these step edges correspond to boundaries of objects in the image) [1]. In range images (3D surfaces), we do not have clear step edges corresponding to the actual edges of the object. For most objects, we have smooth transitions not resembling a step edge. Therefore, applying edge detection on these surfaces do not provide good results. To overcome this problem, we hypothesize that representing the same object surface in spherical coordinates increases the detectability of the object edges. Therefore applying edge detection on this new representation provides improved results. As we detect edges in the spherical representation, we can obtain the cartesian coordinates of the edges and project them back to actual 3D surface to obtain the edge points on the actual surface. Let s start with a simple example to test our hypothesis. We assume a slice of a generic 3D object (at z = ) for demonstration purposes. We can represent the point set at this slice by a parametric space curve in Fig. 1 (a). As can be seen,

Range Image Registration with Edge Detection in Spherical Coordinates 747 the curve is composed of two parts. However, applying edge detection directly on this representation will not give good results, since we do not have step edge like transition between those curve parts. We also plot the spherical coordinate representation of the same curve in Fig. 1 (b). We observe that the change in the curve characteristics is more emphasized (similar to a step edge) in spherical coordinates. This edge can easily be detected by an edge detector. Now, we can explore our hypothesis further for range images. 1.4 1.2 1.3 1.25 y 1.8.6.4.2.8.6.4.2.2.4.6.8 x (a) c(t) in cartesian coordinates R(θ) 1.2 1.15 1.1 1.5 1.95.5 1 1.5 2 2.5 3 3.5 θ (b) r(θ) in spherical coordinates Fig. 1. A simple example emphasizing the effect of changing the coordinate system on detecting edges 2.2 A Function Representation in Spherical Coordinates for Range Images In practical applications, we use either a laser range sensor or a structured light scanner to obtain the range image of an object. Both systems provide a depth map for each coordinate position as z = f(x, y). Our aim is to represent the same point set in spherical coordinates. Since we have a function representation in cartesian coordinates, by selecting a suitable center point, (x c,y c,z c ), we can obtain the corresponding function representation R(θ, φ), in terms of pan (θ) and tilt (φ) anglesas: R(θ, φ) = (x x c ) 2 +(y y c ) 2 +(z z c ) 2 (1) where ( ( ) ( )) y yc (x xc ) (θ, φ) = arctan, arctan 2 +(y y c ) 2 (2) x x c z z c This conversion may not be applicable for all range images in general. However, for modeling applications, in which there is one object in the scene, this conversion is valid. 2.3 Edge Detection on the R(θ, φ) Function As we apply cartesian to spherical coordinate transformation and obtain the R(θ, φ) function, we have similar step like changes corresponding to physically

748 O. Sertel and C. Ünsalan meaningful segments on the actual 3D object. In order to detect these step like changes, we tested different edge detectors on R(θ, φ) functions. Based on the quality of the final segmentations obtained on the 3D object, we picked Marr and Hildreth s [11] zero crossing edge detector. Zero crossing edge detector is basedonfilteringeachr(θ, φ) bythelogfilter: F (θ, φ) = 1 πσ 4 ( θ 2 + φ 2 2σ 2 1 ) exp ( θ2 + φ 2 ) 2σ 2 where σ is the scale (smoothing) parameter of the filter. This scale parameter can be adjusted to detect edges in different resolutions, such that a high σ value will lead to rough edges. Similarly, a low σ value will lead to detailed edges. To label edge locations from the LoG filter response, we extract zero crossings with high gradient magnitude. Our edge detection method has some desirable characteristics. If the object is rotated around its center of mass, the corresponding R(θ, φ) function will only translate. Therefore, the new edges obtained will be definitely same as in the original representation. We provide edge detection results for (one of the) single view of bird object in both cartesian and spherical coordinate representations in Fig. 2. (3) (a) Edges detected in cartesian coordinates (b) Edges detected in spherical coordinates Fig. 2. Edge detection results for a patch of the bird object As can be seen, edges detected from the spherical representation are more informative than the edges detected from the cartesian coordinate based initial representation. If we look more closely, the neck of the bird is detected both in cartesian and spherical representations. However, smooth transitions such as the eyelids, wings, the mouth, and part of the ear of the bird is detected only in spherical coordinates. These more representative edges will be of great use in the registration step. 3 Model Registration Using the ICP Algorithm The ICP algorithm can be explained as a cost minimization function in an iterative manner [1]. We use the ICP algorithm in two modes. In the first mode,

Range Image Registration with Edge Detection in Spherical Coordinates 749 we apply ICP on the edge points obtained by our method. This mode is fairly fast and corresponds to a crude registration. Then, we apply ICP again to the whole crudely registered data set to obtain the final fine registration. Applying this two mode registration procedure decreases the time needed for registration. It also leads to lower registration error compared to applying ICP alone from the beginning. We provide the crude registration result of the first and second bird patches in Fig. 3. As can be seen, the crude registration step using edge points works fairly well. Next, we compare our two step registration method with registration using ICP alone on several range images. (a) Edges after registration (b) Patches after registration Fig. 3. Crude registration of the first and second bird patches 4 Registration Results We first provide the final (crude and fine) registration results of two pair-wise range scans of the red dino and bird objects in Fig. 4. As can be seen, we have fairly good registration results on these patches. Next, we quantify our method s registration results and compare it with ICP. We first provide alignment errors for four pair of patches in Fig. 5. In all figures, we provide the alignment error of the ICP algorithm wrt. iteration number (in dashed lines). We also provide the alignment errors of our crude (using only edge points) and fine alignment (using all points after crude alignment) method in solid lines. We label the iteration step, we switch from crude to fine alignment, by a vertical crude, fine alignment line in these figures. As can be seen, in our alignment tests the ICP algorithm has an exponentially decreasing alignment error. Our crude alignment method performs similarly on all experiments while converging in fewer iterations. It can be seen that our crude alignment reaches almost the same final error value. In order to have better visual results, we need to apply the fine registration for a few more iterations. We provide the final registration results for the bird object from three different viewpoints, including the final edges in Fig. 6. As can be seen, all patches are perfectly aligned and form the final 3D representation of the object. The edge points obtained correspond meaningful locations on the final object model.

75 O. Sertel and C. Ünsalan (a) The 1. and 2. red dino patches (b) The 6. and 7. red dino patches (c) The 5. and 6. bird patches (d) The 17. and 18. bird patches Fig. 4. Final registration results for the pairs of red dino and bird patches. Each registered patch is labeled in different colors. 35 3 error for our method error for ICP alone crude,fine registration line 3 25 error for our method error for ICP alone crude,fine registration line alignment error 25 2 15 1 5 alignment error 2 15 1 5 1 2 3 4 5 6 7 8 iteration number (a) The 1. and 2. red dino patches 5 1 15 2 iteration number (b) The 6. and 7. red dino patches 1.6 1.4 1.2 error for out method error for ICP alone crude,fine registration line 1.4 1.2 1 error for our method error for ICP alone crude,fine registration line alignment error 1.8.6.4 alignment error.8.6.4.2.2 2 4 6 8 1 iteration number (c) The 5. and 6. bird patches 2 4 6 8 1 iteration number (d) The 17. and 18. bird patches Fig. 5. Comparison of alignment errors on four pairs of red dino and bird patches

Range Image Registration with Edge Detection in Spherical Coordinates 751 (a) view 1 (b) view 2 (c) view3 Fig. 6. The final registration of all bird patches with edge points labeled Finally, we compare the total iteration times to register all the pathes of nine objects in Table. 1 in terms of CPU timings (in sec.). The numbers in parenthesis represents the number of total scans of that object. In this table, the second column (labeled as ICP alone) corresponds to constructing the model (from all patches) using ICP alone. The third and fourth columns correspond to crude and fine registration steps of our method, (labeled as Crude registration and Fine registration respectively). The timings in the third column include the edge detection and coordinate conversion steps. The fifth column indicates to the total time needed for our method for registration (labeled as Crude + Fine reg.). The last column corresponds to the gain if we switch from ICP alone to our two mode registration method. While performing registration tests, we used a PC with an AMD Athlon CPU with 35 MHz. clock speed, with 2 GB RAM. Object Table 1. Comparison of the CPU timings (in sec.) over nine objects ICP alone Crude registration Fine registration Crude + Fine reg. Gain red dino (1) 862.42 7.46 174.22 181.68 4.75 bird (18) 1185.94 11.77 241.37 253.14 4.68 frog (18) 216.51 17.22 242.55 259.77 8.11 duck (18) 3292.84 27.27 646.33 673.61 4.89 angel (18) 2298.81 3.59 91.2 94.61 2.44 blue dino (36) 478.4 26.31 145.12 171.43 4.46 bunny (18) 735.51 9.44 161.5 17.95 4.3 doughboy (18) 1483.77 6.77 233.38 24.15 6.18 lobster (18) 2964.7 19.18 586.66 65.85 4.89 Average 219.6 17.33 471.24 488.57 4.48 As can be seen in Table 1, on the average we have a gain of 4.48 over nine objects. At the end, for both methods we obtain the same or similar registration errors. If we can tolerate crude alignment for any application, our gain becomes 126.37. We should also stress that, we also obtain the edge information of the 3D model constructed as a byproduct of our method. This edge information can be used to solve classification and matching problems.

752 O. Sertel and C. Ünsalan 5 Conclusions We introduced an edge based ICP algorithm in this study. Our method differs from the existing ones, in terms of the edge extraction procedure we apply. Our edge detection method allows us detecting smooth edges on object patches. Our method not only registers object patches, it also provides the edge points of the registered patches, hence the 3D model constructed. These edge points may be of use in 3D object recognition and classification. Acknowledgements We would like to thank Prof. Patrick J. Flynn for providing the range images. References 1. Besl, P.J., McKay, D.N.: A method for registration of 3-d shapes. IEEE Trans. on PAMI 14 (1992) 239 256 2. Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. International Journal of Computer Vision 13 (1994) 119 152 3. Turk, G., Levoy, M.: Zippered polygon meshes from range images. Proceedings of SIGGRAPH (1994) 311 318 4. Soucy, M., Laurendeau, D.: A general surface approach to the integration of a set of range views. IEEE Trans. on PAMI 17 (1995) 344 358 5. Liu, Y.: Improving ICP with easy implementation for free-form surface matching. Pattern Recognition 37 (23) 211 226 6. Lee, B., Kim, C., Park, R.: An orientation reliability matrix for the iterative closest point algorithm. IEEE Trans. on PAMI 22 (2) 125 128 7. Jiang, X., Bunke, H.: Edge detection in range images based on scan line approximation. Computer Vision and Image Understanding 73 (1999) 183 199 8. Sappa, A.D., Specht, A.R., Devy, M.: Range image registration by using an edgebased representation. In: Proc. Int. Symp. Intelligent Robotic Systems. (21) 167 176 9. Specht, A.R., Sappa, A.D., Devy, M.: Edge registration versus triangular mesh registration, a comparative study. Signal Processing: Image Communication 2 (25) 853 868 1. Sonka, M., Hlavac, V., Boyle, R.: Image Processing, Analysis and Machine Vision. 2. edn. PWS Publications (1999) 11. Marr, D., Hildreth, E.C.: Theory of edge detection. Proceedings of the Royal Society of London. Series B, Biological Sciences B-27 (198) 187 217