An accurate and fast point-to-plane registration technique

Transcription

1 Pattern Recognition Letters 24 (23) An accurate and fast oint-to-lane registration technique Soon-Yong Park *, Murali Subbarao Deartment of Electrical and Comuter Engineering, College of Engineering and Alied Science, State University of New York at Stony Brook, Stony Brook, NY , USA Received 29 January 23; received in revised form July 23 Abstract This aer addresses a registration refinement roblem and resents an accurate and fast oint-to-(tangent) lane technique. Point-to-lane aroach is known to be very accurate for registration refinement of artial 3D surfaces. However, the comutation comlexity for finding the intersection oint on a destination surface from a source control oint is hindering the algorithm from real-time alications. We introduce a novel oint-to-lane registration technique by combining the high-seed advantage of oint-to-rojection technique. In order to find the intersection oint fast and accurately, we forward-roject the source oint to the destination surface and reroject the rojection oint to the normal vector of the source oint. We show that iterative rojections of the rojected destination oint to the normal vector converge to the intersection oint. By assuming the destination surface to be a monotonic function in a new 2D coordinate system, we show contraction maing roerties of our iterative rojection technique. Exerimental results for several objects are resented for both air-wise and multi-view registrations. Ó 23 Elsevier B.V. All rights reserved. Keywords: Range registration; oint-to-lane; Contractive rojection oint. Introduction Registration refinement of multile range images is an essential ste in multi-view 3D modeling. When the range images are coarsely registered by a riori knowledge of registration arameters, registration refines the rigid transformation arameters to minimize alignment error between overlaing range surfaces. There have been many * Corresonding author. Tel.: ; fax: addresses: arksy@ece.sunysb.edu (S.-Y. Park), murali@ece.sunysb.edu (M. Subbarao). investigations on the refinement roblem (Cambell and Flynn, 2; Rusinkiewicz and Levoy, 2). For a air-wise registration, the registration roblem can be considered to be an error minimization of the rigid body transformation between control oint sets (Arun et al., 987) on the air of surfaces. Control oint sets are decided by matching either geometric (Besl and McKay, 992), hotometric (Lensch et al., 2; Weik, 997), or geometric and hotometric (Bernadini et al., 2; Johnson and Kang, 997) structures of the range surfaces. In this aer, we address a registration roblem in the first category, which considers only geometric structure for control oint matching /$ - see front matter Ó 23 Elsevier B.V. All rights reserved. doi:.6/s (3)57-

2 2968 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) Based on the method of control oint matching, three aroaches can be considered in general. Point-to-oint, e.g. iterative closest oint (ICP) algorithm (Besl and McKay, 992), oint-to-(tangent) lane (Chen and Medioni, 992; Bergevin et al., 996), and oint-to-rojection (Blais and Levine, 995) techniques are well-known registration techniques. The ICP algorithm is one of the common techniques for refinement of artial 3D surfaces (or models) and many variant techniques have been investigated. However, searching the closest oint in the ICP algorithm is a comutationally exensive task. In order to accelerate the seed of closest oint searching, some searching techniques are commonly emloyed, for examle a kd-tree searching, z-buffering, or a closest-oint caching (Nishino and Ikeuchi, 22; Benjemaa and Schmitt, 999). In contrast, oint-to-rojection aroach finds the corresondence of a source control oint by rojecting the source oint onto a destination surface from the oint of view of the destination (Blais and Levine, 995; Neugebauer, 997). When the source oint is backward rojected to the image lane of the destination surface, the destination control oint is the forward rojection of the image oint. This aroach makes registration very fast, because it does not involve any searching ste to find the corresondence. However, one of its disadvantages is that the result of registration is not as accurate as those of the others (Rusinkiewicz and Levoy, 2). Among three aroaches, oint-to-(tangent)- lane technique is known to be the most accurate (Pulli, 999; Rusinkiewicz and Levoy, 2). From a source control oint, the matching control oint is the rojection of the source oint onto the tangent lane at a destination surface oint which is the intersection of the normal vector of the source oint (Bergevin et al., 996; Chen and Medioni, 992; Gagnon et al., 994). However, finding the intersection on the destination surface is also comutationally exensive. One of the acceleration techniques is first searching the closest oint, and finding the intersecting surface (or the triangle) from its neighboring triangles (Rusinkiewicz and Levoy, 2). Gagnon (Gagnon et al., 994) tries to find the intersection on a 2D grid of range images along the rojection of the normal vector of the source oint. In this aer, we address the registration roblem to refine a air of range surfaces as well as multi-view range surfaces. We roose an accurate and fast oint-to-lane registration technique. We combine advantages of oint-to-lane and ointto-rojection techniques for fast control oint searching. We emloy the accuracy from the ointto-lane technique and the seed from the ointto-rojection technique. In order to find the intersecting control oint, we roject a source oint to the destination surface, re-roject the rojection oint to the normal vector of the source oint. We show that iterative normal rojections converge to the intersection oint. By assuming the destination surface as a monotonic function in a new 2D coordinate system, we show a contraction maing roerty of our registration technique (Naylor and Sell, 982). Therefore, we call our technique as contractive rojection oint (CPP) algorithm. Exerimental results for several 3D models are resented for many air registrations as well as a multi-view registration. 2. Descrition of registration techniques In this section, we briefly describe the basic rincile of three registration refinement techniques. Suose there is a control oint set P on a source surface S P, and another control oint set Q on a destination surface S Q.IfwehaveK control oints on each surface and k ¼ ;...; K, then the registration roblem is estimating a rigid body transformation T ¼½RjtŠ which minimizes an alignment error measure such that ¼ XK k¼ kq k ðrp k þ tþk 2 : ðþ In general, source control oint set P is selected by samling the source surface-randomly or uniformly, and filtered by some constraints to delete unreliable control oints. The destination control oint set Q is the conjugate of the source oint set, which is determined by a matching criterion. Point-to-oint technique (or ICP technique) is the most common technique. From a source con-

3 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) trol oint on the source surface, the ICP algorithm searches the closest oint q on the destination surface. Fig. (a) shows a basic diagram of the ICP algorithm. An error metric d s is the distance between two control oints. In order to search the closest oint, a searching technique, e.g. kd-tree, is commonly used. However, when a multi-view registration concerned, we need to rebuild kd-trees of the multile surfaces at every iteration of registration. Point-to-lane registration is another common technique. It searches the intersection on the destination surface from the normal vector of the source oint. As shown in the Fig. (b), the destination control oint q is the rojection of onto the tangent lane at q which is the intersection from the normal of. One of the revious investigations is done by Chen and Medioni (Chen and Medioni, 992). To find control oints, they use a root searching technique similar to Newton Rahson technique in an orthograhic coordinate system. Bergevin et al. (996) also use a similar technique with that of Chen and Medioni. They emloy an image-based control oint searching technique to reduce searching time. Point-to-rojection aroach is known to be a fast registration technique. As shown in the Fig. (c), this aroach determines a oint q which is the conjugate of a source oint, by forwardrojecting from the oint of view of the destination O Q. In order to determine the rojection oint, is first backward-rojected to a 2D oint Q on the range image lane of the destination surface, and then Q is forward-rojected to the destination surface to get q. This algorithm is very fast because it does not include any searching ste to find the corresondence. However, one of its disadvantages is that the result of registration is not as accurate as those of the others (Rusinkiewicz and Levoy, 2). 3. Contractive rojection oint (CPP) technique 3.. Combining oint-to-lane and oint-to-rojection techniques In this section, we roose a novel oint-tolane registration technique by combining the fast searching roerty of the oint-to-rojection technique. Suose there are two artial surfaces S P and S Q as shown in Fig. 2. They are assumed to be acquired from two different views P and Q of an object and coarsely registered to a common (world) coordinate system through transformation matrices T P and T Q, resectively. Let us also suose there is a source control oint on S P. Then the roblem in the oint-to-lane registration is finding the intersection oint q s on S Q as shown in the figure. The oint q s is an intersection on S Q by the normal vector ^ of. One of the tyical methods of searching the intersection is first finding a triangle (when the surface consists of triangles) which is intersected by ^. Then the intersection oint q s can be interolated by three vertices on the triangle. However, this is a comutationally exensive task and it may take several minutes to find all conjugates of S Q q s q s ' q q d s q' q d s q d s O P Tangent lane S P V Q q (a) (b) (c) Fig.. Three common registration techniques: (a) oint-tooint, (b) oint-to-lane, (c) oint-to-rojection. O Q I Q Fig. 2. Finding the intersecting oint q s from by the roosed algorithm.

4 297 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) hundreds of control oints. On the contrary, the oint-to-rojection technique can find a matching destination oint from a source oint within a millisecond on a tyical ersonal comuter, because it directly emloys the arametric surface of the destination range image. We consider a new oint-to-lane registration technique by emloying advantages from both techniques. The main idea is using iterative ointto-rojections to determine the intersection on the destination surface. Let us back-roject in Fig. 2 to a 2D image oint q ¼ M Q T Q ; ð2þ where M Q is the ersective rojection matrix of view Q to the image lane I Q, and T Q is the transformation matrix from the camera coordinate system of view Q to the world coordinate system. Then let us forward-roject q to a new 3D oint q. Forward-rojection q is the range of the arametric surface at the destination image oint q. The oint q is comuted by interolating a grid of destination image lane and transformed back to the world coordinate system. Now let us consider another rojection of q to the normal vector ^ at. Then we obtain a new 3D oint, such that a ¼ðq Þ^; ð3þ ¼ þ a^: If we iterate the same rojections above using the new control oint, then we get the next source oint 2, and so on. If there is an intersection on S Q by ^, then a oint q i (or i ) for the ith rojection will converge to q s when i goes to infinity, such that lim k i q i k!: ð4þ i! However in real situation, only small number of rojections can make a convergence measure c ¼k i q i k ð5þ become close to zero. If we find all convergence oints fq s g for all corresonding source f g,we can find all rojection oints fq sg on tangent lanes at fq s g and use them as control oint sets in the oint-to-lane registration Contraction maing roerty of CPP In order to show the validity of the roosed algorithm, we show contraction maing roerties of the algorithm. When a source control oint converges to an intersection, the roosed technique shows contraction maing roerties such that the convergence measure c becomes close to zero after a coule of normal rojections. The definition of the contraction maing is as follows. Definition 3.. Let ðx; dþ be a metric sace and f : X! X. We say a contraction maing, if there is a real number k; 6 k <, such that dðf ðxþ; f ðyþþ 6 kdðx; yþ for all x and y in X. Let us consider a 2D coordinate system as shown in Fig. 3. When there are two different view oints P and Q, the new axes consist of the viewing vector ^V Q of Q and the normal vector ^ of the source control oint. And the oint becomes the origin of the coordinate system. Now the surface S Q becomes a contour on the 2D lane, which is the intersection of the surface with the 2D lane. As shown in the figure, the roosed algorithm rojects i to a new oint iþ, iteratively until the convergence measure in Eq. (5) becomes very small. Practically seaking, distance from to the contour is very small, because two surfaces are assumed to be registered coarsely at the beginning. Therefore we can consider that all forward-rojecting lines from the view origin of Q to all control oint i are almost arallel. Then the following q s n q Fig. 3. Contraction maing roerty of a new 2D coordinate system. S Q V Q

5 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) theorem shows contraction maing roerties of the roosed algorithm. Theorem 3. () A surface S Q is a monotonic function with resect to ^v Q. (2) A vector roduct ^v Q ^ <, because we only consider a source oint which is seen from the view Q. Therefore there is always a rojection of q i to ^. (3) If there is an intersection q s, a function f : i! iþ (or q i! q iþ ) is a contraction maing when k iþ ; iþ2 k < kk i ; iþ k, where 6 k <. (4) Then, there is a 3D oint q s on the surface such that f ðq s Þ¼q s Convergence condition In the ideal situation, the CPP algorithm always converges to a convergence oint. However, in a real situation, it may diverge or enter to a nonconvergence cycle. Therefore we need to find out the convergence condition according to the contraction coefficient k. Three ossible cases are shown in the Fig. 4. In the Fig. 4(a), a source control oint is converging and the coefficient is 6 k <. In this case, we can use the convergence oint as a matching control oint of its conjugate. However, as shown in the Fig. 4(b), the control oint could diverge if the coefficient is k >. This could haen when the tangent normal at a oint on the surface S Q has a high angle with resect to ^V Q. However, we reduce the effect of diverging control oints by discarding a destination oint if the convergence measure c is greater than an initial measure kq k. The last case is when the maing enters into a non-convergence cycle and k ¼, as shown in the Fig. 4(c). This case could haen when a segment of the destination curve is overlaing with the rojection line from q i to iþ. We also remove this kind of oints by checking the error measure for 3 or 4 iterations. If a measured value reeats for 3 or 4 iterations, we consider the maing enters a non-convergence cycle and quit the maing CPP algorithm The roosed CPP algorithm can be easily imlemented by a recursive searching rogram. The following function is a seudo code of art of the algorithm. The function searches a 3D oint q, the intersection of a control oint on the destination surface Dest. N c is the maximum normal rojections and D c is threshold for the convergence error c. Vec3d Recursive ProjectionðVec3d& ;Vec3d&^; Vec3d&q ;Mesh Dest;int cntþ { if (cnt > N c ) return ; q ¼ M q ; flag ¼ OnObjectð q ; ; Dest; q Þ; if (! flag) return NULL; a ¼ðq Þ^; 2 ¼ þ a^; c ¼k 2 q k; if ( c < D c) return 2 ; return Recursive Projectionð 2 ; ^; q ; Dest; cnt þ Þ; } S Q S Q S Q V V (a) (b) (c) V Fig. 4. Three cases of rojection: (a) convergence ( 6 k < ), (b) divergence (k > ), (c) infinity loo (k ¼ ).

6 2972 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) After finding all matching control oints, we use two control oint sets f g and fq sg for estimating a rigid transformation T between two surfaces. The transformation is comuted by a SVD (Singular Value Decomosition) technique iteratively until two surfaces converge (Arun et al., 987). 4. Exerimental results 4.. Test objects We test our algorithm for three objects. The test objects are shown in the Fig. 5. The wave object is a air of synthetic 3D sinusoidal surfaces. Two surfaces have degree of hase difference in the XY range image lane and mm of translation along the Z axis. Their height is 5 mm and we add random noise to every image oint with maximum 2% of the height. The second object angel is a air of range images obtained from a laser range finder. This object is one of the models from the Ohio State UniversityÕs Range Image Database. The two images has 2 of rotation angle. For the third object otatohead in Fig. 5(c), we test both airwise and multi-view registration. We obtain eight range images of the object. Range images of the object have some erroneous oints on their surfaces. The number of oints and triangles on the test objects are shown in the Table. They are the number of oints and triangles on the surface of the first view, but the numbers of the others are also similar. Table Test objects Objects Wave Angel Potatohead Number of oints (view ) Number of triangles (view ) Convergence error P_i-Q_i (mm) rojection 2 rojections 5 rojections rojections 2 rojections Iterations Fig. 6. Comarison of convergence error with resect to different number of rojections. (wave object) Registration error with resect to rojection numbers In order to decide an aroriate number of normal rojections from q i to iþ, we aly the CPP technique to a noiseless wave object to lot convergence error c with resect to different numbers of rojections. In Fig. 6, the convergence error is lotted for 5 iterations of registration. At each iteration, RMS errors of all control oints are lotted, for which we use about 3 airs. As shown in the figure, the convergence error decreases if the number of rojections increases. However, its decrease rate is saturated after 5 or 6 rojections. Therefore, we use 5 normal rojections for all following exeriments Pair-wise registration Fig. 5. Point clouds models of test objects: (a) wave, (b) angel, (c) otatohead. We comare our algorithm with the oint-torojection and the ICP algorithm. We also use a kd-tree searching technique for ICP. We test all three techniques in the same registration condition on a Pentium III GHz ersonal comuter. As far as the wave object is concerned, we know the real conjugate oints between two range surfaces.

7 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) RMS registration error (mm) % noise 5% noise 7% noise 8% noise 9% noise % noise 2% noise (a) Iterations RMS registration error (mm) % noise 5% noise 7% noise 8% noise 9% noise % noise 2% noise (b) Iterations Fig. 7. Registration error of wave object: (a) CPP, (b) ICP. Therefore, we measure RMS distance error between two ground truth oint sets. For the other objects, we measure d s distance shown in the Fig.. However, the distance error in each aroach cannot be comared directly because they use different error metric. To fairly comare them, we normalize each error by using its initial error. Because the initial condition of all aroaches is Normalized RMS error of ds (mm) Proosed ICP with kd-tree Point-Projection (a) Iterations Normalized RMS error of ds (mm) Proosed ICP with kd-tree Point-Projection Original oint-lane (b) Iterations Fig. 8. RMS error results of (a) angel, (b) otatohead. the same each other, we normalize the RMS errors to watch the decrease rate of them. Registration results of CPP and ICP for wave object are lotted in Fig. 7. We lot the results for 5 iterations, and for different noise rate on the objectõs surface. Two algorithms converge well with low noise rate, however the ICP algorithm fails to converge when noise rate is more than 9%. The results of the other objects are shown in the Fig. 8. Fig. 8(a) shows the results for the object Table 2 Registration error and rocessing time after 5 iterations (N c ¼ 5, D c ¼ : mm, and d s is normalized) Method Point-rojection ICP with kd-tree Proosed CPP Wave with Ground truth (mm) % noise Time/Itr (ms) Wave with Ground truth (mm) % noise Time/Itr (ms) Angel d s (mm) Time/Itr (ms) Potatohead d s (mm) Time/Itr (ms)

8 2974 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) angel. The oint-to-rojection result shows a bad registration result. The roosed technique shows better convergence rate than other techniques. Fig. 8(b) is the results for otatohead. This result only shows the registration of the first air of images. In this figure, we also lot the result of original oint-to-lane technique, which use a brute force searching to find matching control oints. The result of the roosed technique is almost the same as that of the original technique. Table 2 shows the results of registration error and rocessing time after 5 iterations. As shown in the revious figure, our CPP technique has the smallest registration error. As far as the comutation time is concerned, the oint-to-rojection technique is faster than the other aroaches as we exected. The roosed technique and the ointto-oint technique show similar results but they deend on tyes of objects. This is because we need some comutations for searching intersection oints as shown in the CPP seudo code Number of convergence As described in an earlier section, we have three tyes of convergence Convergence, Divergence, and Infinity loo in searching of intersections. After some exeriments, we find that CPP searching converges well in most cases to intersections on destination surface. However, sometimes it diverges or enters to an infinity loo. Even though we have some number of non-convergence oints, we discard them and use only convergent control oints for deriving transformation matrix. The numbers of converging and non-converging oints are different for every iteration. However, Table 3 shows average numbers of them for 5 iterations on each object. Table 3 Average number of convergence and non-convergence control oints Object Convergence Divergence Infinity loo Wave (5% noise) Angel Potatohead Multi-view registration Multi-view registration is a more difficult roblem than a air-wise registration, because registration error should be evenly distributed on all overlaing surfaces. We regard the first view RMS error of ds (mm) view view2 view3 view4 view5 view6 view (a) Iteration RMS error of ds (mm) view view2 view3.5 view4 view5 view6 view (b) Iteration RMS error of ds (mm) view view2 view3 view4 view5 view6 view (c) Iteration Fig. 9. Results of multi-view registration for otatohead after 3 iterations. (a) Point-to-rojection, (b) ICP with kd-tree, (c) roosed (CPP).

9 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) Table 4 Multi-view registration error and rocessing time after 5 iterations. (N c ¼ 5, D c ¼ : mm) Method Point-rojection ICP with Proosed kd-tree RMS error d s (mm) Time (s) Time/Itr (s) of the otatohead object as a reference view and register all the other views to the reference view. In order to find the matching control oints from a source view, we search on its neighboring viewõs surfaces. For examle, if the ith view is concerned, we search the matching control oints on its neighborhoods, ði þ Þth view and ði Þth view. After finding all matching oints for every view oint, excet the first view, we comute rigid transformations and refine all surfaces simultaneously. We test a multi-view registration for the 8 views of the otatohead object. Fig. 9 lots their results. In this figure we do not normalize the error, because the results are obvious to comare. Both the oint-to-rojection and the ICP techniques show bad registration results. Both of them fail to register the multi-view images. In contrast, the CPP algorithm register all range images well. Table 4 shows registration error and rocessing time for this exeriment. The ICP with kd-tree technique take more time, because it needs to refine kd-trees of all surface (excet the reference view) for every iteration. Table 4 shows that our CPP technique has advantages over ICP and oint-to-rojection techniques for registration of multile range images. 5. Conclusions We address a registration refinement roblem and resent an accurate and fast oint-to-(tangent)lane registration technique. In order to find an intersection oint on a destination surface, we roject a source control oint to the destination surface, re-roject the rojection oint to the normal vector of the source oint. We show that iterative rojections of the rojected destination oint to the normal vector converge to the intersection oint. By assuming the destination surface to be a monotonic function in a new 2D coordinate system, we show contraction maing roerties of our contractive rojection oint technique. In an alternative technique, finding convergence oint could be imlemented by another method such as the Newton Rahson method. Instead of rojecting the destination control to the vector normal, the tangent vector at the destination oint can be used to find the intersection with the vector normal. Exerimental results show that our aroach is very accurate and fast for both air-wise registration and multi-view registration roblems. References Arun, K., Huang, T., Blostein, S., 987. Least-squares fitting of two 3D oint sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 9 (5). Benjemaa, R., Schmitt, F., 999. Fast global registration of 3D samled surface using a multi-z-buffer technique. Image and Vision Comuting 7, Bergevin, R., Soucy, M., Gagnon, H., Laurendeau, D., 996. Toward a general multi-view registration technique. IEEE Transactions on Pattern Analysis and Machine Intelligence 8 (5). Bernadini, F., Martin, I.M., Rushmeier, H., 2. High-quality texture reconstruction from multile scans. IEEE Transactions on Visualization and Comuter Grahics 7 (4), Besl, P.J., McKay, N.D., 992. A method of registration of 3D shaes. IEEE Transactions on Pattern Analysis and Machine Intelligence 4 (2), Blais, G., Levine, M., 995. Registering multiview range data to create 3D comuter object. IEEE Transactions on Pattern Analysis and Machine Intelligence 7 (8). Cambell, R.J., Flynn, P.J., 2. A survey of free-form object reresentation and recognition techniques. Comuter Vision and Image Understanding 8, Chen, Y., Medioni, G., 992. Object modeling by registration of multile range images. Image and Vision Comuting (3), Gagnon, H., Soucy, M., Bergevin, R., Laurendeau, D., 994. Registration of multile range views for automatic 3D model building. IEEE Comuter Vision and Pattern Recognition, Johnson, A., Kang, S., 997. Registration and integration of textured 3D data. IEEE Conference on Recent Advances in 3-D Digital Imaging and Modeling. Lensch, H., Heidrich, W., Seidel, H., 2. Automated texture registration and stitching for real world models. In:

10 2976 S.-Y. Park, M. Subbarao / Pattern Recognition Letters 24 (23) Proceedings of Pacific Grahics 2, IEEE Comuter Society Press, Naylor, A., Sell, G., 982. Linear oerator theory in engineering and science. Sringer-Verlag, New York. Neugebauer, P., 997. Geometrical cloning 3D objects via simultaneous registration of multiview range images. IEEE Conference on Shae Modeling and Alications, Nishino, K., Ikeuchi, K., 22. Robust simultaneous registration of multile range images. In: The 5th Asian Conference on Comuter Vision (ACCV22), Pulli, K., 999. Multiview registration for large data sets. In: Proceedings of 2nd International Conference on 3D Digital Imaging and Modeling. Rusinkiewicz, S., Levoy, M., 2. Efficient variants of the ICP algorithm, In: Proceedings of 3D Digital Imaging and Modeling, Weik, S., 997. Registration of 3-D artial surface models using luminance and deth information. In: Proceedings International Conference on Recent Advances in 3-D Digital Imaging and Modeling,. 93.