Partial 3D-Object Retrieval Using Level Curves

Transcription

1 International Conference of Soft Computing and Pattern Recognition Partial 3D-Object Retrieval Using Level Curves Amine Mahiddine, Djamal Merad, Pierre Drap, Jean-marc Boï LSIS UMR CNRS 7296 Centre National de la Recherche Scientifique Marseille, France Abstract 3D object recognition from 3D scenes, is one of the challenges of several researchers in the field of computer vision, engineering and Robotics. The occlusion is one of the problems that we can found. One of the possible solutions in this situation is to find a part of an object in the scene that can be identified. For this reason, we are mainly interested to partial shape retrieval methods. In this paper, we present a new approach for 3D partial object retrieval based on level curves matching. Our approach can be used as an alternative solution for classification-based methods. We generate in the off-line step a dataset by using a viewing sphere to extract levels curves at different points of view. The level curves are a set of 2D planar contours that are the projection of points on several perpendicular planes. The level curves of each query partial object is compared with a set of level curves that define one 3D object from the dataset. The number of matched curves between a partial object and complete object represent the weight of that class. The class with the heavy weight is identify as the class of the query object. Keywords-3D Partial Matching; 2D partial curve matching; 3D-object recognition; 3D shape retrieval is still a research field in the exploration phase, knowing that until now it is difficult to find an automatic method which is efficient enough to identify a part of a 3D model within an occlusion. Many existing 3D shape retrieval methods requiring a complete surface model of a query object in order to find it in the dataset [1] [2] [3] [4] [5] and others seek to find a signature that will be invariant to rotation of the object [6] [7] [8]. We are mainly interested to partial shape retrieval methods and we refer the reader to a survey article by Liu [9]. In this work, we present a novel approach that deals with partial objects. The purpose is to find a partial object in a dataset that contains full 3D objects (point clouds). Our algorithm consists of creating the dataset phase, finding the correspondence between two curves. For creating the dataset, we take as input a set of 3D objects. Then, the level curves are produced by using a viewing sphere. In our experiments, we use one part per objects but we can take more. A 3D partial object is represented by level curves obtained from one viewpoint, each curve from this latter is compared with all curves in the dataset. Our matching algorithm used for comparison among curves is based on 2D planar curves alignment by using the intrinsic properties of curves which are the curvature and the arc-length. The properties are used by Sebastian et al. [10] for whole-towhole matching curves. So, the first contribution of this work is to use a viewing sphere to store the contours of the object from several viewpoints, in order to create a database with complete information about the 3D objects. The second contribution is to extract level curves that represent the contours of the object at various levels, in order to reduce the problem of matching between two 3D point clouds to 2D planar curves alignment which is widely discussed in the literature. The paper is organized as follows: Section I presents existing algorithms used for partial shape retrieval. In Section II, we describe in details our approach, creation of the dataset, curves extraction and the matching between curves. Section III covers experimental part and the paper concluded in Section IV. I. RELATED WORK An important literature has been provided for partial shape retrieval. We can find a number of different approaches that address this problem by using local descriptors-based methods or view-based methods. Local descriptors-based methods aim to extract the description (or signature) from the surface shape in the neighborhood of points whereas the methods based on view generate a set of 2D images of a 3D model from different points of view. Partial shape retrieval boils down to compare views. Johnson and Hebert [8] introduced the concept of spin images where they compute a 2D histogram of the 3D points projections on the cylindrical coordinates, Liu et al. [9] propose to use this signature with Monte-Carlo sampling on the surface model. Rusu et al. [11] propose FPFH (Fast Point Feature Histograms) an optimized method for real time use that characterizes the local geometry of a 3D point and stores the information in 16-bin histograms. Malassiotis et al. [12] extract a descriptor from snapshots of the surface over each point using a virtual camera oriented perpendicularly to the surface around the point. Cornea et al. [13] used the curve-skeleton of a 3D shape and compared between curves, by using Earth Mover s Distance [14] to evaluation the partial similarity. Sun et al. [15] generate a sequence of 2D planar contour by projecting the geodesic /14/$ IEEE 77

2 circles onto the tangent plane. In this work, we introduced a new approach for Partial 3D object retrieval from a database by using level curves. Level curves are 2D planar curves that are the projection of points onto the cutting planes. The originality of this work is to use a viewing sphere to store the contours of the object from several viewpoints in order to create a database with complete information about the 3D objects. Next, the extraction of level curves that present the contours of the object at various levels in order to reduce the problem of matching between two 3D point clouds to a comparison between 2D planar curves. II. APPROACH 3D object models are used in many applications, such as computer vision, computer graphics and Computer-aided design. We can find a large number of databases of 3D models on the Web such as: Stanford 3D Scanning Repository [16], Princeton Shape Benchmark Database [17], NTU 3D Model database [18] and 3D Keypoint Detection benchmark [7]. First of all the database for test must be created. In this work, we used some 3D models from [16][7] and by using viewing sphere on each 3D model (see Fig. 5), we create our dataset where 3D models are represented by level curves obtained from each viewpoint. After the establishment of the dataset, the level curves for all partial models are computed from one direction that best define those models (Section II-A). To find out if a partial model is part of an existing 3D model of our dataset, we have to match level curves of that partial model against all other in our dataset (Section II-B). A. Curves extraction As noted above, our approach is based on curve matching, the fact that we believe that the best descriptions of 3Dobjects are their forms, led us to think of the level curves. Those curves can be extract by cutting out point clouds (3D models) using a several planes with a regular step (see Fig. 3(b)). We implement in this work is an extension of the framework presented in [19] with a new graphic interface for users (see Fig. 2), where we can find two important functions. The first one is Cutting according to one direction, in this function the user have to select the direction of view using the viewer 3D, the threshold which corresponds to the smallest distance between a point and the cutting plane and the step which correspond to the distance between two successive planes. The second function is Cutting with several direction, the directions used here are taking within the viewing sphere where the vectors formed by each point on the sphere and the center of the 3D object represent one direction of view (see Fig. 5). We note that we have added a new parameter to this function that defines the number of viewing angles. A very low angle means a description almost full of the object, Figure 1. illustrative diagram of our approach however, it also means more curves for matching and large computation time. Figure 2. New graphic User Interface for 3D partial object retrieval process The curve obtained at one level is computed by projecting the points on the plane (level) if the distances between this 78

3 latter and the points are less than a threshold. At that time, the projected points present a set of unordered points. The steps to get an organize set of 2D points that present one curve are listed below. 1) Compute the 2D coordinates of the points relative to the plane. 2) Select one point randomly and find its nearest neighbor to get the first segment which will be added to the new list of points. 3) For the remaining points, compute the distance between the first point and the last on the list, if the first distance is less than the second, the new point is added to the top of the list, else it will be added to the end. Recall that in our approach, we used the curvature and the arc-length of the curve, we will define later how to better parameterized the curve for this use. Figure 4. Curve, (a) Point cloud of dragon head Figure 5. Viewing sphere, (red spheres) represent view directions each level, present a 2D planar curve which is parameterized in Cartesian coordinates as: Figure 3. (b) Level curves Level curves extraction from one direction B. Curves matching The division of 3D-object (point clouds) by planes perpendicular to the direction of view in order to obtain level curves, produced several planar curves. In our approach, we reduce the problem of matching between two point clouds by using descriptors or signature, to a problem of matching among curves. Finding the best fit between two curves is the center of interest of several filed as computer graphic, computer vision and so on. The set of points extracted from f(u) =[x(u),y(u)] T (1) where u is from 1 to the number of points at one level, x and y are Cartesian coordinates of points. The purpose of curve matching is to find the longest common subcurve of two curves, and compute the rotation angle and translation vector to fit these curves along their common portion. In the literature, There has been a considerable research interest in matching between curves [20][21][22] [23]. We are inspired by the work presented by Wolfson et al. [24]. The authors used the curvature w.r.t the arclength as a signature for a curve. Cui et al. improved this signature by using the integral of unsigned curvatures. We just used the original method proposed by Wolfson for ease of implementation. To compute the curvature of the curve, 79

4 the first and second derivatives of x and y both must exist and continuous. Cubic spline approximation are used to present curves as a polynomial of order 4 as showed in equation below. Then, these curves are parameterized by arc-length as presented in [25]. curve(u) = 4 C i u i 1 = C 1 + C 2 u + C 3 u 2 + C 4 u 3 (2) i=1 Matching process aims to extract the best part from one curve that can be matched with whole or just part from the second curve, to solve this problem, we need to find the position where the query curve aligns the best curve from the dataset. To compute measure the similarity between the query curve and each curve from the dataset, we slide the signature of the query curve on the current curve of the dataset. The small euclidean distance refers to the position of the best fitting. To remove the false matches, we added a new step to compute the euclidean transformation between points on the matched parts. The alignment error is computed by using RMS (root means square) error, which represent the sum of distance among points of matched parts. This error and the similarity measure can be used as indications on the quality of the curve matching. The Fig. 4 show an example of curve extraction from the complete 3D model of dragon. This curve are matched with the illustrated curves in Fig. 6(a) and Fig. 6(b) to present the problem of the direction of parameterization (blue arrows) as it is highlighted in [26]. Therefore, to solve this problem, each curve from the dataset is matched by using the first and second direction of the query curve parameterization, then the best match is keeping. III. EXPERIMENTS AND RESULTS We performed 3D partial object retrieval experiments using the publicly available database [7]. The Fig. 7 presents only five objects that we selected from the database to test our implementation. Our dataset is created first offline. During the creation, level curves of each complete 3D object are computed. As we have explained in Section II-B, matching between query 3D partial object and each 3D object in the dataset can be executed simultaneously. Our implementation takes 4.42 seconds to match between two curves on a 2.3 GHz machine. Each query 3D partial object shares visually similar subparts with at least one complete 3D object in the dataset. Our approach is different from methods based on classification. Therefore, we can add more models at any time into our database without repeating the learning phase. Fig. 8 shows an example of results for matching of one curve from the partial part of the dragon (see Fig. 6(a)) with some of the complete 3D objects of the dataset. Following some experiences, we noticed that when the similarity is less than 1, it is often mean that the curve has a low variation of the curvature (e.g. a small arc), which can be matched (a) (b) (c) (d) Figure 6. Level curves extraction from one direction, (a) and (b) two curves with different direction of the parameterization, (c) matching between (a) and the curve showed in the Fig. 4, (c) matching between (b) and the curve showed in the Fig. 4 80

5 Figure 7. Example of dataset, (Top) Complete 3D models. (Down) Partial models. Figure 8. 3D partial object retrieval results according to the similarity and the root mean square error with all curve present in the query object. To solve this kind of problem, when we found a possible matching between two curves, we compare curves that correspond to upper and lower level of each level curves. Using this solution, we performed our results by removing the false matching. In our experiments, we chose five complete 3D objects with one part from these objects. As we explained in Section II, we created a dataset that contains several level curves of the object. For each query part (see Fig. 7), we extract one level curves from one direction. All curves of query partial object are matched with curves in the dataset. Next, the cardinality of each class are computed. Fig. 9 showed the confusion matrix of all matching process. The bright diagonal of that matrix means that each partial object is identify in the dataset. The precision for this experiment is 95.6% with recall of 61.2%. The results present in this section are interesting and shows that our method is effective for finding a partial objects from dataset. IV. CONCLUSION &FUTURE WORK In this paper, we propose a new approach for 3D partial object retrieval based on level curves. The main contributions were the use of viewing sphere to extract local geometry information from several viewpoints and reduce the problem of matching between two 3D models to the comparison between 2D planar curves. Our approach is an alternative for the identification of 3D partial objects without Figure 9. Confusion matrix of matching five level curves of partial object with the dataset. The bright diagonal indicates high occurrence of matching between partial object and the class that corresponds to the complete model training and classification. In the future, we plan to use our approach for registration between two 3D models, we focus to test the signature proposed by Cui et al. [26], where the authors demonstrated that is invariant to the scale change and euclidean transformation (rotation and translation). 81

6 ACKNOWLEDGMENT This work has been supported by The National Agency of Research of FRANCE within the framework of the GROPLAN project. REFERENCES [1] Zhang, C., Chen, T.: Efficient feature extraction for 2d/3d objects in mesh representation. In: Image Processing, Proceedings International Conference on. Volume 3. (2001) vol.3 [2] Paquet, E., Rioux, M., Murching, A.M., Naveen, T., Tabatabai, A.J.: Description of shape information for 2-d and 3-d objects. Sig. Proc.: Image Comm. 16(1-2) (2000) [3] Ohbuchi, R., Otagiri, T., Ibato, M., Takei, T.: Shapesimilarity search of three-dimensional models using parameterized statistics. In: Computer Graphics and Applications, Proceedings. 10th Pacific Conference on. (2002) [4] Horn, B.K.P.: Extended gaussian images. Proceedings of the IEEE 72(12) (Dec 1984) [5] Corney, J., Rea, H., Clark, D., Pritchard, J., Breaks, M., Macleod, R.: Coarse filters for shape matching. Journal of Computer Graphics and Applications, IEEE 22(3) (May 2002) [6] Chua, C., Jarvis, R.: Point signatures: A new representation for 3d object recognition. International Journal of Computer Vision 25(1) (1997) [7] Mian, A., Bennamoun, M., Owens, R.: 3d recognition and segmentation of objects in cluttered scenes. In: Application of Computer Vision, WACV/MOTIONS 05 Volume 1. Seventh IEEE Workshops on. Volume 1. (Jan 2005) 8 13 [8] Johnson, A., Hebert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. Pattern Analysis and Machine Intelligence, IEEE Transactions on 21(5) (May 1999) [9] Liu, Y., Zha, H., Qin, H.: Shape topics: A compact representation and new algorithms for 3d partial shape retrieval. In: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on. Volume 2. (2006) [10] Sebastian, T.B., Klein, P.N., Kimia, B.B.: On aligning curves. IEEE TPAMI 25(1) (2003) [11] Rusu, R., Blodow, N., Beetz, M.: Fast point feature histograms (fpfh) for 3d registration. In: Robotics and Automation, ICRA 09. IEEE International Conference on. (May 2009) [12] Malassiotis, S., Strintzis, M.: Snapshots: A novel local surface descriptor and matching algorithm for robust 3d surface alignment. Pattern Analysis and Machine Intelligence, IEEE Transactions on 29(7) (July 2007) [13] Cornea, N., Demirci, M., Silver, D., Shokoufandeh, A., Dickinson, S., Kantor, P.: 3d object retrieval using manyto-many matching of curve skeletons. In: Shape Modeling and Applications, 2005 International Conference. (June 2005) [14] Keselman, Y., Shokoufandeh, A., Demirci, M., Dickinson, S.: Many-to-many graph matching via metric embedding. In: Computer Vision and Pattern Recognition, Proceedings IEEE Computer Society Conference on. Volume 1. (June 2003) I 850 I 857 vol.1 [15] Sun, Y., Abidi, M.: Surface matching by 3d point s fingerprint. In: Computer Vision, ICCV Proceedings. Eighth IEEE International Conference on. Volume 2. (2001) vol.2 [16] Levoy: The stanford 3d scanning (2014) [17] Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton shape benchmark. In: Shape Modeling International. (June 2004) [18] Chen, D.Y., Tian, X.P., Shen, Y.T., Ouhyoung, M.: On visual similarity based 3d model retrieval. Comput. Graph. Forum 22(3) (2003) [19] Seinturier, J., Riedinger, C., Mahiddine, A., Peloso, D., Boï, J., Merad, D., Drap, P.: Towards a 3d based platform for cultural heritage site survey and virtual exploration. In: XXIV CIPA International Symposium Recording, Documentation and cooperation for Cultural Heritage,. Volume XL-5/W2. (september 2013) vol.3 [20] Younes, L.: Computable elastic distances between shapes. SIAM Journal on Applied Mathematics 58(2) (1998) [21] Michor, P.W., Mumford, D.: Riemannian geometries on spaces of plane curves. J. Eur. Math. Soc. (JEMS 1 48 [22] Joshi, S.H., Klassen, E., Srivastava, A., Jermyn, I.: Removing shape-preserving transformations in square-root elastic (sre) framework for shape analysis of curves. In Yuille, A.L., Zhu, S.C., Cremers, D., Wang, Y., eds.: EMMCVPR. Volume 4679 of Lecture Notes in Computer Science., Springer (2007) [23] Tabia, H., Daoudi, M., Vandeborre, J.P., Colot, O.: A new 3d-matching method of nonrigid and partially similar models using curve analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on 33(4) (April 2011) [24] Wolfson, H.: On curve matching. Pattern Analysis and Machine Intelligence, IEEE Transactions on 12(5) (May 1990) [25] Wang, H., Kearney, J., Atkinson, K.: Arc-length parameterized spline curves for real-time simulation. In: In in Proc. 5th International Conference on Curves and Surfaces. (2002) [26] Cui, M., Femiani, J., Hu, J., Wonka, P., Razdan, A.: Curve matching for open 2d curves. Pattern Recogn. Lett. 30(1) (January 2009)