Skeleton Path Based Approach for Nonrigid 3D Shape Analysis and Retrieval

Transcription

1 Skeleton Path Based Approach for Nonrigid 3D Shape Analysis and Retrieval Chunyuan Li and A. Ben Hamza Concordia Institute for Information Systems Engineering Concordia University, Montréal, QC, Canada Abstract. In this paper, we propose a skeleton path based approach to analyze and retrieve nonrigid 3D shapes. The main idea is to match skeleton graphs by comparing the geodesic paths between skeleton endpoints. Our approach is motivated by the fact that the path feature is stable in the presence of articulation of components. The experimental results demonstrate the performance of our proposed method in terms of robustness to symmetry, discrimination against different graph structures, and high efficiency in nonrigid shape retrieval. Keywords: Skeleton Path, Nonrigid 3D Shapes, Retrieval. 1 Introduction With the increase in the number of scanned 3D objects, 3D shape analysis and retrieval is becoming popular in the fields of computer vision, computer graphics, and computer aided design. Previous efforts have been, however, mainly devoted to rigid 3D models, and thus how to efficiently and effectively analyze and compare nonrigid shapes is still a challenging problem. The curve skeleton, which integrates geometrical and topological features of the object, is an important shape descriptor. Shape similarity based on skeleton matching usually performs better than mesh surface or other shape descriptors in the presence of articulation of components, especially for non-rigid shape. As pointed out in [7], the curve-skeleton provides characteristics like part/component matching, registration and visualization, intuitiveness, and articulated transformation invariance. Nonrigid shape matching is one of the most challenging problems in contentbased 3D object retrieval. The aim of the SHREC 21 Shape Retrieval Contest of Non-rigid 3D Models is to evaluate and compare the effectiveness of different 3D retrieval methods [1]. For a 3D retrieval algorithm, the shape descriptor and the similarity discrimination are two key components. The global and local isometry-invariant descriptor proposed recently by Wu et al. [18] captured well the global and local information. Also, Agathos et al. [1] proposed a retrieval methodology based on a graph-based object representation. This method makes use of a meaningful new mesh segmentation and the Earth mover s distance (EMD) similarity measure. J.K. Aggarwal et al. (Eds.): IWCIA 211, LNCS 6636, pp , 211. c Springer-Verlag Berlin Heidelberg 211

2 Skeleton Path Based Approach for Shape Analysis and Retrieval 85 In recent years, several skeleton based shape analysis and retrieval methods have been proposed. Sundar et al. [16] encoded the geometric and topological information in the form of a skeleton graph and used graph matching techniques for skeleton matching and comparison. Cornea et al. [7] enhanced the framework by using a new skeletonization algorithm and an extension of the manyto-many matching algorithm. Au et al. [2] presented a fast and fully automatic correspondence algorithm that allows matching of a wide variety of shapes with semantically similar structures but with different geometric details. Specifically, they attempted to find a one-to-one semantic correspondence between two sets of feature nodes of curve skeletons. However, determining the similarity between two given shapes does not necessarily require finding an exact correspondence between their shape components. Our work is partly a 3D extension of the 2D path similarity skeleton graph matching approach proposed by Bai. et al [4]. Unlike [4], a major goal of our approach is to discover the symmetry instead of finding correspondences. Siddiqi et al. [15] introduced a medial surface based method and obtained state-of-the-art performance on McGill Articulated Shape Benchmark [15]. Partly motivated by Leonardo s Vitruvian Man, which describes the perfect human form in geometrical terms, we propose a skeleton path feature to represent each component of a non-rigid 3D shape, assuming that this feature descriptor is isometry-invariant, i.e. invariant to the object s variational representation, rotation, translation, scaling, and nonrigid bending. A skeleton path refers to the geodesic paths between two endpoints in the curve-skeleton, as shown in Fig. 1(a), where these shortest paths are represented as sequences of radii of the maximal balls at the corresponding skeleton points. We also benefit from the fact that the proportions of the curve skeleton length for different components are different and are almost constant. Although we do not explicitly consider the topological structure of the skeleton graphs, we do not, however, completely ignore this structure. It is worth pointing out that this topological structure is implicitly represented by the fact that overlapping parts of the geodesic paths are similar. Therefore, our approach is flexible enough to perform extremely well on nonrigid 3D shapes. The rest of the paper is organized as follows. Section 2 describes the proposed approach. The experimental results using the proposed algorithm are provided in Section 3. Finally, we conclude in Section 4. 2 Proposed Approach Our proposed method may be described as a two-phase approach: 1) Skeleton path acquisition, which includes curve-skeleton extraction, endpoint detection, and path construction. 2) Endpoints matching, which consists of finding an ordered sequence of end nodes and post-processing of this new sequence. For convenience and efficiency, we adopt the curve-skeleton extraction algorithm developed by Cornea et al. [8]. Nevertheless, our algorithm can be generally applied on curve-skeletons with satisfactory homotopic and centered properties [9].

3 86 C. Li and A. Ben Hamza (a) (b) Fig. 1. (a) Our skeleton path based algorithm describes a 3D cow model as many paths between end nodes. (b) Symmetric components that are discovered by our algorithm are represented by end nodes rendered with the same colors. Note that only part of the results is shown. The proposed curve-skeleton extraction algorithm works on a volumetric representation of a 3D object. It uses a generalized potential field generated by charges placed on the surface of the object. Given a 3D vector field, we use concepts from vector field visualization to identify two types of seed points that we will use to construct a curve-skeleton: critical points and high divergence points. Skeleton segments are discovered using a force-following algorithm on the underlying vector field, starting at each of the identified seed points. The force-following process evaluates the vector (force) value at the current point and moves in the direction of the vector with a small pre-defined step. At critical points, where the force vanishes, the initial directions are determined by evaluating the eigenvalues and eigenvectors of the Jacobian at the critical point. More details about computing the curve-skeleton can be found in [8]. 2.1 Skeleton Path We first describe the initial steps for building the skeleton graphs. The following definitions apply to continuous skeletons as well as to curve-skeletons of 3D models (composed of voxels).

4 Skeleton Path Based Approach for Shape Analysis and Retrieval 87 Definition 1. A skeleton point having only one adjacent point is an endpoint (the skeleton endpoint); a skeleton point having three or more adjacent points is a junction point. If a skeleton point is not an endpoint or a junction point, it is called a connection point. Definition 2. The endpoint in the skeleton graph is called an end node, and the junction point in the skeleton graph is called a junction node. Definition 3. The shortest path between a pair of end nodes on a skeleton graph is called a skeleton path. Based on the curve skeletons that are extracted using the method described above, we provide details on how we detect endpoints and construct the skeleton path. A. Skeleton Endpoint Detection Since the curve-skeleton consists of many segments that have two ends (i.e. segment endpoints), we detect the skeleton endpoints by considering the distance between these end nodes. We denote the set of all the N segment endpoints of an input skeleton by P = {p 1,p 2,..., p N }. For simplicity, let p k P denote the testing endpoint. Given a threshold ε RD,letQ = {q : p k q ε RD,q P, q p k } be the nearest neighbor endpoint set. We consider p k as a skeleton endpoint if the size of Q is and as a junction point if the size is larger than 2. The rest of the segment endpoints and all the other skeleton points are connected points (see Fig. 2). Fig. 2. Illustration of the critical skeleton points using our method on a 3D airplane (left). On the right, (a) endpoint, (b) connected points between two segments, (c) junction points, and (d) connected points in a segment. For better viewing, please see the original color pdf file. B. Skeleton Path Construction After endpoints are detected, we construct the skeleton path as the shortest path between two given nodes (see Fig. 1). Suppose there are N end nodes {v i } i=1,...,n in the skeleton graph G to be matched. Path Length Percentage. Let Γ =(γ ij )beann N path length matrix, where γ ij denotes the geodesic distance (shortest path length) from the i-th end

5 88 C. Li and A. Ben Hamza node v i to the j-th end node v j. To preserve scale-invariance, we normalize the matrix Γ by the overall curve skeleton length L skel. In this way, we obtain an N N length percentage matrix L =(l ij ), where l ij = γ ij /L skel. Path Radius Vector. Let p(v m, v n ) denote the skeleton path from v m to v n. We sample p(v m, v n )withm equidistant points, which are all skeleton points. Let R m,n (t) be the radius of the maximal ball at the skeleton point with index t in p(v m, v n ). We define a vector of the radii of the maximal balls at the M sample points on p(v m, v n ) as follows: R m,n = ( R m,n (t) ) t=1,2,...,m =(r 1,r 2,...,r M ). (1) The distance transform value for each point is equal to the radius of maximal inscribed ball. Suppose there are N voxels in the original 3D model, then to make the proposed method invariant to scale, we normalize R m,n (t) in the following way: DT (t) R m,n = N 1/N i=1 DT(s i), (2) where s i varies over all N voxels in the model [4]. Path Distance. The model dissimilarity between two skeleton paths is called a path distance. If R =(r i ) i=1,...,m and R =(r i ) i=1,...,m denote the vectors of radii of two model paths p(u, v)andp(u, v ) respectively, then the path distance is defined as ϕ ( p(u, v),p(u, v ) ) = M i=1 (r i r i )2 r i + r i + α (l l ) 2 l + l, (3) where l and l are the length percentages of p(u, v) andp(u, v ) respectively. The parameter α is a weight factor [4]. In order to make our representation scale invariant, the path lengths are normalized. We include the path length percentages in Eq. (3), since the percentage is not reflected in the sequences of radii (all paths are sequences of M radii). Thus, our path representation and the path distance are scale invariant. 2.2 Endpoints Matching Sorting endpoints by summing path length percentage. In a skeleton graph G with N end nodes {v i } i=1,...,n, each end node has the skeleton paths to all other end nodes in the graph. Let T (v i )= N j=1 l ij be the total path length percentages of the end node v i. Thus, given an end node v k, there is a corresponding length percentage T (v k ). We order all the end nodes in G following these percentages, by ranking an endpoint with a higher percentage at the top of the list. Therefore, we obtain an ordered end node sequence S = {u 1, u 2,...,u N }.

6 Skeleton Path Based Approach for Shape Analysis and Retrieval 89 Endpoints Distance. Let G and G be two graphs to be matched, and denote by S = {u i } i=1,...,k+1 and S = {u i } i=1,...,n+1 their respective ordered end node sequences, with K N. Similar to shape contexts [5], the matching cost c(u i, u j ) between u i and u j is based on the paths to all other end nodes in G and G that emanate from u i and u j, correspondingly. Then, we compute the path distances between the two sequences and obtain a matrix Φ = ( ϕ(u i, u j )) of the path distances computed using Eq. (3). As suggested by Bai et al. [4], we use the optimal subsequence bijection (OSB) to compute the dissimilarity value: c(u i, u j)=osb ( ϕ(u i, u j) ) (4) Global Matching. Using Eq. (4), we compute the total dissimilarity matrix C(G, G )= ( c(u i, u j )) 1 i K+1 between G and G using the Hungarian algorithm. For each end node v i in G, the Hungarian algorithm can find its corre- 1 j N+1 sponding end node v i in G.SinceG and G may have different numbers of end nodes, the total dissimilarity value should include a penalty for end nodes that did not find any partner. To achieve this, we simply add additional rows with a constant value κ so that C(G, G ) becomes a square matrix. This constant value κ is the average of all the other values in C(G, G ). The intuition behind using the Hungarian algorithm is that we want to have a globally consistent one-to-one assignment of all end nodes with possibly assigning some end nodes to κ, which represents a dummy node. This means that we seek a one-to-one correspondence of the end nodes in the skeleton graphs (with possibly skipping some nodes by assigning them to a dummy node). However, the Hungarian algorithm does not preserve the order of the matched sequences. This does not influence the final score, since we can change the order only for similar symmetric end nodes. This is also part of the reason why we can detect symmetric components. 3 Experimental Results In this section, we evaluate the performance of the proposed method in three parts: Symmetric components discovery, matching between different graph structures, and illustration of the recognition performance of our method on McGill Articulated Shape Database. 3.1 Symmetric Components Discovery Given an end node sequence S = {u 1, u 2,...,u N } of a skeleton G, weobtaina new sequence Ŝ by changing the order of certain end nodes in a set C G and compute the dissimilarity to the original. If the result C(S, Ŝ) islessthanagiven threshold ε C, we consider the components containing these end nodes in C to be symmetric. Obviously, for a given nonrigid 3D shape, it is possible that there are more than one such set C, whose size may be larger than 2. In other words, there exist many symmetric component groups and that each group might have more

7 9 C. Li and A. Ben Hamza Fig. 3. The curve skeleton and discovered symmetric components indicated by end nodes with the same color Table 1. The matrix of dissimilarity values between the skeleton graph with the corresponding end nodes exchanged and the original. The colored values are symmetric node pairs than 2 components. We now give a simple example illustrating our symmetry components discovery approach. Fig. 3 shows the curve skeleton and the results on a 3D cow model. The end nodes, displayed with the same color, indicate the symmetric components. As we observe, the left front leg and the right front leg are symmetric components, they are shown in yellow color. The back legs in green and horns are displayed in blue. We indexed the end nodes so that symmetric nodes are clear to be presented. The matrix with elements indicating the dissimilarity if the corresponding nodes are exchanged is shown in Table 1. The dissimilarities between two most symmetric end nodes are marked with colored numbers. Here, we choose the parameters ε C =.1,M =5,α= 1. Besides the symmetry discovery of the 3D cow model in Fig. 3, we tested the process on several other examples. In Fig. 4(a), we first discover the symmetry in both the hands and the legs of a dancer. And the head is not symmetric to any part of the human body. Obviously, our method finds the correct symmetric components even in large variability due to articulation. Secondly, Fig. 4(b) shows the result of an eight-leg octopus, which demonstrates that the proposed

8 Skeleton Path Based Approach for Shape Analysis and Retrieval 91 (a) (b) (c) (d) Fig. 4. (a) Symmetry discovery of a dancer; (b) Symmetry discovery of an octopus; (c) Symmetry discovery of a crab; (d) Symmetry discovery of a chair method is able to discover symmetric groups with numerous members. A crab with eight legs and two eyes in Fig. 4(c) illustrates that our method works correctly in the situation when there are many symmetric groups with different quantities of member. Finally, we show the result of a four-leg chair in Fig. 4(d). It demonstrates that the proposed method also performs well in the presence of a rigid shape, even when its skeleton graph is not a tree. 3.2 Matching Skeletons with Different Graph Structures For skeleton graphs with the same number of end nodes, they might have very different graph structure. Sometimes there are similar path radii vectors. But path length percentage will enhance the performance. For example, shapes like snake and spectacle have two endpoints in their curve-skeletons. Moreover, the skeleton graph of the snake is a tree, whereas the skeleton graph of the spectacle is not a tree. To evaluate the performance of our proposed algorithm on distinguishing the topological difference, we use a small database that contains four nonrigid 3D shapes: Two spectacles and two snakes as shown in Fig. 5. The parameter M for this database was set to M = 5. We also show the results with α = in Fig. 6(a) and α = 1 in Fig. 6(b). By observing the rankings, it is evident that both of them could discriminate the skeleton graphs with different structures. Although the shortest paths between end nodes of the two classes are similar, the proposed method is, however, able to distinguish the structural difference between a closed loop and a line better by considering the length percentages. No matter how shapes, e.g. spectacles or snakes, deform due to articulation, the length percentages are always almost constant. Moreover, as

9 92 C. Li and A. Ben Hamza Fig. 5. Top: Two spectacles and their curve-skeletons; Bottom: Two snakes and their curve-skeletons (a) (b) Fig. 6. Comparison between α =andα = 1 on a small database. The distance between query and the given shape is also displayed. a matter of fact, shapes in different classes have different length percentages, which lead to more effective discrimination. 3.3 Retrieval on McGill 3D Articulated Shape Database Based on the above experimental results, our algorithm is validated to be robust to symmetry and discriminative to different graph structures. We demonstrate it further on McGill Articulated Shape Database with 255 objects divided into ten categories, namely, Ants, Crabs, Spectacles, Hands, Humans, Octopuses, Pliers, Snakes, Spiders, and Teddy Bears. Sample models from this database are shown in Fig. 7.

10 Skeleton Path Based Approach for Shape Analysis and Retrieval 93 Fig. 7. Sample shapes from McGill Articulated Shape Database. Only two shapes for each of the 1 classes are shown. Skeleton Path based Methods. Retrieving shapes that are similar to a given query shape from a database involves shape matching. However, determining the similarity between two given shapes does not necessarily require finding an exact correspondence between their shape components. In this section, we extend the shape similarity measure discussed in Section 3 to shape retrieval, and we also propose five methods based on the skeleton path. In the sequel, we will use the following abbreviations: SH: We denote the method in section 3 as SH, since it uses the square matrix with penalty and Hungarian algorithm. SDP: We denote the method that uses the square matrix with penalty and dynamic programming algorithm as SDP. NSH: We denote the method that uses the matrix, which is not square and without penalty, and the Hungarian algorithm as NSH. EMS: We define the dissimilarity from the query to a shape in the dataset as the sum of minimum endpoint distance of the query to all endpoints of the latter, and denote it as EMS. PMS: We define the dissimilarity from the query to a shape in the dataset as the sum of minimum skeleton path distance of the query to all skeleton paths of the latter, and denote it as PMS. Here we use the parameters M =5andα = 5. In our comparative analysis, we have used the precision/recall curve to measure the retrieval performance. Ideally, this curve should be a horizontal line at unit precision. For each query shape, we use the first 77 returned shapes with descending similarity rankings (i.e., ascending Euclidean distance ranking), dividing them into 11 groups accordingly. The retrieval results of the 5 skeleton path based methods on the whole McGill Articulated Shape Database are shown in Fig. 8. Obviously, PMS provides a much better performance that the other methods because it fully exploits the original information that skeleton paths carry. By finding the minimum value of skeleton path distances, we might establish a potential corresponding relationship between paths. However, as a matter of fact, there is no veracious global path correspondence. As for EMS, the second best method, we assume that the endpoints with minimum distance are corresponding to each other, although it may fail to find the global endpoints correspondence. Furthermore, SH and SDP are almost neck and neck in terms of retrieval accuracy, and both are superior to NSH, which demonstrate that the penalty plays a key role in shape

11 94 C. Li and A. Ben Hamza Fig. 8. Precision-recall plot of the proposed skeleton path based methods discrimination. Our implementation was done in MATALB on a Intel Core 2 Duo with 2. GHz. To give an idea about the timing: constructing the skeleton path takes on average slightly less than 1 minute for a 3D model; most of this time is actually consumed by Dijkstra s algorithm for finding the shortest path between two end nodes. 4 Conclusions We proposed a skeleton path based technique that is able to detect symmetric components, discriminate different graph structure and retrieve nonrigid 3D shapes. We represented a nonrigid shape by a set of geodesic paths between skeleton endpoints. These paths were compared using sequence matching. By detecting symmetric components, our framework is shown to be consistent with human semanteme based on curve-skeleton. Also, we found that it is possible to discover multiple components in a symmetric group. In addition, the proposed approach could enhance the performance of distinguishing the topological difference. Finally, our skeleton path based approach is shown to be effective, efficient and easily understandable for articulated 3D shape retrieval. References 1. Agathos, A., Pratikakis, I., Papadakis, P., Perantonis, S., Azariadis, P., Sapidis, N.: Retrieval of 3D articulated objects using a graph-based representation. The Visual Computer 26, (21) 2. Au,O.K.-C.,Tai,C.-L.,Cohen-Or,D.,Zheng,Y.,Fu,H.:Electorsvotingforfast automatic shape correspondence. In: Proc. Eurographics, vol. 29 (21)

12 Skeleton Path Based Approach for Shape Analysis and Retrieval Au, O.K.-C., Tai, C.-L., Chu, H.-K., Cohen-Or, D., Lee, T.-Y.: Skeleton extraction by mesh contraction. ACM Transactions on Graphics 27 (28) 4. Bai, X., Latecki, L.J.: Path similarity skeleton graph matching. IEEE Trans. Pattern Analysis and Machine Intelligence 3, (28) 5. Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Analysis and Machine Intelligence 24, (22) 6. Chen, D.-Y., Tian, X.-P., Shen, Y.-T., Ouhyoung, M.: On visual similarity based 3D model retrieval. Computer Graphics Forum 22, (23) 7. Cornea, N.D., Demirci, M.F., Silver, D., Shokoufandeh, A., Dickinson, S., Kantor, P.B.: 3D object retrieval using many-to-many matching of curve skeletons. In: Proc. Int. Conf. Shape Modeling and Applications, pp (25) 8. Cornea, N.D., Silver, D., Yuan, X., Balasubramanian, R.: Computing hierarchical curve-skeletons of 3D objects. The Visual Computer 21, (25) 9. Cornea, N.D., Silver, D., Min, P.: Curve-skeleton properties, applications, and algorithms. IEEE Trans. Visualization and Computer Graphics 13, (27) 1. Lian, Z., Godil, A., Fabry, T., Furuya, T., Hermans, J., Ohbuchi, R., Shu, C., Smeets, D., Suetens, P., Vandermeulen, D., Wuhrer, S.: SHREC 21 Track: Nonrigid 3D shape retrieval. In: Proc. Eurographics Workshop on 3D Object Retrieval, pp. 1 8 (21) 11. Kazhdan, M., Chazelle, B., Dobkin, D., Finkelstein, A., Funkhouser, T.: A reflective symmetry descriptor. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 22. LNCS, vol. 2351, pp Springer, Heidelberg (22) 12. Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton shape benchmark. In: Proc. Shape Modeling International, pp (24) 13. Shinagawa, Y., Kunii, T.L., Kergosien, Y.L.: Surface coding based on Morse theory. IEEE Computer Graphics and Applications 11, (1991) 14. Siddiqi, K., Shokoufandeh, A., Dickinson, S., Zucker, S.: Shock graphs and shape matching. International Journal of Computer Vision 35, (1999) 15. Siddiqi, K., Zhang, J., Macrini, D., Shokoufandeh, A., Bouix, S., Dickinson, S.: Retrieving articulated 3-D models using medial surfaces. Machine Vision and Applications 19, (28) 16. Sundar, H., Silver, D., Gagvani, N., Dickinson, S.: Skeleton based shape matching and retrieval. In: Proc. Shape Modeling International, pp (23) 17. Wang, Y.-S., Lee, T.-Y.: Curve skeleton extraction using iterative least squares optimization. IEEE Trans. Visualization and Computer Graphics 14, (28) 18. Wu, H.-Y., Zha, H., Luo, T., Wang, X.-L., Ma, S.: Global and local isometryinvariant descriptor for 3D shape comparison and partial matching. In: Proc. CVPR, pp (21) 19.