3-D Morphing by Direct Mapping between Mesh Models Using Self-organizing Deformable Model

Transcription

1 3-D Morphing by Direct Mapping between Mesh Models Using Self-organizing Deformable Model Shun Matsui Ken ichi Morooka Hiroshi Nagahashi Tokyo Institute of Technology Kyushu University Tokyo Institute of Technology Abstract This paper presents a new method for 3-D morphing between mesh models using a deformable model called SDM (Self-organizing Deformable Model). The SDM is a mesh model which deforms its shape by learning a target surface. Since the model doesn t depend on a specified shape, we can use various mesh models as SDMs. This enables us to use a SDM for making a correspondence between mesh models. It is also possible to define the correspondence based on their model features. We demonstrate some examples of SDM-based morphing. 1 Introduction Morphing is a key technique in computer animation. It is a process of transforming a given source shape (initial model) into another one (final model) smoothly. Compared with 2-D image morphing, 3-D morphing with 3-D shape deformation enables us to make interpolated images from various view-points and to generate good visual effects. Therefore, 3-D morphing is widely used in movies, games, virtual reality and so on. There are several ways to describe 3-D object models in CG and many morphing methods have been proposed for models in different descriptions[5]. Among them, a description by triangle meshes is often used because of the goodness of the expression for complex models. It is necessary to solve a corresponding problem and an interpolating problem for 3-D morphing between mesh models[1]. The former problem is to construct one-toone correspondences between mesh models which have different data structures. On the other hand, the latter problem is to make intermediate shapes for morphing. There are many methods for morphing between mesh models and most of them focus on the corresponding problem. In this paper, we also discuss the corresoponding problem between genus-0 mesh models. One major method for solving the corresoponding problem is to make a synthesis mesh with interpolating primitive region[2],[3]. It is to create one mesh model which have the data structure of two mesh models and whose vertices have several coordinates. In this case, the mesh fusion yields increasing mesh data. Also, it is difficult for a user to control it. Another major method is to make a correspondence by using low-resolusion mesh models and parameterize from an initial model to a target model[6],[12]. However, these methods require some kind of experience of making low-resolusion models. We proposed a new method for 3-D morphing based on a Self-organizing Deformable Model (SDM). The SDM is a mesh model which deforms its shape by learning a target surface. First, we use an initial model and a final model as two SDMs, respectively and we perform SDM-mappings of them onto a common surface defined as a target surface. Next, compared with the mapping results, we make a rough correspondence between them. Then, we use the resulting model as SDM and the final model as a target surface, respectively and perform a SDM-mapping again. After these processes, a new model we get is to be an approximate model of the final model which has the same topological information of the initial model. Consequently, we make one-to-one correspondence for each vertices of the initial model and the new model which have the same index and interpolate their coordinates to make their intermediate models. Additionally, it is important for 3-D morphing to be able to control the morphing according to structural features of models. A SDM-mapping realizes it by setting some constraints to the movement of the SDM s vertices. 2 Self-organizing Deformable Model 2.1 Notations A SDM is a mesh model which can deform its shape by learning a target surface[9]. The target surface should be represented as a set of vertices which have normal vectors defined at their neighborhood. Usually, we represent the target surface as a mesh model. In this paper, the SDM and the target surface are assumed to be genus-0 mesh models. By using notations adopted in [7], they are represnted as M = (P, K), where P is a set of the vertices positions p i with i = 1,..., N v and N v is the number of vertices of M. K is an abstract simplicical complex which contains all topological information of M. The complex K is composed of three types of subsets of simplices: a set K v of vertices v = {i} K v, a set K e of edges e = {i, j} K e, a set K f of faces f = {i, j, k} K f. p i represents a position of a vertex i K v. An edge e has two vertices i and j on its both ends and a triangle face f is composed of three vertices i, j and k. 2.2 SDM-Mapping A SDM-mapping changes the coordinates of vertices in P, while preserving its topological information in K. Given an SDM M S and a target surface M G, a topologypreserving mapping Φ by the SDM is formulated as Φ : M S = (P S, K S ) M (G S) = (P (G S), K S ). (1) Here, M (G S) is a resulting model which approximates the shape of the target surface M G and has the same topological information of the SDM M S. Generally, such a mapping is non-linear and nonunique. A SDMmapping realizes such a non-linear mapping by deforming its shape in terms of self-organizing map[4]. Particularly, we represent the relation between such a topologypreserving mapping Φ S G from M S onto M G and M (G S)

2 Initial Model Deformed Final Model Figure 1: Direct Mapping from an initial model to a final model. as M (G S) = Φ S G (M S ). (2) A SDM-mapping is composed of the following steps. [Algorithm 1] 1. Initialize a time parameter to t = Select a control point j K G v randomly. 3. Determine its winner vertex i K S v by the following criterion. i = arg min SD(i, j) (3) i K S v We define a signed distance SD(i, j) between i and j as SD(i, j) = n G j (ps i pg j ), (4) where n G j is a normal vector of j K G v. 4. For the winner vertex i K S v and its neighbors, adapt their positions according to the topological distance from i as p S i p S i + ɛ(t)λ(i i ) p S i, (5) where ɛ(t) is a learning rate which adapts the movement degree from p S i to p G j. λ(i i ) is called a neighborhood function which defines the degree of adaptation according to the topological distance between i and i. In our method, we defined them as ɛ(t) = ɛ s ( ɛ f ɛ s ) t T λ(i i ) = exp{ 1 2 ( L(i,i ) σ(t) )} σ(t) = σ s ( σ f σ s ) t T, (6) where L(i, i ) represents the shortest path between i and i. ɛ s (σ s ) and ɛ f (σ f ) are the initial value and the final value of ɛ(σ), respectively. p S i is the movement vector for each vertex i. 5. t t If t < T, go to the step 2. Otherwise go to the step For minimizing energy between M S and M G, we define the distance D(M S, M G ) between them as D(M S, M G ) = 1 3 i K S v E(i) Γ i Ω i, (8) where Γ i is a set of triangle patches f Kf S which include a vertex i Kv S and Ω i is a set of the control point j Kv G on M G corresponded to its closest vertex among all vertives of M S. E(i) is a energy function for i of SDM M S and it is defined as E(i) = {H(j, f)} 2, (9) j Ω i f Γ i where H(j, f) is a function which returns the distance between j Kv G and f Kf S. In the following steps, we minimize the distance D(M S, M G ). 8. For each vertex i K S v, calculate a position p S i by p S i = p S i + w(p G j p S i ); j Ω i, (10) where w is a step value which defines the degree of the movement from p S i to p G j. 9. For all positions calculated by (10), decide the next position p S i of the vertex i K S v as follows. p S i = arg min E(i) (11) p S i 10. If the distance D(M S, M G ) satisfies the following equation, finish the deformation. Otherwise, go to step 8. ξ is a threshold value. D(M S, M G ) < ξ (12) p S i = p G j ps i (7)

3 Initial Model step 1: step 2 Mediator Final Model step 2 step 3: Deformed step 1: Figure 2: Direct Mapping using a mediator model 3 Correspondence Problem between Mesh Models 3.1 Direct Mapping using only a SDM-mapping In our method, we can get a new model which has the same topological information of a given initial model and approximates a given final model by using a SDMmapping. Therefore, it seems available to perform a SDM-mapping using an initial model and a final model as a SDM and a target surface, respectively ( as shown in Fig.1). In this case, the user can estimate the SDMmapping result viscerally and it simplifies the user s control of the SDM-mapping discussed below. However, the method isn t applicable for mesh models which have complex shapes. Conseqently, we use another method for direct mapping between mesh models using a mediator model. We detail the method in the next section. 3.2 Direct Mapping using a Mediator Model Because of the complexity discussed in 3.1, we use a mediator model for direct mapping between mesh models (as shown in Fig.2). At first, an initial model and a final model are respectively mapped onto the same mediator model by using SDM-mappings. Next, we make a rough correspondence between the mapping results via the mediator model. Finally, we perform an additional SDM-mapping using the semi-approximated model and the final model as a SDM and a target surface, respectively. A process for making a correspondence between mesh models using a mediator model is composed of the following steps. [Algorithm 2] 1. Let M I, M F and M C be an initial, a final and a mediator model, respectively. M I and M F are SDMs and M C is a target surface. Then, perform the following SDM-mappings according to the algorithm 1. M (C I) = Φ I C (M I ) M (C F ) = Φ F C (M F ) (13) After this process, both M (C I) and M (C F ) approximate the shape of M C and have the same topological information as those of M I and M F, respectively. 2. For the vertex i K (C I) v, (i) Select the vertex j K v (C F ) which minimizes the distance d(i, j) as the corresponding vertex j for i j = arg min j d (i, j), (14) where d (i, j) is the Euclidian distance between p (C I) i P (C I) and p (C F ) j P (C F ). (ii) Assign the coordinates p F j PF of M F to the coordinates p (C I) i p (C I) i p F j (15) After transforming the coordinates, we get a semiapproximate model M (F I) of M F. 3. Let the defomed model M (F I) and M F be a SDM and a target surface, respectively. Perform a SDMmapping Φ (F I) F as follows. M (F I) = Φ (F I) F (M (F I) ) (16) Then, we get a new mesh model M (F I) which approximates the shape of M F and has the same topological information as that of M I. In step 2, the one to one correspondence between the vertices of M (C I) and M (C F ) is not obtained usually. Therefore, some vertices of M (F I) may have the same coordinates after the step 2. Such a condition doesn t preserve the topological information compared with M I. This leads a self-intersection in morphing and it is not desirable. Consequently, we get a new model M (F I) by performing Φ (F I) F. It involves the movement of the vertices of M (F I) depending on the topological distance

4 between them. Then, M (F I) becomes an mesh model approximating the shape of M F and has the same topological information as that of M I. 3.3 Control of SDM-mapping It is important for 3-D morphing to make a flexible correspondence. For example, making structural feature correspondences between models (eye to eye, nose to nose, etc) is desirable for 3-D face morphing. Therefore, it is neccesary to control the SDM-mapping according to user s purpose. A SDM-mapping realizes it by setting some constraints to the movement of the SDM s vertices. In particular, some vertices g n Kv S and corresponding control points h n Kv G are selected manually. Then, we get a set of pairs X as X = {(g 1, h 1 ), (g 2, h 2 ),..., (g N, h N )}, (17) (a) (b) (c) (d) (e) (f) where N is the number of pairs selected by user. Then, we modify the determination of the winner vertex in (3) as follows. { i = g n ; ( if j = h n ) arg min i K S v SD (i, j) ; ( otherwise ) (18) In our method, one can control the correspondence between mesh models by setting the constraints to the movement of the SDM s vertices. 3.4 Interpolating Problem Using the method represented in the previous section, we get a mesh model M (F I) which approximates a final model M F and has the same topological information as that of a initial model M I. We make the intermediate models for morphing animation by making the oneto-one correspondence between the vertices of M I and M (F I) which have the same indices. Let the vertex position of the intermediate model M M be p M i P M, it is represented as p M i = (1 f (t))p I i + f (t) p(f I) i (0 t 1). (19) A time function f (t) is a function which changes smoothly 0 f (t) 1. (g) (h) (i) Figure 3: Mesh Models used in our experiments; Top : mesh models used as an initial model (a) Rabbit, (b) Pig, (c) Armadillo; Midium : mesh models used as a final model (d) Balljoint, (e) Lion, (f) Santa; Bottom : mesh models used as a mediator model (g) Sphere,(h) Animal, (i) Cylinderman. 4.2 Direct Mapping between Mesh Models Fig.4 shows the results of the mapping from an initial model onto a final model. Normals of the initial models are mapped on them and their shapes approximate the final models, respectively. Thus, the initial models are mapped onto the target models while the topological information may be preserved. In our experiments, we used mediator models which have the same skeleton data structure with mesh models for the mappings of Pig Lion and of Armadillo Santa. One can make a correspondence easily by using such mediator models. 4 Experimental Results For examining the effectiveness of our method, we made some experiments. 4.1 Mesh Models Fig.3 shows some examples of mesh models used in our experiments. Table.1 and 2 show the specifications of mesh models and the combinations of the morphing, respectively. Three mesh models as mediators are shown in Fig.3 (g) (i) (a sphere, an approximated quadrupedal model and an approximated human model). We set the parameters in algorithm 1 as ɛ s = 0.5, ɛ f = 0.005, σ s = 3.0 and σ f = 0.1, respectively. 4.3 Morphing Table 1: Detail of mesh models. # vertex # patch Rabbit 6,706 13,408 Balljoint 3,429 6,854 Pig 14,058 28,112 Lion 8,154 18,304 Armadillo 17,299 34,584 Santa 7,580 15,156 In this paper, we define a time function f(t) as f (t) = t (0 t 1). (20)

5 (a) (b) (c) Figure 4: Mapping Results; (a) Rabbit Balljoint, (b) Pig Lion, (c) Armadillo Santa. Figure 5: Morphing in the part of the face with some constraints for feature points. Table 2: Combination of mesh models. Ex.1 Ex.2 Ex.3 Initial Model Rabbit Pig Armadillo Final Model Balljoint Lion Santa Mediator Sphere Animal Cylinderman Fig.5 shows the morphing focusing on the face part between Pig and Lion. We set 5 feature points as constraints in the part. By setting them, the face morphing becomes a natural morphing according to feature points. Fig.6 shows the morphing from Rabbit to Balljoint. We have no constraints between them because they have no semantic corresponding parts. In our method, the correspondence can be constructed without user s interposition. Fig.7 shows the morphing from Pig to Lion. We set 4 feature points as constraints for them except the face feature points. Fig.8 shows the morphing from Armadillo to Santa. In this morphing, 6 feature points were given as their constraints. 5 Conclusion In this paper, we present a new method for 3-D morphing by direct mapping from a mesh model onto another one by using a SDM-mapping. First, an initial model and a final model are mapped onto a mediator, respectively. Next, we make a rough correspondence between the mapped models via the mappings. Then, the vertex coordinates of the mapped initial model is transposed to the corresponding vertex coordinates of the mapped final model and we perform a SDM-mapping again. After these processes, we can realize the direct mapping between mesh models. Thus, the new method we proposed needs neither combining two meshes nor making low-resolution models. In addition, some constraints for feature points of two mesh models enables us to control a mapping according to the structural features of the mesh models. Besides, we can standardize the description of some mesh models by describing their data structures using the method we present. It enables to perform multitarget morphing[8]. As future works, we will realize the morphing based on our method between the mesh models whose genus are non-zero[14]. In this paper, we set some constraints for only feature points of the models. Therefore, we need to extend the constraints including the discriminative regions such as a head, a body and arms[11]. We are also examining the effectiveness of the SDM for deformation transfer[13], expression cloning[10] and another techniques. References [1] Alexa, M.: Recent Advances in Mesh Mophing, Technical report, Interactive Graphics Systems Group (2002). [2] Kanai, T., Suzuki, H. and Kimura, F.: 3D Geometric Metamorphosis based on Harmonic Map, The Visual Computer, pp (1997). [3] Kent, J, R., Carlson, W. and Parent, R.: Shape transformation for polyhedral objects, SIGGRAPH 92 Proceedings, pp (1992).

6 Figure 6: Morphing from Rabbit to Balljoint. Figure 7: Morphing from Pig to Lion. Figure 8: Morphing from Armadillo to Santa. [4] Kohonen, T.(ed.): Self-Organizing Maps, Springer- Verlag, Berlin Heidelberg New York (1996). [5] Lazarus, F. and Verroust, A.: Three-dimensional metamorphosis : a survey, The Visual Computer, pp (1998). [6] Lee, A., Dobkin, D., Sewldens, W. and Schroder, P.: Multiresolution Mesh Morphing, SIGGRAPH 99, pp (1999). [7] Lee, A., Sweldens, W., Schröder, P., Cowsar, L. and Dobkin, D.: MAPS: Multiresolution Adaptive Parameterization of Surfaces, SIGGRAPH 98 Proceedings, pp (1998). [8] Michikawa, T., Kanai, T., Fijita, M. and Chiyokura, H.: Multiresolution Interpolation Meshes, In 9th Pacific Conference on Computer Graphics and Applications,IEEE, pp (2001). [9] Morooka, K. and Nagahashi, H: Self-organizing Deformable Model : A New Method for Fitting Mesh Model to Given Object Surface, International Symposium on Visual Computing, LNCS 3804, pp (2005). [10] Noh, J. and Neumann, U.: Expression cloning, SIG- GRAPH 01 Proceedings, pp (2001). [11] Schölkopf, B., Steinke, F. and Blanz, V.: Object Correspondence as a Machine Learning Problem, Proceedings of 22th International Conference on Machine Learning (2005). [12] Schreiner, J., Asirvsthsm, A., Praun, E. and Hoppe, H.: Inter-Surface Mapping, SIGGRAPH 04 Proceedings, pp (2004). [13] Sumner, R. and Popovic, J.: Deformation Transfer for Triangle Meshes, SIGGRAPH 04 Proceedings, pp (2004). [14] Takahashi, S., Kokojima, Y. and Ohbuchi, R.: Explicit Control of Topological Transitions in Morphing Shapes of 3D Meshes, Proc. Pacific Graphics 2001, pp (2001).