Real-Time Space-Time Blending with Improved User Control

Transcription

1 Real-Time Space-Time Blending with Improved User Control Galina Pasko 1, Denis Kravtsov 2,, and Alexander Pasko 2 1 British Institute of Technology and E-commerce, UK 2 NCCA, Bournemouth University, UK dkravtsov@bmth.ac.uk Abstract. In contrast to existing methods of metamorphosis based on interpolation schemes, space-time blending is a geometric operation of bounded blending performed in the higher-dimensional space. It provides transformations between shapes of different topology without necessarily establishing their alignment or correspondence. The original formulation of space-time blending has several problems: fast uncontrolled transition between shapes within the given time interval, generation of disconnected components, and lack of intuitive user control over the transformation process. We propose several techniques for more intuitive user control for space-time blending. The problem of the fast transition between the shapes is solved by the introduction of additional controllable affine transformations applied to initial objects in space-time. This gives more control to the user. The approach is further extended with the introduction of an additional non-linear deformation operation to the pure space-time blending. The proposed techniques have been implemented and tested within an industrial computer animation system. Moreover, this method can now be employed in real-time applications taking advantage of modern GPUs. 1 Introduction The shape transformations in computer animation and games include linear transformations, non-linear deformations, and metamorphosis or morphing (transformation between two given shapes). We consider the following aspects of the general type metamorphosis problem: two given initial shapes can have arbitrary topology; the alignment or overlapping of the shapes is not required; one-to-one correspondence is not established between the boundary points or other shape features. A brief survey of existing approaches to shape metamorphosis is given in the next section. Implicit surfaces [2] and FRep solids [8] seem to be most suitable for the given task. The solution to the above stated metamorphosis problem has been proposed in [11], where a space-time blending operation was introduced, which is a geometric bounded blending [9] operation in a higher-dimensional space. Bounded blending is a special operation performing blending union set Corresponding author. R. Boulic, Y. Chrysantou, and T. Komura (Eds.): MIG 2010, LNCS 6459, pp , c Springer-Verlag Berlin Heidelberg 2010

2 Real-Time Space-Time Blending with Improved User Control 147 theoretic operation only within a specific area specified by a bounding solid (i.e. the blended shape exists only within some solid). In contrast to other existing metamorphosis operations based on the interpolation, space-time blending is based on increasing the dimensionality of an object, function-based bounded blending [9], and consequent cross-sectioning for animation. The original formulation of space-time blending in [11] has several problems: fast uncontrolled transition between shapes within the given time interval, generation of disconnected components during the metamorphosis and lack of intuitive user control over the transformation process. The problem of fast transition between shapes was addressed in [10] using smoothing of the generalized halfcylinders which undergo bounded blending. Here, we describe several additional techniques aimed to user-controllable space-time blending. First, additional controllable affine transformations are applied to initial objects in space-time. Then, the approach is extended by additional deformation operations in space-time applied to the initial half-cylinders. Both affine transformations and deformations provide more control to the user and help avoid the generation of disconnected components. Further extensions such as automatic generation of control points for deformations, dynamic texturing and non-linear time sampling are briefly discussed. Finally, the described approach to controllable space-time blending is implemented on GPU to allow interactive rendering rates. 2 Other Works The existing approaches to 2D metamorphosis include physically-based methods [15,16], star-skeleton representation [17], warping and distance field interpolation [3], wavelet-based [21] and surface reconstruction methods [18]. A detailed survey on 3D metamorphosis can be found in [7]. Practically all existing approaches are based on one or several of the following assumptions: equivalent topology, polygonal shape representation, shapes alignment and possibility of establishing vertex-to-vertex correspondence. The desired type of behavior can be obtained using skeletal implicit surfaces [20] with manual or automatic establishment of correspondence between scalar field source points. Metamorphosis of more general shapes based on skeletal implicit surfaces can be handled using hierarchical tree structures similar to those used in FRep. Galin et al. [5] proposed to automatically establish correspondence between such tree structures. Trees of the source and the target objects can be matched with additional manual control from the animator. Metamorphosis of arbitrary FRep objects can be described using the linear (for two initial solids) or bilinear (for four initial solids) function interpolation [4], but it can produce poor results for misaligned objects with different topology. Turk and O Brien [19] proposed a more sophisticated approach based on interpolation of surface points (with assigned time coordinates) using radial basis functions in 4D space. This method is applicable to misaligned surfaces with arbitrary topology thus satisfying some of the requirements to the general type metamorphosis mentioned earlier. However, for the initially given implicit surfaces this requires time consuming surface sampling and interpolation steps. Aubert and Bechmann [1] proposed a method allowing

3 148 G. Pasko, D. Kravtsov, and A. Pasko them to produce metamorphosis sequences involving topological changes of the intermediate shapes. This is achieved through the application of a free-form deformation applied to a higher-dimensional space-time object. On the other hand, the topological complexity of the resulting shapes is quite limited, because they have to be produced by the user-controlled FFD of the initial shapes. 3 Metamorphosis Using Space-Time Blending A blending operation in shape modeling is used to generate smooth transition between two given shapes (curves or surfaces). In a number of works the term blending is used to designate metamorphosis of 2D shapes, but we use it here in the way traditional to geometric and solid modeling. Blending set-theoretic operations (intersection, union, and difference) provide an approximation of the exact results of the pure set-theoretic operations by smoothing sharp edges and vertices. In the case of blending union of two disjoint solids with additional material, a single resulting solid with a smooth surface can be obtained. This property of the blending union operation is the basis of the approach to shape metamorphosis called space-time blending [11]. Space-time blending is technically a bounded blending operation [9] performed in space-time on objects defined by continuous real functions. Let us illustrate the approach proposed in [11] step by step for the given 2D shapes (Figs. 1-4): 1. two initial shapes are defined on the xy-plane (union of two disks and a cross in Fig. 1); 2. each shape is considered as a cross-sections of a generalized half-cylinder in 3D space (a generalized half-cylinder bounded by a plane from one side) as it is shown in Fig. 2; 3. the axes of both generalized half-cylinders are parallel to some common straight line in 3D space, for example, to the coordinate z-axis, and the bounding planes of two half-cylinders are placed at some distance to give space for making the blend; 4. apply the added material bounded blending union operation to the halfcylinders (Fig. 3); Fig. 1. Initial 2D shape (union of two disks) and final 2D shape (cross) for metamorphosis Fig. 2. Two generalized half-cylinders with the given 2D shapes as cross-sections

4 Real-Time Space-Time Blending with Improved User Control 149 Fig. 3. Cross-sections of the bounded blending union steps of metamorphosis Fig. 4. Bounded blending union of two generalized half-cylinders. Note the sharp upper and lower edge in the areas of the boundaries of the half-cylinders. 5. adjust parameters of the blend in order to obtain a satisfactory intermediate 2D shape in one or several 2D cross sections by planes orthogonal to z-axis (Fig. 4); 6. considering additional z-coordinate as time, make consequent orthogonal cross-sections along z-axis (Fig. 4) and combine them into 2D animation. As it was mentioned, a bounded blending operation is applied in order to achieve the desired result. The following definition of the bounded blending set-theoretic operation is used: F bb (f 1,f 2,f 3 )=R(f 1,f 2 )+disp bb (f 1,f 2,f 3 ), (1) where R(f 1,f 2 ) is an R-function corresponding to the type of the set-theoretic operation [13,8], the arguments of the operation f 1 (X) andf 2 (X) are defining functions of two initial solids, and disp bb (f 1,f 2,f 3 ) is a displacement function depending on the defining function of the bounding solid f 3 (X). The formulation for blending operations with the blend bounded by an additional bounding solid [9] was used with the following displacement function: where r 2 1(f 1,f 2 )= { disp bb (r) = ( f1 a 1 ) 2 + (1 r 2 ) 3 1+r,r <1 2,r 2 = 0,r 1 ( f2 a 2 ) 2,r 2 2(f 3 )= { r 2 1,r r1 2 2 > 0 +r2 2, (2) 1,r 2 =0 ( ) 2 f3,f 3 > 0 a 3, 0,f 3 0 where the numerical parameters a 1 and a 2 control the blend symmetry, and a 3 allows us to control the influence of the function f 3 on the overall shape of the blend. The application of the bounded blending union to the generalized halfcylinders is illustrated in figs. 3 and 4. The half-cylinders are bounded by the planes z = 1 andz = 1 to make the gap [ 1, 1] along z-axis between them. The bounding solid for the blend in this case is an infinite slab orthogonal to z-axis and defined by the function f 3 as an intersection of two half-spaces with

5 150 G. Pasko, D. Kravtsov, and A. Pasko Fig. 5. Linear metamorphosis of a torus into the union of two cylinders (shown at the left) Fig. 6. Space-time blending of a torus and a union of two cylinders the definitions z 10 and z 10. The blending displacement from the exact union of two generalized half-cylinders takes zero value at the boundaries of the bounding solid (planes z = 10 and z = 10). This operation results in the exact initial 2D shapes obtained at the cross-sections outside the bounding solid. The parameters a 0 - a 3 of the bounded blend influence the blend shape and respectively the shape of the intermediate cross-sections. The main idea of space-time blending between 3D shapes is the same: each shape is considered a 3D cross-section of a 4D half-cylinder defined in space-time. Fig. 5 demonstrates intermediate steps of 3D metamorphosis with the simple linear interpolation between the defining functions and fig. 6 illustrates space-time blending applied to the same objects. 4 Control Techniques for Space-Time Blending The main problem with the applied bounded blending (fig. 4) and the resulting animation (fig. 3) is that the most significant part of the shape transformation happens in the [-1,1] time interval with the intermediate 2D shape changing very fast from the initial to the final cross-section, which appears as a shape jump in the produced animation. The main reason for this jump is that the bounded blending is applied to a generalized half-cylinder bounded by a plane orthogonal to the axis. This set-theoretic subtraction applied to a cylinder results in the sharp edge of the half-cylinder boundary (as seen in fig. 2). This sharp edge remains a significant feature of the blended half-cylinders (see edges at the top and bottom parts of the shape in fig. 4). To avoid the fast transition between the shapes in the given interval, it was proposed in [10] to use smoothing of half-cylinders which undergo bounded blending. Other problems of the original space-time blending are the generation of disconnected components during the metamorphosis and lack of intuitive user control over the process, where changing the parameters of bounded blending is not enough to achieve the desirable effect. To address these problems, we propose to add two more types of time-dependent transformations with their own control parameters: affine transformations (section 4.2) and deformations with control points (section 4.3).

6 Real-Time Space-Time Blending with Improved User Control 151 Fig. 7. Frames of animation based on the bounded blending (note the movement of the intermediate shape in the direction of the final shape and overall smoother transition) Fig. 8. Examples of extracted thick features 4.1 Applying Affine Transformations From the previous figures it can be observed that space-time cylinders are aligned with the time axis. In fact, we can extrude initial 2D shapes along an arbitrary axis in space-time. One way is the interpolation along the axis going through the centers of both objects (fig. 7). This is similar to the first order interpolation of the offsets used to shift both objects. This can be especially useful when both objects are placed at a significant distance from each other. In certain circumstances it is desirable to manipulate the transition from one shape to another in a specific way. For instance, the user may want to align particular features of the shapes or have more control over the intermediate transformation process when sizes of the shapes vary significantly. For these situations, we introduce rotation and scale affine transformations defined along the time axis (figs. 9, 10). This also helps reducing the influence of fast transitions of the defining function values of both objects and thus results in smoother transition. Without these transformations the volume of the intermediate shape needs to be significantly increased to avoid having disjoint components. But increase of the volume leads to even faster transitions between the shapes. Thus affine transformations provide more control over the interim modifications of the shape as well as help reducing the rate of the transition. All required time-dependent affine transformations can be generated automatically based on the estimated bounding shapes of the objects. Alternatively, the user has an opportunity to modify these parameters in order to achieve the desired effect. We can apply the same technique to 3D objects. Fig. 11 demonstrates two initial 3D objects and a number of steps of the transition between them. We have applied affine transformations to adjust the size and location of the intermediate object. 4.2 Applying Additional Deformations For certain models and values of the parameters used for the definition of spacetime blending, it is possible to observe disjoint components of source and destination objects appearing during the transformation process (Fig. 12). One way of resolving this issue is the addition of user controlled deformations. The

7 152 G. Pasko, D. Kravtsov, and A. Pasko Fig. 9. User guided rotation around time axis to align object features Fig. 10. User guided scale along time axis Fig. 11. Space-time blending between 3D shapes (on the left) extended by affine transformations (translation and scaling) appearance of the disconnected component in fig. 12 can be explained by the significant difference in the distances between the initial torus and the final union of two cylinders. The transition can be improved via the introduction of timedependent deformations in addition to the pure space-time blending. We can apply time-dependent deformations in the direction from the source object to the destination object. For the example of fig. 12, this can be done with the help Fig. 12. Disjoint components seen during the metamorphosis using space-time blending Fig. 13. Metamorphosis of a torus into the union of two cylinders using two additional deformations

8 Real-Time Space-Time Blending with Improved User Control 153 of a non-linear space mapping ( warping ) intuitively controlled by two points [14] as illustrated by fig. 13. Although the user can define the control points for the deformation interactively, these points can be also generated automatically based on the properties of the objects being blended. To do so we need to find a set of internal points with extreme values of the defining function. These points are located inside thick features of the model, i.e. the areas situated at the the extreme distances from the object s boundary: D p src = Nsrc i=1 p srci : F src (p srci ) > 0, F src (p srci ) >F src (p srci + p); p > 0 (3) D p dst = N dst j=1 p dstj : F dst (p dstj ) > 0, F dst (p dstj ) >F dst (p dstj + p); p > 0 where D is a set of N points (N src = N dst ) used to define non-linear spacemapping, F src and F dst are defining functions of the source and destination objects respectively. We find the locations of the aforementioned points performing distance transform of the functional object using Euclidean metrics, i.e. performing voxelization of the data set to retrieve a data set containing Euclidean distances to the surface of the resulting object. For every voxel in this dataset we analyze its 26 neighbours. If any of the neighbours is located further from the surface than the current one, we remove the voxel. The resulting voxels satisfy the requirement defined in equation 3. After this initial filtering similar to erosion operation we consider internal voxels with the highest distance value and remove all the voxels in their neighbourhood bounded by the distance to the surface. The desired number of points is then extracted from a subset of points located in the proximity of these thick features of the object. The user can choose the number of points retrieved in this fashion and give a hint of how close to each other he wants the retrieved points to be (fig. 8). The retrieved points are located on the medial surface of the object. Once extreme internal points are retrieved, we need to establish correspondence between the points of the source and destination object. This is be done using the distance metrics and mutual locations of these points. The pairs of points are then used for the definition of the time-dependent deformation. Location of every destination point used in the deformation is defined over time as follows: p dsti (t) =p srci (1 f(t)) + p dsti f(t); f(t) [0; 1] Thus, areas around fixed number of selected local centers of the source object will be shifted over time in the direction of the areas of interest of the destination object. This can help avoiding disjoint components, as we perform further spacetime blend between the objects already modified around local centers. The main advantage of this approach is the opportunity for the user to have more control over the intermediate phases of the metamorphosis. The user can choose which parts of the objects should be modified first and the trajectory of their motion.

9 154 G. Pasko, D. Kravtsov, and A. Pasko Transitions between particular features can be achieved with a set of non-linear deformations applied in arbitrary order. This results in another powerful expressive tool given to the animator. Besides, the balance between the deformation and the pure space-time blending can be found by selecting appropriate time schedules for both processes. The example shown in fig. 13 demonstrates two simple deformations applied to the initial torus. Each deformation is directed towards the center of one of the cylinders and fills intermediate space between the objects, thus resulting in the smoother and more natural transition. 4.3 Dynamic Texturing and Non-linear Time Sampling Information used for affine transformations can also be employed for the improvement of time-dependent texture coordinates generation (Fig. 14). This approach is based on the method for static texture coordinates calculation described in [12]. Color at point p on the surface is defined as C(p, t) = f(n(p),t) T (S xy (p, t)) + g(n(p),t) T (S xz (p, t)) + h(n(p),t) T (S yz (p, t)) where n(p) is the normal evaluated at point p, T (u, v) is the texture color at point (u, v), f,g,h are scalar blending functions based on normal direction and current time value. 2D coordinates used for texture sampling are defined in the following way: S ij (p, t) =( α i (t) p i + dp i (t), α j (t) p j + dp j (t) ) where α is the scaling coefficient, i, j {x, y, z} and dp is the offset retrieved from affine transformations. Employing this information can help to reduce the sliding of the projected textures on the surface and align it with the dimensions of the objects. 5 Implementation and Discussion It is also worth mentioning that any of the aforementioned parameters used to define space-time blending can be made time-dependent. The user can dramatically change the transition varying blending parameters over time, thus adding another degree of control. This, for instance, could be used to increase the influence of source or destination object at any given moment of time or dramatically modify the intermediate object. Due to non-linear nature of defining functions of the objects and the properties of the bounded blending operation, transition between the objects can not be expected to be a linear process. But we can adjust the visual rate of this transition performing non-uniform sampling over time. In the simplest case the time step can be adjusted depending on the estimated change of the area or volume of the shape as follows:

10 Real-Time Space-Time Blending with Improved User Control 155 Table 1. Average frames per second values for different models on an NVIDIA GeForce 8800 Ultra, 768 MB of RAM Grid resolution Ape to torii Pumpkin to coach for polygonization 64x64x x128x perform discretization 2. estimate area or volume 3. estimate dv = area difference or volume difference 4. update time based on dv 5. repeat step 1 Discretization can be done using a simple voxelization or polygonization procedure in 2D or 3D space. It allows us to get an approximate measure of space occupied by the intermediate object. This information can be used to estimate the rate of change of the occupied area/volume at different time instants. The described process helps linearizing the resulting transition adding or removing fixed amount of material at every step. We have further developed the approach to general type shape metamorphosis on the basis of the bounded space-time blending. In this paper, we extended the proposed approach in three directions: additional control of the metamorphosis is introduced using affine transformations for more precise control over the transition between the objects and an additional deformation operation. We have implemented our method and integrated it into a popular animation modeling system Autodesk R Maya R (Fig. 15). It allows us to comfortably work with the model continually receiving visual feedback. We are able to produce 2D and 3D animation sequences in near-real time. This is rather important, as it allows Fig. 14. Pumpkin to coach space-time blending with applied textures Fig. 15. Controlled space-time blending plug-in for Maya system

11 156 G. Pasko, D. Kravtsov, and A. Pasko performing a large number of iterative modifications of the model to achieve visually pleasing results. Finally, parameters of the model are exported as a kernel for NVIDIA R CUDA R SDK. All the evaluations and rendering of the model are performed on the GPU in real-time using the approach described in [6]. The frame rates of dynamic model evaluation are shown in Table 1. These frame rates include the time required for function evaluation in the volume, polygonization and rendering. Model Ape-to-torii (fig. 11) consists of 53 FRep entities, while model Pumpkin-to-coach (fig. 14) is assembled using 65 FRep entities. We believe that this approach can appear to be useful for computer animation and games, as it allows us to produce interesting transition effects between arbitrary 2D/3D objects. The proposed technique can be used to visualise the metamorphosis between complex entities created by the artist or between usergenerated objects in an easy way in real-time. There are a number of issues that require additional consideration. First of all, space-time blending does not by itself guarantee the elimination of unwanted disjoint components. It only provides a method allowing the user to overcome this problem and have more control over the intermediate process of shape transformation. It is thus desirable to extract the topological critical points in space-time. This information can then be employed for the generation of more detailed animation sequences in the interval of the fast shape transitions. This information could also prove to be useful for the creation of a tool simplifying the exploration of 4D and higher dimensional objects. We also plan to research the problem of volumetric attributes transfer in order to apply extended space-time blending to hypervolume objects. Another direction of further research is the application of space-time blending for advanced shape modeling operations such as lofting. References 1. Aubert, F., Bechmann, D.: Animation by deformation of space-time objects. Computer Graphics Forum 16, (1997) 2. Bloomenthal, J.: Introduction to Implicit Surfaces, 1st edn. The Morgan Kaufmann Series in Computer Graphics. Morgan Kaufmann, San Francisco (August 1997) 3. Daniel, C., Levin, D., Solomovici, A.: Contour blending using warp-guided distance field interpolation. In: VIS 1996: Proceedings of the 7th Conference on Visualization 1996, p IEEE Computer Society Press, Los Alamitos (1996) 4. Fausett, E., Pasko, A., Adzhiev, V.: Space-time and higher dimensional modeling for animation. In: CA 2000: Proceedings of the Computer Animation, Washington, DC, USA, p IEEE Computer Society, Los Alamitos (2000) 5. Galin, E., Leclercq, A., Akkouche, S.: Morphing the blobtree. Comput. Graph. Forum 19(4), (2000) 6. Kravtsov, D., Fryazinov, O., Adzhiev, V., Pasko, A., Comninos, P.: Polygonal- Functional Hybrids for Computer Animation and Games. In: GPU Pro: Advanced Rendering Techniques, pp AK Peters Ltd., Wellesley (2010) 7. Lazarus, F., Verroust, A.: Three-dimensional metamorphosis: a survey. The Visual Computer 14(8/9), (1998) 8. Pasko, A., Adzhiev, V., Sourin, A., Savchenko, V.: Function representation in geometric modeling: Concepts, implementation and applications. The Visual Computer (11), (1995)

12 Real-Time Space-Time Blending with Improved User Control Pasko, G., Pasko, A., Kunii, T.: Bounded blending for function-based shape modeling. IEEE Computer Graphics and Applications 25(2), (2005) 10. Pasko, G., Pasko, A., Ikeda, M., Kunii, T.: Advanced Metamorphosis Based on Bounded Space-time Blending. In: MMM 2004: Proceedings of the 10th International Multimedia Modelling Conference, pp IEEE Computer Society, Los Alamitos (2004) 11. Pasko, G., Pasko, A., Kunii, T.: Space-time blending. Computer Animation and Virtual Worlds 15(2), (2004) 12. Peytavie, A., Galin, E., Merillou, S., Grosjean, J.: Arches: a Framework for Modeling Complex Terrains. Computer Graphics Forum (Proceedings of Eurographics) 28(2), (2009) 13. Rvachev, V.: Methods of Logic Algebra in Mathematical Physics. Naukova Dumka, Kiev (in Russian) 14. Schmitt, B., Pasko, A., Schlick, C.: Shape-driven deformations of functionally defined heterogeneous volumetric objects. In: GRAPHITE 2003: Proceedings of the 1st International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, p ACM, New York (2003) 15. Sederberg, T., Greenwood, E.: A physically based approach to 2 d shape blending. In: SIGGRAPH 1992: Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques, pp ACM, New York (1992) 16. Sederberg, T., Gao, P., Wang, G., Mu, H.: 2-d shape blending: an intrinsic solution to the vertex path problem. In: SIGGRAPH 1993: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp ACM, New York (1993) 17. Shapira, M., Rappoport, A.: Shape blending using the star-skeleton representation. IEEE Comput. Graph. Appl. 15(2), (1995) 18. Surazhsky, T., Surazhsky, V., Barequet, G., Tal, A.: Blending polygonal shapes with different topologies. Computers and Graphics 25(1), (2001) 19. Turk, G., O Brien, J.: Shape transformation using variational implicit functions. In: SIGGRAPH 1999: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp (1999) 20. Wyvill, B., McPheeters, C., Wyvill, G.: Animating Soft Objects. The Visual Computer 2(4), (1986) 21. Zhang, Y., Huang, Y.: Wavelet shape blending. The Visual Computer 16(2), (2000)