Object Oriented Discrete Modeling: a Modular Approach for Human Body Simulation

Size: px

Start display at page:

Download "Object Oriented Discrete Modeling: a Modular Approach for Human Body Simulation"

Oliver Russell
6 years ago
Views:

1 Object Oriented Discrete Modeling: a Modular Approach for Human Body Simulation 1 and Stéphane Craighero 2 1 TIMC IMAG CNRS UMR 5525 Institut Albert Bonniot, Faculté de Médecine, La Tronche Grenoble Cedex, France 2 CHU Grenoble Department of Radiology, Hôpital Michallon, La Tronche Grenoble, France Emmanuel.Promayon@imag.Fr Tel: , Fax: , timc.imag.fr/emmanuel.promayon Keywords: dynamic simulation, physically based object, functional modeling, object oriented modeling, soft tissue, respiration. Abstract There are two actual challenges in the simulation of the human body for medical purpose: the ability to model complex structures and their interactions, and the ability to accurately represent reality. In this article we show further developments of a model presented some years ago to address the former and we present some solutions concerning the latter. In this model soft tissues are considered as three dimensional elastic bodies. Bones are considered as three dimensional rigid bodies. All these objects are dynamically moving in a viscous medium and are submitted both to internal cohesive forces and to external attractive or interactional forces. Three examples show the qualitative evaluation of this model. The first example demonstrates our modular approach by building a complex object made of soft tissues, muscles, bones and passive layers. The second example show a more realistic object, where the bones of the arm are displaced by a muscle contraction and relaxation. The third example, for which this model was originally developed, shows a human trunk object, reduced o the essential structures, that could simulate the trunk movements. The remaining of the article look into a more quantitative evaluation of the model. 1. Introduction Two major directions are taken by researchers to model human soft tissues [Del98][]: the biomechanical approach and the computational discrete approach. The biomechanical approach is mainly based on the Mechanics of Continuous Media, and use framework such as the Finite Element Method (FEM). It offers the advantage of being based on a strong theoretical background. The are two kinds of inconvenients when one apply this method to generate the computer aided medical simulation: the computation time cost and the difficulties to build complex assemblage of where rigid, elastic and active structures objects. The first inconvenient was recently being addressed by using pre calculation [Cot99], simplification of complex law [Pic00], or use of simple type of material [Kee99]. Computational discrete approach, such as mass spring networks [Lee95], are often viewed as a fast but not easily controllable or assessable way of building such complex assembly. The extraction of physical parameter such as Young modulus and Poisson coefficient by measuring isolated tissue is possible [Fun93], even if difficult on actually living tissue samples. These measurements are extremely useful to set the model parameters, especially for FEM based method. However, the complexity of tissue interactions and the wide range of possible situations make the backward validation process, i.e. from the simulation back to the real data, becomes difficult Many studies focus on realistic looking aspects of human body or animals simulation in movements [Sch97] [Wil97] [Aub00]. The accuracy of the generated movement are not compared to real movements. As our aim is to include multiple dynamic interactions and properties and to be able to produce real time simulations, the present study is based on a discrete approach. The discrete models, by nature, allows us to exactly decide which phenomenon is taken into account in the simulation. To handle new phenomenon, new processes are to be added in the model core algorithms. However, as the main

2 drawbacks of these models is the control and assessment of parameters, we investigates here two different ways of doing so: qualitatively and quantitatively. 2. Methods This work proposes to model different kind of living structures (tissues, bones) by using the physically based mechanical interactions of individual object with defined mechanical identity. More precisely, living structures are defined as 3D physical object with characteristic sizes, in which mechanic features are based on force balance equations. In the present study, we illustrate the potentiality of such an object oriented approach for modeling anatomical and functional living structures. 2.1 The approach Our hypothesis lead us to add new features to a discrete model developed in [Pro96] and [Pro97]. This discrete model is based on computer graphics modeling [Ter87]. Natural motions and realistic looking flexible and elastic objects are efficiently modeled and simulated by means of physically based computer graphics models. These models use a small amount of data (object geometries and relations between objects). From this, an animation motor (using forces, energies, or direct displacements), integrates movement and deformation laws to compute the evolution of shapes and positions. Generally, constraints are added to control the movement or deformation or to model complex physical properties. [Pro97] modeled the human abdomen using an elastic and incompressible 3D object and the human diaphragm using an elastic and contractile membrane. These objects were linked together as well as with a rigid object, modeling the rib cage. An explicit algorithm was developed to compute the dynamic evolution of these objects as well as to insure the necessary constraints. Incompressibility and connections were considered as constraints. This model, allowing for rigid, incompressible elastic or contractile objects is the groundwork that we extended to model more complex and realistic objects in the present article. The model is basically explained in previous work, we only detail here the object oriented approach and some improvement in the concept or implementation. 2.2 The object oriented framework To model living structures, we need four different kinds of components: rigid components (to model skeleton), deformable components (to model soft tissues), active deformable components (to model muscles), outer and inner layer to represent organs or tissues that are to be taken into account in a particular simulation. Layers components are passives: they follow underlying active structures at a given distance (inward or outward). To be able to model individual behaviors, each organ or tissue is considered as an entity, each of them having its own history, properties and actions. To be able to manipulate such entities computer models and algorithms are needed. Moreover, as each of these objects follows a general scheme, the object oriented programming framework is used. This approach gives a strong structure for the methodological organization of concepts and their relationships. The object oriented framework allows us to describe properties and actions for a whole category of objects, called a class. Each object is thus a particular instance of a particular mold and has an individual behavior. Moreover the object oriented approach allows to organizes classes into a hierarchical representation depending on the properties to be modelled. The behavior laws described in a class could depend on intrinsic properties, interactions, and could change along time; but it could also be a specialization of another class. For example a muscular tissue could be considered as an elastic tissue in which contractile properties are added. Intrinsic properties are mainly dynamic properties such as elasticity, contractility, and incompressibility. But they could also be bio chemical or genetic properties. Interactions between instances could be in relation with the environment, e.g. mobility or motor functioning, or with other instances, e.g. collision and adhesion. Evolution of the laws could also be set. For example a tissue could become less elastic along time. In our model a component is called a region. There are four different kinds of region: (rigid) solid region, elastic region, muscular region and layer region. A set of masses (called elements) control the region surfaces and are organized using triangles; this is similar to a mass spring network (but more efficient in term of stability, as described in [PBP97]). Elements are themselves organized depending on the region and properties they are modeling. Figure 1 shows the organization diagram of our model. This results in a highly parallel organization. 2

3 Figure 1. Simplified UML diagram of the classes used to represent the objects of the model. 2.3 Simulation forces To generate forces and dynamic, a mass is assigned on each element. Forces are exerted on the nodes to generate displacements and deformations. Three kinds of forces can be used in our model: force fields (e.g. gravitation force), locally applied forces (e.g. user manipulation), and Linear Actuator Forces (LAFs). We introduce a LAF when a target position P target is known for a given node P of the contour. To minimize the distance PP target a spring is created between P and P target. It generates a force that linearly attracts P towards P target. The expression of the LAF is: F = k LAF (P P target), where k LAF is the stiffness of the spring. LAFs can be seen as potential forces that tends to minimize a distance (e.g. muscle contraction, elasticity). LAFs permit to model any kind of forces that could be defined by target positions. P target can be dependent on geometry or on constraints. Spring mass networks are known to be difficult to parameter and adjust. Therefore, to model elasticity we define a local shape memory [Pro96]. The elastic property of an object region is simply its ability to come back in its original shape once deformed. To model this property we define a local shape coordinate system where each node is defined relatively to its neighbors by three parameters: α, β and γ. These three scalars are initialized at the rest shape. During the simulation, given the neighbor positions, α, β and γ, a target position is computed for each node. This target position satisfies the local shape and defines a LAF. An elastic object is defined by a contour where the shape memory force is applied to all the elements. The unique elasticity parameter is the stiffness k elasticity used by the corresponding LAF. Another LAF is used to define muscular forces. Once the contraction directions on the muscular region contours are defined, target positions for muscular elements can be defined. The LAF could then be activated by varying the stiffness coefficient during the simulation. 2.4 Constraints and loads Forces are not often sufficient to model complex behaviors. Constraints are added to maintain some conditions like non penetrating area or incompressibility. Our algorithm considers constraints as non quantified force components: they are solved using a direct projection algorithm based on the gradient vector of the constraint function. Thereby it is possible to handle the total incompressibility of a contour [Pro97]. New improvements of the previously published method allows for real time computation of this particular constraint and thus for any kind of triangulated surface. The model is also improved to take into account local forces know in intensity and direction at any time during the dynamic simulation. This allow to add external forces due to the surgeon or its instruments. 2.4 Resolution The following algorithm is used to compute the evolution of the dynamic system from time t to time t+dt: 1. for each region, for each element: compute the sum of applied forces. 2. Use a discrete integration scheme for equation of motion (e.g. Newton Cotes) to compute the new velocity and position of the elements, without considering any constraints. 3. Take local and global constraints into account, i.e. impose given displacements to the elements. 4. Adjust the velocity to include the imposed displacements. 3. Results 3.1 Qualitative evaluation of the model In order to validate the model through qualitative simulation, we present here three examples. The physically based model is first validated using a complex object where all different properties are implied (figure 2). The second example validates the muscle region, the solid region, their links and the incompressibility constraint. It models a biceps muscle contraction (figure 3). The third example focuses on a more complex object that models how the respiratory 3

L i g n e 3 L i g n e 7 0 L i g n e 1 4 6 L i g n e 2 2 6 L i g n e 3 0 6 L i g n e 3 8 6 Int.

4 L i g n e 3 L i g n e 7 0 L i g n e L i g n e L i g n e L i g n e Int. Workshop on Deformable Modeling and Soft Tissue Simulation Bonn, Germany, November 14 15, 2001 movements could be generated using our framework (figure 4). The trunk object has two layer regions representing the skin and the lungs. The choice of using passive regions means that we consider here the skin and the lung deformations as outputs of the model. The respiratory movements generated by the model is shown in Figure Quantitative evaluation of the model To really evaluate this model, it should be compared to real data. The model parameters should be set so that the resulting simulations would be comparable to reality. One thing is to be realistic the other is to reproduce exact simulation. We first studied the elastic properties of our model. In order to achieve this step, our model was used at another scale : we simulated red blood cells (RBC) using a simple elastic region. Mechanical properties of the living red blood cells can be extracted from micro manipulation by optical tweezers, see [Hén99]. The elastic LAF (local shape memory) proved to be able to accurately simulate RBCs [Pro00]. Considering the simulation of respiratory movements, we first tried to evaluate our model and to fit its parameter by comparing its output with data captured by an optoelectronic system (ELITE System, BTS Milan, Italy) synchronized with physiological measurements (lung volume for example) [Car96]. The ELITE system is composed by 4 video cameras which track hemispheric markers attached to the skin. A mesh was build, which deformation was driven by the ELITE data. The trunk model was iteratively register to this mesh (Figure 6). Direct comparison between geometric markers and geometric output of our model is not easy and is not carried on any more for two reasons: the difficulty to compare the skin object to the marker s mesh. The markers are attached on the skin but placed to match underlying anatomical references. Our skin layer is simply an envelop covering our underlying structures. To find the skin elements that correspond to each marker is not always possible. the lack of suitable registration algorithm to transform a generic trunk object to fit real data. To fit a particular subject, the trunk object has to be transformed (scaled, translated and rotated). As the trunk object is composed by elastic as well as rigid articulated bodies, the matching algorithm should be able to process each region depending on their types. The algorithms found in the literature (such as [Cou00]) are not directly suitable. Another approach consists in comparing not the geometry but the physiological or physical measured data with the synthetic generated data. Figure 7 shows the resulting volume/pressure curve observed in the lung during a generated respiratory movement. Note that although the shape looks not so far from a real volume/pressure curve, the resulting hysteresis is not yet acceptable. 4. Conclusions In the purpose of simulating movements or deformations of living structures, some simplifications have to be made. The object oriented approach presented here allows to build complex object and to model different kind of interactions. In order to produce more realistic simulations, one has to focus on how the model generates the movements. In the trunk object, the movements are generated by the diaphragm contraction. To improve the way the modeled diaphragm contracts, the real diaphragm contraction has to be studied in more details. A new Magnetic Resonance Imaging acquisition protocol is being design to reconstruct 3D diaphragm during respiration [Cou01]. The first results demonstrate the feasibility of the protocol to extract diaphragmatic surface movements (see figure 8). The use of these data as an input for our model is currently under study. t=0 t=150 Figure 2. A complex object (a) is made of four regions: a muscular region (left), a rigid region (center), an elastic region (right), and a layer region which envelops everything. The muscle is being contracted and relaxed following a given function that control the muscle behavior. The function is set so that the contraction and relaxation phases last the the same time, as shown by the muscular region volume curve (b). The inside volume is kept constant during simulation, thus the muscle contraction implies volume Volume variations t=150 t t=300 t=300 4

deformations of the remote elastic region. The passive layer region enables us to visualize the resulting global deformation.

Fixed shoulder blade and humerus; contraction and relaxation of the biceps (incompressible muscular region).

The hand is composed by a set bones, linked to the arm with elastic links. Figure 5. Simulation of a respiratory cycle, contraction and relaxation of the diaphragm.

Elevation of the ribs is present as in a normal quiet breathing movement. Figure 4.

5 deformations of the remote elastic region. The passive layer region enables us to visualize the resulting global deformation. Biceps (ellipsoïd) Wrist (7 solid solid) t=0 t=300 t=1000 t=1300 t = Figure 3. Fixed shoulder blade and humerus; contraction and relaxation of the biceps (incompressible muscular region). The simulation gives a satisfactory result for the force transmission between muscle and bones as well as for the hand movement fluidity. The hand is composed by a set bones, linked to the arm with elastic links. Figure 5. Simulation of a respiratory cycle, contraction and relaxation of the diaphragm. The downward movement of the diaphragm and the outward movement of the abdomen are qualitatively satisfactory. Elevation of the ribs is present as in a normal quiet breathing movement. Figure 4. The human trunk object is composed by articulated rigid regions (rib cage), a muscular region (diaphragm), an elastic region (abdomen) and two layer regions (lungs and skin). Each rib modeled by an independent region. The rib cage elasticity is controlled by the elastic properties set at the joints along the sternum. The incompressibility constraint is applied to the compartment composed by the diaphragm and the abdomen. Figure 6. ELITE system (top) and integration of their data in the model (bottom). 5

Figure 7. Volume/Pressure curve observed in the lung object after the simulation of a respiratory movement. Figure 8.

From the MRI images (a) the diaphragm and the rib cage surfaces are reconstructed (b), a diaphragm surface is reconstructed at different respiratory phases (c).

[Car96] [Cha00] a c Carnevali P, Ferrigno G, Aliverti A, Pedotti A, "A new method for 3D optical analysis of chest wall motion", Technology and Health Care, vol 4, pp 43 65, 1996.

b [Cot99] [Cou00] Cotin S, Delingette H, Ayache N, "Real Time Elastic Deformations of Soft Tissues for Surgery Simulation", IEEE Transactions on Visualization and Computer Graphics, 5(1):62 73, 1999.

6 Figure 7. Volume/Pressure curve observed in the lung object after the simulation of a respiratory movement. Figure 8. 3D reconstruction of the diaphragm and the rib cage from echo planar MRI images acquired during breathing. From the MRI images (a) the diaphragm and the rib cage surfaces are reconstructed (b), a diaphragm surface is reconstructed at different respiratory phases (c). References [Aub00] Aubel A and Thalmann D, "Realistic Deformation of Human Body Shapes", proc. Computer Animation and Simulation 2000, pp , Interlaken, [Car96] [Cha00] a c Carnevali P, Ferrigno G, Aliverti A, Pedotti A, "A new method for 3D optical analysis of chest wall motion", Technology and Health Care, vol 4, pp 43 65, Chabanas M and Payan Y, "A 3D Finite Element model of the face for simulation in plastic and maxillo facial surgery", Proc. MICCAI 2000, pp , b [Cot99] [Cou00] Cotin S, Delingette H, Ayache N, "Real Time Elastic Deformations of Soft Tissues for Surgery Simulation", IEEE Transactions on Visualization and Computer Graphics, 5(1):62 73, Couteau B, Payan Y, Lavallée S, "The Mesh Matching algorithm : an automatic 3D mesh generator for finite element structures", Journal of Biomechanics, vol. 33(8): , [Cou01] Coulomb M, Ferretti G, Thony F, Paramelle P J, Craighero S, Bricault Y, "Imagerie du diaphragme et de sa région chez l adulte", feuillets de radiologie, vol 41, pp , in french, [Del98] [Fun93] [Hén99] [Kee99] [Lee95] [Pic00] [Pro96] [Pro97] [Pro00] [Sch97] [Ter87] [Wil97] Delingette H, "Towards realistic soft tissue modeling inmedical simulation", proc. of the IEEE: special issue on virtual and augmented reality in medicine, 86(3): , Fung, "Biomechanics: Mechanical Properties of Living Tissues", New York: Springer Verlag, Hénon S, Lenormand G, Richert A, Gallet F, "A new determination of the shear modulus of the human erythrocyte membrane using optical tweezers", Biophysical Journal, vol 76, pp Keeve E, Girod S, Kikinis R, Girod B, "Deformable Modeling of Facial Tissue for Craniofacial Surgery Simulation", Computer Aided Surgery, John Wiley & Sons Inc., New York, invited paper, 3(5): , Lee Y, Terzopoulos D, Waters K, "Realistic Modeling for Facial Animation", Proc. Computer Graphics, SIGGRAPH 95, vol 29, pp 55 62, Picinbono G., Delingette H., Ayache N., "Real Time Large Displacement Elasticity for Surgery Simulation: Non Linear Tensor Mass Model", Proc. MICCAI 2000, pp , Promayon E, Baconnier P, Puech C. "Physically based deformations constrained in displacements and volume", Proc. Computer Graphic Forum, Eurographics 96, 15(3): , Promayon E, Baconnier P, Puech C, "Physically based model for simulating the human trunk respiration movements", Proc CVRMED II MRCAS III. Lecture notes in Computer Science, vol 1205, pp , Springer Verlag Promayon E., Martiel J. L., Tracqui P. "Physically based 3D simulations of cellular traction forces, migration and cytokinesis of cells adherent to an elastic substratum", SFB Workshop on Simulations of Polymer and Cell Dynamics, Bonn University, june Scheepers F, Parent R, Carlson W, May S, "Anatomy Based Modeling of the Human Musculature", Proc. Computer Graphics, SIGGRAPH 97, vol 31, pp , Terzopoulos D, Platt J, Barr A, Fleischer K. "Elastically Deformable Models". Proc. Computer Graphics, SIGGRAPH 87, 21(4): , Wilhelms J and Van Gelder A, "Anatomically Based Modeling", Computer Graphics, SIGGRAPH 97, vol 31, pp ,

Volumetric Deformable Models for Simulation of Laparoscopic Surgery

Volumetric Deformable Models for Simulation of Laparoscopic Surgery S. Cotin y, H. Delingette y, J.M. Clément z V. Tassetti z, J. Marescaux z, N. Ayache y y INRIA, Epidaure Project 2004, route des Lucioles,