Contents. 1 Introduction 5

Transcription

1 Acknowledgments For this Master Thesis work I had been hosted at the Bioimaging Group 1 led by Professor Claudio Lamberti at the University of Bologna in Italy. He has been my co-supervisor and I thank him for welcoming me in his laboratory and for being always available. I thank Alessandro Sarti who has plethora of ideas, among them that the work of Andreas Wiegmann could be extended for our purposes, and for the constructive and keen discussions with him. It wouldn t have been such a fantastic experience if I didn t experienced good comradeship with my colleagues: Cristiana, Roberto, Andrea and Gian- Luca. I did the development work at the Vis.I.T. lab of CINECA 2. Roberto Gori has been my co-relator at CINECA. He collaborated on the ViSu project and implemented most of the software. I thank him for he has a solution for every problem and he shares easily his mastering of programming. The work has been performed under the Project HPC-EUROPA (RII3- CT ), with the support of the European Community - Research Infrastructure Action under the FP6 Structuring the European Research Area Programme. 3 This Transnational Access European Program to High Performance Computing facilities enabled me to get access to the highperformance computers to carry out this work. The Post-Graduate Course of Bio Medical Engineering 4, is organized by the University of Patras, Greece, in collaboration with twenty other European universities. The courses, taught by outstanding professors coming from all over Europe, gave me a broad vision on Bio Medical Engineering, and a new starting point in life. 1 Bioimaging Group, Biomedical Engineering Laboratory, Department of Electronics, Computer Science and Systems, University of Bologna, Viale Risorgimento 2, I Bologna, Italy 2 Vizualisation and Information Technology Laboratory, CINECA, Inter-University Consortium, via Magnanelli 6/3, I Casalecchio di Reno (Bologna), Italy

2 Contents 1 Introduction 5 2 Previous Work The ViSu project Introduction Benefits Workflow System description Components Results Conclusions Consortium Maxillo-facial surgical planning: other works Zürick group INRIA University of Erlangen-Nuremberg ZIB Maxillo-facial surgical planners Theory Elasticity theory Explicit Jump Immersed Interface Method (EJIIM) EJIIM for Laplacian Interface-to-Grid Jump Distribution EJIIM in Two-Dimensions The Lamé equation in 2D Traction Boundary Conditions in 2D The interface in 2D Normals and tangents to the interface Grid-to-Interface extrapolation of function and derivative values Cross-derivatives

3 5 Extension of EJIIM in Three-Dimensions The Lamé equation in 3D Traction Boundary Conditions The Interface in 3D Normals and tangents to the interface in 3D Grid-to-Interface extrapolation of function and derivative values 54 6 Implementation: Programming of the Solver Prototyping Development Computers Results Preliminary results One-Dimension: Poisson problem Two-Dimensions: Poisson problem Two-Dimensions: Cantilever problem Two-Dimensions: Skin Deformation The Computer Assisted Surgery case: Skin Deformation Prediction Discussion and Conclusions 64 3

4 List of Figures 2.1 Workflow System architecture The PHANToM from Sensable, Inc Components Data storage structure User interface Patients table and report function Interface to the virtual surgery Data Base: all patients data, as CT scans, reconstructed 3D models, planning hypotheses and simulation outputs, can be managed Transfer HTML form Patient browser Eight steps of the surgical planning Example of surgical intervention planning on the skull of a real patient Normal and tangent to the interface D Poisson problem on irregular domain Left hand side matrix Visualization of the left-hand side matrix Cantilever problem D Skin Deformation Planned bone movements (left) and Predicted soft-tissues (right) Validation of the results

5 Chapter 1 Introduction Cranio-facial surgery is a surgical branch regarding study and treatment of any kind of disease (malformations, trauma and tumors) affecting the face. Peculiar to this surgery is that any surgical procedure has not only functional but even aesthetical implications such important for all patients life. Anatomical and functional complexity of the face and skull, characterized by the presence of the eyes, ear, nose, mouth, facial nerve at the proximity of very important organs as the brain and the respiratory system, makes this area extremely hazardous for any skilled surgeon or fellows in training. Computer-based surgery simulation represents a rapidly emerging and increasingly important area that combines information technology and real time systems for the common purpose of improving health care. It allows the surgeon to operate on a virtual representation of the patient head and to predict the outcome of the real intervention. Several virtual interventions can be executed and evaluated before to proceed to the real surgery. A collection of tools has been developed by a research group in Bologna, Italy. Their software, ViSu, enables the surgeon to experiment with different surgical procedures to find the best aesthetical solution; and to give the patient a chance to appreciate the hypothetical outcome of the real surgical intervention. It is presented in Chapter Previous Works, Section: the ViSu project. Other groups contributed to this research field and proposed parallel solutions. There are briefly described in Chapter Previous Works, Section: Other Works. Common to all groups is a software pipelining the acquisition of the patient s image data, the segmentation, the three-dimensional visualization, the definition of osteotomy lines by the surgeon, thanks to the user interface, the forecast of the appearance of the patient after the planned surgery, the tools for post-op validation. The simulation of the soft tissue remodelling (as a consequence of the planned new geometry of the skull) is one crucial step that this paper addresses. We try to improve that step and present a 5

6 new computational model. We keep the same partial differential equations for the elasticity of the skin. But, we correct the boundary conditions at the skin-bone and skin-air boundary, and we improve the way the very irregular domain (i.e. shape) of the skin s face can be handled to solve effectively the equations. The theory for this approach is presented in Chapter Theory. In Chapter 2D we develop the theory for the two-dimensional case. The ideas underneath the so-called Explicit Jump Immersed Interface Method (EJIIM) for solving PDE s in irregular domains is due to Wiegmann. In this Chapter we actually give some clues about how his method can be implemented. In Chapter 3D we extend Wiegmann s theory to the three-dimensional case, while still giving clues about how it can be implemented. The Chapter Implementation simply describes the prototyping and development phases of this project. The programming languages and libraries used are referenced. The Chapter Results is self-explained. The next Chapter gives the Discussion and Conclusions. 6

7 Chapter 2 Previous Work 7

8 2.1 The ViSu project Introduction The ViSu project features an integrated system for computer-aided surgery, with an user-interface based on the mouse or a haptic device used as a virtual bistoury. The main difference with the approach taken by others for this simulation is that the simulation is done directly on the grid of the 3D CT data, having taken an eulerian approach instead of the classical lagrangian. This physically based computational model allows the simulation of soft tissue deformation as a consequence of the surgery planning that is, predicts the post-surgical aspect of the patients face. Along comes an interactive user interface providing a way for the surgeon to handle the data s, trace osteotomy lines for the planning, and visualize the results Benefits The key-points of the system are: a physically based computational model allowing the simulation of soft tissue remodelling as a consequence of the planned new geometry of the skull. This module allows simulating and predicting the post-surgical aspect of the patients face, working directly on the 3D CT data; an interactive user interface for surgery planning, improving the 3D data manipulation to provide a more natural way to the surgeon to execute the virtual operation. numerical/visual tools for results visualisation and for comparison between prediction and post-operation CT data for error evaluation; portable software modules suitable both for the integration within a PACS server in a hospital environment or even into the graphics workstation, as a standalone solution; data handler modules to keep track of all the data involved in the planning, and perform data compression for speeding up the data exchange in case of a distributed framework; transparent access to patients data (input, output, and parameters of the simulation) using a web application, that allows surgeons to easily compare different planning hypotheses and share the results. The benefits of VISU system are, for the surgeon: it enables to experiment with different surgical procedures, improving the quality of the intervention, reducing operation time and forecasting any surgical complication; 8

9 it provides young surgeons a useful tool to learn surgical practice and to train their skills; for the patient it allows the patient to appreciate an hypothetical result of the surgical intervention. for the hospital it enhances the quality of the health care service Workflow CT images acquisition A 3D CT scan is acquired from the patient before the surgery. The acquisition is performed employing a High Speed Spiral CT using parameters and modalities specified in a suitable acquisition protocol that we have defined according with Rizzoli Hospitals Radiology Unit directives. More specificaly, data consist of images in the ACR-NEMA DICOM3 format. Access to patient data A consistent interface to different DBMS and PACS protocols is provided. CT scan data, parameters defining planning hypothesis (osteotomies) and simulation output are handled. Tissue segmentation, classification and 3D reconstruction A set of automatic tools was developed to process 2D real patients CT slices and obtain 3D volumes necessary for surgery simulation. By greyscale thresholding, soft and hard tissues can be easily differentiated and classified. About 120 slices are interpolated and volumes are obtained. Surgical Planning A 3D graphical interface allows the user to interact directly with hard and soft tissues. Using mouse and keyboard and, optionally, a force feedback 3D virtual scalpel (haptic device), osteotomy lines can be traced and anatomical regions can be moved and relocated according to the established surgical procedure. Numerical Simulation A physically based simulation kernel computes the soft tissue deformation caused by the new geometry of bones. Output Visualization The original 3D model of the face is warped following the physically based displacement field obtained by numerical simulation. 9

10 Figure 2.1: Workflow System description IT and medical requirements As shown in 2.2, VISU is a distributed system composed by several components: a Web server, an application server and several data input clients and simulations clients. The actual geographical distribution of VISU components is shown. A Data Input Client needs access to a Hospital PACS to retrieve patients information and CT studies. Data is transferred to a server component through the Internet and imported by surgeons on a simulation client. This implementation allows different ways of system installation, such as a minimal all-in-one set-up, acquiring the server and providing it with simulation functionalities, or an outsourcing form, displacing web services outside the clinical structure. A this moment there are a data input client connected to the PACS of the Rizzoli Hospital, a Web server and an application server at CINECA and three simulation clients respectively at Bufalini Hospital, CINECA and DEIS. System requirements Simulation Client 10

11 Pentium III-IV, 1-2 Gb RAM graphic card OpenGL 32/64 Mb RAM haptic device PHANTOM Data Input Client low memory capacity low computing power high bandwidth Web Server / Application Server medium memory capacity medium computing power high disk capacity high bandwidth Performance Requested time for data transfer: 6 min (bandwidth 8 Mbit/s) Requested time for intervention simulation: 15 min Figure 2.2: System architecture 11

12 Input visualization and surgical planning A 3D graphical interface allows the user to interact directly with the hard and soft tissues reconstructed models. The code has been written in Tcl/Tk and based on the Vtk library to be cross-platform. Either a mouse, or a virtual bistoury can be used to interact with the 3D CT data. Osteotomy lines can be traced and anatomical regions can be moved and relocated according to the established surgical procedure. Relocations are quantified in terms of translational and rotational parameters. At the end of the surgical planning a new geometry of bones is hypothesized. We have developed an interface, written in C++, to replace the mouse by a haptic device used as a virtual bistoury. The introduction of the haptic tool allows the surgeon to simulate intervention on a computer screen or virtual environment with a sense of presence in the real surgical environment. The hardware device is a PHANToM desktop by Sensable Inc.which provides 6 degrees of freedom positional sensing and 3 degrees of freedom force feedback. These technical specifications meet widely maxillo-facial surgeons requirements, the end medical user of our application. Since PHANToM device is in the form of a stylus that can be moved on an x, y and z axis, we thought to use it as a real bistoury, and use the button in a similar way to a mouse button, for activating or picking up target bones. Figure 2.3: The PHANToM from Sensable, Inc. 12

13 Numerical Simulation Description of tissues visco-elastic behavior is a crucial element to estimate deformations after external interventions. A physically based simulation kernel computes the soft tissue deformation caused by the new geometry of bones. The displacements of hard tissues (bones) are imposed in the planning phase. The displacements of soft tissues are modelled by classical continuous mechanics equations. The equation system is discretized using centered finite difference schemes respect to the natural grid of the CT image itself, avoiding any need of regridding and mesh tuning. The associated matrix is solved using iterative and multiscale methods. The result of the simulation is the displacement vector field associated to every voxel of the CT image.boundary conditions are defined to not affect displacement of soft tissues. Description of tissue viscoelastic behaviour is a crucial element to estimate deformations after external interventions. A physically based simulation kernel computes the soft tissue deformation caused by the new geometry of bones. The displacements of hard tissues (bones) are imposed in the planning phase. The displacements of soft tissues are modelled by classical continuous mechanics equations: µ 2 u(x) + (µ + λ) ( u(x)) = 0 for x soft tissue 2 u(x) = 0 for x air u(x) = u 0 (x) for x hard tissues du(x) dn = 0 for x volume boundary (2.1) The equation system is discretized using centered finite difference schemes respect to the natural grid of the CT image itself, avoiding any need of re-gridding and mesh tuning. The associated matrix is solved using both iterative and multi-scale methods Components Concerning the architecture we have developed a modular structure to transfer, manage and retrieve VISU information. The software components developed for this aim show a high level of portability, scalability and reemployment and has been implemented using freely available libraries, instruments and languages. Figure 2.4: Structure of VISU system and the connections between components. The architecture follows the typical three-tier model: data flow moves on a client-server- DBMS line. VISU system allows keeping an independent data archive, creating minimum impact on hospital organization. Developers also managed security problems and personal data treatment, according to privacy laws. The system administrator can define a group of operators (such as physicians, assistants and radiologists) able to access the 13

14 information held inside the archive and to transfer CT studies. In fact, a table inside a database, inaccessible by external clients, containing usernames and passwords and different access rights has been created. System modularity allows implementing more sophisticated identification procedures. Possible errors on either the server or the network are foreseen: in case of archive manipulation by a user, misfunctions or error conditions, a recovery procedure which restores the initial situation, leaving the archive in a consistent state, is provided. Moreover, each change operated on the archive is registered inside a log file, containing also useful information about connected clients. Figure 2.4: Components Figure 2.5: VISU data storage structure. The Patients table is the root of the hierarchy and provides a record, containing personal data and kind of pathology, for each patient. The Exams table contains a record for each CT study. Technical details related to the CT study, saving directory and name of the file containing the data volume for the simulation are included in this table. The Interventions table provides information about surgical interventions performed on patients. The Segmentations table contains data related to the regions the surgeon needs to relocate. The Simulations table provides information about surgical planning. The Parameters table shows the values of the roto-translation matrix for each region involved by the operation. The data input client block represents the application used to acquire CT studies, stored inside the hospital PACS as a group of DICOM images, and to transfer them to the VISU server in anonymous form and compressed in a single zip file. This client has been developed using Tcl/Tk multi-platform language and shows a simple and intuitive interface to supply 14

15 query/retrieve and move functionality on a DICOM archive. Figure 2.5: Data storage structure Figure 2.6: shows the main window of the application as it appears to the user, and 2.6, the Tcl/Tk user interface. The zip file transfer can be achieved as a CORBA service located on the Application Server. The request arguments are personal data and username and password for user authentication. The useful commands that realize query/retrieve and move operations and DICOM file modification are provided by the freeware packages DICOM Toolkit by OFFIS and Central Test Node by Mallinckrodt Institute of Radiology. The database has been implemented using MySQL DBMS. For more efficiency, data are not grouped in BLOB fields, but directly inside the file system. The Application Server contains a simple CORBA server, written in Java, able to accept requests from remote clients. Intruding client who connects to this port, still cannot send data because a keyword exchange is performed between client and server. Database updating is achieved easily and safely using JDBC-ODBC Java drivers on both Application Server and Web Server. As Web Server we chose Apache Web Server running with Jakarta Tomcat Servlet Container, both freeware. Each service is written in Java language. The first web page runs the user into a form for access authentication. Reviewing and organizing basic clinical data and different hypothesis of intervention are possible through an interactive presentation. Dynamic HTML pages display VISU database tables, also sorting records by a desired criterion and asking compound queries. Figure 2.7: Patients table (left) and report function (right). A report service (Figure 2.7, right) is provided: patients data and CT images, processed with graphic filters, are shown and, if needed, printed. If a VRML 15

16 Figure 2.6: User interface browser is available, the user can also display the 3D models of soft and hard tissues and a 3D animation that shows tissue remodeling. All the data relative to an intervention planning are handled: input CT scans, all the parameters defining the planning hypotheses (osteotomies) and simulation outputs as shown in Figure 2.8. The Interface to the virtual surgery Data Base shows all patients data, as CT scans, reconstructed 3D models, planning hypotheses and simulation outputs, that can be managed. Figure 2.7: Patients table and report function Another service allows to transfer CT studies and update VISU archive: the user is asked to fill in a HTML form, shown in Figure 2.9, containing patient s personal data, data of the exam and the zip file containing DI- COM images. The code does not require changes if further data need to be transferred in the HTML form in the future. To prevent insertion mistakes 16

17 Figure 2.8: Interface to the virtual surgery Data Base: all patients data, as CT scans, reconstructed 3D models, planning hypotheses and simulation outputs, can be managed. and omissions, a check on sent data has been prepared: on the client side, a script in Javascript language and, on the server side, the code itself realizes the check on the form. This second way assures correctness also for clients who do not employ a browser to send a HTTP request. The simulation client provides to the user all the facilities to perform the surgical simulation. One key component of the user interface is the data front-end, able to help the surgeon to keep track of all the data involved in the surgical simulation. The main task of this component is to provide a consistent interface to the different DBMS or PACS protocol. Figure 2.10: The user can browse through the patient data and to select input data, that are locally cached, to avoid unnecessary network transmission (all the data are time stamped). All the data are be stored and transferred in compressed form. Of course security, privacy and more in general private data managing ethical aspects, always relevant in medical applications, are kept in necessary consideration as described in the previous section. By selecting suitable value of CT parameters, soft tissue and bone tissue can be separated. We developed a set of automatic and semi-automatic tools to process 2D real patients CT slices and obtain 3D volumes necessary for surgery simulation. A 3D graphical interface allows the user to interact directly with the hard and soft tissues reconstructed models. The code has been written in Tcl/Tk and based on the Vtk library to be cross-platform. 17

18 Figure 2.9: Transfer HTML form Figure 2.10: Patient browser 18

19 Using a force feedback 3D virtual reality scalpel (Haptic device) or simply a mouse, osteotomies lines can be traced and anatomical regions can be moved and relocated according to the established surgical procedure (see Figure 2.11). Relocations are quantified in terms of translation and rotational parameters. At the end of the surgical planning a new geometry of bones is hypothesized. The surgeon up to now has traced osteotomy lines, cutting off the part involved in the operation. We are substituting the mouse with a virtual scalpel. Thus, doing these incisions, the surgeon will be able to feel and touch bones as with a real scalpel. The software paints interested zone and pressing button on the stylus, he will be able to move and relocate bones. Figure 2.11: Eight steps of the surgical planning. We have developed a C++ library (called GHOSTVtk) to replace the 19

20 mouse by a haptic device used as a virtual blade. The introduction of the haptic tool allows the surgeon to simulate intervention on a computer screen or virtual environment with a sense of presence in the real surgical environment. The hardware device is a PHANToM desktop by Sensable Inc., which provides 6 degrees of freedom positional sensing and 3 degrees of freedom force feedback. These technical specifications meet widely maxillofacial surgeons requirements, the end medical user of our application. Since PHANToM device is in the form of a stylus that can be moved on an x, y and z axis, we thought to use it as a real bistoury, and use the button in a similar way to a mouse button, for activating or picking up target bones Results The result of the simulation is the displacement vector field associated to each voxel of the CT image. The original CT image is warped following the physically based displacement field obtained by numerical simulation. The new hypothetical appearance of the patient can be visualized in the same graphical interface. Figure 2.12: Example of surgical intervention planning on the skull of a real patient. Figure 2.12: Top left: From CT data, hard tissues and soft tissues 3D models are reconstructed. Right: Bones have been selected, cut and relocated. In this phase the operator can experiment different surgical procedure. Down left: After planning, numerical simulation provides prediction 20

21 Patients Procedures 9 Le Fort I maxillary advancement 6 Le Fort I high maxillary advancement extended to zigoma Bilateral sagittal split osteotomy Genioplasty (2 cases) Mandibular border osteotomy 3 Bilateral sagittal split osteotomy Genioplasty (1 case) 1 Le Fort III 1 Le Fort III, fronto orbital advancement Table 2.1: Surgical Procedures of patients faces final aspect by simulating the mechanical behavior of real soft tissues (remodelling) in response to bones replacement. The application of our approach for modelling the elastic deformation of human tissue in response to movement of bones has been tested both on the Visible Human Data Set of the National Library of Medicine and on the CT datasets of real patients. The set of patients involved in our research is for 50 females and 50 males, aged between 10 and 37 with average age of 25.1 years, affected by cranio-facial deformities. The surgeons performed several kinds of procedures. For the validation of the application, no extra CT acquisitions have been required, because pre- and post-operative exams were just suitable. Table 2.1 shows the validated cases. The surgery simulation system has been tested for usability in the daily clinical routine and, in spite of simplicity of the physical model, a study for evaluation and improvement of accuracy and reliability carried out that outputs of the system are consistent with the established tolerance. The accuracy and repeatability of the simulation procedure has been performed following a suitable validation procedure developed in accordance with surgeons. The system accuracy is defined as the ability to simulate correctly the surgery result. It was computed as the percent ratio between the number of simulations for which the simulation error was less than a threshold and the total number of simulations performed. The threshold was set as the distance between two CT slices during the reconstruction process (2mm). The simulation error was computed as the difference between the simulation result and the surgery result and it was obtained comparing the simulated and the post-operative CT soft tissues surfaces. The system repeatability is the extent to which the surgery simulation yields the same results on repeated trials. It was computed as the percent ratio between the total number of simulations for which the planning error was less than the threshold and the total number of simulations performed. As for the computation of the system accuracy, the threshold was set to 2mm. The planning error 21

22 was computed as the difference between the planning result and the real result and it was obtained comparing the planned and the post-operative CT hard tissues surfaces. The first step of the validation procedure was the rigid registration of the pre- and post-operative CT volumes. The algorithm we utilized was based on mutual information and it was applied to sub-volumes not involved in the surgery. The obtained roto-translation matrix was applied to post-operative surfaces to get the reference system of the pre-operative CT data and of the simulated ones. The error computation was based on Hausdorff distance computation obtained by horizontal slicing between geometric models (simulated and post-operative, obtained from CT data set acquired from the patient at least 3 months after surgery and planned and post-operative), relating to the interested region. More than the 80resulted lower than the 2 mm established maximum tolerance Conclusions A complete system has been developed, able to be used in a clinical routine to plan maxillo-facial surgical intervention. The proposed structure, through the development of a portable, scalable and progressive computational kernel, embedded in a distributed architecture and in a portable user interface, allows to build a real time surgical planning tool with reasonably low HW requirements. Moreover a cross-platform SW has been used. The 80% of the studied cases resulted consistent with the validation procedure established in accordance with clinicians. The integration of a distributed data management component allow to further reduce the global time required for the complete surgical planning, enabling surgeons to quickly evaluate different intervention hypotheses, and helping them in the data acquisition and archiving tasks. For this reason a distributed web interface has been developed into virtual surgery environment, to make the application distributed and suitable for clinicians involved in. Actually the mean overall time required for planning and simulating an intervention has decreased down to minutes Consortium The consortium was formed by: Department of Electronics, Computer Science and Systems (DEIS) of the University of Bologna: a research and teaching excellence centre in Electronics, Automatic Control, Telecommunications, Computer Science, Operation Research and Biomedical Engineering. CINECA SuperComputer Center: a Consortium consisting of 18 Italian Universities and CNR - National Research Council. It promotes the use of the most advanced computing systems to support public and 22

23 private scientific and technological research, providing a computer processing service to all members of the Consortium and to the Ministry for Education, University and Research. Azienda Ospedaliera Bufalini di Cesena: Cesena s sanitary complex satisfies the request of sanitary and social safety expressed by the components of the local community and by everyone requiring its services, assuring essential and personalized interventions, oriented to expedience and innovation. Siemens Medical Solutions Health Services Italia: It offers the most comprehensive suite of information solutions to the healthcare industry worldwide. They provide a complete portfolio of application, consulting, and technology services to help customers improve their quality of care, productivity, and financial performance. 23

24 2.2 Maxillo-facial surgical planning: other works Zürick group The Swiss Federal Institute of Technology (ETH) and the University Hospital of Zürich are collaborating on facial surgery simulation, using finite elements. In [14], Koch, Gross et al. propose a simulator in which the geometric modelling is done with the animation software Alias 1, in order to produce tetrahedra s. Data s are volumetric (CT scan of the head) and surfacic (range laser scan of the face). The equation is a spring-mass model, in which they differentiate the stiffness coefficient based on the underlying anatomical structure (bone, skin, muscle, dark fat, bright fat). Boundary conditions, i.e. rigid and non-rigid areas, are differentiated manually, by an interactive painting program. Similarly, stretching and bending parameters have different values in different parts of the face. In [11] and [12], they describe better the worklow of their approach, comparing it to the traditional approach for surgical planning and followup. A somehow qualitative validation is presented in this paper. The error distribution in term of radial distance between simulated and real postsurgical surface, acquired by a laser scan, in displayed in a figure with a color code representing the distance. It seems that they achieved 3 millimeters. Roth, Gross, et al. propose an extension of linear elasticity towards incompressibility and nonlinear material behavior, in [18]. In [13], the planner distant itself from its initial surgical purpose, to become an Emotion Editor. Thanks to the Facial Action Coding System (FACS) and their implementation of the muscle model, they are able to simulate expressions such as laughter or sadness INRIA Cottin, Delingette, Ayache and other researchers of INRIA Sophia-Antipolis, France, work on topics related to surgical training and simulation, such as real-time cutting, deformations and force feed-back. In [4], they illustrate the potential of general modelling tools for surgery simulation, using maxillofacial surgery as a testbed. No conclusion is given regarding the accuracy of the prediction as it was not the goal of the work University of Erlangen-Nuremberg Keeve, Girod, Teschner, at the Telecommunications Laboratory of the University of Erlangen; Keeve, now at the Center of Advanced European Studies and Research, cæsar, Germany; and some other researcher involving the Surgical Planning Lab,Department of Radiology, Brigham and Women s Hospital, Harvard. 1 Alias Systems, a division of Silicon Graphics Limited 24

25 [9] and recently [15] presents two different tissue models integrated in an interactive surgical simulation system, which allow the preoperative visualization of the patient s postoperative appearance. They combine a reconstruction of the patient s skull from computer tomography with a laser scan of the skin. They took both approaches of mass spring and finite elements for calculating the impact of a procedure on the skin deformation. A somehow subjective color-code validation is presented in [5] The performance topic is tackled in [23], with a simulation time for the mass-spring approach claimed to be less than 10 seconds. About the FEM approach, it would be 10.5 minutes for a specific examples, in 1996, on a SGI High Impact workstation, presnted in [10]. The initial configuration would be 47 seconds. 2 The interactive aspect on the planning (with spring-mass) is emphasized in [22] ZIB The CAS project of the Zuse Institute Berlin, Germany, summarizes as follow: import of medical image data in DICOM format tissue classification and segmentation with subvoxel accuracy reconstruction of topological correct 3D surface models generation of tetrahedral grids of arbitrary and adaptive resolution diagnosis and cephalometric analysis using 2D/3D visualization techniques interactive osteotomy planning on 3D models interactive rearrangement of bony structures 3D prediction of the facial tissue using adaptive FE methods A exhaustive list of related publication is available on the web-site of the group [8]. The numerical Finite Elements approach for the simulation is presented by Zachow, Gladiline et al. in [28]. The mechanical model is the Lamé equation. The facial tissue simulation for the prediction took 5 minutes. A quantitative validation is done very recently in [19]. In this paper, Zachow, Hierl et al. compare the alignment of the Pre- and the Postoperative Skull, by the Hausdorf distance [17]. They compare the deviation between 2 What the initial configuration comprises is not clear. It is too short to be the simulation time required by the surgeon to trace the osteotomy lines. The tessellation step (i.e. initial meshing), which should be the most time-consuming process, is not described. 25

26 predicted skin surface and postoperative result for a homogeneous tissue model with varying POISSON ratio for two patients, for one part. They compare the mean prediction error for an inhomogeneous tissue model with varying POISSON ratios and elastic moduli (differentiated for muscle and for the embedding tissue), for another part. Overall, and depending on the parameters and methods used, they find a mean prediction error of varying from 1 to 1.5mm Maxillo-facial surgical planners Other groups developed software tools for maxillo-facial surgery that do not actually include a soft-tissue engine able to predict appearance after surgery. In most cases this feature is part of their future plans. The FU-Berlin 3 developed MeVisTo-Jaw [7], which is a collection of tools for 3D cephalometrical analysis. A virtual reality environment is built from the patients volume dataset; the surgeon defines bone segments from skull and jaw bones, employing visual and force-feedback devices to define subvolumes; parameters are transferable to the Operating Room. Neumann, Siebert et al. explains the force-feed back system for bone cutting in [16], using a joystick-like device. 3 Department of Medical Informatics, University Hospital Benjamin Franklin, Free University of Berlin 26

27 Chapter 3 Theory 27

28 3.1 Elasticity theory This section introduces the partial differential equation that model the soft tissue, in its displacement formulation, known as the Lamé equation. The basis for its establishment are simple physical laws and continuum mechanics. We start from a linear elastic model for an isotropic, non-uniform, quasi-incompressible material. Let a body occupy a space S. When the body is deformed, every particle takes up a new position. For example, a particle P with original coordinates P = P(a i ) is moved to the place Q = Q(x i ), i = 1, 2, 3. The displacement vector u is then defined as u i = x i a i (3.1) Due to the impenetrability of matter the mapping M : A X is continuous and one-to-one, therefore it has unique inverse a i = a i (x j ), i = 1, 2, 3, j = 1, 2, 3, (3.2) in the whole domain, and the displacement can be rewritten as u i (x j ) = x i a i (x j ) (3.3) Consider an infinitesimal line element connecting the point P(a i ) to a point P (a i +da i ). The square of the infinitesimal Euclidean norm-2 between the points is given by ds 2 0 = δ ij a i x l a j x m dx l dx m, (3.4) where δ ij is the Kronecker delta. The difference between the deformed and the original Euclidean distance is then given by ( ) ds 2 ds 2 a α a β 0 = δ ij δ αβ dx l dx m, (3.5) x i x j and this allows us to define the Cauchy strain tensor e ij = 1 ( ) a α a β δ ij δ αβ. (3.6) 2 x i x j It is easy to check that the strain tensor can be given in terms of the displacement vector and after some manipulations it becomes e ij = 1 ( uj + u i u ) α u α. (3.7) 2 x i x j x i x j If the displacement is very small the square and products of partial derivatives of u i are negligible with respect to the linear terms, and the strain tensor reduces to e ij = 1 ( uj + u ) i. (3.8) 2 x i x j 28

29 The relationship between the strain and the related stress is a property of the material and it is described by a constitutive equation σ ij = Φ(e kl ). (3.9) If the stress tensor is linearly proportional to the strain tensor, the stressstrain relationship is given by the tensor of the elastic constants C ij, that is a symmetric tensor with rank 4 and 81 elements (3 4 ). In this case the constitutive equation takes the name of Hookean elastic solid σ ij = C ijkl (e kl ). (3.10) If the material is isotropic, i.e. when the elastic properties are identical in all directions, the σ e relationship reduces to the simple form σ ij = λe αα δ ij + 2µ(e ij ), (3.11) where the constants µ and λ are called Lamé constants. By classical continuum mechanics results the equation of motion of a solid body with density ρ is given by ρ du i dt = σ ij x ij + F i (3.12) and it is obtained as a combination of the continuity equation, the momentum equation and Newtons law. The vector F represents the sum of the external forces acting on the body. The equation of motion for isotropic Hookean solids can be obtained by substituting in 3.12 the corresponding stressstrain relationship If we consider small displacements the strain is expressed in terms of the displacements by 3.8 and finally we obtain ρ du dt = µ 2 u + (λ + µ) ( u) + F. (3.13) Since we are interested in the steady state solution of 3.12 we consider null the time derivatives and write the system of coupled equations of the static equilibrium for an isotropic elastic material in cartesian coordinates: µ 2 u + (λ + µ) ( (u, v, w)) + F x = 0 µ 2 v + (λ + µ) ( (u, v, w)) + F y = 0 µ 2 w + (λ + µ) ( (u, v, w)) + F z = 0 (3.14) where u = u(x), v = v(x), w = w(x) are the displacements of the point x = (x, y, z) in the x, y, and z directions. 29

30 3.2 Explicit Jump Immersed Interface Method (EJIIM) Standard finite differences approximations fails when applied to irregular domains because they do not resolve correctly the interface, that is, usually, the boundary between the domain of computation and the rectangular grid. One idea to overcome this problem is to use instead finite elements, but in this case considerable efforts are to be spend on constructing the mesh. In case the grid construction is not affordable, it has been proposed to correct the finite difference stencil in the neighborhood of the interface. This is the key idea of the Explicit-Jump Immersed Interface Methods, due to Wiegmann who presented it in his PhD thesis [26] and in a joint article with Bube [27]. Standard differences, in the O(h 2 ) approximations, could be u x (x i, y j ) u(x i+1, y j ) u(x i 1, y j ) 2h (3.15) u(x i, y j ) u(x i+1, y j ) + u(x i 1, y j ) + u(x i, y j+1 ) + u(x i, y j 1 ) 4u(x i, y j ) h 2 (3.16) or, in compact way, using a stencil notation, u x (x i, y j ) 1 [ ] (3.17) 2h u(x i, y j ) 1 1 h (3.18) 1 In case that an interface cuts through the stencil used by standard differences, we can either modify the stencil, or add corrective terms. Let s introduce first the notation for jumps in a function u and its derivatives at a point α: [u (m) ] α = lim x α u(m) (x) lim + x α u(m) (x) (3.19) Derived and additional notations are: [ ] = [ ] α, [u (1) ] = [u x ], [u (2) ] = [u xx ],..., and by u (0) we mean the function u itself. Lemma 1 Interface occur at α = 0, which lies between two arbitrarily located grid points (labelled here h and h + ) with grid spacing h. We have h + = h + h. Let x j α < x j+1, h = x j α, h + = x j+1 α. Suppose u continuous on the left and on the right of the interface, and continuously derivable up the desired order. So, for u C 4 [x j h, α) C 4 (α, x j+1 + h], 30

31 the following approximations hold to O(h 2 ): u x (x j ) u(x j+1) u(x j 1 ) 2h u x (x j+1 ) u(x j+2) u(x j ) 2h 1 2h 1 2h u xx (x j ) u(x j+1) 2u(x j ) + u(x j 1 ) h 2 1 h 2 u xx (x j+1 ) u(x j+2) 2u(x j+1 ) + u(x j ) h h 2 2 (h + ) m [u (m) ] (3.20) m! m=0 2 (h ) m [u (m) ] (3.21) m! m=0 3 (h + ) m [u (m) ] (3.22) m! m=0 3 (h ) m [u (m) ] (3.23) m! We now have enough material to solve the u xx = f equation on D R, where R = [0, 1] and D = [α 1, α 2 ]. We wish to discretize the Laplacian with zero Dirichlet boundary condition on the whole rectangle R, introducing jumps so that the solution is actually zeros on [0, α 1 ) and (α 2, 1]. It can m=0 be done by imposing the following conditions 1 : g 1 = [u] α1 = u α 1, since u = 0 in D. The g jumps are unknown but can g 2 = [u] α2 = u α 2 be approximated using one-sided extrapolation. We discretize the interval [0, 1] with n + 1 uniformly spaced points (meshwidth h = 1/n). j and k are chosen such as x j α 1 < x j+1 < x k α 2 < x k+1. We write out the linear system ( A Ψ D T I )( U g ) = ( F 0 ) and (3.24) The vector U contains the grid variables [x 1, x 2,...,x n 1 ] T. The vector of jumps g is [g 1, g 2 ] T. A is the discretization of the Laplacian, with: 2 1 A = h (3.25) The right hand side F = [0,...,0, f(x j+1 ), f(x j+2 ),...,f(x k 1 ), f(x k ), 0,...,0] T (3.26) The Ψ n 1 2 matrix holds the two column vectors distributing the g jumps on the grid. For g 1 it is: Ψ α1 = [0, 0,...,0, h+ h 2, h+ h 2, 0,...,0, 0]T (3.27) 1 The sign in u α simply says that the value of u at α is taken inside the domain D. 31

32 with the non-zero entries at j and j + 1. The D2 n 1 T matrix holds the grid-to-interface extrapolation coefficients, given for instance by one-sided (mono-variate) quadratic approximation. I is the appropriately dimensioned identity matrix. 32

33 3.2.1 EJIIM for Laplacian Standard differences, in the O(h 2 ) approximations, for the Laplacian, is: u(x i, y j ) u(x i+1, y j ) + u(x i 1, y j ) + u(x i, y j+1 ) + u(x i, y j 1 ) 4u(x i, y j ) h 2 (3.28) or, in compact way, using a stencil notation, u(x i, y j ) 1 1 h (3.29) Interface-to-Grid Jump Distribution In the second derivative approximation, u xx (x j ) u(x j+1) 2u(x j ) + u(x j 1 ) h 2 1 h 2 u xx (x j+1 ) u(x j+2) 2u(x j+1 ) + u(x j ) h h 2 3 (h + ) m [u (m) ] (3.30) m! m=0 3 (h ) m [u (m) ] (3.31) m! m=0 the first term enters the discretization of the second derivative (or 1D Laplacian), that is, the A matrix. In the second term, [u (m) ] are the jumps, and enter the g matrix. They may be known or unknown. The rest, that is the distribution of the jump coefficients onto the grid, enter the Ψ matrix. Let s develop the summation, and get u xx (x j ) u(x j+1) 2u(x j ) + u(x j 1 ) h 2 1 ( h 2 [u] + h + [u x ] + (h+ ) 2 2 [u xx ] + (h+ ) 3 6 ) [u xxx ] (3.32) u xx (x j+1 ) u(x j+2) 2u(x j+1 ) + u(x j ) h ( h 2 [u] + h [u x ] + (h ) 2 2 [u xx ] + (h+ ) 3 6 ) [u xxx ] (3.33) which hold to O(h 2 ). We may want to drop the higher correction terms if we can afford a lower order correction. The u xx (u) = 0 equation, with an interface occurring as before between x j and x j+1 is discretized as AU + Ψ 0 g 0 + Ψ 1 g 1 = 0, (3.34) 33

34 with the correction up to only the first derivative. A is the discretization of the Laplacian, with: 2 1 A = h (3.35) It s dimension are n 1 n 1 if the [0, 1] interval has been discretized with n 1 point. Thus h = 1/n 1. The Ψ 0 and Ψ 1 matrices are a all-zero vectors expect in j and j + 1 Ψ 0 = [0, 0,...,0, 1 h 2, 1 h 2,0,...,0, 0]T (3.36) Ψ 1 = [0, 0,...,0, h+ h 2, h h 2, 0,...,0, 0]T (3.37) If we set a parameter 0 α < 1 which says where between x j and x j+1 the interface is cut (often 0.5), and define α = h+ h (1 α) = h h (3.38) (3.39) and remember n 1 = 1/h, we write Ψ 0 = [0, 0,...,0, n 2 1, n 2 1, 0,...,0, 0] T (3.40) Ψ 1 = [0, 0,...,0, αn 1, (1 α)n 1, 0,...,0, 0] T (3.41) 34

35 Chapter 4 EJIIM in Two-Dimensions We aim at providing a numerical solver for a particular elliptic interface problem, here the Lamé partial differential equation. The solver can be break into four components: 1. Jump conditions Partial differential equations, Differential geometry 2. Interface and mesh geometry 3. Finite Difference EJIIM 4. The solution of the linear systems Numerical linear algebra 35

36 4.1 The Lamé equation in 2D Recall the Lamé equation, that is, the Navier-Stokes equation in its displacement formulation: µ 2 u + (µ + λ) ( U) = f (4.1) where µ and λ are the Lamé constants, u = (u, v) T and f = (f u, f v ) T. Setting c = µ λ+µ and f = 1 λ+µ f, the equation becomes c 2 u + (µ + λ) ( U) = f (4.2) We can develop it a bit further: ( ) ( c 2 u u + (µ + λ) v x + v ) ( ) fu = y f v (4.3) Or fully, { c 2 u + 2 u x v x y = f u c 2 v + 2 u x y + 2 v y 2 = f v (4.4) The equation 5.4 is discretised and, in matrix-form, looks like: ( )( ) ( ) c + Dxx D xy U F u = D xy c + D yy V F v In matrix compact form: (4.5) AU = F 1 (4.6) The complete system, with the Explicit Jump formulation, where Ψ distributes jumps on the interface, D T performs grid-to-interface interpolation, J holds the jumps coefficients, is: ( ) ( ) ( ) A Ψ U F1 D T = (4.7) I J Some definitions F 2 Divergence Gradient Laplacian div(f) F = F x x + F y y (4.8) grad(f) f = ( f x, f y )T (4.9) 2 f 2 f x f y 2 (4.10) 36

37 4.2 Traction Boundary Conditions in 2D Traction boundary conditions are more easily expressed in the local coordinates of the boundary, i.e. derivatives in the normal and tangents directions. The traction boundary condition couples the three Cartesian displacement variables. Jumps in local coordinates Recall the traction boundary condition σ(u)n = g on Γ 2 Ω (4.11) In Cartesian coordinates, u = (u, v) T is the vector of displacements in the x and y directions, σ is the stress tensor expressed in (x, y) coordinates, n = (n 1, n 2 ) is the inward normal to the boundary (given in (x, y) coordinates) and g is a vector of surface forces applied at that boundary, also given in (x, y) coordinates. We form a (n,t) right hand coordinate system, in which t = (t 1, t 2 ) = ( n 2, n 1 ) is tangents to the boundary. The displacements in the new local coordinate system are ξ = u n η = u t We think of gradient as row vectors, so ( ) ux u u = y v x v y and use the notation ( ) ( ) ξn η n (u n)n (u t)n = = (u n)t (u t)t ξ t η t ( n T ( u)n n T ) ( u)t t T ( u)n t T ( u)t (4.12) The (symmetric) stress tensor of the Lamé equation, with µ and λ the Lamé constants, is 1 u + ut σ = µ + λtr( u)i (4.13) 2 One may rewrite the Traction Boundary conditions in these local coordinates that are implied by the geometry of the boundary and change with the boundary. That is ( ) { ( ) ( )} ( ) g n 2ξn ξ = µ t + η n ξn + η + λ t 0 1 g t ξ t + η n 2η t 0 ξ n + η t 0 1 tr() is the Trace of the matrix, i.e. only its diagonal elements appear 37

38 E Eν (1+ν)(1 2ν) Using µ = 2(1+ν) and λ =, where E and ν are the Elasticity coefficient and the Poisson coefficient, respectively, it can be expressed as a system of two equations E 2(1 + ν) 2ξ n + Eν (1 + ν)(1 2ν) (ξ n + η t ) = g n E 2(1 + ν) (ξ t + η n ) = g t (4.14) Solving the system for ξ n and η n, i.e. the displacement variable in local coordinates, in the normal direction we get ξ n = g ξ ν 1 ν η t (4.15) η n = g η ξ t The vector g = ( g ξ, g η ) T is the vector of applied forces in the local coordinates of the boundary. In case of Traction-Free boundary conditions it will be null; otherwise it is { g ξ g η = (1+ν)(1 2ν) E(1 ν) g n = 2(1+ν) E g t (4.16) We now have a complete set of jump conditions for Traction on Γ 2, in local coordinates. There are two for displacements (one per dimension) and four for the first derivatives: [u] = u [v] = v (4.17) [ξ n ] = ξ n = g ξ + ν 1 ν η t [η n ] = η n = g η + ξ t [ξ t ] = ξ t (4.18) [η t ] = η t The jump conditions in first derivatives can be seen in matrix-form: ( ) [ξn ] [η n ] ( g ξ g = η ) ( ν + 1 ν η t ξ ) t [ξ t ] [η t ] 0 0 ξ t η t (4.19) Jumps in Cartesian coordinates Equation 5.12 can be used for coordinate transformation of jumps: ( )( ) ( ) n T [ux ] [u y ] (n,t,k ) [ξn ] [η = n ] [v x ] [v y ] [ξ t ] [η t ] t T Or, using the orthonormality of (n,t): ( ) [ux ] [u y ] = ( n,t ) ( )( ) [ξ n ] [η n ] n T [v x ] [v y ] [ξ t ] [η t ] t T (4.20) (4.21) 38

39 Summary for implementation of the Traction boundary condition Thanks for the following definitions: the matrix of first-derivative jumps in Cartesian coordinates, ( ) [ux ] [u [U] y ] (4.22) [v x ] [v y ] the matrix of first-derivative jumps in local coordinates, ( ) [ξn ] [η [X] n ] [ξ t ] [η t ] (4.23) the input matrix of function-derivatives at the interface, in Cartesian coordinates (found by Lagrange Polynomial Interpolation for example), ( U u x u ) y vx vy (4.24) the input matrix made of the normal and tangents vectors to the boundary, ( ) n1 t N 1 (4.25) n 2 t 2 the a priori unknown matrix of function-derivatives at the interface, in local coordinates, ( X ξ n ηn ) ξt ηt (4.26) Then, X = N T U N (4.27) The jumps in first derivatives, for the Traction boundary conditions, are: [X] Traction = ( 1 0 ) ( g ξ, g η) + ( ) X + ( 1 0 ) ( ν 1 ν η t, ) ξ t (4.28) Then the solution [U] Traction = N [X] Traction N T (4.29) 39

40 4.3 The interface in 2D The problem has to be solved on a domain Ω, a subset of the rectangular domain R (Ω R). The interface is Ω. The geometry of interfaces, or boundaries, can be described by splines, or by level set methods which is very convenient is case of moving boundaries problems. Both had been used in 2D. An easiest approach is to simply use piecewise linear approximations. For a segmented image, the domain Ω correspond a certain value (let s say 1); the embedding domain R\Ω to another one (let s say 0). One representation of the interface between the 0 s and the 1 s is to encode all [0 1] and [1 0] transitions in each direction (x and y ). To simplify, we decide that the interface cuts the grid at exactly one half between the 0 and the 1. That relative distance we call α and we set it s value to 0.5. Keeping it as a parameter ensure possible extension of the method, for some other particular application. Thus, the interface geometry is a table containing the coordinates of the outside pixels (of value 0), the coordinates of the inside pixels (of value 1) and α. Implicitly, it also says if the transition occur in x or y, but we prefer to keep two separate tables, so that other algorithms are easier. If there are more than two areas (receiving segmentation values of, let s say, 2, 3...), then we consider other tables with the appropriate transitions. For ViSu, the rectangle (parallelepiped) is the medical data-set, a CT of the cranial region. The domain of computation, the skin, or soft-tissue deformable body, on with the partial differential equation of elasticity has to be applied, has a value of 1. The air, or embedding domain 0. The nonmoving bones 2. The moving bones 3 or more, if they consist of different part. To summarize: 0: air 1: soft-tissue 2: non-moving hard-tissue 3+: moving hard-tissues Transitions of interest are, along with their implied boundary conditions (BC): [0 1] Traction-free BC [2 1] Zero-Dirichlet (displacement) BC [3 1] Dirichlet (displacement) BC 40

41 Y T j-1 j j+1 X i-1 N i Figure 4.1: Normal and tangent to the interface 4.4 Normals and tangents to the interface Each intersection between the interface and the grid occurs on a vertical or horizontal mesh line. We consider the six neighbor pixels, as in 4.1. Normalizing, summing and averaging the vectors traced between the interface point and each domain-pixel gives us a good approximation of the normal to the interface at this point. The tangent is then simply T = ( n 2, n 1 ) if N = (n 1, n 2 ). 41

42 4.5 Grid-to-Interface extrapolation of function and derivative values When the value of a function (or its derivatives) is not known at an interface point, we can extrapolate it (them) from the function value of the neighbors. We present here a weighted least square fit of a bi-variate quadratic approximation. We may wish to consider cubic approximation for a third order method, but a second order one is sufficient for this application. Even, there will be some cases where there are not enough points available, so we must revert to a simple linear, first order approximation, which is the minimum we can afford to approximate the first derivatives we need. The approximation will be implicitly one-sided because we usually want the solution to be zero outside the domain. This is a general method to impose boundary conditions and jump conditions on non-grid-aligned boundaries and interfaces. Tri-variate approximation will be used for the three-dimensional case. We denote the coordinates of the boundary-mesh intersection by (x α, y α ) and a generic marked grid point by (x i, y j ). A marked grid point is a grid point which will enter the approximation of the function value or derivatives. We select all grid points which are not more than two pixels away than the interface point, and only those which belong to the domain. That is, we select (x i, y j ) S D R such as the subset S is a 5 by 5 box centered on the grid point closest to the interface point, D is the domain, R is the whole rectangular grid. The restriction S selects the values U S of a grid function U at the marked grid points. Let h i = x i x α and k j = y j y α. The bi-quadratic polynomial p should satisfy u ij = p(x i, y j ), where p(x i, y j ) = p 0 + p 1 h i + p 2 k j + p 3 h 2 i + p 4h i k j + p 5 kj 2, for each of the marked grid point. Over 6 grid points, the linear system is of course over-determined, and we use a least square fit. To allow better approximation, we introduce weights, whose values are higher closer to the interface point to be extrapolated. We use for the weights, w ij = 1/(1 + d ij ) where d ij = h 2 i + k2 j, the distance to the center of S. The weighted least square problem for the coefficients of p is then min p n l wij(l) 2 (p(x i(l), y j(l) ) u ij(l) ) 2 (4.30) l=1 where n l is the number of marked points and the minimum is taken over all bi-quadratic polynomial p. Letting W = diag((w ij(l) ) n l l=1 ), P = [p 0, p 1,...,p 5 ] T and using the n l 10 matrix M with rows corresponding to grid points, i.e. l th row M l = [1, h i(l), k j(l), h 2 i(l), h i(l)k j(l), kj(l) 2 ], (4.31) we find, for a given grid function U, the coefficients of the weighted least 42

43 squares fit polynomial P = (M T W 2 M) 1 M T W 2 SU (4.32) Since p was derived with the origin (x α, y α ), the function value and derivatives of U at the boundary point are approximated as follow: u(x α, y α ) = p 0 + O(h 3 ) u x (x α, y α ) = p 1 + O(h 2 ) u y (x α, y α ) = p 2 + O(h 2 ) u xx (x α, y α ) = 2p 3 + O(h) u xy (x α, y α ) = p 4 + O(h) u yy (x α, y α ) = 2p 5 + O(h) (4.33) Using rows of the matrix D = S(M T W 2 M) 1 M T W 2 S (4.34) where S = diag(1, 1, 1, 2, 1, 2), we can express conveniently different kind of boundary conditions. For example, the Dirichlet boundary condition, for unknown jumps in u, [u] = u will be expressed as D 1. Another example, a condition on the directional derivative normal to the boundary will be written as n 1 D 2 + n 2 D 3, where N = (n 1, n 2 ) is the unit normal to that boundary point. 43

44 4.6 Cross-derivatives The approximation for cross derivatives u xy {u(x i+1, y j+1 ) u(x i 1, y j+1 )} {u(x i+1, y j 1 ) u(x i 1, y j 1 )} 4h 2 (4.35) is identical to u xy {u(x i+1, y j+1 ) u(x i+1, y j 1 )} {u(x i 1, y j+1 ) u(x i 1, y j 1 )} 4h 2 (4.36) if the function u is smooth. This is of course correct if we assume a uniform mesh with x = y = h. If we compute the differences in y first, then u y (x i+1, y j ) u(x i+1, y j+1 ) u(x i+1, y j 1 ) 2h u y (x i 1, y j ) u(x i 1, y j+1 ) u(x i 1, y j 1 ) 2h (4.37) (4.38) it will have to be corrected via Lemma XXX. If, instead we compute the difference in x first, the approximations for u x (x i, y j+1 ) and u x (x i, y j 1 ) will need to be corrected via Lemma XXX. To avoid giving preference to either one or the other direction, we compute the average between the two approaches u xy (x i, y j ) u(x i+1,y j+1 ) u(x i 1,y j+1 ) 2h u(x i+1,y j 1 ) u(x i 1,y j 1 ) 2h 2h u(x i+1,y j+1 ) u(x i+1,y j 1 ) 2h u(x i 1,y j+1 ) u(x i 1,y j 1 ) 2h 2h (4.39) In the case of smooth function we simply add the same terms, but in the presence of an interface, different corrections will be required. Instead of trying to find the corrections for one particular grid point, which is dependent on the geometry of the interface close-by, we wish to determine the influence of one (grid-boundary) intersection on the grid. Every intersection will always affect six grid points: the two on each side of the grid segment which is intersected, and the four closest. Their relative geometry will depend on whether the intersection occurs on a horizontal or vertical mesh line. For example, an intersection α occurs between (x i, y j ) and (x i+1, y j ), so along a x mesh line (a jump in x ). The discretization of 2 / x y will affect (x i, y j ) and (x i+1, y j ) as outer differences, and (x i, y j 1 ), (x i, y j+1 ), (x i+1, y j 1 ) and (x i+1, y j+1 ) as inner differences. Let s have an interface point α and its six neighbors. The grid point a 44

45 will be affected like u xy (x i, y j ) = u xy (a) u(x i+1,y j+1 ) u(x i 1,y j+1 ) 2h u(x i+1,y j 1 ) u(x i 1,y j 1 ) 2h { 2h u(xi+1,y j+1 ) u(x i+1,y j 1 ) 2h u(x i 1,y j+1 ) u(x i 1,y j 1 ) 2h 2h 1 2h 2 (h + ) m m=0 m! [ m+1 ] } u y x m α (4.40) and the grid point d similarly by { u xy (x i+1, y j ) = u xy (d) 1 u(xi+2,y j+1 ) u(x i,y j+1 ) 2h u(x i+2,y j 1 ) u(x i,y j 1 ) 2h 2 2h 1 2 (h + ) m [ m+1 ] } u 2h m! y x m α m=0 u(x i+2,y j+1 ) u(x i2,y j 1 ) 2h u(x i,y j+1 ) u(x i,y j 1 ) 2h 2h (4.41) The corrections are called outer differences because they are related to the second differentiation. For the point b, the correction is (dropping the standard differences and using the stencil notation instead): u xy (x i, y j 1 ) = u xy (b) (b) h 2 3 (h + ) m m=0 m! [ m ] u x m α (4.42) We find identical corrections for c, e and f. However the sign of the correction changes, depending if the grid point lies with the domain Ω (minus sign) or outside, in Ω + (plus sign). These are the inner differences because they are related to the first differentiation. Implementation of Cross-Derivatives Remembering that n 1 = 1/h and n 2 = 1/k, dropping the correction terms higher than first derivatives, and rewriting for non-uniform meshes, we get, for jumps in x Ψ xy,a = 1 4 n 2[u y ] Ψ xy,d = 1 4 n 2[u y ] Ψ xy,b = 1 8 n 1n 2 [u] αn 2[u x ] Ψ xy,c = 1 8 n 1n 2 [u] αn 2[u x ] Ψ xy,e = 1 8 n 1n 2 [u] αn 2[u x ] Ψ xy,f = 1 8 n 1n 2 [u] αn 2[u x ] 45

46 where 0 α < 1, the relative distance between the intersection and the related grid point. Similarly, for jumps in y we get Ψ xy,a = 1 4 n 1[u x ] Ψ xy,d = 1 4 n 1[u x ] Ψ xy,b = 1 8 n 1n 2 [u] αn 1[u y ] Ψ xy,c = 1 8 n 1n 2 [u] αn 1[u y ] Ψ xy,e = 1 8 n 1n 2 [u] αn 1[u y ] Ψ xy,f = 1 8 n 1n 2 [u] αn 1[u y ] Ψ here too is a sparse matrix but has six entries per column, and as many column as there are interface points. 46

47 Chapter 5 Extension of EJIIM in Three-Dimensions 47

48 5.1 The Lamé equation in 3D Recall the Lamé equation, that is, the Navier-Stokes equation in its displacement formulation: µ 2 u + (µ + λ) ( U) = f (5.1) where µ and λ are the Lamé constants, u = (u, v, w) T and f = (f u, f v, f w ) T. Setting c = µ λ+µ and f = 1 λ+µ f, the equation becomes c 2 u + (µ + λ) ( U) = f (5.2) We can develop it a bit further: u ( c 2 v u + (µ + λ) x + v y + w ) = z w Or fully, c 2 u + 2 u + 2 v x 2 x y + w2 x z = f u c 2 v + 2 u x y + 2 v + w2 y 2 y z = f v c 2 w + 2 u x z + 2 v y z + w2 f u f v f w (5.3) z 2 = f w (5.4) The equation 5.4 is discretised and, in matrix-form, looks like: c + D xx D xy D zz U F u D xy c + D yy D yz V = F v D xz D yz c + D zz W F w In matrix compact form: (5.5) AU = F 1 (5.6) The complete system, with the Explicit Jump formulation, where Ψ distributes jumps on the interface, D T performs grid-to-interface interpolation, J holds the jumps coefficients, is: ( ) ( ) ( ) A Ψ U F1 D T = (5.7) I J Some definitions F 2 Divergence Gradient Laplacian div(f) F = F x x + F y y + F z z (5.8) grad(f) f = ( f x, f y, f z )T (5.9) 2 f 2 f x f y f z 2 (5.10) 48

49 5.2 Traction Boundary Conditions Traction boundary conditions are more easily expressed in the local coordinates of the boundary, i.e. derivatives in the normal and tangents directions. The traction boundary condition couples the three Cartesian displacement variables. Jumps in local coordinates Recall the traction boundary condition σ(u)n = g on Γ 2 Ω (5.11) In Cartesian coordinates, u = (u, v, w) T is the vector of displacements in the x, y and z directions, σ is the stress tensor expressed in (x, y, z) coordinates, n = (n 1, n 2, n 3 ) is the inward normal to the boundary (given in (x, y, z) coordinates) and g is a vector of surface forces applied at that boundary, also given in (x, y, z) coordinates. We form a (n,t,k) right hand coordinate system, in which t = (t 1, t 2, t 3 ) and k = (k 1, k 2, k 3 ) are two tangents to the boundary. The displacements in the new local coordinate system are ξ = u n η = u t χ = u k We think of gradient as row vectors, so u x u y u z u = v x v y v z w x w y w z and use the notation ξ n η n χ n ξ t η t χ t = ξ k η k χ k = (u n)n (u t)n (u k)n (u n)t (u t)t (u k)t (u n)k (u t)k (u k)k n T ( u)n n T ( u)t n T ( u)k t T ( u)n t T ( u)t t T ( u)k k T ( u)n k T ( t)n k T ( u)k (5.12) The (symmetric) stress tensor of the Lamé equation, with µ and λ the Lamé constants, is 1 u + ut σ = µ + λtr( u)i (5.13) 2 1 tr() is the Trace of the matrix, i.e. only its diagonal elements appear 49

50 One may rewrite the Traction Boundary conditions in these local coordinates that are implied by the geometry of the boundary and change with the boundary. That is g n g t g k + λ = µ E 2ξ n ξ t + η n ξ k + χ n ξ t + η n 2η t η k + χ t χ n + ξ k χ t + η k 2χ k ξ n + η t + χ k ξ n + η t + χ k ξ n + η t + χ k Eν Using µ = 2(1+ν) and λ = (1+ν)(1 2ν), where E and ν are the Elasticity coefficient and the Poisson coefficient, respectively, it can be expressed as a system of three equations E 2(1+ν) 2ξ Eν n + (1+ν)(1 2ν) (ξ n + η t + χ k ) = g n E 2(1+ν) (ξ t + η n ) = g t (5.14) E 2(1+ν) (χ n + ξ k ) = g k Solving the system for ξ n, η n and χ n, i.e. the displacement variable in local coordinates, in the normal direction we get ξ n = g ξ ν 1 ν (η t + χ k ) η n = g η ξ t (5.15) χ n = g χ ξ k The vector g = ( g ξ, g η, g χ ) T is the vector of applied forces in the local coordinates of the boundary. In case of Traction-Free boundary conditions it will be null; otherwise it is g ξ g η g χ = (1+ν)(1 2ν) E(1 ν) g n = 2(1+ν) E g t (5.16) = 2(1+ν) E g k We now have a complete set of jump conditions for Traction on Γ 2, in local coordinates. There are three for displacements (one per dimension) and nine for the first derivatives: [u] = u [v] = v (5.17) [w] = w

51 [ξ n ] = ξ n = g ξ + ν 1 ν (η t + χ k ) [η n ] = η n = g η + ξ t [χ n ] = χ n = g χ + ξ k [ξ t ] = ξ t [η t ] = η t [χ t ] = χ t [ξ k ] = ξ k [η k ] = η k [χ k ] = χ k The jump conditions in first derivatives can be seen in matrix-form: [ξ n ] [η n ] [χ n ] g ξ g η g χ [ξ t ] [η t ] [χ t ] = [ξ k ] [η k ] [χ k ] Jumps in Cartesian coordinates ν 1 ν (η t + χ k ) ξt ξ t ηt ξ k χ t ξ k η k χ k (5.18) (5.19) Equation 5.12 can be used for coordinate transformation of jumps: n T [u x ] [u y ] [u z ] [ξ n ] [η n ] [χ n ] t T [v x ] [v y ] [v z ] (n,t,k) = [ξ t ] [η t ] [χ t ] (5.20) k T [w x ] [w y ] [w z ] [ξ k ] [η k ] [χ k ] Or, using the orthonormality of (n,t,k): [u x ] [u y ] [u z ] [v x ] [v y ] [v z ] [w x ] [w y ] [w z ] = (n,t,k) [ξ n ] [η n ] [χ n ] [ξ t ] [η t ] [χ t ] [ξ k ] [η k ] [χ k ] n T t T k T (5.21) Summary for implementation of the Traction boundary condition Thanks for the following definitions: the matrix of first-derivative jumps in Cartesian coordinates, [U] [u x ] [u y ] [u z ] [v x ] [v y ] [v z ] [w x ] [w y ] [w z ] (5.22) the matrix of first-derivative jumps in local coordinates, [ξ n ] [η n ] [χ n ] [X] [ξ t ] [η t ] [χ t ] (5.23) [ξ k ] [η k ] [χ k ] 51

52 the input matrix of function-derivatives at the interface, in Cartesian coordinates found by Lagrange Polynomial Interpolation for example, U u x u y u z vx vy vz wx wy wz (5.24) the input matrix made of the normal and tangents vectors to the boundary, n 1 t 1 k 1 N n 2 t 2 k 2 (5.25) n 3 t 3 k 3 the a priori unknown matrix of function-derivatives at the interface, in local coordinates, ξ X n ηn χ n ξt ηt χ t (5.26) ξ k η k χ k Then, X = N T U N (5.27) The jumps in first derivatives, for the Traction boundary conditions, are: ( 1 ) ) [X] Traction = 0 ( g ξ, g η, g χ ) + X 0 Then the solution + ( ( ) ( ν 1 ν (η t + χ k ), ξ t, ξ k ) (5.28) [U] Traction = N [X] Traction N T (5.29) 52

53 5.3 The Interface in 3D As for 2D, the interface is represented by piecewise linear segments. There will be now three directions (x, y and z ). Let s take an example. A patient who undergoes a surgery with only one moving bone will have a segmented dataset containing: 0 in the air, 1 in the soft-tissues, 2 in the non-moving bones, and 3 in the moving bones. The types of transition that have to be kept are: [0 1] for the Skin-Air interface. The Boundary Condition which will be associated with this interface will be a Traction-Free BC. [1 2] for the Skin-Non moving bone interface, with a zero-dirichlet BC. [1 3] for the Skin-Moving bone interface, with a Dirichlet BC. For each type of transition, there will be a sub-category for the direction. So here, a [0 1] in x table, a [0 1] in y, then z ; a [1 2] in x ; and so on. In each of these tables is stored the coordinates of the interface point. The normal and tangents to the interface at that point will be embedded. 5.4 Normals and tangents to the interface in 3D The interface is not any more a curve but a surface, in 3D. Actually, since we have chosen piecewise segments of lines to characterize the interface, our surface will also be piecewise. It is not described explicitly, only by the twodimensional transitions. In determining the normal to an interface point, we consider the neighbors voxels of that point, trace vectors between the point and the outside voxels (the 1 ones) and average. Thus the normal direction to the interface is found. A cross product with an arbitrary vector gives a tangent tho the interface. A cross-product between the normal and the first tangent gives a second tangent. 53

54 5.5 Grid-to-Interface extrapolation of function and derivative values When the value of a function (or its derivatives) is not known at an interface point, we can extrapolate it (them) from the function value of the neighbors. Similarly to 2D, we present here a weighted least square fit of a tri-variate quadratic approximation. If (x, y, z) α is the interface point and (x, y, z) i,j,k are the coordinates of points around α, the equation for the least square fit is the polynomial p(x, y, z) = p 0 +p 1 h+p 2 k+p 3 l+p 4 h 2 +p 5 k 2 +p 6 l 2 +p 7 hk+p 8 hl+p 9 kl (5.30) where h i = x i x α, k j = y j y α and l k = z j z α (we got rid of the indices in the equation for simplicity). The points around α is a subset of the whole dataset, that is, a 5 by 5 by 5 centered on the interface point. Each point must satisfy the equation so, if there are m points, there will be m equations. We must ensure that m is greater than 10 so that there will be a value for each parameter p i. Since p was derived with the origin (x α, y α ), the function value and derivatives of U at the boundary point are approximated as follow: u(x α, y α, z α ) = p 0 + O(h 3 ) u x (x α, y α, z α ) = p 1 + O(h 2 ) u y (x α, y α, z α ) = p 2 + O(h 2 ) u z (x α, y α, z α ) = p 3 + O(h 2 ) u xx (x α, y α, z α ) = 2p 4 + O(h) (5.31) u yy (x α, y α, z α ) = 2p 5 + O(h) u zz (x α, y α, z α ) = 2p 6 + O(h) u xy (x α, y α, z α ) = p 7 + O(h) u xz (x α, y α, z α ) = p 8 + O(h) u yz (x α, y α, z α ) = p 9 + O(h) We use the same matrix approach than in 2D. 54

55 Chapter 6 Implementation: Programming of the Solver 6.1 Prototyping The theory for the explicit jump immersed interface method was tested using Matlab, The Mathworks, Inc.[24] Matlab, The Language of Technical Computing, is a high-level technical computing language and interactive environment for algorithm development, data visualization, data analysis, and numerical computation. This language was chosen for it allows to test rapidly ideas. The syntax is simple and everything necessary for matrix computation and linear solving is available, without the need of external libraries. One-dimensional and twodimensional EJIIM were tested with this. However three-dimensional EJIIM would soon require large memory, for holding the matrices, not talking about the requirements for solving the equations. One disadvantage of Matlab under Windows is that parameters are passed by value in functions, not by pointer. As a result, the usage of the memory is not optimal, and our application could not be extended in 3D. 6.2 Development For best implementation of our solver, we chose C++, the language of Scientific Programming by excellence. For one part the rest of the ViSu software had already been developed in this language, with VTK[25] as the graphic library. PETSc, the Portable, Extensible Toolkit for Scientific Computation[2, 1, 3], was used for solving the linear system, so we kept the same. PETSc is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modelled by partial differential equations. It employs the MPI standard for all message-passing communication. 55

56 For small matrix computations, we used MTL[20], The Matrix Template Library, which has been implemented as a C++ STL[21] (Standard Template Library). 6.3 Computers The Matlab tests and the coding has been done on standard personal computers. The computation for the 3D deformations had been carried out by a IBM Cluster 1350 CLX 1, whilst it was actually not necessary. Italy 1 at CINECA the Italian Inter-University Consortium, Casalecchio di Reno (BO), 56

57 Chapter 7 Results 7.1 Preliminary results This section deals about diverse tests, performed at each stage of the development, to verify that the code was working well One-Dimension: Poisson problem Figure 7.1: 1D Poisson problem on irregular domain We ran the Matlab scripts with the following Partial Differential Equations: { d 2 dx 2 u(x) = f(x) on [a 1, a 2 ] u(x) = 0 on [0, a 1 ) and (a 2, 1] (7.1) 57

58 and the following Dirichlet Boundary Conditions: { u(a1 ) = u l u(a 2 ) = u r (7.2) The Figure 7.1 shows the results with f = 10, a 1 = 0.33, a 2 = 0.66, a l = 5 and a r = 7. The interval [0, 1] is discretized uniformly by 100 spaced points. Although the equation had been discretized over the whole interval, the Boundary Conditions ensure that the solution is null outside the domain, as expected Two-Dimensions: Poisson problem For the two dimensions, we run the same Poisson equation, 2 u = f, with Dirichlet boundary conditions. Here the jump is given in the normal direction to the interface, in local coordinates. As expected, Figure 7.2 shows a plateau inside the domain (for f = 0 that is in fact, a Laplace equation), and zero solution outside Figure 7.2: Left hand side matrix In Figure 7.3 the shape of the left hand side matrix for this problem. The regular discretization of the Laplacian is recognizable on the top-left part of the matrix. The top-right holds the Ψ coefficients that is, distribution of the jump, and we can see the interface. The top-bottom holds the D T that is, the grid-to-interface approximations. 58

59 nz = 7624 Figure 7.3: Visualization of the left-hand side matrix Two-Dimensions: Cantilever problem These two previous examples did not include any cross-derivatives, so we set up another problem. Let s have a cantilever beam, anchored on the left in the wall, and loaded on the right. Clearly, the anchored part has null Dirichlet and von Neumann boundary conditions, the top and bottom part have Traction-Free boundary conditions, and the loaded right part is constrained by the charge. The elasticity equation for the beam chosen is the Lamé equation which had been developed previously: with the Boundary Conditions: µ 2 u + (µ + λ) ( U) = f (7.3) [u] = 0 and [u ] = 0 on Γ 1 [u] = u and [u ] = TF on Γ 2 [u] = u 0 and [u x ] = u x on Γ 3 (7.4) Where Γ 1,2,3 are the left, top/bottom, right parts of the beam. Here TF represents the Traction Free boundary conditions studied before. In Figure 7.4 we display the results in the displacement formulation Two-Dimensions: Skin Deformation We designed a two-dimensional example into which soft-tissue is constrained by two moving bones (up and down), and one non-moving bone (right). The 59

60 Figure 7.4: Cantilever problem small vectors at each discretization point in Figure 7.5 are the displacement vectors. 60

61 Figure 7.5: 2D Skin Deformation 7.2 The Computer Assisted Surgery case: Skin Deformation Prediction For our study we took the data s of a patient who already was included in the original ViSu. For this patient a CT before surgery was available, as well as a CT three months after surgery. Validation of the initial solving method has already been done on this data, so it will allow us to compare it with the new approach that is, the one with EJIIM which is the concern of this paper. The CT acquisition protocol was: Type High Speed Spiral CT helical mode Slice thickness 3mm Pitch 2 Kw 120 ma 160 Acquisition matrix 512 X 512 pixel 61

62 The patient underwent a Le Fort I high axillary advancement. Figure 7.6 (left) shows the selection of the moving bones and the osteotomies that the surgeon traced. Figure 7.6: Planned bone movements (left) and Predicted soft-tissues (right) The surgeon repositioned the bones and the computation kernel predicted the appearance of the patient s face as in Figure Figure 7.6 (right). A validation study is performed between the real outcome and the simulated results. The metric validation corresponds to the maximum distance between boundaries in horizontal slices (Hausdorff metric), as in Figure 7.7. The error appears to be 5mm. This is worse than for the previous approach: that will be discussed in the next Chapter. 62

63 Figure 7.7: Validation of the results 63