Computational Biology Group (CoBI), D-BSSE, ETHZ Lecture 10: Image-Based Modelling Prof Dagmar Iber, PhD DPhil MSc Computational Biology 2015
Contents 1 Image-based Domains for Simulations Staining & Imaging of the Tissue Image Processing Image Alignment Image Averaging Image segmentation Mesh Generation Model Simulation 2 Image-based Growth Fields for Simulations Basic Algorithms Extensions of the Algorithms Evaluation of Algorithms Final 2D Algorithm 3D Algorithms 3 Image-based Modelling 4 Model-based Domains & Growth Processes Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 2 / 65
Human Embryonic Development Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 3 / 65
Image-based Domains for Simulations Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 4 / 65
15. Mai 2015 5 / 65 A Pipeline for Image-based Modelling Tissue / Organ of Interest Labelling & Imaging 3D Image Image Processing & Segmentation Static Computational Domain Clonal Analysis of realistic shape Morphing Cell Tracking Mesh Generation & Import Growth Fields Moving, growing Domains of Realistic Shapes Model Formulation Computational Models of Organ Development Parameter Optimization Gene Expression Domains: wildtype & mutants APPLICATIONS: - Define underlying Mechanisms - Test Consistency of Hypotheses with Biological Data - Predict & Explain Mutant Phenotypes - Biomedical Applications 1 Tissue Staining & Imaging 2 Image Processing Image Alignement & Averaging Image Segmentation 3 Mesh Generation 4 Model Simulation on Embryonic Domain Mesh Import Model Formulation PDE Solution
15. Mai 2015 6 / 65 1. Staining & Imaging of the Tissue 1 obtain imaging data of the tissue of interest 2 If different sub-structures of interest label tissue accordingly. Data from Erkan Uenal The staining and imaging technique of choice depends on the tissue, the sub-structure of interest, and the desired resolution.
15. Mai 2015 7 / 65 2. Image Processing Data from Erkan Uenal Several image processing software packages are available to perform these steps, e.g. Amira Imaris Drishti Simpleware Meshlab Rhino 3-D Slicer Alternatively, image processing can also be done with MATLAB.
15. Mai 2015 8 / 65 2.1 Image Alignment If multiple image recordings of the organ or tissue are available at a given stage, then the 3D images can be aligned and averaged. The alignment procedure is a computationally non-trivial problem. Data from Erkan Uenal In Amira a number of iterative hierarchical optimization algorithms (e.g. QuasiNewton) are available as well as similarity measures (e.g. Euclidean distance) to be minimized.
15. Mai 2015 9 / 65 2.2 Image Averaging Average pixel intensities of corresponding pixels in multiple datasets of the same size and resolution. helps to assess the variability between embryos identifies common features. reduces variability due to experimental handling UT averaging of badly aligned datasets can result in loss of biologically relevant spatial information run the alignment algorithm several times, starting with different initial positions of the objects, which are to be aligned.
15. Mai 2015 10 / 65 2.3 Image segmentation During image segmentation, the digital image is partitioned into multiple subdomains, usually corresponding to anatomic features and gene expression regions. A variety of algorithms are available for image segmentation, most of which are based on differences in pixel intensity. Data from Erkan Uenal
15. Mai 2015 11 / 65 2-D Virtual Sections 3D Simulations are expensive and it can be helpful to extract 2-D sections first. A variety of algorithms are available for image segmentation, most of which are based on differences in pixel intensity. Data from Erkan Uenal
15. Mai 2015 12 / 65 3. Mesh Generation To carry out finite element methods (FEM)-based simulations of the signaling networks, segmented images are subsequently converted into meshes of sufficient quality. Data from Erkan Uenal
15. Mai 2015 13 / 65 3.1 Mesh Quality The quality of the mesh can be assessed according to the following two parameters: Mesh size: The linear size of the mesh should be much smaller than any feature of interest in the computational solution, i.e. if the gradient length scale in the model is 50 µm then the linear size of the mesh should be at least several times less than 50 µm. The ratio of the sides of the mesh elements: The length of the shortest side to the longest side should be 0.1 or more. To confirm the convergence of the simulation, the model must be solved on a series of refined meshes.
15. Mai 2015 14 / 65 3.1 Mesh Import To exchange meshes between image processing software and the simulation software, suitable file formats need to be chosen. To exchange between AMIRA and COMSOL: save AMIRA mesh as I-DEAS universal data format read into Gmesh and save as Nastran Bulk data file change file extension from UNV to DAT import into COMSOL
15. Mai 2015 15 / 65 3. Model Simulation Available PDE solver include Commercial: Ansis, Abaqus, COMSOL Open Source: DUNE, FEniCS, FreeFEM, LiveV Data from Erkan Uenal
Image-based Growth Fields for Simulations Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 16 / 65
15. Mai 2015 17 / 65 Reaction-diffusion equation on a growing domain c i t + (c i u) = D i c i + R(c i ). (1) x x indicates that the time derivative is performed while keeping x constant. The terms u c i and c i u describe advection and dilution, respectively. If the domain is incompressible, i.e. u = 0, the equations further simplify.
15. Mai 2015 18 / 65 Image-based Displacement Fields Data from Odysse Michos, Sanger Insitute, Cambridge, UK
15. Mai 2015 19 / 65 Image-based Displacement Fields To obtain the displacement field from experimental data, tissue geometries need to be extracted at sequential time points. This requires the following steps: 1 imaging of the tissue at distinct developmental time points 2 image segmentation 3 meshing of the segmented domain 4 warping (morphing) of images at various developmental stages. Subsequently a mathematical regulatory network model can be solved on the deforming physiological domain. In the following we will discuss the different steps in detail.
Calculating the Displacement Field To simulate the signaling models on growing domains we need to determine the displacement fields between the different stages. The displacement field between two consecutive stages can be calculated by morphing two subsequent stages onto each other. In other words we are looking for a function which returns a point on a surface at time t + t which corresponds to a point on a surface at time t. Data from Erkan Uenal Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 20 / 65
15. Mai 2015 21 / 65 Landmark-based Bookstein algorithm Data from Denis Menshykau The landmark-based Bookstein algorithm (Bookstein, 1989), which is implemented in Amira, uses paired thin-plate splines to interpolate surfaces over landmarks defined on a pair of surfaces. The landmark points need to be placed by hand on the two 3D geometries to identify corresponding points on the pair of surfaces.
15. Mai 2015 22 / 65 Limitations of landmark-based Bookstein Algorithm The exact shape of the computed warped surface therefore depends on the exact position of landmarks; landmarks must therefore be placed with great care. While various stereoscopic visualization technologies are available this process is time-consuming and in parts difficult for complex surfaces such as the epithelium of the embryonic lung or kidney, in particular if the developmental stages are further apart.
15. Mai 2015 23 / 65 Alternative Approaches to determine displacement fields (a) displacement field of entire domain (b) local displacement field
15. Mai 2015 24 / 65 Basic Algorithms 1 Minimal Distance Mapping 2 Normal Mapping: intersection of C 1 normal vector and C 2 3 Diffusion Mapping: This mapping is obtained by solving the diffusion equation u u = 0, (2) t for steady flow: u = 0 4 Uniform Mapping: interpolate N points equidistantly along both curves and connect them
Extensions of the Algorithms 1 Reverse Mapping: apply algorithm from C 2 to C 1 2 Transformation Mapping: scale and align before mapping 3 Curve Segment Mapping: curves are split into segments according to the intersection points Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 25 / 65
15. Mai 2015 26 / 65 Extensions of the Algorithms: Curve Segment Mapping (a) global mapping (b) segment mapping
15. Mai 2015 27 / 65 Evaluation of Algorithms 1 Qualitative Evaluation 2 Theoretical Reasoning 3 Quantitative Evaluation
15. Mai 2015 28 / 65 Evaluation of Algorithms Qualitative Evaluation
15. Mai 2015 29 / 65 Minimal Distance Mapping (a) minimal distance mapping (b) reverse minimal distance mapping (c) transformed minimal distance mapping
15. Mai 2015 30 / 65 Minimal Distance Mapping (d) Minimal Distance (e) Reverse Minimal Distance
15. Mai 2015 31 / 65 Normal Mapping (a) normal mapping (b) reverse normal mapping(c) transfomed normal mapping
15. Mai 2015 32 / 65 Normal Mapping (d) Normal Mapping (e) Reverse Normal Mapping
15. Mai 2015 33 / 65 Diffusion Mapping (a) Normal Mapping fails (b) Minimal Distance fails
15. Mai 2015 34 / 65 Diffusion Mapping (c) solution of diffusion equation (d) streamlines (e) diffusion mapping
15. Mai 2015 35 / 65 Diffusion Mapping (f) diffusion mapping (g) reverse diffusion mapping (h) transformed diffusion mapping
15. Mai 2015 36 / 65 Uniform Mapping (a) good (b) bad
15. Mai 2015 37 / 65 Transformed Mapping (a) minimal distance mapping (b) C 1 scaled indicated by dashed line
15. Mai 2015 38 / 65 Transformed Mapping (c) minimal distance mapping from C t onto C 2 (d) displacement field starting points are transformed back onto C 1
15. Mai 2015 39 / 65 Summary of Qualitative Evaluation 1 Minimal distance fails when mapping onto a curve with much bigger curvature 2 Normal mapping leads to crossings if C 1 is locally concave and far away from C 2 3 Diffusion and uniform mapping solve these problems but in many cases do not give very nice results
15. Mai 2015 40 / 65 Theoretical Reasoning Minimal distance mapping will always be orthogonal on a smooth and closed curve minimal distance solution normal mapping solution Normal mapping is preferred: more natural, no crossings if point density is low, better solutions on boundary and other non smooth regions
Quantitative Evaluation of Mapping Algorithms Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 41 / 65
15. Mai 2015 42 / 65 Quantitative Evaluation of Mapping Algorithms ( ) 2 Li QM 1 = 1. αl i (3) A i QM 2 = ( < A i > 1)2 (4) QM = QM 1 QM 2 = ( L i 1) 2 A i ( αl i < A i > 1)2 (5)
15. Mai 2015 43 / 65 Quality Measures applied to Circle Ellipse Mapping QM 1 QM 2 QM minimal distance mapping 13.417 22.446 301.116 reverse minimal distance 0.128 0.134 0.017 transformed minimal distance 1.690 2.709 4.578 normal mapping 0.103 0.398 0.041 reveres normal 56.054 2.447 137.164 transformed normal 0.084 0.397 0.033 diffusion mapping 0.494 1.168 0.577 reverse diffusion 1.589 0.088 0.140 transformed diffusion 0.224 0.559 0.125
Quality Measures applied to Circle Ellipse Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 44 / 65
15. Mai 2015 45 / 65 Quality Measures applied to Kidney Data (a) minimal distance mapping (b) reverse minimal distance mapping
15. Mai 2015 46 / 65 Quality Measures applied to Kidney Data (c) normal mapping (d) reverse normal mapping
15. Mai 2015 47 / 65 Quality Measures applied to Kidney Data (e) reverse diffusion mapping (f) uniform mapping
15. Mai 2015 48 / 65 Quantitative Evaluation of Mapping Algorithms QM( =1h) QM( =2h) minimal distance 1.755E-2 4.530E-2 reverse minimal distance 1.937E-2 2.728E-2 normal 0.591E-2 2.037E-2 reverse normal 8.686E-2 5.734E-2 diffusion 1.163E-2 3.42E-2 reverse diffusion 3.896E-2 1.989E-2 uniform mapping 0.120E-2 0.673E-2
15. Mai 2015 49 / 65 Quantitative Evaluation of Mapping Algorithms QM( =4h) QM( =8h) minimal distance 55.563E-2 11.664 reverse minimal distance 14.530E-2 125.240 normal 9.247E-2 0.304 reverse normal 143.190 160.660 diffusion 40.144E-2 12.190 reverse diffusion 0.104 0.454 uniform mapping 0.951E-2 0.280
15. Mai 2015 50 / 65 Final 2D Algorithm For each curve intersection segment: 1 try normal mapping 2 if crossings occur try normal mapping with scaling 3 if crossings occur do reverse diffusion For open curves: first scale and connect!
15. Mai 2015 51 / 65 3D Algorithms (g) minimal distance mapping (h) normal mapping (i) diffusion mapping
3D Normal Mapping applied to Lung Data Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 52 / 65
Image-based Modelling Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015 15. Mai 2015 53 / 65
15. Mai 2015 54 / 65 Image-based Modelling Once the correspondence between two surfaces has been defined, a displacement field can be calculated by determining the difference between the positions of points on the two surface meshes as illustrated for the embryonic lung sequence in the next slide.
Image-based Displacement Fields Computational Biology Group (CoBI), D-BSSE, ETHZ 15. Mai 2015 55 / 65 Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015
15. Mai 2015 56 / 65 Image-based Displacement Fields (a,b) The segmented epithelium and mesenchyme of the developing lung at two consecutive stages. (c) The displacement field between the two stages in panels a and b. (d) The growing part of the lung. The coloured vectors indicate the strength of the displacement field. (e) The solution of the Turing model (Equations 10) on the segmented lung of the stage in panel a. (f) Comparison of the simulated Turing model (solid surface) and the embryonic displacement field (arrows). The images processing was carried out in AMIRA; the simulations were carried out in COMSOL Multiphysics 4.3a.
15. Mai 2015 57 / 65 Simulation of Signaling Dynamics using FEM To carry out the FEM-based simulations the mesh and displacement field need to be imported into a FEM solver. To avoid unnecessary interpolation of the vector field, the displacement field should be calculated for exactly the same surface mesh as was used to generate the volume mesh. A number of commercial (COMSOL Multiphysics, Ansis, Abaqus etc) and open (FreeFEM, DUNE etc) FEM solvers are available.
15. Mai 2015 58 / 65 Reaction-diffusion equation on a growing domain c i t + (c i u) = D i c i + R(c i ). (6) x x indicates that the time derivative is performed while keeping x constant. The terms u c i and c i u describe advection and dilution, respectively. If the domain is incompressible, i.e. u = 0, the equations further simplify.
15. Mai 2015 59 / 65 Prescribed Growth In prescribed growth models an initial domain and a spatio-temporal velocity or displacement field are defined. The domain with initial coordinate vectors X is then moved according to this velocity field u(x, t), i.e. X(t) t = x t = u(x, t) (7) X
15. Mai 2015 60 / 65 Model-based Displacement Fields The velocity field u(x, t) can be captured in a functional form that represents either the observed growth and/or signaling kinetics. In the simplest implementation the displacement may be applied only normal to the boundary, i.e. u = µn, (8) where n is the normal vector to the boundary and µ is the local growth rate.
15. Mai 2015 61 / 65 Model-based Displacement Fields Growth processes often depend on signaling networks that evolve on the tissue domain. The displacement field u(x, t) may thus dependent on the local concentration of some growth or signaling factor. We then have u = µ(c)n, (9) where c is the local concentration of the signaling factor.
15. Mai 2015 62 / 65 Model Implementation These approaches can be readily implemented in the commercially available finite element solver COMSOL Multiphysics. Details of the implementation will be discussed in the tutorial.
15. Mai 2015 63 / 65 Example: Schnakenberg Turing Model c 1 t + (c 1u) = c 1 + γ(a c 1 + c1 2 c 2 ) c 2 t + (c 2u) = d c 2 + γ(b c1 2 c 2 ); (10) a, b, γ, and d are constant parameters in the Turing model.
Prescribed Domain Growth under Control of a Signaling Model. The figure shows as an example a 2D sheet that deforms within a 3D domain according to the strength of the signaling field normal to its surface, i.e. u = µc12 c2 n, where c1 and c2 are the two variables that are governed by the Schnakenberg-type Turing model. Computational Biology Group (CoBI), D-BSSE, ETHZ 15. Mai 2015 64 / 65 Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015
15. Mai 2015 65 / 65 Thanks!! Thanks for your attention! Slides for this talk will be available at: http://www.bsse.ethz.ch/cobi/education