Lecture 10: Image-Based Modelling

Transcription

1 Computational Biology Group (CoBI), D-BSSE, ETHZ Lecture 10: Image-Based Modelling Prof Dagmar Iber, PhD DPhil MSc Computational Biology 2015

2 Contents 1 Image-based Domains for Simulations Staining & Imaging of the Tissue Image Processing Image Alignment Image Averaging Image segmentation Mesh Generation Model Simulation 2 Image-based Growth Fields for Simulations Basic Algorithms Extensions of the Algorithms Evaluation of Algorithms Final 2D Algorithm 3D Algorithms 3 Image-based Modelling 4 Model-based Domains & Growth Processes Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

3 Human Embryonic Development Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

4 Image-based Domains for Simulations Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

5 15. Mai / 65 A Pipeline for Image-based Modelling Tissue / Organ of Interest Labelling & Imaging 3D Image Image Processing & Segmentation Static Computational Domain Clonal Analysis of realistic shape Morphing Cell Tracking Mesh Generation & Import Growth Fields Moving, growing Domains of Realistic Shapes Model Formulation Computational Models of Organ Development Parameter Optimization Gene Expression Domains: wildtype & mutants APPLICATIONS: - Define underlying Mechanisms - Test Consistency of Hypotheses with Biological Data - Predict & Explain Mutant Phenotypes - Biomedical Applications 1 Tissue Staining & Imaging 2 Image Processing Image Alignement & Averaging Image Segmentation 3 Mesh Generation 4 Model Simulation on Embryonic Domain Mesh Import Model Formulation PDE Solution

6 15. Mai / Staining & Imaging of the Tissue 1 obtain imaging data of the tissue of interest 2 If different sub-structures of interest label tissue accordingly. Data from Erkan Uenal The staining and imaging technique of choice depends on the tissue, the sub-structure of interest, and the desired resolution.

7 15. Mai / Image Processing Data from Erkan Uenal Several image processing software packages are available to perform these steps, e.g. Amira Imaris Drishti Simpleware Meshlab Rhino 3-D Slicer Alternatively, image processing can also be done with MATLAB.

8 15. Mai / Image Alignment If multiple image recordings of the organ or tissue are available at a given stage, then the 3D images can be aligned and averaged. The alignment procedure is a computationally non-trivial problem. Data from Erkan Uenal In Amira a number of iterative hierarchical optimization algorithms (e.g. QuasiNewton) are available as well as similarity measures (e.g. Euclidean distance) to be minimized.

9 15. Mai / Image Averaging Average pixel intensities of corresponding pixels in multiple datasets of the same size and resolution. helps to assess the variability between embryos identifies common features. reduces variability due to experimental handling UT averaging of badly aligned datasets can result in loss of biologically relevant spatial information run the alignment algorithm several times, starting with different initial positions of the objects, which are to be aligned.

10 15. Mai / Image segmentation During image segmentation, the digital image is partitioned into multiple subdomains, usually corresponding to anatomic features and gene expression regions. A variety of algorithms are available for image segmentation, most of which are based on differences in pixel intensity. Data from Erkan Uenal

11 15. Mai / 65 2-D Virtual Sections 3D Simulations are expensive and it can be helpful to extract 2-D sections first. A variety of algorithms are available for image segmentation, most of which are based on differences in pixel intensity. Data from Erkan Uenal

12 15. Mai / Mesh Generation To carry out finite element methods (FEM)-based simulations of the signaling networks, segmented images are subsequently converted into meshes of sufficient quality. Data from Erkan Uenal

13 15. Mai / Mesh Quality The quality of the mesh can be assessed according to the following two parameters: Mesh size: The linear size of the mesh should be much smaller than any feature of interest in the computational solution, i.e. if the gradient length scale in the model is 50 µm then the linear size of the mesh should be at least several times less than 50 µm. The ratio of the sides of the mesh elements: The length of the shortest side to the longest side should be 0.1 or more. To confirm the convergence of the simulation, the model must be solved on a series of refined meshes.

14 15. Mai / Mesh Import To exchange meshes between image processing software and the simulation software, suitable file formats need to be chosen. To exchange between AMIRA and COMSOL: save AMIRA mesh as I-DEAS universal data format read into Gmesh and save as Nastran Bulk data file change file extension from UNV to DAT import into COMSOL

15 15. Mai / Model Simulation Available PDE solver include Commercial: Ansis, Abaqus, COMSOL Open Source: DUNE, FEniCS, FreeFEM, LiveV Data from Erkan Uenal

16 Image-based Growth Fields for Simulations Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

17 15. Mai / 65 Reaction-diffusion equation on a growing domain c i t + (c i u) = D i c i + R(c i ). (1) x x indicates that the time derivative is performed while keeping x constant. The terms u c i and c i u describe advection and dilution, respectively. If the domain is incompressible, i.e. u = 0, the equations further simplify.

18 15. Mai / 65 Image-based Displacement Fields Data from Odysse Michos, Sanger Insitute, Cambridge, UK

19 15. Mai / 65 Image-based Displacement Fields To obtain the displacement field from experimental data, tissue geometries need to be extracted at sequential time points. This requires the following steps: 1 imaging of the tissue at distinct developmental time points 2 image segmentation 3 meshing of the segmented domain 4 warping (morphing) of images at various developmental stages. Subsequently a mathematical regulatory network model can be solved on the deforming physiological domain. In the following we will discuss the different steps in detail.

20 Calculating the Displacement Field To simulate the signaling models on growing domains we need to determine the displacement fields between the different stages. The displacement field between two consecutive stages can be calculated by morphing two subsequent stages onto each other. In other words we are looking for a function which returns a point on a surface at time t + t which corresponds to a point on a surface at time t. Data from Erkan Uenal Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

21 15. Mai / 65 Landmark-based Bookstein algorithm Data from Denis Menshykau The landmark-based Bookstein algorithm (Bookstein, 1989), which is implemented in Amira, uses paired thin-plate splines to interpolate surfaces over landmarks defined on a pair of surfaces. The landmark points need to be placed by hand on the two 3D geometries to identify corresponding points on the pair of surfaces.

22 15. Mai / 65 Limitations of landmark-based Bookstein Algorithm The exact shape of the computed warped surface therefore depends on the exact position of landmarks; landmarks must therefore be placed with great care. While various stereoscopic visualization technologies are available this process is time-consuming and in parts difficult for complex surfaces such as the epithelium of the embryonic lung or kidney, in particular if the developmental stages are further apart.

23 15. Mai / 65 Alternative Approaches to determine displacement fields (a) displacement field of entire domain (b) local displacement field

24 15. Mai / 65 Basic Algorithms 1 Minimal Distance Mapping 2 Normal Mapping: intersection of C 1 normal vector and C 2 3 Diffusion Mapping: This mapping is obtained by solving the diffusion equation u u = 0, (2) t for steady flow: u = 0 4 Uniform Mapping: interpolate N points equidistantly along both curves and connect them

25 Extensions of the Algorithms 1 Reverse Mapping: apply algorithm from C 2 to C 1 2 Transformation Mapping: scale and align before mapping 3 Curve Segment Mapping: curves are split into segments according to the intersection points Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

26 15. Mai / 65 Extensions of the Algorithms: Curve Segment Mapping (a) global mapping (b) segment mapping

27 15. Mai / 65 Evaluation of Algorithms 1 Qualitative Evaluation 2 Theoretical Reasoning 3 Quantitative Evaluation

28 15. Mai / 65 Evaluation of Algorithms Qualitative Evaluation

29 15. Mai / 65 Minimal Distance Mapping (a) minimal distance mapping (b) reverse minimal distance mapping (c) transformed minimal distance mapping

30 15. Mai / 65 Minimal Distance Mapping (d) Minimal Distance (e) Reverse Minimal Distance

31 15. Mai / 65 Normal Mapping (a) normal mapping (b) reverse normal mapping(c) transfomed normal mapping

32 15. Mai / 65 Normal Mapping (d) Normal Mapping (e) Reverse Normal Mapping

33 15. Mai / 65 Diffusion Mapping (a) Normal Mapping fails (b) Minimal Distance fails

34 15. Mai / 65 Diffusion Mapping (c) solution of diffusion equation (d) streamlines (e) diffusion mapping

35 15. Mai / 65 Diffusion Mapping (f) diffusion mapping (g) reverse diffusion mapping (h) transformed diffusion mapping

36 15. Mai / 65 Uniform Mapping (a) good (b) bad

37 15. Mai / 65 Transformed Mapping (a) minimal distance mapping (b) C 1 scaled indicated by dashed line

38 15. Mai / 65 Transformed Mapping (c) minimal distance mapping from C t onto C 2 (d) displacement field starting points are transformed back onto C 1

39 15. Mai / 65 Summary of Qualitative Evaluation 1 Minimal distance fails when mapping onto a curve with much bigger curvature 2 Normal mapping leads to crossings if C 1 is locally concave and far away from C 2 3 Diffusion and uniform mapping solve these problems but in many cases do not give very nice results

40 15. Mai / 65 Theoretical Reasoning Minimal distance mapping will always be orthogonal on a smooth and closed curve minimal distance solution normal mapping solution Normal mapping is preferred: more natural, no crossings if point density is low, better solutions on boundary and other non smooth regions

41 Quantitative Evaluation of Mapping Algorithms Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

42 15. Mai / 65 Quantitative Evaluation of Mapping Algorithms ( ) 2 Li QM 1 = 1. αl i (3) A i QM 2 = ( < A i > 1)2 (4) QM = QM 1 QM 2 = ( L i 1) 2 A i ( αl i < A i > 1)2 (5)

43 15. Mai / 65 Quality Measures applied to Circle Ellipse Mapping QM 1 QM 2 QM minimal distance mapping reverse minimal distance transformed minimal distance normal mapping reveres normal transformed normal diffusion mapping reverse diffusion transformed diffusion

44 Quality Measures applied to Circle Ellipse Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

45 15. Mai / 65 Quality Measures applied to Kidney Data (a) minimal distance mapping (b) reverse minimal distance mapping

46 15. Mai / 65 Quality Measures applied to Kidney Data (c) normal mapping (d) reverse normal mapping

47 15. Mai / 65 Quality Measures applied to Kidney Data (e) reverse diffusion mapping (f) uniform mapping

48 15. Mai / 65 Quantitative Evaluation of Mapping Algorithms QM( =1h) QM( =2h) minimal distance 1.755E E-2 reverse minimal distance 1.937E E-2 normal 0.591E E-2 reverse normal 8.686E E-2 diffusion 1.163E E-2 reverse diffusion 3.896E E-2 uniform mapping 0.120E E-2

49 15. Mai / 65 Quantitative Evaluation of Mapping Algorithms QM( =4h) QM( =8h) minimal distance E reverse minimal distance E normal 9.247E reverse normal diffusion E reverse diffusion uniform mapping 0.951E

50 15. Mai / 65 Final 2D Algorithm For each curve intersection segment: 1 try normal mapping 2 if crossings occur try normal mapping with scaling 3 if crossings occur do reverse diffusion For open curves: first scale and connect!

51 15. Mai / 65 3D Algorithms (g) minimal distance mapping (h) normal mapping (i) diffusion mapping

52 3D Normal Mapping applied to Lung Data Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

53 Image-based Modelling Computational Biology Group (CoBI), D-BSSE, ETHZ Prof Dagmar Iber, PhD DPhil Lecture 11 MSc Mai / 65

54 15. Mai / 65 Image-based Modelling Once the correspondence between two surfaces has been defined, a displacement field can be calculated by determining the difference between the positions of points on the two surface meshes as illustrated for the embryonic lung sequence in the next slide.

55 Image-based Displacement Fields Computational Biology Group (CoBI), D-BSSE, ETHZ 15. Mai / 65 Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015

56 15. Mai / 65 Image-based Displacement Fields (a,b) The segmented epithelium and mesenchyme of the developing lung at two consecutive stages. (c) The displacement field between the two stages in panels a and b. (d) The growing part of the lung. The coloured vectors indicate the strength of the displacement field. (e) The solution of the Turing model (Equations 10) on the segmented lung of the stage in panel a. (f) Comparison of the simulated Turing model (solid surface) and the embryonic displacement field (arrows). The images processing was carried out in AMIRA; the simulations were carried out in COMSOL Multiphysics 4.3a.

57 15. Mai / 65 Simulation of Signaling Dynamics using FEM To carry out the FEM-based simulations the mesh and displacement field need to be imported into a FEM solver. To avoid unnecessary interpolation of the vector field, the displacement field should be calculated for exactly the same surface mesh as was used to generate the volume mesh. A number of commercial (COMSOL Multiphysics, Ansis, Abaqus etc) and open (FreeFEM, DUNE etc) FEM solvers are available.

58 15. Mai / 65 Reaction-diffusion equation on a growing domain c i t + (c i u) = D i c i + R(c i ). (6) x x indicates that the time derivative is performed while keeping x constant. The terms u c i and c i u describe advection and dilution, respectively. If the domain is incompressible, i.e. u = 0, the equations further simplify.

59 15. Mai / 65 Prescribed Growth In prescribed growth models an initial domain and a spatio-temporal velocity or displacement field are defined. The domain with initial coordinate vectors X is then moved according to this velocity field u(x, t), i.e. X(t) t = x t = u(x, t) (7) X

60 15. Mai / 65 Model-based Displacement Fields The velocity field u(x, t) can be captured in a functional form that represents either the observed growth and/or signaling kinetics. In the simplest implementation the displacement may be applied only normal to the boundary, i.e. u = µn, (8) where n is the normal vector to the boundary and µ is the local growth rate.

61 15. Mai / 65 Model-based Displacement Fields Growth processes often depend on signaling networks that evolve on the tissue domain. The displacement field u(x, t) may thus dependent on the local concentration of some growth or signaling factor. We then have u = µ(c)n, (9) where c is the local concentration of the signaling factor.

62 15. Mai / 65 Model Implementation These approaches can be readily implemented in the commercially available finite element solver COMSOL Multiphysics. Details of the implementation will be discussed in the tutorial.

63 15. Mai / 65 Example: Schnakenberg Turing Model c 1 t + (c 1u) = c 1 + γ(a c 1 + c1 2 c 2 ) c 2 t + (c 2u) = d c 2 + γ(b c1 2 c 2 ); (10) a, b, γ, and d are constant parameters in the Turing model.

64 Prescribed Domain Growth under Control of a Signaling Model. The figure shows as an example a 2D sheet that deforms within a 3D domain according to the strength of the signaling field normal to its surface, i.e. u = µc12 c2 n, where c1 and c2 are the two variables that are governed by the Schnakenberg-type Turing model. Computational Biology Group (CoBI), D-BSSE, ETHZ 15. Mai / 65 Prof Dagmar Iber, PhD DPhil Lecture 11 MSc 2015

65 15. Mai / 65 Thanks!! Thanks for your attention! Slides for this talk will be available at: