Modeling Human Embryos Using a Variational Level Set Approach

Transcription

1 Modeling Human Embryos Using a Variational Level Set Approach Uffe Damgaard Pedersen June 1, 2004

2 Abstract At fertility clinics, living human embryos are evaluated by visual inspection of light microscopy images. The criteria to assess the quality are number of blastomeres (cells) of the embryo, regularity of size and degree of fragmentation. Today, this evaluation process is based on interpretation of 3D information projected to 2D images. It is believed that an overall improvement in the process could be achieved, if the process could be based on 3D information with quantitative measurements instead. Unfortunately, due to limitations in the acquisition process, the 3D information cannot be obtained directly. Previous work on building 3D models has used solid spheres to simulate the position and size of the blastomeres. This thesis main objective is to make use of the level set methodology to address this problem. A new 3D model is developed by using a variational level set approach to model the blastomeres and the interior of the Zona Pellucida as elastic objects with individual surface tensions, thereby enabling them to deform due to mutual object interaction. Positional information from microscopy images is embedded as soft constraints requiring that an obtained curve must be located on an object s surface and that the object must be located inside a bounding generalized cylinder passing through this curve. There is no quantitative standard to measure the results against, but results are promising and correspond well to human interpretation of the light microscopy images. 2

3 Preface This work constitutes Uffe Damgaard Pedersen s Master of Science Thesis, a mandatory part of the Master of Science (MSc) in Information Technology (Multimedia Technology) from the IT University of Copenhagen. The work was initiated in September 2002 and has been carried out under supervision of assistant research professor Ole Fogh Olsen, IT University of Copenhagen. A paper [10] based on this work was accepted and presented at the 2nd IEEE Workshop on Variational, Geometric and Level Set Methods October 2003 in Nice, France Uffe Damgaard Pedersen udp@itu.dk IT University of Copenhagen Glentevej 67 DK-2400 Copenhagen NV Denmark 3

4 Table of Contents Abstract...2 Preface...3 Table of Contents...4 Table of Figures...7 Table of tables Introduction Background The Level Set Method Human Embryo evaluation Goal...11 PART I Level Set Methods Curves in space and time Parametric curves Implicitly defined curves Signed distance functions Discretization Curves in time Entropy conditions and weak solutions The Level Set Method The Level Set formulation Comparison of explicit and implicit definition Marker Particles Level Set Method versus Marker Particle Approach Re-initialization of a signed distance function Crossing times Partial differential equation approach Variational Level Set Approach Multiphase Systems The Variational Level Set Approach The Variational Level Set Formulation

5 4.2.2 System Energy Constraints Energy minimization...33 PART II Human Embryo Analysis Background In Vitro Fertilization Embryo morphology The membrane of a blastomere Blastomere Shape Qualitative evaluation criteria Hoffman modulation contrast light microscopy Microscopy image series of a human embryo Analysis Information in images Shape and volume information Positional information Concave objects Discussion Modeling the human embryo Modeling a blastomere Restricting an object with respect to available information...47 PART III A Variational Level Set Approach to Model a Human Embryo A Level Set Model of the Human Embryo Energies and constraints Energy minimization System behavior summary Implementation Energy minimizing process Calculation of evolution equations Advancing the level sets Re-initialization of a level set Evaluation of system stationarity Gradient precautions Boundary of computational domain

6 9 Test and results System behavior test Energies Velocities Test of the human embryo in 2D System behavior test Energy minimization & volume conservation Stationarity & conclusion Conclusion Bibliography

7 Table of Figures Figure 1: A 3D plot of an implicitly given circle Figure 2: Curve evolution examples Figure 3: Swallow tale and entropy solution Figure 4: Huygens' principle Figure 5: Front propagation with unit normal speed Figure 6: Front to level set representation Figure 7: Marker particles - a discrete parameterization of a curve Figure 8: Two-phase and multiphase systems Figure 9: Steps of the In Vitro Fertilization process Figure 10: Human embryo and its corresponding morphology model Figure 11: A single blastomere extracted from a human embryo Figure 12: Embryo quality at day Figure 13: Sequence of HMC microscopy images Figure 14: Equal equator contours obtained from unequal objects Figure 15: Equator contour variation due to focal height Figure 16: Equator contours of concave objects Figure 17: Shape modeling of blastomere Figure 18: Positional restriction of an object Figure 19: A level set model of a human embryo Figure 20: Sign convention for the system s level set functions Figure 21: System Overview Figure 22: Energy minimizing process Figure 23: Four cases of overlap Figure 24: Blastomere and cylinder level sets Figure 25: Energy plots of tested system energy Figure 26: Surface and cylinder velocity fields Figure 27: Results of 2D human embryo test Figure 28: Energies and volume in time for human embryo 2D test Figure 29: Stationarity and energy in time for human embryo 2D test

8 Table of tables Table 1: Boundary scheme for central differences gradient Table 2: Boundary scheme for second order ENO gradient scheme Table 3: Energy results of the surface energy test Table 4: Energy results energies in the four overlap cases Table 5: Velocity results for surface velocity and Lagrange multiplier Table 6: Setup information for level sets

9 1 Introduction 1.1 Background Originally, I was interested in finding a subject in the areas of simulation of physical phenomenon and computer graphics. I considered simulation of water, water surfaces and water drops. In my search, I came by the article Capturing the behavior of drops and bobbles in which it was shown how to simulate drops and bobbles with a method called the Level Set Method. This seemed very fascinating and gave me an interest for going deeper into the theory behind this method. During the first meetings with my supervisor, I was presented to the following problem. Physicians, when making fertility assessments, want more information about the Human Embryo. My supervisor and I agreed it would be interesting to make a model for this using the Level Set Method. The plan was as follows: I should acquire knowledge of the Level Set Method starting with the article and then use this knowledge to make a solution on how to model a human embryo. 1.2 The Level Set Method Motivation: when simulating water, a natural approach would be to simulate the surface, or more correct the interface between the water and the surroundings. If we imagine there are buoys in the water surface, we can track the water by tracing the buoys position. Some questions, however, arise; how many buoys should we use; what happens if the buoys come to close or to far from to each other; what if the water parts in two, etc. Even though this method seems straight ahead, there are things to consider and things to deal with along the way. Another way to approach this is to let the surface be represented implicitly. Instead of using tracking points following the surface (buoys) we could use a grid of points where every point holds the distance to the surface. This is called implicit representation because the surface position is not known explicitly. The implicit representation demands more points than the explicit, but because the points are fixed a lot of bookkeeping is avoided (add/remove buoys). The Level Set method 9

10 uses an implicit representation of the interface, where also a velocity function is added. There are several methods to create a velocity function. In an addition to the Level Set method, called Variational Level Set method, an energy expression is set for the objects one wish to simulate and these are then sought minimized as a result of a velocity function. This method has shown useful for simulating physical phenomenon, for instance soap bobbles and falling drops. 1.3 Human Embryo evaluation An estimate is that around 15 percent of all couples encounter fertility problems and in some cases it is necessary to seek professional assistance. The process of assisting pregnancy by fertilization outside the body is called In Vitro Fertilization. The first baby born of this procedure was born in England in Two years later around sixteen thousand children were born of different kinds of assisted fertilizations. At fertility clinics, medical experts evaluate living human embryos by visual inspection of microscopy images. The aim is to select the candidate embryo with the highest probability for pregnancy and to avoid unwanted multiple siblings. The criteria to assess the quality are number of cells (blastomeres) in the embryo, regularity of size and degree of fragmentation. Today, this evaluation process is based on interpretation of 3D information projected to 2D images. It is believed that an overall improvement in the process could be achieved, if the process could be based on 3D information with quantitative measurements instead. Unfortunately, due to limitations in the acquisition process, the 3D information cannot be obtained directly. Instead a 3D model based on the 2D information available, knowledge of the systems physical behavior and necessary assumptions could be developed. A more thorough presentation and analysis of the problem statement and data will be given in Part II. 10

11 1.4 Goal The main goals of this Master of Science Thesis have been: To acquire knowledge on the Level Set method To develop a new elastic model for a human embryo, using a variational level set approach. Most effort has been put into the development of a new model for Human Embryo. 11

12 PART I LEVEL SET METHODS 12

13 2 Curves in space and time This chapter gives a short presentation of curve representation and curves evolution. 2.1 Parametric curves 2 2 A parametric curve C existing in is a function C : I where I is the parameter interval[ 0, 1]. A pair of functions gives the curve and the parameter p is a real number: C ( p) = [ x( p), y( p) ], p [ 0, 1] (1.1) The curve is assumed initially to be smooth, simple and closed. Thus all derivatives should be nonzero, the curve should not intersect itself and C ( 0) = C ( 1). For each input p we obtain a pair of coordinates in 2D, thus the curve is obtained by varying p in the complete interval. The first derivative of the curve with respect to its parameter is the velocity vector also called the tangent vector. The normalized tangent of the curve is: C x p, y p p T = = (1.2) C x + y 2 2 p p p The normalized normal to the curve is: Tˆ yp, x p N = = Tˆ x + y 2 2 p p (1.3) where ˆT is the normalized tangent rotated 270 degrees. The arch length of a curve at from p is given by: s( p) = C ( p ) dp (1.4) 0 p p 13

14 The curve is said to be parameterized by the arch length if C s 1 and therefore is s( p) = p p0 for any p. It is common to denote the parameter variable by s for a curve parameterized by arch length: [ ] C ( s) = x( s), y( s), 0 s S (1.5) For a curve parameterized by the arch length the velocity vector is by definition normalized so the normalized tangent becomes: T = [ x, y ] s s (1.6) For a curve parameterized by arch length the normalized normal becomes: N = [ y, x s s ] (1.7) Curvature κ of the curve tells how fast the curve is moving away from the tangent of the curve in a given point. Curvature is the length of the second derivative s component, (usually the second derivative is called the acceleration vector) in the direction of the normal. For a curve parameterized by the arch length the curvature simplifies to: κ = C ss (1.8) 2.2 Implicitly defined curves For the parametric curve, we can obtain its position by varying the parameter variable in the complete interval so the curve is given explicitly in the parametric representation. For a curve given implicit, we need to solve a given equation in order to find the position of the curve. A curve existing 2 is given implicitly by: 2 C = { x, y ϕ( x, y) = 0 } (1.9) 14

15 We can interpret the value of the ϕ ( xy, ) as the z-coordinate in a 3D Cartesian coordinate system and plot the function. The resulting graph s intersection with the xy -plane is where ϕ ( x, y) = 0 and thereby where the curve C is located. In simple cases it is possible to make conversions between explicit and implicit representations, but not in the general case. Example of a circle given by an implicit function We consider the implicit function given by: 2 2 ϕ ( x, y) = x y + 50 (1.10) Solving the above equation will give a circle. We notice that points inside the regions of the circle are positive and points outside the region are negative. To test if a given point lies inside or outside a curve is a simple sign test when using the implicit representation. The function is shown in figure 1 where the points where the circle is formed at ϕ ( x, y) = 0 are illustrated by a curve Figure 1: A 3D plot of an implicitly given circle. The gradient of an implicit curve is defined as: ϕ ϕ ϕ, = x y (1.11) 15

16 The normalized normal: ϕ N = (1.12) ϕ The curvature: ϕ κ = ϕ (1.13) Where v v v v 2 x y (1.14) ( ) = 1 div = + For the normalized normal and the curvature terms above, it is assumed that the gradient is non-vanishing Signed distance functions A subset of implicit functions is the signed distance functions. They share properties with the implicit functions but have some interesting additional properties too. Given + a region of interest Ω, the interior is denoted Ω the exterior is called Ω and the interface separating the interior from the exterior is denoted Ω. A signed distance function ϕ is given by: + d( x ), for x Ω ϕ( x ) = d( x ), for x Ω (1.15) 0, for x Ω Where d is a distance function giving the distance from a given point x to the closest point on the interface. d x = x x for all x Ω (1.16) ( ) min( ) I I 16

17 Since ϕ is a signed distance function the gradient length becomes ϕ( x ) = 1. Therefore, the equation for the normalized normal for implicit functions simplifies to N = ϕ 2.3 Discretization When discretizing the explicit representation we replace the real parameter t by a finite set of points tt 0, 1,.., T. The curve is dicretized into line segments and the number of points yields the resolution. For the implicit defined curves we need not 2 only to discretize the curve but the entire domain. This is of course impossible and therefore a sub region t 2 D is dicretized: {( i, j) 1,1 x y i m j n} (1.17) The result is a grid of size m times n, where the sign of each point tell if the point is inside or outside of the curve. Both discrete representations lack information. In the explicit representation, we have the curves position in sample points and need to interpolate between these. In the implicit representation, we need more advanced interpolation methods to obtain the curve s location. 2.4 Curves in time A curve parameterized by arch length s and time t is given by: C( s, t) = x( s, t), y( s, t), 0 s S, 0 t T (1.18) The geometrical functions do not depend on time so the gradient, the normal and the curvature are simply: The tangent: T( st, ) = [ x, y s s ] (1.19) 17

18 The normalized normal: ( st, ) = [ y, x ] N (1.20) s s The curvature: κ ( st) = C ( s) (1.21), ss We are interested in evolving a curve over time and will decide velocity. Since only velocity in the direction of the normal is geometrical relevant we will only consider speed in the direction of the normal. The curve is evolved in time under the above speed function and a family of curves α ( st, ) is obtained. The speed function F describes the speed in the direction of the normal to the front. A general speed function can depend on: F( L, G, I ) (1.22) L is local properties local geometric information, i.e. curvature G is global properties i.e. shape and position, area enclosed by the curve, the length of curve and energy minimizing differential equations. I is independent properties independent of the shape, i.e. a passive flow. 18

19 C( x, y, t ) 1 N N C( x, y, t ) 0 N κ N C( x, y, t ) 1 κn κn N κn κn κn Figure 2: Curve evolution examples 2.5 Entropy conditions and weak solutions If we consider a smooth curve with for example a unit speed function, it can develop corners as the front propagates. This is illustrated in the figure 3, where an initial sinusoidal front is propagated with the speed function F=1. The exact solution to the problem is given by x( st, ) = n( st, = 0 ) t+ x( st, = 0 ) (1.23) As the front develops, a corner is formed and at this corner, the derivatives are no longer continuous and the normal is ambiguously defined. This calls for a weak solution, a solution that weakly satisfies the definition of differentiability. At this state, the question of how to continue the propagation arises. If the curve represents a geometrical curve, a solution could be the swallowtail at the left where the front is allowed to pass through itself. On the other hand, if the curve represents the interface between two regions, the breaking and crossing of the front seems inappropriate. Since the front is moved by a unit speed function, the solution we are interested in should only consist of points where the distance from the initial curve equals time t. The solution on the right is an alternative solution to this problem with this property. It can be build through Huygens principle, where the curve is considered a waterfront. Each point on the curve emanates with the unit speed function and resulting envelope of the wave front is formed by the first arrivals, see figure 4. 19

20 Figure 3: Swallow tale and entropy solution Illustration from [6] Figure 4: Huygens' principle Illustration from [6] In the studies of flame propagation Sethian posed a so-called entropy condition to the problem of propagating fronts with a unit speed function [6]. The idea was to view the boundary as a source for a propagating flame and with the property that each point in the domain can only be ignited once. This is similar to Huygens principle and yields the same solution. The term entropy refers to the organization of information, stating that new information cannot be created during the evolution of a problem. The opposite, loss of information, occurs when a shock is formed which results in an irreversible solution. This is illustrated in figure 5, showing that the shock is preserved resulting in a ridge to the left, but turning the speed direction results in a smooth curve. 20

21 Figure 5: Front propagation with unit normal speed. A shock is formed (left).a rarefaction (right). Illustration from [6] 21

22 3 The Level Set Method The idea of the Level Set Method [2] is to represent an n-dimensional interface of interest as the zero level set of a ( n + 1) dimensional function and the ( n + 1) dimensional function is a signed distance function to the interface. If we have a 2D curve the equivalent level set representation is the zero level set of a 3D signed distance function with positive sign convention inside region and negative outside. A 2D example of a simple curve can be seen on the figure below. The level set method is said to be an Eulerian approach that captures the front rather than track it. Ω y ϕ ( xyt,, ) x t=0 y x Figure 6: Front to level set representation 3.1 The Level Set formulation Given a closed hyper-surface, Γ ( t ), bounding a region, Ω, we want to compute its motion in time influenced by a velocity field, v. Let us embed the surface as the zero level set ofϕ : ϕ ( xt, ) =± d (1.24) Where d is the signed distance from x to Γ ( t ) with these properties: 22

23 ϕ( xt, ) > 0 forx Ω ϕ( xt, ) < 0 forx Ω ϕ( xt, ) = 0 forx Ω (1.25) The boundary of Ω is the hyper-surfaceγ : Γ ( t) = x ϕ( x, t) 0 = = Ω (1.26) The evolution equation of ϕ ( xt, ) is: ϕ + v ϕ = 0 t (1.27) If we split the velocity vector into its components by the system defined by the normal and the tangent vector, thus v = vnn + vt t we get: ϕ + + = t ( vn n vt t ) ϕ 0 (1.28) Since N and ϕ point in the same direction and T is perpendicular ton, that is T ϕ = 0, we have: ϕ + vn n ϕ = 0 t (1.29) We can rewrite the terms in the above equation by: N 2 ϕ ϕ ϕ= ϕ= = ϕ (1.30) ϕ ϕ Inserting this in equation (1.29) gives us a new evolution equation describing the level set functionϕ s evolution in time: 23

24 ϕ + v n ϕ = 0 t (1.31) This equation is often referred to as the level set equation and was originally presented in [2]. The interface given by the zero level set is moved by the speed functionv in the normal direction if v is positive and in the opposite direction if v n k is negative. n Much of the challenge in interface problems is finding or approximating a speed function. If the task is to simulate physic behavior, the function may be given by known equations but in most cases, it is not. In chapter 4 we will take a look at the Variational Level Set Approach, where a speed function is obtained through an energy minimizing process. 3.2 Comparison of explicit and implicit definition In the previous chapter, we looked at general curve evolution. In this chapter, we will look at how to describe the curve or interface. The description of the interface determines which numerical methods that can be used. Two different approaches will be examined; the marker particles and level set method. We will look at the properties and compare the advantages and disadvantages of both perspectives Marker Particles A traditional approach to model moving interfaces is to discretize the lagrangian equation of motion given in the previous chapter and thereby representing the interface by so-called marker particles. These are tracked in time and the interface is reconstructed by connecting these particles by line segments in 2D or as polygons in 3D. The idea is illustrated in figure 7. 24

25 Figure 7: Marker particles - a discrete parameterization of a curve This approach is very intuitive but has some serious drawbacks: when marker particles come close together the calculations become very sensitive and computed curvature can change drastically from one particle to the next. Three fixes to target these problems are in general used: Smoothing of the speed function to keep distance between marker particles reasonable. Redistribution of marker particles according to arc length Filtering techniques to remove oscillations in the particle system The cost of using these fixes is that the equations of motions are altered in a nonobvious way resulting in smoothing of the singularities of the interface. Other issues to target are: Topologic changes - split/merge of regions When region shrinks or expands Formation of corners (singularities) If we consider two regions merging into one or a region splitting into two, it may require insertion or removal of particles and restructuring of the binding line segments. In 2D it takes a lot of work, but in 3D it is very difficult. When regions 25

26 shrink or expand, again insertions or removal is required and redistribution of the marker particles along the arc lines. This Lagrangian approach is based on a local rather than global representation of the front, it suffers from instability and topological limitations and a lot of hard work and bookkeeping are required Level Set Method versus Marker Particle Approach Topological changes The way the level set method handles topological changes is very simple. The front is described by the zero level set and there are no restrictions regarding splitting or merging of the level set, in fact it is done automatically without any form of bookkeeping. Compared to how the marker particle treats topological changes, the level set method is very appealing especially in 3D applications. Considering shrinking or expanding of regions, the marker particle approach has to remove or add particles and redistribute the distance between them on the curve. In the Level set method this is again done automatically. Local geometric information Local geometric information like normal and curvature is available and can be determined at a given point on curve using the level set approach. Higher dimensions The level set method can be used on curves, 3D and in higher dimension. In 3D, a lot of the bookkeeping of particle relations becomes difficult using marker particles. Performance issues If we disregard bookkeeping the marker particles method is a lot faster than the level set method. This is due to the overhead of extra dimension of data to be calculated in the level set method. There are some efficient schemes like narrow band methods that can speed up the level set methods considerably. 26

27 3.3 Re-initialization of a signed distance function Re-initialization is the process of rebuilding a level set function to ensure that it has the property of being the signed distance function to the interface. Only in the simplest cases of speed functions, the signed distance function is preserved as the level set function evolves. This is not the case for example for speed functions calculated on the entire computational domain in a multiphase environment, such as the speed functions obtained by the variational level set approach which will be presented later in this chapter. Another reason to re-initialize could be if one use a narrow band level set method, an optimization where the level set is only calculated in a band around the zero level set. When the front approaches the edge of the computational domain (the edge of the band) the band has to be reconstructed as a signed distance function. Several approaches to re-initialization exist; we will look at two different approaches: the Crossing times approach and the partial equation approach. A known problem for applying re-initialization procedures to signed distance functions is that the location of the interface has a tendency of moving under this procedure. This is obviously not desirable and something to consider when choosing a re-initialization method Crossing times The idea is to solve the Eikonal equation, T = 1, since time equals distance, by evolving the zero level set with a speed function F=1 in the direction of the normal. We compute the boundary s arrival time T for every grid point as the boundary crosses it. When a sign shift occurs the time is notated and that way we can obtain the distance map on one side of the interface. The procedure has to be applied twice, once for the interior and once with speed functions F=-1 for the exterior. A disadvantage of this re-initialization procedure is that it is very slow and for bigger datasets and especially in the 3D, the procedure is not recommendable. 27

28 3.3.2 Partial differential equation approach If we assume that an implicit function ϕ is close to being a signed distance and that it only needs to be straightened a little a procedure suggested in [] can be applied to the implicit function. The idea is to solve the partial differential equation to steady state: t ( )( ) ϕ + S ϕ ϕ = (1.32) Where S is a sign function given by: 1, if ϕ0 > 0 S( ϕ0) = 1, if ϕ0 < 0 0, if ϕ0 = 0 (1.33) The term ( ϕ 1) will try to straighten the dataset to obtain the gradient property of the signed distance function, ϕ = 1 and the direction is determined by the sign function, S. 28

29 4 Variational Level Set Approach The original Level Set Methodology [2] was designed to apply to problems with a clear distinction between inside and outside, i.e. object and background (non-object). Such a system can be represented by a level set where the positive part corresponds to the object, the negative part to the background and the zero level set to the boundary separating object from background. In the field of fluids mechanics, the level set method can be used to model two-phase flows (systems of two fluids). One phase is associated with the positive part of the level set while the other is associated with the negative part of the level set. This binary nature, either object or background, either phase one or phase two, limited the applications of the original level set method Multiphase Systems To generalize the level set method to handle multiphase systems (systems of more than two phases), Merriman et al. proposed in [1] to associate phases in a multiphase system with a level set each. The positive part of the level set representing the inside of the associated phase, the zero level set representing its boundary and the negative part of the level set representing the outside of the phase. Using multiple level sets introduces potential problems: the possibility of overlap and vacuum regions in the system. Early attempts to address these issues added a correction step to the evolution process removing vacuum and overlap by moving the zero level sets. The idea of the Variational Level Set Approach we will look at in next section was to incorporate the overlap and vacuum handling into the evolution equations for the level sets instead. ϕ + ϕ 1 + ϕ 2 + ϕ ϕ 3 + Figure 8: Two-phase and multiphase systems 29

30 4.2 The Variational Level Set Approach In [7] Zhao et al. proposed a new approach to model multiphase systems called a Variational Level Set Approach. It combines the level set method with a variational formulation by Reitich and Soner[4] and as in [1], phases are represented by a level set each. The behaviour of the system is modelled by energy terms and a stationary solution is searched for in an energy minimizing process. The phases can be coupled through a linear system. The overlap and vacuum issues in multiphase systems are addressed by constraints to the solution and incorporated into the resulting evolution equations The Variational Level Set Formulation In a multiphase system with n phases, each phase is associated with a level set function, ϕ, ϕ,.., ϕn. The energy E of the system is given by the integral over an 1 2 energy-functional in the entire domain Ω. The variational level set formulation to the multiphase problem is: Minimize E = f ( ϕ, ϕ,.., ϕn ) (1.34) Ω 1 2 Subject to the constraints Ω g ( ϕ, ϕ,.., ϕ ) = C, i = 1,.., m i 1 2 n i (1.35) System Energy The role of the system energy is to be minimized in an energy minimizing process. In the case of a physical based system the energy functional can i.e. represent physical properties like surface tension or i.e. potential energy. The energy functional can also be a penalty functional for unwanted behavior in the system or a combination of physical properties and penalties. Before looking at some examples of energy and constraint terms, some useful functions will be presented first. 30

31 The Heaviside function: H x ( ) 1, x 0 = 0, x < 0 (1.36) The Dirac delta function: d dx δ ( x) = H ( x ) (1.37) The volume (area) of a phase is then determined by: H ( ϕ ) (1.38) i Ω The surface (length) of a phase is: δ ( ϕ ) ϕ i i (1.39) Ω Then the bulk energy, the gravitational energy and the surface energy is given by: E E = ρ H( ϕ ) (1.40) bulk i i Ω = ρ H( ϕ ) gh (1.41) gravitational i i Ω = δ( ϕ ) ϕ γ (1.42) E surface i i i Ω where ρ i is the density, g is the magnitude and direction of the gravitational force, h is a height function and γi is the surface tension Constraints Constraints like volume (area) conservation laws or rules to prevent the development of vacuum or overlapping regions can be applied to the system as constraints to the 31

32 solution. If the phases in the system represent uncompressible fluids, the area (2D) or volume (3D) should be conserved. Overlap and Vacuum An important property of a multiphase system is to prevent overlap of phases and the development of vacuum regions. Overlap occurs at points where more than one level set is positive and vacuum occurs at points where all level sets are negative. At any point in the domain exactly one level set function must be present. The constraint can then be expressed as: Ω n i= 1 H ( ϕ ) 1= 0 i (1.43) Since it would require a huge set of constraints and thereby be impractical to enforce, it is replaced by a least square condition: 1 2 Ω 2 n H ( ϕi ) 1 = ε i= 1 (1.44) where ε > 0 and as small as numerical possible. The constraint is a degenerated constraint but good results were showed in [1] using it. Volume (Area) Conservation Some systems may require volume (area) conservation, i.e. a system of noncompressible fluids. The volume (area) of a phase is given by equation(1.38). The constraint for volume (area) conservation is simply: H ( ) = A i (1.45) Ω Where A i is a constant. 32

33 Geometrical Constraints Ω H ( ϕ ) H ( ψ ) = 0 i i (1.46) Energy minimization In [7] Rosen s gradient projection method [5] is used for the optimization process. The search direction for a local extreme is obtained by projecting the negative gradient on all active constraints. The evolution equations for the level sets in the system are: m ϕi f g j = λj, i = 1,..., n (1.47) t ϕ ϕ i j= 1 i where is the Lagrange multipliers and the constraints satisfy: λ i d dt Ω g ( ϕ, ϕ,... ϕ ) = 0, j = 1,..., m j 1 2 n (1.48) The Lagrange multipliers can be found as solution to the linear system of equations: m λ ( ϕg j ϕg k) = ( ϕgj ϕ f ) k k= 1 Ω Ω j = 1,.., m, for ϕ = ( ϕ,.., ϕ ) 1 m (1.49) In the case of no active constraints, the optimization process turns into a gradient descent. The energy functional might have local extremes, so the obtained solution might only be a local minimum. To remove stiffness around the zero level set, δ( ϕ) is replaced by ϕ. 33

34 PART II HUMAN EMBRYO ANALYSIS 34

35 5 Background This chapter will introduce the background information about In Vitro Fertilization, embryo morphology, evaluation of human embryos and the light micro images. This is not meant as an in-depth study of the process, but more to give an overview of the problem. The presented information will be the foundation of the discussion in the next chapter that will lead to the 3D model. 5.1 In Vitro Fertilization The process of performing fertilization outside the body of a female is called In Vitro Fertilization and the first baby of this technique was born in 1978 in England. The process includes basically these five steps: Ovary stimulation Egg retrieval Insemination Embryo Evaluation Egg transferring Figure 9: Steps of the In Vitro Fertilization process. Ovary stimulation The first step of the process is to give the woman a concentrated form of the natural hormone responsible for stimulating ovulation. In a normal menstrual cycle, one follicle (the fluid-filled sack within the ovaries containing the eggs) is matured and released. The hormone treatment will stimulate the ovaries to produce multiple follicles. 35

36 Egg retrieval When the eggs, through ultrasound inspection, are determined to be mature, the retrieval of the eggs can begin. The eggs are removed from the follicles and placed in separate dishes. Insemination The eggs are allowed to mature six to eight hours before insemination can begin. The insemination can be performed either by adding a number of sperm cells to each dish or by injecting a sperm cell directly into each egg. After fertilization, the cleavage begins, dividing the egg into first two cells and then four etc. These cells are called blastomeres. Embryo evaluation The development of the embryo is now being monitored and light microscopy images are obtained. The images are used to judge the quality of each embryo to select the best candidate embryos. Egg transferring One or more candidate embryos are transferred to the uterus after approximately 72 hours after the egg retrieval process. We are interested in the fourth step of the process, the evaluation of the human embryos and will in the next section talk about the morphology of the embryo and the criteria used in this evaluation step. 5.2 Embryo morphology Figure 10 shows an image of a human embryo and the morphology model used by biologists. The model is a high level model where details have been left out, but it is adequate in the consent of In Vitro Fertilization examinations. According to the model, the interior of the embryo consists of cells called blastomeres and smaller subparts referred to as fragments. The part of the blastomere containing the DNA information is called nucleus. The elastic membrane enclosing the interior of the embryo is called Zona Pellucida. 36

37 Blastomere Zona Pellucida Nucleus Fragments Figure 10: Human embryo and its corresponding morphology model. The membrane Zona Pellucida encloses the interior, consisting of four blastomeres (cells) and fragments. Left: HMC image, right: The corresponding morphology model used by biologists The membrane of a blastomere Membranes of animal cells are very thin layers of material that are semi-permeable and protect the interior of the cell against the surroundings. Some molecules such as oxygen and water can pass through the membrane while larger molecules and ions cannot. Transportation of water from a region of higher water concentration through a semi-permeable membrane to a region of lower water concentration is called osmosis. In plant cells the membrane is covered by a cell wall that allows the plant cell to maintain a higher difference in pressure across the membrane. Animal cells do not have cell walls and can only maintain smaller difference in pressure across the membrane. So the cells will either shrink or expand to equalize a higher difference in pressure across the membrane Blastomere Shape The shape and strength of an animal cell is determined by thin wire-shaped proteins. This construction is called the cell s cytoskeleton and works in principle like wiring a tent. The cytoskeleton possesses some elasticity, but if a cell is exposed to too much stress, the cytoskeleton might be damaged, thereby changing the cell s shape permanently. In figure 11, an image of an isolated blastomere of a human embryo is shown. It has not been possible to find specific information about the shape and strength characteristics of blastomeres. By inspection of a series of images, including the one 37

38 shown in the figure below, it seems reasonable to assume that the natural shape of a blastomere might be approximately like a sphere. Figure 11: A single blastomere extracted from a human embryo. 5.3 Qualitative evaluation criteria In general, the quality determination of the embryo starts 48 hours after the in vitro fertilization called day two. At this point the second cell division is expected to have taken place resulting in an embryo with four blastomeres. Light microscopy images are obtained and the images are inspected. Several qualitative issues are considered, i.e. the shape of the objects, but the major criteria used to assess the quality are: Number of blastomeres in the embryo Regularity of blastomere size Degree of fragmentation Embryos with higher cell numbers and regular appearing cells (blastomeres) and little or no fragmentation have a higher overall chance of implanting than do their counterparts with less cells, more irregularity and more fragmentation. [Advanced Fertility Center of Chicago] 38

39 Figure 12: Embryo quality at day 3 Left embryo is graded high quality, middle is graded less and right is graded poor quality. In figure 12, three embryos at the day three are shown. The first embryo has good regularity in size and shape and will get a high grade. The second embryo has some degree of fragmentation and some irregularity. The third embryo has a high degree of fragmentation and lacks blastomere development. 5.4 Hoffman modulation contrast light microscopy The embryos are transparent objects with refractive properties. When light travels through the embryo, it undergoes a change in phase and is deflected influenced by the different subparts of the embryo. Using a setup of lenses and modulation filters as described in [9] the optical gradients can be transformed into intensity variations in the microscopy image. This technique, called the Hoffman Modulation Contrast technique (HMC), converts positive gradients into bright intensities and negative gradients into dark intensities. Thus, the image formation is a result of the embryo s optical properties, the optics and the modulation filters of the microscope. The resulting images, see figure 13, gives an impression of viewing a 3D object illuminated by a light source placed in the top left corner. In the next section we will talk about how to retrieve an image series of the human embryo and what kind of information these images offer. 5.5 Microscopy image series of a human embryo When the quality of a human embryo is to be assessed, a series of microscopy images are obtained and inspected. It may vary from clinic to clinic; but in principle these are the steps for a semi-automatic images retrieval process: 39

40 The first step is to manually find the height in which the Zona Pellucida is in focus and calibrate the height axis of the coordinate system. Next step is to obtain a series of images with equal height steps. The image series in this thesis are obtained in the range ± 40µ m with 5 µ m steps. The in-focus heights of the different elements are located either by visual inspection or by an image analysis system. If we examine the images in Figure 13, we can see that every object is present in each image, but in different focus. For example if we look at image number 9, the Zona Pellucida is in focus, which we might expect due calibration. If we move up or down from this image we will notice that Zona Pellucida becomes more blurred. 40

41 1. z = 40µ m 2. z = 35µ m 3. z = 30µ m 4. z = 25µ m 5. z = 20µ m 6. z = 15µ m 7. z = 10µ m 8. z = 5µ m 9. z = 0µ m 10. z = 5µ m 11. z = 10µ m 12. z = 15µ m 13. z = 20µ m 14. z = 25µ m 15. z = 30µ m 16. z = 35µ m Figure 13: Sequence of HMC microscopy images. The images are focused through a range and the contrast enhancement gives the impression of object depth. The objects equator contours have same size and shape through the range but are blurred depending on the distance to the focal plane. 41

42 6 Analysis The objective is to develop a 3D model of the human embryo that can support the embryo evaluation step of the in vitro fertilization process explained in 5.1. The human embryo evaluation is currently based on qualitative measures and interpretation of 2D images. Quantitative measures such as volume information, shape characteristics and 3D positional information would be preferable. The model will be based on the morphology model from section 5.2. The available information, an obtained series of 2D light microscopic images of a human embryo, will be examined with respect to what kind of information that can be extracted from it. Information about blastomere properties is limited, so assumptions will have to be made to develop a model. 6.1 Information in images A HMC microscopic image series of the human embryo are the result of focused projections of 3D to 2D in a range of focal heights as described in section 5.4. To summarize the properties of HMC microscopic images: The contrast is most significant where light tangents the object. A so-called equator curve is formed where light tangents an object. Spatial information is difficult to infer in regions without significant contrast. The usual approaches for generating 3D models from volume data, for example CT or MR scanning data, cannot be used. To decide an alternative modeling approach, in the following sections it will be examined what information that can be extracted Shape and volume information Shape information is only available at the equator contour. The location of the equator contour will be discussed in the next sub section. With regard to volume information, an example with three different objects and their equator contours are shown in figure 14. The first is a sphere, the second a larger sphere vertically restricted by an object (only the cross section is shown) and the third a smaller sphere 42

43 horizontally restricted by two objects. Depending on the positioning and shape of the restricting objects, the equator curves for the three spheres might equal. Therefore, it is not possible to extract volume information directly from equator contours. Figure 14: Equal equator contours obtained from unequal objects Left: A relaxed sphere. Middle and right: spheres restricted by their surroundings Positional information We are interested in finding the positional information about the different parts of the human embryo. These parts will in the following discussion be referred to as objects. For an object with an equator contour located in a plane parallel to the focal plane, for example a perfect sphere, the equator contour s height information is given by the focus height of the image (or by interpolation between two images) where the equator contour is most in focus. For an object with a different orientated equator contour, different parts of the equator contour will be in focus at different focal heights and it is not possible to obtain an equator contour not partly blurred. The situation is illustrated in figure

44 Actual equator contour Focal plane Actual equator contour Focal plane Equator contours Figure 15: Equator contour variation due to focal height Focal plane in different heights results in variations of equator contours. For the object in figure 15 (left), the focal plane is placed at the highest point of the object s actual equator contour. The obtained contour is sharp to the right where the focal plane tangents the real contour and becomes more blurred at the left as the distance between the focal plane and the real equator contour increases. For the object in figure 15 (right), the focal plane is placed at the center height of the actual equator curve. The obtained equator contour is sharp in the middle and becomes more blurred as the distance between the focal plane and the real equator contour increases. An accurate description of the relationship between the actual equator contour and an obtained equator contour might require a more in-depth study of the used optics and the image formation process. But it can be observed that sharpness decreases as distance between the focal plane and the actual equator contour increases Concave objects As illustrated in figure 16, multiple equator contours can be obtained from a single concave object. To infer 3D geometry without other information than in the microscopy images would be very difficult since we cannot distinguish one concave object from three convex objects in the obtained images. 44

45 fp fp fp Equator contours Figure 16: Equator contours of concave objects For a concave object, different equator contours at different heights can be obtained. This makes it difficult to distinguish one concave object from three convex objects in an obtained image series Discussion Equator contour orientation in the general case Approaches to handle objects with equator contours not parallel to the focal plane: 1. Approximating that all objects have equator contours parallel to the focal plane and using the focal height giving the best average sharpness of the obtained equator contour. 2. Analyze the image formation process to construct an image formation function and to derive an inverse function if it exists. 3. Vary the focal plane, choose selective points for evaluation of sharpness and interpolate along these. 6.2 Modeling the human embryo The available information The objective is to model a human embryo based on available information. The available information was examined in the previous section and found to be: Shape information is only available at the equator contours 45

46 Volume information is not available, only area of cross section Positional information is only available at equator contours An equator contour can be obtained for a convex object and several equator contours for a concave object Developing a model based on a very limited amount of information requires assumptions. 3D information is only available at the equator contours. It needs to be decided how the information would be expected to be outside. Assumptions Blastomeres and Zona Pellucida possess minimal surface properties Blastomeres do not overlap Blastomeres or Zona Pellucida Blastomeres and Zona Pellucida are convex objects Restricting objects to known information We are interested in a system where objects can be deformed due to mutual interaction without overlapping and where object volumes are preserved. It should be possible to restrict blastomeres and Zona Pellucida with respect to positional to given equator contour curves, respecting geometrical constraints (bounding cylinders and plates) Modeling a blastomere The shape of a blastomere is determined by a structure of thin wire-shaped proteins called the cytoskeleton, as explained in section Two different models, a string model and a minimal surface model, for modeling a blastomere will be compared to select one. String model In a blastomere model, the thin wire-shaped proteins can modeled by strings as illustrated in figure 17 (left). Attaching the strings at given points at the membrane of the blastomere will fix the shape of the blastomere. Temporary surface deformations due to mutual interaction are possible. The string forces determine the elasticity of the blastomere. 46

47 A minimal surface model Assuming that the neutral shape of a blastomere is like a sphere, the blastomere can be modeling as a membrane encapsulating a fluid, as illustrated in figure 17 (right). With a higher pressure inside the membrane than the surroundings, the surface will obtain a minimal surface. Mutual interaction of blastomeres will result in temporary deformations of their surfaces. The elasticity of the blastomere is determined by the pressure. Membrane Strings Membrane p Figure 17: Shape modeling of blastomere The shape of a blastomere is modeled by strings (left) and by surface tension (right). The string model offers a more specific description of the blastomere s shape compared to the minimal surface model. But without general information about the cytoskeleton and with shape information only given at the equator contour, specification of strings placement is not possible. For modeling a human embryo based on the available information, the minimal surface model is found useful. In the next section, a method to restrict a minimal surface object with respect to available shape and positional information will be explained Restricting an object with respect to available information An isolated object with minimal surface properties will try to obtain the shape of a sphere. Mutual object interaction might cause temporary deformations of the objects surfaces or cause the objects to move. Since the available information to the model is very limited, it is important that the objects comply with the available information. Restrictions for the objects with respect to shape and positional information are needed. 47

48 Cylinder and plate restrictions The equator contour curve is formed by the points where rays of light tangents the surface of the object. The rays of light can be considered to form a cylinder passing through the obtained equator contour. The relationship between the cylinder and the object is that all points on the surface of the object have to be located on the cylinder or inside the cylinder. The cylinder can be used as a restriction to the object in the model, by requiring that the object does not intersect the cylinder. It ensures that all points on the surface of the object will stay inside the cylinder, but this does not prevent the object from shrinking or moving vertically and thereby not staying attached to the equator contour. To address this issue, a plate can be formed by the equator contour and inserted as another restriction to the object in the model, requiring that the plate has to be included in the object. The combination of the plate and the cylinder restricts the object to stay attached at the equator contour, to stay inside the cylinder and thereby to comply with the available information. The principle is illustrated in figure 18. Equator contour Plate Cylinder Object restricted by cylinder Object restricted by cylinder and plate Figure 18: Positional restriction of an object Using the equator contour, a bounding cylinder and a restricting plate can be generated. The cylinder prevents the object from expanding beyond the equator contour and the plate prevents the object from shrinking. 48

49 The presented method applies to convex objects, but a similar approach can be used to concave objects by inserting one cylinder and one plate for each equator contour. The problem as mentioned earlier is to distinguish one concave object from several convex objects. 49

50 PART III A VARIATIONAL LEVEL SET APPROACH TO MODEL A HUMAN EMBRYO 50

51 7 A Level Set Model of the Human Embryo In the previous chapter, we discussed conceptually how to construct an elastic model of a human embryo. In this chapter we will model the human embryo mathematically by applying level set methodology to it and state it as a variational level set problem. We will design a system where the objects in the model are represented by level sets. Object properties will be modeled mathematical as energies or constraints. Using Rosen s gradient projection method we will obtain evolution equations for the level sets that will minimize the system s energy in time. The system The system will include Zona Pellucida and blastomeres as illustrated in figure 19. The blastomeres and Zona Pellucida will be assigned physical properties such as a surface tension forces, a non overlapping property and volume conservation. Each blastomere and Zona Pellucida will be attached to an individual geometrical curve by a pair of geometrical constraining objects, a cylinder and a plate, see figure 19. Blastomere Zona Pellucida Cylinder Plate Figure 19: A level set model of a human embryo Level sets in the system Zona Pellucida and the blastomeres are dynamic objects and will be represented by dynamic level sets denoted ϕ0 and ϕ,.., 1 ϕn respectively in the system. The geometrical constraints, the bounding cylinders and the restricting plates, are static in 51

52 time and will be represented by static level sets denoted ψ and χ respectively. The interfaces of the objects are represented by the zero level sets. The sign convention of the level sets is illustrated in figure 20, where gray represents positive sign, white represents negative sign and black represents the zero level set in the level sets. i i Zona Pellucida Blastomere Cylinder Plate Figure 20: Sign convention for the system s level set functions. Grey denotes positive sign and white denotes negative sign. 7.1 Energies and constraints We are interested in modeling the properties of the blastomeres and Zona Pellucida. The properties are surface tension forces, non-overlapping properties and volume conservation. To assign these properties to the level sets we define energies in the system and constraints to the solution. Surface tension energy The surface tension energy for the level set ϕ i, a blastomere or Zona Pellucida, can be described as: Esurface = γδ i ( ϕi ) ϕi (3.1) Ω where γ i is the surface tension coefficient. Overlap of blastomere or Zona Pellucida The energy expressing overlap between the level set Pellucida, and the other level can be described as: ϕ i, a blastomere or Zona 52

53 n Eoverlap = H ( ϕi ) H ( ϕn ) H ( ϕi ) (3.2) Ω j= 0 Geometrical constraining object energies Each level set ϕ is assigned two constraining objects: a cylinderψ and a plate χ. i The energy expressing overlap between blastomere and Zona Pellucida respectively: ϕ i and its bounding cylinder is for a i i E = H ( ϕ ) H ( ψ ) (3.3) cylinder i i Ω Ecylinder = H ( ϕ0) H ( ψ0) (3.4) Ω The energy expressing overlap between the compliment of a blastomere or the Zona Pellucida and its restricting plate is respectively: Eplate = H ( ϕi ) H ( χi ) (3.5) Ω Eplate = H ( ϕ0) H ( χ0) (3.6) Ω Volume (area) constraint Area or volume preservation constraint: H ( ϕ ) = A i i (3.7) Ω where is the constant enclosed volume (area) for the blastomere or Zona A i Pellucida. 53

54 7.2 Energy minimization Overlapping regions and the surface tension energies are to be minimized under the constraint of volume preservation. The energy of the system is the sum of the energies defined above so the resulting system energy to be minimized is: E = Esurface + Eoverlap + Ecylinder + Eplate (3.8) Subject to the volume preservation constraint given in equation (3.7) Obtaining evolution equations for the level sets Using Rosen s gradient projection method as described in chapter 4, we obtain the evolution equations of the systems, a coupled set of partial differential equations for the dynamic level set functions. For a blastomere the evolution equation becomes: n ϕ i ϕ i = ϕ i γ i H ( ϕn) H ( ϕi) H ( ψi) H ( ) t + ϕ χi λ i (3.9) i j= 0 Vcylinder Vplate Vsurface V overlap where the Lagrange multiplier is: λ i = Ω n ϕ i δ( ϕi) ϕ i γ i H ( ϕn ) H ( ϕi) H ( ψi) H ( χ + i) ϕ i j= 0 cylinder plate surface overlap (3.10) δ( ϕ ) ϕ Ω i i For Zona Pellucida, the evolution equation is: ϕ 0 t = ϕ 0 n ϕ 0 γ 0 H ( ϕn ) H ( ϕ0) H ( ψ0) H ( ) + ϕ χ0 λ 0 (3.11) 0 j= 0 Vcylinder Vplate Vsurface V overlap Where the Lagrange multiplier is: 54

55 λ 0 = Ω n ϕ 0 δ( ϕ0) ϕ 0 γ i H ( ϕn ) H ( ϕ0) H ( ψ0) H ( χ + 0) ϕ 0 j= 0 cylinder plate surface overlap (3.12) δ( ϕ ) ϕ Ω 0 0 We notice that the evolutions equations for the blastomeres and Zona Pellucida are almost identical, only difference is the change of sign in the cylinder and plate parts of the equations. This is caused by difference in sign convention for the blastomere level sets and the Zona Pellucida level set. 7.3 System behavior summary Energy The energy of the system, given by equation (3.8) is the sum of blastomere overlapping, blastomeres overlapping their bounding cylinders, their plates or the Zona Pellucida and the blastomeres surface areas energies. Minimization Through a gradient descent approach the system energy will be minimized under volume constraint. For every time step we obtain an evolution equation (3.9) for each level set function with a corresponding Lagrange multiplier(3.10). Velocities The velocity part of the evolution equation decides whether the level set is moved in positive or negative normal direction. It consists of four local terms (one curvature term and three overlapping terms) and a global term (the Lagrange multiplier). The local terms are based on local properties and affect the level set locally, while the global term can either make the level set grow or shrink. Surface Area (tension) The first local contributor to the velocity function in the evolution equation is the surface area term. The term is in fact a curvature regulating term (see figure 2) and 55

56 the sign of the contribution depends on the sign of the curvature. This will cause a decrease of variations in the surface, thereby minimizing the surface area. Overlap When a blastomere overlap another blastomere, the overlap term becomes non zero in the region of overlap thereby contributing negative to the velocity function in this area. If this is the only contribution to the velocity function, it will enforce the interfaces to be moved away from each other. Similar, the term concerning cylinder overlap becomes active in the cylinder overlap region and contributes to the velocity function. Plate overlap works similar too, but due to reversed sign convention, the velocity contribution is positive instead. Lagrange multiplier The Lagrange multiplier represents the volume conservation and works globally. Stationary Solution In the energy minimization process we are interested in finding a minimum energy functional. We search by using a gradient descent approach which means that we travel in the negative first derivative direction. As the gradient approaches zero we approach a minimum energy of the system and we are aware that this minimum might only be a local minimum. This means that our system converges as the evolution equations approaches zero all along the zero level set for all our level sets. Due numerical errors, these evolution equations might not reach a zero. Instead, we will require that they are as small as possible, 56

57 8 Implementation In the previous chapter, a human embryo was modeled mathematically using level set methodology and formulated as a variational level set problem. Level sets, energies and constraints were defined and evolution equations for the level sets were derived. In this chapter, the implementation of the model will be described. First, an overview of the system will be given and later, discretization and implementation of equations and terms will be explained and handling of input and output will be explained. System overview The system consists of three modules as illustrated in figure below. The energy minimization module is the core of the system while the two other modules are supporting modules generating input and visualizing output. In the first module level set data files are generated from annotated image files. In the second module level set data files are red. An energy minimization process is performed on the level sets and level set data files are generated. In the third module, level set data files are read and visualized. Generation of input Energy minimization Visualization Figure 21: System Overview 57

58 8.1 Energy minimizing process The energy minimization process consists of four steps as illustrated in figure 22. In the first step of the process, the Lagrange multipliers and the evolution equations are calculated for the active level sets in the system. The obtained evolution equations are used in the second step to advance the level sets. In the third step the level sets are re-initialized (at every nth iteration) to secure their signed distance properties. In the fourth step of the process, the stationarity of the system is measured and evaluated. The energy minimizing process runs until a desired stationarity is obtained. Figure 22: Energy minimizing process In the following sections, the four steps will be explained more in details Calculation of evolution equations To implement the evolution equations for the level sets the evolution equations and the Lagrange multipliers (3.9) to (3.12) need to be discretized with respect to time and space. For a blastomere, the discretized evolution equation and Lagrange multiplier is respectively: m n m+ 1 m m ϕ i m m m+ 1 ( ) ( ) ( ) ( ) ) ϕ = ϕ + t ϕ γ H ψ H χ H ϕ H ϕ λ i i i i + m i i i i i ϕ i V V j = 0 cylinder plate (3.13) V V overlap surface 58

59 λ = m n m m ϕ i m m δ( ϕi ) ϕ γ H i i ( ϕn ) H ( ϕi ) H ( ψi) + H ( χi) m ϕ i j 0 cylinder plate surface overlap (3.14) δ( ϕ ) ϕ Ω = m + 1 i m m i i Ω where m denotes iteration number and t denotes the time step length. Integrals are replaced by summations over the discrete finite domain Ω. To implement these equations and to make a robust system good approximations to the terms in the equations are needed. Approximating the Heaviside and the delta function The Heaviside function and the delta function are approximated like in [7]. A transition bandα gives a slightly smoothing effect and removes stiffness when calculating the evolution equations at the boundary of the local active regions, i.e. a region of blastomere overlap. Heaviside function: 1, x > α 1 x 1 πx = 1+ + sin, α 2 α π α 0, x < α H ( x) α x (3.15) Dirac s delta function: δ 0, x > α d x = H x = 1 πx dx 1 + cos, x α 2α α ( ) ( ) α α (3.16) Approximating the gradient The evolution equations push information in the level sets. To make a good approximation to the gradients in the evolution equation (3.13) we need to take the information flow in the level set into considerations. The velocities, V, V, Voverlap and λi cylinder, propagate information in positive or negative direction normal to the front. The gradients for these must be approximated by entropy satisfying schemes plate 59

60 respecting the information flow. The term V surface is a curvature regulating term and can be considered to propagate information in both directions. This gradient can be approximated using central differences. A second order space convex ENO scheme has been selected for approximating the gradients in the evolution equation. By evaluating the direction of the velocity terms, it selects the correct direction of information for calculating the gradients. Due to the Heaviside terms in V, V and V the direction of information for these cylinder overlap velocities are known ahead and the scheme can be simplified. The resulting scheme for a simple time stepper is: plate m+ 1 m ϕ = ϕ + t ϕv + ϕv + ϕv + ϕv + + i ϕ max ( λi, 0) + i ϕ min ( λi, 0) i i i isurface i icylinder i iplate i ioverlap (3.17) Where the gradients are given by: ϕ= max ( A, 0) min ( B, 0) max ( C, 0) min ( D, 0) max ( E, 0) min ( F, 0) ϕ= min ( A,0) max ( B,0) min ( C,0) max ( D,0) min ( E,0) max ( F,0) x 2 0y 2 0z 2 2 ϕ= D ϕ + D ϕ + D ϕ (3.18) and ( ) A = D ϕ+ x m D ϕ, D ϕ, x 1 x x + x x 2 ( ) B = D ϕ+ x m D ϕ, D ϕ, + x 1 + x+ x + x x 2 ( ) C = D ϕ+ y m D ϕ, D ϕ, y 1 y y + y y 2 ( ) D = D ϕ+ y m D ϕ, D ϕ, + y 1 + y+ y + y y 2 ( ) E = D ϕ+ z m D ϕ, D ϕ, z 1 z z + z z 2 ( ) F = D ϕ+ z m D ϕ, D ϕ + z 1 + z+ z + z z 2 x if x y, xy > 0 m( x,y) = y if x > y, xy > 0 0 ifxy 0 (3.19) For calculating the Lagrange multipliers, the gradient terms are calculated by using central differences. 60

61 Approximating the curvature Curvature is approximated similar to the approximations used in [1] but has been extended to three dimensions: i, j, k ϕx ϕ ϕy ϕ x y ϕz ϕz ( ϕ) ( ϕ) ( ϕ) ( ϕ) ( ϕ) ( ϕ) ϕ = + + ϕ x y z 1 i+ 2, j, k 1 i 2, j, k 1 i, j+ 2, k 1 i, j 2, k i, j, k+ i, j, k (3.20) Where: ϕ ϕ x = ϕ ϕ i+ 1 2, j, k x = i 1 2, j, k ϕ y = ϕ 1 ϕ ϕ ij, + 2, k y = ϕ z ϕ ϕ ϕ ij, 1 2, k ijk,, ijk,, 1 2 = z = D + x ijk,, + x 2 1 0y 0y 1 0z 0z ( D ϕijk,, ) + ( 2( D ϕi+ 1, jk, + D ϕijk,, )) + ( 2( D ϕi+ 1, jk, + D ϕijk,, )) D x ϕ ϕ ijk,, 2 2 x 2 1 0y 0y 1 0z 0z ( D ϕijk,, ) + ( 2( D ϕijk,, + D ϕi 1, jk, )) + ( 2( D ϕijk,, + D ϕi 1, jk, )) D + y ijk,, y 2 1 0x 0x 1 0z 0z ( D ϕijk,, ) + ( 2( D ϕij, + 1, k+ D ϕijk,, )) + ( 2( D ϕij, + 1, k+ D ϕijk,, )) D y ϕ ϕ ijk,, 2 2 y 2 1 0x 0x 1 0z 0z ( D ϕijk,, ) + ( 2( D ϕijk,, + D ϕij, 1, k) ) + ( 2( D ϕijk,, + D ϕij, 1, k) ) D + z ijk,, z 2 1 0x 0x 1 0y 0y ( D ϕijk,, ) + ( 2( D ϕijk,, D ϕijk,, )) + ( 2( D ϕijk,, D ϕijk,, )) D z ϕ ϕ ijk,, 2 2 z 2 1 0x 0x 1 0y 0y ( D ϕijk,, ) + ( 2( D ϕijk,, + D ϕijk,, 1 )) + ( 2( D ϕijk,, + D ϕijk,, 1) ) (3.21) 2 2 And D, D and given by: D + 0 denotes forward, backward and central differences respectively 61

62 ϕ ϕ ϕ ϕ ϕ ϕ D ϕ, D ϕ, D ϕ x x 2 x ϕ ϕ ϕ ϕ ϕ ϕ D ϕ, D ϕ, D ϕ y y 2 y ϕ z i, j, k+ 1 ϕ + i, j, k ϕ ϕ ϕ ϕ z 0z D ϕi, j, k, D ϕi, j, k, D ϕi, j, k z z 2 z (3.22) + x i+ 1, jk, ijk,, x ijk,, i 1, jk, 0x i+ 1, jk, i 1, jk, i, j, k i, j, k i, j, k + y i, j+ 1, k i, j, k y i, j, k i, j 1, k 0y i, j+ 1, k i, j 1, k i, j, k i, j, k i, j, k i, j, k i, j, k 1 i, j, k+ 1 i, j, k Advancing the level sets To implement the evolution equation, we use a simple time stepping algorithm of the form: m 1 m ϕ ϕ t ϕ + = V (3.23) In the above equation, m is the iteration and t is the step length in time. The Heaviside function and the curvature term will be implemented as for the Lagrange calculation Re-initialization of a level set The signed distance property of the level sets is not maintained under the evolution of the level sets in time. The purpose of the re-initialization step is to address this problem. The crossing times method described in has been selected for this step. The crossing times method is computational expensive but accurate and does not move the interface during the re-initialization. The following ENO scheme is used: m 1 m ϕ + i = ϕi + t + i ϕ max ( v, 0) + iϕ min ( v, 0) (3.24) The two gradient terms are calculated like in equation (3.18). Interface crossing of each grid points is detected by sign shift and a value for the grid point in the new level set is calculated by time of occurrence. The process is run twice, once with the velocity function v = 1 and once with the velocity function v = 1. 62

63 8.1.4 Evaluation of system stationarity For the system to be stationary, the level sets should stop moving and to obtain this the evolution equations should be zero: n i= 0 Ω ϕ i = 0 (3.25) t Since the interfaces of the objects are represented by the zero level sets, only the zero level sets are interesting with respect to measuring and evaluating the stationarity of the system. To calculate the activeness of the zero level sets, their movements can be summed and normalized with respect to contributing grid points. The total movement of the zero level sets is then measured as: Q = n i= 0 ( m ϕi < B ) n ϕ i= 0 ( m+ 1) ( m) i i M i ϕ (3.26) where M i is the number of grid points lying in some distance B to the zero level set Gradient precautions A continuous signed distance function is discontinuous at local maximums, resulting in undefined gradients at the local maximums. For a discreet signed distance function, the gradient at local maximums will be zero or close to zero. In the system the gradient is used in the evolution equation, in the calculation of the Lagrange multiplier and in the curvature term. The influence of a zero gradient in these equations will be examined to decide if precautions should be taken in the implementation. The evolution equation In the general level set evolution equation in(1.31), the level set is evolved in time by applying a velocity function to the gradient. For point in the level set with a zero 63

64 gradient, the evolution equation turns zero and the point will not evolve. In time flat regions might develop in the level set. The Lagrange multiplier The calculation of the Lagrange multiplier in equation (3.14) involves two gradient terms, one in the numerator and one in the denominator. Both are multiplied by the delta function that in the discrete approximation returns zero for points with larger distance than α to the interface. For a zero gradient to influence the numerator it has to be located within distance α to the interface. For the denominator to be zero, all points in the interior of the object are required to be located within distanceα to the interface. The curvature term Curvature is calculated in (3.20) and (3.21) by calculating the divergence of the gradient. In the calculation of the normalized partial derivatives in (3.21), a zero gradient in the denominator is only possible if the partial derivative in the numerator is also zero. Therefore, a zero gradient in the curvature calculation is not considered a numerical problem. Gradient precautions The Lagrange multiplier is closely connected to the evolution equation, so zero gradients should be treated equal in these two equations. For the evolution equation, points with a zero gradient will not evolve, but the re-initialization step will recalculate the signed distance function and thereby remove flat regions. For the Lagrange multiplier, assuming that re-initialization is done often, only very narrow or small objects with zero gradients near to the interface can cause problems. Even though the re-initialization step will correct some problems, a zero denominator in the Lagrange multiplier equation (3.14) is still possible. Therefore, zero gradients in equation (3.14) and (3.13) has been chosen to be replaced by the ideal gradient length in a signed distance function: if ( ϕ < ε) ϕ = 1 (3.27) where ε is a small positive number close to zero. 64

65 For the curvature term, it does not make sense to take the same precaution in equation (3.21) since a zero gradient yields a zero partial derivative. Instead, a precaution against division by zero can be taken, assigning a normalized partial derivative to zero if a zero gradient is detected Boundary of computational domain The output of functions like the Heaviside function and the delta function is calculated on a single grid point in the computational domain. Functions like difference functions or curvature functions use grid points in some neighborhood to calculate their output. At the boundaries of the computational domain, some of the grid points necessary for calculation a function might not be available. It has been selected to make use of as much information as possible. In the following discussions, the x-dimension is assumed to be[ 1, N] for the computational domain. Central differences gradient The calculation of the gradient using central differences requires a four neighborhood for a grid point in 2D and a six neighborhood in 3D. At the boundaries of the computational domain not all these grid points are available. To compensate for the missing information it has been chosen to replace the central difference term by a forward differences term at the minimum boundary and by a backward term at the maximum boundary of the computational domain. In the table below, the scheme for the partial derivative with respect to x is shown for the calculation of the central differences gradient in equation(3.18). Table 1: Boundary scheme for central differences gradient n 1 2..N-1 N 0x D ϕ x D + ϕ 0x D ϕ x D ϕ 65

66 ENO Scheme gradient To calculate the gradient at a point using the second order ENO scheme in (3.17), nine grid points are necessary in 2D and thirteen grid points in 3D. For a first order ENO scheme the terms A and B in (3.17) are replaced by backwards and forwards differences respectively. A first order ENO scheme requires only four points in 2D and six points in 3D. In the calculation of A, B, C, D, E and F in(3.19), the highest order scheme possible to construct on available information will be used. At the first point in the computational domain, n=1, it is not possible to construct a backwards term for A. At the second point, n=2, only points for constructing a first order backwards term for A are available. In the range [3, N-1] points are available for constructing second order backwards terms for A. In the last point, n=n, only a first order backwards term can be constructed for A. For calculating B in equation(3.19), a second order forward term can be constructed in the range [2,N-2]. At the points n=1 and n=n-1 only a first order term can be constructed for B and at n=n it is not possible to construct a forward term for B. The calculations for A and B are shown in the table below. The calculation of C, D, E and F are equivalent to the calculation of A and B. Table 2: Boundary scheme for second order ENO gradient scheme N A n B N-1 N x ϕ + 2 x D ϕ ( ϕ ) x x x + D m D,D 2 N-2 x x ϕ N-1 x D + ϕ x ϕ + ϕ ϕ 2 +x x ( ) + x + x+ x D m D,D x D + ϕ x D ϕ N 0 Curvature To calculate the curvature (3.20) in a given grid point, all points around it in a square in 2D or 3 3 cube in 3D are needed. For all points at n=1 or n=n, all necessary points are not available for calculating the curvature. Both the forward term and

67 backward term are needed in the calculation, so to try to construct some scheme at the boundary will be a difficult task. The curvature will instead be assigned to zero at the boundary. The result is that the level set will not be evolved by curvature at the boundaries. Since our main concern is curvature near the zero level set and by assuming that the zero level set stay off the boundaries, the lack of curvature flow at the boundaries is acceptable. 67

68 9 Test and results The system models a human embryo by a variational level set approach. Zona Pellucida and blastomere are represented by level sets. Physical properties, nonoverlapping criteria and geometrical constraints are modeled as energies in the system. Velocities for the level sets evolution equations are derived using Rosen s gradient projection method to minimize the energy of the system in time under a volume conservation constraint. Energies Regions of blastomere or Zona Pellucida overlap, cylinder overlap or plate overlap induces energies in the system. The velocities in the obtained evolution equations (3.9) and (3.11) are expected to minimize these overlap energies by minimizing the regions of overlap. The surface energy is expected to be minimized by straightening the surfaces. Volume conservation Volume conservation is imposed to the system as a constraint in the energy minimizing process. It contributes globally to the velocities in the blastomeres and Zona Pellucida s evolution equations by the Lagrange multipliers in equation(3.10) and (3.12). Without requiring volume conservation in the system, the volumes of the blastomeres and Zona Pellucida would shrink to zero to make the surface energies vanish. The Lagrange multiplier is expected to keep the sum of velocities zero at the object s interface and thereby conserving the object s volume. Solution with minimum energy We are interested in obtaining a stationary solution with minimum system energy. The gradient descent approach leads us to a local minimum and as we approach this energy minimum, the velocities at the interfaces should vanish. This means that the system converge when the velocities at the interface reach zero, thereby resulting in a solution with (local) minimum energy. Measuring the velocities at the interface measures the degree of stationarity in the system. 68

69 In conclusion the behavior of the system is controlled by the velocities in the evolution equations that will minimize the energies in the system: Surface energies Blastomere and Zona Pellucida overlap energies Geometrical constraint energies Volume conservation constraints The solution we are interested in should be a solution with: Minimum energy Stationary at the zero level sets Test objectives Based on the description of the system and the behavior of the system the following test objectives can be derived: 1. Test of system behavior 2. Test of volume conservation constraint 3. Test of convergence of the system To reach the first and second objectives the system will be examined in the fundamental overlapping cases. The energies in the system and the velocities in the evolution equation will be examined to verify system behavior. The system will be tested in time and volume conservation will be examined. The third objective will be reached by testing the system on real input data. Different setups will be used and we will examine system convergence. 9.1 System behavior test In this section we will test the behavior of the system. The four fundamental cases of overlap in the system are shown in figure

70 First we will examine the energies in the system. Then we will examine the velocity fields and finally examine the evolution equations that evolve the blastomeres in time. Cylinder Overlap Plate Overlap Overlap Blastomere Overlap Zona Pellucida Figure 23: Four cases of overlap Energies Input data Two level sets of dimension have been generated for this test, a blastomere and a cylinder level set, both generated as signed distance functions. The blastomere s zero level set forms a circle of radius 30 and center in (50, 50). The cylinder s zero level set is generated by two lines, x=25 and x=75. The two level set are shown in figure 24. In the test the transition band delta and Heaviside functionsδ ( x) and H ( x). α α α is set to 1 in the discrete 70

71 Figure 24: Blastomere and cylinder level sets The blastomere level set (left) and the cylinder level set (right) Non-overlap energy test The test is initiated with a blastomere in the system and no restricting objects. Energy is expected to be located at the blastomere s interface. The energy of the system is expected to be equal to the blastomere s surface energy: E surface Non-overlap energy test results The resulting system energy values of the test are shown in table 3. The only contributor to the system energy is the surface energy. An energy plot of the system is shown in figure 25 (left), where black represents zero energy and white represents maximum energy. The energy plot forms a circle with radius 40 and centered at (50, 50). The results agree with our expectations. Table 3: Energy results of the surface energy test E surface E overlap E cylinder E plate E

72 Figure 25: Energy plots of tested system energy Energy plot of the blastomere (left) and the cylinder overlapping case (right). Overlap energy test: The cylinder case A cylinder overlapping the blastomere is inserted into the system. The expected result is an increase in system energy. Energy is expected to be located at regions where the cylinder and the blastomere overlap and at the surface of the blastomere. The system energy is expected to be the sum of the surface energy and the cylinder overlapping energy: E surface + E cylinder Test results The resulting system energy values of the test are shown in table 4. The contributors to the system energy are the surface energy and the cylinder overlap energy. An energy plot of the system is shown in figure 25 (right). As expected, the energy is located at the surface of the blastomere and at the regions where the blastomere overlaps the cylinder. For the overlapping regions, the energy is smaller at x=25 and at x=75. The transition band of the discrete Heaviside function causes this in the cylinder energy term. The energy at the surface of the blastomere seems smaller (except where both surface and overlap energy is present) compared to 72

73 the previous test with a blastomere alone. This is caused by normalization of intensity values in the energy plot. Table 4: Energy results energies in the four overlap cases Overlap case E surface E overlap E cylinder E plate E Cylinder Summary or energy test results For a system with a blastomere alone, the test results showed that the energy of the system is the surface energy of the blastomere and that the energy is located at the surface of the blastomere. For a system with a blastomere overlapping a cylinder, the system energy is the sum of the cylinder overlapping energy and the surface energy. The energy is located at the region of overlap and at the surface of the blastomere Velocities The velocities at the zero level set are induced by the energy terms in the system and the volume preservation constraint. We will examine the surface velocity, the Lagrange multiplier and the cylinder velocity. Surface velocity and Lagrange multiplier test Surface velocity is expected in the system due to the curvature of the surface. The Lagrange multiplier is expected to be active to prevent the blastomere from shrinking. The velocity field for the single blastomere is expected to be: Vsurface λ The Lagrange multiplier is expected to be equal to the average sum of the surface velocities at the zero level set. The average sum of the resulting velocities at the zero level set is expected to zero for the Lagrange multiplier to prevent the blastomere from shrinking. 73

74 The surface velocity in evolution equation (3.9) contains a curvature term. Higher level sets in a signed distance function will have higher curvature. We expect the surface velocity field to peak at the center of the blastomere and flatten as the distance to the center increases. Results The test results are shown in table 5 and a plot of the surface velocity field is shown in figure 26 (left). The average sum of velocities at the zero level set is very close to zero with a difference considered a numerical error. Due to the sign, the result will be at small area loss after evolving the level set. The shape of the surface velocity field is as expected. Higher velocities will be applied to higher-level sets and the signed distance property of the level set is not preserved. Mean value over the entire domainω of the level set: ( v) Ω µ Ω = Ω v 1 (3.28) Mean value over the interface Ω of the level set: µ v = ( ) Ω Ω Ω v δ( ϕ) ϕ (3.29) Table 5: Velocity results for surface velocity and Lagrange multiplier µ Ω ( V surface ) λ µ Ω ( V surface λ) ( Vsurface λ) e-015 Ω Cylinder overlap velocities By studying the cylinder overlapping velocity in the evolution equation(3.9), we notice that the velocity is induced by the cylinder s positive level set. The expected result is a velocity field at the region of the cylinder level set. 74

75 Results The obtained result is shown in figure 26 (right). In the region of the cylinder, negative velocities of magnitude 1 are induced regardless of overlap. For a blastomere that does not overlap the cylinder, the negative part of the blastomere level set will still be moved in negative direction. The re-initialization procedure will correct this by straightens the level set. Figure 26: Surface and cylinder velocity fields Surface velocity field (left) and cylinder velocity field (right) 9.2 Test of the human embryo in 2D We will now test a human embryo consisting of Zona Pellucida and two blastomeres and their respectively restricting cylinders and plates. These are the test objectives: Test system behaviour Test minimization of system energy Test volume conservation in the system Test stationarity of the system Input data The level sets used in this test are of dimension Zona Pellucida is constructed with a zero level set as a circle and the two blastomeres are constructed with zero level sets as ellipses; see setup information in table 6. The level sets are re- 75

76 initialized to signed distance functions. Each cylinder has been constructed with zero level set defined by two lines with distance 1 from its plate. In the test the transition α δ ( ) ( ) α α band is set to 1 in the discrete delta and Heaviside functions x and H x. The time step length t = and re-initialization step length t= The surface tension has been set to 1 for Zona Pellucida and 0.5 for the blastomeres. The level sets are re-initialized for every 25 step. Circle: ( x x ) + ( y y ) = R (3.30) Ellipse: ( x x ) 2 ( ) 2 0 y y0 + = 1 (3.31) 2 2 a b Table 6: Setup information for level sets x0 y0 R a b Zona Pellucida Blastomere Blastomere Plate for Zona Pellucida Plate for blastomere Plate for blastomere System behavior test The human embryo setup is shown in the first image of figure 27. The initial positioning and shape of the two blastomeres results in a region of overlap in the center of Zona Pellucida. The cylinders and plates have been placed to restrict the blastomeres and Zona Pellucida to some equator contour curves. The plate for Zona Pellucida has for visualization matters been selected to be small and will not influence the test. 76

77 The expected behavior of the system is that regions of blastomere overlap will vanish in time by moving the blastomeres. Blastomeres will be restricted by their cylinder and plate constraining objects while respecting the minimal surface properties. Results The result, a sequence of level sets, is shown in figure 27. We notice that the region of blastomere overlap is reduced in time and has vanished at time5 & 6. To maintain the areas of the blastomeres, the blastomeres are moved apart. Since the cylinders and plates restrict the blastomeres, this causes the blastomeres to be squeezed against the cylinders and against Zona Pellucida. By examining the shape of blastomere1 at the left corner of its plate, we see that the shape of the blastomere changes from round at time1 to a more flat shape in time. The blastomeres are kept inside their cylinder at a distance of 1 grid point. This is caused by the transition band of 1 in H ( x). α By examining the shape of the blastomere at the corners again, we notice that when a blastomere is squeezed against its cylinder, it is slightly broader at the height of the plate than at the rest of the cylinder. At these points, both the velocity field of the plate and the cylinder is active with opposite directions, so the plate velocity will neutralize the cylinder velocity slightly Energy minimization & volume conservation To test the energy minimization and the volume conservation in the system, we will examine the energies in the system and the volumes of the objects in time. We expect the evolution equations of the system to minimize the system energy in time. As the region of overlap vanishes, the overlap energy should vanish too. The elliptic shape of the blastomeres deforms and becomes more flat, which should result in increased surface energy. Due to numerical errors, we might expect some area loss in the system. Area loss will result in decreased surface energy as the objects become smaller, decreased cylinder energy and increased plate energy. Results The obtained energy functions are showed in figure 28. The system energy decreases in time, to reach a minimum level after approximately 4000 iterations. The major 77

78 contributors to the system energy are the object overlapping energy and the surface energy, while the contributions of the cylinder and the plate energies are relatively small. Area loss in the system calculated over the first iterations is 0.077%. Using a simple time stepper with a step length of 0.1, the area loss is acceptable and we can conclude that volume conservation works. We notice that when the overlap energy reaches zero, the surface energy reaches its maximum, corresponding to maximum stress upon the blastomeres to avoid overlap. The surface energy falls after this maximum and follows the slope of the area function. For the cylinder and plate energy functions, we notice opposite slope characteristics with a maximum of cylinder energy at the minimum of plate energy. This correspondent well to the functionality of restricting the object s zero level set between the cylinder and the plate. 78

79 Figure 27: Results of 2D human embryo test 79