Level Sets Methods in Imaging Science

Transcription

1 Level Sets Methods in Imaging Science Dr. Corina S. Drapaca Pennsylvania State University University Park, PA 16802, USA Level Sets Methods in Imaging Science p.1/36

2 Textbooks S.Osher, R.Fedkiw, Level Set Methods and Dynamic Implicit Surfaces, Applied Mathematical Sciences vol. 153, J.A.Sethian, Level Set Methods and Fast Marching Methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science, Cambridge University Press, 2 nd edition, Handbook of Mathematical Models in Computer Vision, edited by N.Paragios, Y.Chen, O.Faugeras, Springer, Geometric Level Set Methods in Imaging, Vision, and Graphics, edited by S.Osher, N.Paragios, Springer, Level Sets Methods in Imaging Science p.2/36

3 Level Set Methods Originally introduced by S.Osher, J.A.Sethian, Journal of Computational Physics 79, pg.12-49, Extensively used in computer vision CAD (computer-aided design) optimal design and control computational geometry computational fluid mechanics computational physics (two-phase flows, shocks, solid-fluid coupling, epitaxial growth, etc.) image processing (restoration, segmentation, registration, deblurring/denoising, reconstruction of surfaces from unorganized data points, etc.) Level set methods add dynamics to implicit surfaces. (Osher & Fedkiw book) useful in analyzing and computing the motion of an interface bounding a (multiply connected) open region under a given velocity field. Level Sets Methods in Imaging Science p.3/36

4 Implicit Functions In R n : (Sub) domains are n-dimensional. Interface: the border between two subdomains (or between the inside and outside of a domain). Interfaces have dimension n 1 and codimension 1. Interface representations: explicit: list of all the interface points using a parametrization of the interface. implicit: the isocontour of a (implicit) function φ( x) = C, x = (x 1,..., x n ) R n, C=const. For simplicity, C = 0. We will consider only the case of simple closed interfaces. A simple (left) and a non-simple (right) closed interface. Level Sets Methods in Imaging Science p.4/36

5 Implicit Functions Examples: Implicit function φ(x) = x 2 1 in R defining the inside region Ω = ( 1, 1) and the outside region Ω + = (, 1) (1, + ) as well as the boundary (interface) Ω = { 1, 1} (left). Implicit function φ(x, y) = x 2 + y 2 1 in R 2 defining the inside region (unit open disk) Ω = {(x, y) R 2 x 2 + y 2 < 1}, the outside region Ω + = {(x, y) R 2 x 2 + y 2 > 1} and the interface (unit circle) Ω = {(x, y) R 2 x 2 + y 2 = 1} (right). Level Sets Methods in Imaging Science p.5/36

6 Implicit Functions The explicit representations of the two interfaces in our examples are: in R: a list of points { 1, 1} (dimension 0) in R 2 : one possible paramaterization of the unit circle is: x = cos s, y = sin s, 0 s 2π We assume that the interfaces parametrizations are such that the interior regions are on the left side as the parameters increase. The connectivity (ordering) of the points of an interface contains important information about the topology (shape) of the interface. In addition, a moving interface can change its topology and hence its connectivity. Active contour: name used in the image processing literature for a moving (dynamic) interface. Level Sets Methods in Imaging Science p.6/36

7 Limitations of the explicit representation of an interface: it is hard to parametrize arbitrary curves in R 2 and hyper-surfaces in R n, n 3. the connectivity of hyper-surfaces in R n, n 3 is very difficult to represent; hence explicit interfaces cannot handle easily interfaces with holes and topological changes. Advantages of implicit interfaces: Going from R 2 to R n, n 3 is straightforward: For example, the zero isocontour of the implicit function φ(x, y, z) = x 2 + y 2 + z 2 1 is the implicit representation of the unit sphere, the interface between the inside open unit ball Ω = {(x, y, z) R 3 φ(x, y, z) < 0} and its exterior open region Ω +. Topological changes and discontinuous interfaces are handled naturally, there are no problems with the connectivity of implicit interfaces. Implicit functions make Boolean operations, geometry and calculus operations easy to apply and use. Level Sets Methods in Imaging Science p.7/36

8 Boolean operations: If φ 1 and φ 2 are two implicit functions, then the implicit representation of: the union of the interior regions of φ 1 and φ 2 is φ = min(φ 1, φ 2 ). the intersectionof the interior regions of φ 1 and φ 2 is φ = max(φ 1, φ 2 ). the complement of the interior region of φ 1 is φ = φ 1. Level Sets Methods in Imaging Science p.8/36

9 Boolean operations: If φ 1 and φ 2 are two implicit functions, then the implicit representation of: the union of the interior regions of φ 1 and φ 2 is φ = min(φ 1, φ 2 ). the intersectionof the interior regions of φ 1 and φ 2 is φ = max(φ 1, φ 2 ). the complement of the interior region of φ 1 is φ = φ 1. Geometry: If φ is a smooth enough implicit function then: gradient of the interface: ( φ φ( x) =, φ,.., φ ), x = (x 1, x 2,.., x n ) R n x 1 x 2 x n unit outward normal to the interface: N = φ φ mean curvature of the interface: κ = N = ( ) φ φ Level Sets Methods in Imaging Science p.8/36

10 Boolean operations: If φ 1 and φ 2 are two implicit functions, then the implicit representation of: the union of the interior regions of φ 1 and φ 2 is φ = min(φ 1, φ 2 ). the intersectionof the interior regions of φ 1 and φ 2 is φ = max(φ 1, φ 2 ). the complement of the interior region of φ 1 is φ = φ 1. Geometry: Convex regions have κ > 0, and concave regions have κ < 0. Level Sets Methods in Imaging Science p.8/36

11 Calculus operations: Characteristic functions χ and χ + of the interior region Ω and, respectively, exterior region Ω + are: { χ 1, if φ( x) 0 (i.e. x Ω ) ( x) = 0, if φ( x) > 0 and χ + ( x) = { 0, if φ( x) 0 1, if φ( x) > 0 (i.e. x Ω + ) Since χ ± are functions of a multidimensional variable x, it is easier to replace these characteristic functions with the one-dimensional Heaviside function H(φ) = { 0, if φ 0 1, if φ > 0 Level Sets Methods in Imaging Science p.9/36

12 Calculus operations: Volume (area, length) integrals of a function f are: f( x)χ ( x)d x = f( x)(1 H(φ( x)))d x, R n R n and R n f( x)χ + ( x)d x = R n f( x)h(φ( x))d x, representing the integrals of f over Ω and, respectively, Ω +. Dirac delta distribution: the directional derivative of the Heaviside function H in the normal direction of the normal N: ˆδ( x) = H(φ( x)) N = H (φ) φ( x) = H (φ) φ( x) = δ(φ) φ( x) φ( x) φ( x) since φ( x) φ( x) = φ( x) 2 and δ(φ) = H (φ) (the one-dimensional Dirac delta distribution). Level Sets Methods in Imaging Science p.10/36

13 Calculus operations: Surface (line or point) integral of a function f over the boundary (interface) Ω between the inside Ω and outside Ω + regions is: fds = f( x)ˆδ( x)d x = f( x)δ(φ) φ( x) d x. Ω R n R n By embedding the volume and surface integrals in higher dimensions, the above formulas avoid the need for identifying inside, outside, or boundary regions, making the numerical integration easier. The numerical approximations of the above integrals require that the Heaviside function H and the Dirac distribution δ be smooth. Level Sets Methods in Imaging Science p.11/36

14 Calculus operations: Possible regularizations of H and δ: C 2 regularization of H H 2,ǫ (φ) = δ 2,ǫ (φ) = dh 2,ǫ(φ) dφ 0, [ if φ ǫ φ 2 ǫ + 1 ( )] π φ ǫ sin, if φ ǫ ǫ 1, if φ > ǫ = 0, [ if φ > ǫ ǫ + 1 ( )] π φ ǫ cos, if φ ǫ ǫ C regularization of H H,ǫ (φ) = 1 ( ( )) φ 2 π arctan, δ,ǫ (φ) = dh,ǫ(φ) = 1 ǫ dφ π As ǫ 0, H 2,ǫ, H,ǫ H and δ 2,ǫ, δ,ǫ δ. ǫ ǫ 2 + φ 2 Level Sets Methods in Imaging Science p.12/36

15 Level Set Functions What is a good choice of an implicit function φ? Signed distance functions: a subset of the implicit functions φ that are positive on the exterior region, negative in the interior region, and zero on the interface. Examples of signed Euclidean distance functions: for the interface { 1, 1} in R: φ(x) = x 1 has φ = 1, x 0. for the unit circle {(x, y) R 2 x 2 + y 2 = 1} in R 2 : φ(x, y) = x 2 + y 2 1 has φ = 1, x 0. for the unit sphere {(x, y, z) R 3 x 2 + y 2 + z 2 = 1} in R 3 : φ(x, y, z) = x 2 + y 2 + z 2 1 has φ = 1, x 0. All signed Euclidean distance functions φ satisfy φ = 1 almost everywhere, so N = φ, κ = φ. However, avoid using the above expressions for N and κ since φ 1 when using numerical approximations. Level Sets Methods in Imaging Science p.13/36

16 Motivational Example of the Level Set Method in 2D: Sea Level = interface; Oceans = interior region; Mountains = exterior region; φ(x, y, τ) = 0 moving interface; φ(x, y, 0) = signed Euclidean distance y z y x x a. Initial Circle b. Initial Surface y z y x x c. Circle at time d. Surface at time Level Sets Methods in Imaging Science p.14/36

17 Motivational Example of the Level Set Method in 2D: The surface on the right is called the level set function (geometric active contour or signed distance function). It accepts as input any point in the plane and hands back its height as output. The red cross-section through the surface is called the zero level set. It is the collection of all points that are at height zero. Basic idea: instead of moving the red interface in 2D, we move the surface (level set function) in 3D. Mathematically, the level set method tracks tbe motion of an interface as the zero level set of the signed Euclidean distance function. Level Sets Methods in Imaging Science p.14/36

18 Evolution Equation for the Level Set Function φ: The level set value of a particle moving on the interface with path x(τ) is always zero: φ( x(τ), τ) = 0 Differentiate the above with respect to τ φ τ + φ( x(τ), τ) d x(τ) dτ = 0 with φ( x(0), 0) given. If the particle velocity is known (1) d x(τ) dτ = V then: (2) φ τ + V φ( x(τ), τ) = 0 Level Sets Methods in Imaging Science p.15/36

19 Evolution Equation for the Level Set Function φ: When V = F N = F φ φ, then (3) φ τ + F φ = 0 The level set function φ is used not only to represent the interface but also to evolve the interface. The level set equation (evolution equation): equation (2) or the equivalent equation (3). Use equation (2) when the interface is moved by an externally generated velocity field V not dependent on the level set function φ only the physics of the problem of interest evloves the interface. Level Sets Methods in Imaging Science p.16/36

20 Evolution Equation for the Level Set Function φ: Use equation (3) when the interface is moved by a self-generated velocity field V that depends directly on the level set function φ both - the geometry and the physics of the problem of interest - contribute to the evolution of the interface The level set equation (2) or (3) is the global Eulerian representation of the interface evolution the interface is captured by the level set function φ as opposed to being tracked by interface elements as done by the local Lagrangian formulation (1). Level Sets Methods in Imaging Science p.17/36

21 Example: Motion by mean curvature The level set equation describing mean curvature flow is: (4) φ τ = bκ φ, with b a positive scalar. When b > 0 circles in 2D shrink to a single point and disappear. When b < 0 the problem is ill-posed: instabilities develop as the circles in 2D grow instead of shrink. Shrinking sphere (right) and breaking dumbbell (left) (J.M. Fried, 1995) Level Sets Methods in Imaging Science p.18/36

22 Example: Motion in the normal direction The level set equation is: where a is a real valued scalar. φ τ + a φ = 0 If φ is initially a signed distance function, it stays a signed distance at all times This is not true in general for arbitrary velocity fields! An interface propagating at constant speed can form corners as it evolves. At corner points the interface (level set function) is not differentiable and a weak solution must be constructed. The correct weak solution, the entropy solution, comes from Sethian s entropy condition: Once a corner has developed, the solution is no longer reversible; some information about the solution is forever lost. No point-wise correspondence! Level Sets Methods in Imaging Science p.19/36

23 Application to hydrocephalus:(drapaca et al., 2005) Horizontal section of a normal (left) and hydrocephalic (right) brain. Note the large ventricles and severely compressed brain tissue in the hydrocephalus condition. Level Sets Methods in Imaging Science p.19/36

24 Application to hydrocephalus:(drapaca et al., 2005) Horizontal section of a hydrocephalic brain before (left) and 3 months after (right) shunt implantation Level Sets Methods in Imaging Science p.19/36

25 Application to hydrocephalus:(drapaca et al., 2005) Horizontal section of a hydrocephalic brain before (left) and 3 months after (right) shunt implantation The pre-shunted ventricular CSF-tissue boundary (left), evolution in time of the ventricular wall for a = 1 (centre) and one of the evolved curve (right) Level Sets Methods in Imaging Science p.19/36

26 Application to hydrocephalus:(drapaca et al., 2005) Horizontal section of a hydrocephalic brain before (left) and 3 months after (right) shunt implantation A 3D ventricular surface shrinking with a = 1 (original 3D surface (blue), evolved 3D surface (purple)) Level Sets Methods in Imaging Science p.19/36

27 Example: Propagation of a cosine curve An interface propagating at a speed F = 1 ǫκ, ǫ > 0 stays smooth during the evolution process. As ǫ 0, this solution approaches the entropy solution obtained for the constant speed case. The constant speed acts as an advection term, while the curvature dependent term has a diffusive, regularizing effect on the interface. Viscosity solution for F = κ (left) and the entropy solution for F = 1 (right) Level Sets Methods in Imaging Science p.20/36

28 Advantages of the level set method: No parameterization. Automatic handling of topology changes. Easy computation of geometric properties. Mathematical proofs and numerical stability. Easy to implement numerical schems. Challenges of the level set method: Computationally expensive Narrow band algorithm (Adalsteinsson & Sethian, 1995) PDE-based fast local level set method (Peng & Merriman & Osher et al., 1999) GPU implementation (Lefohn et al., 2004) Fixed uniform resolution Octree-based level sets (Losasso & Fedkiw & Osher, 2006) Level Sets Methods in Imaging Science p.21/36

29 Challenges of the level set method: Need a periodic reinitialization Extension velocities (Adalsteinsson & Sethian, 1999) Signed distance conservation model (Gomes & Faugeras, 2004) Need a mesh extraction step Marching cubes algorithm (Lorensen & Cline, 1987) Numerical diffusion Particle level set method (Enright, Fedkiw et al., 2002; Osher & Fedkiw book, ch.9) Limited to codimension 1 Local level set method for any codimension (Min, 2004; Osher & Fedkiw book, ch.10) Limited to closed surfaces model open surfaces using two level set functions (Liao, 2003; Li et al., 2006) Level Sets Methods in Imaging Science p.22/36

30 Challenges of the level set method: Cannot track a region of interest on the surface Producing a suitable model for the speed function F (normal component of the velocity). F may depend on the geometry and the physics of the problem. The information of the tangential component of the velocity is not used Cannot handle interfacial data No point-wise correspondence No control on topology Pons et al., 2004, 2006 report progress on level sets with tangential velocities. Work on level set methods with topological constraints was done by Han et al., 2003; Alexandrov, Santosa, 2004; Pons et al., Level Sets Methods in Imaging Science p.23/36

31 Reinitialization equations: As the interface evolves according to, for example, (3), φ will drift away from its initialized value as signed distance and sometimes it may lead to unbounded values of φ. If φ is not a signed distance function at a time t, then its zero isocontour will not be the evolved interface at t. Reinitializing φ occasionally to be a signed distance function will also ensure that φ stays smooth enough such that its spatial derivatives are computable. The numerical scheme will stay stable (Merriman, Bence & Osher, 1994) Recall that φ is a signed distance function if φ = 1. Level Sets Methods in Imaging Science p.24/36

32 I. Reinitialization using the extension velocities model Peng et al., 1999 (following Rouy & Tourin, 1992) embedded the constraint φ = 1 into a dynamic scheme and solve the equations (5) φ + φ = 1 in Ω+ τ φ φ = 1 in Ω τ until the steday state ( φ τ = 0) is reached. Reinitialization equation: combination of the two equations in (5) of the form: (6) φ τ + S(φ) ( φ 1) = 0, S(φ) = φ φ2 + φ 2 ( x) 2. S(φ) has smoothing effects on the numerical solution φ; x is the step in the x direction of the numerical grid. Level Sets Methods in Imaging Science p.25/36

33 II. Reinitialization using the fast marching method The signed distance is the solution of the Eikonal equation: (7) φ = 1 Use the fast marching method to solve (7). Fast marching method (FMM): designed for problems in which the front is always moving forward or backward (the speed does not change its sign). the front crosses each forward (backward) grid point only once. it is a very fast method (tree algorithms). Level Sets Methods in Imaging Science p.26/36

34 II. Reinitialization using the fast marching method FMM uses: upwind difference operators to approximate the gradient, and the Dijkstra idea of a one-pass algorithm for computing the shortest path on a network. Dijkstra s method for a network in which there is cost assigned to entering each node: Put the starting point in a set called Accepted. Call the grid points which are one link away from the Start Neighbors. Compute the cost of reaching each of these Neighbors. The smallest of these Neighbors must have the correct cost. Remove it, and call it Accepted. Add any new Neighbors to this point that are not already Accepted. Find the cost of reaching all Neighbors. Repeat the previous step until all points are Accepted. Level Sets Methods in Imaging Science p.26/36

35 II. Reinitialization using the fast marching method Find the shortest path from Start to Finish in the given network (left), and shortest path shown in red (right). Level Sets Methods in Imaging Science p.26/36

36 II. Reinitialization using the fast marching method To solve (7), FMM can be run separately for grid points outside and inside the interface. Basic idea: builds the signed distance function φ using only upwind values starting with the smallest value of φ (first arrivals). Algorithm for the fast marching method Tag the points on the interface as Known. Tag the points that are one grid point away from the interface as Trial. Tag all the other points as Far. Begin loop: If A is a Trial point with the smallest φ value, add it to Known and remove it from Trial. Tag as Trial all the neighbours of A that are not Known. If the neighbour is in Far remove it from Far and add it to the set Trial. Recompute the values of φ at all the Trial neighbours of A according to equation (7). Return to the top of the loop. Level Sets Methods in Imaging Science p.26/36

37 Algorithm for the fast marching method Update procedure for the fast marching method Level Sets Methods in Imaging Science p.26/36

38 III. The signed distance conservation model Gomes & Faugeras, 2004 changed the level set equation (3) in such a way that at each time instant φ is the signed distance function. The idea is to introduce a new function B such that B and φ satisfy: B( x) = F( x), for φ( x) = 0; φ τ = B; φ = 1 Differentiating the above last two equations: Since φ τ = ( ) φ τ = B, φ φ φ τ ( ) φ, we get: φ B = 0, τ i.e. B is constant along the characteristics of φ. = 0 Level Sets Methods in Imaging Science p.27/36

39 III. The signed distance conservation model Arnold, 1983 proved that the characteristics of distance functions are straight lines of (nonunique) equation f(λ) = x λ φ( x), x R n. The point y = x φ( x) φ( x) is the closest point to x on the zero level set of φ (φ( y) = 0) also located on the characteristic of φ through x. Since B is constant on the characteristics of φ and B = F on φ = 0, it follows that B( x) = B( y) = F( x φ( x) φ( x)). The new level set equation that conserves the signed distance function during the evolution process is: (8) φ τ + F( x φ( x) φ( x)) = 0. By using equation (8) instead of the classic level set equation (3), no reinitialization is needed. Level Sets Methods in Imaging Science p.28/36

40 Level sets with a point correspondence (LSPC) Level set methods convey a purely geometric description the point-wise correspondence is lost cannot handle interfacial data restricts the range of possible applications Tangential velocities have no effect on the shape (geometry) of the level set funciton φ, but it affects point correspondences and the evolution of interfacial data (contains information about the physics of the problem) Enright test: a circle is entrained by vortices and stretched out very thin before the flow time reverses returning the circle to its original form. Level Sets Methods in Imaging Science p.29/36

41 Level sets with a point correspondence (LSPC) Basic idea: advect the point coordinates with the same speed as the level set function Introduce a correspondence function ψ pointing to the initial interface φ and ψ are the steady-state solutions of: φ + v φ τ = 0 ψ τ + J ψ v = 0 with φ( x, 0) and ψ( x, 0) given. J ψ is the Jacobian of ψ. If f 0 is a function of interfacial data (related to the physics of the problem) then the evolution of f = f 0 is given by: ( ) f ψ τ + v f = ( f 0) τ + J ψ v = 0. Level Sets Methods in Imaging Science p.30/36

42 Level sets with a point correspondence (LSPC): Examples (Pons et al., 2004, 2006) A rotating and shrinking circle: initial (left) and final (centre) interface, point correspondence (right). Level Sets Methods in Imaging Science p.31/36

43 Level sets with a point correspondence (LSPC): Examples 2D evolutions with (bottom) and without (top) an area preserving tangential velocity. Expanding (left column) and shrinking (right column) square. Level Sets Methods in Imaging Science p.32/36

44 Level sets with a point correspondence (LSPC): Examples 3D unfolding of a cortex with a tumor (left) with (right) and without (centre) an area preserving tangential velocity. Cortex unfolding to a simplified geometry allows for easier visualization and analysis of functional and structural properties of the cortex. Level Sets Methods in Imaging Science p.32/36

45 Remarks on the type of PDEs Hamilton-Jacobi equation: a hyperbolic PDE of the form (9) φ τ + H( x, τ, φ) = 0 The level set equation (2): φ τ + V φ = 0 is of form (9) with H( x, τ, φ) = V ( x, τ) φ. The level set equation (3): φ τ + F φ = 0 is of form (9) with H( x, τ, φ) = F φ when F depends only on x, τ and/or φ. Level Sets Methods in Imaging Science p.33/36

46 Remarks on the type of PDEs Gomes & Faugeras equation (8): φ τ + F( x φ φ) = 0 is not a Hamilton-Jacobi equation since F depends on φ. The equation of the motion by mean curvature (4): ( ) φ φ τ = b φ φ contains second order spatial derivatives of φ and thus is not a Hamilton-Jacobi equation. This is a parabolic equation. Level Sets Methods in Imaging Science p.34/36

47 General level set algorithm Initialize/reinitialize the level set function φ at τ = τ n. Construct/approximate V φ or F φ. Evolve φ using equation (2) or (3) for τ = τ n + τ. Example of Matlab code function signeddistance = ellipse(x,y,x0,y0,xradius,yradius) dist2=(x-x0). 2./xradius 2 +(y-y0). 2./yradius 2 ; if (1-dist2 >= 0) signeddistance=(1-dist2). (1/2) ; else signeddistance=-(dist2-1). (1/2) ; end; Level Sets Methods in Imaging Science p.35/36

48 Example of Matlab code function d = lsm-normaldir-2d(data, a, T, deltax, deltay, deltat) Ny=size(data,1); Nx=size(data,2); syminus=cat(1,data(1,:),data(1:ny-1,:)); syplus=cat(1,data(2:ny,:),data(ny,:)); sxminus=cat(2,data(:,1),data(:,1:nx-1)); sxplus=cat(2,data(:,2:nx),data(:,nx)); Ixminus=(sxminus-data)./deltax; Ixplus=(sxplus-data)./deltax; Iyminus=(syminus-data)./deltay; Iyplus=(syplus-data)./deltay; Ixpm=(sxplus-sxminus)./(2 deltax); Iypm=(syplus-syminus)./(2 deltay); mag=(ixpm. 2 +Iypm. 2 ). (1/2) ; data=data-a. deltat. mag; d=data; Level Sets Methods in Imaging Science p.36/36

49 Example of Matlab code M-file test-lsm-normaldir-2d m=zeros(64,64); for i=1:64 for j=1:64 m(i,j)=ellipse(i,j,32,32,30,30); end; end; a=2; dx=1; dy=1; dt=1;t=10; contour(m,[0,0], b ); hold on; [Nx,Ny]=size(m); mevolved=zeros(nx,ny,t+1); for i=1:t mevolved(:,:,i+1)=lsm-normaldir-2d(mevolved(:,:,i),a,t,dx,dy,dt); contour(mevolved(:,:,i+1),[0,0], g ); hold on; end; contour(mevolved(:,:,t+1),[0,0], r ); Level Sets Methods in Imaging Science p.36/36

50 Example of Matlab code Evolution of a initial ellipse (blue) with constant speed for 10 time steps. The last contour is red. Level Sets Methods in Imaging Science p.36/36