Deformable Contour Method: A Constrained Optimization Approach

Transcription

1 Deformable Contour Method: A Constrained Optimization Approach Abstract. In this paper, a class of deformable contour methods using a constrained optimization approach of minimizing a contour energy function satisfying an interior homogeneity constraint is proposed. The class is defined by any positive potential function describing the contour interior characterization. An evolutionary strategy is used to derive the algorithm. A similarity threshold T v can be used to determine the interior size and shape of the contour. Sensitivity and significance of T v and σ (a spreadness measure) are also discussed and shown. Experiments on noisy images and the convergence to a minimum energy gap contour are included. The developed method has been applied to a variety of medical images from CT abdominal section, MRI image slices of brain, brain tumor, a pig heart ultrasound image sequence to visual blood cell images. As the results show, the algorithm can be adapted to a broad range of medical images containing objects with vague, complex and/or irregular shape boundary, inhomogeneous and noisy interior, and contour with small gaps. 1 Introduction Image segmentation is a fundamental issue in computer vision. As one of the most successful image segmentation techniques, deformable contour methods have received a tremendous amount of attention in a variety of areas, including robot vision, pattern recognition, and biomedical image processing. Typically, deformable contour methods (Kass, Witkin, and Terzopoulos 1988), (Cohen and Cohen 1993), (Xu and Prince 1998) model image segmentation as a contour energy minimization problem, which is dependent only on image gradient (boundary-based). A traditional difficulty with these approaches is that, due to the presence of other objects in an image and noise, there are numerous energy minima. Consequently, undesired contour energy minima, instead of target boundary, are often generated. As a solution, balloon force concept was brought up in (Cohen 1991), which tries to push the contour nearby the target boundary in order 1

2 to locate it. This concept is extensively applied in later literatures (Casellas, Kimmel and Sapiro 1997), (Siddiqi, Lauziere, Tannenbaum, and Zucker 1998), (McInerney and Terzopoulos 1999). But as balloon forces are functions of image gradient or constants, it can be still difficult for the balloon force to distinguish the neighborhood of target boundary from those of undesired contour energy minima, when image gradient information is either noisy or inaccurate in identifying target boundary. As an alternate solution to the problem of multiple contour energy minima, non-deterministic approaches (Lundervold and Storvik 1995), (Grzeszczuk and Levin 1997) search for a global contour energy minimum. The problem with these methods is that, in a complex image context, the target boundary is usually a local energy minimum. To make the target boundary be the global energy minimum, two conditions are often needed. Either a major modification of energy function, incorporating specific prior knowledge on target boundary (Lundervold and Storvik 1995), or a preset mask (Grzeszczuk and Levin 1997) constraining the contour searching space within a neighborhood close to target boundary is required. Region-based deformable contour method is one way to avoid the problem of boundary-based methods, by considering boundary extraction as minimizing regionbased energy functions (Chan and Vese 2001) (Samson, Blanc-Feraud, Aubert, and Zerubia 2000). That means some prior knowledge of the region and image structure, such as the number and mean brightness of the objects in the image, is often required. Such information is not readily available in many implementations. Another problem is that these methods are only based on region features and may obtain inaccurate contours due to lack of image gradient information. 2

3 In this work, we focus on integrating region and boundary information into one framework. Specifically, our strategy is to add the region information to the existing boundary-based deformable contour formulation. Since region information is more global and less susceptible to noise, it will make the approach more robust and precise. We note here that in the literature of deformable contour methods, similar ideas have been proposed in other image segmentation methods as well, although with rather different natures (Chakraborty and Duncan 1999), (Jermyn and Ishikawa 2001), (Zhu and Yuille 1996). In (Chakraborty and Duncan 1999), a model based boundary finder that integrates region and boundary information is presented. The method is robust to noise and is rather successful in many applications but it may have difficulty in handling complex shape and poor initialization. In (Jermyn and Ishikawa 2001), an original framework incorporating information from both region and boundary is proposed. However, this method still requires that the functions incorporating region information, or the integral of the region characterization function over the region, be expressed by an integral over its boundary. In (Zhu and Yuille 1996), an image segmentation method named region competition is presented, which combines the favorable features of snakes/balloons and region growing. However, the method may have difficulties dealing with over-segmentation and is not easy to be implemented for the boundary extraction of a single object. This work takes the viewpoint that region information can be introduced as extra constraints within the contour energy minimization framework. With this in mind, the contour energy minimization problem is then formulated as searching for an energy minimum contour with its interior satisfying a constraint of region features. The constraint can be any function characterizing the contour interior structure, such as 3

4 homogeneity, texture, and color. The introduction of the constraint is to limit the contour searching in a subspace of contours with desirable interior properties, whereby undesirable local energy minima are eliminated and target boundaries are much easier to be located. To solve this constrained contour energy minimization problem, a new method derived from evolution strategy is developed. The method incorporates not only boundary features (by minimizing contour energ but also a general class of modeling information from contour interior region (by imposing the constraint). As a special case, a constraint function based on region homogeneity inside the contour is proposed. The versatility and robustness of the method with this constraint is then evaluated by using challenging biomedical image applications where extracting contours with gaps, complex shapes, and interior noises are common requirements. High computational complexity due to energy minimization by evolution strategy is avoided by limiting the contour searching space within a small set of contour individuals. In many cases, the results outperform by far other conventional deformable contour methods. The problem statement and the proposed approach are introduced in Section 2. Algorithm description and evaluation are provided in Section 3 and Section 4, respectively. Medical applications and experimental comparison are discussed in Section 5 and 6. Finally, conclusions are contained in Section 7. 2 Problem Statement and an Evolution Strategy Approach: We formulate the problem as follows. We first start with boundary model. 2 Let Φ be an open domain subset of R and I ( :Φ R be the image intensity function. Assume that target boundary Γ (s) Φ is a closed contour with minimum energy, 4

5 E ( Γ ( s)) = MinE (1) where s is the parameter of Γ (s) and E (.) is an energy function, 1 1 E ( Γ( s)) = ds (2) p ds Γ( s) 1+ G * I( x( s), y( s)) Γ ( s) where p = 1 or 2, G * I is the absolute value of the gradient of I( smoothed by a Gaussian filter N(0, σ 2 0 ). It should be noted that E (.) taking the form of Eq. (2) is not strictly necessary and can be replaced by energy functions such as of (Kass, Witkin, and Terzopoulos 1988), (Casellas, Kimmel and Sapiro 1997). Let region Ω be the interior of Γ (s). To characterize this region Ω, we assume Γ(s) Γ(s) that there exists a characteristic function D( satisfying the following condition, D ( T 0 if ( Ω V Γ( s ) (3) where T V is a given constant. D( can be any arbitrary function of region features including colour and texture. From Eq. (3), a very general class of the modelling information from region can be incorporated into the contour energy minimization framework. The resulting constraint of Eq. (3) can be illustrated by Fig D( T v if ( Ω Γ(s) D( <T v if ( is outside of Γ (s) Fig. 2.1 An Illustration of T v distribution Considering all the difficulties associated with the various natures that exist in image segmentation problems, defining a general D( that can be universally applied is a 5

6 challenging task. In this paper, we bring up a D( function that is based on the region homogeneity properties. Let D ( = A( B( (4) with I ( I 0 1 A( = and B( = e σ, where I 0 is the average intensity over G*I( region Ω Γ(s). A ( represents a measurement of smoothness as a function of point gradient, and B ( is a measurement of smoothness as a potential function of deviation from an average brightness value I 0 of the interior region. Therefore, D( as given in (4) can be considered as a homogeneity measure of smoothness at an interior point (. A large value of the gradient and a large deviation of the point brightness from I 0 produce a much smaller value of D(. We are now ready to give the problem statement. Our problem is to find a closed contour C(s, t) enclosing region Ω c, such that is minimized with the constraint 1 1 E( C( s, t)) = ds (5) p ds C 1+ G * I( x( s), y( s)) C I( I0 σ 1 D( = e T (6) 2 V 1+ G * I for all ( Ω c. Our solution to the constrained optimization problem is to use evolution strategy to deform C(s, t) until an optimum C(s) is reached. The evolution strategy is to mimic the process of evolution, the deriving process for emergence of complex and well-adapted organic structure (Buck, Hammel, and Schwefel 1997). It is a recursive process in which 6

7 a population of individuals, the parents, mutates and recombines to generate a large population of offspring. These offspring are then evaluated according to a fitness function and as a result, a best subset of offspring is selected to replace the existing parents. There are three main factors to consider: 1. The representation of individual contours Individual contours are represented by states. 2. Variation of states There are two types of state variation scheme: mutation and recombination. The recombination of contours requires a much higher computational complexity and is not being used in our application. C(s, t) varies by satisfying the constraint of Eq. (2), and mutates (Buck, Hammel, and Schwefel 1997) by adding a Gaussian σ p N(0, 1) perturbation, where the notation N(b 1, b 2 ) indicates the Gaussian random variable with mean b 1 and standard deviation b 2. Notice that maximizing F = ( D( Tv) dxdy is equivalent to satisfying Eq. (6). With the analysis shown in Ω C the Appendi the corresponding curve evolution formula, C( s, t) r = ( D( T ) N (7). v t The contour driven by Eq. (7) grows outwards when D( >T v, and stops at points when D( = T v. To incorporate the mutation process, we have C( s, t) t r = [ D( x( s, t), y( s, t)) T σ ( s, t) N(01, )]N. v p Let σ ( s, t) = τd( with 0 τ 1, p C ( s, t) r = [ D( (1 τ N(0,1)) T ]N V t 7

8 To reduce the point brightness noise contamination, in Eq. (6), I( is replaced with a local average of size 3 by 3 denoted as M(. To further improve the efficiency of the mutation process is to let τ be a function depending on the local image brightness distribution as Z1 τ = Z e (8) 1 with Z 1 I ( M ( = and σ = 1 2σ. σ 1 The goal is to generate a velocity perturbation only when needed. For a small Z 1 indicating a rather smooth neighborhood, Eq. (8) produces a small τ to cause a small perturbation. A large Z 1 indicating a rough neighborhood also causes a small perturbation. Only when Z 1 is between these two extremes, the random perturbation is assigned a large value. The final evolution formula is C( s, t) t M ( I0 1 r σ = e (1 τn (0,1)) Tv N (9) G* I 3. Selection scheme -- We apply a modified ( µ, λ) selection scheme (Buck, Hammel, and Schwefel 1997), where N µ = and λ = N. The notation ( µ, λ) indicates µ parents 2 create λ > µ offspring by means of mutation, and the best µ offspring individuals are deterministically selected to replace the parents. However, the acceptance of temporary deterioration might also make ( µ, λ) selection drift away from the contour energy minimum. To avoid this, we select the state with lowest energy from µ survivors in ( µ, λ) selection and compare it to the state of lowest energy selected from previous selections. We save the state with lowest energy among all the selection processes as the output contour when the algorithm is over. 8

9 The described evolutionary method involves a contour evolution (Malladi, Sethian and Vemuri 1995) formulation that maximizes an interior F function under perturbations so that the minimum E contour can be evaluated among the generated admissible contour set. The algorithm terminates under the following two conditions: 1. The maximum F is reached in less iterations than a predefined maximum iteration number N T and resulting minimum E contour is recorded as the solution. 2. The total number of iterations N T has been reached and the resulting minimum E contour is recorded as the solution. The resulting effect of the contour point velocity perturbation is equivalent to an adjustable T V. Lowering T V to zero makes N T as the only stopping control. Therefore, the contour continues to expand with D(.)>0 for all image pixels and many local minimum E solutions are found in the process. To achieve the minimum E contour for a given T V setting, the original level set (Malladi, Sethian and Vemuri 1995) is needed which requires a much longer computation time while approximated solutions can be obtained in much less computation times using the fast marching approach (Sethian 1996). An approximated solution is reached when the maximum contour point velocity (the maximum F is considered reached) is less than a preset velocity. A smaller T V value produces a larger interior area that requires a larger iteration number in order to reach a boundary contour. 3 Algorithm Description and Parameter Setting Experiments Like most deformable contour methods, the proposed method is a semi-automatic image segmentation method. However, as our discussion proceeds, we can see that the performance of the method is very robust to the parameter settings thus it has a good 9

10 potential to be applied in many automatic image segmentation applications. In the following, we will illustrate the experiments that evaluate the effect of different settings of T v, and σ of D( on the final results. Notice that T v is a similarity threshold value set to isolate the interior as shown in Fig. 2.1, and σ the measure of the spreadness in D(. As we will show later, for a given value of T v, σ is proportional to the contour interior intensity standard deviation. Experiments are designed to show the relationship among A(, B(, D( in Eq. (4) and the resulting contours for different settings of T v and σ. The steps of the proposed algorithm can be summarized as follows: 1. Input the Gaussian filtered image I(, an initial interior location (x 0, y 0 ), and parameters: σ (spreadness in D( ), V TH (stopping velocity threshold), N C (number of candidate contours), N T (total number of iterations) and T V. 2. Generate a contour around the 5 by 5 region centered at (x 0, y 0 ); duplicate another (N c - ) C ( q,0) (,0) 1) copies; Fix I = I, where ˆC q I is the average intensity inside the initial contour 0 0 C(q,0) and set the number of iterations to zero. 3. For each contour from the N C contours performs the following: 0 i). Solve Eq. (9) using fast marching or level set method for 10 iteration (, ) ii). Let ˆC q t (, ) I = I and fix it, where ˆC q t I is the average intensity inside the current contour C(q, t). Stop the whole process and record the minimum E contour as the solution, when the maximum point velocity is less than V TH, otherwise continue; 4. Compute E of Eq. (5) and select N C /2 contours of the smallest E from the N C contours; record the minimum E contour as a potential solution and its corresponding iteration number. 10

11 5. Duplicate each contour of the selected N C /2 contours once to constitute a set of new N C contours; Stop the whole process when the number of iterations is larger than N T, otherwise, go to Steps 3. The recorded minimum E contour is the solution contour. Fig. 3.1a shows the cross sectional brightness profile of Fig. 3.1e image of a circular brightness platform of size 256 by 256. The platform top can be characterized as a smooth brightness surface at 180 gray scales. Cross sectional profiles of A(, B( with σ=3, and D( of Fig. 3.1e are shown in Fig. 3.1b, 3.1c and 3.1d, respectively. Each represents a different measure of smoothness with D( ) = A( )B( ). The degree of B( drop-off is controlled by both σ and I 0 values. With the T v settings of 0.9, 0.1 and 10-5, the respective resulting circular contours are shown in Fig. 3.1f, 3.1g and 3.1h. The resulting contour diameters in Fig. 3.1f, 3.1g and 3.1h are approximately 40, 72 and 124 pixels, respectively as illustrated in Fig. 3.1d. These results show that the lower the similarity threshold value, the larger is the circular contour. In other words, T v controls the interior size and location of the resulting contour. For the Fig. 3.2a image of an MRI front view brain image size 256 by 256, A(, B( with σ =20, and D( are shown in Fig. 3.2b, 3.2c and 3.2d all in logarithmic base 10 scale, respectively. The dynamic ranges of A, B and D are between 0.63 and 1, 5.16x10-4 and 1, and 6.5x10-5 and 1, respectively. It is obvious that B has a greater influence on D at region near the final contour. The interior brightness average I 0 ranges from to With these images reflecting the pixel similarity measure, one can roughly visualize the resulting contour by looking at these images. As one can see from the resulting contours of Fig. 3.2e to 3.2i for a decreasing T v value as shown in Table 1, 11

12 lowering the T v setting has the same effect as increasing the iteration number or the size of the contour interior. A much lower T v like forces the algorithm to march passing a visible but lighter contour region to locate the minimum E contour as shown in Fig. 3.2i. For each T v setting, there is only a single minimum E contour. To test the sensitivity of the algorithm to the settings of σ, we set T v = 0.01 and conduct the experiments by increasing the value of σ as shown from Fig. 3.3a to 3.3f. One can also observe from the resulting contours of Fig. 3.3a to Fig. 3.3f for an increasing σ value shown in Table 2, increasing σ has the effects of increasing standard deviation inside the contour and the size of the contour interior. A small σ results a more homogenous contour interior as shown in Fig. 3.3a while a large σ indicates just the opposite. It is also rather clear that the algorithm is not sensitive to the settings of σ and when σ is in a range from 30 to 100, the resulting contours vary insignificantly. To further illustrate the effect of A( of Eq. (3), we set A( = 1, T v = 0.01 and conduct the experiments. The final contours with global energy minimum are shown in Fig. 3.4a and 3.4b with σ=10 and 20, respectively. As illustrated in Table 2, it can be seen that as A( is set as 1, the resulting contours are more tended to march over edge points and the energies of the resulting contours are much higher. Fig. 3.1a A cross-sectional brightness profile of Fig. 3.1e Fig. 3.1b A cross-sectional A( values of Fig. 3.1e 12

13 Fig. 3.1c A cross-sectional B( values of Fig. 3.1e Fig. 3.1d A cross-sectional D( values of Fig. 3.1e Fig. 3.1e Image of the circular platform profile of Fig. 3.1a Fig. 3.1f Resulting contour for T v =0.9 Fig. 3.1g Resulting contour for T v =0.1 Fig. 3.1h Resulting contour for T v =

14 Table 1. Algorithm performance for different T v T v MinE * Iteration Number or Size* Contour Figure Number e f g h i E & I are approximate values since we only compute them after every 10 iterations. Fig. 3.2a Original Image Fig. 3.2b Scaled image of log 10 A( of Fig.3.2a Fig. 3.2c Scaled image of log 10 B( of Fig. 3.2a with σ=10, I 0 =131 Fig. 3.2d Scaled image of log 10 D( of Fig. 3.2a Fig. 3.2e Resulting contour of T v =0.1 Fig. 3.2f Resulting contour of T v =

15 Fig. 3.2g Resulting contour of T v =0.01 Fig. 3.2h Resulting contour of T v =0.006 Fig. 3.2i Resulting contour of T v =3x10-4 Fig. 3.3a Resulting contour of σ = 10 Fig. 3.3b Resulting contour of σ = 20 Fig. 3.3c Resulting contour of σ = 30 Fig. 3.3d Resulting contour of σ = 60 Fig. 3.3e Resulting contour of σ = 70 15

16 Fig. 3.3f Resulting contour of σ = 100 σ Table 2. Algorithm performance for different σ Standard Deviations MinE * Iteration Number or Size* Contour Figure Number a b c d e f a b E & I are approximate values since we only compute them after every 10 iterations. Fig. 3.4a Resulting contour of 10 = σ and A( = 1 Fig. 3.4b Resulting contour of σ = 20 and A( = 1 4 Algorithm Evaluations There are two evaluations being conducted in this section. First, the performance of the algorithm in noisy image is evaluated, and second, the convergence to a suitable minimum E gap contour is studied. 16

17 For noisy image evaluations, Gaussian and salt-and-pepper noise of different levels are added in Fig We select real biomedical objects with clear contours like left and right kidneys, and contour with a lower contrast like the spleen, for our study. Increasing levels of additive noise are then introduced to evaluate the performance of the method until it fails to provide a visually acceptable contour. Signal-to-noise ratio (SNR) is used to measure the noise level, which is defined in decibels (db). For an m by n image, u( is the pixel brightness of the original image, v( is the pixel brightness of the contaminated image. SNR m x n x y = 1 y= 1 = 10 log10 m n = 1 = 1 u( u mean u( v( where u mean is the spatial mean brightness of u(. The numerator is the variance of the original image. Fig. 4.1a is the original 512 by 512 CT abdominal image with 256 gray level. Fig. 4.1b is the segmentation result. For the additive Gaussian noise, the above two kidney boundaries can be extracted as shown in Fig. 4.3b from Fig. 4.3a with variance of 10,000 (SNR = -7.37dB) and the spleen can be extracted as shown in Fig. 4.2b from Fig. 4.2a with variance 3000 (SNR = -2.72dB). 2 2 Fig. 4.1a Original abdominal image Fig. 4.1b Segmentation results of Fig. 4.1a 17

18 For salt-and-pepper noise, the two kidneys boundaries are extracted as shown in Fig. 4.5b from Fig. 4.5a with 30% noise (SNR = dB), and the spleen boundary is extracted as shown in Fig. 4.4b with 10% noise (SNR = -7.95dB). In all 4 cases, further additional noise would produce images difficult for the human eyes to distinguish. Fig. 4.2a Gaussian noise σ 2 = 3000 and SNR = -2.72dB Fig. 4.2b Segmentation results of Fig. 4.2a Fig. 4.3a Gaussian noise 2 σ = 10000, SNR = 7.37dB Fig. 4.3b Segmentation results of Fig. 4.3a Fig. 4.4a salt-and-pepper noise (10%), SNR = -7.95dB Fig. 4.4b Segmentation results of Fig. 4.4a 18

19 We also evaluate the effectiveness of energy minimization in our evolution strategy in locating a gap contour. Fig. 4.6a shows a cell image under the microscope. The cells in the image are irregular and there are overlapping cell boundaries. Without remembering the global minimum E, the evolving curve would race out of the gap nonstop. With our method, we are able to extract the cell boundary as shown in Fig. 4.6b. Fig. 4.7 shows the energy fluctuation of one of the competing contours at iteration steps in attaining the solution shown in Fig. 4.6b. The horizontal axis is the iteration number, and the vertical axis is the lowest energy in the 16 competing contours at each competition step with σ=100. The energy are compared every 10 iterations for each contour. By observing the energy evolution process, we can see that the energy fluctuates at the beginning, then decreases gradually. After it reaches the historical minimum of at iteration 1130 (see Fig. 4.7), the energy increases indicating that the contour runs out of the gap. By preserving the contour with the lowest energy shown in Fig. 4.8, we obtain the desired cell boundary shown in Fig. 4.6b. Fig. 4.5a salt-and-pepper noise (30%), SNR = dB Fig. 4.5b Segmentation result of Fig. 4.5a 19

20 Fig. 4.6a Original cell image Fig. 4.6b The extracted boundary of one cell Fig. 4.7 Energy Fluctuation of a competing contour Fig. 4.6a Fig. 4.8 Lowest energy for cell boundary extraction 5 Illustrative Examples The developed algorithm has been applied to a variety of medical images from a CT abdominal section, MRI image slices of brain, brain tumor, a pig heart ultrasound image sequence to visual blood cell images. The main purpose of these applications is to evaluate and demonstrate both the applicability and versatility of the algorithm. The images vary in image types, contour types, and have different degrees of difficulties in segmentation. In all of these examples, N C is set between 2 and 4 copies, σ=10, Tv=0 with V TH =0.05, and N T =20,000. Any changes from these settings are indicated in specific illustrations. Notice that a large N C is to provide additional choices under more trying conditions and a large σ is to minimize the effect of B(. The setting of Tv=0 with 20

21 V TH =0.05 is practically approximate to the setting of Tv=0.05. With a rather large N T, each experiment stops when V TH value is reached, and the minimum E contour is recorded as the solution contour. The first set of data is a slice of abdominal CT images of size 512 by 512, with 256 gray level as shown in Fig. 5.1a. With this image, the goal is to find contours of liver, stomach, spleen, two kidneys, two blood vessels, an intestinal section, and the spinal cord. The resulting contours are shown in Fig. 5.1b. Each starts with a 5 by 5 initial contour arbitrarily placed inside the desired region. This image illustrates a varying degree of difficulties in the automatic contour extraction. The kidneys, spleen, stomach with food, blood vessels and intestinal section have rather clear and distinct boundaries and homogenous interiors. These objects are among the easiest contours to be extracted and the algorithm has no difficulties with them. The spinal cord has a clear boundary with sharp corners and inhomogeneous interior. The algorithm overcomes these above difficulties by increasing the value of σ to σ =30. The most challenging task here is the liver. It is a much larger contour as compared to others with scattering small abbreviations inside the liver. It has a smooth and distinguishable boundary that is very close to the left kidney, and has a tiny crossover to the stomach at the upper right portion. By increasing the number of candidate contours N C to 8, the liver is successfully extracted. Except spine, in all of the above contour extraction, the selection of σ is not crucial since boundaries are rather clear, and interiors are rather smooth (σ can be set from 10 to 30 in all these applications). For the brain coronal MRI T1 images, there are 188 slices with size 256 by 256 cross sectional images of 256 gray levels. The physical resolution is 1.016mm by

22 mm with a 1mm thickness. Brain images have a complex shape. Our goal is to extract intracranial contour, sulci, and the boundary between white and gray matter. Here, two slices are selected for evaluation: 1) Fig. 5.2a for the segmentation of the intracranial boundary, sulci, and gray matter/white matter boundary, 2) Fig. 5.3a for the segmentation of cerebrum and cerebellum. For the segmentation of intracranial boundary, a high σ is applied (we choose σ =200 in this application) and the result is shown in Fig. 5.2b. The sulci problem is among our hardest problem due to its highly curvilinear structure of rather thin boundary with inhomogeneous brightness interior. In this case, we decrease σ to σ =20 and increase the number of candidate contour to N C = 6. The result is shown in Fig. 5.2c. For the gray matter/white matter contour isolation and identification, the value of σ is further decreased (σ = 10). The result is shown in Fig. 5.2d. There are some difficulties of different gray level differentials between gray and white matter at different parts of the brain. In terms of absolute brightness, there are gray matter brightness levels at one region of a brain that is close to the white matter brightness levels at another region of the brain. Here, the evaluation of the initialization robustness is rather important due to the complexity of the brain contour. As shown in Fig. 5.4a, 5.4b, 5.5a, 5.5b, with rather different initial contours and the above parameter setting, the resulting contours are essentially similar. For Fig. 5.3a, a smaller σ (σ =10) as compared to sulci isolation is used to find the contour of the cerebrum. Once the cerebrum contour is identified and removed, the cerebellum contour extraction process (σ can be set as 30 to 100) can then be processed. The results are shown in Fig. 5.3b and Fig. 5.3c, respectively. The removal of the 22

23 obtained cerebrum contour is helpful for accurately locating cerebellum contour because of the elimination of crossover possibilities, which is a difficulty in this method. A test of the algorithm has also been conducted on an axial MRI brain tumor data set consisting of 74 slices as computed in (Zhu and Yan 1997). The pixel size is mm by mm with slice thickness 0.8 mm. In this evaluation, the same testing sequence discussed in (Zhu and Yan 1997) is used. In (Zhu and Yan 1997), slices intersect the tumor. The contour extraction process starts with slice 39 and progress on both sides to 21 and 56. In the proposed algorithm, the same procedures are also executed. The initial contour is around a square of 30 pixel size, which contains the tumor with an inward deformation. Two slices, Fig. 5.6a, Fig. 5.6b (slice 37, 49, same as Fig.2 (c) (f) in (Zhu and Yan 1997)), are shown here with the zoomed image shown in Fig. 5.7a and Fig. 5.7b, respectively. Zoomed results with only a portion of the image are shown in Fig. 5.8a and Fig. 5.8b. Comparing with Fig. 11 (c) and (f) in (Zhu and Yan 1997), finer and more accurate close contours are obtained using the proposed method. The algorithm has also been tested on an ultrasound pig heart image sequence with 9 images. Two ultrasound images of size 640 by 480, 256 gray level with various degrees of gaps as selected to be shown in Fig. 5.9a and Fig. 5.9b are used. Note also that ultrasound images have a higher noise level as compared to CT and MRI images. Contour extraction results are shown in Fig. 5.10a and Fig. 5.10b. It is realized that the initial contour maybe arbitrary but has to be farther away from gaps for a successful contour extraction. The algorithm is also tested on two blood cell images obtained under microscope shown in Fig. 5.11a and Fig. 5.12a. Please be noted that the cells have both overlapping 23

24 irregular boundaries. To obtain a satisfactory result, the algorithm needs a larger number of candidate contours (from 16 to 32), a smaller N T = 3000, and a rather large σ (σ =100). Results are shown in Figure 5.11b and 5.12b. A word of caution is that all the above discussions are based on visual observation of individuals who are not a professional image reader of these biomedical images. Fig. 5.1a Abdominal CT image, size 512*512 Fig. 5.1b Abdominal CT image segmentation result Fig. 5.2a A slice of brain MRI image Fig. 5.2b Intracranial contour Fig. 5.2c Sulci contour Fig. 5.2d Gray/white matter contour 24

25 Fig. 5.3a Another slice of brain MRI image Fig. 5.3b Cerebrum Fig. 5.3c Cerebrum and cerebellum Fig. 5.4a Initial contour One Fig. 5.4b Resulting contour with the initial contour One Fig. 5.5a Initial contour Two Fig. 5.5b Resulting contour with the initial contour Two 25

26 Fig. 5.6a Slice 37 of axial brain tumor image Fig. 5.6b Slice 49 of axial brain tumor image Fig. 5.7a Zoomed Image of Fig. 5.6a Fig. 5.7b Zoomed Image of Fig. 5.6b Fig. 5.8a Segmentation result of Fig. 5.7a Fig. 5.8b Segmentation result of Fig. 5.7b Fig. 5.9a A slice of ultrasound pig heart image, size 480*640 26

27 Fig. 5.9b Another slice of ultrasound pig heart image, size 480*640 Fig. 5.10a Segmentation result of Fig. 5.9a Fig. 5.10b Segmentation result of Fig. 5.9b 27

28 Fig. 5.11a Original cell image one Fig. 5.11b Segmentation result of Fig. 5.11a Fig. 5.12a Original cell image two Fig. 5.12b Segmentation result of Fig. 5.12a 6 Experimental Comparisons In this paper, we only show a simple comparison with three other methods to provide an indication of the efficiency of our approach. The first two methods are geodesic snakes (Casellas, Kimmel and Sapiro 1997), and area & length active contours (Siddiqi, Lauziere, Tannenbaum, and Zucker 1998), using 1 g( = as the edge detection 2 1+ G * I function. The third method is T-snake (McInerney and Terzopoulos 1999). We select two images of Fig. 6.1a an MRI brain image and Fig. 6.2a a CT kidney image from Fig. 4.2a. Similar to all other three methods, a Gaussian filter N(0, 1) is applied to both images as a 28

29 preprocessing operation. To provide an objective comparison, a set of seven white dots in Fig. 6.1a and a set of three dark dots in Fig. 6.2a are used as initial candidate locations for all four methods including ours. Each method using an initial candidate location provides a resulting contour. The best contour (best of seven resulting contours for Fig. 6.1a, and best of three resulting contours for Fig. 6.2a) of each method from all initial candidate locations is selected for comparison. These best results are shown in Fig. 6.1b to Fig. 6.1e for Fig. 6.1a and Fig. 6.2b to Fig. 6.2e for Fig. 6.2a. Comparing these resulting contours, our proposed method has the best contours 1. Fig. 6.1a Brain image Fig. 6.1b Topology snake Fig. 6.1c Geodesic snake Fig. 6.1d Area & Length active contour 1 According to the evaluation by Professor Michael Behbehani of Molecular & Cell Physiology Dept. of University of Cincinnati, Dr. Nie and his colleagues in University of Kentucky hospital. 29

30 Fig. 6.1e The proposed method Fig. 6.2a Original image Fig. 6.2b Topology snake Fig. 6.2c Geodesic snake Fig. 6.2d Area & length active contour Fig. 6.2e The proposed method 30

31 7 Conclusions We have introduced a framework of deformable contour methods based on constrained optimization. The framework can conveniently incorporate region features such as color, and texture. By limiting the contour searching space to a subset of contours with desirable interior features, the algorithm derived from the proposed framework is more robust to noises and in many cases, performs better than other conventional deformable contour methods. In this paper, we have only used D( I( I 0 1 = e σ to access the variation of contour interiors. This, however, 2 1+ G* I does not constitute a limitation to the proposed method. In fact, our method can be easily extended to handle other measures of interior features, such as the similarity of color and texture (Zhu and Yuille 1996). It is also rather straightforward to incorporate shape information into D ( function. All these present a very desirable feature in many segmentation applications where the information from multiple sources is needed in a simple framework. As for the computational complexity, it usually takes 30 seconds to 5 minutes for our algorithm to obtain most of the segmentation results in the ultra 10 Sun workstations. So the computational complexity is rather comparable to other deformable contour methods. Acknowledgement: The authors wish to thank Dr. D. Hill of the Image Processing Group at UMDS, London, UK for providing the MRI brain tumor data, Professor Michael Behbehani of Molecular & Cell Physiology Department of University of Cincinnati, Dr. Nie and his colleagues in University of Kentucky hospital for providing the evaluations of the experimental 31

32 comparison results on MRI brain & CT abdominal images. Dr. S. K. Holland of the Imaging Research Center at Children s Hospital Research Foundation is also acknowledged for providing the MRI T1 brain data. The authors also wish to thank reviewers for their comments and valuable suggestions. Appendix I In order to maximize F = ( D( TV ) dxdy, suppose a surface Ω C d( Ψ( = d( if ( Ω otherwise C, where d ( is the distance from point ( x, to C ( s, t). Then consider a function 0 Ρ( Ψ( x, ) = 1 if if Ψ( < 0 Ψ( 0 So 0 Ρ( Ψ( x, ) = 1 if ( Ω otherwise C With this, F can be written as, Φ F = ( D( TV ) Ρ( Ψ( ) dxdy Then So where df dt dρ dψ = ( D( TV ) dxdy dψ dt Φ df dt dψ = ( D( TV ) δ ( Ψ) dxdy dt Φ ( x, Φ composes the domain of intensity image. δ (Ψ) is a Dirac function satisfying 32

33 δ ( Ψ) = 0 if Ψ 0 So df dt = ( { Ψ= 0} ( D( T V dψ ) δ (0) dxdy dt df Since limδ ( α) > 0, to make 0, let α 0 dt dψ = ( D( TV ) Ψ if ( { Ψ = 0} dt where Ψ Ψ Ψ = x,. To keep Ψ being a surface composed of level set contours y { Ψ = ±d, d R} without crossing each other, let every level set contour move with the same velocity as the zero level set of Ψ, dψ = ( D( TV ) dt Ψ= 0 Ψ In a curve evolution formula, the above equation can be written as, C( s, t) t = ( D( T V r ) N Reference: Atkins, M. and Mackiewich, B Fully automatic segmentation of the brain in MRI. IEEE Trans. on Medical Imaging, Vol. 17, No.1, pp Buck, T., Hammel, U., and Schwefel, H Evolutionary computation: comments on the history and current state. IEEE Transaction on Evolutionary Computation, Vol. 1, No. 1, pp Casellas, V., Kimmel, R., and Sapiro, G Geodesic active contours. International Journal of Computer Vision. Vol. 22, No. 1, pp

34 Chalana, V., Linker, D., Haynor D., and Kim, Y A multiple active contour model for cardiac boundary detection on echocardiographic sequences. IEEE Trans. on Medical Imaging, Vol. 15, No. 3, pp Cham, T. and Cipolla, R Automated B-spline curve representation incorporating MDL and error minimizing control point insertion strategies. IEEE Trans. on PAMI, Vol.21, No.1, pp Chan, T. and Vese, L Active Contours without Edges, IEEE Trans on Image Processing, Vol. 10, No. 2, pp Chakraborty, A. and Duncan, J.S Game-Theoretic Integration for Image Segmentation, IEEE Trans. on PAMI, Vol. 21, No. 1, pp Chiou, G. and Huang, J A neural network-based stochastic active contour model (NNS-SNAKE) for contour finding of distinct features. IEEE Trans. Image Processing, Vol. 4, No. 10, pp Cohen, L On active contour models and balloons CVGIP: Image Understanding. Vol. 52, No.2, pp , March. Cohen, L. and Cohen, I Finite element methods for active contour models and balloons for 2-D and 3-D images. IEEE Trans. on PAMI, Vol.15, pp Cohen, L. and Kimmel, R Global minimum for active contour models: a minimal path approach. International Journal of Computing Vision, Vol. 24, No.1, pp Fenster, S. and Kender, J Sectored snakes: evaluating learned-energy segmentation. ICCV, pp Fok, Y., Chan, Y. and Chin, R Automated analysis of nerve cell images using active contour models. IEEE Trans. on Medical Imaging, Vol. 15, No

35 Friedland, N. and Rosenfeld, A Compact object recognition using energy- functionbased optimization. IEEE Trans. on PAMI, Vol. 14, pp Grzeszczuk, R. and Levin, D Brownian String, Segmenting images with stochastically deformable contours. IEEE Trans. on PAMI, Vol. 19, No. 10, pp Jermyn, I. and Ishikawa, H Globally optimal regions and boundaries as minimum ratio weight cycles. IEEE Transaction on Pattern Analysis and Machine Intellegence, Vol. 23, No. 18, pp Kass, M., Witkin A., and Terzopoulos, D Snakes: active contour models. International Journal of Computer Vision, Vol. 1, No.4, pp Lundervold, A. and Storvik, G Segmentation of brain parenchyma and cerabrospinal fluid in multispectral magnetic resonance images. IEEE Trans. on Medical Imaging, Vol. 14, No.2, pp Malladi, R., Sethian, J. and Vemuri, B Shape Modeling with Front propagation. IEEE Trans on PAMI, Vol. 17, No.2, pp McInerney, T. and Terzopoulos, D Topology adaptive deformable surfaces for medical image volume segmentation. IEEE Trans on Medical Imaging, Vol. 18, No.10, pp Pien, H., Desai, M. and Shah, J Segmentation of MR images using curve evolution and prior information. International Journal of Pattern Recognition and Artificial Intelligence, Vol. 11, No. 8, pp Ranganath, S Contour extraction from cardiac MRI studies using snakes. IEEE Trans. Medical Imaging, Vol. 14, No. 2, pp

36 Samson, C., Blanc-Feraud, L., Aubert, G. and Zerubia, J A Level Set Model for Image Classification, International Journal of Computer Vision, Vol. 40, No. 3, pp Sethian, J A fast marching level set method for monotonically advancing fronts. Proc. Nat. Acad. Sci., Vol. 93, No. 4. Siddiqi, K., Lauziere, Y.B., Tannenbaum, A. and Zucker, S.W Area and length minimizing flows for shape segmentation, IEEE Trans on Image Processing, Vol. 7 pp Stovik, G A Bayesian approach to dynamic contours through stochastic sampling and simulated annealing. IEEE Trans. on PAMI, Vol.16, No.10, pp Toennies, K. and Rueckert, D Image segmentation by stochastically relaxing contour fitting. Medical Imaging 1994: Image Processing, vol. 2167, pp , Bellingham, Wash: Int l Soc. Optical Eng. Xu, C. and Prince, J Snakes, shape, and gradient vector flow. IEEE Trans. on Image Processing, Vol. 7, pp Yezzi, A., Kichenssamy, S., Kumar, A., Olver, P., and Tannebaum, A A geometric snake model for segmentation of medical imagery. IEEE Trans. Medical Imaging, Vol. 16, No. 2, pp Zhong, M The motion-constrained active contour model and its medical imaging application. MS. Thesis, University of Cincinnati. Zhu, S. and Yuille, A Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation, IEEE Trans. on PAMI, Vol. 18, No.9, pp

37 Zhu Y. and Yan, H Computerized tumor boundary detection using a Hopfield neural network. IEEE Trans. on Medical Imaging, vol. 16, pp