Evolutionary Optimization of Filter Parameters for Image Segmentation

Transcription

1 Evolutionary Optimization of Filter Parameters for Image Segmentation Timothy J. Rupe 30th November 2004 Abstract This paper explores the use of evolutionary strategies (ES) for parameter selection of image segmentation algorithms. Normally, segmentation algorithms are tuned by hand by experimenting with parameters, and finding out what happens. This can be a time consuming and computationally expensive process. ES give us a way to explore possible parameter combinations without the user having to do a lot of guess work. Through experimentation, we find that this method does work well, however the goal of minimizing human interaction is very difficult to accomplish. The last experiment took over thirteen hours to complete, and required human interaction about every five minutes. 1

2 Contents 1 Introduction Problem Statement Goals Background 4 3 EA Design and Implementation Input Parameter File Input Data File Output Data File Representation Fitness Function Population Initialization Parent Selection Recombination Mutation Evaluation Survivor Selection Termination Condition User Manual 8 5 Experiments and Results Experiment # Experiment # Experiment # Experiment # Conclusions 15 7 Future Work 17 References 18 2

3 1 Introduction This paper explores the use of evolutionary strategies (ES) to select parameters for existing image partitioning algorithms. The algorithms partition an image into disjoint regions that follow two basic properties. The regions must be homogeneous, encompassing data that share common characteristics such as color and texture. They must also be heterogeneous in relation to each other, having distinct boundaries. Formally, the problem is described in the following way [2]: Let I be an image and H be a homogeneity predicate; then the segmentation of I is a partition of I into a set of regions {R 1, R 2,...,R n } such that: 1.1 Problem Statement n i=1 R = I, R i R j =, i j The Morphology Net project, headed by Dr. Ann Maglia and Dr. Jennifer Leopold, is focused on creating and managing a library of animal morphological data. The centerpiece of this data library is the three dimensional (3D) reconstruction of anatomy. A 3D representation of an animal allows for different measurements, comparisons, and study methods for the physical structure of an animal. Modern techniques such as Computerized Axial Tomography (CT) scan and magnetic resonance imaging (MRI) have been considered for imaging an animal for 3D reconstruction. These methods work well for large species such as dogs and humans, but the image resolution is too poor for very small animals. For example, herpetologists often need to study small frogs only two centimeters long, and using a CT scan or MRI is impossible. Physically dissecting the frog is difficult, and taking measurements between internal body parts is prone to error. An alternative method involves taking very thin axial slices of the animal from nose to tail. These slices are digitally captured, and stored as computer image files. Using this method, high resolution data can be captured from very small animals. Once the slices have been created, they still need to be processed to create a 3D model. This is performed by creating bounded regions encompassing body parts of interest. Once the bounded regions for all slices have been created, they can be combined together to create 3D bounded regions defining the anatomical representation of the animal. The problem with this method involves the time consuming process of defining bounded regions. Currently this step is performed by hand using computer desktop software. Each region of interest needs to be traced and adjusted. One animal may be sliced into dozens of slices, and will often require weeks of painstaking tracing to complete the reconstruction of just one animal. 1.2 Goals Currently it is possible to simply write a program using common image filters to generate regions for a given image. The problem here is two-fold. First, the number of possible parameter and filter combinations is astronomical. Finding an optimal or 3

4 near optimal solution by evaluating every possible permutation or performing a random search is unacceptable. Secondly, each image is unique, and an acceptable set of parameters for one image may not work well for another. An alternative method is to use an evolutionary algorithm (EA) to search for an optimal set of filters and accompanying parameters. An EA allows for a directed search of potential parameter sets, and allows for incremental steps towards better combinations. The specific EA I am using is the evolutionary strategies (ES) method. The ES method can optimize the segmentation process, potentially requiring as little human intervention as possible. Using an ES based program, a person could supervise the process, giving guidance by grading the algorithm s progress. 2 Background Computer vision, texture recognition, and image segmentation are well researched fields. Most methods involve static algorithm parameters for determining image segments [6]. Using evolutionary algorithms is a fairly new method for image segmentation. A primary concern for segmentation is the need to constantly tweak the algorithm parameters, and evolutionary algorithms is one way to minimize the need for supervision by people [2]. One research program that relies heavily on these topics is the Visible Human Project (VHP), hosted by the National Library of Medicine. The VHP provides thin axial slices of both a man and a woman, which are much like the axial slices of small animals used by Morphology Net. Fortunately, powerful tools such as the Insight Visualization Toolkit (ITK) have been developed by the National Library of Medicine, and are perfect for use in this project. The ITK is a segmentation and registration development library. It provides filters and constructs for segmenting both 2D and 3D images, as well as registering and aligning layered data from multiple sources. This library provides well made filters usable for image segmentation as well as a plug-in interface for new custom filters. Using filters provided by the ITK, I am providing a method for dynamically modifying ITK filters in order to maximize the segmentation quality. 3 EA Design and Implementation Most tasks involving image processing require all source images to have very similar properties, otherwise custom filters have to be created for each image. One way to create a more general purpose solution is to use stochastic operations. To create a process that can be used over a wider variety of images, EA can be used. The nature of EA allows for a more dynamic search of optimal filter properties[3], but because we don t want a pure random search, the EA must have direction through the use of a fitness function. One method is to use human intervention in the guidance and training of an EA [4]. Human intervention is likely necessary for most image processing tasks because the quality or fitness of results can be very subjective. However, great care 4

5 Population Offspring Tournament Input Output thorax.bmp thorax_result.bmp neck.bmp neck_result.bmp head.bmp head_result.bmp Table 1: Example Input Parameter File must be made to minimize the need for human interaction in order to improve the overall performance of the algorithm. An EA isn t required to directly perform the image processing but can be used to tune the parameters of various filtering algorithms[5]. This has the benefit of having well known algorithms do the dirty work of raw manipulation while the EA guides the overall process. In order to simplify the overall process, I chose which image filters and algorithms to use for the tests, and then limited the evolutionary process to parameter selection. This keeps the complexity of the overall project down, but still allows for experimentation using ES. Future projects will allow for filter selection as well. 3.1 Input Parameter File The parameter file contains a list of unique problem descriptions, where each line describes the desired parameters for a unique run as seen in table1. Each line contains the following information: 1. Population size 2. Number of offspring created for each generation 3. Tournament Size 4. Input image filename 5. Output image filename 3.2 Input Data File The input data file consists of a standard bitmap image. The image format can be one of several common varieties, such as jpeg, bmp, tiff, and gif. For my purposes, I will be using the bmp format because it doesn t involve any compression, and has a simple implementation. The input data will not be altered in any way. 3.3 Output Data File The output data file is represented as a bitmap image, with distinct colored shapes depicting individual regions created by the segmentation algorithms. The image format can be specified by file extension, or defaults to the same format used by the input image. Eventually, the data will be saved as vector data, allowing easy importation into 3D modeling programs. 5

6 Conductance Its Threshold Scale Gradient σ 1 σ 2 σ 3 σ 4 float integer float float boolean float Table 2: Examples of Individual Representation 3.4 Representation Each individual in the segmentation problem represents a set of parameters for the image processing filters. A common representation of an individual can be described as an array of real values, the first half of which represent filter parameters and the other half represents the standard deviation parameters used in ES uncorrelated mutation with n step sizes, where n represents the number of parameters. The representation I use differs slightly to accommodate the input parameters expected by the image manipulation filters. The image filters use floating point, integer, and boolean values, so the representation I use reflects these needs. The use of boolean values negates the need to use a mutation parameter used by ES because a boolean value is never partially changed. For the image filters I have chosen, an example of two randomly generated individuals is shown in table Fitness Function Unfortunately, the segmentation quality of an image is highly subjective. Depending on the desired results of the human operator, a candidate solution can score well or poorly. This means a more subjective type of image evaluation is needed. The method by which a candidate solution s fitness is evaluated is based on points and age, which allows more flexible image evaluation, but requires human intervention to award points. During the evaluation phase as described later, the person performing image evaluations gives points to those individual believed to be performing better than other images presented. As the EA is progressing through each generation, surviving individuals accumulate a growing score and age. The fitness of an individual is based on their score/age ratio where Fitness = score age. The goal is to keep individuals with a good fitness ratio. Individuals who age, but don t score well, will suffer by having a lower fitness. Individuals who keep getting points have the chance to keep their fitness higher despite increasing age. This fitness method gives better scoring individuals in the population a high probability of survival, while allowing new individuals a chance to compete. Currently, individuals are awarded a fixed number of points by the user. This amounts to a pass/fail type system, but future implementation may allow for more flexible methods of awarding points. 6

7 Figure 1: Child Creation 3.6 Population Initialization A small initial population (for example 10) of candidate solutions is created and the resulting segment images are presented to the user. The user chooses which candidate solutions to keep, which are used to create children to fill the remaining population slots. This method allows for a guided initialization so initial population will have a higher average fitness, and the user will not have to evaluate as many poor performers. 3.7 Parent Selection For each child to be created, two parents must be selected from the population. Parent selection is performed using a tournament. For a given tournament, n individuals are randomly chosen from the population, and the best is selected as a parent. The size of the tournament can be adjusted to increase or decrease the selective pressure. For the creation of λ children, 2λ parents must be selected using this method. 3.8 Recombination The creation of a child from two parents is performed in two stages. The first stage fills the parameter values for the child by performing local recombination. For each parameter in the child, the value is chosen randomly from one or the other parents. This is continued until all parameter values in the child are filled. The second stage fills the standard deviation values of the child by using global recombination, using the average of the same value for the rest of the population. Figure 1 shows an example of the creation of a child from two parents. 3.9 Mutation The child mutation is based on uncorrelated mutation with n step sizes, and is performed in two steps. First, the standard deviation values σ are mutated, and then based 7

8 on the new values, the parameter values x are mutated. Formally, the standard deviation values and parameter values are mutated in the following way [1]: σ = σ i e τ N(0,1)+τ N i(0,1) x i = x i + σ i N i (0, 1) Note that because one of the parameters is a boolean value, there is no σ value associated with it, and is mutated by randomly flipping it with a probability of 1 n Evaluation The evaluation of individuals is the one point in the program where user intervention is required. The evaluation process starts by taking the parameters of an individual and along with the input image, pass the values to the image segmentation algorithm. The resulting image is the graphical representation of the found segment regions. The resulting image is then viewed by the user, and if the user chooses, the individual that represents this image is given points. The point award system is completely dependent on the decision of the user. This means that a user may not always award the same individual points between generations. That can happen because another image performs better, or the user simply changed their mind. This makes the results of this program unpredictable Survivor Selection In order to minimize the amount of human intervention, a high selective pressure method is desirable. Survivor selection uses a deterministic rank based method, utilizing the (µ, λ) method. All children are added to the population, and the worst ranking individuals are simply removed from the population Termination Condition There is no target fitness value, and the desired output is dependent entirely on the needs of the user. The evolutionary cycle will continue indefinitely until the user feels that either the segmentation process has produced a satisfactory result, or has given up after finding only unsatisfactory results. 4 User Manual The current release of the program is console based. The immediate disadvantage of this is felt when the user must view image segmentation results. The user must manually open the image results with an image viewer when evaluating individuals. A future version will use a graphical interface, and all images will be automatically presented to the user for evaluation. The use of this program is fairly easy. The user creates a configuration file described earlier containing desired parameters for the program. The program is then run along 8

9 Population Offspring Tournament Input Output image.bmp image_result.bmp Table 3: Experimental Parameters with the name of the parameter file name in the command line. The program reads the parameter file, checks for legitimate values and a valid input image, and starts the evolutionary process. Every time a population individual needs to be evaluated, the image is saved to disk, and the file name is displayed. The user opens the image, and depending on the quality of the results, the user gives a pass or fail score on the command line. This is repeated for every evaluation during the lifetime of the evolutionary process. Currently, every time an individual needs to be evaluated, the image segmentation must be performed. A future version will cache results, preventing wasted processing time when an individual is evaluated multiple times. This should substantially decrease the overall time to find an acceptable solution. If the user decides to terminate the evolutionary process and end the program, they simply enter the stop command at the next evaluation prompt. The parameters for the individual with the highest fitness value will be displayed, and the program will end. 5 Experiments and Results The experiments are designed to see how well the program works with increasingly more complex data sources. The first image is very simple, and poses no real problem finding optimal parameter values, but the last image poses the most difficult challenge. Due to the nature of this project, each experiment took a significant amount of time to run. Each generation took about an hour to complete due to the processing cost of running the segmentation algorithms. Therefore, few experiments have been run so far, and comparisons between different evolutionary algorithm parameters have not been completed yet. Once a faster method of segmentation has been implemented and the need for human guidance has been reduced, more experiments will be possible, and better EA parameters can be tried. Each experiment used the same EA parameters shown in table 3 with only the input and output images differing. 5.1 Experiment #1 This experiment will test the algorithm s ability to segment a simple image consisting of only three regions, made by splitting image vertically down the center with an encompassing white boarder as seen in figure 2. As expected, the algorithm did an excellent job finding the regions as can be seen in figure 3. In fact, the result was found on only the second generation of evolution. I believe the filters used were robust enough and the input image simple enough that nearly any kind of parameter inputs would have found near optimal results. 9

10 Figure 2: Simple Data Source Figure 3: Simple Data Source Results 10

11 Figure 4: Basic Shapes Data Source 5.2 Experiment #2 This experiment uses a more complex image with multiple regions of different shapes and sizes as seen in figure 4. The results shown in figure 5 are also very good, however this image does show the first signs of small discrepancies. The region shaped like a star was not segmented perfectly. It the tips of the star points, there are a few additional regions shown. I believe this is caused by the sharp angles, in those areas. Despite this small error, the results are more than acceptable to me. If another person were running the program, this may still not be acceptable to them, and they could continue the evolution process until their desires for image quality were met. 5.3 Experiment #3 This experiment uses more complex and overlapping shapes seen in figure 6. Some of the shapes also have outlined boarders, and are the only places where discrepancies are found. It appears that the segmentation filter has difficulty in very small regions. Instead of a line becoming one region, it is broken down int very small individual regions. Still, the quality of the result is very good, and I decided to stop the evolutionary process after three generations. 5.4 Experiment #4 This experiment poses the largest difficulty for this program. The input data file in figure 8 is a cross-section of the thorax in the visible human project. This image is 11

12 Figure 5: Basic Shapes Data Source Results Figure 6: More Complex Data Source 12

13 Figure 7: More Complex Data Source Results Figure 8: Thorax Data Source 13

14 Figure 9: Thorax Result 1 Figure 10: Thorax Result 2 14

15 Figure 11: Thorax Result 3 very complex, with varying colors, textures, and potential region sizes. I expected this experiment to perform fairly poorly. However, it did better than I expected. The results after the first few generations were showing recognizable regions, but the large quantity of tiny regions made the segmentation look noisy. The results are promising, but not usable yet. The results after about ten generations look even better in figure 10, but still show some signs of noisiness. Even with these improvements, the results are not acceptable to me. After thirteen generations, the results as seen in figure 11 are even better. I believe these are acceptable, and I stopped the process here. The segmentation process is not perfect, however most of the primary body tissue areas are well defined, with just a fairly small amount of noise present. Because this experiment had the most generations completed, I was able to generate a fitness graph of its progress. Figure 12 shows the progress of this experiment. As you can see, even though the fitness is in part guided by human intervention, the overall shape of the curve resembles the progress of common EA plots. 6 Conclusions I am partly pleased, and partly disappointed in the results I have found. The accuracy and segmentation quality is very good for all experiments performed. I am especially pleased about the results in experiment #4. I did not expect it to perform as well as it did. However, the time involved to perform these experiments was large. With each generation taking about an hour to complete, and every individual having to be evaluated by me, the time spent working with the program was extremely boring. If this process has any chance to be useful in practice, this limitation must be eliminated. 15

16 Figure 12: Experiment #4 Plot 16

17 7 Future Work There are three primary things I would like to improve in future work. First, a graphical user interface is essential when presenting image data to a user. The current practice of requiring the user to open each image file to view it is impractical. One idea is to present a group of images, and have the user choose the best one. This could possibly be integrated in the tournament selection process. Second, a faster segmentation algorithm must be found. Currently, every segmentation process takes about five minutes. If human intervention is used, time is of essence. Third, human intervention must be eliminated or significantly reduced. Requiring a person to intervene every five minutes for thirteen hours is unacceptable. Even if it is reduced to half an hour, this level of intervention is too much. One direction I plan to explore is the use of a target template. The user can provide an example of what the result should look like, and the program can use it as a goal. The fitness function could use similarity to this goal to rate the individual s fitness. 17

18 References [1] A.E. Eiben and J.E. Smith. Introduction to Evolutionary Computing. Natural Computing Series. Springer-Verlag, Berlin Heidelberg, [2] Chih-Chin Lai and Chuan-Yu Chang. A hierarchical genetic algorithm based approach for image segmentation. In Networking, Sensing and Control, 2004 IEEE International conference on, Vol.2, pages , [3] Cristian Munteanu, V. Lazarescu, and C. Radoi. A new strategy in optimization using genetic algorithms. In Electrotechnical Conference, MELECON 98., 9th Mediterranean, volume 1, pages , May [4] Cristian Munteanu and Agostinho Rosa. Evolutionary image enhancement with user behavior modeling. SIGAPP Appl. Comput. Rev., 9(1):8 14, [5] Will Smart and Mengjie Zhang. Applying online gradient descent search to genetic programming for object recognition. In Proceedings of the second workshop on Australasian information security, Data Mining and Web Intelligence, and Software Internationalisation, pages Australian Computer Society, Inc., [6] S. Wang and J.M. Siskind. Image segmentation with ratio cut. In Pattern Analysis and Machine Intelligence, IEEE Transactions, Vol. 25, Iss. 6, pages , June