Image Segmentation using Gaussian Mixture Models

Transcription

1 Image Segmentation using Gaussian Mixture Models Rahman Farnoosh, Gholamhossein Yari and Behnam Zarpak Department of Applied Mathematics, University of Science and Technology, 16844, Narmak,Tehran, Iran Abstract. Recently stochastic models such as mixture models, graphical models, Markov random fields and hidden Markov models have key role in probabilistic data analysis. Also image segmentation means to divide one picture into different types of classes or regions, for example a picture of geometric shapes has some classes with different colors such as circle, rectangle, triangle and so on. Therefore we can suppose that each class has normal distribution with specify mean and variance. Thus in general a picture can be Gaussian mixture model. In this paper, we have learned Gaussian mixture model to the pixel of an image as training data and the parameter of the model are learned by EM-algorithm. Meanwhile pixel labeling corresponded to each pixel of true image is done by Bayes rule. This hidden or labeled image is constructed during of running EM-algorithm. In fact, we introduce a new numerically method of finding maximum a posterior estimation by using of EM-algorithm and Gaussians mixture model which we called EM-MAP algorithm. In this algorithm, we have made a sequence of the priors, posteriors and they then convergent to a posterior probability that is called the reference posterior probability. So Maximum a posterior estimation can be determined by this reference posterior probability which will make labeled image. This labeled image shows our segmented image with reduced noises. This method will show in several experiments. Keywords: Bayesian Rule, Gaussian Mixture Model (GMM),Maximum a Posterior (MAP), Expectation-Maximization (EM) Algorithm, Reference Analysis. PACS: c, Mj, Tp INTRODUCTION Automatically image processing means image segmentation(i.e. dividing an image into different types of regions or classes), recognizing of objects and detecting of edges,etc by machine. All of these can be done after segmentation of a pictures. So image segmentation is the most important image problems. In addition noise removing and noise reduction of pictures always are important in classical image problems. In this paper, we do both segmentation and noise reduction with a probabilistic approach. There are many mathematical and statistical methods for image problems, but this paper argues about GMM as a general Gaussian distribution, EM-algorithm and Bayesian rule. But Bayesian framework usually has many difficulties because the posterior probability has complex form. So we must use Markov Chain Monte Carlo algorithms or Variational methods with high computing to find MAP estimation. These methods have worked well in the last decades [1,]. In this paper, a new numerically EM-MAP algorithm base on Bayesian rule has constructed. We have used Bernardo s theory about reference analysis [2] in practice and in image segmentation. it means that in reference analysis a sequence of priors and

2 posteriors are made and they convergent to a posterior probability which is called reference posterior probability. In this paper, we have used this idea. So we have modified EM-algorithm for our image segmentation. After finding reference posterior probability, MAP estimation and pixel labeling can easily make segmented of image. This paper organized as follows. It first reviews of GMM and its properties. Then we introduce the EM-MAP algorithm for learning parameters for a given image as the training data. Choosing initial values of EM-algorithm have discussed in next section. EM -MAP algorithm never can not convergent without choosing a suitable starting points. But these initial information are made by histogram of image. Finally we show some experiments with simulated images. GAUSSIAN MIXTURE MODELS Image is a matrix which each element is a pixel. The value of the pixel is a number that shows intensity or color of the image. Let X is a random variable that takes these values. For a probability model determination, we can suppose to have mixture of Gaussian distribution as the following form f (x) = k p i N(xjµ i ;σi 2 ) (1) i=1 Where k is the number of regions and p i > 0 are weights such that k i=1 p i = 1 N(µ i ;σi 2 1 )= σ p 2pi exp (x µ i ) 2 2σ 2 Whereµ i ;σ i are mean, standard deviation of class i. For a given image X, the lattice data are the values of pixels and GMM is our pixel base model. However, the parameters are θ =(p 1 ;:::;p k ; µ 1 ;:::;µ k ;σ1 2 ;:::;σ k 2 ) and we can guess the number of regions in GMM by histogram of lattice data. This will show in experiments. i (2) EM-MAP ALGORITHM There is a popular EM algorithm for GMM in several papers [3,6]. We modify it to the following algorithm with the name of EM-MAP algorithm. 1. Input:Observed Image in a vector x j ; j = 1;2;:::;n and i 2f1;2;:::;kglabels set 2. Initialize: θ (0) =(p (0) 3. (E-step) 1 ;:::;p(0) k p (r+1) ij ; µ (0) 1 = P (r+1) (ijx j )= (0) ;:::;µ ;σ 2(0) k p (r) i 1 ;:::;σk 2(0) ) N(x j jµ (r) i f (x j ) ;σ 2(r) i ) (3)

3 4. (M-step) n (r+1) 1 ˆp i = n p (r) ij j=1 ˆµ (r+1) i ˆσ 2(r+1) i = = n j=1 p(r+1) x ij j (r+1) n ˆp i n j=1 p(r+1) (r+1) (x ij j ˆµ i ) (r+1) n ˆp i. Iterate steps 3 and 4 until an specify error i.e. i e 2 i < ε 6. Compute p lj = ArgMax i p ( f inal) ; j = 1;2;:::;n (7) ij 7. Construct labeled image corresponding of each true image pixel. This EM-MAP algorithm is a pixel labeling base method such that the labeled image shows each segment or object by different type of labels. Note that formula (3) is Bayes rule, p r i is discrete prior probability in stage r and p(r+1) is discrete posterior probability ij in the next stage. In this algorithm, we make a sequence of prior and then posterior until to get convergence. The labeled images chooses with MAP of the final posterior. (4) () (6) CHOOSING A PRIOR WITH MAXIMUM ENTROPY PROPERTY In EM-MAP algorithm, there are some difficulties. How can we choose? the number of classes the weights the means the variances A practical way that we can guess the prior of parameters is drawing the histogram of the observed image. Not only image histogram gives us four above parameters for using initial values in the EM algorithm, but also this extracted information usually has maximum entropy. This claim has shown in experiments. Besides, the final posterior probability will get a stable entropy. We also compute the number of misclassifications of results. This shows that how much our algorithm is well. EXPERIMENTS In first example, we make three boxes in an image and add it white noise. The observed image and its histogram in the figure 1 and figure 2 are shown. Information extracted

4 FIGURE 1. a)the Observed Image of Boxes,b)Histogram of the Observed Image FIGURE 2. a)labeled image with reduced noises,b)entropy curve in each iteration from the histogram are k = 4; p (0) =(0:03;0:1344;0:076;0:7760) or empirical probability with entropy 0:7411; µ (0) = (40;7;2;2)andσ (0) = (0;0;0;0). The stopping time occurs when L-2 norm of absolute error has very small value. After running EM-MAP, we had ten-times iteration in figure 2 and the entropy of each iteration which goes to a stable or maximum case. We see in segmented image that Blue = l, cyan = 2,Yellow = 3 andred = 4. There is percentage misclassification that is only one red instead of yellow pixel is wrong. In this example, pixel labeling and noise removing are well done. In the second example, some different shapes such as circle, triangle, rectangle, etc have considered. The observed image and its histogram in figure 3 have shown. For EM-MAP finding, we need to get initial values by histogram of this image. We

5 FIGURE 3. a)the Observed Image of Circle,b)Histogram of the Observed Image FIGURE 4. a)labeled image with reduced noises,b)entropy curve in each iteration choose k = 3; p = (0:08; 0:34; 0:8) as empirical probability or relative frequency, µ =(38; 63; 88) and σ =[12; 12; 12] with norm of error less than In figure 4, we made 3 classes Blue = 1,Green = 2 andred = 3. There are only missing of classifications or percentage of If in EM-MAP algorithm, we compute the entropy of the posterior probability in Each stage of iteration, this entropy will be decreasing to reach a stable form. The third example is more complex. The number of components is great. In addition, there is dependent noise in image which noise reduction in this case is more difficult. The true image and observed image have shown in figure. Again,information extraction can find by drawing histogram of observed image, it means

6 FIGURE. a)the True Image of Partition,b)The Observed Image of Partition FIGURE 6. a)histogram of Image,b)The Segmented Image of Partition k = p =(0:0977; 0:1377; 0:06; 0:0693; 0:0361; 0:0361; 0:1182; 0:1904; 0:72; 0:0947) µ =(1;2;2:;3:;4:;:;6;7;7:;8:) σ =(0:;0:;0:;0:;0:;0:;0:;0:;0:;0:) The results have shown in figure 6 with times iteration. In figure 7 the results are shown in 0 times iteration and entropy has reached in a stable case.

7 FIGURE 7. a)labeled image with reduced noises,b)entropy curve in each iteration CONCLUSIONS In this paper, we make a new numerical EM-GMM-Map algorithm for image segmentation and noise reduction. This paper is used BernardoŠs idea about sequence of prior and posterior in reference analysis. We have used known EM-GMM algorithm and we added numerically MAP estimation. Also the initial values by histogram of image have suggested which is caused to convergence of EM-MAP method. After convergence of our algorithm, we had stability in entropy. EM-algorithm is iteration algorithm of first order [3], so we had slow convergence. We used acceleration convergence such as Steffensen algorithm to have the second order convergence. But later we note that in EM- MAP method, the number of classes will reduce to real classes of image. Finally, EMalgorithm is linear iteration method, so our method is suitable for simple images. It is important to note that "for segmentation of real images, the results depend critically on the features and feature models used" [4] that is not the focus of this paper. ACKNOWLEDGMENTS We have many thanks to prof.mohammad-djafari for his excellent ideas. REFERENCES 1. C.Andrieu, N.D.Freitas, A.Doucet, M.I.Jordan, An Introduction to MCMC for Machine Learning, Journal of Machine Learning, 03, 0, pp J.M. Bernardo and A.F.M. Smith, Bayesian Theory, John Wiley & Sons, L.Xu, M.I.Jordan, On Convergence Properties of the EM Algorithm for Gaussian Mixture, Neural Computation, 8, 1996, pp M.A.T.Figueiredo, Bayesian Image Segmentation Using Gaussian Field Prior, EMMCVPR 0, LNCS 377, pp

8 . M.I.Jordan, Z.Ghahramani, T.S.Jaakkola, L.K.Saul, An Introduction to Variational Methods for Graphical Models, Journal of Machine Learning, 1999, 37, pp R.Farnoosh, B.Zarpak, Image Restoration with Gaussian Mixture Models, Wseas Trans. on Mathematics, 04, 4, 3, pp