AN OVERVIEW OF IMAGE SEGMENTATION TECHNIQUES. IN Fabris-Rotelli 1 and J-F Greeff 1. ABSTRACT

Transcription

1 AN OVERVIEW OF IMAGE SEGMENTATION TECHNIQUES IN Fabris-Rotei 1 and J-F Greeff 1 1 Department of Statistics, University of Pretoria, 0002, Pretoria, South Africa, inger.fabris-rotei@up.ac.za ABSTRACT We present an overview of some image segmentation techniques, empoyed to extract regions of interest. Exampes, with comparisons, are presented for Iterative Seection, Baanced Histogram, Otsu s method, Wener agorithm, Integra Image agorithm, Gaussian mixtures and Iterated Conditiona Modes. 1. INTRODUCTION Image segmentation pays an integra part within computer vision, since the identification of regions of interest is usuay the first step in extracting usefu information from an image. We provide an overview of possibe methods of segmenting an image, incuding exampes of resuts. Appications of segmentation techniques incude medica imaging (Pham et a., 2000), image compression (Vaisey & Gersho, 1992), facia recognition (Jain & Par, 2009), scene cassification (Debba et a., 2008), handwriting recognition (Shapiro & Stocman, 2002) and ane detection (Sezgin & Sanur, 2004). A good overview of appications can be found in Shapiro & Stocman (2002). Many methods exist that aim at segmenting images (Shapiro & Stocman, 2002; Mobahi et a., 2011): threshoding to create a bac and white (BW) image; custering iterative methods, such as -means; compression-based segmentation to minimise the data coding ength; histogram-based methods to identify segments; edge detection; region growing to group neighbouring pixes with simiar uminosities; and spit-and-merge methods where the image is repeatedy spit into heterogeneous squares and merged into homogeneous squares. These methods are not mutuay excusive, Otsu s method is both a histogrambased and custering method. Throughout the paper we wi refer to an image matrix of pixe uminosities of size n m as I, where I(i, j) is the uminosity of pixe (i, j) with i = 1, 2,..., n and j = 1, 2,..., m, noting that I (i, j) coud be a vector of uminosity vaues. J-F Greeff is the corresponding and presenting author. Thans to the Department of Statistics and Statomet for support and funding. 1 34

2 2. THRESHOLDING Threshoding creates BW images from gray-scae images. In its simpest form, a pixes with intensities beow an appropriatey chosen threshod eve T are considered to be bacground pixes, represented by white pixes; a pixes with intensities above T are considered the foreground, represented by bac pixes. This is termed threshod above, whie threshod beow considers pixes beow T to be the foreground (Shapiro & Stocman, 2002). Other threshoding techniques incude band threshoding, p-tie threshoding, optima threshoding and adaptive threshoding (Henden, 2004). Standard threshoding suffers from ac of robustness in the presence of non-stationary and correated noise, ambient iumination, variation of intensity eves within the objects and the bacground, inadequate contrast, the object size reative to the image size, and a ac of an objective performance measure (Sezgin & Sanur, 2004). Suggested soutions incude noise reduction techniques (Nachtegae et a., 2001), adaptive threshoding, and cropping of the image to mae the object the focus of the image. Rider & Cavard (1978) present a specia case of -means custering, caed Iterative Seection (IS), to find an optima T. The agorithm is as foows 1. Assume some initia binary segmentation, 2. cacuate the mean uminosity of the foreground, µ (0) b, and bacground, µ (0) f, 3. cacuate the threshod as T (0) = ( ) 1 µ (0) 2 b +µ (0) f, 4. segment the image using T 0, 5. repeat steps 2 to 4, where µ (α) b, µ (α) f and T (α) are the mean uminosities and threshod vaue at iteration α, unti some predetermined stop criteria is met. The Baanced Histogram (BH) method is presented by Anjos & Shahbazia (2008). It is a recursive mode in which the histogram is baanced to find a weighted mid-point of the grayscae uminosities in the set I = {I(i, j) (i, j)} 1. Define I (0) = { I(i, j) I : I(i, j)<m (0)} and I u (0) = { } I(i, j) I : I(i, j) m (0) for some m (0), and construct the image histogram as f(t)= card{(i,j):i(i,j)=t} n m for a t I The method can be summarised as 1. choose an initia point m (0) for the weighted midpoint of the histogram domain, 2. w = t I (0) f(t) and w u = t I (0) u cacuate the weights of the histogram on either side of m (0) as f(t), 3. remove weight from the outer-edge of the heavier side of the histogram, that is if w < w u, then et I u (1) = I u (0) \ maxi u (0) and I (1) = I (0), or if w w u, then et I (1) = I (0) \ mini (0) and I u (1) = I u (0), 4. recacuate the mid-point m (1) of the new histogram domain I (1) = I (1) (1) I u, 5. repeat steps 3 to 5, unti I (α) is a singe vaue which is then taen to be the threshod for this image. Otsu s Method (OM) is popuar for seecting the optima threshod by minimising the within-custer uminosity variance within the foreground and bacground custers. It can be shown that this is equivaent to maximising the between-custer variance, given by σ 2 B (T ) = p 1(T ) p 2 (T ) [µ 1 (T ) µ 2 (T )] 2, where f(t) is the image histogram, p 1 (T ) = t<t I(t), p 2(T ) = 1 p 1 (T ), µ 1 (T ) = t<t tf(t), and µ 2(T ) = t T tf(t). This reduces the effort needed to obtain the optima threshod by ony requiring estimation of custer means and custer reative frequency (Otsu, 1979; Shapiro & Stocman, 2002). Together with its 1 The definition of I used within this agorithm overcomes the probem of extreme vaues within the domain of the histogram, which may cause the agorithm to converge to an incorrect soution. 2 35

3 effectiveness this method is popuar in practice and iterature and is incuded in many software pacages. Eraser image. T = T = 88. T = 99. Grass image. T = T = 22. T = 140. Rice image. T = T = 127. T = 131. Tan image. T = T = 40. T = 99. Fig. 1: Image resuts for each origina image (eft) based on Iterative Seection (center-eft), Baanced Histogram (center-right) and Otsu s method (right). The threshod in each case is T. In Figure 1 we see that BH underperformed compared to OM and IS and that T for each image is significanty different. This can be attributed to the simpicity of BH, which does not tae into account the sewness of the image histograms. It is possibe to obtain a ess sewed distribution by cropping the image but this requires user interaction. In the rice image this probem does not occur due to the approximatey equa number of ight and dar pixes. We see that the threshods and resuts obtained using IS and OM are simiar. For the eraser image the observed segmentation is very good due to the star bacground/foreground contrast, custer uminosity homogeneity, and ac of ambient iumination. The textured grass image presents a bigger chaenge due to greater custer uminosity variabiity. The appication of a noise reduction fiter (Lim, 1990) to the segmentation provides an improvement by removing the texture effect. The rice image iustrates the issue with ambient ight. The grain edges near the bottom edge are not as ceary defined as those in the centre due to a contrast reduction by the shadow. Adaptive threshoding (Section 3) provides a soution. The tan image shows the effect of the reative size of foreground 3 36

4 and bacground, as we as the effect of ac of contrast. When appied to a cropped tan image, a better segmentation is obtained; because arge sections of the miscassified shrubbery, with simiar uminosities to the tan, are removed, and because the size of the tan reative to the bacground is increased. 3. ADAPTIVE THRESHOLDING In Adaptive threshoding the threshod changes according to the position of the pixe being threshoded. The threshod may be different for each individua pixe, or constant over regions in the image; enabing robustness to ambient ight changes (Wener, 1993). We provide an overview of two agorithms, the Wener agorithm (WA) and the Improved Image agorithm (IIA). In Wener (1993) s agorithm a pixe s threshod is determined as a fraction of the average neighbourhood pixe uminosity. Define s N to be the size of the neighbourhood and t (0, 1) as the reative threshod parameter. Wener discusses methods for determining the average pixe uminosity in each neighborhood, and how to dea with image boundaries. However, the Wener agorithm ony considers pixes in the same row meaning information above and beow the pixe is ignored. Define the neighbourhood of (i, j) as N s (i, j) = {(a, b) I : a [ i s 2, i+ s 2], b = j} and the average neighbourhood uminosity as µ s (i,j) = (i,j) Ns(i,j) I(i,j) card(n s(i,j)). A pixe is cassified as bacground (foreground) if I(i, j) < (1 t) µ s (i, j) (I(i, j) (1 t) µ s (i, j)). Wener (1993) and our own investigation found that s = m and t = 0.15 provide satisfactory resuts for a range of images. Bradey & Roth (2007) propose an improvement on WA, the Improved Image agorithm (IIA). They cacuate the average 8 uminosity within a rectanguar neighbourhood around the pixe to be threshoded. To ighten the computationa oad they introduce the integra image J(i, j) = a i b ji(a, b) and compute J ony once. They redefine the neighbourhood N s (i, j) = {(a, b) I : a [i s /2, i+ s /2], b [ j s, j+ s 2 2] } with i 1 (i 2 ) = min a (max a )N s (i, j), j 1 (j 2 ) = min b (max b )N s (i, j), so that the average neighbourhood uminosity is µ s (i, j) = J(i 2,j 2 ) J(i 2,j 1 ) J(i 1,j 2 )+J(i 1,j 1 ) (i 2 i 1 )(j 2 j 1 ). In Figure 2 we iustrate the mared improvement of IIA over WA and OM, with two images subject to ambient ight. The WA seems more robust to ambient ight. Miscassification does sti occur near the shadow borders and areas with a ong run of simiar uminosities. The atter is addressed by IIA. A cear improvement is seen when comparing the resuts for WA and IIA. The miscassification in the top and bottom sections of the QR codes are not observed in the IIA resuts, and the miscassification in the centre aong the shadow edges disappears. Overa, the QR codes are more accuratey represented, even near the shadow edges. In the journa image, the ine beow the tite is more accuratey segmented, with no missing sections. The text is generay more ceary defined and the dots running perpendicuar to the tite are mosty removed. One important aspect to highight is the fact that both WA and IIA extract information not visibe to the human eye. This is ceary seen in the journa image, where initiay not visibe parts of the tite and text are extracted. In regions where shadows create a uniform band of uminosities, a sight deviation in uminosity sti triggers a segmentation resut, maing the technique usefu in data recovery or 4 37

5 image compression. An investigation into a smaer neighbourhood size, s, and arger reative threshod, t, coud yied vauabe insight into extracting such information within these regions. Bradey & Roth (2007) showed that IIA is approximatey 2.5 times sower than WA, but this is negigibe in practice. Fig. 2: Image resuts for each image (eft) based on Otsu s method (center-eft), the Wener agorithm (center-right) and the Integra Image agorithm (right). (Journa image from Bradey & Roth (2007).) 4. GAUSSIAN MIXTURES Gaussian Mixtures (GM) are a specia case of mixture distributions, described by McLachan & Pee (2000), in which a distribution is obtained by superimposing two or more norma distributions by means of a inear transformation. The probabiity density function of such a GM is given by f(x) = K =1 π f (x) = K =1 π (2π) d 2 Σ 1 2 exp{ 1 (x µ 2 ) Σ 1 (x µ )} for x R d where each of the K components of f are normay distributed in d-dimensions, with mean vector µ and covariance matrix Σ ; and have mixing coefficients π [0, 1], such that K =1 π = 1. By atering K, we are abe to fit f to any continuous distribution with an arbitrary degree of accuracy (Bishop, 2006). In grayscae image segmentation we tae d=1 and K the number of custers to identify; whereas for more genera scenarios, such as coour images, we et d 1, specificay d = 3 for RGB images. The probem is then to fit f to the observed pixe uminosities by estimating the 3K parameters of f, through maximum ieihood estimation and expectation-maximisation estimation (Bishop, 2006). The responsibiities of pixe (i, j) are given by γ(z ) = E [z I(i, j)] = π f (I(i,j) K i=1 π if i (I(i,j)) for = 1, 2,..., K, that is, the conditiona expected vaues of the eements z 1, z 2,..., z K, indicating to which custer the pixe beongs, given the pixe s uminosity. Pixe (i, j) is custered to the custer with the highest responsibiity (Bishop, 2006). Figure 3 presents image histograms with the fitted GM overaid, and the image segmentation. For each image on the eft K =2 was used, whie for those on the right K =3 (eraser image) and K =4 (rice image) was used. In the cases of K > 2 we have aso grouped two or more custers to form a BW image with ony two custers. The resuts of the eraser image with K = 2 are simiar to those obtained using OM. For K = 3, however, we observe that the GM extracts the grain of the wooden des from the bacground. There are three distinct peas in the distribution for the image histogram, impying at east three custers. Merging the two custers removes amost a of the miscassified pixes around the bottom of the image. 5 38

6 We see that GM with K =2 underperforms OM in the rice image, due to the ambient ight. The histogram dispays three distinct peas, and a section of the domain with a neary constant frequency, suggesting at east four custers in the image, supported by the we-fitting GM, with K =4. Intuitivey, this is expected since there are four natura custers - the grains, bacground, and inside and outside the shadow. In this case the GM outperforms even the IIA which is specificay designed to dea with ambient ight. π = (0.59; 0.41) and µ = (15; 190). π = (0.41; 0.18; 0.41) and µ = (10; 27; 191). π = (0.81; 0.19) and µ = (95; 183). π = (0.23; 0.4; 0.22; 0.16) and µ = (61; 102; 137; 185). Fig. 3: Histogram of the pixe uminosities with the fitted GM (eft) and the image resuts (right). In the case of more than 2 custers, the appropriate custers are merged to create two custers (far right). 5. ITERATED CONDITIONAL MODES Iterated Conditiona Modes (ICM) is an agorithm introduced by Besag (1986) to reduce the noise in dirty pictures. It taes into account both features of each pixe and spatia information based on a Marov Random Fied of each pixe to be custered (Debba et a., 2008). ICM, within the context of noise remova, is based on the assumption that neighbouring pixes tend to have simiar uminosities, or other features, and that each pixe is corrupted independenty with a given probabiity. Consider an image I in which there are K custers of pixes which we woud ie to detect and extract. { Then, for each } iteration of the agorithm, indexed by α, define ω (α) ij the custer of pixe x=(i, j); C (α) = x : ω (α) ij = the set of pixes beonging ( ) ( ) to custer ; N (α) = card C (α) the number of pixes in custer ; N (α) ij () = card C (α) N(x) the number of pixes in a neighbourhood N(x), of x in custer ; µ (α) = 1 I(x) the d-dimensiona N (α) x C (α) mean vector of custer ; ν (α) = 1 K ( ) ( ) n m =1 I(x) µ (α) x C (α) I(x) µ (α) the tota within-custer variance. ICM minimises the tota within-custer variance, by assigning and reassigning each pixe in the image to a cass, whie taing spatia information into account. We proceed as foows, (1) initiaise the parameters using ony the uminosity of each pixe. We used a mutivariate K-means custering procedure (see Hartigan & Wong (1979)); (2) cacuate C (α), N (α), µ (α) and ν (α) for each = 1, 2,..., K; (3) ( ) ( ) cacuate Λ(x, ) = I(x) µ (α) I(x) µ (α) βν (α) N ij () and find = arg min {Λ (x, )} for 6 39

7 a x; (4) set ω (α+1) ij = for each pixe in the image to recassify the pixe to custer ; (5) repeat 2 through 4 unti C (α+1) = C (α) for a = 1, 2,..., K or unti some predetermined stop criterion is met. If the agorithm converges the sets C are the custers of pixes, grouped according to their iey custer membership, taing into account the spatia ocation of the pixe. The function Λ is simiar to the function minimised within the K-means framewor, but with a second term, caed the spatia penaisation term (Debba et a., 2008), aowing for the incusion of spatia information. In effect, we are reducing the within-custer sum of square deviations by a mutipe of the number of pixes in the neighbourhood which are in the custer. Since if a pixe is surrounded by many pixes which beong to custer, it is more iey that that pixe aso beongs to custer, due to our first assumption, we increase the ieihood that wi minimise Λ, by reducing the size of Λ(x, ) by a constant reated to our spatia information. Debba et a. (2008) suggest that a good choice for the parameter β is 1.5. A arger vaue eads to a smoother image based more heaviy on spatia data, whie a ower vaue eads to a custering simiar to K-means. Figure 4 iustrates the effect of the β parameter. Overa, it was found that ICM is an effective segmentation method due to the incusion of spatia information as we as we as more robust than adaptive threshoding. 6. CONCLUSION β = 0.5 β = 1 β = 1.5 β = 2 β = 2.5 β = 5 Fig. 4: Comparison of the effect of the β parameter in ICM with K = 5. We have presented an overview of image segmentation techniques for grayscae images. Iterative Seection, the Baanced Histogram, Otsu s method, the Wener agorithm, the Integra Image agorithm, Gaussian mixtures and Iterated Conditiona Modes were discussed. We concude for genera segmentation Otsu s method is the preferred technique. It is, however, not robust in the presence of ambient ight, in which case we recommend the use of the Integra Image agorithm. A more robust aternative for incuding spatia information is provided by the Iterated Conditiona Modes agorithm. The Gaussian mixtures agorithm provides an effective method for fitting a distribution to the image histogram. It provides a good segmentation, especiay when mutipe custers are identified and grouped. Due to their short processing times the BH and WA are usefu as an initia segmentation for more compex methods. REFERENCES Anjos, A. & Shahbazia, H. (2008). Bi-Leve Image Threshoding - A Fast Method. Biosignas, 2, Besag, J. (1986). On the statistica anaysis of dirty pictures. J R Stat Soc, 48(3),

8 Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer, first edition. Bradey, D. & Roth, G. (2007). Adaptive Threshoding using the Integra Image. J Graphics, GPU, Game Toos, 12(2), Debba, P., Stein, A., van der Meer, F., Carranza, E., & Lucieer, A. (2008). Fied Samping from a Segmented Image. In ICCSA 2008, voume 5072 of Lect Notes Comput Sc (pp ). Springer Berin / Heideberg. Hartigan, J. & Wong, M. (1979). A K-means custering agorithm. App Stat-J Roy St C, 28, Henden, P. (2004). Exercise in Computer Vision. unpubished manuscript, NTNU Facuty of IT. Jain, A. & Par, U. (2009). Facia mars: Soft biometric for face recognition. In P 16th IEEE Int Conf Image Process (pp ). Lim, J. S. (1990). Two-Dimensiona Signa and Image Processing. Engewood Ciffs, N.J.: Prentice Ha. McLachan, G. J. & Pee, D. (2000). Finite Mixture Modes. New Yor: Wiey. Mobahi, H., Rao, S., Yang, A., Sastry, S., & Ma, Y. (2011). Segmentation of Natura Images by Texture and Boundary Compression. Int J Comput Vision, 95(1), Nachtegae, M., Van der Ween, D., Van De Vie, D., Kerre, E., Phiips, W., & Lemahieu, I. (2001). An overview of cassica and fuzzy-cassica fiters for noise reduction. In 10th IEEE Int Conf Fuzzy Sys, voume 1 (pp. 3 6). Otsu, N. (1979). A Threshod Seection Method from Gray-Leve Histograms. IEEE T Syst Man Cyb, SMC-9(1), Pham, D., Xu, C., & Prince, J. (2000). Current Methods in Medica Image Segmentation. Annu Rev Biomed Eng, 2, Rider, T. W. & Cavard, S. (1978). Picture Threshoding Using an Iterative Seection Method. IEEE T Syst Man Cyb, 8(8), Sezgin, M. & Sanur, B. (2004). Survey over image threshoding techniqies and quantitative performance evauation. J Eectron Imaging, 13(1), Shapiro, L. & Stocman, G. (2002). Computer Vision. Upper Sadde River, NJ: Prentice Ha. Vaisey, J. & Gersho, A. (1992). Image compression with variabe boc size segmentation. IEEE T Image Process, 40(8), Wener, P. (1993). Adaptive Threshoding for the DigitaDes. Technica Report EPC , Ran Xerox. 8 41