MRF-based texture segmentation using wavelet decomposed images

Transcription

1 Pattern Recognition 35 (2002) MRF-based texture segmentation using wavelet decomposed images Hideki Noda a;, Mahdad N. Shirazi b, Ei Kawaguchi a a Department of Electrical, Electronic and Computer Engineering, Kyushu Institute of Technology, 1-1 Sensui-cho, Tobatu-ku, Kitakyushu, Japan b Communications Research Laboratory, Iwaoka, Nishi-ku, Kobe, Japan Received 28 March 2000; received in revised form 12 January 2001; accepted 12 March 2001 Abstract In recent textured image segmentation, Bayesian approaches capitalizing on computational eciency of multiresolution representations have received much attention. Most of the previous researches have been based on multiresolution stochastic models which use the Gaussian pyramid image decomposition. In this paper, motivated by nonredundant directional selectivity and highly discriminative nature of the wavelet representation, we present an unsupervised textured image segmentation algorithm based on a multiscale stochastic modeling over the wavelet decomposition of image. The model, using doubly stochastic Markov random elds, captures intrascale statistical dependencies over the wavelet decomposed image and intrascale and interscale dependencies over the corresponding multiresolution region image.? 2002 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved. Keywords: Image segmentation; Texture; MRF; Wavelet; Multiresolution; Unsupervised 1. Introduction An important problem in image processing is segmentation of an image into disjoint regions which may possess the same average gray level but dier in the spatial distribution of gray levels (texture). Segmentation of images based on textural features is a critical preliminary operation in various image processing applications ranging from computer vision to remote sensing. In the last decade, there has been considerable interest in Bayesian estimation techniques in conjunction with Markov random elds (MRFs) for segmenting textured images. These methods use distinct stochastic processes Corresponding author. Tel.: ; fax: address: noda@know.comp.kyutech.ac.jp (H. Noda). to model textures covering each region and typically an MRF with smooth spatial behavior to model a region image. Segmentation of an image is then achieved by nding an approximate Maximum a posteriori (MAP) estimate of the unknown region image given the observed image. Recently, motivated by the importance of utilizing information at various scales, a number of authors proposed multiresolution approaches to textured image segmentation, mainly to capitalize on computational eciency of multiresolution models. Multiresolution approaches make it possible to capture and utilize correlations over a set of neighborhoods of increasing size by making use of multiresolution representations for the region image and single [1,2] or multiresolution representations [3,4] for the observed image. Bouman et al. [1] used a Gaussian autoregressive model for the observed image. The MAP estimate of the region image at the coarsest /02/$22.00? 2002 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved. PII: S (01)

2 772 H. Noda et al. / Pattern Recognition 35 (2002) resolution is approximated rst, using iterated conditional modes (ICM) [5], and the result is then used as an initial condition for segmentation at the next ner level of resolution, and the process is continued until individual pixels are classied at the nest resolution. Krishnamachari et al. [3] used a Gauss Markov random eld (GMRF) to model the observed image at each resolution with the assumption that the random variables at a given resolution are independent of the random variables at other levels. It was assumed that the GMRF parameters at the nest resolution are known. The MAP estimate is approximated at each resolution using ICM rst at the coarsest level and then progressively at ner levels. Comer et al. [4] proposed a multiresolution Gaussian autoregressive model for the observed image which takes into account the correlation between adjacent levels of resolutions. It was assumed that the parameters of the Gibbs distribution of the region process are known. Segmentation is achieved there as the maximum posterior marginals (MPM) estimate rather than the MAP estimate. Most of the previous multiresolution approaches, however, have been based on modeling of the region and=or observed image over the lattice structure which corresponds to the Gaussian pyramid decomposition [6] of the observed image. In this paper, we propose a multiresolution Bayesian approach based on modeling over the lattice structure which corresponds to the wavelet decomposition [7] of the observed image. The wavelet decomposition of an image, as opposed to the Gaussian pyramid decomposition, results in nonredundant and direction (horizontal, vertical, diagonal) sensitive features at dierent scales and hence allows more selective feature extraction in space-frequency domain. It might also be easier to model the nonredundant and direction sensitive subbands of a wavelet decomposed image than to model each subband of a Gaussian pyramid decomposed image where the image at a resolution contains all the information in the lower resolutions. The proposed modeling scheme captures, over the wavelet pyramidal lattice, signicant intrascale and interscale statistical dependencies in the region image and intrascale statistical dependencies in the observed image, using doubly stochastic MRFs. To estimate model parameters, a version of the expectation-maximization (EM) algorithm [8] is used where the Baum function is approximated using the mean-eld-based decomposition of a posteriori probability of the region process [9]. The mean-eld-based decomposition is also used in nding the MAP estimate of the region process. The paper is organized as follows. In Section 2, a doubly stochastic MRF model, usually used to model textured images, is described. The proposed multiscale stochastic model of textured images in wavelet domain is presented in Section 3. The corresponding EM-based parameter estimation and MAP-based image segmentation algorithms are given in Section 4. Simulation results are given in Section 5, followed by concluding results in Section Ordinary textured image modeling An image consisting of dierent regions of textures is usually modeled by a hierarchical Markov random eld (HMRF) which consists of two layers. In the following we give a brief description of MRF and the HMRF and then introduce a typical specic model for textured images Markov random eld Let L = {(i; j); 1 6 i; j 6 N } denote a nite set of sites of an N N rectangular lattice. Let L denote the (i; j)-pixel s neighborhood of a random eld 1 L dened on L. Let C denote the set of cliques C associated with which contains the (i; j)-pixel, i.e., (i; j) C. For example, in the rst-order neighborhood, = {(i; j 1); (i 1;j); (i; j +1); (i +1;j)} and C = {{(i; j)}; {(i; j); (i; j 1)}; {(i; j); (i 1;j)}; {(i; j); (i; j+1)}; {(i; j); (i+1;j)}} which consists of one singleton and four doubleton cliques. Let the random eld L = { ;(i; j) L} be a Markov random eld dened on L with s taking values from a common local state space Q. It is well known that an MRF is completely described by a Gibbs distribution [10] p(x L )= 1 exp{ U(x Z L )}; (1) where x L is a realization of L from the conguration space = Q N N and U(x L )= U(x C ) (2) (i;j) L C C is the global energy function whereas U(x C ) is the clique energy function and Z = exp{ U(x L )} (3) x L is the partition function. For details on MRFs and related concepts such as the neighborhoods and cliques, see Ref. [11] A two-layered hierarchical Markov random eld An image comprising dierent textures can be considered as a realization of a collection of two interacting random variables ( L ;Y L ) dened on the lattice L: the 1 In this paper, x A and f(x A ) denote the set {x a1 ;:::;x al } and the multivariable function f(x a1 ;:::;x al ), respectively, where A = {a 1 ;:::;a l }.

3 H. Noda et al. / Pattern Recognition 35 (2002) region label process L = { ;(i; j) L} and the observation process Y L = {Y ;(i; j) L}. 2 The observation process is assumed to be a function of the region label process. The interacting processes ( L ;Y L ) can be characterized completely by a joint probability p(x L ;y L )or equivalently by p(x L ) and p(y L x L ). We have already specied p(x L ) in Eqs. (1) (3); thus to characterize ( L ;Y L ) it suces to specify the conditional probability p(y L x L ). Assuming that observed images are realizations of MRFs with neighborhoods which are functions of the underlying region labels, i.e., Y = Y (x ), a general expression for p(y L x L ) can be given as p(y L x L )= 1 exp{ U(y Z L x L )}; (4) Y where the global conditional energy function is U(y L x L )= U(y C x ) (5) (i;j) LC C Y (x) and the conditional partition function is Z Y = exp{ U(y L x L )}: (6) y L Y (x L ) In Eq. (6), Y (x L )= (i;j) L Q Y (x ), where Q Y (x )is a set from which Y takes a value A specic model comprising multi-level logistic MRF and GMRFs Consider an image consisting of M distinct textural regions. Let be an indicator vector taking values from the vector set Q = {e 1 ;:::;e M }, where e m, for 1 6 m 6 M,istheM dimensional unit vector whose mth component is 1 and all other components are 0. Then, x takes e m when the region label of (i; j)-pixel is m. To model the hidden region label process, we adopt a multi-level logistic MRF (LMRF) with the second-order neighborhood system. In this model all clique energies are assumed to be zero except for the doubleton clique energies which are given by { if x = x + U(x ; x + )= (7) otherwise: In Eq. (7), N = {(0; 1); (0; 1); (1; 0); ( 1; 0); (1; 1); ( 1; 1); ( 1; 1); (1; 1)}. For example when = (0; 1), x + = x i;j+1. The local conditional probability 2 In order to model region boundaries, another process like the line process [11] can be added resulting in modeling by three processes. In this paper, we prefer not to use such a model for the sake of simplicity. for the hidden region label process is given by p(x x )= exp{ N U(x ; x + )} x Q exp{ N U(x ; x + )} : (8) The observed textures are modeled by GMRFs with the second-order neighborhood systems characterized by the following local conditional probability density functions (pdfs) 1 p(y y ; x = e m)= (2 m) 1=2 [ exp 1 (y m) ] 2 m (y + m) 2 m : N (9) Here m, m and m stand for the mean, variance and the pair clique s interaction parameter, all depending on the region label m. The interaction parameters (prediction coecients) are assumed to be symmetric, i.e., m =, m where =( i; j) if =(i; j). 3. Textured image modeling in wavelet domain 3.1. Image modeling in wavelet domain Let y L denote the original image and w L = W(y L ) the J -level wavelet transformed image. In what follows, we model the collection of two interacting random variables (x L ;w L ), which form a doubly stochastic random process, using HMRFs. For the J -level wavelet transform, the lattice L can be decomposed as L = {LL(J ); LH(J ); HL(J ); HH(J );:::;LH(1); HL(1); HH(1)} (10) which corresponds to 3J +1 subbands of the decomposed image w L. Here the rst L or H, respectively refers to a low or high frequency passband in the horizontal direction and the second L or H refers to a low or high frequency vertical passband (see Fig. 1). Using these sublattices, x L and w L can be partitioned as 3 x L = { x LL(J ) ; x LH(J ) ; x HL(J ) ; x HH(J ) ;:::;x LH(1) ; x HL(1) ; x HH(1) }; (11) w L = { w LL(J ) ;w LH(J ) ;w HL(J ) ;w HH(J ) ;:::;w LH(1) ; w HL(1) ;w HH(1) }: (12) 3 For the sake of convenience, the indices for sublattices are put at upper position.

4 774 H. Noda et al. / Pattern Recognition 35 (2002) Fig. 1. An example of 2-level wavelet transform: (a) original image, (b) wavelet transformed image, (c) sublattices. Considering that every hidden label process at the same scale should be identical 4 and denoting the label process at the sth scale (level), dened on the lattice L s = {(i; j); 1 6 i; j 6 N=2 s },asx (s) we get x LL(J ) = x LH(J ) = x HL(J ) = x HH(J ) = x (J ) ; x (J ) = {x (J ) }; (i; j) L J ;. x LH(1) = x HL(1) = x HH(1) = x (1) ; x (1) = {x (1) }; (i; j) L 1 : p(x L ) can then be written as p(x L )=p( x LL(J ) ; x LH(J ) ; x HL(J ) ; x HH(J ) ;:::;x LH(1) ; x HL(1) ; x HH(1) ) (13) = p(x (J ) ;:::;x (1) ): (14) As for p(w L x L ), considering that decomposed subimages of the same scale can constitute a vector image, i.e., w (J ) =(w LL(J ) ;w LH(J ) ;w HL(J ) ;w HH(J ) ) T for (i; j) L J and w (s) =(w LH(s) ;w HL(s) ;w HH(s) ) T for (i; j) L s at s =1;:::;J 1, and assuming that w (s) = {w (s) ;(i; j) Ls} is dependent on the corresponding hidden label process x (s), it can be written as p(w L x L )=p( w LL(J ) ;w LH(J ) ;w HL(J ) ;w HH(J ) ;:::; w LH(1) ;w HL(1) ;w HH(1) x (J ) ;:::;x (1) ) (15) = p( w LL(J ) ;w LH(J ) ;w HL(J ) ;w HH(J ) x (J ) ) = p(w LH(s) ;w HL(s) ;w HH(s) x (s) ) (16) J 1 J p(w (s) x (s) ): (17) 4 To be exact, this should be true after adjusting the origins of the coordinates of all subimages Multi-level logistic MRF and GMRFs in wavelet domain To model the hidden region label process in the wavelet domain, which has a multiscale structure composed of x (J ) ;:::;x (1), we adopt a LMRF with interscale neighborhood as well as intrascale neighborhood. The chosen neighborhood is the second-order neighborhood for the intrascale neighborhood and, one parent and four children for the interscale neighborhood. The neighborhood of the (i; j)-pixel at the sth scale, (s) ; is explicitly described as follows: (s) = { (i; j +1) (s) ; (i; j 1) (s) ; (i +1;j) (s) ; (i 1;j) (s) ; (i +1;j+1) (s) ; (i 1;j 1) (s) ; (i 1;j+1) (s) ; (i +1;j 1) (s) ; ([(i +1)=2]; [(j +1)=2]) (s+1) ; (2i 1; 2j 1) (s 1) ; (2i 1; 2j) (s 1) ; (2i; 2j 1) (s 1) ; (2i; 2j) (s 1) }; (18) where the indices (s); (s + 1) and (s 1) show the scale levels and [ ] shows the function to produce the greatest smaller integer. In this model, clique energies are given only for doubleton cliques as { ; x(s) + )= s s if x (s) = x (s) + otherwise (19) for the intrascale cliques, where N = {(0; 1); (0; 1); (1; 0); ( 1; 0); (1; 1); ( 1; 1); ( 1; 1); (1; 1)}, as { ; s;s+1 if x (s) x(s+1) = x (s+1) i i pj p )= pj p (20) otherwise s;s+1 for the interscale (to parent) clique, where i p =[(i + 1)=2]; j p =[(j +1)=2], and as { ; s 1;s if x (s) x(s 1) = x (s 1) (2i;2j)+ (2i;2j)+ c )= c (21) s 1;s otherwise

5 H. Noda et al. / Pattern Recognition 35 (2002) for the interscale (to child) clique, where c N c = {( 1; 1); ( 1; 0); (0; 1); (0; 0)}. Correlations of region labels are considered through these clique energy parameters; for example, a larger value of s 0 represents a stronger tendency that region labels for members of relevant intrascale clique are the same. Using these clique energies, the local conditional probability of x (s) in the wavelet domain is given by 4.1. Parameter estimation The EM method [8] is an iterative method to perform the maximum likelihood (ML) estimation with incompletely observed data. It is considered that the observed image w L, the J -level wavelet transformed image, is incomplete data and the set consisting of the observed p(x (s) x (s) )= exp{ [ N U(x(s) ; x(s) + )+U(x(s) x (s) Q exp{ [ N U(x(s) ; x(s) ; x(s+1) i pj p + )+U(x(s) )+ c N c ; x(s+1) i pj p )+ c N c ; x(s 1) (2i;2j)+ c )]} ; x(s 1) (2i;2j)+ c )]} : (22) Note that the clique energies in Eq. (20) are removed at the coarsest J th scale and those in Eq. (21) removed at the rst scale. The vector image w (s) at the sth scale in Eq. (17) is modeled by multidimensional GMRFs with the second-order neighborhood systems characterized by the following local conditional pdfs: p(w (s) w W (s) { exp 1 2 (w(s) ŵ (s);m = m (s) ; x (s) = e m)= + 1 (2) K=2 (s) m 1=2 ŵ (s);m ) T ( m (s) ) 1 (w (s) ŵ (s);m ) B m;(w (s) (s) + (s) N Here K denotes the dimension of w (s) three for s =1;:::;J 1), ŵ (s);m } ; (23) m ): (24) (four for s = J and denotes the predicted, and m (s), m (s) and vector using neighboring vectors w W (s) B m; (s) stand for the mean vector, covariance matrix of the prediction error vectors (w (s) ŵ(s);m ) and spatial interaction parameter matrix for pairwise cliques, all depending on the class label m. The correlation between w (s) and w (s) + is considered through B(s) m;. The spatial interaction parameter matrix is reminiscent of the prediction coecient matrix in the linear vector prediction and therefore can be simply referred to as the prediction matrix. The prediction matrices are assumed to be symmetric, i.e., B (s) m; = B (s) m;. 4. Unsupervised segmentation algorithm We have already proposed an unsupervised textured image segmentation method where the mean-eld-based decomposition of a posteriori probability is used for parameter estimation and image segmentation [9]. Image segmentation for wavelet transformed images can follow the method for original images, although the procedures for wavelet transformed images become more complex. image w L and the unobservable region label image x L (in fact a set of region indicator vectors, x L ) is complete data. The EM method consists of the expectation step (E-step to obtain the Baum function) and the maximization step (M-step): E-step: Q( (p) )= p(x L w L ; (p) ) log p(x L ;w L ; ): x L M-step: (25) (p+1) = arg max Q( (p) ): (26) Here = { ; W (s); s =1;:::;J} represents the set of all parameters to be estimated. = { s; t;t+1 ; s =1;:::;J; t =1;:::;J 1} is the set of MRF parameters of the region process and W (s) = { m (s) ; (s) m ; B m;; (s) N; m =1;:::;M} is the set of GMRF parameters for w (s). (p) is a provisionally estimated set of at the pth iteration. The Baum function in Eq. (25) represents the sum over all possible congurations of x L, 5 and it is dicult (practically impossible) to calculate this. To overcome this problem, we use the mean-eld-based decomposition of a posteriori probability to calculate the Baum function [9]. The a posteriori probability p(x L w L ; (p) ) in Eq. (25) can be written as p(x L w L ; (p) ) = p(w L x L ; (p) )p(x L ; (p) ) x L p(w L x L ; (p) )p(x L ; (p) ) : (27) Using the mean eld approximation, p(w L x L ; (p) ) and p(x L ; (p) ) are decomposed as [12] J p(w L x L ; (p) )= p(w (s) x (s) ; (p) ) W (s) 5 Since x L = {x (J ) ;:::;x (1) }, in Eq. (25) becomes equal to Q L J L 1.

6 776 H. Noda et al. / Pattern Recognition 35 (2002) J p(w (s) (i;j) L s p(x L ; (p) )=p(x (J ) ;:::;x (1) ; (p) J p(x (s) (i;j) L s w W (s) ; x (s) ; (p) ); (28) W (s) ) x (s) ; (p) ); (29) where denotes the mean eld for. Substituting Eqs. (28) and (29) into Eq. (27) and replacing J x L (i;j) L s by J (i;j) L s x (s) Q, we get the following decomposition for p(x L w L ; (p) ): J p(x L w L ; (p) ) p(x (s) w (s) ; w W (s) where p(x (s) w (s) ; w W (s) = p(w (s) x (s) Q (i;j) L s ; x (s) ; (p) ); (30) ; x (s) ; (p) ) w W (s) ; x (s) ; (p) p(w (s) )p(x (s) W (s) w W (s) ; x (s) ; (p) )p(x (s) W (s) x (s) ; (p) ) x (s) ; (p) ) : (31) p(x (s) w (s) ; w W (s) ; x (s) ; (p) ) is considered as a local a posteriori probability (LAP) and hereafter we write it as (x (s) ) for short. Then the LAPs for all region indicators form a vector (LAP vector), =( (x (s) = e 1 );:::; (x (s) = e M )) T.Itis reasonable to use the LAP vector as the mean eld of x (s), x (s) [9]. Then, using Eq. (31) (x (s) ) is computed by (x (s) ) = x (s) p(w (s) where w W (s) {z (p) kl p(w(s) Q w W (s) ; x (s) ; (p) w W (s) ; x (s) ; (p) )p(x (s) W (s) z (p) (s) ; (p) ) )p(x (s) W (s) z (p) (s) ; (p) ) ; (32) is simply used for w W (s) and z (p), (s) }. Note that in order to calculate, we need z(p), those ; (k; l) (s), the LAP vector for x (s) (s) for x (s). Therefore the LAP vectors can be calculated by iterative procedures popular in numerical analysis. In p(x (s) z (p) ; (p) (s) ) the doubleton clique energies in Eqs. (19), (20) and (21) should be changed as ; z(s) + )= s(x(s) )T z (s) + + s(1 (x(s) )T z (s) + ) = s(1 2(x (s) )T z (s) + ); (33) ; z(s+1) i pj p )= s;s+1 (1 2(x (s) )T z (s+1) i pj p ); (34) ; z(s 1) (2i;2j)+ c )= s 1;s (1 2(x (s) )T z (s 1) (2i;2j)+ c ): (35) Now we can approximately calculate the Baum function with the mean-eld-based decomposition of p(x L w L ; (p) ), J (k;l) L s kl (x (s) kl ), and with the same decomposition for log p(x L ;w L ; ). Q( (p) ) J kl (x (s) x L (k;l) L s kl ) J logp(w (s) w W (s) (i;j) L s J + log p(x (s) z (p) ; (s) ) (i;j) L s J = z (i;j) L s x (s) Q log p(w (s) + (p+1) = arg max W (s) W (s) J (s);(p) (x (s) ) w W (s) ; x (s) ; W (s)) (s);(p) z (x (s) ) (i;j) L s x (s) Q log p(x (s) z (p) (s) ; x (s) ; W (s)) ; ): (36) Once the Baum function is obtained, the M-step is carried out straightforwardly as follows: (p+1) = arg max z (i;j) L s x (s) Q (s);(p) (x (s) ) logp(w (s) w W (s) ; x (s) ; W (s)) (37) J z (i;j) L s x (s) Q log p(x (s) z (p) (s) (s);(p) (x (s) ) ; ) : (38) The provisional estimate of the LMRF s parameters in Eqs. (33) (35), by Eq. (38), cannot be given in a mathematically closed form and needs to be calculated using an appropriate numerical optimization method. In the following experiments we used the Newton method to estimate. The provisional estimate of GMRFs parameters W (s), by Eq. (37), can be expressed in a mathematically closed form as follows. The reestimate of the

7 H. Noda et al. / Pattern Recognition 35 (2002) mean vector (s);(p+1) m (s);(p+1) m = (i;j) L s is given as / (m)w (s) (i;j) L s (m) ; (39) where (m) represents (x (s) = e m). Assuming B m; (s);(p+1) s = B (s);(p+1) m; s, the reestimates of the prediction matrices B m; (s);(p+1) s can be obtained by solving the following matrix equation: (B (s);(p+1) m; 1 A (s) st = ; B m; (s);(p+1) 2 ; B m; (s);(p+1) 3 ; B (s);(p+1) 13 A(s) A(s) A(s) 34 A (s) 11 A(s) 12 A(s) A (s) 21 A(s) 22 A(s) A (s) 31 A(s) 32 A(s) A (s) 41 A(s) 42 A(s) 43 A(s) 44 A (s) 0t = (i;j) L s m; 4 ) =(A (s) 01 ; A(s) 02 ; A(s) 03 ; A(s) 04 ); (40) (m)(w (s) + s + w (s) s 2 (s);(p+1) (w (s) + t + w (s) t 2 m (s);(p+1) (i;j) L s (i;j) L s ) T / m ) (m) ; (41) (m)(w (s) }/ +w (s) t 2 m (s);(p+1) ) T (i;j) L s m (s);(p+1) )(w (s) + t (m) ; (42) where s, t N = {(0; 1); (1; 0); (1; 1); ( 1; 1)} (if s t then s t). For example 1 =(0; 1), 2 =(1; 0), 3 =(1; 1), 4 =( 1; 1). Finally, the reestimate of the covariance matrix m (s);(p+1) is given as m (s);(p+1) (i;j) L s = (m)(w (s) ŵ (s);m (i;j) L s (m) )(w (s) ŵ (s);m ) T ; (43) ŵ (s);m = (s);(p+1) m Image segmentation B m; (s);(p+1) s (w (s) + s + w (s) s 2 m (s);(p+1) ): (44) Segmentation of an image into regions of dierent textures amounts to estimation of the hidden region process x L. In principle, it is carried out with the nally estimated parameters. However, a provisional segmentation is also possible with provisionally estimated parameters at each iteration in the EM method. In the following we describe the segmentation in this situation. Given a wavelet transformed image w L and an estimated parameter set (p), the provisional estimation of x L is carried out by maximizing the a posteriori probability p(x L w L ; (p) ) (MAP estimation). 6 x (p) L, arg max p(x L w L ; (p) ): (45) x L As seen from Eqs. (30), (31) and (32), this global optimization problem is approximately decomposed into the local optimization problems using the LAPs (x (s) for x (s) s x (s);(p) = arg max x (s) Q (x (s) )s ): (46) Since segmentation in a ner resolution is generally favorable, the estimated region image at the rst scale, x (1);(p) can be used as a segmentation result. As a better alternative, the averaged LAP over all scales z (1);(p) (x (1) )= 1 J i J sj s (x (s) i sj s ); (47) where (i 1 ;j 1 )=(i; j), i s =[(i 1)=2 s 1 +1] and j s =[(j 1)=2 s 1 + 1], can be used to obtain x (1);(p). In the following experiments, the averaged LAP z (1);(p) (x (1) )is utilized Initial parameter estimation To start iterative procedures in the EM method, initial values of MRF parameters should be given in advance. These initial values are derived as follows. A vector image at each scale, w (s) is divided into small blocks, for example, consisting of 8 8 pixels. Assuming a single 6 To solve this optimization problem, the stochastic relaxation algorithm known as the simulated annealing (SA) [11] can be used. However, we prefer not to use the SA since it demands formidable computation.

8 778 H. Noda et al. / Pattern Recognition 35 (2002) Fig. 2. Segmentation results: (a) given image, (b) segmented image by using given original image, (c) by only LL(1) image, (d) by 1-level decomposed images, and (e) by 2-level decomposed images. Fig. 3. Segmentation results: (a) given image, (b) segmented image by using given original image, (c) by only LL(1) image, (d) by 1-level decomposed images, and (e) by 2-level decomposed images. Fig. 4. Segmentation results: (a) given image, (b) segmented image by using given original image, (c) by only LL(1) image, (d) by 1-level decomposed images, and (e) by 2-level decomposed images. Fig. 5. Segmentation results: (a) given image, (b) segmented image by using given original image, (c) by only LL(1) image, (d) by 1-level decomposed images, and (e) by 2-level decomposed images.

9 H. Noda et al. / Pattern Recognition 35 (2002) Fig. 6. Segmentation results: (a) given image, (b) segmented image by using given original image, (c) by only LL(1) image, (d) by 1-level decomposed images, and (e) by 2-level decomposed images. Fig. 7. Segmentation results: (a) given image, (b) segmented images for the number of regions M =2; 3 and 4 by using given original image, (c) by only LL(1) image, (d) by 1-level decomposed images, and (e) by 2-level decomposed images. texture for each block, a set of texture parameters (composing a vector) is estimated. Assuming that the number of regions is known, these parameter vectors derived from all blocks are classied into the known number of regions by using a clustering method. Then the texture parameters for each dierent region are again estimated using all blocks classied to the same region and are used as initial texture parameters (0). W Appropriate positive values can be used as initial region (s) parameters, { s; t;t+1 ; s =1;:::;J;t=1;:::;J 1} in Eqs. (33) (35). In the following experiments we use 0.5 for all ss and t;t+1 s. 5. Simulation results To evaluate the performance of the proposed unsupervised segmentation method, we applied the method

10 780 H. Noda et al. / Pattern Recognition 35 (2002) Fig. 8. Segmentation results: (a) given image, (b) segmented images for the number of regions M =2; 3 and 4 by using given original image, (c) by only LL(1) image, (d) by 1-level decomposed images, and (e) by 2-level decomposed images. to seven images shown in Figs. 2(a) 8(a). Among them, Figs. 2(a) 6(a) are synthesized textured images consisting of three natural textures from the Brodatz album [13]. All part (b)s of the gures are derived segmentation results using given original images, part (c)s are those using only the LL(1) subimage w LL(1), part (d)s are those using four decomposed subimages of the 1-level wavelet transform, and part (e)s those using seven decomposed subimages of the 2-level wavelet transform. For synthesized images, for which true region label images are available, segmentation error rates can be calculated as the number of misclassied pixels over the total number of pixels and each error rate is written at the bottom of the corresponding gure. We believe that error mainly comes from lack of modeling capability to model textured images, i.e., if statistical correlations in textured images are appropriately taken into account in modeling, segmentation performance should be improved. From the bad segmentation results shown in all part (b)s of gures, it is seen that the resolution of these original images is too ne; these original images are too coarse to be modeled accurately by the GMRF with the second-order neighborhood. Using the LL(1) subimage, i.e., down-sampled image, some textured images such as the one given in Fig. 2(a) can be segmented fairly well, but general segmentation results are still bad (see all part (c)s of gures). Comparing the results shown in part (d)s of all gures with those in part (b)s and part (c)s, it is seen that the use of a set of wavelet decomposed images generally improves segmentation performance. Furthermore, by comparing the results shown in part (e)s of all gures with those in part (d)s, we can see that it is very eective to consider statistical correlations over multiple scales by using multi-level wavelet decomposed images. Segmentation results for real images are shown in Figs. 7 and 8 for several numbers of regions (the

11 H. Noda et al. / Pattern Recognition 35 (2002) number of regions M =2; 3 and 4). 7 Although performance evaluation is dicult for real images without true region labels, the segmentation results subjectively seem to show a similar tendency for synthesized images. At least we may be able to say that the results using multi-level wavelet decomposed images shown in part (e)s of all gures are better than others from the viewpoint of textured image segmentation Conclusions In this paper, we presented a wavelet-based multiresolution Bayesian approach to the problem of segmenting textured images. The approach makes use of the modeling power of MRF models and the multiscale and highly discriminative nature of the wavelet representation, and underlies a multiscale segmentation algorithm which, as shown by experimental results, uses eectively the statistical regularities over multiple scales. Incorporating interscale correlations, which exist over wavelet decomposition of the observed image, into the model and the corresponding segmentation algorithm is an issue for further research. Extension to Markov modeling over wavelet packet decomposition [14] is another interesting issue to be addressed. Acknowledgements This work was partly supported by Basic Research 21 for Breakthroughs in Info-communication Project of Japan Ministry of Posts and Telecommunications. References [1] C.A. Bouman, B. Liu, Multiple resolution segmentation of textured images, IEEE Trans. Pattern Anal. Mach. Intell. 13 (2) (1991) [2] C.A. Bouman, M. Shapiro, A multiscale random eld model for Bayesian image segmentation, IEEE Trans. Image Process. 3 (2) (1994) [3] S. Krishnamachari, R. Chellappa, Multiresolution Gauss Markov random eld models, Technical Report, University of Maryland, Collage Park, [4] M.L. Comer, E.J. Delp, Segmentation of textured images using a multiresolution Gaussian autoregressive model, IEEE Trans. Image Process. 8 (3) (1999) [5] J.E. Besag, On the statistical analysis of dirty pictures, J. Roy. Stat. Soc. B 48 (3) (1986) [6] P.J. Burt, E.H. Adelson, The Laplacian pyramid as a compact image mode, IEEE Trans. Commun. COMM-31 (1983) [7] C.K. Chui, An Introduction to Wavelets, Academic Press, Boston, [8] A.P. Dempster, N.M. Laird, D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Stat. Soc. 39 (1) (1977) [9] H. Noda, M.N. Shirazi, B. Zhang, E. Kawaguchi, Mean eld decomposition of a posteriori probability for MRF-based unsupervised textured image segmentation, Proceedings of the ICASSP 99, Vol. 6, 1999, pp [10] F. Spitzer, Markov random elds and Gibbs ensembles, Amer. Math. Mon. 78 (1971) [11] S. Geman, D. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6 (1984) [12] H. Noda, M.N. Shirazi, Data-driven segmentation of textured images using hierarchical Markov random elds, Trans. IEICE J77-D2 (5) (1994) (in Japanese. An English translated version is available in Systems and Computers in Japan 259(5) (1995) ). [13] P. Brodatz, Textures A Photographic Album for Artists and Designers, Dover, New York, [14] T. Chang, C.-C.J. Kuo, Texture analysis and classication with tree-structured wavelet transform, IEEE Trans. Image Process. 2 (4) (1993) [15] C.S. Won, H. Derin, Unsupervised segmentation of noisy and textured images using Markov random elds, CVGIP: Graphical Models Image Process. 54 (4) (1992) About the Author HIDEKI NODA received B.E. and M.E. in Electronics Engineering from Kyushu University, Japan, in 1973 and 1975, respectively, and Dr. Eng. degree in Electrical Engineering from Kyushu Institute of Technology, Japan, in He worked in Daini-Seikosha Ltd. from 1975 to 1978, in National Research Institute of Police Science, Japan National Police Agency from 1978 to 1989 and then in Communications Research Laboratory, Japan Ministry of Posts and Telecommunications from 1989 to In 1995, he moved to Kyushu Institute of Technology where he is now an associate professor in the Department of Electrical, Electronic & Computer Engineering. His research interests include speaker and speech recognition, image processing and neural networks. 7 Estimation of M is another important problem but it is beyond the scope of this paper. It might be possible by information-theoretic approaches [15]. 8 In this paper, images are assumed to be composed of multiple textures. Therefore non or less textured regions in real images are likely to be segmented into one region even though they dier in average gray level.

12 782 H. Noda et al. / Pattern Recognition 35 (2002) About the Author MAHDAD NOURI SHIRAZI was born in 1963 in Iran. He received his M.Sc and Ph.D degrees in Electrical Engineering from Tottori University and Kobe University, Japan, respectively. In 1993 he became a post-doctoral research fellow at the Communications Research Laboratory of Japan Ministry of Posts and Telecommunications, funded by the Japan Science and Technology Agency. Since 1995 he has been at the same laboratory, currently as a senior research scientist. His research interests include neural networks, pattern recognition, and image processing. About the Author EIJI KAWAGUCHI received the Dr. Eng. degree from the Department of Electronics Engineering at Kyushu University, Japan in Currently, he is a professor of Computer Engineering at Kyushu Institute of Technology. His research interests include speech recognition, pattern understanding, image processing and knowledge engineering as well as natural language understanding and semantic modeling.