v = (V,,v2,..., v, ) and the set of arc E c v X v, and a

Proceedings of the First International Conference on Machine Learning and Cybernetics, Beijing, 4-5 November 2002 TIME-VARIABLE-PARAMETRIC RELAXATION LABELING AND ITS APPLICATION IN TEXTURE SEGMENTATION YUN-TAO QIAN'~), QI WANG(=', CHING Y. SUEN'~) School of Computer Science, Zhejiang University, Hangzhou 310027, China (b) CENPARM1,Concordia University, Montreal, Canada E-MAIL: ytqian@mail.hz.zj.cn, suen@cenpami.concordia.ca Abstract: Relaxation labeling (RL) is a very important tool applied for exploitation of contextual information to reduce local ambiguity and achieve a global consistency. In image segmentation, RL is always used for post-processing the initial segmentation results, which is based on region or boundary information. However, few papers discuss the combination of region and boundary information in RL process. An timevariable-parametric RL (TW-RL) is proposed in this paper, which aims at incorporating boundary strength map into region based relaxation process by time-variable compatibility coemcient and neighboring size. This method can remove scattered false segmented regions and maintain real boundaries at the Same time. Compared with other region and boundary information hybrid methods, TW-RL is very simple and easy to be realized in computer. Its application to texture image segmentation is also discussed. Keywords: Relaxation labeling; Time-variable parameter; Texture segmentation; Region-based segmentation; Boundary strength map. 1 Introduction Relaxation labeling (RL) is a class of mechanisms designed to reduce labeling ambiguities and achieve a global consistency by taking local relationships in the object and label array into account. It has been applied successfully to a broader field such as edge post-processing, segmented region refinement, shape matching, scene labeling and optimization 11.2,3.41. It is widely recognized that RL is a simple and local computation method for accomplishing global, complex task, and its performance is strongly dependent on the application environment and the choice of updating rule and compatibility coefficients. In image segmentation, RL play an important role in improving the initial segmentation results, including region based RL, boundary based RL, or as a hybrid of the two. Region based segmentation is the process of dividing the given image into homogeneous regions with respect to certain features, and which is always completed with partitioning clustering such as fuzzy c-mean clustering algorithm 'I ' in histogram space. However, most of partitioning clustering methods do not take into account the spatial connectivity constraints, therefore many little scattered regions are produced. Like region segmentation, boundary detection also produces many short discontinuous edges owing to local gradient measure sensitive to noise [61, In order to remove and relabel these false regions or edges, RL enforces the local spatial constraints on the initial segmentation result, and produces an improved result with global consistency. Although many region or boundary based RL algorithms have been developed ".*', only few papers discuss how to combine region and boundary information in RL. But many works have proved that region and boundary information are mutually complementary, and their combination is an effective way to improve segmentation performance ['I. In this paper, a new RL algorithm, namely time-variableparametric RL (TVR-RL) is proposed. TVR-RL can be used for the combination of region and. boundary information in the process of region relabeling, which not only imposes spatial connectivity constraints on region relabeling, but also incorporates boundary information into this relaxation process. Different from the traditional RL, its compatibility coefficients are changed according to iteration time, and its updating process is divided several sub-processes with different sizes of neighborhood. Its application in texture image segmentation is also discussed in this paper. In the rest of this paper, first we review RL algorithm and its application in image segmentation. In section 3, the main idea and algorithm of TVP-RL are presented. Its application in texture image segmentation is discussed in section 4. Finally, the proposed algorithm is summarized, and the conclusions of our work are given in section 5. 2 A review on RL and its application in image segmentation The problem of RL can be summarized as: given a graph G = (V,E) with the set of nodes v = (V,,v2,..., v, ) and the set of arc E c v X v, and a setoflabelsh=(/1,,/2,,...,~~],everynodevi~v is associated with a membershiplprobabilistic set corresponding to all labels. RL selects a best consistency 0-7803-7508-4/02/$17.00 02002 IEEE 486

Proceedings of the First International Conference on Machine Learning and Cybernetics, Beijing, 4-5 November 2002 label among several possible choices for every node according to the compatibility function by iterative and parallel technique. In iterative procedure, the probability of the labeling of each node is being updated as time goes on, and its iterative probability updating equations can be written as [I]: and where p,!(a)'is the probability of the ith node being a member of the class /1 at the iteration t, and 4: (a) is a Support function that represents an influence on the labeling of the ith'node from its neighboring nodes. r-,(a,a') is the compatibility coefficient between the pair of ith and jth nodes, and R(i) is the neighborhood of the ith node. pn is an initial probabilistic labeling assignment. The final global consistency result is achieved after the relaxation process reached a stable state. In [4], a convergence theorem has been proved that in the stable state, each node has crisp label assignment p,(a) or has the same support function value qi (A) for all class labels. In image segmentation, RL is an effective method to improve the segmentation results that are derived from region or boundary based segmentation approacks. In region-based segmentation, the features of the pixels belonging to the same region are always different from each other due to noise and weakness of the feature extraction algorithm. Therefore, partitional clustering method without local spatial connectivity constraints always produces many little scattered false segmented regions. If every pixel has been assigned a probability set of class label, RL can update this initial probability set by introduction of spatial connectivity constraints "I. The compatibility coefficients of RL are derived from the spatial connectivity constraints, which can be determined by a priori knowledge, probabilistic distribution, or training data. Learning compatibility coefficients from training data is not suitable for our problem, because in practice the training data are difficult to obtain. The simplest method is based on a priori knowledge, but its precision is not good. Rosenfeld et al. [I] developed a method to use initial probabilities pn instead of real probabilistic distribution. This "nonleaming" method has been widely used in the computer vision domain. Its compatibility coefficients can be calculated as in which Then relaxation process can be completed by equations ( 1-2). For boundary based segmentation, RL is used to link up discontinuous boundary lines, and remove scattered short boundary lines. 3 TVP-RL Algorithm Obviously, in RL based segmentation, the affect between the two pixels is dependent on their distance and compatibility coefficients. If two pixels are far from each other and their compatibility coefficients are small, the affinity between them will be little, and vice versa. Therefore, the performance of RL based region relabeling is strongly dependent on the choice of neighboring size, if the neighboring size is large, the scattered small regions can he easily incorporated into their nearby large segmented regions, but the real boundaries of the segmented regions will he changed seriously; on the contrary, if the neighboring size is small, although the change of the boundary will be little, some of the scattered small regions can not be removed. We think that this dilemma arises from the space-invariable compatibility coefficients, i.e they depend only on the position of pixel j relative to pixel i, and not on their absolute positions. The fact is that the spatial connectivity constraints are not identical to all pixels, especially if two pixels come from different classes, spatial connectivity constraint does not exist between them, even though they are very close to each other. In order to remove scattered false segmented regions and maintain real boundary positions at the same time, besides spatial connectivity constraints, we should introduce boundary information. The simplest method is to redefine the compatitiility coefficients by 1 rg(a.1'), the path fmm ilo jdoes r'g (2.1') = not go through boundary. (6) 0, otherwise. If the real boundaries can be obtained and the neighboring size is large enough, the performance of this IU will be very good. Unfortunately in most cases the correct boundaries are very difficult to obtain by the existing boundary detection algorithms. To counteract the above, a TVP-RL algorithm is proposed. From the convergence theorem of RL, we know that the relaxation process generates two types of stable nodes whose probabilistic labeling set will not change. One type, of stable node means those nodes that have an unambiguous label, i.e. the node belongs to only one class, and the second type of stable node refers to those nodes whose support function values from other nodes' are identical no matter what label is given to those nodes. Obviously the second type of stable node occurs with very 487

Proceedmgs of the Fkst International Conference on Machine Learning and Cybernetics, Beijing, 4-5 November 2002. little probability, because if the compatibility coefficients or the labels of other pixels change, its stability will be broken. Therefore, we mainly discuss the first type of stable node. The speeds of approximating to the first type of stable node are not identical for all nodes, which are dependent on their initial probabilities, and compatibility coefficients. Although by now we can not give a precise analysis of the convergence speed, it is obvious that if the affinity between a node i and its neighbor is little, it will take more iterations to become an unambiguous label, because (1 + qi (1)) in equation (1) approximates 1 for all classes, and the change of probabilities of this node is very little after each iteration. Based on this fact, TVP-RL is proposed, which lets the different groups of nodes converge to fixed nodes at different times through building a special time-variant compatibility function. In image segmentation, it firstly makes those pixels with weak boundary strength approximate to the unambiguous label, then the pixels with medium boundary strength begin to approximate to the unambiguous label, finally all pixels converge to be unambiguous labels. In this process, at begin, the neighboring size can be given a large value and boundary constraint can be given a large weight in compatibility function, so that two pixels belonging to different classes have few opportunities to affect each other at the early stage, and in later period the neighboring size and the weight of boundary information become smaller, so that the real boundary positions can be remained. Before TVP-RL algorithm is given, we introduce the definition of layer in neighborhood. A neighboring size is assumed to be (2L + I)', the neighborhood of a pixel can be divided into L layers according to their distances with the central pixel, and the central pixel is defined as the Gth layer (Figure 1). I" I+ I i I I-iLq'C' 2 Layer 1 Layer 0 Figure 1: The definition of the layer in neighborhood. Different from the ordinary relaxation process in which the parameters (the compahbility coefficients and the size of neighborhood) are fixed throughout the iterative process, in TVP-RL algorithm, the compatibility coefficients are not only dependent on the spatial connectivity constraints and boundary strengths, but also dependent on the iteration time f. where the definition of r, (A, 1') is the same as equation (3). f(t) increases by one after every p iteration. f(t) = [tlpi (8) where "[I" represents integer operation. S' is derived from boundary strength map S that will be discussed in the next section. If a pixel j within the neighborhood of a pixel i has a little boundary strength, but the shortest path between i and j must pass a pixel with large boundary strength, it is possible that there exists a real boundary between them according to canny's or Mm's edge detection algorithm [61, and these two pixels i and j should have little affinity between them. So the s' should be [ S( j), j E layero, max(s'(j'),s(j)), je layerm, S'( j) = (9) and j'e layer (rn- 1) and j'is nearest to j. In TVP-RL process, the neighboring size (2L + 1)' is also decreased as the iteration time t increases L = Lo - f (r) (10) In this algorithm, we define a parameter p and the initial neighboring size as 10 and 17*17 respectively. 4 TVP-RL applied to texture image segmentation In order to verify TVP-RL algorithm, we do many experiments on texture image segmentation. Texture image segmentation plays an important role in many computer vision systems, ranging from geophysical surveying and aeriallsatellite image recognition to industrial and biomedical surface inspection, document image analysis and image retrieval. A lot of texture segmentation approaches have been reported in literature (e.g. see [IO]), but it is still an open problem, and to date only limited successes have been reached. The performance of unsupervised texture image segmentation is dependent on two factors: texture feature extraction and segmentation algorithm. To achieve a good segmentation result, the set of texture features must have a good discrimination power, and a segmentation algorithm must be able to synthesize all the useful segmentation information such as global and local statistics of the texture features, spatial constrains and non-stationary nature of the feature image. In our experiments, wavelet-statistical measures are used as texture features, Fuzzy C-means clustering algorithm (FCM) based region segmentation method and gradient based edge detection method are used to obtain initial region and boundary segmentation information, and TVP- RL is used for segmented region relabeling (Figure 2). 400

Proceedings of the First International Conference on Machine Learning and Cybernetics, Beijing, 4-5 November 2002 Inputimag obtain fuzzy clustering result. The final clustering result provides each pixel a fuzzy membership/probability set,u for all classes. Boundary strength map is chosen as boundary information, which is based on the first derivative of the feature images. Boundary information should have good performance under noisy conditions, to achieve this each feature image I is subjected to Gaussian smoothing, then the first derivative is computed. I'= (1 * G)'= I * G' G = exp(- * * x'+y' 202 1 (13) Figure 2 The flow chart of texture image segmentation Wavelet theory provides a precise and unified framework for spatial-frequency analysis. It attracts great interests from texture feature extraction, and many statistical texture features derived from wavelet representation have been proposed [lll. Here we propose a new set of directional wavelet-statistical texture features, which is based on first and second order statistical information: energy, variance, contrast and angular second moment, but compared with Wouwer's texture features [I 11, the directional information existed in wavelet representation is fully considered in our measures, i.e. we use different shapes of statistical window for different directional wavelet coefficients. The experiments will illustrate that our texture features have As we have more than one wavelet-statistical feature image, it is a problem on the gradient of multi-image. The boundary strength map S is derived from the average module of first derivative of all feature images. Our experiments aimed at comparisons between the performances of TVP-RL and ordinary RL on segmented region relabeling. In addition, we also do comparisons between the discrimination power of wavelet features and wavelet-statistical features. Here wavelet features refer to the coefficients of discrete wavelet frame. The original 300*300 texture image is shown in Figure 3(a), which consists of five texture patterns. From Figure 3(b-c), we find the segmentation result based on waveletstatistical features is better than that based on wavelet features. Figure 3(d-g) are ordinary RL results on figure 3(c) with 3*3, 7*7, 11*11 and 17*17 neighboring sizes respectively. The boundary strength map is shown in figure 3(h). Figure 3(i-n) are TVP-RL processes over iteration excellent discrimination power. numbers 10, 20, 30, 40 and 50 respectively ( p = 10 ), Having obtained the feature images, an initial regionbased segmentation be generatd. C-mem a boundary shength map can is chosen for region based segmentation, and its objective function can be defined as [5]: with probabilistic constraint: where C is the cluster number, v, is the central vector of the ith cluster.,!$ is membership which indicates the degree of vector xj belonging to the ith cluster, m is a constant that controls the fuzzy degree. There exists an iterative method to minimize the objective function, and which are better than those of ordinary RL algorithm. In addition, even at a specific neighboring size, the ordinary relaation result is good, how to determine this neighboring size is also a problem, It is obvious the oerformance of TVP-RL is deoendent on the initial segmentation result and boundary strength map, if. we assume that an ideal boundary map can be obtained (Figure 3(0)), TVP-RL can reach a correct segmentation result (Figure 3(p)). In order to evaluate our algorithm further, we take many test images from the Brodatz database, in which a cross shaped mask is used io superimpose one texture on left-top of another. The experiment results show the improvements brought by wavelet-statistical texture features instead of wavelet features range from 5.4% to 50%, and the improvements brought by TVP-RL instead of ordinary RL range from 2.3% to 27%. " 489

Proceedmgs of the First International Conference on Machine Learning and Cybernetics, Beijing, 4-5 November 2002 5 Conclusions In this paper, TVP-RL and its application in texture image segmentation are presented. Relaxation process is an important way to improve segmentation performance by imposing some local constraints, and in which spatial connectivity constraints are widely used. In the existing relaxation methods, compatibility coefficients between two pixels are only dependent on their relative rather than absolute positions, therefore in most cases they are not enough to produce a good segmentation result. Boundary extraction is based on the gray gradient of image, so it can compensate for spatial connectivity constraints. TVP-RL is proposed to deal with the combination of boundary strength map and spatial connectivity constraints. In TVP-RL, compatibility coefficients and neighboring size are changed with iteration time in relaxation process, and every pixel converges to a stable node at different speeds according to its initial state and neighboring environment. Compared with other region and boundary hybrid segmentation methods, TVP-RL is very simple and easy to be realized. TVP-RL performed very well in the experiments on texture segmentation, and provided better results than ordinary RL method. detection," IEEE Trans. Pattern Anal. Machine Intell., vo1.8, no.6, pp.679-698, 1986. [7] J.Y. Hsiao and A.A. Sawchuk, "Unsupervised textured image segmentation using feature smoothing and probabilistic relaxation," Computer Vision, Graphics, Image Processing, vo1.48, pp. 1-21, 1989. [81 P. Andrey and P. Tarroux, "Unsupervised segmentation of Markov random field modeled textured images using selectionist relaxation," IEEE Trans. Pattern Anal. Machine Intell.,) v01.20, no.3, pp.252-262, 1998. [91 J.F.Haddon and J.F.Boyce, "Image segmentation by unifying region and boundary information," IEEE Trans. Pattern Anal. Machine Intell., v01.12, no.9, pp.929-949, 1990. [IO] T.R. Reed and J.M.H. Buf, "A review of recent texture segmentation and feature extraction techniques," Image Understanding, vo1.57, no.3, pp.359-372, 1993. [I 11 G. Van de Wouwer, P. Scheunders, and D. Van Dyck,. "Statistical texture characterization from discrete wavelet representations," IEEE Trans. Image Processing, vo1.8, no.4, pp.592-598, 1999. Acknowledgements This work was supported in part by National Natural Science Foundation of China under Grant 60103018. References [I] A. Rosenfeld, R.A. Hummel, and S.W. Zucker, "Scene labeling by relaxation operations," IEEE Trans. Syst., Man, Cybern., vo1.6, pp.420-433, 1976. [2] M. Pelillo and M. Refice, "Learning compatibility coefficients for relaxation labeling processes," IEEE Trans. Pattern Anal. Machine Intell., vol.16, no.9, pp.933-945, 1994. [31 R.A. Hummel and S. W. Zucker, "On the foundations of relaxation labeling processes," IEEE Trans. Pattern Anal. Machine Intell., vo1.5, pp.267-287, 1983. [4] S.W. Zucker, "Relaxation processes for scene labeling convergence, speed and stability," IEEE Trans. Syst., Man,Cyhern.,voI.8, pp.41-48, 1978. [5] J.C. Bezdek. Pattern Recognition With Fuzzy Objective Function Algorithms. Plenum Press, New York, 1981. [6] I. Canny, "A computational approach to edge 490

Proceedrngs of the First Internaiional Conference on Machine Laming and Cybernetics, Beiiing, 45 November 2002 (4 In]. t U,) ip'l.. Figure 3: (a) original image, (b-c) fuzzy C-mean clustering based segmentation with wavelet and waveletstatistical features respectively, (d-g)'ordinaty RL results with 3*3, 7*7, 11*11 and 17*17 neighboring sizes respectively, (h) boundaty strength map, (i-n) TVP-RL process over iteration number 10, 20, 30, 40 and 50 respectively, (0) ideal boundary map, (p) TVP-RL result with ideal bounday. 491