; Robust Clustering Based on Global Data Distribution and Local Connectivity Matrix

Transcription

1 1997 IEEE International Conference on Intelligent Processing Systems October 28-3 I, Beijing, China Robust Clustering Based on Global Data Distribution and Local Connectivity Matrix Yun-tao Qian Dept. of Computer Sci. & Eng. Northwestern Polytechnical University Xi'an 717:!, P.R.China Rong-chun Zhao. Dept. of Computer Sci. & Eng. Northwestern Polytechnical University Xi'an 7172, P.R.China Abstract - A new method to clustering analysis, which is based on integration of graph theoretical method and fuzzy objective fimction algorithm, is developed. The connectivity mritriv derived from fuzzy limited neighborhood graph and the measurement for similarity and dissimilarity are utilized to build a new fuzzy objective function that iinifles global data distribution and local spatial information. In some sense, both the traditional graph theoretical method and objective function algorithm are special cases of our algorithm. I. INTRODUCTION CIustering is an important tool of data analysis, which has been widely used in data compression, pattern recognition, economics and other xience fields. Clustering analysis attempts to partition the data points into subgroups according to their properties without prior knowledge and training examples, but, if nothing is known about data, the problem is not solvable. Various clustering approaches present different knowledges concerning the prototype or partition criterion, which can be classified into two categories in general a) local information based methods including hierarchicall clustering algorithms [ 31 and graph theoretic techniques [4] 161, and b) prototype of data distribution based methods including object function algorithm [I] and statistics method [2]. In the first class of approach, clustering i:3 realized at a local level, and the partition criterion is based on the local information such as spatial relation between points and the local area statistics. These methods are flexiable and non-parameter, which prepurpose very little assumption on data characteristics. However they run into trouble when the clusters are close together and the boundaries are indistinct, and they are sensitive to the random noise. Differ from the local information based method, the second class of approach completes clustering in a whoie feature space. Since they are based on the assumption on prototype of data distribution, if the data distribution is not conformable to the assumped prototype, they will become less effective. For example, Fig.1 includes four data structures. fuzzy c-mean algorithm (FCM) that is a common objective function method is suitable for Fig.l(a) and (b), but (c) and (d) are not processed with FCM; on the contrary, graph theoretical method can produces a successful clustering results for fig.l(c)(d), but has some of difficults for figl.(a)(b). As these two classes of approaches consider the local and global information of data points respectively, they are compensative rather than competitive. In this paper, we propose an algorithm to unify the fuzzy objective function method and graph theoretical method, which utilizes the global data distribution and local connectivity matrix. 6 OO oo G o," :oo."o 8 O " io % o o o OB :do ' ~ ~ $ 8 "v O oqooooo oodo (c) (4 Fig.1 Four data structures used for clustering ; /97/$1. 1!197 IEEE

2 11. THE ALGORITHM Our algorithm is based on global data distribution and local connectivity matrix. At first, the connectivity matrix is obtained by graph theoretical method, then a new objective function is built which incorporates the connectivity matrix into its parameters, at the same time, the assumption on global data distribution is also reflected in this objective function. This method is an improvement on the traditional objective function algorithms that only consider the global data distribution, but not consider the local information. To facilitate the discussion, at first, fuzzy objective function algorithm and graph theoretical method are introduced briefly, furthermore, a fuzzy graph used for describing the connective relation is also presented. A. Fuzzy Objective Function Algorithm Objective function based clustering is dependent on the definition of similarity and dissimilarity with distance measurement. The data set is divided into subsets according to their similarities and dissimilarities, the data points belonging to the same subset are close to each other in a specific measurement, whereas those belonging to the different subset are far from each other. After the objective function is derived from distance measurement, the goal of clustering analysis is to minimize the objective function, and the clustering problem is translated into optimal programming problem. Various distance measurements determine the forms of clustering, and now people have proposed many objecti9e function algorithms such as line clustering, shell clustering and ellipsoid clustering, however objective function algorithm is very sensitive to the data distribution. Recently, there has been an increasing use of fuzzy set theory for clustering, which partitions a data set into fuzzy subsets that have not precise boundaries, and the range of membership values is [O, 11, which differ from the crisp clustering in which the value of memberships is (,l). In general, fuzzy objective function is defined as 111: c N C cp;j = 1, j = 1,..., N. i=l where c is the number of clusters, D(zj,H;) is the distance from point xi to the cluster kernel H;, H;, is membership which indicates the degree of point zj belonging to cluster i, and m is a constant that controls the fuzzy degree. B. fizzy Graph Theoretical Method The key problem of graph theoretical clustering is to translate the data set into non-orientation graph G = (V, E), V = (VI,...,UN) is a set of vertices that corresponts to the data set, and E = (el,..., em) is a set of distinct edges, and each edge e, = (z,, z3) is a pair of vertices. There are many algorithms including Voronoi diagram, relative neighborfood graph, Gabriel graph and p skeletons for determining the edge set [5]. Here Gabriel graph is selected as base for building non-orientation graph. At firsr, definition of Gabriel influence region is given as [5]: P+q d(p,q) rp,q = B (-p 7) (2).. d(z, Y) = Iz -.> Yl B(z, r) = {Y : +, Y) 5 (3) (4) then the edge set E can be computed by: (p, q) E E, if and only if rp,q n V = (5) Although Gabriel graph is effective in some clustering problem, this graph only considers the information of region between two vertices, does not consider the information of regions around the two vertices. Urquhart proposed a modified Gabriel graph whose influence region is defined as [SI: (6) where a is a constant, CY =.25 is choosen in our experiments. Then by (5) edge set can be obtained, a connectivity matrix RN~N can be also derived from edge set and vertice set, and the value of its element is determined by: 1, if (x~, xi) E E; = {, otherwise. (7) As the value of element is 1 or, we call this nonorientation graph as crisp graph. Crisp graph will lose much useful information, and it is not suitable for ambiguous and noise environment, although it is very simple. In this paper, a fuzzy edge set is presented, in which each pair of vertices have a edge with fuzzy membership that is an indicator of connectivity strength. Thus the value of element in connectivity matrix is redefined as: n L ai = e 8J

3 A,, is the number of points in influence region I n V. X is a constant that control the fuzzy degree, if X -+, the fuzzy graph degenerates to crisp graph. In graph theoretical methods, connectivity matrix is only source of information used for clustering, but in our method, the fuzzy connectivity matrix will be incorporated into fuzzy objective function in the following processing as local information of data. C. Robust Algorithm In order to overcome the drawbacks of objective function algorithm and graph theoretical approach, a new fuzzy objective function is presented which includes global data distribution information and fuzzy connectivity matrix. The fuzzy objective function is given as: subsets. The performance measurement for cluster validity is common used to estimate the optimal number of subsets. In this paper, normalized partition entropy is selected as performance measurement, which is defined as [I]: P(U1 c) = F(U, c)/ll - (c/n)i ( 1) N c F(u,c) = -xxk tklogo(k tk)/n (11) k=l i=l where e is the number of subsets, and a is logarithmic base. If (U*,C*) is the solution of the following problem: min P(v,c) 2<c<N c* is considered as optimal number of subsets. e N 111. EXPERIMENTAL RESULTS. In all experiments, global data distribution is supposed spherical distribution, so the distance measurement in (1) and (9) is defined as the distance from the data point tb the center of cluster: a=1 where rlk is the element of fuzzy connectivity matrix R, and cy1 is a constant that determines the importance of the local information, the larger the value of al, the more important the local information. And third term on the right side is employed to avoid generating ordinary solution, where 81 is a positive constant. The remained work is to comput the fuzzy membership /.L,~ by minimizing this o bjective function (9), which is a constrained nonlinear optimal problem. There exist many methods to solve this problem. Here, multiplier methods is employed, which has been proved superiority over ordinary penalty methodls in convergence properties for constrained minimization 171. Especially they are suitable for multi-dimensional problems with many nonlinear contraints such as our problem, where gradient projection methods and the reduced gradient method, or Newton and quasi-newton versions of them, may encounter difficulties due to large dimensionality. The people who are not familiar to multiplier met,hods can be refered to [8]. D. Cluster Validity For objective function method, the key obstacle to be overcome is the determination of the suitable number of U; is the center of ith cluster. Fig.S(a) is the iris data set that has three subsets, two of which are overlapping, and three subsets are shown in fig.2(b) with different symbols. Fig.2(c) is our clustering result with c = 3 that is optimal cluster number determined by performance measurement function shown in fig.2(d). It can be found that all points can be correctly classified except few points in overlaping region, and the normalized partition entropy for cluster validity is effective in selecting the optimal number of subgroups. But irii data can not be processed correctly with graph theoretical method. Fig.3(a) shows an two dimensional data set drawn from two Gaussian distributions with different density. The results of FCM and our method are shown in fig3.(b) and (c) respectively. In fig3.(b), some data points belonging to the low density cluster are misclassified as belgnging to high density cluster. Fig.4(a) shows a data set including two different distribution. FCM algorithm generates incorrect result shown in fig.li(b), but our algorithm can yield a correct solution shown in fig.4(c). Through above experiments, it is demonstrated that our algorithm has an advantage over the single information based approaches.

4 I n OO O O b S Number of clusters (4 Fig.2. Iris data is used for clustering analysis. (a) Iris data. (b) Three subsets of iris data is shown with different symbols, two subsets have overlapping region. (c) Clustering using our robust algorithm with 3 clusters. (d) The performance measurement as a function of the number of clusters in data. Fig.3. Synthetic data is used for clustering which includes two Gaussian distributions with different parameters. (a) Synthetic data. (b) Clustering using FCM algorithm. (c) Clustering using our robust algorithm

5 IV. Oc&G In this paper, a new fuzzy clustering based on integration of global data distribution and local connectivity matrix is developed, which overcomes the drawbacks of the traditional objective function algorithm and graph theoretical method, and improves the robust of clustering in complicated situations. In some sense, both the graph theoretical method and objective function algorithm are special cases of our algorithm. As the clustering problem is translated into a constrained nonlinear optimal problem in our method, the computing time will become very long when the large data are processed. In general, random sampling can be used to reduce the size of data, moreover, the quality of clustering will not be affected seriously in most cases. Since our algorithm includes the local spatial information, it can be applied to clustering based image segmentation. Multithresholding image segmentation based on our algorithm is in progress. O 8 V. REFERENCES [I] J.C.Bezdek, Pattern Recognition With Fuzzy Objective Function AIgorithms, Plenum Press, New York, R.Wilson and M.Spann, A new approach to clustering, Pattern Recognition, vo1.23, no.12, 199, pp [3] F.Murtagh, Survey of recent advances in hierarchical clustering algorithms, The Computer Journal, vo1.26, no.4, 1983, pp C.T.Zahn, Graph-theoretic methods for detecting and describing gestalt clusters, IEEE %u~s. Computer, v1.2, 1971, pp (51 J.W.Jaromczyk and G.T.Toussaint, Relative neighborhood graphs and their relatives, Proc. IEEE, vo1.8, n.9, 1992, pp [SI R.Urquhart, Graph theoretical clustering based on limited neighborhood sets*, Pattern Recognition, vo1.15, no.3, 1982, pp D.P.Bertsekas, Multiplier method: A survey, Automatica, v1.12, 1976, pp Fig.4. Synthetic data is used for clustering which includes two different distributions. (a) Synthetic data. (b) Clustering using [8] M.R.Hestenes, Multiplier and gradient methods, J. FCM algorithm. (c) Clustering using our robust algorithm. Opt Theory Appl., vo1.4, 1969, pp