Cluster Center Initialization Method for K-means Algorithm Over Data Sets with Two Clusters

Transcription

1 Available online at Proceia Engineering 4 (011 ) International Conference on Avances in Engineering Cluster Center Initialization Metho for K-means Algorithm Over Data Sets with Two Clusters Chun Sheng Li * Department of Mathematics an Computational Science Guang Dong University of Business Stuies Guangzhou China Abstract This paper efines nearest neighbor pair an puts forwar four assumptions about nearest neighbor pairs base on which a center initialization metho for K-means algorithm over ata sets with two clusters is buil. Experiments on real ata sets show that the propose metho is not preferable but at least comparable to the ones in literatures. The contribution of the propose metho is to open up a new approach to evising center initialization metho for K-means algorithm. 011 Publishe by Elsevier Lt. Open access uner CC BY-NC-ND license. Selection an/or peer-review uner responsibility of ICAE011. Key wors: cluster center initialization k-means algorithm nearest neighbor rule 1. Introuction Clustering is an unsupervise pattern classification metho that ivies a set of given ata into clusters such that ata in the same group are more similar to each other than to ata from ifferent clusters. Clustering is one of the most important tasks in ata analysis. It has been use for ecaes in image processing an pattern recognition. Many clustering algorithms were propose to perform a partition on the given ata set. The goal of any clustering algorithm is to reveal the natural structure of the ata set. Crisp k-means algorithm [1] is a partitioning clustering metho that ivies ata into k mutually excessive groups. It is simple an fast to run an most popular with researchers an practitioners. However Crisp k-means algorithm is very sensitive to the initial cluster centers that have a irect impact on the formation of final clusters. Crisp k- Means algorithm may be strappe in a local optimum or generates an empty cluster for the given ata set because of inappropriate initial cluster centers. Therefore it is quite important to provie k-means algorithm with goo initial cluster centers. * Corresponing author. Tel.: aress: lcs581084@sina.com Publishe by Elsevier Lt. oi: /j.proeng Open access uner CC BY-NC-ND license.

2 Chun Sheng Li / Proceia Engineering 4 ( 011 ) Several methos propose to solve the cluster initialization for k-means algorithm. The common characteristics of the methos in literatures are that they try to search proper initial cluster centers in any ata sets with ifferent number of clusters. This is only a goo ream as shown by applications an comparative researches [ 3]. There may be many ifferences among ata sets with ifferent number of clusters. A uniform solution to initial cluster center for any ata sets seems impossible. In contrast a class of ata sets with the same number of clusters may have some common characteristics which may contribute to searching proper initial clusters. Base on this assumption this paper evotes to searching initial clusters especially in ata sets with two clusters. The propose metho may be a little reasonable but it opens up a new approach to initial cluster centers.. Center initialization metho base on nearest neighbor groups Supposing that X x x is a ata set where x x x x T 1 between x j an x k enote by (x j x k ) is efine as x n T x x x x x x j k j k j k (1) For any ata point x in X its nearest neighbor x in X is efines as x x x y arg min yx j 1 j j mj.the issimilarity () Each atum point x in X an its nearest neighbor x constitute a pair of ata points (x x ) which is calle a nearest neighbor pair. All the nearest neighbor pairs compose a set x x is calle a set of nearest neighbor pairs. The issimilarity between two nearest neighbor pairs (x x ) an (y y ) is efine as x x y y min a ba x x b y y (3) B x X which Assumption I for any point x in X x an its nearest neighbor x are either in the same cluster or on the overlapping of two clusters. Assumption II the more issimilar the nearest neighbor pair (x x ) is with (y y ) the more probable they are in ifferent clusters. For a ata set of two clusters we evote to search two nearest neighbor pairs that are in ifferent clusters an very issimilar with each other. To reach this objective the nearest neighbor point of each atum point x in X is searche an the nearest neighbor pairs form a set B. Assumption II inicates that two nearest neighbor pairs that are most issimilar are more probable to be in ifferent clusters. So we search two nearest neighbor pairs in B that are most issimilar an enote them by x 1 an x where x i represents the nearest neighbor pair (x i x i ) i=1 an satisfies Equ.(4). x x x x max x x y y x x B y y 1 1 B (4) They are most probable to be in ifferent clusters an selecte as two initial clusters. However assumption I inicates that x i an x i i=1 may be on the overlapping of two clusters. When this case happens they can not be selecte as the initial cluster centers an we have to search another two nearest neighbor pairs in B that are most issimilar with each other an (x i x i ) i=1. We enote them by x 3 an x 4 where x i represents the nearest neighbor pair (x i x i ) i=34. x 3 an x 4 satisfy the following equations. x 3 arg max y y xi xi y y x x y y 1 1 x x (5) y y B i1

3 36 Chun Sheng Li / Proceia Engineering 4 ( 011 ) arg max y y y y x x y y x x 1 1 y y B Since x x x x x x x x hols (x 3 x 3 ) an (x 4 x 4 ) may be in ifferent clusters but not certain to be so. When they are in ifferent clusters we can select them as initial cluster centers. If they are in the same cluster only one of them can be selecte as the initial cluster center. By now we obtaine four nearest neighbor pairs (x i x i ) i=1 34. Among them (x 1 x 1 ) an (x x ) may be in ifferent clusters an selecte as initial clusters but not certain to be so so o (x 3 x 3 ) an (x 4 x 4 ). We have to evise a scheme to pick out the initial cluster centers from (x i x i ) i=1 34. To o so we introuce another nearest neighbor pair (x 5 x 5 ) that satisfies Equ.(7). x arg min y y B y y x x 4 5 i i1 i It is probable that (x 5 x 5 ) is on the overlapping of two clusters. With the help of (x 5 x 5 ) either (x 1 x 1 ) (x 5 x 5 ) an (x x ) or (x 3 x 3 ) (x 5 x 5 ) an (x 4 x 4 ) can be treate as the borerline between two clusters. If (x 1 x 1 ) (x 5 x 5 ) an (x x ) are treate as the borerline seen in Fig.1(a) (x 3 x 3 ) an (x 4 x 4 ) are in ifferent clusters an can be selecte as the initial cluster centers. If (x 3 x 3 ) (x 5 x 5 ) an (x 4 x 4 ) are treate as the borerline seen in Fig.1(b) (x 1 x 1 ) an (x x ) can be selecte as the initial cluster centers. Another case can not be neglecte that is when (x 3 x 3 ) an (x 4 x 4 ) are in the same cluster seen in Fig.1(c) only one of (x 3 x 3 ) an (x 4 x 4 ) an one of (x 1 x 1 ) an (x x ) can be selecte as the initial cluster centers. (7) (6) x x x (a) (b) (c) Fig.1 the borerlines of clusters (x i represents (x i x i )) For a ata set of two clusters which of the above three cases is suitable for it? Consiering the complexity of ata structure we only put forwar the following assumptions that can not be proofe. Assumption III if C(x -x 5 -x 3 -x 4 -x )<C(x -x 3 -x 5 -x 4 -x ) an C(x 1 -x 5 -x 3 -x 4 -x 1 )<C(x 1 -x 3 -x 5 -x 4 -x 1 ) hol where C(x -x 5 -x 3 -x 4 -x )=(x x 5 )+(x 5 x 3 )+(x 3 x 4 )+(x 4 x ) others are efine similarly the close curves x 1 -x 5 -x -x 1 an x 3 -x 5 -x 4 -x 3 are the borerlines of two clusters respectively (seen in Fig.1(c)). Two v v max x y x x x y x x. initial cluster centers v 1 an v are selecte by Otherwise the close curves x 1 -x 3 -x 5 -x 4 -x 1 an x -x 3 -x 5 -x 4 -x or x 3 -x -x 5 -x 1 -x 3 an x 4 -x -x 5 -x 1 -x 4 (seen in Fig.1(a)-(b)) are the borerlines of two clusters respectively Assumption IV when the close curves x 1 -x 5 -x -x 1 an x 3 -x 5 -x 4 -x 3 are not the borerlines of two clusters at all if C(x 1 -x 3 -x 5 -x 4 -x 1 )+ C(x -x 3 -x 5 -x 4 -x C(x 3 -x -x 5 -x 1 -x 3 )+ C(x 4 -x -x 5 -x 1 -x 4 ) hol then x 1 an x are in ifferent clusters an selecte as the initial cluster centers (seen in Fig.1(a)). Otherwise x 3 an x 4 are in ifferent clusters an selecte as the initial cluster centers (seen in Fig.1(b)). Base on the above analysis the propose cluster center initialization metho for K-means algorithm over ata sets of two clusters can be given in Algorithm I. we enote it by CIT. 4

4 Chun Sheng Li / Proceia Engineering 4 ( 011 ) Algorithm I center initialization algorithm for K-means algorithm over ata sets of two clusters (CIT) 1 compute the issimilarity between any pair of ata points in X using formula (1); for any atum point x in X fin its nearest neighbor x using formulae () an constitute a set B of nearest neighbor pairs; 3 fin two most issimilar nearest neighbor pairs (x 1 x 1 ) an (x x ) using formulae (3)-(4); 4 fin the thir most issimilar nearest neighbor pairs (x 3 x 3 ) using formula (5); 5 fin the fourth most issimilar nearest neighbor pairs (x 4 x 4 ) using formula (6); 6 fin the nearest neighbor pair (x 5 x 5 ) on the overlapping of two clusters using formula (7); 7 select two initial cluster centers accoring to assumptions III-IV. 3. experiments Three real ata sets of two clusters are employe to test the propose cluster center initialization metho CIT. Their brief escriptions are given in Table I. Table I brief escription of ata sets Data set Number of Number of Number of monks_ [6] Ionosphere [6] Liver-isorer [6] attributes ata clusters To emonstrate the effectiveness of CIT it was compare with two existe cluster center initialization methos enote by k-tree [4] an CCIA [5] respectively. Two inexes pattern recognition rate of HCM (har c-means)[1] whose initial cluster centers are from each cluster center initialization metho an CPU time of each cluster center initialization metho are employe to evaluate the cluster center initialization methos. The comparative results are liste in Table II that shows CIT is comparable to ktree an CCIA in terms of pattern recognition rate of HCM [1]. In terms of CPU time CIT is a little computationally expensive than CCIA an k-tree in case of small size an low imensional ata sets. Although CIT is not preferable to k-tree an CCIA it is still worth investigating for it opens up a new approach to evising cluster center initialization metho. Table II comparative results on real ata sets Data set monks_ Ionosphere Liverisorer 4. Conclusions an iscussions Pattern recognition rate of HCM (%) CIT k-tree CCIA CPU time (secon) CIT k-tree CCIA This paper gives the efinition of nearest neighbor pair an four assumptions base on which a new cluster center initialization metho for HCM over ata sets of two clusters enote by CIT is built. The propose cluster center initialization metho CIT evotes to searching two nearest neighbor pairs that are most issimilar an in ifferent clusters but not on the overlapping of two clusters. The means of each searche nearest neighbor pairs are selecte as two initial cluster centers. Experiments on real ata sets show that CIT is comparable to k-tree an CCIA in terms of pattern recognition rate of HCM an CPU

5 38 Chun Sheng Li / Proceia Engineering 4 ( 011 ) time. Although CIT oes not prefer to center initialization methos in literatures it is still valuable for it opens up a new approach to evising cluster center initialization methos. Acknowlegements This paper is supporte by the program of Guang Dong University of Business Stuies grante by 10BS References [1] MacQueen J.B Some methos for classification an analysis of multivariate observation. In: Le Cam L.M. Neyman J. (Es.) Berkeley Symposium on Mathematical Statistics an Probability. University of California Press pp [] Pena J.M. Lozano J.A. Larranaga P An empirical comparison of four initialization methos for the k-means algorithm. Pattern Recognition Lett. 0 (10) [3] Douglas Steinley Michael J. Brusco 007 Initializing K-means Batch Clustering: A Critical Evaluation of Several Techniques Journal of Classification 4 (1) [4] Stephen J. Remon Conor Heneghan 007 A metho for initialising the K-means clustering algorithm using k-trees Pattern Recognition Letters 8 (8) [5] Khan S. S. Ahma A. 004 Cluster center initialization algorithm for k-means clustering Pattern Recognition Letter 5 (11) [6]The UCI Machine Learning Repository