A hybrid sequential approach for data clustering using K-Means and particle swarm optimization algorithm

Transcription

1 MultCraft Internatonal Journal of Engneerng, Scence and Technology Vol., No. 6, 00, pp INTERNATIONAL JOURNAL OF ENGINEERING, SCIENCE AND TECHNOLOGY 00 MultCraft Lmted. All rghts reserved A hybrd sequental approach for data clusterng usng K-Means and partcle swarm optmzaton algorthm Sandeep Rana *, Sanjay Jasola, Rajesh Kumar * School of ICT, Gautam Buddha Unversty, Greater Noda, INDIA Department of Electrcal and Computer Engneerng, Natonal Unversty of Sngapore, SINGAPORE * Correspondng Author: e-mal: srana.t@gmal.com, Tel , Fax Abstract Clusterng s a wdely used technque of fndng nterestng patterns resdng n the dataset that are not obvously known. The K-Means algorthm s the most commonly used parttoned clusterng algorthm because t can be easly mplemented and s the most effcent n terms of the executon tme. However, due to ts senstveness to ntal partton t can only generate a local optmal soluton. Partcle Swarm Optmzaton (PSO) technque offers a globalzed search methodology but suffers from slow convergence near optmal soluton. In ths paper, we present a new Hybrd Sequental clusterng approach, whch uses PSO n sequence wth K-Means algorthm for data clusterng. The proposed approach overcomes drawbacks of both algorthms, mproves clusterng and avods beng trapped n a local optmal soluton. Experments on four knds of data sets have been conducted. The obtaned results are compared wth K-Means, PSO, Hybrd, K-Means+Genetc Algorthm and t has been found that the proposed algorthm generates more accurate, robust and better clusterng results. Keywords: Cluster Centrod, Global Optmzaton, K-Means clusterng, Partcle Swarm Optmzaton (PSO). Introducton In general, clusterng nvolves parttonng of a gven multdmensonal data vector set nto subsets based on the closeness or smlarty among the data of same knd (Mtra and Acharya, 004). Clusterng algorthms have been used n data mnng and machne learnng wth many applcatons arsng from a wde range of problems, ncludng exploratory data analyss, mage segmentaton, securty, medcal mage analyss (Zhang and Chen, 004), web handlng and mathematcal programmng (Pyle, 999), (Panov et al., 008). Owng to the huge amount of data collected n databases, cluster analyss has recently become a hghly actve area of research. Clusterng has been defned as the process of groupng a data set n a way that the smlarty between data wthn a cluster s maxmzed whle the smlarty between data of dfferent clusters s mnmzed (Rajan and Saravanan, 008), (Xndong, 004), so the clusterng algorthms have to focus on the n-house groupng based on certan crtera. The research n ths area has focused on fndng an effcent, fast and effectve cluster analyss algorthm to handle large databases. Most clusterng algorthms belong to two groups: herarchcal clusterng and parttoned clusterng. The herarchcal approach produces a nested seres of parttons consstng of clusters ether dsjont or ncluded one nto the other. In herarchcal clusterng, an objectve functon s used locally as the mergng or splttng crteron. In general, herarchcal algorthms cannot provde optmal parttons for ther crteron. In contrast, parttoned methods assume the gven number of clusters to be found and then look for the optmal parttons based on the object functon (Jan et al., 999). However, n many applcatons, herarchcal approaches are unpractcal for clusterng. In such crcumstances, the parttoned clusterng approach whch drectly mnmzes the sum of squares dstance s more applaudable. The tradtonal way to deal wth such problems s to use some heurstcs such as the well-known K-Means algorthm (Zalk, 008). The K-Means algorthm s one of the most popular methods for clusterng multvarate quanttatve data (Tsa and Chu, 008). It s a method commonly used to automatcally partton a data set nto k groups. K-Means algorthm generates a fast and effcent soluton. The basc K-Means algorthm works wth the objectve to mnmze the mean squared dstance from each data pont to ts nearest centre. There are no effcent solutons known to any of these problems and some formulatons are NP-hard. The use of classcal optmzaton methods suffers from the problem of stckng to local mnma, also the ntalzaton of classcal methods s

2 68 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp another mportant ssue. These two drawbacks are also present n the K-Means algorthm and hence the cluster result s senstve to the selecton of the ntal cluster centrods and converges to the local optma. Therefore, the ntal selecton of the cluster centrods decdes the man processng of K-Means algorthm and the partton result of the dataset as well. The K-Means algorthm searches the local optmal soluton n the vcnty of the ntal soluton to refne the partton result. An approxmaton algorthm for solvng clusterng problem wth arbtrary dmensons was proposed (Kumar et al., 00). A flterng algorthm based on kd-tree ncreased the speed of clusterng process (Kanungo et al., 00). Local approxmaton based heurstc was used for K-Means clusterng and proved t through an emprcal study (Kanungo et al., 00). However, f good ntal clusterng centrods can be obtaned usng any of the other technques, the K-Means would work well n refnng the clusterng centrods to fnd the optmal clusterng centers. The same dea s proposed n ths paper to determne ntal ponts for K-Means algorthm by some other global optmzaton search algorthms. Evolutonary and bo-nspred algorthms eradcate some of the above mentoned dffcultes and are quckly replacng the classcal methods n solvng practcal problems (Chen and Fun, 004). The Partcle Swarm Optmzaton (PSO) s one of the nature-nspred populaton based stochastc optmzaton algorthms. It s a Swarm Intellgence (SI) technque based on the observatons of the collectve behavor n decentralzed and self-organzed systems (Kennedy and Eberhart, 995). Its examples can be found n nature, ncludng bee colones, ant colones, brd flockng, anmal herdng, bactera modelng and fsh schoolng (Kennedy et al., 00). The partcles search locally but the nteracton wth each other leads to the emergence of global behavor (El-abd and Kamal, 005). The PSO algorthm can be used to generate good ntal cluster centrods for the K-Means. In ths paper, we present a hybrd sequental clusterng approach that can avod beng trapped n a local optmal soluton. Ths paper s organzed as follows. The K-Means algorthm s most commonly used algorthm because of ts ease of mplementaton. Secton detals the workng of K-Means algorthm and also descrbes major drawbacks whch are to be rectfed. Secton 3 detals the standard PSO and the related ssues about accuracy and convergence to optmal solutons. Secton 4 descrbes the basc requrements of sequental clusterng approach. The development and workng of the approach s elaborated n the secton 4. Secton 5 dscusses smulaton and expermental results made on some standard test systems and draws nferences on the cluster formaton from the results obtaned. Fnally, secton 6 concludes the paper.. The K-Means Clusterng Algorthm Developed between 975 and 977 by J. A. Hartgan and M. A. Wong, K-Means clusterng s one of the older predctve modelng methods (Mtra and Acharya, 004). In K-Means clusterng a set of n observatons n d-dmensonal space (an nteger d) s gven and the problem s to determne a set of c ponts to mnmze the mean squared dstance from each data pont to ts nearest center wth whch each observaton belongs. No exact polynomal-tme algorthms are known for ths problem. The problem can be set up as an nteger programmng problem but because solvng nteger programs wth a large number of varables s tme consumng, clusters are often computed usng a fast, heurstc method that generally produces good (but not necessarly optmal) solutons (Jan et al., 999). The K-Means algorthm s one such method where clusterng requres less efforts. In the begnnng, number of cluster c s determned and the centre of these clusters s assumed. Any random objects as the ntal centrods can be taken or the frst k objects n sequence can also serve as the ntal centrods. n Gven a set of observatons ( x, x,..., x ), where each observaton s a d-dmensonal real vector, then K-Means algorthm clusterng ams to partton the n observatons nto c sets ( c < n) as Z = ( z, z,..., z c ) to mnmze a measure of dsperson wthn the clusters. The standard K-Means algorthm mnmzes the wthn-cluster sum of squares dstance accordng to the equaton () gven below. j f = arg mn( X µ ) z c j= j X Z () j j where µ s the mean of Z. There are two ssues n creatng a K-Means clusterng algorthm: the optmal number of cluster and the centre of cluster. In many cases, number of cluster s gven then the mportant part s where to put cluster centre so that scattered ponts can be grouped properly. Centre of cluster can be obtaned by frst assgnng any random pont and then optmzng the mean dstance as gven n equaton (). The process s repeated untl all the centre postons are optmzed. The drawback of standard clusterng algorthm s that they gnore measurement errors, or uncertanty, assocated wth the data. If these errors exst, then these can play a sgnfcant role n decdng clusters and cluster centers. In general, the algorthm does not acheve a global mnmum of f over the assgnments. In fact, snce the algorthm uses dscrete assgnment rather than a set of contnuous parameters, the "mnmum" t reaches cannot even be properly called a local mnmum (Cu et al., 005). Despte these drawbacks, the algorthm s used farly frequently because of ts ease of mplementaton (Tsa and Chu, 008). The result of K- Means algorthm s hghly dependent upon ts ntal selecton of cluster centers and before clusterng t must be prevously known and fxed.

3 69 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp The Partcle Swarm Optmzaton Algorthm PSO explots a populaton of ndvduals to probe promsng regons of the search space. In analogy wth evolutonary computaton methods, a swarm s smlar to populaton and a partcle s smlar to an ndvdual. PSO follows a stochastc optmzaton method based on Swarm Intellgence (VenderMerwe and Engelbrecht, 003). The fundamental dea s that each partcle represents a potental soluton whch t updates accordng to ts own experence and that of neghbors. The PSO algorthm searches n parallel usng a group of ndvduals. Indvduals or partcles n a swarm, approach to the optmum through ther present velocty, prevous experence and the experence of ts neghbors (Sh and Eberhart 998). PSO searches the problem doman by adjustng the trajectores of movng ponts n a multdmensonal space. The moton of ndvdual partcles for the optmal soluton s governed through the nteractons of the poston and velocty of each ndvdual, ther own prevous best performance and the best performance of ther neghbors. 3 In PSO, swarm s composed of a set of partcles P = { p, p, p,... p n }. The poston of partcle corresponds to a canddate soluton of the optmzaton problem. At any tme step k, the partcle p has two vectors assocated: poston and veloctyv k. Both the nformaton vectors have been recorded n every tme step and help n further movement of partcle. The best poston that partcle p has ever vsted tll tme step k s known as personal best and represented by vector X k pbest k. The best poston of all the partcles s known as global best and represented by gbest k. The movement of partcle n search space depends on the nformaton t receves from neghborhood N P. The neghborhood relatons between partcles are commonly represented as a graph G { V, E} =, where each vertex n V corresponds to a partcle n swarm and each edge n E relates connectons between them. The basc PSO algorthm conssts of three steps, namely, generaton of partcles and ther nformaton, movements and new nformaton vector. Ths can be consdered as generatng partcle s postons and veloctes, velocty update, and fnally, poston update. Frst, the postons, X k, and veloctes, V k, of the ntal swarm of partcles are randomly generated usng upper and lower bounds on the search varables values, LB and UB, as expressed n equatons () and (3). In equatons () and (3), rand s a unformly dstrbuted random varable that can take any value between 0 and. Ths ntalzaton process allows the swarm partcles to be randomly dstrbuted across the search space. 0 ( ) X = LB + rand UB LB () ( ) LB + rand UB LB V0 = t (3) The movement of partcle n the next tme step s functon of ts current velocty and partcle current poston whch s the objectve functon to be optmzed. There are three parts n velocty update: the frst part shows the current speed of partcle.e. shows ts present state, the second part s known as the cognton term whch shows the thought of the partcle tself and the last part s socal term that shows the ablty of nformaton sharng among the swarms. The ntal velocty V k s updated frst usng the nformaton of pbest k and gbest k to V k + for next teraton. Good convergence of the search space and avodng trappng n local mnma can be ensured by usng some random parameters, represented by the unformly dstrbuted varables, rand. The velocty update formula uses the current velocty, partcle personal memory and swarm memory nfluence as gven n equaton (4). ( ) ( pbestk X k gbestk X k ) Vk + = wv { k + c rand + crand t t Current Velocty Partcle Personal Memory Consderaton Swarm Memory Consderaton Where c and c are two postve acceleraton constants responsble for degree of nformaton consderaton of personal and swarm memory respectvely and w s an nerta weght whch s usually lnearly decreasng durng the teratons. The nerta weght w plays a role of balancng the local and global search. Tsa and Chu (008) proposed generalzed models and technques for tunng these parameters. Poston update s the last step n each teraton. The Poston of each partcle s updated usng ts velocty vector gven by equaton (5). (4)

4 70 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp k + k k + X = X + V t (5) The three steps of velocty update, poston update, and ftness calculatons are repeated untl a desred convergence crteron s met. PSO algorthm s very fast, smple and easy to understand and mplement. It also has a very few parameters to adjust (Kennedy et al., 00) and requres lttle memory for computaton. PSO also has major draw backs, such as when the search space s hgh ts convergence speed becomes very slow near global optmum. Another PSO problem s ts nature to a fast and premature convergence n md optmum ponts. 4. Hybrd Sequental Clusterng Algorthm The ssues related to global and local mnmum play an mportant role when data sets and attrbutes assocated are very large and the classfcaton based on clusterng s mportant and crtcal. In case of certan data sets lke medcal, securty, fnance etc. the error generated because of K- Means clusterng algorthm s not acceptable. The objectve functon of the K-Means algorthm s not convex and hence t may contan many local mnma. Bo-nspred algorthms have advantages of fndng global optmal soluton. The process of random searchng and nformaton sharng make these algorthms best tool for fndng global solutons (Sadu et al., 009). We have used one of such algorthm.e. Partcle Swarm Optmzaton (PSO) for data clusterng. In ths secton we am to propose a hybrd sequental clusterng algorthm based on combnng the K-Means algorthms and PSO algorthms. The motvaton for ths dea s the fact that PSO algorthm, at the begnnng stage of algorthm starts the clusterng process due to ts fast convergence speed and then the result of PSO algorthm s tuned by the K-Means near optmal solutons. Flow chart of proposed algorthm s shown n Fgure. START NUMBER OF CLUSTERS CALCULATE THE CENTROID CLUSTER CENTRE MOVEMENT BASED ON K-MEANS CALCULATE INTER-CLUSTER AND INTRA-CLUSTER DISTANCES DATA MOVEMENT NO YES CLUSTERING BASED ON PSO END NO TOTAL ITERATIONS REACHED YES PSO BASED CLUSTERING Fgure.. Flow chart of Hybrd Sequental Clusterng Algorthm The combnaton of K-Means algorthm and PSO wll generate the better result compared to the result of ndvdual algorthm. Ths algorthm wll remove the drawbacks of both algorthm (K-Means Algorthm and PSO Algorthm) and uses the advantage of both algorthms for producng the best optmzed result. The algorthm of the proposed scheme s gven below.

5 7 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp Algorthm : Hybrd Sequental Clusterng Algorthm. Intalzaton: Randomly generate partcles where each partcle represents a feasble soluton.e. cluster soluton. The number of partcles s taken as product of dataset dmenson and number of clusters to be generated.. Intalzaton of partcle poston and velocty: Each canddate soluton possesses a poston, whch represents the soluton n search space and velocty for the movement of partcles for fndng global optmal soluton. The poston and velocty ntalzaton s made by usng equatons () and (3). 3. Evaluaton of ftness: The ftness value of each partcle s computed by the followng ftness functon. j objectve functon( f ) = x z, =,..., n, j =,..., c (6) Where n and z are the number of datasets and clusters, respectvely and x s data pont and j z s cluster centre. The value of objectve functon s stored as partcle personal best and best of all personal best s recorded as global or swarm best. 4. Poston and velocty update: The search for the global optmal soluton s made through dynamcally updatng the partcles n swarm. The velocty update wll be made usng equaton (4) whch s functon of ntal velocty, the partcle own best performance and the swarm best performance. Poston update wll be made usng equaton (5) by addng ncremental change n poston n each step. Though partcles have been ntalzed by equaton () and (3) forcng them to search them wthn the boundary but n case they move out of boundary they are reset to the boundary value. 5. Steps -4 are repeated tll the termnaton condton s reached. 6. Place n ponts nto the space represented by the objects that are clustered wth cluster centre obtaned from PSO algorthm. These ponts represent ntal group centrods. 7. Assgn each object to the group that has the closest centrod. 8. When all objects have been assgned, recalculate the postons of the c centrods usng equaton (). 9. Repeat Steps 6 and 7 untl the centrods no longer move. PSO algorthm s a probablstc approach to fnd the optmal soluton and hence n every run t generates a new optmal soluton near around global optmal pont. It s normally suggested to take 0 runs of the algorthm and fnd the mean value of t for further processng. Although PSO s a good clusterng method, t does not perform well when the dataset s large or complex. K-Means s added n sequence to the PSO to obtan better result through further refnement n cluster formaton. The PSO algorthm s used at the ntal stage to help dscoverng the vcnty of the optmal soluton by a global search. The result from PSO s used as the ntal seed of the K-Means algorthm, whch s appled for refnng and generatng the fnal result. 5. Result and Dscusson In ths secton, detals of the overall results of the proposed algorthm are dscussed. A complete program usng MATLAB has been developed to fnd the optmal soluton. Ths secton has been dvded nto two subsectons. Frstly the workng of the proposed scheme and refnement n the cluster centers s llustrated. Secondly to evaluate the performance of the proposed clusterng algorthm, few experments have been conducted on two artfcal generated data set problems and another two wth standard data mnng benchmark problems. Subsecton : Only sx data wth two attrbutes are selected to create a dataset, to gve a graphcal vew of the workng of proposed Hybrd Sequental clusterng algorthm to frame two clusters. The data set s developed by random number generaton n

6 7 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp a range [0, ]. PSO s appled frst to the data set and the obtaned results are shown n Fgure for two cluster formulaton. The obtaned results from PSO are then processed through K-Means algorthm for further refnng the cluster formaton. Fgure 3 shows the workng of proposed Hybrd Sequental clusterng algorthm. It can be seen that the cluster centers are further shfted. It can also be seen that the centers are movng more toward the centre of cluster and both centers are movng far away from each other.e. maxmzaton of dstance between the cluster centers. It s found that the further refnement n the cluster centers lead to more composte and condensed cluster formaton also t s also observed that the cluster formaton only usng PSO s not suffcent. In PSO, the value of w = and c = c =.5 has been taken for obtanng best results. The populaton sze s chosen to be 0 and the entre algorthm s run for 0 teratons. The average results of 0 smulatons runs are then passed to K-Means algorthm. Data Cluster Center Cluster Old cluster Center Cluster New cluster Center Cluster Cluster Cluster Center Old cluster Center New cluster Center Fgure. Cluster Centre and cluster formaton by PSO for 6-data wth -attrbutes Fgure 3. Cluster Centre refnement by K-Means Algorthm for 6-data wth -attrbutes For better understandng of the algorthm the complexty s further ncreased n the problem. Agan 6 data are taken but wth ncrease n attrbutes. The consdered data has 3 attrbutes now. The results have been presented n Fgure 4 wth PSO algorthm. Fgure 5 shows the result of our proposed hybrd sequental clusterng algorthm n three dmensonal spaces wth ncreased attrbutes. The Fgures 3 and 5 clearly show the mprovement n the cluster centers. It s also observed that the ntra cluster dstance ncreased and nter cluster dstance s mnmzed. The proposed algorthm results n the formaton of more compact and more separable clusters and thus ncreases accuracy whle new data have been added up. Old Cluster Centers Cluster Center Cluster Center New Cluster Centers Data Fgure 4. Cluster Centre and cluster formaton by PSO Algorthm for 6-data wth 3-attrbutes Fgure 5. Cluster Centre refnement by K-Means Algorthm for 6-data wth 3-attrbutes Subsecton : Ths Subsecton presents comparson of the proposed scheme wth K-Means, PSO, Hybrd, K-Means+Genetc algorthm. The accuracy and robustness of our proposed algorthm have been tested on four dfferent problems. The classfcaton problems are as follows:

7 73 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp () Artfcal problem I: Ths problem formulaton s made as per followng classfcaton rule (VenderMerwe and Engelbrecht, 003). f ( z ) or( ( z ) and( z z ) ) y( z) = (7) 0 otherwse A total of 400 data vectors were randomly created, wth z and z n a range [-, ]. () Artfcal problem II: Ths s a -dmensonal problem wth 4 unque classes (Merwe and Engelbrecht, 003). The problem s nterestng n that only one of the nputs s really relevant to the formaton of the classes. A total of 600 patterns were drawn from four ndependent bvarate normal dstrbutons, where classes were dstrbuted accordng to equaton (8) for =,... ~ 4, where p s the mean vector and s the covarance matrx; m = -3, m = 0, m3 = 3 and m4 = 6. = m N µ, = (8) () Wne Problem: These data are the results of a chemcal analyss of wnes grown n the same regon n Italy but derved from three dfferent cultvars. The analyss determned the quanttes of 3 consttuents (nputs) found n each of the three types of wnes (classes). These data are collected of 78 nstances (data vectors) ( Hence, ths s a classfcaton problem wth "well behaved class structures. There are 3 nputs, 3 classes and 78 data vectors. (v) Irs Data Set: Ths s perhaps the best known database to be found n the pattern recognton lterature. The data set contans 3 classes of 50 nstances each, where each class refers to a type of rs plant ( One class s lnearly separable from the other ; the latter are NOT lnearly separable from each other. The man purpose of our proposed Hybrd Sequental clusterng algorthm s to compare the qualty of the respectve clusterng, where qualty s measured accordng to the followng three crtera: The quantzaton error as defned n equaton (9) Qe = uuur p j Where d( X, z ) c uuur j= p j d( X, z ) / N 0 c s dstance to centrod, N 0 s number of data vectors to be clustered, c s the number of cluster to be formed. The ntra-cluster dstances,.e. the dstance between data vectors wthn a cluster, where the objectve s to mnmze the ntra-cluster dstances and s gven by equaton (0). c j j Intra = uuur X z (0) n j = The nter-cluster dstances,.e. the dstance between the centrods of the clusters, where the objectve s to maxmze the dstance between clusters s gven by equaton () (9) j Inter = mn( z z () The results obtaned from the fve clusterng algorthms (K-Means, PSO, Hybrd, K-Means+Genetc algorthm and Hybrd Sequental clusterng algorthm ) are summarzed n Tables (-3). In these algorthms, every run generates a new soluton so the values reported are averaged over 30 smulatons, wth standard devatons to ndcate the range of values to whch the algorthms converge. Table presents the comparson on the ftness of solutons,.e. the quantzaton error. It can be noted that the results obtaned through the Hybrd Sequental clusterng algorthm has the smallest average quantzaton error. The devatons n the results obtaned are mnmzed n the proposed algorthm. All other algorthms have better soluton n one or another case but there s no unformty n the soluton obtaned. It s only the proposed Hybrd Sequental clusterng algorthm whch generates best among them.

8 74 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp Table. Comparson of K-Means, PSO, Hybrd, K-Means+Genetc Algorthm and Hybrd Sequental clusterng algorthm wth Quantzaton Error Algorthm Artfcal Artfcal Wne Irs Data Set problem I problem II K-Means 84 ± ± ± ±0.5 PSO 69 ± ± ± ±0.094 Hybrd 68 ± ± ± ±0.43 K-Means+Genetc algorthm 7 ± ± ± ±0.8 Hybrd Sequental clusterng algorthm 64 ± ± ± ±0.09 Table. Comparson of K-Means, PSO, Hybrd, K-Means+Genetc Algorthm and Hybrd Sequental clusterng algorthm wth Intra-cluster Dstance Algorthm Artfcal Artfcal Wne Irs Data Set problem I problem II K-Means ±0.085 ± ± ±45 PSO 3.86 ± ± ± ±0.86 Hybrd 3.83 ± ± ± ±04 K-Means+Genetc algorthm 3.89 ± ± ± ±35 Hybrd Sequental clusterng algorthm ± ± ± ±04 Table 3. Comparson of K-Means, PSO, Hybrd, K-Means+Genetc Algorthm and Hybrd Sequental clusterng algorthm wth Inter-cluster Dstance Algorthm Artfcal Artfcal Wne Irs Data Set problem I problem II K-Means.77 ± ± ± ±0.09 PSO.4 ± ± ±4 8 ±0.086 Hybrd.5 ± ± ±0. 5 ±0.097 K-Means+Genetc algorthm.5 ± ± ± ±0.097 Hybrd Sequental clusterng algorthm.779 ± ± ± 94 ±0.089 Table and Table 3 prersent the comparson of algorthms consderng ntra- and nter-cluster dstances. These parameters are cosdered to ensure compact clusters wth lttle devaton from the cluster centrods and larger separaton between the dfferent clusters. It can be seen from the results that Hybrd Sequental clusterng algorthm successfully obtan better results than ts counterparts. It has been seen that for frst two problems PSO generate better soluton than K-Means, hybrd or K-Means+Genetc algorthm but for the other two the other algorthms are better whle the proposed algorthm generates better soluton among all of them. It s also seen that the devaton n results obatned by proposed Hybrd Sequental clusterng algorthm s much less than ts counter parts and hence proves ts stablty. It s because ntal clusterng made by PSO s further tuned wth K-Means algorthm whch has capablty of obtanng better local optmal soluton. Hence, the proposed soluton always generates better soluton than ts counter algorthms. 6. Concluson Ths paper nvestgated the applcaton of the PSO n sequence wth K-Means to clusterng problem. Fve algorthms are tested, namely a standard K-Means, PSO, K-Means+ Genetc algorthm, Hybrd approach and the Hybrd Sequental clusterng algorthm, where the swarms fnd the clusters centre and further refnng s obtaned through K-Means algorthm. The Hybrd Sequental

9 75 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp clusterng algorthm s compared wth two PSO approaches, K-Means clusterng and wth Genetc algorthm, whch shows that the proposed clusterng algorthm have better convergence to lower quantzaton errors, and n general, larger nter-cluster dstances and smaller ntra-cluster dstances. The varaton n the solutons obtaned for dfferent cases s also reported mnmum n the proposed algorthm. It can be concluded that the drawback of fndng optmal soluton by K-Means can be mnmzed by usng PSO over t. The varatons n PSO algorthm and ts hybrdzaton wth K-Means algorthm s proposed for future research. References Chen C. Y. and Fun Y., 004. Partcle swarm optmzaton algorthm and ts applcaton to clusterng analyss. IEEE Internatonal Conference on networkng sensng and Control, pp Cu X., Potok, T. E. and Palathngal. P., 005. Document clusterng usng partcle swarm optmzaton. Internatonal Journal of Pattern Recognton and Artfcal Intellgence, Vol. 9, No 3, pp El-abd M. and Kamel M., 005. Informaton exchange n multple cooperatng swarms. IEEE swarm Intellgence Symposum, pp Jan A. R., Murthy M. N. and Flynn P. J., 999. Data clusterng: A Revew. ACM Computng Surveys, Vol. 3, No 3, pp Kanungo T., Mount D.M., Netanyahu N., Patko C., Slverman R. and Wu A.Y., 00. An effcent K-Means clusterng algorthm: Analyss and mplementaton. IEEE Trans. Patterns Analyss and Machne Intellgence, Vol. 4, No 7, pp Kanungo T., Mount D.M., Netanyahu N., Patko C., Slverman R. and Wu A.Y., 00. A local search approxmaton algorthm for K-Means clusterng. Computatonal Geometry: Theory and Applcatons, SoCG 0, pp Kumar A., Sabharwal Y. and Sen S., 00. Lnear tme approxmaton schme for clusterng problems n any dmensons. Journal of ACM, Vol.57, No.pp 5:-3. Kennedy J. and Eberhart R. C., 995. Partcle swarm optmzaton. IEEE Internatonal Conference on Neural Networks, Perth Australa, Vol. 4, pp Kennedy J., Eberhart R. C. and Sh Y., 00. Swarm ntellgence. Morgan Kaufmann. Mtra S. and Acharya T., 004. Data Mnng. Wley Publcatons. Panov P., Dzerosk S. and Soldatova L., 008. OntoDM: An ontology of data mnng. IEEE Internatonal Conference on Data Mnng Workshops, pp Pyle D., 999. Data preparaton for data mnng. Morgan Kaufmann. Rajan J. and Saravanan V., 008. A framework of an automated data mnng system usng autonomous ntellgent agents. Internatonal Conference on Computer Scence and Informaton Technology, pp Sadu A., Kumar R. and Kavasser R.G., 009. Optmal placement of phasor measurement unts usng partcle swarm Optmzaton. World Congress on Nature & Bologcally Inspred Computng, pp Sh Y. and Eberhart R. C., 998. Parameter selecton n partcle swarm optmzaton. Evolutonary Programmng, Vol. 44 of Lecture Notes n Computers Scence, Sprnger. pp Tsa C. Y. and Chu C. C., 008. Developng a feature weght self-adjustment mechansm for a K-Means clusterng algorthm. Computatonal Statstcs and Data Analyss, Vol. 5, pp Vander Merwe D.W. and Engelbrecht A. P., 003. Data clusterng usng partcle swarm optmzaton. Congress on Evolutonary Computaton, Vol., pp Xndong W., 004. Data mnng: Artfcal ntellgence n data analyss. IEEE/WIC/ACM Internatonal Conference on Intellgent Agent Technology. Zalk K. R., 008. An effcent k'means clusterng algorthm. Pattern Recognton Letters, Vol. 9, pp Zhang D. Q. and Chen S. C., 004. A novel kernelzed fuzzy c-means algorthm wth applcaton n medcal mage segmentaton. Artfcal Intellgence n Medcne, Vol. 3, No, pp Bographcal notes Sandeep Rana receved hs master degree from U.P Techncal Unversty INDIA n Computer Applcaton. He served as a lecturer n Sharda Unversty. Currently he s workng as Assstant System Manager n Gautam Buddha Unversty (GBU) and also persung Ph. D. from GBU. He s lfe tme member of Indan Socety of Techncal Educaton and member of Internatonal Assocaton of Computer Scence and Informaton Technology. Hs area of nterest s Artfcal Intellgence and Data Mnng. Sanjay Jasola s Professor and Dean of School of Informaton and Communcaton Technology at Gautam Buddha Unversty, Greater Noda, Inda. Hs research papers have been publshed n several Internatonal and natonal journals. He has also worked n Wawasan Open Unversty, Malaysa and completed several nternatonal consultancy assgnments. He s recpent of Gold Medal for nnovaton n open and dstance learnng from IGNOU, New Delh. He s a fellow of

10 76 Rana et al. / Internatonal Journal of Engneerng, Scence and Technology, Vol., No. 6, 00, pp Insttuton of Engneers and Insttuton of Electroncs and communcaton Engneers, Inda. Hs research nterest ncludes wreless networkng, moble communcaton, Open educatonal resources. Rajesh Kumar s Assocate Professor n the Department of Electrcal Engneerng at the Malavya Natonal Insttute of Technology (MNIT), INDIA. Presently he s Post Doctorate Research Fellow n the Department of Electrcal and Computer Engneerng at the Natonal Unversty of Sngapore (NUS), SINGAPORE on leave from MNIT. He has been actve n the research and development of Intellgent Systems and applcatons more than ten years, and s nternatonally known for hs work n ths area. Dr. Kumar has publshed over a hundred and twenty artcles on the theory and practce of ntellgent control, evolutonary algorthms, bo and nature nspred algorthms, fuzzy and neural methodologes, power electroncs, electrcal machnes and drves. He has receved the Career Award for Young Teachers n 00 from Government of Inda. Dr. Kumar s a Senor Member IEEE, Member IE (INDIA), Fellow Member IETE, Senor Member IEANG and Lfe Member ISTE. Receved August 00 Accepted October 00 Fnal acceptance n revsed form November 00