Cluster Feature-Based Incremental Clustering Approach (CFICA) For Numerical Data

Transcription

1 JCSNS nternatonal Journal of Computer Scence and Network Securty, VOL.0 No.9, September Cluster Feature-ased ncremental Clusterng pproach (CFC For Numercal ata.m.sowanya and M.Shash, epartment of Computer Scence. & Systems Engneerng, College of Engneerng, ndhra Unversty, Vsakhapatnam. Summary ata clusterng s a hghly valuable feld of computatonal statstcs and data mnng. maor dffculty n the desgn of modern data clusterng algorthms s that, n maorty of applcatons, new data sets are dynamcally appended nto an exstng massve database and t s not vable to perform data clusterng from scrape every tme new data nstances get added up n the database. he development of clusterng algorthms whch handle the ncrementally updated data ponts, has receved ncreasng nterest among the researchers. hs paper presents a more effcent cluster feature-based ncremental clusterng approach (CFC for numercal data sets. ntal clusterng s performed on the statc database wth the help of k-means clusterng algorthm. hen, by makng use of cluster feature computed from the ntal clusters, the ncrementally updated data ponts are clustered. Subsequently, the closest par of clusters s merged to obtan better cluster accuracy. Fnally, the proposed approach has been valdated wth the help of real datasets presented n the UC machne learnng repostory. he expermental results demonstrated that clusterng accuracy of the proposed ncremental clusterng approach s mproved sgnfcantly. Key words: ata mnng, Clusterng, ncremental clusterng, k-means algorthm, Cluster Feature, Mean, Farthest neghbor ponts, Clusterng accuracy (C. ntroducton n vew of the fast growth of World Wde Web, nternet users have overwhelmng quanttes of onlne nformaton that requre automatc data mnng technques. Manual analyss of data s a vague process n whch quck nformaton retreval by the user s mpossble []. Clusterng s the unsupervsed classfcaton of patterns nto groups. t groups a set of obects nto dfferent subsets such that obects of the same cluster are hghly equvalent to one another. he varous applcatons of clusterng nclude mage segmentaton, nformaton retreval, web pages groupng, market segmentaton, and scentfc and engneerng analyss []. ata clusterng s a prmary data mnng method and an effcent technque for data analyss. Contnuous dumpng of each and every raw data set nto an exstng massve database requres the desgn of new clusterng algorthms. soluton to handle ths problem s to ntegrate a clusterng algorthm that functons ncrementally [3]. ncremental clusterng algorthms permt a sngle or a few passes over the whole dataset to put the updated tem nto the cluster. Wth respect to the sze of the set of obects, algorthms and number of attrbutes, ncremental clusterng algorthms are of scalable nature [4]. he model of ncremental algorthms for data clusterng s necessated by realstc applcatons where the demand sequence s not known n advance and the algorthm should keep a constantly fne clusterng usng a restrcted set of operatons resultng n a soluton of herarchcal structure [5]. n ths paper, we report a more effcent approach, Cluster Feature-based ncremental Clusterng (CFC. he statc database s gven to the k-means clusterng algorthm for ntal clusterng. he cluster obtaned s used to compute the cluster feature, whch conssts of mean and p-farthest neghbor ponts of the cluster. he computed cluster feature s used for clusterng the ncremental database, wheren the devsed dstance measure s used to fnd the approprate cluster for each ncomng data pont. t the same tme, a new cluster s formed f the computed dstance measure s above the predefned threshold level. Subsequently, the cluster feature s updated each tme after fndng the relevant cluster for every ncomng data pont. he closest cluster par for a set of processed data ponts, s merged usng the mean of the cluster. hs procedure s repeated for each data pont avalable n the ncremental database and fnally, the resultant cluster s obtaned.. CFC algorthm Most of the clusterng algorthms presented n the lterature are used for clusterng the statc database. Nowa-days, research communty has shfted focus to the ncremental databases for real-world applcatons and the necessty of handlng dynamcally updated data ponts. Motvated by ths research nterest, we have developed an ncremental clusterng approach for numercal data sets. ntal clusterng and handlng of ncremental data ponts Manuscrpt receved September 5, 00 Manuscrpt revsed September 0, 00

2 74 JCSNS nternatonal Journal of Computer Scence and Network Securty, VOL.0 No.9, September 00 are two mportant steps of the ncremental clusterng approaches. We made use of K-means clusterng algorthm (Conventonal clusterng algorthm for ntal clusterng and then, the proposed approach, for clusterng the ncremental data ponts has been appled. Let S be the orgnal data base (statc database and ΔS be the ncremental database. Our ultmate am s to cluster the database S + ΔS, n whch the data ponts from the ncremental database ΔS are updated over tme. he procedure used for the proposed ncremental clusterng approach s gven n fgure. nput: Statc atabase S, ncremental atabase ΔS, Mergng hreshold M, hreshold N, k, t. Output: he resultant cluster, C R Parameters: Δy : ncomng data pont CF : Cluster feature ( Δ y : stance Measure m : Mean value E : Eucldean stance ( q : p-farthest neghbor ponts : farthest neghbor pont Q Method:. Call k-mean (S, k. For from to k a Compute CF = { ( m, } 3. For each pont Δy n ΔS b For from to k q Compute { m, Q } Calculate dstance, ( Δ y < c dd Δy to th cluster, f mn ( Δ y d Formng a new cluster, f mn ( Δ y e Update CF = { ( m, } q f fter processng t data ponts n ΔS N N 4. he resultant cluster, C R Subroutne: k-mean ( S *, k * Compute E n between mean ponts Merge the closet cluster par; f E < M Update the mean of merged cluster Go to step 3.f. (. Choose ntal means m, m,.., m k* from * S. Untl there are no change n the mean a Use the estmated mean to cluster the data ponts b For from to k*. Replace m wth the mean of data ponts for cluster c Go to step (, a 3. he set of k cluster Fg. Proposed ncremental Clusterng pproach. ntal clusterng wth statc database t frst, the clusterng s performed on the statc database ( S, where the data ponts do not change over tme. Here, for clusterng the statc database ( S, we have used the k-means clusterng algorthm so that, k number of clusters are obtaned. K-means algorthm s one of the most popular data clusterng methods because of ts smplcty and computatonal effcency... K-means clusterng algorthm K-means clusterng s a well-known parttonal algorthm that ams to partton n data ponts nto k clusters, n whch each data pont belongs to the cluster wth the nearest mean ssume the statc data base, S comprsng of n data ponts y, y, L, yn so that every data pont s n R, the problem of dentfyng the mnmum varance clusterng of the dataset nto k clusters s none other than fndng k means m } ( =,, L, k n R whereas {

3 JCSNS nternatonal Journal of Computer Scence and Network Securty, VOL.0 No.9, September n n = [ mn d ( y, m ] s mnmzed, where d( y, m ndcates the Eucldean dstance between y and m. he ponts { m } ( =,, L, k are termed as cluster centrods. he problem n the aforementoned equaton s to obtan k cluster centrods, n whch the average squared Eucldean dstance (mean squared error, MSE between a data pont and ts nearest cluster centrod s mnmzed [6]. Steps: ntalze k means, one for each cluster. Compute the dstance E ( y, m (descrbed n defnton of each k centrod wth data ponts y n S. 3 ssgn data pont y to cluster C whose dstance s less. 4 Update the k centrods based on the membershps of new clusters. 5 Repeat Step to step 4, untl there s no movement of the data ponts between the clusters. efnton : (stance Measure For calculatng the dstance between the centrod and the data ponts, we make use of the Eucldean dstance [7]. he Eucldean dstance ( E between ponts y and m s the length of the lne segment ym. n Cartesan coordnates, f y = y, y, L, y and ( l m = ( m, m, L, ml are two ponts n Eucldean l - space, afterwards the dstance from y to m s formulated by: l m + ( y m + + ( yl ml = ( y m = E ( y, m = ( y L. Clusterng of ncremental database We obtan k number of clusters, whch are represented n a set, c = { c, c, L, ck }; k from the k-means algorthm usng the statc database S. fter that, usng the proposed approach, we cluster the ncremental database Δ S consstng of r data ponts Δ y, Δy, L, Δy r ; r. he motvaton behnd ths approach s that, n ncremental database, data ponts Δ y are added over tme. hese changes should be reflected n the resultant cluster C R wthout extensvely affectng the current clusters, c. fter the database s updated, there are three possbltes for clusterng the updated ponts as shown n the fgures to 4.ssume and are the ntal clusters for an nstance and represents the new ncomng data ponts. Case : ddng wth the exstng cluster. Fg.. Representaton Case : Formaton of a new cluster. of addng to the exstng cluster Fg. 3. Representaton of formaton of a new cluster Case 3: Possblty to merge the exstng clusters when updated ponts are n between the exstng two clusters. Fg. 4. Representaton of mergng the closest cluster par

4 76 JCSNS nternatonal Journal of Computer Scence and Network Securty, VOL.0 No.9, September 00 he proposed approach s desgned by makng use of the above three possbltes and the steps of the approach s gven n ths sub-secton... Computaton of Cluster Feature (CF he cluster feature ( CF s computed for every cluster c whch s obtaned from the k-means algorthm. n the proposed approach, mean and p -farthest neghbor ponts are used to represent the CF, whch s denoted as, ( CF = { m, q }; p, where m s the mean ( of the cluster c and q S gves the p -farthest neghbor ponts of cluster c. he p -farthest neghbor ponts of the cluster c are calculated as follows: he Eucldean dstance value E s calculated n between the data ponts wthn cluster c and the mean of correspondng cluster m. hen, the data ponts are arranged n descendng order wth the help of the measured Eucldean dstance. Subsequently, the top p - farthest neghbor ponts for every cluster are chosen from ( the sortng lst and these ponts ( q ; p are known as p -farthest neghbor ponts of the cluster c wth respect to the mean value m... Fndng the approprate cluster n ths step, we ntate the clusterng process for dynamcally updatng data ponts avalable n the ncremental database, Δ S. t frst, the farthest neghbor pont Q whch s nearer to the ncomng data pont Δ y s dentfed for every cluster c.we make use of the p - farthest neghbor ponts for dentfyng the farthest neghbor pontq. n order to fnd the farthest neghbor pont Q, we calculate the Eucldean dstance descrbed n defnton, for an ncomng data pont E Δ y wth each p -farthest neghbor ponts of every cluster c. hen, the p -farthest neghbor ponts are sorted based on the computed dstance measure and thereby, for each cluster, we take one pont, known as farthest neghbor pont Q havng mnmum dstance. fter that, we fnd the ( dstance Δ y of every cluster c wth the ncomng data pont Δ y. he ncomng data pont Δ y s assgned to the current cluster havng mnmum dstance only f the calculated dstance s less than the predefned threshold level, N. Otherwse, the ncomng data pont Δy s separately formed as a new cluster so that, the number of cluster s ncremented wth one. stance measure: stance strategy s an mportant for effectvely dentfyng the approprate cluster of an ncomng data pont Δ y. n the proposed approach, we have used dfferent dstance strategy that makes use of three ponts such as mean m, farthest neghbor pont Q and the ncomng data pont Δ y. For each cluster c, the Eucldean dstance E s calculated for the followng two set of ponts: mean and ncomng pont ( m, farthest neghbor pont and ncomng pont ( Q, mean and farthest neghbor pont m, Q. When new data pont ( Δ y s updated n the statc database S, the dstance ( Δ y s calculated usng the followng equaton. he cluster qualty of the proposed approach s mproved sgnfcantly f these three measures are ncorporated nto the dstance value for dentfyng the approprate cluster. ( = E ( m, Δy + E ( Q, Δy E Δy where, E ( m Eucldean dstance between the ponts m and Δ y ; E ( Q Eucldean dstance between the ponts Q and Δ y ; E ( m, Q Eucldean dstance between the ponts m and Q..3 Updatng of Cluster Feature fter addng the ncomng data pont Δ y to the cluster, updatng of CF s mportant for further processng. he cluster feature ( CF of the ncremented cluster C (the cluster c wth ncomng data pont Δ y s updated by : ( takng the mean of the ncremented cluster C ( fndng the p -farthest neghbor pont of the ncremented cluster C. For updatng of CF, we frst calculate the mean of the ncremented cluster ( m and update the frst ( m, Q

5 JCSNS nternatonal Journal of Computer Scence and Network Securty, VOL.0 No.9, September component of the cluster feature CF wth m. hen, Eucldean dstance E s calculated for every data ponts n the ncremented cluster C wth the updated mean m.he data ponts are sorted based on the computed dstance measure and new set of top p -farthest neghbor ponts are chosen from the sorted lst. Such a way, the new cluster feature s formed for the ncremented cluster C, ( whch s represented as, { m, q } CF = : p, hs process of updatng s applcable only to the ncremented cluster C whch s updated recently. Fndng the approprate cluster (sub-secton.. and updatng of cluster feature (sub-secton 3..3 are teratvely performed for t number of data ponts n the ncremental database Δ S...4 Mergng of closest cluster par Once the t number of data ponts n the ncremental database Δ S are processed wth the proposed approach, there s an urgent requrement to merge the closest clusters. o avod the formaton of a non-optmal clusterng structure whch may occur durng the ncremental clusterng process, the mergng strategy s used n the proposed approach that gudes the ncremental clusterng process wth hgh qualty and at the same tme, the proposed approach generates the reasonable number of clusters. For mergng the closest cluster par, we make use of the CF employed n the prevous steps. he mean of every cluster dentfed after fnshng the t number of teraton s used for mergng the closest cluster pars. f t data ponts are processed wth the proposed approach, then we perform the mergng strategy where, the closest cluster par s dentfed by makng use of the mean and the Eucldean dstance. he procedure used for mergng process s descrbed as: ( Calculate the Eucldean dstance E n between the mean of the cluster wth other cluster ( Select the cluster par havng mnmum dstance ( Merge the chosen cluster par, f the dstance of the chosen cluster par s less than the mergng threshold level, M (v Recalculate the mean of the merged cluster (v Repeat step ( to (v untl no cluster par s merged. Fnally, the resultant cluster C R s obtaned from the mergng process after processng all data ponts n the ncremental database. 3. Results We have mplemented the proposed ncremental clusterng approach usng two statc datasets rs dataset [8] and wne dataset [9] from the UC machne repostory. rs dataset comprses 3 classes of 50 nstances each, n whch each class defnes to a type of rs plant. he attrbutes are sepal length n cm, sepal wdth n cm, petal length n cm, petal wdth n cm and class label. Wne dataset comprses 3 classes of 78 nstances. hese data are the outcome of a chemcal analyss of wnes grown n smlar regons n taly but resultng from three dverse cultvators. he analyss resolves the quanttes of 3 consttuents dentfed n each of the three types of wnes. he attrbutes are lcohol, Malc acd, sh, lcalnty of ash, Magnesum, otal phenols, Flavanods, Nonflavanod phenols, Proanthocyanns, Color ntensty, Hue, Prolne and class label. o convert these statc problems nto dynamc problems, the data ponts n each database are dvded nto groups. each group consstng of dentcal number of data ponts from each class. ntally, we gve the frst group of data ponts to the k-means algorthm for ntal clusterng. t provdes the k- number of clusters. hen, we compute the cluster feature for ntal cluster and by makng use of the cluster feature; we fed the second group of data ponts to the proposed approach ncrementally. For each data pont from the second group, we compute the desgned dstance measure and assgn the data ponts to the correspondng cluster f the mnmum dstance of the cluster s less than the mnmum threshold ( N =0. Otherwse, t forms as a separate cluster. Subsequently, the cluster feature s updated for each data ponts based on the proposed approach. Once we process the t ( t =0 set of data ponts, the mergng process s done n accordance wth the mergng threshold ( M = 4. Fnally, we obtan the set of resultant cluster from the mergng process. he performance of the proposed approach s evaluated on rs dataset and wne dataset usng Clusterng ccuracy (C. We have used the clusterng accuracy descrbed n [37, 38] for evaluatng the performance of the proposed approach. he evaluaton metrc used n the proposed approach s gven below, Clusterng ccuracy, C = N Clusterng Error, CE = = C where, N Number of data ponts n the dataset; Number of resultant cluster ; X Number of data ponts occurrng n both X

6 78 JCSNS nternatonal Journal of Computer Scence and Network Securty, VOL.0 No.9, September 00 cluster and ts correspondng class. he expermental results of the proposed approach are shown n fgure 5 and fgure 6. We calculate the clusterng accuracy of the resultant clusters by changng the k-value (order of ntal clusterng and at the same tme, clusterng error s also calculated Clusterng ccuracy (C k rs dataset Wne dataset Fg. 5. Clusterng ccuracy vs. nput order of ntal clusterng ( k Clusterng Error (CE k Fg. 6. Clusterng Error vs. nput order of ntal clusterng ( k 4. Concluson rs dataset Wne dataset n ths paper, we have developed a more effcent approach for clusterng ncremental database usng cluster feature. he proposed approach has two modules namely, ntal clusterng ncremental clusterng. Frst, ntal clusters have been obtaned by usng the conventonal parttonal clusterng algorthm - k-means algorthm then, cluster features have been obtaned usng the ntal cluster and these cluster features have been used for clusterng the ncremental database n accordance wth the devsed dstance measure. Fnally, by makng use of the mean value, closest par of clusters has been merged after processng the set of data ponts. For expermentaton, we have employed the real datasets obtaned from the UC machne learnng repostory. he expermental results ndcated that the proposed ncremental clusterng approach s effcent n terms of clusterng accuracy. References [] Seokkyung Chung and enns McLeod, "ynamc Pattern Mnng: n ncremental ata Clusterng pproach", Journal on ata Semantcs, Vol., pp. 85-, 005. [] Pham,.. and ffy,.. Clusterng technques and ther applcatons n engneerng, Proceedngs- nsttuton of Mechancal Engneers Part C Journal of Mechancal Engneerng Scence, Vol: ; No:, pp: , 007. [3] Chen-Yu Chen, Shen-Chng Hwang, and Yen-Jen Oyang, "n ncremental Herarchcal ata Clusterng lgorthm ased on Gravty heory", Proceedngs of the 6th Pacfc- sa Conference on dvances n Knowledge scovery and ata Mnng, Pages: 37-50, 00. [4] an Smovc, Namta Sngla, "Metrc ncremental Clusterng of Nomnal ata", CM, pp: 53-56, 004. [5] mtrs Fotaks, "ncremental algorthms for Faclty Locaton and k-medan", heoretcal Computer Scence, Vol: 36, No: -3, pp: 75-33, 006. [6] FHM.M., SLEM.M., orkey F.., Ramadan M.., "n effcent enhanced k-means clusterng algorthm", Journal of Zheang Unversty scence, vol. 7, no.0, pp , 006. [7] Eucldean dstance from Eucldean_dstance [8] rs dataset from [9] Wne dataset from Wne [0] Zengyou He, Xaofe Xu, Shengchun eng, Clusterng mxed numerc and categorcal data: cluster ensemble approach, abs/cs/ [] Z. Huang, Extensons to the k-means algorthm for clusterng large data sets wth categorcal values, ata Mnng and Knowledge scovery, vol., no.3, pp , September 998. [] Chung-Chan Hsu and Yan-Png Huang: "ncremental clusterng of mxed data based on dstance herarchy", Expert Systems wth pplcatons, Vol: 35, No: 3, pp: 77-85, 008. [3] Eucldean dstance from Eucldean_dstance [4] Gavn Shaw & Yue Xu, "Enhancng an ncremental Clusterng lgorthm for Web Page Collectons", Proceedngs of the 009 EEE/WC/CM nternatonal Jont Conference on Web ntellgence and ntellgent gent echnology, Vol: 3, pp: 8-84, 009. [5] Jawe Han, Mchelne Kamber, ata Mnng Concepts and echnques, Harcourt nda Prvate Lmted, 00.

7 JCSNS nternatonal Journal of Computer Scence and Network Securty, VOL.0 No.9, September M.Sowanya receved her M.ech. egree n Computer Scence and technology from ndhra Unversty. She s presently workng as an ssstant Professor n the department of Computer Scence and Systems Engneerng, College of Engneerng (utonomous, ndhra Unversty, Vsakhapatnam, ndhra Pradesh, nda. She s pursung her Ph. from ndhra Unversty. Her areas of nterest nclude ata Mnng and atabase management systems. M.Shash receved her.e. egree n Electrcal and Electroncs and M.E. egree n Computer Engneerng wth dstncton from ndhra Unversty. She receved Ph. n 994 from ndhra Unversty and got the best Ph. thess award. She s workng as a professor and HO of Computer Scence and Systems Engneerng at ndhra Unversty, ndhra Pradesh, nda. She receved CE career award as young teacher n 996. She s a co-author of the ndan Edton of text book on ata Structures and Program esgn n C from Pearson Educaton Ltd. She publshed techncal papers n Natonal and nternatonal Journals. Her research nterests nclude ata Mnng, rtfcal ntellgence, Pattern Recognton and Machne Learnng. She s a lfe member of SE, CS and a fellow member of nsttute of Engneers (nda.