Optimizing of Fuzzy C-Means Clustering Algorithm Using GA

Transcription

1 World Acadey of Scence, Engneerng and Technology Internatonal Jornal of Copter and Inforaton Engneerng Optzng of Fzzy C-Means Clsterng Algorth Usng GA Mohanad Alata, Mohaad Molh, and Abdllah Ran Internatonal Scence Index, Copter and Inforaton Engneerng waset.org/pblcaton/3398 Abstract Fzzy C-eans Clsterng algorth (FCM) s a ethod that s freqently sed n pattern recognton. It has the advantage of gvng good odelng reslts n any cases, althogh, t s not capable of specfyng the nber of clsters by tself. In FCM algorth ost researchers fx weghtng exponent () to a conventonal vale of 2 whch ght not be the approprate for all applcatons. Conseqently, the an objectve of ths paper s to se the sbtractve clsterng algorth to provde the optal nber of clsters needed by FCM algorth by optzng the paraeters of the sbtractve clsterng algorth by an teratve search approach and then to fnd an optal weghtng exponent () for the FCM algorth. In order to get an optal nber of clsters, the teratve search approach s sed to fnd the optal sngle-otpt Sgenotype Fzzy Inference Syste (FIS) odel by optzng the paraeters of the sbtractve clsterng algorth that gve n least sqare error between the actal data and the Sgeno fzzy odel. Once the nber of clsters s optzed, then two approaches are proposed to optze the weghtng exponent () n the FCM algorth, naely, the teratve search approach and the genetc algorths. The above entoned approach s tested on the generated data fro the orgnal fncton and optal fzzy odels are obtaned wth n error between the real data and the obtaned fzzy odels. Keywords Fzzy clsterng, Fzzy C-Means, Genetc Algorth, Sgeno fzzy systes. I. ITRODUCTIO ATTER recognton s a feld concerned wth achne P recognton of eanngfl reglartes n nosy or coplex envronents. In spler words, pattern recognton s the search for strctres n data. In pattern recognton, grop of data s called a clster []. In practce, the data are sally not well dstrbted; therefore the "reglartes" or "strctres" ay not be precsely defned. That s, pattern recognton, by ts very natre, an nexact scence. To deal wth the abgty, t s helpfl to ntrodce soe "fzzness" nto the forlaton of the proble. For exaple, the bondary between clsters cold be fzzy rather than crsp; that s, a data pont cold belong to two or ore clsters wth dfferent degrees of ebershp. In ths way, the forlaton s closer to the real- Manscrpt receved March 8, M. Alata and M. Molh, Assstant professor are wth Mechancal Engneerng Departent, Jordan Unversty of Scence and Technology (eal: alata@jst.ed.jo). Abdllah Ran, Gradate stdent, s wth Mechancal Engneerng Departent, Jordan Unversty of Scence and Technology. world proble and therefore better perforance ay be expected. Ths s the frst reason for sng fzzy odels for pattern recognton: the proble by ts very natre reqres fzzy odelng (n fact, fzzy odelng eans ore flexble odelng-by extendng the zero-one ebershp to the ebershp n the nterval [0,], ore flexblty s ntrodced). The second reason for sng fzzy odels s that the forlated proble ay be easer to solve coptatonally. Ths s de to the fact that a non-fzzy odel often reslts n an exhastve search n a hge space (becase soe key varables can only take vales 0 and ), whereas n a fzzy odel all the varables are contnos, so that dervatves can be copted to fnd the rght drecton for the search. A key proble s to fnd clsters fro a set data ponts. Fzzy C-Means (FCM) s a ethod of clsterng whch allows one pece of data to belong to two or ore clsters. Ths ethod was developed by Dnn [2] n 973 and proved by Bezdek [3] n 98 and s freqently sed n pattern recognton. Ths what we want fro the optzaton s to prove the perforance toward soe optal pont or ponts [4]. Ls [5] dentfes three an types of search ethods: calclsbased, eneratve and rando. Hall, L. O., Ozyrt, I. B. and Bezdek, J. C. [6] descrbe a genetcally gded approach for optzng the hard (J ) and fzzy (J ) c-eans fnctonal sed n clster analyss. Or experents show that a genetc algorth aelorates the dffclty of choosng an ntalzaton for the c-eans clsterng algorths. Experents se sx data sets, ncldng the Irs data, agnetc resonance and color ages. The genetc algorth approach s generally able to fnd the lowest known J vale or a J assocated wth a partton very slar to that assocated wth the lowest J vale. On data sets wth several local extree, the GA approach always avods the less desrable soltons. Deterorate parttons are always avoded by the GA approach, whch provdes an effectve ethod for optzng clsterng odels whose objectve fncton can be represented n ters of clster centers. The te cost of genetc gded clsterng s shown to ake seres of rando ntalzatons of fzzy/hard c-eans, where the partton assocated wth the lowest J vale s chosen, and an effectve copettor for any clsterng doans. The an dfferences between ths work and the one by Bezdek et al [6] are: 670 scholar.waset.org/ /3398

2 World Acadey of Scence, Engneerng and Technology Internatonal Jornal of Copter and Inforaton Engneerng Ths work sed the least sqare error as an objectve fncton for the genetcs algorth bt Bezedek [6] sed J as an objectve fncton. Ths work optzed the weghtng exponent wthot changng the dstance fncton bt Bezdek [6] keeps the weghtng exponent 2.00 and ses two dfferent dstance fnctons to fnd an optal vale. II. THE SUBTRACTIVE CLUSTERIG The sbtractve clsterng ethod asses each data pont s a potental clster center and calclates a easre of the lkelhood that each data pont wold defne the clster center, based on the densty of srrondng data ponts. The algorth: C x j k x k c Ths teraton wll stop when. x 2 (2) (3) Internatonal Scence Index, Copter and Inforaton Engneerng waset.org/pblcaton/3398 Selects the data pont wth the hghest potental to be the frst clster center Reoves all data ponts n the vcnty of the frst clster center (as deterned by rad), n order to deterne the next data clster and ts center locaton Iterates on ths process ntl all of the data s wthn rad of a clster center The sbtractve clsterng ethod [7] s an extenson of the ontan clsterng ethod proposed by R. Yager [8]. The sbtractve clsterng s sed to deterne the nber of clsters of the data beng proposed, and then generates a fzzy odel. However, the teratve search s sed to optze the least sqare error fro the odel beng generated and the test odel. After that, the nber of clsters s taken to the Fzzy C-Means Algorth. III. THE FUZZY C-MEAS CLUSTERIG ALGORITHM Fzzy C-Means (FCM) s a ethod of clsterng whch allows one pece of data to belong to two or ore clsters. Ths ethod s freqently sed n pattern recognton. It s based on nzaton of the followng objectve fncton: J C j x Where s any real nber greater than, t was set to 2.00 by Bezdek. s the degree of ebershp of x n the clster j; x s the th of d-densonal easred data ; c j s the d-denson center of the clster and * s any nor expressng the slarty between any easred data and the center. Fzzy parttonng s carred ot throgh an teratve optzaton of the objectve fncton shown above, wth the pdate of ebershp and the c j clster centers by: j 2 () ax ( k { } + ( k ) < ε Where ε s a ternaton crteron between 0 and and k are the teraton steps. Ths procedre converges to a local n or a saddle pont of J. The algorth s coposed of the followng steps:. Intalze U [ ] atrx, U(0) 2. At k-step: calclate the centers vectors C(k)[c j ] wth U(k) c 3. Update U(k), U(k+) C k x x. x j k 2 If U(k+) - U(k) <ε then STOP; otherwse retrn to step 2. IV. THE GEETICS ALGORITHM The GA s a stochastc global search ethod that cs the etaphor of natral bologcal evolton. GAs operates on a poplaton of potental soltons applyng the prncple of srvval of the fttest to prodce (hopeflly) better and better approxatons to a solton [9, 0]. At each generaton, a new set of approxatons s created by the process of selectng ndvdals accordng to ther level of ftness n the proble doan and breedng the together sng operators (4) (5) (6) 67 scholar.waset.org/ /3398

3 World Acadey of Scence, Engneerng and Technology Internatonal Jornal of Copter and Inforaton Engneerng Internatonal Scence Index, Copter and Inforaton Engneerng waset.org/pblcaton/3398 borrowed fro natral genetcs. Ths process leads to the evolton of poplatons of ndvdals that are better sted to ther envronent than the ndvdals that they were created fro, jst as n natral adaptaton. V. RESULTS AD DISCUSSIO A coplete progra sng MATLAB prograng langage was developed to fnd the optal vale of the weghtng exponent. It starts by perforng sbtractve clsterng for npt-otpt data, bld the fzzy odel sng sbtractve clsterng and optze the paraeters by optzng the least sqare error between the otpt of the fzzy odel and the otpt fro the orgnal fncton by enterng a tested data. The optzng s carred ot by teraton. After that, the genetcs algorths optzed the weghtng exponent of FCM. The sae way, bld the fzzy odel sng FCM then optze the weghtng exponent by optzng the least sqare error between the otpt of the fzzy odel and the otpt fro the orgnal fncton by enterng the sae tested data. Fg. at Appendx A shows the flow chart of the progra. The best way to ntrodce reslts s throgh presentng for exaples of odelng of for hghly nonlnear fnctons. Each exaple s dscssed, plotted. Then copared wth the best error of orgnal FCM wth weghtng exponent ( 2.00). A. Exaple - Modelng a Two Inpt onlnear Fncton In ths exaple, a nonlnear fncton was proposed: sn( x) sn( y) z * x y The range X [-0.5, 0.5] and Y [ ] s the npt space of the above eqaton, 200 data pars are obtaned randoly (Fg. 2). Fg. 2 Rando data ponts of eqaton (7); ble crcles for the data to be clstered and the red stares for the testng data (7) Frst, the best least sqare error was obtaned for the FCM of weghtng exponent ( 2.00) whch s (0.026 wth 53 clsters). ext, the optzed least sqare error of the sbtractve clsterng s obtaned by teraton that s (0.05 wth 52 clsters). We cold see here that the error proves by (0%). Then, the clsters nber s taken to the FCM algorth, the error s optzed to (0.004 wth 52 clsters) that eans the error proves by (30%) and the weghtng exponent () s (.449). Reslts are better shown n Table I at Appendx A. B. Exaple 2 - Modelng a One Inpt onlnear Fncton In ths exaple, a nonlnear fncton was proposed also bt wth one varable x: sn( x) y x The range X [-20.5, 20.5] s the npt space of the above eqaton, 200 data pars were obtaned randoly and shown n Fg. 3. Fg. 3 Rando data ponts of eqaton (8); ble crcles for the data to be clstered and the red stares for the testng data Frst, the best least sqare error s obtaned for the FCM of weghtng exponent ( 2.00) whch s (5.898*e-7 wth 78 clsters). ext, the least sqare error of the sbtractve clsterng s obtaned by teraton whch was (*e-0 wth 24) clsters snce ths error pre-defned f the error s less than (*e-0). Then, the clsters nber s taken to the FCM algorth, the error s (.2775*e-2) wth 24 clsters and the weghtng exponent () s (.7075). The proveent s (4*e7) % and the nber of clsters proved by 74%. Reslts are better shown n Table 2 at the Appendx. C. Exaple 3 - Modelng a One Inpt onlnear Fncton In ths exaple, a nonlnear fncton was proposed: y ( x 3) 3 0 (9) The range X [, 50] s the npt space of the above eqaton, 200 data pars were obtaned randoly and are shown n Fg. 4. (8) 672 scholar.waset.org/ /3398

4 World Acadey of Scence, Engneerng and Technology Internatonal Jornal of Copter and Inforaton Engneerng The whole reslts are better shown n Table IV at Appendx A. Internatonal Scence Index, Copter and Inforaton Engneerng waset.org/pblcaton/3398 Fg. 4 Rando data ponts of eqaton (9); ble crcles for the data to be clstered and the red stares for the testng data Frst, the best least sqare error s obtaned for the FCM of weghtng exponent ( 2.00) whch s (3.3583*e-7 wth 88 clsters). ext, the least sqare error of the sbtractve clsterng s obtaned by teraton whch s (.6988*e-7 wth 03 clsters) snce the least error can be taken fro the teraton. Then, the clsters nber s taken to the FCM algorth, the error was (2.289*e-8 wth 03 clsters) and the weghtng exponent () s Here we cold see that the nber of clsters s redced fro 88 to 03 clsters that ean the nber of rles s redced and the error s proved by 4 tes. Reslts are better shown n Table III at Appendx A. VI. COCLUSIO In ths work, the sbtractve clsterng paraeters, whch are the rads, sqash factor, accept rato, and the reject rato are optzed sng the GA. The orgnal FCM proposed by Bezdek s optzed sng GA and another vales of the weghtng exponent rather than ( 2) are gvng less approxaton error. Therefore, the least sqare error s enhanced n ost of the cases handled n ths work. Also, the nber of clsters s redced. The te needed to reach an opt throgh GA s less than the te needed by the teratve approach. Also GA provdes hgher resolton capablty copared to the teratve search de to the fact that the precson depends on the step vale n the for loop fncton whch s ax eqal to 0.00 for the rads paraeter n the sbtractve clsterng algorth, bt for GA, t depends on the length of the ndvdal and the range of the paraeter whsh s for the rads paraeter also. So GA gves better perforance and has less approxaton error wth less te. Also t can be conclded that the te needed for the GA to optze an objectve fncton depends on the nber and the length of the ndvdal n the poplaton and the nber of paraeter to be optzed. 673 scholar.waset.org/ /3398

5 World Acadey of Scence, Engneerng and Technology Internatonal Jornal of Copter and Inforaton Engneerng APPEDIX Orgnal Data Testng data Orgnal fncton reslts Sbtractve clsterng odel Least sqares error Change lest sqares paraeters Internatonal Scence Index, Copter and Inforaton Engneerng waset.org/pblcaton/3398 Testng data Orgnal fncton reslts Store reslts: centers To FCM Least sqares error The weghtng exponent of the optal least sqares Orgnal Data FCM clsterng odel Fg. The flowchart of the software Change the weghtng exponent by GA f the nber of generatons s not reached TABLE I THE RESULTS OF EQUATIO (7): M IS THE WEIGHTIG EXPOET AD TIME I SECODS Iteraton Error Clsters M Iteraton for the Sbtractve clsterng FCM Te sbtractve and GA Error Clsters Error for FCM Iteraton TABLE II THE RESULTS OF EQUATIO (8): M IS THE WEIGHTIG EXPOET AD TIME I SECODS Error Clsters e-7 78 Iteraton for the Sbtractve clsterng FCM Te sbtractve and GA Error Clsters Error M for FCM e e Iteraton TABLE III THE RESULTS OF EQUATIO (9): M IS THE WEIGHTIG EXPOET AD TIME I SECODS Error Clsters M e-7 88 Iteraton for the Sbtractve clsterng FCM Te sbtractve and GA Error Clsters Error 2440 for FCM.6988 e e scholar.waset.org/ /3398

6 World Acadey of Scence, Engneerng and Technology Internatonal Jornal of Copter and Inforaton Engneerng TABLE IV THE FIAL LEAST SQUARE ERRORS AD CLUSTERS UMBER FOR THE ORIGIAL FCM AD FOR THE FCM WHICH THEIR UMBERS OF CLUSTERS WERE GOT FROM THE ITERATIVELY OR GEETICALLY OPTIMIZED SUBTRACTIVE CLUSTERIG The fncton The orgnal FCM (2) Iteraton then genetcs Error Clsters Error Clsters ew () Eq (7) Eq (8) e e Eq (9) e e Internatonal Scence Index, Copter and Inforaton Engneerng waset.org/pblcaton/3398 REFERECES [] L-Xn Wang, A Corse n Fzzy Systes and Control, (Prentce Hall, Inc.) Upper Saddle Rver, J 07458; 997: [2] J. C. Dnn, A Fzzy Relatve of the ISODATA Process and Its Use n Detectng Copact Well-Separated Clsters, Jornal of Cybernetcs 3; 973: [3] Bezdek, J. C., Pattern Recognton wth Fzzy Objectve Fncton Algorths, Plen Press, Y, 98. [4] Beghtler, C. S., Phllps, D. J., & Wld, D. J., Fondatons of optzaton (2 nd ed.). (Prentce-Hall) Englewood Clffs, J, 979. [5] Ls, R. and Jaakola T. H. I., Optzaton by Drect Search and Systeatc Redcton of the Sze of Search Regon, AIChE Jornal 973; 9(4): [6] Hall, L. O., Ozyrt, I. B. and Bezdek, J. C., Clsterng wth a genetcally optzed approach, IEEE Trans. Evoltonary Coptaton 999; 3(2), [7] Ch S. Fzzy odel dentfcaton based on clster estaton. 994, Jornal of Intellgent and Fzzy Systes; 2: [8] Yager, R. and D. Flev, Generaton of Fzzy Rles by Montan Clsterng, Jornal of Intellgent & Fzzy Systes, 994; 2 (3): [9] Gnat, D., Genetc Algorth - A Fncton Optzer, SF Report, Departent of Copter Scence, Unversty of Maryland, College Park, MD20740, 988. [0] Wang, Q. J., Usng Genetc Algorths to Optze Model Paraeters, Jornal of Envronental Modelng and Software 997; 2(): 675 scholar.waset.org/ /3398