A penalized fuzzy clustering algorithm

Transcription

1 Proceedngs of the 6th WSEAS Internatonal Conference on Appled Computer Scence, Tenerfe, Canary Islands, Span, December 6-8, 26 3 A penalzed fuzzy clusterng algorthm Mn-Shen Yang a, Wen-Lang Hung b and Cha-Hsuan Chang a a Department of Appled Mathematcs, Chung Yuan Chrstan Unversty, Chung-L 3223, Tawan, ROC b Department of Appled Mathematcs, Natonal Hsnchu Unversty of Educaton, Hsn-Chu,Tawan,ROC Abstract: In ths paper, we propose a penalzed nter-cluster separaton (PICS) fuzzy clusterng algorthm by addng a penalty term to the nter-cluster separaton (ICS) algorthm. Numercal comparsons are made for several fuzzy clusterng algorthms accordng to crtera of accuracy and computatonal effcency. The results show that the PICS has better accuracy and effcency. Image segmentaton s an mportant step n any mage analyss system. Exstng varous segmentaton methods for magnetc resonance mage (MRI) have been used to dfferentate abnormal and normal tssues. We apply the PICS algorthm to the MRI segmentaton of an ophthalmc patent. In these MRI segmentaton results, we fnd that PICS provdes useful nformaton as an ad to dagnoss n ophthalmology. Key Words: Fuzzy clusterng; fuzzy c-means (FCM); nter-cluster separaton (ICS); penalzed ICS; mage segmentaton; Magnetc resonance mage (MRI). Introducton Cluster analyss s a method of groupng data wth smlar characterstcs nto larger unts of analyss. Snce Zadeh [7] frst artculated fuzzy set theory whch gave rse to the concept of partal membershp, based on a membershp functon, fuzzness has receved ncreasng attenton. Fuzzy clusterng, whch produces overlappng cluster parttons, has been wdely studed and appled n varous areas. In fuzzy clusterng, the fuzzy c-means (FCM) clusterng algorthm s the best known and most powerful methods used n cluster analyss ([]). The dea of penalzaton s mportant n statstcal learnng. For example, rdge regresson shrnks the regresson coeffcents by mposng a penalty on ther sze. Based on penalty dea, we added a penalty term to the nter-cluster separaton (ICS) clusterng algorthm ([6]) and then proposed the penalzed ICS (PICS). Numercal comparsons are made wth several fuzzy clusterngs accordng to crtera of accuracy and computatonal effcency. MRI segmentaton provdes mportant nformaton for detectng a varety of tumors, lesons, and abnormaltes n clncal dagnoss. As descrbed by Yang et al. ([2]), most medcal mages often present overlappng gray-scale ntenstes for dfferent tssues. MRI medcal magng uncertanty s wdely presented n data because of the nose and blur n acquston and the partal volume effects orgnatng from the low resoluton of the sensors. In partcular, borders between tssues are not clearly defned and membershps n the boundary regons are ntrnscally. Therefore, fuzzy clusterng methods are sutable for the MRI segmentaton (see [4],[7], [8], [],[2]). In ths paper, the PICS algorthm s appled to the segmentaton of magnetc resonance mage (MRI) of an ophthalmc patent. In these MRI segmentaton results, we fnd that PICS provdes useful nformaton as an ad to dagnoss n ophthalmology. 2 A PICS fuzzy clusterng algorthm Let X = {x,, x n } R s be a data set and let c be a postve nteger greater than one. A partton of X nto c clusters s represented by mutually dsjont sets X,, X c such that X X c = X or equvalently by the ndcator functons u,, u c such that u (x) = f x s n X and u (x) = f x s not n X for all =,, c. Ths s known as clusterng X nto c clusters X,, X c by a hard c-partton {u,, u c }. A fuzzy extenson allows u (x) to take on values n the nterval [, ] such that = u (x) = for all x n X. In ths case, {u,, u c } s called a fuzzy c-partton of X ([?]). Thus, the FCM objectve functon J F CM s defned as

2 Proceedngs of the 6th WSEAS Internatonal Conference on Appled Computer Scence, Tenerfe, Canary Islands, Span, December 6-8, 26 4 ([]) J F CM (u, a) = n j= = c u m j x j a 2 where u = {u,, u c } s a fuzzy c-partton wth u j = u (x j ) beng the membershp of the data pont x j n cluster, a = {a,, a c } s the cluster centers, the weghtng exponent m s a fxed number greater than one establshng the degree of fuzzness and the notaton x j a denotes the Eucldean dstance between the data pont x j and the cluster center a. Thus, the FCM clusterng algorthm s an teraton through the necessary condtons for mnmzng J F CM wth the followng update equatons: j= a = um j x j, and u j = j= um j x j a 2/(m ) k= x j a k 2/(m ). The FCM algorthm s a well-known and powerful method n clusterng analyss. One of mportant parameters n the FCM s the weghtng exponent m. When m s close to one, the FCM approaches the hard c-means algorthm. When m approaches nfnty, the only soluton of the FCM wll be the mass center of the data set. Therefore, the weghtng exponent m plays an mportant role n the FCM algorthm. Recently, Yu and Yang [5] provded the theoretcal analyss for selectng the parameters n some generalzed FCM algorthms that ncludng FCM. The mxture maxmum lkelhood approach to clusterng s a remarkable model-based clusterng method. Scott and Symons [9] proposed the so-called classfcaton maxmum lkelhood (CML) procedure, named frst n Bryant and Wllamson [2], that many of the commonly used clusterng procedures correspond to applcatons of the maxmum lkelhood approach for normal groups wth varous restrctons on the covarance matrces and wth the ndcator classfcaton varables of group membershp assocated wth the data treated as unknown parameters. Yang [] made the fuzzy extenson of the CML procedure n conjuncton wth fuzzy c-parttons and called t a class of fuzzy CML procedures. On the other hand, the dea of penalzaton s mportant n statstcal learnng. For example, rdge regresson shrnks the regresson coeffcents by mposng a penalty on ther sze. Combnng the CML procedure and penalty dea, Yang ([]) added a penalty term to the FCM objectve functon J F CM and then extended the FCM to the so-called penalzed FCM (PFCM). Thus, the PFCM objectve functon s defned as follows: n J P F CM (u, a) = u m j x j a 2 j= = w n j= = c u m j ln α where w,, α and = α =. The necessary condtons for a mnmum of J P F CM (u, a) are and j= a = um j x j, α = u j = j= um j j= um j, = j= um j ( x j a 2 w ln α ) /(m ) k= ( x j a k 2 w ln α k ) /(m ). Based on the numercal results of Yang and Su [3], the PFCM s more accurant than FCM. Furthermore, the PFCM has been appled n varous areas (cf. [4], [5], [4]). On the other hand, by mnmzng the FCM objectve functon and smultaneously maxmzng the nter-cluster separaton (ICS) measure, Özdemr and Akarun [6] proposed the ICS clusterng algorthm wth the objectve functon J ICS (u, a) = n ( µ m j x j a 2 n j= = γ c c a a t 2), t= where the parameter γ. Thus, the update equatons for the ICS algorthm are as follows (see [6] and [6]): a = µ j = n j= µm j x j 2γ c t= a t j= µm j 2γ, n x j a 2/(m ) k= x j a k 2/(m ). We see that the PFCM algorthm has more accuracy than the FCM method. It means that the penalty term can mprove the performance of FCM. To mprove the performance of ICS, we consder addng the penalty term ( w j= = um j ln α ) to ICS and call the penalzed ICS (PICS). The PICS objectve functon s gven by J P ICS (u, a) = J ICS (u, a) n w u m j ln α. j= =

3 Proceedngs of the 6th WSEAS Internatonal Conference on Appled Computer Scence, Tenerfe, Canary Islands, Span, December 6-8, 26 5 The update equatons for mnmzers of J P ICS (u, a) are α = a = j= um j ; =, 2,, c, k= j= um kj n j= um j x j 2γ c t= a t j= um j 2γ ; n =, 2,, c, u j = x j a 2 w ln α ) /(m ) k= ( x j a k 2 w ln α k ) /(m ) ; =, 2,, c; j =, 2,, n. Thus, the PICS algorthm can be summarzed as follows: PICS Algorthm Set the teraton counter l = and choose the ntal values a (), =,, c and the ntal values µ () j, =,, c; j =,, n. Step. Fnd α (l+) usng (2); Step 2. Fnd a (l+) usng (3); Step 3. Fnd µ (l+) j usng (4); IF max z (l+) z (l) < ε, THEN stop; ELSE l = l + and go to step. 3 Numercal comparsons and applcaton to MRI segmentaton In ths secton, we make a comparson of four dfferent algorthms: FCM, PFCMCS and PICS, accordng to the bvarate normal mxtures of two classes under the accuracy and computatonal effcency crtera. The accuracy of an algorthm s measured by the mean squared error (MSE) that s the average sum of squared error between the true parameter and ts estmate n N repeated trals. The computatonal effcency of an algorthm s measured by the average numbers of teratons (NI) n N repeated trals. Let N 2 (a, Σ) represent the bvarate normal wth mean vector a and covarance matrx Σ. As the separaton between subpopulaton s determned by varyng the parameters of subpopulatons, wthout loss of generalty we gve that one subpopulaton bvarate normal s mean vector a = and dentty covarance matrx I and the other s mean vector a 2 and dentty covarance matrx I. That s, we consder the random sample of data drawn from α N 2 () + α 2 N 2 (a 2 ) wth α 2 = α. We also desgn varous bvarate normal mxture dstrbutons shown n Table. In Tests A and A2, we consder a well-known clusterng problem [3] where there s an nordnate dfference n the number of members n each cluster. But Test A3 has almost equal sze n each cluster and Test A4 has well-separated clusters. Table. Varous bvarate normal mxture dstrbutons for the numercal tests Test mxture model A.N 2 () +.9N 2 ( ( A2.3N 2 () +.7N 2 ( ( A3.5N 2 () +.5N 2 ( ( A4.N 2 () +.5N 2 ( ( 3 In each test, we consder the sample sze n =, ɛ =. and N = 5. The MSE s calculated by 2N N 2 k= = â (k) a 2 where â (k) s the estmated mean vector for the kth tral and a s the true mean vector. How to select m depends on the user. Because most researchers have used m = 2, we also choose m = 2 n ths secton. Next, we choose w =.5, and 2 n PFCM, and γ =.3,.5 n ICS. The numercal results are shown n Table 2. Compared PFCM wth FCM, we see that PFCM wth w =. can lead to a MSE reducton of 27.6% n Test A, 27.6% n Test A, 44.% n Test A2, 49.4% n Test A3 and 4.9% n Test A4, respectvely. These results llustrate the penalty term ( w j= = um j ln α ) to the FCM objectve functon can mprove the accuracy of FCM. Ths s why we add the penalty term ( w j= = um j ln α ) to the ICS objectve functon. Compared PICS wth ICS, we see that: () PICS (γ =.3) wth w =. can lead to a MSE reducton of 3% n Test A, 47.9% n Test A2, 48.8% n Test A3 and 9.3% n Test A4, respectvely; () PICS (γ =.5) wth w =. can also lead to a MSE reducton of 33.% n Test A, 54.9% n Test A2, 55.% n Test A3 and 9.9% n Test A4, respectvely. Based on the above results, we fnd that the reducton percentage of PICS s greater than PFCM. It llustrates that the effect of the penalty term ( w j= = um j ln α ) on ICS algorthm s sgnfcant. Moreover, we also fnd that: () PICS (γ =.3) wth w =. has the smallest MSE n Test A; ()

4 Proceedngs of the 6th WSEAS Internatonal Conference on Appled Computer Scence, Tenerfe, Canary Islands, Span, December 6-8, 26 6 PICS (γ =.5) wth w =. has the smallest MSE n Tests A2 and A3; () PFCM and PICS (γ =.3) wth w =. have good accuracy n Tests A2 and A3; (v) As we expected, fve dfferent algorthms have good accuracy n Test A4. Table 2. Accuracy and computatonal effcency for dfferent clusterng algorthms Test FCM PFCM ICS w =.5 w =. w = 2. γ =.3 γ =.5 A MSE NI A2 MSE NI A3 MSE NI A4 MSE NI Table 2. (Contnued) Test PICS (γ =.3) PICS (γ =.5) w =.5 w =. w = 2. w =.5 w =. w = 2. A MSE NI A2 MSE NI A3 MSE NI A4 MSE NI represents the smallest value. Next, we use PICS (γ =.3,.5 and w =.) n a real case study of MRI segmentaton to dfferentate between normal and abnormal tssues n ophthalmology. The MRI data sets are from a 2-yr old female patent that had been analyzed by Yang et al. [2]. She was dagnosed wth retnoblastoma of her left eye, an nborn malgnant neoplasm of the retna wth frequent metastass beyond the lacrmal crbrosa. The MRI mages showed an ntra-muscle cone tumor mass wth hgh T-weght sgnal mages and low T2-weght sgnal mages n the left eyeball. The tumor measured 2 mm n dameter and occuped nearly the entre vtreous cavty. There was a shady sgnal abnormalty all along the optc nerve reachng as far as the optc chasma near the bran. Here we analyzed two MRI data sets. The frst MRI data set s llustrated n Fgs. & 2. The second MRI data set s shown n Fg. 3. We frst attempt to cluster the full sze mages (Fgs. & 2) nto the same fve clusters as used by Yang et al. [2]. The categores are as follows: Muscle tssue, connectve tssue, nervous tssue, the lens, and tumor tssue. Accordng to Yang et al. [2], a wndow segmentaton (Fg. 3 for the second MRI data set) can be used to enhance areas of the tumor to better detect small tumors. We also apply PICS (γ =.3,.5 and w =.) to a wndow segmentaton llustrated n Fg. 3. The lens and muscle tssue are excluded from the wndow so that the orgnal fve categores are reduced to three; connectve ts-

5 Proceedngs of the 6th WSEAS Internatonal Conference on Appled Computer Scence, Tenerfe, Canary Islands, Span, December 6-8, 26 7 sue, nervous tssue and tumor tssue. A gray scale hstogram comparson shows that there are actually three peaks appearng n the segmentaton wndow mage. The two pctures (Fgs. & 2) were processed at pxels. The pctures are clustered nto four tssue classes and one tumor class. From the red crcle on the full sze two dmensonal MRI n Fg., we can clearly detect whte tumor tssue at the chasma. PICS wth γ =.3, w =. (see Fg..) and γ =.5, w =. (see Fg..2) are able to dstngush the tumor from the healthy tssue usng fve clusters. MRI medcal magng uncertanty s wdely presented n the collected data because of nose n the partal volume effects orgnatng from the low resoluton of the sensors. Another factor causng uncertanty s the fact that the eyeball moves durng the magng and t s dffcult to control ths movement, especally n younger patents. A dstorted MR mage, shown n Fg. 2, s used here to llustrate how PICS wth γ =.3, w =. and γ =.5, w =. are able to detect tumorous tssue, despte uncertanty. Fg. 3 n the second MRI data set was processed at pxels. From ths pcture, one leson was clearly seen n the MR mage. However, some fuzzy shadows of lesons were suspected of tumor nvason. These suspected abnormaltes are not easly ascertaned to be tumorous. For the purpose of detectng these abnormal tssues, a wndow of the area around the chasma s selected from the orgnal MR mages as shown n Fg. 3. We then appled PICS wth γ =.3, w =. and γ =.5, w =. to the wndow selecton as llustrated n Fgs. 3. We can see occult lesons (red crcles) clearly enhanced wth Fgs. 3. and 3.2. Ths shows that PICS wth γ =.3, w =. and γ =.5, w =. gve good results n MRI segmentaton. 4 Conclusons In ths paper we added a penalty term to the ICS algorthm [6] and then extended the ICS to the socalled penalzed ICS (PICS). Numercal comparsons are made for several fuzzy clusterngs accordng to crtera of accuracy and computatonal effcency. The results show that the PICS s better. Fnally, the PICS algorthms are appled n the segmentaton of the magnetc resonance mage (MRI) of an ophthalmc patent. In these MRI segmentaton results, we fnd that PICS provdes useful nformaton as an ad to dagnoss n ophthalmology. References: [] J.C. Bezdek, Pattern Recognton wth Fuzzy Objectve Functon Algorthms (Plenum Press, New York, 98). [2] P.G. Bryant and J.A. Wllamson, Maxmum lkelhood and classfcaton: a comparson of three approaches, n: W. Gaul and M. Schader, Eds., Classfcaton as a Tool of Research (North-Holland, Amsterdam, 986) [3] R.O. Duda and P.E. Hart, Pattern Classfcaton and Scene Analyss (Wley, New York, 973). [4] J.S. Ln, K.S. Cheng and C.W. Mao, Segmentaton of multspectral megnetc resonance mage usng penalzed fuzzy compettve learnng network, Computers and Bomedcal Research 29 (996) [5] S.H. Lu and J.S. Ln, Vector quantzaton n DCT doman usng fuzzy possblstc c- means based on penalzed and compensated constrants, Pattern Recognton 35 (22) [6] Özdemr and Akarun, Fuzzy algorthm for combned quantzaton and dtherngeee Trans. Image Processng (2) [7] D.L., Pham and J.L., Prnce, Adaptve fuzzy segmentaton of magnetc resonance mageseee Trans. Med. Imagng 8 (999) [8] W.E., Phllps, R.P., Velthuzen, S., Phuphanch, L.O., Hall, L.P., Clarke, M.L., Applcaton of fuzzy c-means segmentaton technque for dfferentaton n MR mages of a hemorrhagc globlastoma multforme, Magnetc Resonance Imagng 3 (995) [9] A.J. Scott and M.J. Symons, Clusterng methods based on lkelhood rato crtera, Bometrcs 27 (97) [] J., Sucklng, T., Sgmundsson, K., Greenwood and E.T., Bullmore, A modfed fuzzy clusterng algorthm for operator ndependent bran tssue classfcaton of dual echo MR mages. Magnetc Resonance Imagng 7 (999) [] M.S. Yang, On a class of fuzzy classfcaton maxmum lkelhood prosedures, Fuzzy Sets and Systems 57 (993) [2] M.S., Yang, Y.J., Hu, K.C.R., Ln and C.C.L., Ln, Segmentaton technques for tssue dfferentaton n MRI of Ophthalmology usng fuzzy clusterng algorthms, Magnetc Resonance Imagng 2 (22) [3] M.S. Yang and C.F. Su, On parameter estmaton for normal mxtures based on fuzzy clusterng algorthms, Fuzzy Sets and Systems 68 (994) 3-28.

6 Proceedngs of the 6th WSEAS Internatonal Conference on Appled Computer Scence, Tenerfe, Canary Islands, Span, December 6-8, 26 8 [4] M.S. Yang and N.Y. Yu, Estmaton of paramrters n latent calss models usng fuzzy clusterng algorthms, European Journal of Operatonal Research 6 (25) [5] J. Yu and M.S. Yang, Optmalty test for generalzed FCM and ts applcaton to parameter selectoneee Trans. Fuzzy Systems 3 (25) [6] J. Yu and M.S. Yang, A note on the ICS algorthm wth correcton and theoretcal analyss, IEEE Trans. Image Processng 4 (25) [7] L.A. Zadeh, Fuzzy setsnform. Control 8 (965)