A Comparative Study of Fuzzy Classification Methods on Breast Cancer Data *

Transcription

1 Comparatve Study of Fuzzy Classfcaton Methods on Breast Cancer Data * Rav. Jan, th. braham School of Computer & Informaton Scence, Unversty of South ustrala, Mawson Lakes Boulevard, Mawson Lakes, S 5095 rav.an@unsa.edu.au Computer Scence Department, Oklahoma State Unversty, 700 N Greenwood venue, Tulsa, Oklahoma OK 7406, US. ath.abraham@eee.org bstract In ths paper, we examne the performance of four fuzzy rule generaton methods on Wsconsn breast cancer data. The frst method generates fuzzy f-then rules usng the mean and the standard devaton of attrbute values. The second approach generates fuzzy f-then rules usng the hstogram of attrbutes values. The thrd procedure generates fuzzy f-then rules wth certanty of each attrbute nto homogeneous fuzzy sets. In the fourth approach, only overlappng areas are parttoned. The frst two approaches generate a sngle fuzzy f-then rule for each class by specfyng the membershp functon of each antecedent fuzzy set usng the nformaton about attrbute values of tranng patterns. The other two approaches are based on fuzzy grds wth homogeneous fuzzy parttons of each attrbute. The performance of each approach s evaluated on breast cancer data sets. Smulaton results show that the Modfed grd approach has a hgh classfcaton rate of %. Keywords: Fuzzy systems; Breast cancer dagnoss. Introducton Breast cancer s the most common cancer n women n many countres. Most breast cancers are detected as a lump/mass on the breast, or through selfexamnaton/mammography []. Screenng mammography s the best tool avalable for detectng cancerous lesons before clncal symptoms appear. Surgery through a bopsy or lumpectomy have been also been the most common methods of removal. Fne needle aspraton (FN) of breast masses s a cost-effectve, non-traumatc, and mostly nvasve dagnostc test that obtans nformaton needed to evaluate malgnancy. Recently, a new less nvasve technque, whch uses super-cooled ntrogen to freeze and shrnk a non-cancerous tumor and destroy the blood vessels feedng the growth of the tumour, has been developed [] n the US. Several rtfcal Intellgence (I) technques ncludng neural networks and fuzzy logc [3-5] are successfully appled to a wde varety of decsonmakng problems n the area of medcal dagnoss. In ths paper we examne the performance of four drect rule generaton methods that nvolve no tme-consumng tunng procedures on breast cancer data [6]. The frst method generates fuzzy f-then rules usng the mean and the standard devaton of attrbute values. The second approach generates fuzzy f-then rules usng the hstogram of attrbutes values. The thrd approach generates fuzzy fthen rules wth certanty of each attrbute nto homogeneous fuzzy sets. In the fourth approach, only overlappng areas are parttoned. Ths approach s a modfed verson of the thrd approach. In the frst two approaches, a sngle fuzzy f-then rule s generated for each class. That s, the number of fuzzy f-then rules s the same as the number of classes. These methods were reported n [7-8]. The man advantage of fuzzy rule-based systems s that they do not requre large memory storage, ther nference speed s very hgh and the users can carefully examne each fuzzy f-then rule. Ths paper s organsed as follows. Secton descrbes the characterstcs of fuzzy systems. Secton 3 descrbes rule generaton methods. Secton 4 provdes detals of the Wsconsn breast cancer data, and Secton 5 descrbes smulaton and results. The fnal secton provdes some conclusons relatng to the performance of fuzzy systems when appled to the breast cancer data.. Fuzzy Systems Fuzzy logc was nvented by Zadeh [9] n 965 for handlng uncertan and mprecse knowledge n real world applcatons. It has proved to be a powerful tool for decson-makng, and to handle and manpulate mprecse and nosy data. The frst maor commercal applcaton was n the area of cement kln control. Ths requres that an * Presented at the 7th Internatonal Work Conference on rtfcal and Natural Neural Networks, IWNN 03, Span, 003.

2 operator montor four nternal states of the kln, control four sets of operatons, and dynamcally manage 40 or 50 "rules of thumb" about ther nterrelatonshps, all wth the goal of controllng a hghly complex set of chemcal nteractons. One such rule s "If the oxygen percentage s rather hgh and the free-lme and kln-drve torque rate s normal, decrease the flow of gas and slghtly reduce the fuel rate". The noton central to fuzzy systems s that truth values (n fuzzy logc) or membershp values (n fuzzy sets) are ndcated by a value on the range [0.0,.0], wth 0.0 representng absolute Falseness and.0 representng absolute Truth. fuzzy system s characterzed by a set of lngustc statements based on expert knowledge. The expert knowledge s usually n the form of f-then rules. Defnton : fuzzy set n X s characterzed by a membershp functon whch s easly mplemented by fuzzy condtonal statements. For example, f the antecedent s true to some degree of membershp, then the consequent s also true to that same degree. If antecedent Then consequent Rule: If varable s low and varable s hgh Then output s bengn Else output s malgnant In a fuzzy classfcaton system, a case or an obect can be classfed by applyng a set of fuzzy rules based on the lngustc values of ts attrbutes. Every rule has a weght, whch s a number between 0 and, and ths s appled to the number gven by the antecedent. It nvolves dstnct parts. The frst part nvolves evaluatng the antecedent, fuzzfyng the nput and applyng any necessary fuzzy operators. For Example, Unon: Intersecton: Complement: µ B ) = Mn[ µ ), µ B )] µ B ) = Mn[ µ ), µ B )] µ ) = µ ) where µ s the membershp functon. The second part requres applcaton of that result to the consequent, known as nference. To buld a fuzzy classfcaton system, the most dffcult task s to fnd a set of fuzzy rules pertanng to the specfc classfcaton problem. fuzzy nference system s a rule-based system that uses fuzzy logc, rather than Boolean logc, to reason about data. Its basc structure ncludes four man components () a fuzzfer, whch translates crsp (real-valued) nputs nto fuzzy values; () an nference engne that apples a fuzzy reasonng mechansm to obtan a fuzzy output; (3) a defuzzfer, whch translates ths latter output nto a crsp value; and (4) a knowledge base, whch contans both an ensemble of fuzzy rules, known as the rule base, and an ensemble of membershp functons known as the database. The decson-makng process s performed by the nference engne usng the rules contaned n the rule base. These fuzzy rules defne the connecton between nput and output fuzzy varables. 3. Rule Generaton Procedure In ths secton, we explan each of four approaches examned n ths paper. The performance of each approach s examned n the next secton by computer smulatons on breast cancer data sets. Let us assume that we have an n-dmensonal c-class pattern classfcaton problem whose pattern space s an n- dmensonal unt cube [0,] n. We also assume that m patterns x p = (x p l,...,x p n ), p =,,...,m, are gven for generatng fuzzy f-then rules where x p [0,] for p =,,..., m, =,,...,n. In computer smulatons of ths paper, all attrbute values are normalzed nto the unt nterval [0,]. 3. Rule Generaton Based on the Mean and the Standard Devaton of ttrbute Values In ths approach, a sngle fuzzy f-then rule s generated for each class. The fuzzy f-then rule for the k th class can be wrtten as If x s and... and x n s n then Class k, () where s an antecedent fuzzy set for the th attrbute. The membershp functon of ) = exp µ ) ( σ ) s specfed as where µ s the mean of the th attrbute values x p of Class k patterns, and σ s the standard devaton. Fuzzy f-then rules for the two-dmensonal two-class pattern classfcaton problem are wrtten as follows: The membershp functon of each antecedent fuzzy set s specfed by the mean and the standard devaton of attrbute values (see Fgure ). For a new pattern x p = (x p3,x p4 ), the wnner rule s determned as follows: k k { } * * 3 p3 ). p4 ) = max p3 ). p4 ) k =, (3) For each attrbute, 0 membershp functons f h (), h=,,...,0 were used. The fuzzy partton was used only for calculatng the hstogram. 3. Rule Generaton Based on the Hstogram of ttrbute Values In ths method the use of hstogram an antecedent membershp functon and each attrbute s parttoned nto several fuzzy sets. We used 0 membershp functons f h (.), h=,,...,0 for each attrbute n computer smulatons as shown n Fgure. ()

3 Fgure : Mean and standard devaton of attrbutes values Fgure 3: Normalsed hstogram of class patterns 3.3 Rule Generaton of Based on Smple Fuzzy Grd Membershp 0 ttrbute Value Fgure : Fuzzy partton for calculatng the smoothed hstogram k The smoothed hstogram m ) of Class k patterns for the th attrbute s calculated usng the 0 membershp functons f h (.) as follows: ( ) k m ) = k f x p x Class k h m p for βh- x β h, h=,,...,0 where m k s the number of Class k patterns, β h, β h s the h th crsp nterval correspondng to the 0.5-level set of the membershp functon f h (.): β = 0, β0 =, (5) β h = h 0 for h=,,...,9 The smoothed hstogram n (4) s normalzed so that ts maxmum value s. n example of such a normalzed hstogram s shown n Fgure 3, whch s the hstogram of Class patterns for the 3 rd attrbute of breast cancer data. s n the frst approach based on the mean and the standard devaton, a sngle fuzzy f-then rule n () s generated for each class n the second approach. (4) (6) Fgure 4: n example of fuzzy partton In the frst two approaches, a sngle fuzzy f-then rule was generated for each class usng the nformaton about tranng patterns. On the contrary, many fuzzy f-then rules are generated n the thrd approach by parttonng each attrbute nto homogeneous fuzzy sets. In Fgure 4, we show an example of such a fuzzy partton where the two dmensonal pattern space s parttoned nto 5 fuzzy subspaces by fve fuzzy sets for each attrbute (S: small, MS: medum small, M: medum, ML: medum large, L: large). sngle fuzzy f-then rule s generated for each fuzzy subspace. Thus, the number of possble fuzzy f-then rules n Fgure 4 s 5. One dsadvantage of ths approach s that the number of possble fuzzy f-then rules exponentally ncreases wth the dmensonalty of the pattern space. For copng wth ths dffculty, some G-based rule selecton approaches have been proposed to fnd a compact rule set []. The number of fuzzy f-then rules can be also decreased by feature selecton []. Because the specfcaton of each membershp functon does not depend on any nformaton about tranng patterns, ths approach uses fuzzy f-then rules wth certanty grades. The local nformaton about tranng patterns n the correspondng fuzzy subspace s used for determnng the consequent class and the grade of certanty. 3

4 In ths approach, fuzzy f-then rules of the followng type are used: If x s and... and x n s Then n Class C, wth CF = CF, =,,..., N (7) where ndexes the number of rules, N s the total number of rules, s the antecedent fuzzy set of the th rule for the th attrbute, C ; s the consequent class, and CF s the grade of certanty. The consequent class and the grade of certanty of each rule are determned by the followng smple heurstc procedure: Step : Calculate the compatblty of each tranng pattern x p =(x p,x p,,x pn ) wth the -th fuzzy f-then rule by the followng product operaton: ( ) ( p) ( ) π x p = x... n x pn, p =,,..., m. (8) Step : For each class, calculate the sum of the compatblty grades of the tranng patterns wth the -th fuzzy f-then rule R : n βclass k ( R ) = π p ), k=,,...,c xp class k where class k ( R ) (9) β the sum of the compatblty grades of the tranng patterns n Class k wth the -th fuzzy f-then rule R. Step 3: Fnd Class value class k ( R ) β : * that has the maxmum β = Max{ ( R ),..., c ( R )} k* class class class β β (0) If two or more classes take the maxmum value or no tranng pattern compatble wth the -th fuzzy fthen rule (. e., f β Class k(r )=0 for k =,,..., c ), the consequent class C can not be determned unquely. In ths case, let C be φ. If a sngle class takes the maxmum value, the consequent class C s determned by (7). Step 4: If the consequent class C s 0, let the grade of certanty CF be CF = 0. Otherwse the grade of certanty CF s determned as follows: CF ( β * ( R ) β ) class k = c β class k k = ( R ) () where β = k = * k k β ( R ) Class k ( c ) 3.4. Rule Generaton Based on Fuzzy Partton of Overlappng reas In the thrd approach, the shape of each membershp functon was specfed wthout utlzng any nformaton about tranng patterns. smple modfcaton of the thrd approach s to partton only overlappng areas. Ths approach s llustrated n Fgure 5. small small ttrbute value (a) Smple fuzzy grd approach Overlappng area (b) Modfed fuzzy grd approach large large Fgure 5: Fuzzy partton of each attrbute Ths approach generates fuzzy f-then rules n the same manner as the smple fuzzy grd approach except for the specfcaton of each membershp functon. Because ths approach utlzes the nformaton about tranng patterns for specfyng each membershp functon as n the frst and second approaches, the performance of generated fuzzy fthen rules s good even when we do not use the certanty grade of each rule n the classfcaton phase. For example, the classfcaton boundary n Fgure 5 was obtaned by generatng nne fuzzy f then rules wthout certanty grades. In ths approach, the effect of ntroducng the certanty grade to each rule s not large when compared wth the thrd approach. In computer smulatons of the next secton, we used fuzzy f-then rules wth certanty grades n ths approach, as n the thrd approach. 4

5 4. Wsconsn Dagnostc Breast Cancer Data The Wsconsn breast cancer dataset [6] was obtaned from a repostory of a machne-learnng database Unversty of Calforna, Irvne. Ths data set has 3 attrbutes (30 realvalued nput features) and 569 nstances of whch 357 are of bengn and are of malgnant class. Table shows the statstcal detals of the data. Table : Statstcal detals of the data Class Frequency Percent Vald Percent Cumulatve Percent Total Several studes have been conducted based on ths database. For example, Bennet and Mangasaran [0] used lnear programmng technques, obtanng a 99.6% classfcaton rate on 487 cases (the reduced database avalable at the tme). However, dagnostc decsons are essentally black boxes, wth no explanaton as to how they were attaned. Fgure 6 shows 3D plot of data. Ten real-valued features are computed for each cell nucleus: a) radus (mean of dstances from center to ponts on the permeter) b) texture (standard devaton of gray-scale values) c) permeter d) area e) smoothness (local varaton n radus lengths) f) compactness (permeter^ / area -.0) g) concavty (severty of concave portons of the contour) h) concave ponts (number of concave portons of the contour) ) symmetry ) fractal dmenson ("coastlne approxmaton" - ) Fgure 6: 3D plot of Wsconsn data 5. Smulaton Results and Dscussons We examned the performance of four dfferent approaches, and the emprcal results are summarzed n Table. Table. Classfcaton rates for breast cancer data Mean and Standard Devaton 9.% Hstogram 86.7% Smple Grd 99.73% Modfed Grd 6.57% s evdent, the performance of smple grd and mean and standard devaton s comparable. But the performance of hstogram and modfed grd approaches s not good enough wth the other approaches. Ths s because n the hstogram approach a sngle fuzzy rule s not enough for each class and the classfcaton of some patterns was reected and n the case of the modfed grd approach the number of fuzzy f-then rules s ncreased exponentally wth the dmensonalty of pattern space. Smple grd approach gave the overall best results wth a classfcaton accuracy of 99.73%. Rule generaton usng mean and standard devaton s easy to mplement as t depends only on the mean and standard devaton of the attrbute values. The modfed grd approach dd not produce the desred accuracy. Moreover, n the grd-based approach, the number of fuzzy f-then rules exponentally ncreased wth the dmensonalty of the pattern space. Thus, a large number of fuzzy f-then rules are usually generated for real-world pattern classfcaton problems. Ths leads to several drawbacks: over-fttng tranng patterns, large memory storage requrement, and slow nference speed. On the contrary, the numbers of fuzzy f-then rules n the frst two approaches are the same as the number of classes. 5

6 6. Concluson and Dscussons In ths paper, we examned the performance of four fuzzy rule generaton methods that could generate fuzzy f-then rules drectly from tranng patterns wth no tmeconsumng tunng procedures. In the frst approach, a sngle fuzzy f-then rule was generated for each class usng the mean and the standard devaton of attrbute values. In the second approach, a sngle fuzzy f-then rule was generated for each class usng the hstogram of attrbute values. The thrd approach generated fuzzy f-then rules by homogeneously parttonng each attrbute. Thus, a pattern space was parttoned nto a smple fuzzy grd. The nformaton about attrbute values was not used for specfyng the membershp functon of each antecedent fuzzy set. The local nformaton of tranng patterns was utlzed when the consequent class and the certanty grade were specfed. The last approach was a modfed verson of the smple fuzzy grd approach. s llustrated n Table, smple grd approach gave the best performance overall whle the mean and standard devaton approach also performed reasonably well. It may be noted that a sngle fuzzy f-then rule for each class s not always suffcent for real-world pattern classfcaton problems. Whle each approach s very smple and has some drawbacks as dscussed above, fuzzy rule-based systems have hgh classfcaton ablty as shown n ths paper. The performance of fuzzy rule based systems can be further mproved by feature selecton and optmzng the rule selecton and varous rule parameters. cknowledgements The authors are grateful to the edtor of ths ournal, John Pattson, revewers (for constructve comments) and a number of readers ncludng Robyn Vast for readng and correctng ths paper. References [] DeSlva, C.J.S. et al. rtfcal Neural networks and Breast Cancer Prognoss The ustralan Computer Journal, 6, pp. 78-8, (994). [] The Weekend ustralan, Health Secton, p 7. July, 3-4, (00). [5] Cos, K.J., et.al. "Usng Fuzzy Sets to Dagnose Coronary rtery Stenos", IEEE Computer, pp , (99). [6] Merz J., and Murphy, P.M., UCI repostory of machne learnng databases. - learn/mlrepostory.html, (996). [7] Jan R. and braham., Comparatve Study of Fuzzy Classfers on Breast Cancer Data, 7th Internatonal Work Conference on rtfcal and Natural Neural Networks, Lecture Notes n Computer Scence- Volume 687, Jose Mra and Jose R. lverez (Eds.), Sprnger Verlag, Germany, pp. 5-59, 003. [8] Ishbuch, H., and Nakashma, T., Study on Generatng Classfcaton Rules Usng Hstogram Edted by Jan L.C., and Jan, R.K., KES 98, pp. 3-40, (998). [9] Zadeh, L.., Fuzzy Logc IEEE Computer, pp (988). [0] Bennett, K.,P., and Mangasaran, O.L.,. Neural network tranng va lnear programmng. In P. M. Pardalos, edtor, dvances n Optmzaton and Parallel Computng, pages Elsever Scence, (99). [] Ishbuch, H. et.al, fuzzy classfer system that generates fuzzy f-then rules for pattern classfcaton problems, Proc. Int. Conf. Evolutonary Computat. Perth, ustrala, vol., pp , (995). [] 3. H. Ishbuch, H., Nakashma,T., and Morsawa, T.,"Smple fuzzy rule-based classfcaton systems performed well on commonly used real-world data sets," Proc. of North mercan Fuzzy Informaton Processng Socety Meetng, Syracuse, pp -4, (997) [3] 4. Nakashma,T., Ishbuch,H., and Morsawa,T., "Input selecton n fuzzy rule-based classfcaton systems," Proc. of 6th Internatonal conference on Fuzzy Systems, Barcelona, Span, July -5, pp , (997). [3] Hayash, Y., "Neural Expert System Usng Fuzzy Teachng Input and Its pplcaton to Medcal Dagnoss", Proceedngs of the Second Internatonal Conference on Fuzzy Logc and Neural Networks, Izuka, Japan, pp , (99). [4] Watanabe, H. et.al. The pplcaton of a Fuzzy Dscrmnaton nalyss for dagnoss of Valvular Heart Dsease IEEE trans. on Fuzzy Systems, (994). 6