Unsupervised Classification Using Immune Algorithm

Transcription

1 Internatonal Journal of Computer Applcatons ( ) Volume 2 o.7, June 2 Unsupervse Classfcaton Usng Immune Algorthm M.T. Al-Muallm Department of Computer Engneerng & Automaton, Faculty of Mechancal an Electrcal Engneerng, Damascus Unversty, Syra R. El-Kouatly Department of etworkng an systems, Faculty of Informaton technology Engneerng Damascus, Syra ABSTRACT Unsupervse classfcaton algorthm base on clonal selecton prncple name Unsupervse Clonal Selecton Classfcaton (UCSC) s propose n ths paper. The new propose algorthm s rven an self-aaptve, t austs ts parameters to the to make the classfcaton operaton as fast as possble. The performance of UCSC s evaluate by comparng t wth the well known K-means algorthm usng several artfcal an real-lfe sets. The experments show that the propose UCSC algorthm s more relable an has hgh classfcaton precson comparng to tratonal classfcaton methos such as K-means. General Terms Pattern Recognton, Algorthms. Keywors Artfcal Immune Systems, Clonal Selecton Algorthms, Clusterng, K-means Algorthm.. ITRODUCTIO Unsupervse classfcaton whch also known as clusterng s efne as the problem of classfyng a collecton of obects nto a set of natural clusters wthout any a pror knowlege. Many clusterng methos were propose. These methos can be bascally classfe nto two categores: herarchcal an parttonal. In contrast to herarchcal clusterng, whch yels a successve level of clusters by teratve fusons or vsons, parttonal clusterng assgns a set of ponts nto K clusters wthout any herarchcal structure. Ths process usually accompanes the optmzaton of a crteron functon []. One of the wely use parttonal metho s the K-means algorthm. It s an teratve hll-clmbng algorthm. Many of stochastc optmzaton methos were use n clusterng problem Such as genetc algorthms GAs [2-5], smulate annealng (SA) [6-8] an tabu search (TS) [9]. Artfcal Immune Systems (AIS) s a fel of stuy evote to the evelopment of computatonal moels base on the prncples of the bologcal mmune system. It s an emergng area that explores an employs fferent mmunologcal mechansms to solve computatonal problems [] A lot of mmune algorthms were evelope amng to fnng solutons to a broa class of complex problems. Applcatons of AIS have nclue the followng areas: clusterng an classfcaton [-5], anomaly etecton [6-8], optmzaton [3, 9], control [2-2], computer securty, learnng, bo-nformatcs, mage processng, robotcs, vrus etecton an web mnng [22]. Many of the mmune algorthms use prncples nspre by the clonal selecton theory of acqure mmunty. The clonal selecton prncple s use by the mmune system to escrbe the basc features of an mmune response to an antgenc stmulus. It establshes the ea that only those cells that recognze the antgens prolferate, thus beng selecte aganst those whch o not. The process of prolferatng calle clonal expanson. The selecte cells are subect to an affnty maturaton process whch mproves ther affnty to the selectve antgens [23]. The frst clonal selecton algorthm was propose by [3] whch name CLOALG an use for optmzaton. Other versons of clonal selecton algorthm are esgne to mprove the performance of CLOALG such as [24-26]. In ths stuy, we tre to nvestgate the possblty of usng clonal selecton algorthm as stochastc optmzaton methos for clusterng. Unsupervse classfcaton algorthm base on clonal selecton prncple name Unsupervse Clonal Selecton Classfcaton (UCSC) s propose. The new algorthm s rven an self-aaptve, t austs ts parameters to the to make the classfcaton operaton as fast as possble. The propose approach has been teste on several artfcal an reallfe sets an ts performance s compare wth the well known K-means algorthm []. Experments show that UCSC algorthm s more relable an has hgh classfcaton precson comparng to tratonal classfcaton methos such as K-means. 2. USUPERVISED CLOAL SELECTIO CLASSIFICATIO (UCSC) 2. Basc Prncple In UCSC, clusterng problem s consere as optmzaton problem an the obectve s to fn the optmal parttons of where the resultng clusters ten to be compact as possble. A smple crteron whch s the wthn cluster sprea s use n UCSC, ths crteron nees to be mnmze for goo clusterng. Unlke K-means whch uses the square-error crteron to measure the wthn cluster sprea, UCSC uses the sum of the Euclean Dstances of the ponts from ther respectve cluster centros as clusterng metrc an uses clonal selecton algorthm as clusterng algorthm whch ensures fnng the global optma when most of others algorthms such as K-means stuck nto local optma. The number of clusters K s suppose to be known an the approprate cluster centers m, m 2,,m k have to be foun such that the clusterng metrc J s mnmze. Mathematcally, 44

2 Internatonal Journal of Computer Applcatons ( ) Volume 2 o.7, June 2 the clusterng metrc J for the K clusters C={C, C 2,, C K } s gven by the followng equaton: K J(,M) x m () where x, =,, are ponts, { } s a partton matrx wtch gven by the eq. (2), M s centro matrx wtch gven by the eq. (3) an m, =,, K s the mean for the C cluster wth ponts. f x C wth oterwse M=[m, m 2,,m K] where K m x (2), =,, K (3) The task of clonal selecton algorthm s to search for the approprate cluster centers wherefore J s mnmze. Base on clusterng crteron, UCSC s suppose to gve rght results f the clusters are compact an hypersphercal n shape. 2.2 Clonal Selecton Algorthm All clonal selecton algorthms have the same basc steps whch are sumerze as follows: generate populaton P of antboes (canate solutons) whle stoppng crteron s not met o { clone P base on ther affnty Submt the result populaton to hypermutaton scheme select the hghest affnty soluton to form new populaton mantan versty n the populaton } select the hghest affnty antboy to form the mmune memory whch s the soluton to the problem. There are certan crtcal ssues that must be taken nto conseraton whle esgnng an runnng a clonal selecton algorthm such as representng the soluton, mantanng versty n populaton, affnty metrc an hypermutaton mechansm. Even a small change n any of these aspects may lea to a conserable change n the performance of clonal selecton algorthms [26]. The UCSC algorthm s summarze as follows: Intalzaton: generate populaton P of n antboes (canate solutons) ranomly For every generaton o: { affnty measure of all antboes n P clone P generatng a populaton PC submt PC to hypermutaton scheme generatng Pm consolate P & Pm affnty measure of all antboes re-select the n hghest affnty to form P Generate new L nvuals (ranomly) replace the L lowest affnty antboes n P wth the new ones } fnally: select the hghest affnty antboy n P whch s the soluton These steps wll be escrbe n etals next. 2.3 Soluton Representng Each antboy n P forms a strng of real numbers representng the K cluster centers. For -mensonal space, the length of the strng s *K number, where the coornates of the centers are localze n sequence. p Ab, Ab,..., Ab ] (4) [ 2 n l [ m, m2,..., m, m2,..., mk Ab ], l =,, n (5) The frst numbers represent the mensons of the frst cluster center; the next postons represent those of the secon cluster center, an so on. 2.4 Affnty Metrc To measure the affnty of an antboy, the clusters are forme accorng to the centers encoe n the antboy uner conseraton, ths s one by assgnng each pont x, =,, to one of the clusters C whose center are the closest to the pont. After the clusterng s one, the new cluster centros are calculate by fnng the mean ponts of the respectve clusters, then clusterng crteron J s calculate by eq. (). The affnty s efne as: aff J The maxmum value of the affnty stanng for the mnmum value of J. Zero s assgne to the affnty f any cluster becomes empty. 2.5 Clonng Antboes n P wll be clone proportonally to ther affntes, the hgher the affnty the hgher the number of clones generate for the antboy. The antboes were sorte n escenng orer accorng to ther affnty an then the amount of clones generate for the antboes was gven by: n nc l roun (7) l Where nc l s number of clones an β s clonal factor. 2.6 Hypermutaton Mechansm Every antboy n P C s submtte to a mutaton that s nversely proportonal to the affnty an ths s one accorng to the followng equatons: * Ab Ab (,) (8) aff e (9) Where Ab * s the resultng antboy of mutate Ab, (,) s a matrx of *K Gaussan ranom varables wth zero mean an stanar evaton =, aff s the affnty of the antboy, whch s normalze n the range [ ].α s a factor that reszes the value of the Gaussan mutaton an t s nversely proportonal to (6) 45

3 Internatonal Journal of Computer Applcatons ( ) Volume 2 o.7, June 2 the affnty, s a factor that controls the range of α. To make the algorthm fast an rven ths factor s gven as: (max mn) () Where max an mn are the maxmum an the mnmum values of the features at all mensons. In ths way, the mutaton probablty epens on the affnty of the antboy an also on the scope of search. 2.7 ew Antboes Generator To generate new ranom solutons, the scope of search whch s the strbuton range n the feature space was etermne. The range of was calculate usng the upper an the lower lmt of the at every menson: UL UL, UL,, UL ] () [ 2 [ LL, LL2,, LL LL ] (2) Where UL, an LL are matrxes of the upper an lower lmts of the features respectvely. Then new ranom soluton s generate usng: Ab LL )) new T T ( ag(( UL LL) ran (3) Where ran s a matrx of *K ranom varables wth unform probablty strbuton wthn the range [ ]. Ths ranom solutons generator s use to nsure a fast an accurate performance of UCSC an accelerate the convergence rate of the algorthm snce all solutons are n the scope of search. 3. EXPERIMETS The UCSC was teste usng several artfcal an real-lfe sets, then compare wth the well known K-means algorthm []. The UCSC was teste wth the followng parameters n=, β =5, L=4, an number of generatons gen=3. For K-means algorthm [] as a maxmum number of teratons was use n case t oes not termnate normally. At every experment the algorthms were run for tmes wth fferent ranom ntal confguratons To prove statstcal evaluaton of the performance. The sets are escrbe below: 3. Artfcal Data Sets Dataset : An artfcal set consstng of overlappng two classes ( patterns each) wth bvarate Gaussan ensty wth the followng parameters: m =(.,.), m 2=(.35,.), 2.. The set s shown n Fgure ()., where Σ s covarance matrx. Fgure.: An artfcal set wth two classes. Dataset 2: An artfcal set consstng of nne classes (25 patterns each) wth bvarate Gaussan ensty wth the followng parameters: m =(.,.), m 2=(.,.5), m 3=(.,.9), m 4=(.5,.), m 5=(.5,.5), m 6=(.5,.9), m 7=(.9,.), m 8=(.9,.5), m 9=(.9,.9), The set s shown n Fgure (2) Fgure.2: An artfcal set 2 wth nne classes Dataset 3: An artfcal set consstng of three classes (5 patterns each) wth trpartte Gaussan ensty wth the followng parameters: m =(,, ), m 2=(2, 2.5, 2.5), m 3=(2, 3, 3), 46

4 Internatonal Journal of Computer Applcatons ( ) Volume 2 o.7, June The set s shown n Fgure (3). Table : classfcaton result for all sets Dataset UCSC K-means Dataset 88% 86% Dataset % 97.33% Dataset % 9.33% Irs set 9% 89.33% Breast Cancer set 96.% 95.7% The experments show that K-means algorthm got stuck at suboptmal solutons even for smple but UCSC not exhbt any such behavor. Table (2) shows the best values of J an ts percentages of the total runs of UCSC an K-means algorthms for every set. As we can see from Table (2) for all sets, UCSC fns better solutons than K-means algorthm an the clusters forme by UCSC are more compact than those forme by K-means algorthm. The results show that UCSC algorthm s more relable than K-means algorthm because t fns the best soluton all the tme unlke K-means whch not fn the best soluton all the tmes. Fgure.3: An artfcal set 3 wth three classes. 3.2 Real-lfe Data Sets The followng real-lfe set has beng teste: - Irs set [27] conssts of 5 four-mensonal patterns n three classes (5 patterns each) represent fferent categores of rs flowers whch have four feature values. The four feature values represent the sepal length, sepal wth, petal length an the petal wth n centmeters. The three classes are: Irs Setosa, Irs Verscolor an Irs Vrgnca. 2- Wsconsn Breast Cancer set [28] conssts of 699 nne-mensonal patterns n two classes whch are Bengn (458 patterns) an Malgnant (24 patterns). The nne features are: Clump Thckness, Unformty of Cell Sze, Unformty of Cell Shape, Margnal Aheson, Sngle Epthelal Cell Sze, Bare ucle, Blan Chromatn, ormal ucleol an Mtoses. 4. RESULTS The best classfcaton results of UCSC an K-means after run for tmes are shown n Table () whch nclues the obtane classfcaton accuracy for all sets. As we can see from Table () the UCSC algorthm proves better accuracy compare wth K-means algorthm. Dataset Table 2: Values of J for the fferent sets. UCSC K-means J percents J percents Dataset 25.4 % % Dataset % % Dataset % % Irs set 97. % % Breast set Cancer % 35.3 % For all experments UCSC foun the soluton n less than 3 generatons 5. DISCUSSIO means algorthm usng several artfcal an real-lfe sets. The experments show that the propose UCSC algorthm s more relable because t fns the best soluton all the tme unlke K-means whch got stuck at sub-optmal solutons. UCSC algorthm has hgh classfcaton precson comparng to K-means algorthm. The new propose algorthm s rven an self-aaptve, t austs ts parameters to the to make the classfcaton operaton as fast as possble. UCSC algorthm has many avantages comparng to other evolutonary algorthms. One s the small populaton sze n= where most of other evolutonary algorthms nee at lest populaton sze of. Secon t foun the soluton n less than 3 generatons. 47

5 Internatonal Journal of Computer Applcatons ( ) Volume 2 o.7, June 2 Base on clusterng crteron use n UCSC t suppose to gve rght results f the clusters are compact an hypersphercal n shape. 6. COCLUSIO In ths paper, unsupervse classfcaton algorthm base on clonal selecton prncple name Unsupervse Clonal Selecton Classfcaton (UCSC) s esgne to fn the optmal partton between the. It uses wthn cluster sprea crteron as a clusterng crteron. The crteron s base on Euclean stance between the n the clusters. The new algorthm s rven an self-aaptve, t austs ts parameters to the to make the classfcaton operaton as fast as possble. UCSC s teste on several artfcal an real-lfe sets an ts performance s compare wth the well known K-means algorthm []. The experments show that UCSC algorthm has classfcaton precson hgher than K-means algorthm whch got stuck at sub-optmal solutons even for smple sets. The new algorthm fns the soluton n thrty generatons only an t uses a small populaton sze n= where most of other evolutonary algorthms nee at lest populaton sze of. UCSC gves goo results f the clusters are compact an hypersphercal n shape 7. REFERECES [] R. Xu, et al., Clusterng. Hoboken, ew Jersey: John Wley & Sons, Inc, 29. [2] G. Babu an M. Murty, "Clusterng wth evoluton strateges," Pattern Recognton, vol. 27, pp , 994. [3] L. Hall, et al., "Clusterng wth a genetcally optmze approach," IEEE Transactons on Evolutonary Computaton, pp. 3 2, 999. [4] U. Maulk an S. Banyopahyay, "Genetc algorthm-base clusterng technque," Pattern Recognton, vol. 33, pp , 2. [5] L. Y. Tseng an S. B. Yang, "A genetc approach to the automatc clusterng problem," Pattern Recognton vol. 34, pp , 2. [6] D. Brown an C. Huntley, "A practcal applcaton of smulate annealng to clusterng," Pattern Recognton, vol. 25, pp. 4-42, 992. [7] R. Klen an R. Dubes "Experments n proecton an clusterng by smulate annealng," Pattern Recognton vol. 22, pp , 989. [8] S. Selm an K. Alsultan, "A smulate annealng algorthm for the clusterng problems," Pattern Recognton vol. 24, pp. 3-8, 99. [9] K. Al-Sultan, "A Tabu search approach to the clusterng problem," Pattern Recognton, vol. 28, pp , 995. [] D. Dasgupta an L. F. ño, Immunologcal Computaton Theory an Applcatons. Boca Raton: CRC Prees, Taylor & Francs Group, 29. [] L.. e Castro an F. J. Von Zuben, "An evolutonary mmune network for clusterng," presente at the Proceeng of Sxth Brazlan Symposum on eural etworks, Ro e Janero, RJ, 2. [2] L.. e Castro an F. J. Von Zuben, "The Clonal Selecton Algorthm wth Engneerng Applcatons," presente at the Proceengs of GECCO, Workshop on Artfcal Immune Systems an Ther Applcatons, Las Vegas, USA, 2. [3] L.. e Castro an F. J. Von Zuben, "Learnng an Optmzaton Usng the Clonal Selecton Prncple," IEEE Transactons on Evolutonary Computaton, vol. 6, pp , 22. [4] L.. e Castro an F. J. Von Zuben, "aet: An Artfcal Immune etwork for Data Analyss," presente at the n Data Mnng: A Heurstc Approach, Iea Group Publshng, USA, 2. [5] A. Watkns an J. Tmms, "Artfcal Immune Recognton System (AIRS): An Immune-Inspre Supervse Learnng Algorthm," Journal Genetc Programmng an Evolvable Machnes, vol. 5, pp , 24. [6] D. Dasgupta an S. Forrest, "ovelty Detecton n Tme Seres Data usng Ieas from Immunology," presente at the Proceengs of The 5th Internatonal Conference on Intellgent Systems, Reno, evaa, 995. [7] P. D'haeseleer, et al., "An Immunologcal Approach to Change Detecton: Algorthms, Analsys an Implcatons," presente at the IEEE Symposum on Securty an Prvacy, Oaklan, CA, 996. [8] S. Forrest, et al., "Self-onself Dscrmnaton n a Computer," presente at the IEEE Computer Socety Symposum on Research n Securty an Prvacy, Oaklan, CA, 994. [9] L.. e Castro an J. Tmms, "An artfcal mmune network for multmoal functon optmzaton," presente at the Proceengs of the 22 Congress on Evolutonary Computaton, Honolulu, HI, 22. [2] F. Gumarães, et al., "Desgn of mxe control systems usng algorthms nspre by the mmune system," Informaton Scence, vol. 77, pp , 5 October 27. [2] D. H. Km an J. H. Cho, "Intellgent tunng of PID controller wth sturbance functon usng mmune algorthm," presente at the IEEE Internatonal Conference on Computatonal Intellgence for Measurement Systems an Applcatons, 24. [22] E. Hart an J. Tmms, "Applcaton areas of AIS: The past, the present an the future," Apple Soft Computng, vol. 8, pp. 9-2, January 28. [23] L.. e Castro an J. Tmms, Artfcal Immune Systems: A ew Computatonal Intellgence Approach: Sprnger, 22. [24] V. Cutello, et al., "Real coe clonal selecton algorthm for unconstrane global optmzaton usng a hybr nversely proportonal hypermutaton operator," presente at the Symposum on Apple Computng archve, Don, France 26. [25] V. Cutello, et al., "An Immune Algorthm for Proten Structure Precton on Lattce Moels," IEEE Transactons on Evolutonary Computaton, vol., pp. -7, 27. [26]. Khlwan, et al., "Fast clonal algorthm," Engneerng Applcatons of Artfcal Intellgence, vol. 2, pp. 6-28, 28. [27] R. A. Fsher, "The use of multple measurements n taxonomc Problems," Ann. Eugencs, vol. 3, pp , 936. [28] O. L. Mangasaran an W. H. Wolberg, "Cancer agnoss va lnear programmng," SIAM ews, vol. 23, pp. -8,