A Serial and Parallel Genetic Based Learning Algorithm for Bayesian Classifier to Predict Metabolic Syndrome

Transcription

1 A Seral and Parallel Genetc Based Learnng Algorthm for Bayesan Classfer to Predct Metabolc Syndrome S. Dehur Department of Informaton and Communcaton Technology Fakr Mohan Unversty, Vyasa Vhar Balasore , Orssa, Inda B. S. P Mshra, R. Roy School of Computer Engneerng KIIT Unversty Bhubaneswar , Orssa, Inda mshra.bsp@gmal.com, lnk2rahulroy@gmal.com S. -B. Cho Soft Computng Laboratory Department of Computer Scence Yonse Unversty 262 Seongsanno, Seodaemun-gu, Seoul , South Korea sbcho@cs.yonse.ac.kr ABSTRACT Ths paper presents a seral and parallel genetc based learnable bayesan classfer for desgnng a prognostc model for metabolc syndrome. The objectve of the classfer s to address the fundamental problem of fndng the optmal weght n the learnable bayesan classfer, by seral GA, and mnmze the response tme by parallel GA. The algorthms exhbt an mproved capablty to elmnate spurous features from the large dataset and ad the researchers n dentfyng those features that are solely responsble for hgh predcton accuracy. The effectveness of the classfer are demonstrated usng metabolc syndrome dataset obtaned from Yonchon County of Korea. Categores and Subject Descrptors H.2.8 [Database management]: Database Applcatons- Data mnng General Terms Algorthms, Performance, Desgn, Expermentaton, Keywords Genetc Algorthm, parallel genetc algorthm, learnable nave Bayesan classfer, metabolc syndrome dseases. Permsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. To copy otherwse, to republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. COMPUTE 11, March 25-26, Bangalore, Karnataka, Inda Copyrght 2011 ACM /11/03...$ INTRODUCTION The metabolc syndrome s a collecton of metabolc dsorder whch ncludes hypertenson, dyslpdema, elevated blood glucose and obesty. Ths dsorder s more common n the adults of 20 years and above. Approxmately 34% of adults met the crtera for metabolc syndrome. Males and females years of age were about three tmes as lkely as those years of age to meet the crtera for metabolc syndrome. In Asan countres, t has become a sgnfcant problem lately due to the change n detary habt and lfe style. Approxmately 5 17% of the males and females of the age years have the prevalence of metabolc syndrome dseases as reported n [9]. Due to the growng nature of the dsease, t has become an area of nterest among the researchers of the world. Bayesan classfers are the smple and yet effectve statstcal based classfer that have been appled to many complex domans. In medcal doman, we can fnd varous prognostc models based on Bayesan classfer [3, 13, 11]. In ths paper, a seral genetc aded learnable Bayesan classfer s proposed where the weghts are determned usng genetc algorthm thereby ncreasng the predctve accuracy. In order to mnmze the response tme, ths paper also presents a parallel genetc based learnng algorthm for the classfer. The performance of the classfer s evaluated wth a real world dataset obtaned from Yonchon County of Korea. The mpact of parameter settng of the genetc operator on the classfcaton accuracy s studed here. Also a comparatve study of the applcaton of smple genetc algorthm and parallel genetc algorthm n the desgn of classfer s carred out based on the classfcaton accuracy and CPU tme (n sec). The rest of the secton s arranged as follows: Secton 2 presents the bascs concepts of learnable nave Bayesan classfer, genetc algorthm and parallel genetc algorthm. Secton 3 and 4 descrbes the seral and parallel genetc aded learnable Bayesan classfer. In Secton 5, we provde the expermental results and analyss of the modelng of the metabolc syndrome. Fnally, concluson s provded n Secton 6.

2 2. BASIC CONCEPTS Ths Secton provdes some of the basc concepts of learnable nave Bayesan classfer, genetc algorthm and parallel genetc algorthms. 2.1 Learnable Bayesan Classfer The nave Bayesan classfer s a probablstc approach to classfcaton. Gven an unclassfed object X =(x 1,x 2..., x n ) the classfer predcts that X belongs to the category havng the hghest posteror probablty condtoned on X. Specfcally, ths classfes object X nto category C f and only f P( C X ) P( C j X ) j The P( C X ) s calculated usng the Bayes Theorem. P C X PX C PC PX C P( C ) The tranng sample s used to determnng the requred q P s calculated as PC where q s q probabltes. The C the total number of samples and q s number of samples of class C. The crucal part of the formulaton of the Bayesan model s the determnaton of P(X C ). The determnaton of P(X C ) can be extremely computatonally expensve and may requre large tranng samples. The nave Bayesan model makes the smplfyng assumpton that P n X C Px C k1 Ths assumpton, called attrbutes condtonal ndependence, greatly smplfes the calculaton. However, nave Bayesan classfer does not provde any assurance for accurate classfcaton of tranng dataset. The tranng set s only used for determnng the probabltes. Thus Yager et al. [13] provded an extenson for nave Bayesan classfer by ntroducng learnng weghts n the model. The formulaton for the P(X C ) s shown n equaton 1 P n x C wj P xk C j1 k1 j Where P(x k C ) s the k th largest of the P(x j C ) and w j =[0, 1] and w j =1. The ntroducton of these weghts provdes wth the addtonal degree of freedom n the form of the assocated weghts. The ncluson of weghts provdes a more general model for the classfer. However, t adds to the classfer the problem of determnng the optmal weghts. The GA can be used as a tool for determnng the optmal values for the weghts. 2.2 Genetc Algorthms In early 1960s several computer scentsts ndependently studed evolutonary systems wth the dea that evoluton could be used as an optmzaton tool for engneerng problems [4, 7, 2]. GAs was k (1) frst descrbed by John Holland n 1960s and further developed by Holland and hs students and colleagues at the Unversty of Mchgan n the 1960s and 1970s. Holland s 1975 book Adaptaton n Natural and Artfcal System presents the GA as an abstracton of bologcal evoluton and gves a theoretcal framework for adaptaton under the GA. Holland s GA s a method for movng from one populaton of chromosomes (e.g., bt strngs representng organsms or canddate solutons to a problem) to a new populaton, usng selecton together wth the genetc operators of crossover, and mutaton. Each chromosomes conssts of genes (e.g., bts), wth gene beng an nstance of a partcular allele (e.g., 0 or 1). Selecton chooses those chromosomes n the populaton that wll be allowed to reproduce, and decde how many offsprng than less ft ones. Crossover exchanges subparts of two chromosomes (roughly mmckng sexual recombnaton between two sngle-chromosome organsms); mutaton randomly changes the values of some locatons n the chromosome and nverson reverses the order of a contguous secton of the chromosome, thus rearrangng the order n whch genes are arrayed n the chromosome. Inverson s rarely used n to-days GAs at least partally because of the mplementaton expense for most representatons. A smple algorthmc form of the GA (wthout nverson) works as follows: Genetc Algorthm Begn Generate an ntal populaton. Evaluate each ndvdual of the populaton. Whle Stop crteron s not reached do Select ndvduals. Perform crossover /* Mutaton wth respectve probabltes*/. Evaluate the resultng chldren. Replacement of the chldren wth parents. End whle Ths process s terated over many generatons. After several generatons, the result s often one or more hghly ft chromosomes n the populaton. We say that a populaton has converged when all the ndvduals are very much alke and further mprovement may only be possble by a favorable mutaton. [1, 6, 12, 8]. 2.3 Parallel Genetc Algorthms The basc dea behnd most parallel programs s to dvde a large problem nto smaller ones and solve each one smultaneously usng multple processors. The hope s that such a dvde and conquer prncple s more effcent, f all processors work on the whole problems. One of the approaches for parallelzng GAs s to use global parallelzaton. In ths parallel GAs, there s a sngle panmctc populaton (just as n a smple GA), but the evaluaton of ftness s dstrbuted among several processors (Fgure 1). There s no communcaton between the processors durng the evaluaton because the ftness of each ndvdual s ndependent from all the others. Communcaton only occurs at the start and at the end of the evaluaton phase.

3 crossover. The real valued crossover s used for updatng each of the weghts and the mask bts are updated usng the one pont ( 1, t1) bnary valued crossover. The updatng of the weghts ( x ) usng real valued crossover s done by equaton (3) Fgure 1: Master Slave Model 3. GENETIC ALGORITHM AIDED BAYESIAN CLASSIFIER The more general famly of nave Bayesan classfer can be constructed usng the equaton (1). The ncluson of weghts provdes a more general model for the classfer. However, t adds to the classfer the problem of determnng the optmal weghts. The genetc algorthm can be used as a tool for determnng the optmal values for the weghts. 3.1 Chromosome Representaton The chromosome of ndvdual s represented as a 2 n matrx where n s the number of the attrbutes. The frst row represent the weghts as real values n the range [0, 1] and the second row represents the mask bt that determnes the ncluson or excluson of the weghts n determnng the probablty of the each attrbute for a gven class C. The pctoral representaton of the weght s shown n Fgure 2. Fgure 2: Chromosome representaton of an ndvdual 3.2 Ftness Evaluaton Durng the executon of the algorthm, each ndvdual s passed through the classfer for evaluaton and the cost s computed based on the classfcaton accuracy obtaned from the parameterzed generalzed Bayesan formulaton n classfyng the known set of samples of known class. The genetc algorthm seeks to maxmze the cost functon. The mathematcal formulaton of the cost functon ( f (x) ) s gven n equaton (2); f ( x) CLaccu, (2) Where CL accu s the classfcaton accuracy. 3.3 Genetc Operators The three genetc operator used for generaton of new set of soluton are selecton, crossover and mutaton. The tournament selecton operator s used for selecton of good ndvduals nto the matng pool. Two ndvduals from the populaton are randomly selected and the one wth the hghest ftness s selected for the matng pool. The selecton of the ndvduals for the tournament s done usng the roulette wheel selecton. The selecton operator s executed untl the sze of the matng pool becomes equal to the sze of populaton. Snce the chromosome s a combnaton of real and bnary values, thus we adopt combnaton of both real valued and bnary valued where (1, t1) (1, t) (2, t) x 0.5[(1 q) x (1 q) x ] (2, t1) (1, t) (2, t) x 0.5[(1 q) x (1 q) x ] 1 nc 1 (2u ) f q 1 1 nc u u 0.5 otherwse Here β s known as the spread factor whch s obtaned by equatng the area under the probablty curve equal to u, where u s a random number between [0 1]. In one pont bnary valued crossover we produce the chld populaton generated by selectng a one pont randomly and swappng the values from the crossover ste among the two parents. The SBX-mutaton operator [5] s used for updatng the real valued weght whereas the normal mutaton operator s used for updatng the mask bts. The SBX-mutaton operator s formulated as gven n equaton (4). where 1, t1 (1, t1) U y x ( x x 1/( (2r ) 1 (2r ) Here the parameter 1) m 1 1/( 1) m ( L) ) f r f r (3) (4) s calculated from the polynomal probablty dstrbuton p( ) 0.5( m m 1)(1 ). m s an external parameter whose value s fxed and r represents a random number n [0 1]. In case of updatng of mask bt usng mutaton operator, the mask bt s flpped when t random number generated s less than the mutaton probablty. To retan the best parent ndvduals from the populaton, we use the concept of the eltsm where both the parent and chld populaton are combned and are sorted n descendng order of ftness. The best N (N s the populaton sze) soluton s retaned for the next generaton evoluton 4. PARALLEL GENETIC ALGORITHM AIDED BAYESIAN CLASSIFIER Durng the desgnng of the learnable Bayesan classfer we found that the computaton of the probablty takes O(n 2 ) operatons, where n s the number of features whch can be a cause for the ncreases n the computaton tme of the ftness functon. Thus the parallelzaton of the Bayesan classfer s proposed for reducng

4 the computaton tme wthout much affectng the predcton accuracy. A herarchy of parallelzaton s provded for the ftness functon evaluaton by usng the master slave model. Frst we provde a class level parallelzaton followed by attrbute label parallelsm. In class level parallelsm we determne the probablty of the features for each class by usng the number of processor equal to the number of class and return the results to the master. Then the master dvdes the populatons whch s send to the slaves for determnng the classfcaton accuracy for each of the ndvduals. The pctoral representaton for the herarchy s shown n Fgure 3. The other operaton for generaton of new set of populaton s carred by the master. Fgure 3: Herarchy of parallelzaton n master-slave model for ftness evaluaton The algorthm for the parallelzaton of the ftness functon s gven algorthm 2. Algorthm 2: Parallelzaton of Ftness Evaluaton For =1 to class label End for Dstrbute the dataset based on the class label to each processor P Determne the probablty for each class label and return the result to master processor. For each processor P Dstrbute the populaton POP t to each processor equally. Determne the classfcaton of each ndvdual. Return the classfcaton accuracy to the master. 5. EXPERIMENTAL STUDY In ths secton, we provde the descrpton of the dataset, expermental setup and fnally results and the dscusson. 5.1 Descrpton of Dataset The modelng of the classfer s carred out usng the metabolc syndrome dataset obtaned from Yonchon County of Korea. The dataset have 1135 samples wth 2 class labels. The dataset has 18 attrbutes and the class attrbute determne the absence or presence of the dsease. Few of ths attrbutes has a least contrbuton to the classfcaton accuracy whch was obtaned from the covarance analyss. 11 mportant attrbute whch are necessary for the predcton s dentfed n [10]. However, we consder all the attrbutes for classfcaton and try to remove the spurous attrbutes and ncrease the classfcaton accuracy. The dataset s dvded nto three groups namely tranng set, valdaton set and testng set. The tranng and valdaton set contans 25% of data, and testng contans 50% of the dataset. 5.2 Expermental Setup The algorthm s mplemented usng C language on a mult core system core 2 Duo wth 4 cores, each of 2.4 GHz, 2GB RAM, under Lnux OS. The communcaton between the processors has been supported by the free avalable MPICH (Message Passng Interface) lbrary. The parameter settng used for the algorthm s shown n Table 1. Table 1: Parameter Settng Processor Populaton Sze 100 Real valued crossover probablty 0.8 Bnary crossover probablty 0.78 Real valued mutaton probablty 0.05 Bnary mutaton probablty Result and Analyss The Bayesan classfer s constructed usng 25% of the tranng dataset. The valdaton set s used for adjustng the learnng vector. The maxmum predctve accuracy attaned the valdaton set and test set s 76% and 72% respectvely wth 11 attrbutes. The parallelzaton of the ftness evaluaton process for adjustng the weght s done. The expermental results of the seral and parallel mplementaton are tabulated n Table 2. The results show that the accuracy of the classfer does not detorates on parallelzaton. However there s a sgnfcant reducton of computaton tme. Ths s llustrated by the speed up graph shown n Fgure 4 Also the mpact of the parameter settng of genetc algorthm s studed. The results are shown n Fgure 5-7. Frst, consderng the crossover probablty, t s found from the Fgure 5(a) that TABLE 2: Predctve accuracy and computaton tme of the seral and parallel model

5 Processor Tranng accuracy (%) Testng accuracy (%) Computaton tme (n sec) s good enough for predctng hgh accuracy. We see that the crossover probablty doesn t have much mpact on tranng accuracy but reduces the testng accuracy. For the mutaton probablty the approprate range s 0.01 to 0.05 and ncreasng the mutaton probablty has the same effect as crossover probablty. The ncrease n populaton doesn t really account for ncrease n accuracy; however t ncreases the computaton tme. Ths s clearly llustrated by Fgure 6. The same result can be seen for the ncrease n teraton (Fgure 7). (a) Crossover Probablty vs. Classfcaton accuracy 6. CONCLUSION Ths paper proposed a genetc and parallel genetc aded learnable classfer and was found to have reached the optmal classfcaton as specfed by [10] wth mnmal set of attrbute. We found that the parallel model has same predcton accuracy as (b) Mutaton probablty vs. Classfcaton accuracy Fgure 5: Impact of crossover and mutaton probablty on classfcaton accuracy REFERENCES Fgure 4: Speedup Graph seral genetc aded learnable Bayesan classfer, yet there s reducton n tme n parallel model. Also the analyss of the mpact of the parameter settng showed that 0.8 of crossover probablty, 0.05 mutaton probablty, 100 populaton and 100 teraton can result n hgh classfcaton accuracy. However, the result of parameter settng s restrcted to ths dataset. Ths needs to be evaluated usng other datasets. Also the classfer performance needs to be evaluated on more datasets for valdaton of ts result. [1] T. Back. Evolutonary Algorthms n Theory and Practce.. New York, Oxford Unv. Press, [2] T. Back, D. B. Fogel, and Z. Mchalewcz. Handbook of Evolutonary Computaton. New York, Insttute of Physcs Publshng and Oxford Unversty Press, [3] R. Blanco, I. Inza, M. Merno, J. Quroga, and P. Larra naga. Feature selecton n bayesan classfers for the prognoss of survval of crrhotc patents treated wth tps. Journal of Bomedcal Informatcs, 38(5): , [4] C. A. C. Coello and A. H. Agurre. Desgn of combnatonal logc crcuts through an evolutonary multobjectve optmzaton approach. Artfcal Intellgence for Engneerng, Desgn, Analyss and Manufacture, 16(1):39 53, January 2002.

6 (a) Populaton Sze vs. Classfcaton accuracy (a) Iteraton vs. Classfcaton accuracy (b) Populaton sze vs. Tme Fgure 6: Impact of populaton sze on classfcaton accuracy and tme [5] K. Deb. Mult-Objectve Optmzaton usng Evolutonary Algorthms. Prentce Hall, [6] K. Deb and D. E. Goldberg. MGA n c: A messy genetc algorthm n c. Techncal report 91008, Illnos Genetc Algorthms Laboratory(IIIGAL), September [7] D. E. Goldberg. Genetc Algorthms n search, optmzaton and machne learnng. Addson-Wesly, [8] C. Mngje and L. Shang. An mproved adaptve genetc algorthm and ts applcaton n functon optmzaton. Journal of Harbn Engneerng Unversty, 28: , [9] L. Mykkanen, J.Kuussto, K.Pyor ala, and M. Laakso. Cardovascular dsease rsk factors as predctors of type 2 (non-nsuln-dependent) dabetes melltus n elderly subjects. Dabetologa, 36: , [10] H.-S. Park and S.-B. Cho. An effcent attrbute orderng optmzaton n bayesan networks for prognostc modelng of the metabolc syndrome. In ICIC (3), pages , (b) Iteraton vs. Tme Fgure 7: Impact of number of teraton on classfcaton accuracy and tme [11] M. L. Raymer, L. A. Kuhn, and W. F. Punch. Knowledge dscovery n bologcal datasets usng a hybrd bayes classfer/evolutonary algorthm. In Proceedngs of the 2nd IEEE Internatonal Symposum on Bonformatcs and Boengneerng, pages , Washngton, DC, USA, IEEE Computer Socety. [12] R. Tanese. Parallel genetc algorthm for a hypercube. In Proceedngs of the Second Internatonal Conference on Genetc Algorthms, pages , [13] M. Wggns, A. Saad, B. Ltt, and G. Vachtsevanos. Evolvng a bayesan classfer for ECG-based age classfcaton n medcal applcatons. Appled Soft Computng, 8(1): , [14] R. R. Yager. An extenson of the nave bayesan classfer. Informaton Scences, 176(5): , 20