A TOPSIS based Method for Gene Selection for Cancer Classification

Transcription

1 Volume 67 No17, April 2013 A TOPSIS baed Method for Gene Selection for Cancer Claification IMAbd-El Fattah,WIKhedr, KMSallam, 1 Department of Statitic, 3 Department of Deciion upport, 2 Department of information technology 2,3 Faculty of Computer and Informatic, 1 Faculty of Commerce (Operation reearch) Zagazig Univerity ABSTRACT Cancer claification baed on microarray gene expreion i an important problem In thi work a new gene election technique i propoed The technique combine TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution) and F-core method to elect ubet of relevant gene The output of the combined gene election technique i fed into four different claifier reulting in four hybrid cancer claification ytem In the propoed technique ome important gene were choen from thouand of gene (mot informative gene) After that, the microarray data et were claified with a K-Nearet Neighbour (KNN), Deciion Tree (DT), Support Vector Machine (SVM) and Naive Baye (NB)The goal of thi propoed approach i to elect mot informative ubet of feature/gene that give better claification accuracy General Term Feature election, claification, machine learning technique, and data mining Keyword-component TOPSIS, Gene Selection, Cancer claification, Neural Network, Deciion Tree, Naive Baye and K-Nearet Neighbour 1 Introduction The goal of microarray data claification i to build an efficient and effective model that can dicriminate between normal and abnormal condition, or claify tiue ample into different clae of dieae However, gene election proce i a major problem becaue of the characteritic of microarray data; the huge number of gene compared to the mall number of ample (high-dimenional data), irrelevant gene, and noiy data In other word, the problem of microarray data i the mall amount of ample againt the huge amount of feature (gene in our cae) To overcome thi challenge, a gene election method i ued to elect a ubet of gene that increae the claifier ability to claify ample accurately The gene election method ha everal advantage, uch a improving claification accuracy, reducing the dimenionality of data, and removing irrelevant and noiy gene [1] In order to improve the performance of claifier algorithm, it i preferred to reduce the dimenionality of the data by deleting redundant, irrelevant or noiy feature [1] In addition, election of an optimal ubet of feature by exhautive earch i impractical and it conume much time a the number of attribute increae, and a proper learning trategy mut thu be devied The relevance of good feature election method ha been dicued in [2], but the recommendation in literature do not give evidence for a ingle bet method for either feature election or claification of microarray data [3]Many feature election algorithm have been propoed in literature All thee method earch for optimal or near optimal ubet of feature that optimize a given criterion The ret of the paper i organized a follow In Section 2, we provide background information on microarray data analyi and dicu ome related work Section 3 we dicu the feature election type filter, wrapper and embedded model Section 4 contain a detailed dicuion of the propoed approach Section 5 contain a detailed dicuion a well a comparion between the propoed approach and everal tate-of-the-art method Finally, in Section 6 we conclude with ome final remark and ugget future reearch direction 2 Microarray data Microarray analye were motivated by the earch for ueful pattern of tumour claification, dieae tate claification, dicovery of new ubtype of dieae or dieae tate, among other Microarray data are generally having large number of feature/gene in comparion to the number of ample or condition (high dimenional data) There are many efficient method for the analyi of microarray data uch a clutering, claification and feature election Microarray technology meaure the expreion level of gene That can be ued in the diagnoi, through the claification of different type of cancerou gene leading to a cancer type [4] A microarray data i commonly repreented a an M N matrix, where M i the number of ample and N i the number of gene in the experiment Each cell in the matrix, a hown in Fig 1, i the level of expreion of a pecific gene in a pecific experiment The lat column to the right how the label or cla that each ample belong to Microarray tudie often involve the ue of machine learning method uch a unupervied and upervied learning Unupervied learning find ubgroup in the data baed on a imilarity meaure between the expreion profile Unupervied learning tudie how ytem can learn to repreent particular input pattern in a way that reflect the tatitical tructure of the overall collection of input pattern[5] In upervied learning, each example i a pair coniting of an input object (typically a vector) and a deired output value (alo called the uperviory ignal) A upervied learning algorithm analyze the training data and produce an inferred function, which i called a claifier [6] Sample1 Sample2 SampleM Gene1 F11 F21 FM1 Gene2 F12 F22 FM2 Gene N F1N F2N FMN Cla C1 C2 CM Figure 1: Matrix repreentation of a microarray dataet 39

2 Volume 67 No17, April 2013 The expreion profile of a ample i can be repreented a a row vector The expreion profile of a gene j can be repreented a a row vector generalization propertie ince it i independent of any pecific claifier Fig 3 repreent the procedure of wrapper It i the ame a that of filter except that the meaurement tage i replaced by a claifier algorithm The wrapper model ue the predictive accuracy of a predetermined mining algorithm to give the quality of a elected feature ubet, generally producing feature better uited to the claification tak at hand However, it i time conuming for high dimenional data [2] 3 Feature Selection Method Feature election technique tudy how to identify and elect mot informative feature for building claification model which can interpret data better Feature election reduce dimenionality of data, o the cot of computation i reduced It alo improve the claification accuracy and the comprehenibility of the model by eliminating redundant and noiy feature Feature election and feature extraction are different The former elect the mot informative gene and maintain the original meaning of the elected feature, which i deirable in ome domain, while the latter create new feature by combining the original feature The feature election algorithm that have been propoed in literature earch for optimal or near optimal ubet of feature that optimize a given criterion Feature election method divided into three clae: filter, wrapper, and embedded method baed on whether the election method depend on the claifier or not [2] Fig2 the Filter Method 31 Filter Method Filter method elect ubet of feature a a pre-proceing tep, independently of the claifier [7] Feature are rated by their dicriminative power with regard to targeted clae Feature ranking approache are ued rank feature by certain ranking criterion Thi ranking i ued a the bae of election mechanim In the propoed feature election method, feature ranking approach i attractive becaue it i imple, calable, and how good empirical ucce Computationally, feature ranking i efficient ince it require only the computation of n core and orting thee core A i hown in Fig 2 the filter have three main phae: feature ubet generation, meaurement, and teting In feature ubet generation tage, a feature ubet i generated Next the meaurement tep i performed, which meaure the information of the current feature ubet Thee two tep will be performed iteratively until the meaurement matche the top criterion The final feature ubet would contain the mot informative feature Finally, the teting tep i done by a claifier algorithm, like Deciion tree (DT) [8], Naive Baye (NB) Claifier [8], K-Nearet neighbour (KNN) [9] or Support vector machine (SVM) [10] 32 Wrapper Method 33 Embedded Method Fig3The wrapper Method Embedded method ue feature ubet earch and evaluation in the proce of building a claifier model a hown in Fig 4 The earch i guided by the learning algorithm [2] Embedded method are computationally le intenive than wrapper and they account for feature dependencie Wrapper method ue the claifier algorithm of interet to evaluate ubet of feature according to their claification accuracy [7] Wrapper can often find a mall ubet of feature with high accuracy becaue the feature match well with the learning algorithm Even though, wrapper typically require more computation effort, it i argued that filter have better Fig4 The Embedded Method 40

3 Volume 67 No17, April Technique for Order Performance by Similarity to Ideal Solution TOPSIS According to thi technique, the bet alternative would be the one nearet to the poitive ideal olution and farthet from the negative ideal olution [11] The poitive ideal olution i a olution that maximize the benefit criteria and minimize the cot criteria, wherea the negative ideal olution i a olution that maximize the cot criteria and minimize the benefit criteria In brief, the poitive ideal olution i compoed of all bet value attainable from the criteria, while the negative ideal olution conit of all wort value attainable from the criteria [12] The TOPSIS method conit of the following tep: Step 1: Etablih a deciion matrix for the ranking The tructure of the matrix can be expreed a follow: Fig5 the Matrix repreentation where row denote the gene i, i = 1,2,, m; column repreent j th ample, j = 1, 2,, n; and x ij i a gene expreion level of gene i and a ample j Step 2: Calculate the normalized deciion matrix R= [r ij ] The normalized value r ij i calculated a: Step 3: Calculate the weighted normalized deciion matrix by multiplying the normalized deciion matrix by it aociated weight The weighted normalized value v ij i calculated a: Where wj repreent the weight of the j th ample Here I calculate w j uing geometric mean by uing the following formula: Where j=1, 2 m Step 4: Determine the poitive-ideal and negative-ideal olution -Similarly, the eparation of each gene from the negative-ideal olution ( ) i a follow: Step 6: Calculate the relative cloene to the ideal olution and rank the performance order The relative cloene of the gene Aj can be expreed a: -Where the CCi* index value lie between 0 and 1 The larger the index value mean the better the performance of the gene 5 Method 51 Colon cancer dataet The Colon data et contain 62 ample collected from cancer patient Among them 40 tumour and 22 normal are from healthy part of the colon of the ame patient a hown in table 1 Two thouand out of around 6500 gene were elected baed on the confidence in the meaured expreion level Datae t #of Gene Table 1Colon data et # of ample # of clae # of poitive ample #of negativ e ample colon Gene election method Not all gene in microarray data needed for claification Microarray data conit of large number of gene in mall ample So ome gene were needed to be choen (mot informative gene) [13] Thi proce i referred to a gene election (feature election) in machine learning 521 T-tet tatitic The t-tet [14],[15] i a tatitical hypothei tet ued to determine the ignificance of the difference between the mean of two independent ample It aume normally ditributed population For unequal variance and unequal or equal ample ize the t tatitic i calculated a follow: Step 5: Calculate the eparation meaure, uing the n- dimenional Euclidean ditance The eparation of each gene from the poitive-ideal olution ( ) i given a: where and are the mean of the two ample; and are the variance etimate of the two ample; n1 and n2 are the ample ize of the two ample The degree of freedom df can be calculated a 41

4 Volume 67 No17, April 2013 The t ditribution can be ued with t tatitic and df degree of freedom a parameter to calculate the correponding p-value Feature are then ranked according to p-value The maller the p-value the more the relevant the feature 522 Signal-to-Noie (SNR) baed Gene Selection Method SNR i a calculated ranking number for each gene to define how well thi gene i mot informative [16] For each gene, thi work ha normalized the gene expreion data by ubtracting the mean of the two clae and then dividing by ummation of the tandard deviation of the two clae Every ample i labelled a either a normal or a cancer ample Signal-to-Noie Ratio (SNR) i computed a follow The SNR i the value of the i th gene; and denote the averaged expreion intenitie of the i th gene over the ample each belonging to normal cae and tumour cae repectively Alo, and denote the tandard deviation of normal cae and tumour cae 523 Fiher' ratio (FR) Fiher-ratio [17] i a ratio between-cla ditance to with-in cla ditance, and it i defined a: Where m 1 and m 2 are the average of the expreion level of a particular gene in normal cla and tumor cla; v 1 and v 2 are the correponding variance In the experiment, given a microarray dataet, we compute the FR value for each gene and rank them in decending order Gene with higher ranking have more dicriminative power for claifying ample into categorie 524 Information gain (IG) Information Gain (IG) i a filter method ued to pre-elect gene and finally produce a ubet of informative gene [18] Information gain i given by Where X and Y are feature, and In our cae, given a microarray dataet, we compute the information gain for each gene and rank them in decending order by their information gain value The higher information value a gene ha, the more effective the gene ued to claify the training data 53 Claifier 531 Deciion Tree (DT ) Deciion tree claifier i a method commonly ued in machine learning It aim to create a model that predict the value of a target variable baed on everal input variable [6],[8],[19] It i a tree where each non-terminal node repreent a tet or deciion on the conidered data item Choice of a certain branch depend upon the outcome of the tet To claify a particular data item, we tart at the root node and follow the aertion down until we reach a terminal node (or leaf node) A deciion i made when a terminal node i approached [20] 532 Support Vector Machine (SVM) Support vector machine (SVM) i a new generation learning ytem baed on recent advance in tatitical learning theory [10] It i an algorithm that attempt to find a linear eparator (hyper-plane) between the data point of two clae in multidimenional pace SVM are well uited to dealing with interaction among feature and redundant feature 533 Neural Network (NN) Neural network (NN) [6] are thoe ytem modelled baed on the human brain working A the human brain conit of million of neuron that are interconnected by ynape, a neural network i a et of connected input/output unit in which each connection ha a weight aociated with it The network learn in the learning phae by adjuting the weight o a to be able to predict the correct cla label of the input [21] 534 K-Nearet Neighbor (KNN) The purpoe of the algorithm i to claify a new object baed on attribute and training ample The K-NN method conit of a upervied learning algorithm where the reult of a new query intance i claified baed on the majority of the K-nearet neighbour category The K-NN method i eay to implement ince only the parameter K (number of nearet neighbour) need to be determined The parameter K i an important factor affecting the performance of the claification proce K-NN claifie a new ample baed on the minimum ditance from the tet ample to the training ample The Euclidean ditance wa ued for calculation in thi paper [13] If an object i near to the K nearet neighbour, the object i claified into the K-object category 535 Naive Baye (NB) The NB claifier i a probabilitic algorithm baed on Baye rule and the imple aumption that the feature value are conditionally independent given the cla [22] Given a new ample obervation, the claifier aign it to the cla with the maximum conditional probability etimate 54 Propoed method The four propoed claification module (J48, SVM, NB and 3NN) receive pre-proceed original high dimenionality microarray dataet a it input The ytem firt tep i reducing the total number of gene in the input dataet to mall ubet uing TOPSIS Method and F-core tet technique a a combined gene election technique Then the elected data will be the data ued by the choen claifier to aign new ample into their correct clae intead of uing the original full data et At thi point we can meaure and record the tet accuracy of the claifier The workflow of the propoed approach i hown in Fig 5 42

5 Accuracy Accuracy % International Journal of Computer Application ( ) Volume 67 No17, April 2013 In the propoed approach, firt we rank gene according to TOPSIS Method and elect 250 top ranked gene from thi method and after that we ue F ditribution a follow: Calculating the mean of the expreion value for each of the n gene (μn1 for the firt cla and μn2 for the econd cla) Calculating the tandard deviation of the expreion value for each of the n gene (σn1 for the firt cla and σn2 for the econd cla) Obtaining the abolute difference between the calculated mean ( μn1 - μn2 ) Calculating the F value for each of the elected gene from TOPSIS METHOD by the following: (F = μn1 - μn2 /(σn1 + σn2)) Arrange the gene according to their F value and chooe the mot informative gene Table 2 accuracy of the claifier with the original Colon data et, and the reduced data et uing 10 fold cro validation Claifier # of ued gene SVM 3NN J48 Naïve Baye Original data et Table 4 Comparion of the propoed method and other method ued in [23], [24] and [25] Method Propoed Propoed [23] [25] [24] with SVM with 3NN Colon After finihing the gene election tep we ue four different claifier to tet the accuracy of the propoed method One of the propoed ytem achieved the highet claification accuracie on the ued dataet with a coniderable number of gene Although the other propoed ytem couldn t reach the ame reult, it contribute in validating the propoed gene election technique 6 Experimental Reult Fig5 Propoed Method We tarted experiment by evaluating performance accuracie of the claifier, SVM, KNN, and deciion tree (J48) on the dataet uing 10-fold Cro Validation (CV) without uing feature election algorithm The reult for the four claifier i hown in table 3 After feature election, the elected feature ubet were evaluated uing four common claification algorithm SVM, KNN, J48, and NB uing 10-fold CV We tet the accuracy of the propoed approach uing different number of gene a howed in table 3 and fig6 uing 10-fold cro validation We ee that when uing data et which ha 100 gene we ee that KNN ha the highet accuracy than other method When uing 50 and 20 gene data et we ee that SVM ha the highet accuracy than the other method When the number of reduced data et i 10 SVM and 3NN have higher accuracy than other claifier When comparing the propoed approach uing 10 gene and uing SVM or 3NN a claifier with the method ued in [23],[24],[25] we ee that the propoed approach give higher claification accuracy than other a hown in table 4 and Fig SVM NN3 48J Claifier Fig6 compared reult of colon data et Claifier 7 Concluion and future work The proce of claifying microarray data mut encloe two main tep; implementing an effective gene election technique and chooing a powerful claifier Thee two tep form the NB Fig7 comparion of our method with [23, 24, 25] original 100 gene 50 gene 20 gene 10 gene 43

6 Volume 67 No17, April 2013 workflow of thi paper trying to go through each tep detail and validating it output The goal of the propoed approach i to extract informative outcome from feature election method to find a ubet of informative feature that have maller ize and better claification accuracy compare to individual method In thi paper, we introduced four new hybrid claification ytem for claifying cancer ample uing microarray gene expreion dataet The main target of the propoed ytem i to get the highet accuracy when claifying the ample uing a mall ubet of informative gene A combination of two gene election technique i introduced to olve the problem of the microarray high dimenionality Thi combined technique preent high performance a it reduce the number of gene SVM, KNN, J48 and NB were choen for claification a they are very efficient binary claification technique and uually give good reult by attenuating their variety of attribute The four ytem were a reult of integrating the propoed gene election technique once with SVM reulting in the Firt CS, another time with KNN reulting in the Second CS, another time with J48 reulting in third CS and another time with NB reulting in fourth CS The four ytem CS were applied public microarray dataet colon When the number of reduced gene data et i 10 we ee that the accuracy i higher than the other reduced data et number Comparion with other verifie the efficiency of the propoed method 8 Reference [1] I H Witten and E Frank, Data Mining: Practical machine learning tool and technique: Morgan Kaufmann, 2005 [2] Y Saey, I Inza, and P Larrañaga, "A review of feature election technique in bioinformatic," Bioinformatic, vol 23, pp , 2007 [3] A Statnikov, C F Aliferi, I Tamardino, D Hardin, and S Levy, "A comprehenive evaluation of multicategory claification method for microarray gene expreion cancer diagnoi," Bioinformatic, vol 21, pp , 2005 [4] A C Lorena, I G Cota, and M C de Souto, "On the complexity of gene expreion claification data et," in Hybrid Intelligent Sytem, 2008 HIS'08 Eighth International Conference on, 2008, pp [5] M A Ranzato, F J Huang, Y-L Boureau, and Y LeCun, "Unupervied learning of invariant feature hierarchie with application to object recognition," in Computer Viion and Pattern Recognition, 2007 CVPR'07 IEEE Conference on, 2007, pp 1-8 [6] S Kotianti, I Zaharaki, and P Pintela, "Supervied machine learning: A review of claification technique," Frontier in Artificial Intelligence and Application, vol 160, p 3, 2007 [7] B Krihnapuram, L Carin, and A Hartemink, "1 Gene expreion analyi: Joint feature election and claifier deign," Kernel Method in Computational Biology, pp , 2004 [8] Y Lu and J Han, "Cancer claification uing gene expreion data," Information Sytem, vol 28, pp , 2003 [9] Y Song, J Huang, D Zhou, H Zha, and C Gile, "Iknn: Informative k-nearet neighbor pattern claification," Knowledge Dicovery in Databae: PKDD 2007, pp , 2007 [10] I Guyon, J Weton, S Barnhill, and V Vapnik, "Gene election for cancer claification uing upport vector machine," Machine learning, vol 46, pp , 2002 [11] G-W Wei, "Extenion of TOPSIS method for 2-tuple linguitic multiple attribute group deciion making with incomplete weight information," Knowledge and information ytem, vol 25, pp , 2010 [12] T Yang and C-C Hung, "Multiple-attribute deciion making method for plant layout deign problem," Robotic and computer-integrated manufacturing, vol 23, pp , 2007 [13] S-B Cho and H-H Won, "Machine learning in DNA microarray analyi for cancer claification," in Proceeding of the Firt Aia-Pacific bioinformatic conference on Bioinformatic 2003-Volume 19, 2003, pp [14] G D Ruxton, "The unequal variance t-tet i an underued alternative to Student' t-tet and the Mann Whitney U tet," Behavioral Ecology, vol 17, pp , 2006 [15] D C Montgomery, G C Runger, and N F Hubele, Engineering tatitic: Wiley, 2009 [16] J-H Hong and S-B Cho, "Lymphoma cancer claification uing genetic programming with SNR feature," Genetic Programming, pp 78-88, 2004 [17] K Mao, P Zhao, and P-H Tan, "Supervied learningbaed cell image egmentation for p53 immunohitochemitry," Biomedical Engineering, IEEE Tranaction on, vol 53, pp , 2006 [18] G Forman, "An extenive empirical tudy of feature election metric for text claification," The Journal of Machine Learning Reearch, vol 3, pp , 2003 [19] W Du and Z Zhan, "Building deciion tree claifier on private data," 2002 [20] G Stiglic, S Kocbek, I Pernek, and P Kokol, "Comprehenive Deciion Tree Model in Bioinformatic," PloS one, vol 7, p e33812, 2012 [21] M A Mazurowki, P A Haba, J M Zurada, J Y Lo, J A Baker, and G D Tourai, "Training neural network claifier for medical deciion making: The effect of imbalanced dataet on claification performance," Neural network: the official journal of the International Neural Network Society, vol 21, p 427, 2008 [22] K Al-Aidaroo, A Bakar, and Z Othman, "Medical Data Claification with Naive Baye Approach," 2012 [23] B Liu, Q Cui, T Jiang, and S Ma, "A combinational feature election and enemble neural network method for claification of gene expreion data," BMC bioinformatic, vol 5, p 136, 2004 [24] T Paul and H Iba, "Identification of informative gene for molecular claification uing probabilitic model building genetic algorithm," in Genetic and Evolutionary Computation GECCO 2004, 2004, pp [25] J Liu and H Iba, "Selecting informative gene uing a multiobjective evolutionary algorithm," in Evolutionary Computation, 2002 CEC'02 Proceeding of the 2002 Congre on, 2002, pp