A novel feature extraction approach for microarray data based on multi-algorithm fusion

Transcription

1 Hypothess Voume 11(1) A nove feature extracton approach for mcroarray data based on mut-agorthm fuson Zhu Jang 1,2 & Rong Xu 1 1State Key Laboratory of Astronautc Dynamcs, X an Satete Contro Center, X an, Chna; 2 Key Laboratory of Fud and Power Machnery, Mnstry of Educaton, Xhua Unversty, Chengdu, Chna; Zhu Jang - Ema: h5525@163.com; * Correspondng author Receved December 09, 2014; Revsed December 31, 2014; Accepted January 08, 2015; Pubshed January 30, 2015 Abstract: Feature extracton s one of the most mportant and effectve method to reduce dmenson n data mnng, wth emergng of hgh dmensona data such as mcroarray gene expresson data. Feature extracton for gene seecton, many serves two purposes. One s to dentfy certan dsease-reated genes. The other s to fnd a compact set of dscrmnatve genes to bud a pattern cassfer wth reduced compexty and mproved generazaton capabtes. Dependng on the purpose of gene seecton, two types of feature extracton agorthms ncudng rankng-based feature extracton and set-based feature extracton are empoyed n mcroarray gene expresson data anayss. In rankng-based feature extracton, features are evauated on an ndvdua bass, wthout consderng nter-reatonshp between features n genera, whe set-based feature extracton evauates features based on ther roe n a feature set by takng nto account dependency between features. Just as earnng methods, feature extracton has a probem n ts generazaton abty, whch s robustness. However, the ssue of robustness s often overooked n feature extracton. In order to mprove the accuracy and robustness of feature extracton for mcroarray data, a nove approach based on mut-agorthm fuson s proposed. By fusng dfferent types of feature extracton agorthms to seect the feature from the sampes set, the proposed approach s abe to mprove feature extracton performance. The new approach s tested aganst gene expresson dataset ncudng Coon cancer data, CNS data, DLBCL data, and Leukema data. The testng resuts show that the performance of ths agorthm s better than exstng soutons. Keywords: feature extracton; robustness; mcroarray data; mut-agorthm fuson Background: Feature extracton s one of the key ssues n the fed of data mnng. Researchers have reazed that n order to use data mnng toos effectvey, data preprocessng s essenta to successfu data mnng. Feature extracton s one of the mportant and frequenty used technques n data preprocessng. It not ony can emnate nformaton redundant, mprove the cassfcaton effcency and acceerate the computatona speed, but aso can reduce the compexty of the cassfer and the error rate of cassfcaton [1]. Feature extracton agorthm can be cassfed nto three fundamentay approaches: wrapper, fter and embedded. Wrapper mode evauates the subset of seected features by usng crtera based on the resuts of earnng agorthms, whe fter methods depends on ntrnsc characterstcs of the data to seect feature subsets wthout nvovng and mnng methods [2]. These methods are often mted n use because they requre a ong computatona tme. To take advantage of these two agorthms, the embedded method s proposed [3]. For a feature extracton agorthms, many rea appcatons, such as computer vson, mcroarray technoogy and vsua recognton, nvove mcroarray data. Robustness of feature extracton for these data aso gets ts attentons n recent years. Gugezen et a. [4] studed the stabty and cassfcaton accuracy of Mnmum Redundancy Maxmum Reevance-based feature extracton. An entropy-based measure for stabty assessment was deveoped by Krzek et a [5]. However, research shows that the extracton resuts of most feature extractons are very senstve to the changes of tranng sets. That s to say, these agorthms have poor robustness [6]. Ths probem s partcuary obvous for mcroarray data set. Even f the ISSN (onne) (prnt) Bonformaton 11 (1): (2015) Bomedca Informatcs

2 tranng data set s sghty changed, the obtaned optmum feature subset w have arge dfference and the performance of the cassfcaton mode w change greaty. Therefore, to mprove the credbty of cassfcaton performance, we need to choose the feature extracton agorthm wth a hgh robustness. Fgure 1: Score-based mut-agorthm fuson. In prevous studes, t s we known that combnng or ntegratng mutpe cassfers, especay uncorreated weak ones, coud greaty mprove the cassfcaton performance [7], but studes for the fuson of varous feature extracton agorthms are a few. Marna Skurchna beeves that there may be usefu nformaton n the unseected features after the feature extracton. The omsson of these features may ead to the poor performance of feature extracton and pattern recognton. So t s suggested to use the method of fuson to utze the usefu nformaton n the negected features [8]. As we a know that feature subsets produced by dfferent feature extracton agorthm may show compementary, and fuson of mutpe agorthms utzes the search abtes of each agorthm to get coser to a goba optma souton. Thus a fuson of the feature subsets may produce a better representaton n feature space. A ot of feature extracton agorthms have been proposed n the terature. But not a feature extracton agorthms can be fused. If two extracton agorthms are smar, the fuson of them can not mprove the stabty of the extracton agorthm greaty [9]. Therefore, the dversty must be consdered whe choosng the feature extracton agorthms. Dfferent types of feature extracton can compement each other and avod overappng. Obvousy, t s not necessary to fuse a feature extracton agorthms, whch s aso mpossbe. To smpfy the cacuaton process and reduce the amount of computaton whe mantanng the dversty of extracton agorthms, n ths study, Fsher Rato, Absoute Weght of SVM (AW-SVM) and Poynoma Support Vector Machne (PSVM) were used to fuse n ths paper. Fsher Rato [9] s one of the basc methods n fter mode of feature extracton. It s a unvarate fter method evauatng each feature ndvduay. Its estmaton standards are drecty obtaned from the data set. It has the characterstcs of sma cacuaton cost, hgh effcency, etc.. AW-SVM [10] s an embedded method that ranks features based on ther correspondng coeffcents n the SVM cassfer. PSVM s a wrapper method based on statstca earnng theory [11]. It has powerfu faut toerance and generazaton abtes. Studes have shown that the generazaton abty of PSVM w not reduce wth the ncreasng of the order. It overcomes the probems of over earnng, ack of earnng, oca mnmum vaue and dmenson dsaster of tradtona machne earnng. Fgure 2: Performance comparsons on coon data: A) Cassfcaton Error; B) AUC; C) Standard Devaton of Error Estmaton; D) Feature Robustness. ISSN (onne) (prnt) Bonformaton 11(1) (2015) Bomedca Informatcs

3 By consderng a these factors and based on the dea of fuson, a nove feature extracton method, poynoma support vector machne based on mut-agorthm fuson (MAF-PSVM), was put forward n ths paper. Fgure 3: Performance comparsons on CNS data: A) Cassfcaton Error; B) AUC; C) Standard Devaton of Error Estmaton; D) Feature Robustness. Methodoogy: Poynoma Support Vector Machne based on Mut-Agorthm Fuson The specfc mpementaton steps of MAF-PSVM are ntroduced n deta. Step one, ntaze the sampe data set. Tranng sampes are custered nto K casses by k-means [12] (The vaue of k n ths paper was 8 by tranng.), dfferent penaty factors are used for each cass. Step two, fuse feature extracton agorthms (n ths paper, the agorthms were Fsher Rato, AW-SVM and PSVM (How to seect approprate kerne functons for dfferent appcatons has been a dffcut probem. Studes have shown that near kerne functon s sutabe for the case of neary separabe data, so poynoma kerne was seected. The other kerne functons can be further studed n the future for us. The generazaton abty of poynoma kerne functons s dfferent. In most cases, the performance of the cassfer can reach to optmum whe the parameter d s taken as 1[13]. Therefore, the vaue of order d of the poynoma kerne was taken as 1)) and conduct feature extracton for custered sampes. The foowng contents ntroduce the fuson method used by ths paper: In our study, we used score-based mut-agorthm fuson methods. Frsty, a score vector contanng scores of a features was produced by each bass crteron. Secondy, a score combnaton agorthm was used to aggregate the mutpe score vectors nto one consensus score vector. Fnay, a feature rankng procedure was empoyed to rank the features based on ther consensus scores. The score-based mut-agorthm fuson procedure s ustrated n Fgure 1. In score aggregatng, the scores produced by dfferent bass crtera w be comparabe. It s essenta to ensure that score normazaton shoud be done before score combnaton s ISSN (onne) (prnt) Bonformaton 11(1) (2015) Bomedca Informatcs

4 performed. The scores are normazed to the range of [0, 1] n ths study. Assume u s the score vector produced by bass crteron, the score normazaton s performed as foows: ' u u mn u u max u mn (1) where u mn and u max are the mnmum and maxmum vaues n vector u. For a the bass crtera, t s assumed that the arger the score, the better the feature. A score combnaton method w be used as foowng: m 1 ' u u m 1 (2) where m s the number of bass crtera used n fuson. Step three, feature extracton resuts of the step two w be used to tran the PSVM cassfer. If the numbers of negatve cass ponts and postve cass ponts have a arge dfference n the tranng data set, and f appyng the same penaty parameter C to the set of postve cass ponts and the set of negatve cass ponts, t means that the one wth more cass ponts w get more attenton. However, we hope that penates to the postve pont and negatve pont are not the same. Accordngy, for propery seected parameter C, C C and C C. Where, and are respectvey the number of postve cass tranng ponts and negatve cass tranng ponts. C s the penaty parameter of postve cass pont. Comparatvey, C s the penaty parameter of negatve cass pont. The PSVM cassfer was but as foowng: 1 d max W ( ) ( x x 1) y y 2 1, 1 s.t. y 0, 1 0 Ccass 1, Cass Index cass1; 0 Ccass2, Cass Index cass2; 0 CcassN, Cass Index cassn; 1,, 1, 1, n C 1,, n (3) Where, represents the Lagrange mutper. cass1,, cassn represent the categores after custerng. Cass Index represents the mark of cass. 1,, n represent the number of sampe ponts n each cass, and C represents the penaty factor of each cass. Step four, regress the sampe data set by traned cassfer, and remove features wth the mnma correaton. Sampe set w be updated. In ths study, n order to gve a more genera and precse measure of the smarty between two feature subsets, the correatons between the dfferent features of two feature subsets w be used. JC ( k) S S SC foowng smarty ndex JC ( [0,1]) that takes n account the ISSN (onne) (prnt) Bonformaton 11(1) (2015) Bomedca Informatcs 0 k (4) Where S and S 0 are feature subsets seected usng the th batch of re-samped data and the fu data respectvey. SC s the sum of absoute correaton vaues between the dssmar features form S and S 0. It w be computed usng the greedy search agorthm. Step fve, see f the codng s over, that s, orgna feature set S= [1, 2,, n] s nu. If so, end the teraton, otherwse repeat step two to four unt achevng the feature extracton. Measurement of feature extracton robustness The defnton of robustness of feature extracton s the senstvty degree of the resuts of feature extracton agorthm to the changes of tranng set. Accordng to ths defnton, the measurement of feature extracton robustness s measurng the smarty among the optma feature subsets seected by the agorthm. The overa robustness of the agorthm can be cacuated as [14]: 2 k k S(s, s ) 1 1 Stot (5) k(k 1) Where, s represents the resuts of feature extracton from tranng set No. ( 1 k). S(s, s ) represents a smarty measure between two feature extracton resuts s and s. At present, there are many types of smarty measurng methods accordng to dfferent representaton ways of feature extracton resuts. The commony used set method [14] was seected by ths study. In ths method, the robustness s measured by Tanmoto dstance [12]. Tanmoto dstance s used to measure the concdence degree of eements between two feature subsets: s s 2 s s Ss(s, s ) 1 s σ s s s s s s I (s I (s s s 1) 0 ) If the parameter s true, the functon I () returns as 1; otherwse t returns as 0. The vaue nterva of S s s [0, 1]. 0 means that the ntersecton of these two sets s an empty set, and a eements are dfferent. 1 means that these two sets are exacty the same, and a eements are the same. The szes of sets measured by Tanmoto dstance can be the same or dfferent. Resuts: Coon cancer data, CNS data, DLBCL data, and Leukema data [15] were separatey adopted for the smuaton test. Amng at these mcroarray data sets, performance evauaton of feature extracton was conducted for the method proposed n ths paper, Fsher Rato, AW-SVM and PSVM n four aspects of dentfcaton error, AUC vaues, standard devaton and robustness. The resuts were shown n Fgure 2 - Fgure 5. It can be seen from the smuaton resuts (n the Fgure 2a, Fgure 3a, Fgure 4a, & Fgure 5a) that the accuracy of feature dentfcaton for the method proposed n ths paper s better than the other three methods. Takng Fgure 2a as an exampe, MAF-PSVM reazes the mnmum dentfcaton error by ony

5 extractng 150 features, and the dentfcaton error at ths tme s 12.96%. However, the dentfcaton errors of Fsher Rato, AW-SVM and PSVM when extractng 150 features are 14.70%, 15.63% and 15.17%, separatey. The area (AUC) n the ROC curve s usuay used to measure the cassfcaton performance. The arger the AUC vaues are, the better the cassfcaton performance w be. So AUC vaues are used to evauate the cassfcaton performance of severa feature extracton methods durng the smuaton test. The AUC vaues of these four methods are shown n Fgure 2b, Fgure 3b, Fgure 4b, & Fgure 5b. It can be seen from these resuts and the dentfcaton errors of a methods shown n Fgure 2a, Fgure 3a, Fgure 4a & Fgure 5a that when extractng 150 features, the AUC vaues of the method proposed n ths paper are better than the other three methods when coon data and CNS data were used. When DLBCL data and Leuk data are used to test, the AUC vaues of MAF-PSVM are cose to the resut of FR. It aso ndcates that the cassfcaton performance of MAF-PSVM s better than the other three methods whe achevng the most accurate extracton of features. Fgure 4: Performance comparsons on DLBCL data: A) Cassfcaton Error; B)AUC; C) Standard Devaton of Error Estmaton; D) Feature Robustness. The standard devatons of a methods are shown n Fgure 2c, Fgure 3c, Fgure 4c, and Fgure 5c. By anayzng the smuaton resuts, t s known that the performance of the method proposed n ths paper s better than the other three feature extracton methods when Coon and DLBCL data are used. For exampe, on coon data (Fgure 2c), when extractng 150 features, the standard devaton of MAF-PSVM s ony Its dentfcaton accuracy s second ony to the Fsher Rato, whch standard devaton s at ths tme; the standard devaton of AW-SVM s 0.547, and that of PSVM s the argest, whch reaches to ISSN (onne) (prnt) Bonformaton 11(1) (2015) Bomedca Informatcs

6 The robustness resuts of a methods durng the feature extracton process are shown n Fgure 2d, Fgure 3d, Fgure 4d, Fgure 5d. By anayzng the smuaton resuts, t s known that the stabty of the method proposed n ths paper does not perform the best. Ths s because the method proposed n ths paper s an embedded feature extracton method. Compared wth other agorthms, t fuy consders the dependence among features durng the feature extracton process. The resuts of ths treatment are that the feature can be extracted more accuratey and the dentfcaton of patterns can be reazed. The smuaton resuts of the estmated cassfcaton error, AUC, and the standard devaton of error estmaton can fuy confrm ths. It s worth mentonng that whe evauatng the performance of a feature extracton method, we need to comprehensvey consder the accuracy, effcency and stabty of feature dentfcaton of the method. Cassfcaton performance shoud be the frst consderaton because a cassfcaton-neffectve extracton resut does not make any sense. Based on ths and the above smuaton anayss resuts, we can concude that the comprehensve performance of MAF-PSVM proposed n ths paper s better than the other three methods durng feature extracton of mcroarray data. Fgure 5: Performance comparsons on Leukema data: A) Cassfcaton Error; B) AUC; C) Standard Devaton of Error Estmaton; D) Feature Robustness. Concuson: The feature extracton for mcroarray data was dscussed and anayzed n ths paper. Accordng to the dea of custerng and nformaton fuson, a nove feature extracton method, poynoma support vector machne based on mut-agorthm fuson (MAF-PSVM), was put forward. The smuaton resuts of measured data show that the dentfcaton error, AUC vaues, standard devaton of error estmaton of the method proposed n ths paper are better than Fsher Rato, AW-SVM and PSVM. It was found out whe anayzng and comparng the robustness of feature extracton of a methods that the method proposed n ths paper does not perform the best. Ths s because t fuy consdered the dependence among features. The new method buds a compromse between the accuracy of feature dentfcaton and the stabty of feature extracton. For the performance evauaton of a knd of feature extracton method, both the stabty of feature extracton and the dentfcaton performance (such as dentfcaton accuracy and effcency) sha be consdered. ISSN (onne) (prnt) Bonformaton 11(1) (2015) Bomedca Informatcs

7 Therefore, the comprehensve performance of the MAF-PSVM s better than other methods. Ths method s more sutabe for the feature extracton of mcroarray data. Acknowedgement: Fundng: the research s supported by key aboratory of fud and power machnery, mnstry of educaton.xhua unversty (sz ) and aboratory of pattern recognton and artfca ntegence. References: [1] Langey P, Artfca Integence : 245 [2] Guyon I et a. Journa of Machne Learnng Research : 1157 [3] Madonado SR et a. Informaton Scences : 115 [4] Gugezen GZ et a. Book seres n Machne Learnng and Knowedge Dscovery n Databases : 455 [5] Krzek PJ et a. Lecture Notes n Computer Scence : 929 [6] Davs CA et a. Bonformatcs : 2356 [PMID: ] [7] Yang F & Mao KZ, IEEE/ACM Trans Comput Bo Bonform : 1080 [PMID: ] [8] Skurchna M et a. Mutpe Cassfer Systems : 165 [9] Fung ES & Nq MK, Bonformaton : 230 [PMID: ] [10] Yng-Xn L et a. Chnese Journa of Computers : 324 [11] Jang H & Chng WK, Bonformaton : 257 [PMID: ] [12] Ge Y & Seafon SC, Bonformatcs : 2052 [PMID: ] [13] Dwork C et a. In Word Wde Web. 2001: [14] Kaouss A et a. Knowedge and Informaton System : 95 [15] Edted by P Kangueane Ctaton: Jang & Xu, Bonformaton 11(1): (2015) Lcense statement: Ths s an open-access artce, whch permts unrestrcted use, dstrbuton, and reproducton n any medum, for non-commerca purposes, provded the orgna author and source are credted ISSN (onne) (prnt) Bonformaton 11(1) (2015) Bomedca Informatcs