Improving KNN Method Based on Reduced Relational Grade for Microarray Missing Values Imputation

Transcription

1 Improvng KNN Method Based on Reduced Relatonal Grade for Mcroarray Mssng Values Imputaton Yun He, De-chang P Abstract Mcroarray gene expresson data generally suffers from mssng values, whch adversely affects downstream analyss. A new smlarty metrc method called reduced relatonal grade was proposed, based on whch we further presented an mproved KNN method named mputaton algorthm for teratvely estmatng mcroarray mssng values. Reduced relatonal grade s an mprovement of gray relatonal grade. The former can acheve the same performance as the latter, whereas the former can greatly reduce the tme complexty. mputes mssng data teratvely by consderng the reduced relatonal grade as smlarty metrc and expandng the set of canddate genes for nearest neghbors wth mputed genes, whch ncreases the effect and performance of the mputaton algorthm. We selected data sets of dfferent knd, such as tme seres, mxed and non-tme seres, and then expermentally evaluated the proposed method. The results demonstrate that the reduced relatonal grade s effectve and outperforms common mputaton algorthms. Index Terms Gene expresson data, reduced relatonal grade, mputaton, teraton A I. INTRODUCTION N mportant part of human genome proect s to analyze and utlze mcroarray gene expresson data, whch records the abundance of gene transcrpts mrna n cells and contans sgnfcant control nformaton about gene functon. Mcroarray analyss technology on gene expresson data has been wdely used n numerous felds related to nvestgatng drug effects, dentfcaton of crtcal genes for dagnoss or therapy and cancer classfcaton. DNA mcroarray technology [1] s one of the most useful tools for montorng gene expresson level, whch can smultaneously analyze the mrna levels of thousands of genes n partcular cells or tssues on a gene chp. In a DNA mcroarray experment, plenty of DNA probes are fxed on a gven spot, and then hybrdzed wth samples of fluorescence labeled DNA, cdna or RNA, so the gene sequence nformaton can be obtaned by detectng the strength of Manuscrpt receved September 02, 2015; revsed December 14, Ths work was supported n part by the Natonal Natural Scence Foundaton of Chna (U ), the Fundaton of Graduate Innovaton Center n NUAA (kf ). Yun He s wth the College of Computer Scence and Technology, Nanng Unversty of Aeronautcs and Astronautcs, Nanng, Chna. (E-mal: @qq.com). De-Chang P s wth the College of Computer Scence and Technology, Nanng Unversty of Aeronautcs and Astronautcs, Nanng, Chna. (E-mal: dc.p@163.com). hybrdzaton sgnal. Even though mcroarray technology s effcent, accurate and low-cost, t stll suffers from the problem of mssng values due to a varety of nternal or external factors n experments. The mssng value can account for 10% and even n some cases, up to 90% of genes have one or more mssng values [2]. All of such ssues as mage corrupton, hybrdzaton falures, nsuffcent resoluton, or dust and scratches on slde can cause gene mssng values. Many data analyss methods, such as prncpal component analyss (PCA), sngular value decomposton (SVD) and herarchcal clusterng, ust can be appled wth complete datasets wthout mssng values. Besdes, t has been found that mssng values mpede mcroarray data analyss and law dscovery [4]. Expermentng repettvely s regarded as a way to solve mssng data problem, but most of them are complex, costly and tme-consumng. Deletng ncomplete genes before analyzng s a smple approach to obtan a complete dataset. Unfortunately, deleton strategy omts ncomplete genes, so t leads to nsuffcent orgnal dataset especally for multvarate data lke mcroarray gene expresson. It generates serous bas and naccurate concluson when mssng rato s large or data dstrbuton of mssng values s non-random [5]. Mssng value mputaton [6]-[7] s a low-cost and effcent approach to recover all mssng data wthout repettve experments [8]. Substtutng mssng values wth the global or class-condtonal mean/mode has been employed to handle mssng values. Furthermore, mputaton before analyss can sgnfcantly mprove the performance of some machne learnng algorthms whch suffer mssng values, namely C4.5, reference [9] shows that KNN mputaton can enhance predcton accuracy of C4.5 over small software proect datasets. We propose an teratve mputaton algorthm based on a knd of novel dstance metrc for predctng gene expresson mssng values, called Iteratve mputaton based on reduced relatonal grade (). measures the smlarty between a gene wth mssng values and ts nearest neghbors wth the reduced relatonal grade, whch mproves gray relatonal grade. Besdes, mputng mssng values teratvely has hgh data utlzaton by usng ncomplete genes (wth mssng values). We expermentally evaluate the proposed algorthm on dfferent knds of publcly avalable mcroarray datasets and the results demonstrate that the reduced relatonal grade acheve smlar performance as gray one when capturng nearness but greatly reduces the tme complexty. Moreover, algorthm outperforms the

2 conventonal KNN method. The rest of ths paper s organzed as follows. It has a bref revew on related work about mputaton methods n Secton II. In Secton III, we ntroduce the concept of gray relatonal grade. Both the reduced relatonal grade and algorthm are proposed n Secton IV, whle the analyss of them are dscussed n Secton V. In Secton VI we conclude ths paper. II. RELATED WORK Throughout ths paper, the orgnal gene expresson matrx (wth mssng values) s denoted by X, whch contans n genes and m attrbutes (n m), where the -th row represents gene x and x [ x 1, x 2,, xm ]. x denotes the expresson level of gene x n sample. All of complete genes from X complete consttute a complete data matrx X (wthout mssng values). The gene wth mssng values s called a target gene, and the set of genes wth avalable nformaton for mputng mssng values n a target gene s referred as ts canddate genes. K nearest-neghbors (KNN) mputaton s defned to fnd K nearest neghbors from complete data sets for the target gene assumed to have a mssng value n attrbute, and then fll n the target gene wth a weghted average of values n attrbute from the K closest genes [10]. The dea of KNN s that obects close to each other are potentally smlar. For a gene expresson dataset, smlar genes n smlar experments wll have smlar expresson, based on whch KNN mplements the mputaton for gene expresson datasets. Reference [10] presented a comparatve study of three knds of mputaton methods, namely Sngular Value Decomposton-based method, KNN and row average, on gene mcroarray data of dfferent mssng rato. Expermental results showed that KNN mputaton has the best robustness and accuracy. Reference [11] synthetcally analyzed the performances of 23 knds of mputaton methods and demonstrated that KNN mputaton has excellent estmaton accuracy. So far, many researches have mproved KNN mputaton and they manly am at two aspects: the order of mputaton and the metrc dstance. Among these researches, a sequental KNN () mputaton [12] method sorts the target genes (wth mssng values) accordng to ther mssng rato and then mputes genes wth the smallest mssng rate frst. Once all mssng values n a target gene are mputed, the target gene wll be consdered as a complete one. Shell Neghbors mputaton [13] flls n an ncomplete nstance by only usng ts left and rght nearest neghbors wth respect to each attrbute, and the sze of the set of nearest neghbors s determned by cross-valdaton method. Exstng KNN mputaton s based on Mnkowsk dstance, whch s a smple supermposed dstance on dfferent attrbutes of two genes wthout consderng the whole data set. Reference [14] showed that gray relatonal grade s more approprate to capture the proxmty between two nstances than Mnkowsk dstance or others. Sngle mputaton that affords sngle estmaton for each mssng data s a knd of common strategy. Nevertheless, sngle mputaton cannot provde effectve standard errors and confdence ntervals because t gnores the uncertanty of the mputed dataset [14]-[15]. Fllng n mssng values wth teratve mputaton s the alternatve of sngle mputaton. Reference [16] presented a nonparametrc teratve mputaton algorthm and confrmed that t outperforms normal sngle mputaton. Iteratve mputaton based on gray relatonal grade () proposed n [14] uses gray relatonal grade as ts smlar metrc to select K nearest neghbors. can obtan great results when dealng wth mssng values n heterogeneous data (contnuous and dscrete data). However, s more sutable for small datasets, because calculatng the grey correlaton degree costs a lot of tme wth the expanson of the data scale. We propose a mputaton algorthm based on n ths paper, and reduced relatonal grade s desgned as the smlar metrc method, whch can sgnfcantly decrease tme complexty and keep the good mputaton performance compared wth the conventonal relatonal grade. III. GRAY RELATIONAL GRADE Gray System Theory (GST) was developed by Deng n 1982 [17]. The System s good at handlng complex systems to get relable results. Gray Relatonal Analyss (GRA), a method of GST, can seek the numercal relatonshp among dfferent subsystems to measure the smlarty. GRA s used to quantfy the trend relatonshp of two systems or two elements n a system. Generally, f the development tendency between two systems s consstent, the relatonal grade s large. Otherwse, t s small. Gray relatonal coeffcent (GRC) s used to descrbe the smlarty between a target and a canddate gene on the attrbute q n a gven dataset. x and x represent the target and the canddate gene respectvely. The GRC s defned as follows: mn t mn k xk x max t max k xk x GRC(x q,x ) x x max max x x q t k k (1) where s the dstngushng coeffcent, [0,1], generally =0.5. Gray relatonal grade takes mean processng to change each seres gray relatonal coeffcent at all attrbutes nto ther average as smlarty metrc of two genes. m 1 GRG(x, x ) GRC(x, x ) (2) k k m k 1 GRG measures the relatonshp between two genes at a global perspectve, whch overcomes the defcency of Mnkowsk dstance. When selectng K nearest neghbors for a target gene, the larger the value of gray relatonal grade s, the hgher smlarty and the less dfference between genes are. Otherwse, the less smlarty and the more dfference. A. Reduced relatonal grade IV. ALGORITHM Gene expresson data matrx of hgh-dmensonal descrbes the expresson levels of thousands of genes n dfferent

3 expermental condtons. Accordng to (1), we must search the whole canddate dataset one tme when calculatng a GRC of each target gene. For gene data matrx, ths knd of approach wll cost too much tme. In ths paper, we propose a new relatonal coeffcent denoted as Reduced Relatonal Coeffcent (RRC), whch s a knd of reduce of GRC. RRC s approprate for measurng the nearness between a target gene x and a canddate gene x at a specfc attrbutes q: Max k{ xk mn k, xk max k } RRC(x q, x ) x x Max { x mn, x max } q k k k k k Where functon Max{,} takes the larger one of two values. mn k and max k refer to the mnmum and maxmum values of attrbutes k respectvely, whch are easly obtaned durng the process of data nputtng or data normalzng. Wth expermentng n many tmes, we found the calculaton result of formula mn t mn k xk x always tends to zero, and the value of max t max k xk x tends to Max k{ xk mn k, xk max k }. In short, equaton (3) s a smplfcaton of (1), and that means the results of mn t mn k xk x and max t max k xk x, whch need a huge amount of computatons, are approxmately replaced by ther extreme, so RRC can greatly reduce the tme complexty. In (3), RRC (x,x ) s valued n [0, 1]. The greater the q value of RRC(x, x ) s, the larger the smlarty between x q and x wll be. If x q q (3) x, RRC(x,x ) 1. On the contrary, f x q and x have completely dfferent values on attrbute q, the value of RRC(x, x ) s mnmal. The reduced relatonal grade between the target gene x and the canddate gene x s defned as follows: k k m k 1 q m 1 RRG(x, x ) RRC(x, x ) (4) Smlarly, RRG s also the mean processng and the greater the RRG s, the larger smlarty between genes s acheved. Assume that the sze of canddate gene dataset s N*m. For any target gene, when calculatng ts GRG wth all of genes n canddate dataset, the tme complexty s O(N*N*m); whle calculatng the RRG, the tme complexty s reduced to O(N*m). The mprovement of tme complexty s amazng, consderng gene expresson dataset s N m. B. Imputaton For a gven target gene x (the value of x t s mssng), calculates the RRG(x, x ) between x and each canddate gene x, then selects K most smlar genes as ts K nearest neghbors and fnally mputes x t wth the weghted average of ts K neghbor genes at attrbute t: yˆ K t k 1 k kt q w x (5) Where w k s the weght of k-th neghbor gene x k to x. w RRG( x, x ) / RRG( x, x ) (6) k k k 1 k K C. Data normalzaton Generally, the smlarty between two genes s domnated by attrbutes wth greater magntude unts. To avod the bas generated by unt dfference and make the data processng convenent, data should be normalzed before calculatng the reduced relatonal grade for mputaton algorthm. In ths paper, we select Mn-Max normalzaton. Orgnal data wll be normalzed nto [0,1]. Assume that max(t) and mn(t) represent the maxmum and mnmum values on attrbute t respectvely, and x t s the expresson value of gene x on attrbute t. Data s normalzed as follows: xt mn(t) xt (7) max(t) mn(t) D. algorthm desgn uses RRG as smlarty metrc method to select K neghbors and teratvely mputes mssng values wth weghted average untl the termnaton condton s satsfed. Algorthm : mpute Input : Output : gene expresson dataset X wth mssng values complete gene expresson dataset 01: Step1 : Intalzaton 02 : FOR each target gene x n X 03: replece all mssng values n x wth row averages 04 : END FOR / / obtanng a complete matrx X 05: 06 : Step2 : Imputaton 07 : h 0 08: complete( 0) 09 : h / / the kth teratve mputaton( h 1,2,3 10 : 11: 12 : normalze X REPEAT complete, get X FOR each target gene x n X complete construct the canddate gene dataset based onx 13: compute RRG x, x 14 : 15: 16: FOR each canddate gene x END FOR elect the K nearest genes mpute all mssng values n x usng equaton(6)? 17 : END FOR / / obtanng X 18: complete( h) 2 ( h) N ( h) (h 1) ˆ ˆ 1 (h) 20 : m maxmu ) complete( h1) FOR each mputaton value y ˆ X, yˆ X 19 : compute 1 N (y y ) 20 : END FOR ( h) complete( h) (h 1) complete( h1) UNTIL ( or reach N the m number of teraton) 3 Heren, generally convergence accuracy s 10 and the maxmum number of teraton s Nm 10. The accuracy of mputaton algorthm s evaluated by the root mean square error () as follows: N 2 (y ˆ y) 1 N (8) Where y s the actual value; ŷ s the mputed value, and N s the total number of mssng values. The smaller the s, the better mputaton accuracy wll be, and t means the estmated value s close to the exact one.

4 A. Data V. EXPERIMENTS AND ANALYSIS Gene expresson data from mcroarray technology s a matrx, whch presents of expresson level of varous genes (rows) under dfferent expermental condtons (columns). In ths study, we used fve publc avalable mcroarray data sets n three dfferent types obtaned from the publc genetc databases: Two data sets (data sets NTS1 and NTS2) from the study n yeast Saccharomyces cerevsae consst of non-tme seres mcroarray data. The NTS1 s a comparson of cdna comng from mex67-5 temperature-senstve mutant and that from Mex67 wldtype stran both at 37, whle NTS2 compares cdna from yra1-1 temperature-senstve mutant wth that from Yra1 wldtype stran both at 37 too. Both NS1 and NS2 have sx samples representng sx experments. The thrd and the fourth data sets (data sets TS1 and TS2) are tme seres data. TS1 tested the transcrptonal response of S.cerevsae to aeraton after anaerobc growth. The sx attrbutes of TS1 stands for how long t has been aerated. TS2 contans the data from a cdc15-2-based sychronsaton, whch s composed of 25 attrbutes mplyng dfferent culture tme. The ffth data set belongs to a study of gene expresson n Salmonella enterca after treatng wth 2mM hydrogen peroxde. It s termed by MIX, and contans both tme and non-tme course data. Data set TABLE I DIMENSION AND TYPES OF THE GENE EXPRESSION DATASETS Orgnal Data Complete Data row column row column Mssng Rate Type the type of data and mssng rato, but n theory, there s no exact formula. Reference [18] desgned a procedure for selectng K automatcally, and demonstrated that K can be set to any value n the range Reference [10] addressed ths queston n KNN method and reported the best results for K s n the range It was fond expermentally that when K s valued n [10, 15], the fluctuaton of K can hardly affect the performance of algorthms. Therefore, we take K=10 n subsequent experments. C. Expermental evaluaton on RRG In order to assess the performance of RRG, GRG was used as a reference. Whether RRG can mprove the system property when compared wth GRG was ndrectly showed by the comparson results of and, snce the only dstncton of these two algorthms les n dfferent smlarty metrc methods. Comparson results are presented n Fg. 1, where Fg. 1(a) dsplay the on TS1 dataset wth mssng rato 5%, 10%, 15%, 20%, respectvely. Smlarly, Fg. 1(b) and Fg. 1(c) are the results on datasets NTS1 and MIX, separately. The results show that the of the second teraton dramatcally decreases when compared wth the frst teraton mputaton both n and algorthms at dfferent mssng rato. So teratve procedure can refne the mputaton value. From Fg 1, the two curves, whch descrbe the performance of and, bascally concde n each subfgure. That ndcates the two mputaton algorthms based on RRG and GRG approxmately have the same mputaton accuracy, whch means that consderng neghbors selecton and weght calculaton, the same results as GRG are acheved by applyng RRG as the smlarty metrc. NTS % NTS % TS % TS % MIX non-tme seres non-tme seres tme seres tme seres mxed seres TS1 5% TS1 10% All of these fve data sets suffer mssng problems, especally TS2. Frstly, these data sets need to be pre-processed by removng genes wth mssng values to obtan complete data sets. Table I shows the dmensons of the orgnal data matrces before and after pre-processng (complete data). Before expermentng, mssng values at dfferent rato were ntroduced nto these fve complete data sets randomly, and then they are analyzed by mputaton algorthm. B. Parameter K KNN method or ts varatons have one thng n common: An approprate K must be selected. The value of K can affect the predcton of KNN method or ts varatons. If K s too large, the smlarty of some neghbors wll be nsuffcent, and too much neghbors may result n mputaton performance reducton; f K s too small, t wll strengthen a few neghbors and the negatve mpact of nose data wll ncrease smultaneously. The value of K s emprcally found related to Iteraton TS1 15% Iteraton Iteraton (a) Expermental results on TS1 TS1 20% Iteraton

5 NTS1 5% NTS1 10% effectvely as well. TABLE II TIME-CONSUMING OF AND FOR TEN ITERATIONS ON TS1 Consumng tme (ms) Dataset TS mssng rato 5% 10% 15% 20% Iteraton NTS1 15% Iteraton NTS1 20% TABLE III TIME-CONSUMING OF AND FOR TEN ITERATIONS ON NTS1 Consumng tme (ms) Dataset NTS1 mssng rato 5% 10% 15% 20% Iteraton (b) Expermental results on NTS1 MIX 5% Iteraton MIX 15% Iteraton MIX 10% Iteraton MIX 20% TABLE IV TIME-CONSUMING OF AND FOR TEN ITERATIONS ON MIX Consumng tme (ms) Dataset MIX mssng rato 5% 10% 15% 20% Overall, RRG has same performance as GRG on mputaton accuracy. Moreover, RRG greatly reduces the tme complexty. D. Expermental evaluaton on In order to evaluate the proposed algorthm wth some mcroarray data sets, two algorthms were selected n our experments. One s the algorthm of sequental KNN mputaton (), the other s the teratve KNN mputaton () [17] by changng normal KNN method nto an teratve mputaton based on teratve prncple., and are appled to fve datasets TS1, TS2, NTS1, NTS2, and MIX at dfferent mssng rato 5%, 10%, 15%, 20% and 25%. The expermental results n showed the phenomenon of predcton accuracy for these three mputaton algorthms n Fg TS Iteraton Iteraton (c) Expermental results on MIX Fg. 1. Expermental results for ten teratons on TS1, NTS1 and MIX datasets for and algorthms. Table II presents tme-consumng scale of the comparson experments on dataset TS1, smlarly Table III and Table IV dsplay the tme-consumng of and on dataset NTS1 and dataset MIX, respectvely. Obvously, RRG proposed n ths paper compared wth GRG can decrease computatonal complexty sgnfcantly, and reduce runtme Mssng rato (d) Comparatve results on TS1

6 TS Mssng rato (e) Comparatve results on TS2 NTS Mssng rato (f) Comparatve results on NTS1 NTS Mssng rato (g) Comparatve results on NTS2 MIX Mssng rato (h) Comparatve results on MIX Fg. 2. Expermental results on datasets TS1, TS2, NTS1, NTS2 and MIX for three algorthms. From Fg 2, we fnd that the accuraces of algorthms decrease whle the mssng rato ncreases generally. Fg. 2(d), Fg. 2(e) and Fg. 2(h) presented the results on tme seres datasets TS1, TS2, and mxed dataset MIX, respectvely. The mputaton accuraces of, and are close to each other over these three datasets, but we can stll fnd out that has the smallest estmaton error. The advantage of s very obvous on non-tme seres datasets NTS1 and NTS2 dsplayed n Fg. 2(f) and Fg. 2(g). The performance of algorthms depends on the type of datasets, and s more approprate for non-tme seres datasets. Hence, compared wth and algorthms, our method has the best performance. VI. CONCLUSIONS In ths work, we proposed a new smlarty metrc method named reduced relatonal grade (RRG), whch s an mprovement of GRG. The performance of RRG was ndrectly assessed and compared wth GRG over three datasets of dfferent types at dfferent mssng rato. Consderng estmaton accuracy, RRG and GRG have the same smlar results, but RRG sgnfcantly decreases the tme complexty. Therefore, RRG s a knd of more effcent method to capture nearness between two nstances compared wth GRG. Based on RRG, we further proposed an mproved KNN method for estmatng mssng values on mcroarray gene expresson data, named mputaton. s ablty to effcently utlze data and t also can mpute mssng values teratvely. We expermentally evaluated the performance of compared wth and algorthms on fve datasets at dfferent mssng rato. The results show that works well on mputng mssng values. It should also be noted that the approprate convergence accuracy and the maxmum number of teraton can affect the performance of mputaton, so how to effcently and reasonably determne them would be further researched. REFERENCES [1] Hohesel J D. Mcroarray technology: beyond transcrpt proflng and genotype analyss, Nature Revews Genetcs, vol. 7, no. 3, pp , [2] Brevern A G D, Hazout S, Malpertuy A. Influence of mcroarrays experments mssng values on the stablty of gene groups by herarchcal clusterng, BMC Bonformatcs, vol. 5, no. 1, pp , [3] Yang Y H, Buckley M J, Dudot S, et al. Comparson of methods for mage analyss on cdna mcroarray data, Journal of Computatonal and Graphcal Statstcs, vol. 11, no. 1, pp , [4] Junger W L, de Leon A P. Imputaton of mssng data n tme seres for ar pollutants, Atmospherc Envronment, vol. 102, pp , Feb [5] García-Laencna P J, Sancho-Gómez J L, Fgueras-Vdal A R, et al. K nearest neghbours wth mutual nformaton for smultaneous classfcaton and mssng data mputaton, Neurocomputng, vol. 72, no. 7-9, pp , [6] Fukuta K, Okada Y. LEAF: Leave-one-out Forward Selecton method for nformaton gene dscovery n DNA mcroarray data, IAENG Internatonal Journal of Computer Scence, vol. 38, no. 2, pp , [7] Okada Y, Okubo K, Horton P, et al. Exhaustve search method of gene expresson modules and ts applcaton to human tssue data, IAENG nternatonal ournal of computer scence, vol. 34, no. 1, pp , 2007.

7 [8] Moorthy K, Saber Mohamad M, Ders S. A Revew on Mssng Value Imputaton Algorthms for Mcroarray Gene Expresson Data, Current Bonformatcs, vol. 9, no. 1, pp , [9] Song Q, Shepperd M, Chen X, et al. Can k-nn mputaton mprove the performance of C4.5 wth small software proect data sets? A comparatve evaluaton, The Journal of Systems and Software, vol. 81, no. 12, pp , [10] Troyanskaya O, Cantor M, Sherlock G, et al. Mssng value estmaton methods for DNA mcroarrays, Bonformatcs, vol. 17, no. 6, pp , [11] Lew A W C, Law N F, Yan H. Mssng value mputaton for gene expresson data: computatonal technque to recover mssng data from avalable nformaton, Brefngs n Bonformatcs, vol. 12, no. 5, pp , [12] Meng F, Ca C, Yan H. A Bcluster-Based Bayesan Prncpal Component Analyss Method for Mcroarray Mssng Value Estmaton, Bomedcal and Health Informatcs, vol. 18, no. 3, pp , [13] Zhang S. Shell-neghbor method and ts applcaton n mssng data, Appled Intellgence, vol. 35, no. 1, pp , [14] Zhang S. Nearest neghbor selecton for teratvely KNN mputaton, The Journal of Systems and Software, vol. 85, no. 11, pp , [15] Rgg S, Rgg D, Rgg F. Handlng mssng data for the dentfcaton of charged partcles n a multlayer detector: A comparson between dfferent mputaton methods, Nuclear Instruments and Methods n Physcs Research A, vol. 780, pp 81-90, Apr [16] Zhang S, Jn Z, Zhu X. NIIA: Nonparametrc Iteratve Imputaton Algorthm, Berln: Sprnger-Verlag, [17] Song Q, Shepperd M. Predctng software proect effort: A grey relatonal analyss based method, Expert Systems wth Applcatons. vol. 38, no. 6, pp , [18] Brás L P, Menezes J C. Improvng cluster-based mssng value estmaton of DNA mcroarray data, Bomolecular Engneerng, vol. 24, no. 2, pp , 2007.