Verdict Accuracy of Quick Reduct Algorithm using Clustering, Classification Techniques for Gene Expression Data

Transcription

1 Verdict Accuracy of Quick Reduct Algorith using Clustering, Classification Techniques for Gene Expression Data T.Chandrasekhar 1, K.Thangavel 2 and E.N.Sathishkuar 3 1 Departent of Coputer Science, eriyar University, Sale, Tailnadu , India ch_ansekh80@rediffail.co 2 Departent of Coputer Science, eriyar University, Sale, Tailnadu , India drktvelu@yahoo.co 3 Departent of Coputer Science, eriyar University, Sale, Tailnadu , India en.sathishkuar@yahoo.in Abstract In ost gene expression data, the nuber of training saples is very sall copared to the large nuber of genes involved in the experients. However, aong the large aount of genes, only a sall fraction is effective for perforing a certain task. Furtherore, a sall subset of genes is desirable in developing gene expression based diagnostic tools for delivering reliable and understandable results. With the gene selection results, the cost of biological experient and decision can be greatly reduced by analyzing only the arker genes. An iportant application of gene expression data in functional genoics is to classify saples according to their gene expression profiles. Feature selection (FS) is a process which attepts to select ore inforative features. It is one of the iportant steps in knowledge discovery. Conventional supervised FS ethods evaluate various feature subsets using an evaluation function or etric to se lect only those features which are related to the decision classes of the data under consideration. This paper studies a feature selection ethod based on rough set theory. Further K-Means, Fuzzy C-Means (FCM) algorith have ipleented for the reduced feature set without considering class labels. Then the obtained results are copared with the original class labels. Back ropagation Network (BN) has also been used for classification. Then the perforance of K-Means, FCM, and BN are analyzed through the confusion atrix. It is found that the BN is perforing well coparatively. Keywords: Rough set theory, Feature Selection, Gene Expression, Quick Reduct, K-eans, Fuzzy C eans, BN. 1. Introduction Feature selection is the process of choosing the ost appropriate features when creating the odel of the process. Most of the feature selection ethods are applied across the entire data set. Once such genes are chosen, the creation of classifiers on the basis of the genes is another undertaking. If we survey the established investigations in this field, we will find that alost all the accurate classification results are obtained based on ore than two genes. Rough sets have been used as a feature selection ethods by any researchers aong the Jensen and Schen, Zhong et al, Wang and Hu et al. The Rough set approach to feature selection consists in selecting a subset of features which can predict the classes as well as the original set of features. The optial criterion for Rough set feature selection is to find shortest or inial reducts while obtaining high quality classifiers based on the selected features. Here we propose a feature selection ethod based on rough set theory for reducing genes fro large gene expression database [1, 4]. Discriinant analysis is now widely used in bioinforatics, such as distinguishing cancer tissues fro noral tissues. A proble with gene expression analysis or with any large diensional data set is often the selection of significant variables (feature selection) within the data set that would enable accurate classification of the data to soe output classes. These variables ay be potential diagnostic arkers too. There are good reasons for reducing the large nuber of variables: 1) An opportunity to scrutinize individual genes for further edical treatent and drug developent. 2) Diension reduction to reduce the coputational cost. 3) Reducing the nuber of redundant and unnecessary variables can iprove inference and classification. 4) More interpretable features or characteristics that can help identify and onitor the target diseases or function types [5]. The rest of the paper is organized as follows: Section 2, briefs about the Rough set theory. Section 3 describes the clustering techniques. Section 4 briefs about classification techniques. Section 5 explains briefly about experiental

2 analysis and results. Section 6 presents a conclusion for this paper. 2. Rough Set Theory Rough set theory (awlak, 1991) is a foral atheatical tool that can be applied to reducing the diensionality of datasets. The rough set attribute reduction ethod reoves redundant input attributes fro datasets of discrete values, all the while aking sure that no inforation is lost. The approach is fast and efficient, aking use of standard operations fro conventional set theory [3]. Definition: Let U be a universe of discourse, X U, and R is an equivalence relation on U. U/R represents the set of the equivalence class of U induced by R. The positive region of X on R in U, is defined as pos(r,x)= U {Y U/R Y X}. The partition of U, generated by IND () is denoted U/. If (x, y) IND (), then x and y are indiscernible by attributes fro. The equivalence classes of the - indiscernibility relation are denoted [x]p. The indiscernibility relation is the atheatical basis of rough set theory. Let X U, the -lower approxiation X and - upper approxiation X of set X can be defined as: X= { x U [x]p X } (1) X= { x U [x]p X φ } (2) Let, Q A be equivalence relations over U, then the positive, negative and boundary regions can be defined as: OS ( Q) = NEG BND X U ( Q) = U ( Q) = X U X X U X X X U X An iportant issue in data analysis is discovering depende ncies between attributes dependency can be defined in the following way. For, Q A, depends totally on Q, if and only if IND () IND (Q). That eans that the partition generated by is finer than the partition generated by Q. We say that Q depends on in a degree 0 k 1 denoted k Q, if k = γ ( Q ) = OS ( Q ) U (3) (4) (5) (6) If k =1, Q depends totally on, if 0 k 1, Q depends partially on, and if k=0 then Q does not depend on. In other words, Q depends totally (partially) on, if all (soe) objects of the universe U can be certainly classified to blocks of the partition U/Q, eploying. In a decision syste the attribute set contains the condition attribute set C and decision attribute set D, i.e. A = C U D. The degree of dependency between condition and decision attributes, γc(d), is called the quality of approxiation of classification, induced by the set of decision attributes[6,10]. 2.1 Quick Reduct Algorith The reduction of attributes is achieved by coparing equivalence relations generated by sets of attributes. Attributes are reoved so that the reduced set provides the sae quality of classification as the original. A reduct is defined as a subset R of the conditional attribute set C such that γr(d)=γc(d). A given dataset ay have any attribute reduct sets, so the setr of all reducts is defined as: Rall = {X X C,γX(D) = γc(d); γx {a}(d) γx(d), a X}. (7) The intersection of all the sets in Rall is called the core, the eleents of which are those attributes that cannot be eliinated without introducing ore contradictions to the representation of the dataset. For any tasks (for exaple, feature selection), a reduct of inial cardinality is ideally searched for. That is, an attept is to be ade to locate a single eleent of the reduct set Rin Rall: Rin = {X X Rall, Y Rall, X Y }. (8) The Quick Reduct algorith shown below[8, 9], it searches for a inial subset without exhaustively generating all possible subsets. The search begins with an epty subset; attributes which result in the greatest increase in the rough set dependency value are added iteratively. This process continues until the search produces its axiu possible dependency value for that dataset (γc(d)). Note that this type of search does not guarantee a inial subset and ay only discover a local iniu. QUICKREDUCT(C, D) C, the set of all conditional features; D, the set of decision features. (a) R {} (b) Do (c) T R (d) x (C-R) (e) ifγr{x} (D) >γt (D)

3 Where γr(d)=card(osr(d)) / card(u) (f) T R{x} (g) R T (h) untilγr(d) = = γc(d) (i) return R It starts off with an epty set and adds in turn, one at a tie, those attributes that result in the greatest increase in the rough set dependency etric, until this produces its axiu possible value for the dataset. Other such techniques ay be found in [8, 9] 3. Clustering Techniques Clustering is the process of grouping data into clusters, where objects within each cluster have high siilarity, but are dissiilar to the objects in other clusters. Siilarities are assessed based on the attributes values that best describes the objects. Often distance easures are used for the purpose. Clustering has its roots in any areas, including data ining, statistics, biology, and achine learning. In this work K-Means, FCM and BN algoriths which are used to classify the data. 3.1 K-Means Algorith K-Means algorith (MacQueen, 1967) is one of a group of algoriths called partitioning ethods. The k-ean algorith is very siple and can be easily ipleented in solving any practical probles. The k-eans algorith is the best-known squared error-based clustering algorith [11]. Consider the data set with n objects, i.e., S = {x i : 1 i n}. 1) Initialize a k-partition randoly or based on soe prior knowledge. i.e. {C 1, C 2, C 3,.., C k }. 2) Calculate the cluster prototype atrix M (distance atrix of distances between k-clusters and data objects). M = { 1, 2, 3,., k } Where i is a colun atrix 1 n. 3) Assign each object in the data set to the nearest cluster - C i.e. x j C if x j - C x j C i 1 j k, j Where j=1, 2, 3,., n. 4) Calculate the average of each cluster and change the k- cluster centers by their averages. 5) Again calculate the cluster prototype atrix M. 6) Repeat steps 3, 4 and 5 until there is no change for each cluster. The k-eans algorith is the ost extensively studied clustering algorith and is generally effective in producing good results. The ajor drawback of this algorith is that it produces different clusters for different sets of values of the initial centroids. Quality of the final clusters heavily depends on the selection of the initial centroids [12]. 3.2 Fuzzy C Means Fuzzy clustering allows each feature vector to belong to ore than one cluster with different ebership degrees (between 0 and 1) and vague or fuzzy boundaries between clusters. Fuzzy c-eans (FCM) is a ethod of clustering which allows one piece of data to belong to two or ore clusters. This ethod (developed by Dunn in 1973 and iproved by Bezdek in 1981) is frequently used in pattern recognition [15]. Algorith Steps: Step-1: Randoly initialize the ebership atrix using this equation, ( )=1 i = 1,2,..k (9) Step-2: Calculate the Centroid using equation, = [ ( )] [ ( )] (10) Step-3: Calculate dissiilarly between the data points and Centroid using the Euclidean distance. Step-4: Update the New ebership atrix using the equation, [ 1 ] ( )= [ 1 ] (11) Here is a fuzzification paraeter, The range is always {1.25, 2} Step-5: Go back to Step 2, unless the centroids are not changing. 4. Classification Techniques 4.1 Back ropagation Networks (BN) BN is an inforation-processing paradig that is inspired by the way biological nervous systes[13,14],

4 such as the brain, process inforation. The key eleent of this paradig is the novel structure of the inforation processing syste. It is coposed of a large nuber of highly interconnected processing eleents (neurons) working in unison to solve specific probles. The data studied by rough sets are ainly organized in the for of decision tables. One decision table can be represented as S = (U, A=C U D), where U is the set of saples, C the condition attribute set and D the decision attribute set. We can represent every gene expression data with the decision table like Table 2. Table2. Microarray data decision table. Decision Condition attributes(genes) Sa attributes ples Gene 1 Gene 2... Gene n Class label 1 g(1,1) g(1,2) g(1,n) Class(1) 2 g(2,1) g(2,2) g(2,n) Class(2) g(,1) g(,2) g(,n) Class() Fig 1: BN Architecture Developing a neural network involves first training the network to carry out the desired coputations. The feedcoonly used for forward neural network architecture is supervised learning. Feed-forward neural networks contain a set of layered nodes and weighted connections between nodes in adjacent layers. Feed-forward networks are often trained using a back propagation-learning schee. Back propagation learning works by aking odifications in weight values starting at the output layer then oving backward through the hidden layers of the network. Neural networks have been criticized for their poor interpretability, since it is difficult for huans to interpret the sybolic eaning behind the learned weights. Advantages of neural networks, however, include their high tolerance to noisy data as their ability to classify patternson which they have not been trained [13,14]. 5. Experiental Results 5.1 Data Sets We use four datasets: leukeia, breast cancer, lung cancer and prostate cancer which are available in the website: [2]. the gene nuber and class contained in four datasets are listed in Table 1. Table1: Suary of the four gene expression datasets. Dataset #Gene Class Leukeia 7129 ALL/AML rostate Tuor/Noral Breast Relapse/Non Relapse Lung 7129 Tuor/Noral In the decision table, there are saples and n genes. Every saple is assigned to one class label. Each gene is a condition attribute and each class is a decision attribute. g(x, y) signifies the expression level of gene y in saple x. [2]. 5.2 Data re-processing, Gene Selection Before applying feature selection algorith all the conditional attributes (saples) are discretized using K- Means discretization [16]. Let us considered U is the set of saples, C the condition attribute set and D the decision attribute set. By applying Quick Reduct Algorith, In prostate gene dataset, gene #20 and #11154 are identified, where as in leukeia dataset gene #4 and #3252 are identified, in breast cancer dataset gene #3 and #22019 are identified, finally in lung cancer dataset gene #4817 as best attribute for finding appropriate decision. Table 3: Features selected by Quick Reduct Algorith Gene Data Leukeia rostate Breast Lung 5.3 Classification erforance Identified Attributes (Genes) #4, #3252 #20, #11154 #3, #22019 #4817 In this section the selected data is clustered by the K- Means and FCM algorith. The data presented in Table 4 and 5 shows the classification perforance of True ositive (T) rate, True Negative (TN) rate, False ositive (F) rate, and False Negative (FN) rate as previously described. Table 5 shows classification perforance of Back ropagation Network. Results are presented both in

5 ters of classification accuracy and classification error [7]. Table 4: K-Means Classification erforance Rate Gene Data K-Means T F TN FN Leukeia rostate Breast Lung Table 5: FCMs Classification erforance Rate FCM Gene Data T F TN FN Leukeia rostate Breast Lung When coparing classification results, where the BN ethod shows a high in classification accuracy, which is deonstrated in Fig. 2. Error Table 7: K-Means, FCM and BN Classification Error Gene Data K-Means FCM BN Leukeia rostate Breast Lung Classification Error K-Means FCM BN Accuracy Table 6: K-Means, FCM and BN Classification Accuracy Gene Data K-Means FCM BN Leukeia rostate Breast Lung Classification Accuracy Fig 2: K-Means, FCM and BN Classification Accuracy K-Means FCM BN Fig 3: K-Means, FCM and BN Classification Error Fig 2 and 3 deonstrated the classification accuracy and error rate of Quick reduct algorith. 6. Conclusion In this paper, Quick reduct algorith based on rough set theory has been studied for gene expression datasets. The reduced feature set has been used to cluster the data using K-Means and FCM algoriths with considering decision attributes. The perforance was evaluated using confusion atrix with positive and negative class values. Further, the selected features with class labels were classified using Back ropagation Network. It was observed that the perforance of the BN is significant. References [1] Jensen, R. and Shen, Q. (2003) Finding rough set reducts with ant colony optiization, roceedings UK Workshop on Coputational Intelligence, pp [2] Xiaosheng Wang, Osau Gotoh, Classification Using Single Genes, pp [3] awlak, Z. (2002) Rough Sets and Intelligent Data Analysis, Inforation Sciences, Vol. 147, pp [4] Changjing Shang and QiangShen, Aiding Classification of Gene Expression Data with Feature Selection: A Coparative Study, International Journal of Coputatonal Intelligence Research. ISSN Vol.1, No.1 (2005), pp

6 [5] Liang Goh, Qun Song, and Nikola Kasabov, A Novel Feature Selection Method to Iprove Classification of Gene Expression Data, Conferences in Research and ractice in Inforation Technology, Vol. 29. [6] radiptamaji and Sankar K. al, Fuzzy rough sets for inforation easures and Selection of relevant genes fro icroarray data, IEEE transactions on systes, an, and cybernetics part b: cybernetics, vol. 40, no. 3, June 2010 [7] C.Velayutha, K.Thangavel, Unsupervised Feature Selection Using Rough Set. roceeding on International Conference, Eerging Trends in Coputing(ICETC- 2011), Mar [8] K.Thangavel,. Jaganathan, A. ethalakshi, M.Karnan, Effective Classification with Iproved Quick Reduct For Medical Database Using Rough Syste, BIME Journal, Volue (05), Issue (1), Dec., [9] K.Thangavel, A. ethalakshi, Feature Selection for Medical Database Using Rough Syste, AIML Journal, Volue (6), Issue (1), January, 2006 [10] QiangShen, Alexios Chouchoulas, A Rough Fuzzy Approach For Generating Classification Rules,ww.e lsevier.co/locate/patcog, attern Recognition 35 (2002) [11] arvesh Kuar, SiriKrishanWasan, Coparative Analysis of k-ean Based Algoriths. IJCSNS International Journal of Coputer Science and Network Security, VOL.10 No.4, April [12] K. A. Abdul Nazeer, M.. Sebastian, Iproving the Accuracy and Efficiency of the k-eans Clustering Algorith. roceedings of the World Congress on Engineering 2009 Vol I WCE 2009, July 1-3, 2009, London, U.K. [13] AshaGowdaKaregowda, A.S. Manjunath, M.A. Jayara, Application of Genetic Algorith Optiized Neural Network Connection Weights for Medical Diagnosis of ia Indians Diabetes. International Journal on Soft Coputing ( IJSC ), Vol.2, No.2, May [14] ing Chang and Jeng-Shong Shih, The Application of Back ropagation Neural Network of Multi-channel iezoelectric Quartz Crystal Sensor for Mixed Organic Vapours.Takang Journal of Science and Engineering, Vol. 5, No. 4, pp (2002). [15] Binu Thoas, Raju G., and SonaWango, A Modified Fuzzy C-Means Algorith for Natural Data Exploration. World Acadey of Science,Engineering and Technology [16] Sellappanalaniappan, Tan Ki Hong, Discretization of Continuous Valued Diensions in OLA Data Cubes.IJCSNS International Journal of Coputer Science and Network Security, VOL.8 No.11, Noveber 2008.