Classifying Datasets Using Some Different Classification Methods

Transcription

1 Intenational Jounal of Engineeing and Technical Reseach (IJETR) ISSN: -869 (O) (P), Volume-5, Issue, June 6 Classifying Datasets Using Some Diffeent Classification Methods Sanaa Abou Elhamayed Abstact The classification methods ae used to classify a new data instance based on the known classifications of the obsevations in the taining set. The main objective of this wok is to compae the pefomance of thee classification methods. The methods ae called the Classification and Regession Tee (CART), K-Neaest Neighbou, (KNN), and Pincipal Component Analysis (PCA). Such methods ae applied on diffeent datasets. Any dataset is patitioned into two sets, one of them is taining set and the othe one is testing set. The pefomance of each method is measued using some measuable citeia. This includes: non-,,, pecision,, and. The adopted methods ae evaluated and compaed using some chosen datasets as testbeds. The coss validation is applied to impove and assesst the pefomance of the classification methods. The classification methods ae implemented and oped by applying MATLAB vesion-4 fo calculating the significant paametes which have a diect effect on the pefomance of the classification methods. Index Tems Classification Methods, CART, KNN, PCA, Coss Validation, Qualification Paametes I. INTRODUCTION A lot of eseach woks have been pesented concening the classification poblems. This involves using classification algoithms, softwae tools, datasets, and classification cies. Examples of such published effots ae biefly mentioned as follows: [] conducted an expeiment using the WEKA envionment by handling fou classification algoithms namely ID, J48, Simple Classification And Regession Tee (CART) and Altenating Decision Tee on the spam dataset. Such classification algoithms ae used to categoize the s as spam o non-spam. The algoithms ae analyzed and compaed in tems of classification. Fom the esults it was found that the highest pefomance is fo the J48 fo the spam datasets containing 46 instances with58 attibutes pe each. [] used decision tee classification algoithms to classify the data into coectly and incoectly instances. Thei wok shows the pocess of WEKA analysis and selection of attibutes to be mined. Also, they povided an evaluation based on the evolutionay classification algoithms to thei datasets and measued the of the obtained esults. [5] pesented an intoduction of text classification and compaed some existing s accoding to time Sanaa Abou Elhamayed, Electonic Reseach Institute, Giza, Egypt complexi, pincipal, and pefomance. They veified that infomation Gain and Chi squae statistics ae the most commonly used and well pefomed methods fo featue selection. Also, they veified that no single epesentation scheme and can be mentioned as a geneal model fo any application. Diffeent algoithms pefom diffeently depending on data collection. [6] pesented a compaative study of diffeent classification and clusteing techniques using WEKA. They tested J48, ID, Bayes netwok classification algoithms. Accoding to thei compaison they veified that J48 algoithm gives the best pefomance consideing both and speed. [7] pesented some classification techniques which ae decision tee, Bayesian netwoks, k-neaest neighbou, neual netwok, and suppot vecto machine. These techniques ae used to uncove hidden pattens within lage amounts of data and pedict thei futue behaviou. They veified that the good data is the fist equiement fo good data exploation. [8] evaluated the pefomance of data mining classification algoithms on vaious datasets. They found that most algoithms can classify datasets with both nominal and numeic class values. But bayes algoithms classify datasets with only nominal class values wheeas linea egession, M5 ules classify datasets only with numeic class value. They found that J48 algoithm pefomed well with % coectly classified instances with least time. [9] applied five diffeent classification methods fo classifying diffeent pes of data based on thei size. The five classification methods ae decision tee, lazy leane, ules based, naive bayes, and egession. They poposed the data using WEKA tool which povides woking with attibutes section and evaluate the pefomance of the classification algoithms accoding to the and the. They found that the lazy leane is much bette than the othes in big datasets while the ules basedis good in small datasets. They also found that the decision tee does not change when a dataset is changed. [] analyzed the pefomance of thee Meta classification algoithms namely attibuted selected, filteed and logitboost. They analyzed the pefomance of the algoithms by evaluating the classification and. They classified the compute files accoding to thei extension and used the WEKA tool fo analyzing the pefomance of the classification algoithms. The dataset is collected fom the compute systems and contains 9 instances and fou attibutes. Befoe stating the classification pocess the taining and the test data ae educed by attibute 48

2 Classifying Datasets Using Some Diffeent Classification Methods selection. Fom the expeimental esults it is obseved that the logitbbost is bette than the othe algoithms. [] used two classification algoithms namely J48 and multilaye pecepton fo seveal datasets fo making a decision which is bette based on the conditions of the datasets. The confusion matix is used to evaluate the classification quali, whee the sum of diagonal is the numbe of coectly classified instances else ae incoectly classified. They found that the multilaye pecepton is a bette algoithm in the most of the cases. [] pesented a study to find the best classification algoithm among bayesien and lazy s. The dataset is collected fom the compute files and has 8 instances and fou attibutes. Bayesian algoithms pedict the class depending on the pobabili of belonging to that class. Lazy algoithms pedict the class depending on the distance fom the test instance and its neighbous. They evaluated the quali of the classifying algoithms consideing some measuable citeia such as:,, F measue, Receive opeating chaacteistics, Tue positive, and kappa statistics. Fom the expeimental esults it is obseved that the lazy s k-neaest neighbou is bette than the othe techniques. The oganization of this pape is as follows: Section pesents the chosen classification methods. Section implements the diffeent datasets and the effective paametes in the classification pocess while section 4 discusses the esults. Section 5 concludes the whole wok. II. THE CHOSEN CLASSIFICATION METHODS. Classification Using CART Method Decision tee is one of the most impotant knowledge epesentation methods which attempt to build a top-down method to educe dimensionali. The eduction of dimensionali is used by eliminating duplicated o edundant attibutes o neglecting less impotant ones. The decision tee method is used in diffeent applications of science and medicine. Decision tees ae tees that classify instances by soting them based on featues values. Each node in a decision tee epesents a featue in an instance to be classified, each leave epesents a class label, and each banch epesents a conjunction of featues that lead to those class labels (a value that a node can assume) [], []. This method is based on ule induction. A distinction between continuous and categoical vaiables is equied to descibe the splitting ules. If the dataset has numeical vaiables then the numbe of possible splits at a given node is one less than the numbe of its distinctly obseved values. If the dataset has M categoical vaiables then those vaiables will be splitted into M subsets. In this method the data set is ecusively splitted into smalle subsets whee each subset contains objects belonging to as few categoies as possible. Fo the best splitting node, gain atio and gini index ae used. To decease the height of the tee, the iegulai of each node must be educed. So, the iegulai I is computed fo all the featues by applying whee p(c) is the popotion of the data that belongs to the class c. The final classification model consists of a tee that defines the classification ule. The steps of the method can be summaized as follows: Input: dataset (a set of featue vectos epesenting instances). Ceate the oot of the tee with the featue that maximizes the gain atio G. Detemine fo the best split by computing the Gini index whee p( ) is the elative fequen of cases belong to class c j. Split the node into banches. 4. Check if banches have data. 5. Repeat. 6. Stop when all banches have no data. 7. Assign classes to teminal nodes. Output: tee of classifying data and qualification paametes. Classification Using KNN Method The k-neaest neighbou method is the most well known classification algoithm because of its simplici. Also, it needs only two paametes to tune which ae distance metic [] obseved that the k-neaest neighbou is bette than the Bayseain algoithms.the k-neaest neighbou is called lazy because it does not build a model until the time that a pediction is equied. It only does wok at the last second. Also, it is a competitive leaning algoithm because it makes a compaison between data instances to make a pedictive decision. The k-neaest neighbou algoithm pedicts the unseen data instance by seaching though the taining dataset fo the k-most simila instances. The pediction attibute of the most simila instances is summaized and etuned as the pediction fo the unseen instance. Fom taining instance to sample instance distance is evaluated and the instance with lowest distance is called neaest neighbou. KNN method is used in many applications such as classification, poblem solving, and function leaning [5]. This method uses the Euclidean distance fo the eal valued data. The steps of the algoithm can be summaized as follows: Input: dataset (a set of featue vectos epesenting instances). Specify a positive intege k.. Split the dataset into taining dataset D and test dataset D z : the taining dataset to make classifications and the test dataset to evaluate the of the algoithm.. Calculate the distance d(x,x) between the test instance z and evey instance in the taining dataset (x,y)є D. 4. Select D z D, the set of k closest taining instances to test the instance z. 5. Find the most common classification of these instances (the majoi class of the k neaest neighbous) () Whee v is a class label, y i is the class label fo the i th neaest neighbous, and i is an indicato function that 49

3 etuns the value of if its agument is tue and othewise. 6. Give this classification to the test instance (the test instance is classified based on the majoi class of its neaest neighbous). 7. Calculate the of the algoithm. 8. Collect the most simila all togethe. Output: qualification paametes.. Classification Using PCA Method Statistical pocedue o leaning uses an othogonal tansfomation to convet a set of obsevations of possibly coelated vaiables into a set of values of linealy uncoelated vaiables called pincipal components. The objective of PCA is to educe the numbe of attibutes (educe the dimensionali) [4]. The steps of the algoithm can be summaized as follows: Input: dataset (a set of featue vectos epesenting instances). Compute the means of each attibute vecto of all the data set by using the equation (): () whee is the mean of the dataset X and d is the numbe of instances.. Subtact the mean fom each of the data dimensions.. Compute the covaiance matix of the whole datasetx by using equation (4) Whee cov i,j is the covaiance between attibutes i and j. 4. Compute the eigenvectos fo each attibute E = (e,e,-----,e m ) of the covaiance matix X is d m. 5. Compute the coesponding eigenvalues Λ = diag (λ,λ,---,λ d ) whee Xe=λe whee λ is a scala eigenvalue. 6. Sot the eigenvectos by deceasing the eigenvalues. 7. Get the eigenvectos with the lagest eigenvalues to fom the educed matix of dimension k. 8. Multiply the oiginal matix with the educed one to fom the matix W with dimension d k (tansfoms d m matix into d k matix). 9. Use the matix W to tansfom the onto the new space. Output: scatteed figues and qualification paametes. III. IMPLEMENTATION WORK To evaluate the pefomance of the adopted classification methods, fou diffeent datasets ae used. The diffeent datasets cove small dataset with lage attibutes, small data set with limited attibutes, lage dataset with limited attibutes, and lage dataset with lage attibutes. The datasets ae Olitos, Glass, Diabetes, and Madelon datasets. Table () shows these datasets. Olitos dataset consists of olive oil instances on measuements on 5 chemical compositions (fat acids, steols, titepenic alcohols) of olive oils fom Tuscany. Thee ae 4 classes coesponding to diffeent (4) Intenational Jounal of Engineeing and Technical Reseach (IJETR) ISSN: -869 (O) (P), Volume-5, Issue, June 6 poduction aeas. Class, Class, Class, and Class 4 contain 5, 5, 4, and obsevations, espectively. The Glass dataset consists of 4 glass of each 9 attibutes which ae: RI: efactive index, Na: Sodium, Mg: Magnesium, Al: Aluminum, Si: Silicon, K: Potassium, Ca: Calcium, Ba: Baium, and Fe: Ion. Thee ae 7 classes coesponding to diffeent pes of glass which ae :building_windows_float_pocessed, building_windows_non_float_pocessed,vehicle_windows_f loat_pocessed,vehicle_windows_non_float_pocessed, containes, tablewae, headlamps. The Diabetes dataset consists of 768 instances of each 8 attibutes which ae:. Numbe of times pegnant. Plasma glucose concentation a hous in an oal glucose toleance test. Diastolic blood pessue (mm Hg) 4. Ticeps skin fold thickness (mm) 5. -Hou seum insulin (mu U/ml) 6. Body mass index (weight in kg/(height in m)^) 7. Diabetes pedigee function 8. Age (yeas) Two class vaiables ( o ). The Madelon is an atificial dataset which consists of 44 instances of each 5 attibutes The Olitos dataset is divided andomly into taining set (9 instances) and test set ( instances). The glass dataset is divided andomly into taining set with 44 instances and test set which 7 instances. The taining and testing sets fo the Diabetes ae 568 instances and instances and fo the Madelon ae 6 instances and 8 espectively. Table : Datasets Name Instances Attibutes Classes Olitos 5 4 Glass Diabetes Madelon 44 5 Coss validation is a popula stgy fo method selection. The main idea of coss validation is to split data once o seveal times, fo estimating the isk of each method: pat of data (taining sample) is used fo taining each method, and the emaining pat (the test sample) is used fo estimating the isk of method. Then, the coss validation selects the method with the estimated isk [Sylvain Alot and Alian Celisse, ]. Softwae is the Classification toolbox fo MATLAB - vesion 4. has been eleased by Milano Chemometics and QSAR eseach Goup. Visit thei website at Hadwae is Intel (R) Pentium 4, CPU. GHZ, and RAM.49 GB. The is evaluated fo the taining set and test set.the hold out method is used to detemine the stopping point. The hold out method used is coss validation. Random subsampling coss validation is applied to split the dataset andomly into taining set and test set, then can calculate the with the test. The qualification of the classification methods ae based on the following paametes: non- (NER) epesents the aveage of the class, (ER), (Ac) is the atio of coectly assigned, pecision (P) is the atio between the of g th class coectly classified and the total numbe of assigned to that class, 5

4 Classifying Datasets Using Some Diffeent Classification Methods (Sn) descibes the abili of the algoithm to ecognize coectly, and (Sp) chaacteizes the abili of the class to eject the of all othe classes. These paametes ae defined as in the following equations: (5) CART KNN PCA Table 6: Evaluation of the Classifies on Diabetes Dataset (6) (7) (8) (9) fo k g () () Whee : is the total numbe of assigned to the g th class n gg is the numbe of belonging to class g and coectly assigned to it. n g is the total numbe of belonging to the g th class. is the total numbe of assigned to the k th class. CART KNN PCA Table 7: Evaluation of the Classifies on Diabetes Test Dataset CART KNN PCA Classifie Table 8: Evaluation of the Classifies on Madelon Dataset CART KNN PCA IV. DISCUSSION OF RESULTS The quali of the classification methods ae evaluated by pecision,,,, non--, and. Each element of the dataset is called an instance and the class it belongs to is called the label and the of the dataset is the pobabili of the to incoectly classify an instance. The chosen datasets ae splitted into taining set and testing set and the paametes that evaluate the quali of the chosen methods ae shown in tables ( to 9) espectively. Table : Evaluation of the s on Olitos Dataset classifie Non eo c y Pecisio n t y t y CART KNN PCA Table : Evaluation of the Classifies on Olitos Test Dataset CART KNN PCA Table 4: Evaluation of the Classifies on Glass Dataset CART KNN PCA Table 5: Evaluation of the Classifies on Glass Test Dataset Classifie Table 9: Evaluation of the Classifies on Madelon Test Dataset CART 5 5 KNN PCA Fom the above tables, it is clea that when inceases, pecision,, and decease and vice vesa fo all classification methods using diffeent datasets. Fom the test set, the classification algoithms ae estimated to be applicable o not. If the, pecision,, and of the algoithm ae acceptable, the algoithm can be used to classify new data. Sensitivi and ae impotant statistical measues of the classification pefomance. Sensitivi measues the popotion of actual positives which ae coectly identified as such. Specifici measues the popotion of negatives which ae coectly identified. As shown fom the tables fo the same numbe of pinciple components, the inceases by inceasing the value. Although the - elationship is globally non-linea, it seems to be patially linea fo some ange values of. Moeove, the - elationship changes by changing the numbe of pincipal components. The pecentage changes by changing the numbe of pincipal components. Such changes may be in an inceasing ode, othes in a deceasing ode while othes ae altenating. If the dataset has lage numbe of attibutes, it is bette to apply the PCA method. It is noticed that the KNN method is not sensitive fo dataset has lage attibutes also takes moe time than the othe two methods. The PCA method is bette than the othe methods fo big data but CART method is moe sensitive. The uning time of the thee 5

5 methods by applying the madelon dataset is shown in table (). Intenational Jounal of Engineeing and Technical Reseach (IJETR) ISSN: -869 (O) (P), Volume-5, Issue, June Table : The Complexi Time of the Thee Methods Using Madelon Dataset Method Time in Seconds CART 5.4 KNN PCA Because of the of the taining set is lowe than the tue the dataset is patitioned in seveal diffeent ways. The aveage scoe ove the diffeent patition is computed to avoid the possible bias intoduced by elying on any paticula division into test and tain sets. The best method is estimated by how the method pefoms by the unknown data and the coss validation is used to measue the using Diabetes dataset and applying PCA and KNN methods as shown in figues (,, ). The PCA aimed to finding the pinciple components with maximum dependence on the esponse vaiables. When the task is egession o classification, it is pefeed to poject the explanatoy vaiables along diections that ae elated to the esponse vaiable. The two-dimensional pojection esults fo the adopted dataset using the chosen method ae shown in figue () Fig. : Coss Validation Eo Rate on Diabetes Dataset fo the Diffeent Segmentations,, 4, 5,, and all data as Computed by KNN Method

6 scoes on canonical vaiable scoes on canonical vaiable scoes on canonical vaiable scoes on canonical vaiable scoes on canonical vaiable scoes on canonical vaiable Classifying Datasets Using Some Diffeent Classification Methods 5 class class class Fig. : Coss Validation Eo Rate on Diabetes Dataset fo the Diffeent Segmentations,, 4, 5,, and all data as Computed by PCA Method class Fig. : The Pojection of Diabetes Dataset fo the Diffeent Segmentations,, 4, 5,, and all Data Coesponding to Attibute by PCA Method class class The choice of k affects the pefomance of KNN method as shown in Figue () and the coss validation is used to choose the optimal value of k. Figue () shows that at each attibute using coss validation with diffeent segmentation. V. CONCLUSION In this wok diffeent classification methods ae discussed and demonstd by applying diffeent datasets. By analyzing the expeimental esults it is obseved that the PCA method has bette esults than othe two algoithms. Also, it is found that the PCA a useful appoach when dealing with lage amount of data. Fo dataset has lage numbe of attibutes it is pefeed to use the PCA method. The CART and KNN algoithms have poo pefomance fo datasets have lage numbe of attibutes. At last, we can say that no one algoithm is the best fo all pes of dataset. The ovefitting is that the method doesn t fit the test as it fits the taining. The coss validation is a way used to pedict the fit of the method. So, coss validation is used to estimate the expected level of fit of a method independent of the taining set. The optimal value of k in the KNN algoithm is obtained by means of coss validation pocedues. Fo all the classification methods the time to classify the instance is elated to the numbe of instances and the numbe of attibutes

7 REFERENCES [] A. K. Shama and S. Sahni, A Compaative Study of Classification Algoithms fo Spam Data Analysis, Intenational Jounal on Compute Science and Engineeing (IJCSE), Vol., May, pp [] T. C. Shama and M. Jain, WEKA Appoach fo Compaative Study of Classification Algoithm, The Intenational Jounal of Advanced Reseach in Compute and Communication Engineeing, Vol., Issue 4, Apil, pp [] S. Saha and R. Chaki, Application of Data Mining in Potein Sequence Classification, The Intenational Jounal of Database Management Systems (IJDMS), Vol. 4, No. 5, Octobe, pp. -8. [4] A. Kuwadeka and J. Neville, Rational Active Leaning fo Joint Collective Classification Models, The Poceedings of the 8 th Intenational Confeence on Machine Leaning, Bellevue, USA,, pp. -8. [5] V. Kode and C. N. Mahende, Text Classification and Classifies: A Suvey, The Intenational Jounal of Atificial Intelligence and Applications (IJAIA), Vol., No., Mach, pp [6] Sonam Nawal and Kamaldeep Mintwal, Compaison the Vaious Clusteing and Classification Algoithms of WEKA Tools, The Intenational Jounal of Advanced Reseach in Compute Science and Softwae Engineeing, Vol. Issue, Decembe, pp [7] S. Neelamegam and E. Ramaaj, Classification Algoithm in Data Mining: An Oveview, The Intenational Jounal of pp Netwok Tends and Technology (IJPTT), Vol. 4, Issue 8, Septembe, pp [8] Nikhil N. Solvithal and R. B. Kulkani, Evaluating Pefomance of Data Mining Classification Algoithm in Weka, Intenational Jounal of Application o Inovation in Engineeing and Management (IJAIEM), Vol., Issue, Octobe, pp. 78. [9] M. Tiwai and M. Misha, Accua Estimation of Classification Algoithms with DEMP Model, The Intenational Jounal of Advanced Reseach in Compute and Communication Engineeing, Vol., Issue, Novembe, pp [] S. Vijayaani and M. Muthulakshmi, Compaative Study on Classification Meta Algoithms, The Intenational Jounal of Innovative Reseach in compute and Communication Engineeing, Vol., Issue 8, Octobe, pp [] RohilAoa and Suman, Compaative Analysis of Classification algoithms on Diffeent datasets Using WEKA, The Intenational and jounal of Compute Application, Vol. 54, No., Septembe, pp. 5. [] S. Vijayaani and M. Muthulakshmi, Compaative analysis of Bayes and lazy Classification Algoithms, The Intenational Jounal of Advanced Reseach in Compute and Communication Engineeing, Vol., Issue 8, August, pp [] X. Wu, V. Kuma, J. R. Quinlan, J. Ghosh, Q. Yang, H. Motoda, G. J. McLachlan, A. Ng, B. Liu, Philip S. Yu, Zhi-Hua Zhou, M. Steinbach, D. J. Hand, and D. Steinbeg, Top Algoithms in Data Mining, KnowlInfSyst, Vol. 4, 8, pp. 7. [4] J. Gou, L. Du, Y. Zhang, and T. Xiong, A New Distance-Weighted K-Neaest Neighbo Classifie, The Jounal of Infomation &Computational Science, Vol. 9, June, pp [5] T. Neo and D. Ventua, A Diect Boosting Algoithm fo the K-Neaest Neighbo Via Local Waping of the Distance Metic, Patten ecognition lettes, Vol.,, pp. 9. [6] F. Fenandez and P. Isasi, Local Featue Weighting in Neaest Potope Classification, The IEEE Tansactions on Neual Netwoks, Vol. 9, No., Januay 8, pp.4-5. [7] O. Sutton, Intoduction of K Neaest Neighbou Classification and Condensed Neaest Neighbou data Reduction, Febuay, pp.-. [8] L. Godon, Using Classification and Regession Tees (CART) is SAS Entepise Mine fo Applications in Public Health, Data Mining and Text Analysis, SAS Global Foum,, pp. -8. [9] W. Y. Loh, Classification and Regession Tees, WIRES data Mining and Knowledge Discovey, John Wily&Sons, Vol., Januay/Febuay,, pp.4. [] T. N. Phyu, Suvey of Classification Techniques in Data Mining, Poceeding of the Intenational Multi-Confeence of Enginees and Compute Scientists, IMECS 9, Mach 8, 9-Hong Kong, pp. -5. [] S. Alot and A. Celisse, A Suvey of Coss Validation Pocedues fo Model Selection, Statistics Suvey, Vol. 4, ISSN: ,, pp Intenational Jounal of Engineeing and Technical Reseach (IJETR) ISSN: -869 (O) (P), Volume-5, Issue, June 6 [] Wikipedia the fee Enclopedia Downloaded 5. [] H. S. Yazdi and N. Salehi, Multi Banch Decision Tee: A New splitting Citeion, Intenational Jounal of Advanced Science and Technology, Vol.45, August 4, pp [4] Downloaded fom website 6. [5] M. AkhilJabba, B. L. Deekshatulu, and Piti Chanda, Classification of Heat Disease Using K-Neaest Neighbo and Genetic Algoithm, The Intenational Confeence on Computational Intelligence:Modling Techniques and Applications, Elsevie,, Vol., pp