Efficiency Comparison of Data Mining Techniques for Missing-Value Imputation

Transcription

1 Journal of Industral and Intellgent Informaton Vol. 3, No. 4, December 2015 Effcency Comparson of Mnng Technques for Mssng-Value Imputaton Jarumon Nookhong and Nutthapat Kaewrattanapat Suan Sunandha Rajabhat Unversty, Bangkok, 10300, Thaland Emal: {jarumon.no, Abstract Ths research proposes to compare the effcency data mnng technques for mssng-value mputaton by Naïve Bayesan, KNN, Lnear Regresson, Decson Tree and Rule Based Classfer (PART).There s adjustng parameters dfferent set. The data was collected by data set of Mushroom Classfcaton (Dscrete data), Glass Type Classfcaton (Contnuous ) and the Balance Scale data (Ordnal ) from UCI Machne Learnng Repostory. The data was analyzed and compared the effcency for each technque by comparng ther performance n mnmzng the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). The result s found that the complete dscrete data was well mputed by Decson Tree, but ths technque needs enough rules to mnmzng an error. For contnuous data, t was well mputed by K-Nearest Neghbor. The last Naïve Bayes was good for the dscrete data and hdden ordnal scale data. II. OBJECTIVES To study sutablty n utlzng of data mnng n mputng mss data havng dfferent format and brngng nput parameter of mnng data analyss, such as Naïve Bayes, KNN, Lnear Regresson, Decson Tree and Rule Based Classfer. III. DELIMITATION OF RESEARCH utlzed n the research was data set brought from UC Irvne Machne Learnng Repostory: UCI. The 3 data sets were used, namely mushroom data set (posonous and non-posonous), glass dentfcaton data set and balance scale data set, all of 3 data sets have dfferent data specfcaton and record number. Ths research determned value of mssng attrbute by choosng wth randomzng technque n each data set for only 1 attrbute used n mputng mssng value and testng errors. Type of mssng data would be n Mssng Completely Random: MCAR. Index Terms mssng value, mputaton, data mnng, errorsd. I. INTRODUCTION In obtanng and data storage, they are sgnfcantly mportant to an analyze or a quanttatve research to obtan accurate research results to be utlzed; however, data storage may have bug or some data s mssng. [1], [2] For solvng such problems, t can be solved by dsposng data record that has problem; however, a lackng of data for utlzng n data analyss may occur. Therefore, n order to be able to brng the data to be utlzed, t s necessary to update data utlzng the man technque, that s estmaton of approxmated smlar data or mputaton. [3] As for the estmaton to brng the data for mputaton n mssng data has several technques. Ths research choose 5 data mnng technques, namely 1) Naïve Bayes 2) K-Nearest Neghbor: KNN) 3) Lnear Regresson 4) Decson Tree and 5) Rule Based Classfer to compare effcency n mputaton of mssng data. In ths research, an adjustment of parameter n each technque of data mnng to obtan dfferent models was conducted and utlzng of Root Mean Square Error: RMSE and Mean Absolute Error: MAE to show the comparson of effcacy of mssng data mputaton. IV. RELATED LITERATURE A. Naï ve Bayes Naïve Bayes' Technque forecasts wth prncple of classfcaton by applyng Bayes Theorem whch s a supervsed learnng; the exercse must have answer keys meanng that type or class of data that wants to create learnng to buld concepts of such class [4] s sutable wth the cases of large number of example sets and attrbute of ndependent example sets [5] whch probablty of data sets to be C class for data havng n attrbute X = {A1, A2,...An} or P(C A1,...,An} from Bayes' Rule; the results wll be as the equaton (1) (1) By fndng P(C A1,...,An) from equaton for ever group I, attrbute at j; the obtaned value wll be brought to compare. The group havng hghest probablty s an answer. Manuscrpt receved December 9, 2014; revsed February 10, do: /j

2 Journal of Industral and Intellgent Informaton Vol. 3, No. 4, December 2015 B. K-Nearest Neghbor: KNN K-Nearest Neghbor: KNN or nearest range cluster s unsophstcated technque, appled to good rregular data n scatterng formaton. A prncple of KNN s to extend boundary for fndng members by measurng dstance, appled n several felds of work, namely Impute Mssng Values [6] and etc. as KNN wll examne certan factors, such as a number of nearby members, dstance measurement and Normalzaton. The K parameter s a number of nterested neghbors whch, normally, would be self-determned. dstance measurement could be conducted by several technques, namely Eucldan Dstance, Manhattan Dstance, Mahalanobs Dstance and etc.; these technques are correlaton neghbors that should apply normalzaton data. As for Eucldan Dstance, t s not necessary to conduct normalzaton data. C. Lnear Regresson Lnear Regresson s a study regardng how ndependent varables affects dependent varables or ndependent varables affects Y value vared n what model. The relatonshp could be explaned by Regresson Model [7] as equaton (2) As Y = Dependent varables X = Independent varables a = Constant b = Slope D. Decson Tree Y a bx (2) Decson Tree s a data mnng technque whch s a learnng model by categorzng data n the sample group nto subgroups, usng attrbutes of data as categorzng tools. The decson tree obtaned from learnng demonstrates attrbutes of data that s a determner of answers and demonstrates how mportant or dfferent of the attrbutes, helpng users to analyze data and decde more accurately. A process starts wth selectng attrbutes to be mode, whch are the attrbutes when dvdng samples nto subgroups makng most members n each subgroups have answers n the most same answers. Measurement of an ablty n categorzng of gan n each mode s able to conduct by relyng on Informaton Theory that brng entropy to be an ndcator of dsorder n data. E. Rule Based Classfer Rule Based Classfer s a technque for categorzng data record by usng f...then... whch the rule brought to create model wll be present n connected model, such as R = (r1 V r2... V rk) as R s groups of rule and r s a rule usng n dvdng or each rule [4] F. Root Mean Square Error: RMSE Root Mean Square Error or Square Root of Mean Standard Devaton s a technque measurng error from value forecast from model wth occurred actual value; f RMSE has low value, t means that the model can forecast value nearly actual value; therefore, f ths value equals zero, t means that there s no error n ths model. RMSE could be calculated as equaton (3). As: Y forecastng RMSE = T t1 ( Y Y ) n 1 2 (3) = Approxmaton from data value model from Yˆ = Actual value of actual data obtaned from calculaton n = Number of sample sze usng n model estmaton G. Mean Absolute Error: MAE Mean Absolute Error: MAE s a technque to measure dfference value between actual value and forecast value from model. If MAE has low value, t means that the model can forecast nearly actual value; therefore, f ths value equals zero, t means that there s no error n ths model. MAE could be calculated as equaton (4). MAE = 1 n n 1 f y f = Approxmaton from data value model from forecastng y = Actual value of actual data obtaned from calculaton n = Number of sample sze usng n model estmaton V. RELATED RESEARCH Narong et al [8] have conducted a study and comparson regardng a technque of mssng-value forecastng wth statstcal method, namely mean, correlaton coeffcent analyss, weghted correlaton coeffcent analyss and dscrmnant analyss by measurng effcacy by MMRE. It s found that the dscrmnant analyss can forecast mssng value to be the most nearly actual data sutable for data havng relatonshp wth each other and clear scatterng. Karung and Payunk [6] have conducted a study regardng replacng of mssng value by weghted nearest technque of mcro array by KNNFSW Impute and compared effcacy wth Row Average KNN. The test demonstrated the better effcacy n term of Normalzed Root Mean Squared Error (NRMSE). A. Preprocessng VI. RESEARCH METHODOLOGY Comparson between effcacy of data mnng technque to mpute mssng data n 5 technques would use such data from UC L (UC Irvne Machne Learnng Repostory, http//:archve.cs.uc.edu/ml/ndex.html), total 3 sets of case study, namely mushroom data set conssted of 23 attrbutes, 8124 records conssted of Nomnal Scale, Glass (4) 306

3 Journal of Industral and Intellgent Informaton Vol. 3, No. 4, December 2015 Identfcaton Set conssted of 11 attrbutes and 214 records comprse contnuous data and nomnal scale and Balance Scale Set conssted of 5 attrbutes and 625 records conssted of Ordnal and Class Label, all of data sets are dfferent n property of data and number of records. Before brngng data to analyze, t s necessary to convert data format to be sutable for analyss n each technque ( Preprocessng), namely Lnear Regresson Analyss has to use numercal data only and etc., whch the converson of data format and attrbute selecton of mssng data wll be present n Table I Sets Mssng Attrbute TABLE I: DATA PREPROCESSING % Tranng Set % Test Set Mushroom Odor 80% 20% a,l,c,y,f, m,n, p,s Glass Identfcat on Balance Scale Debug Class Label Rght- Weght 80% 20% 1,2,3, 5,6,7 80% 20% 1,2,3, 4,5 Numerc Modeln g 1,2,3,4,5, 6,7, 8,9 TABLE II: PARAMETER TUNING OF NAÏVE BAYESIAN DsplayModelInOldFormat UseKernel Estmator Nomnal Modelng FIVE FOUR= 4 FIVE=5 UseSupervsed Dscretzaton TABLE III: PARAMETER TUNING OF K-NEAREST NEIGHBOR: KNN KNN DstanceFuncton DstanceWegthng No, K=1, EucldeanDstance, 1/Dstance, K=3 ManhattanDstance 1-Dstance scope of parameter tunng would present n Table II to Table VI. TABLE VI: PARAMETER TUNING OF RULE BASED CLASSIFIER Confdence Factor 0.25, 0.50, 0.90 Bnary Splt ReducedError Prunng C. Process of Mnng Analyss Unprune When the data for preprocessng s prepared and parameter data s fnshed preparng, the analyss s conducted n each technque of data mnng accordng to the process as Fg. 1: Tranng Set Analyzng (Generated Model) Adjust the Parameters No Set Preprocessng 5 Models? Yes Performance Comparson Test Set Mssng Value Imputaton Error Estmaton Fgure. 1. Mssng value of data n 5 technques. TABLE IV: PARAMETER TUNING OF LINEAR REGRESSION AttrbuteSelectonMet hod M5Method, GreedyMethod Debug ElmnateColnearAttrbute TABLE V: TUNING OF DECISION TREE Confdence Factor 0.25, 0.50, 0.90 Bnary Splt B. Parameter Tunng Desgn ReducedError Prunng UseLaplace In ths research, the researchers tuned parameter value by WEKA for tunng because the parameter tunng would affect a change n model used n dfferent mputaton. The Fgure. 2. Error value from mputaton n mssng value by naïve bayesan technque. VII. RESULTS A. The Results n Comparson of Errors When the data whch s ready to analyze s brought to use n each technque n order to obtan forecastng model utlzed n mssng value. Then, the comparson for errors 307

4 Journal of Industral and Intellgent Informaton Vol. 3, No. 4, December 2015 by MAE and RMSE s conducted. The models and error value s presented n Fg. 2 and Table VI. Fgure. 3. Error value from mputaton n mssng value by k-nearest neghbor technque. TABLE VII: THE LOWEST ERRORS IN EACH MODELING Set Model MAE RMSE Decson Tree KNN Mushroom Rule Based Naïve Bayes Lnear Regresson KNN Rule Based Glass Decson Tree Identfcaton Naïve Bayes Lnear Regresson Naïve Bayes Decson Tree Balance Scale Rule Based KNN Lnear Regresson Fgure. 4. Error value from mputaton n mssng value by lnear regresson technque. Fgure. 7. Comparson for errors by mae and rmse n each mnng data n order to mpute mssng value. Fgure. 5. Error value from mputaton n mssng value by decson tree. Fgure. 6. Error value from mputaton n mssng value by rule based classfer. B. Summarze of Error Comparson In the comparson of data mnng effcacy n order to mpute mssng value, error between forecastng data of mputaton and actual data was used. As the results n comparson of errors are presented n Table VII When the data n Table VII was brought to convert nto comparson chart, t demonstrates a clear dfference. The comparson chart s as Fg. 7: VIII. CONCLUSION In the comparson of data mnng effcacy for mputng mssng value, the researchers chose 5 technques avalable n WEKA software, namely Naïve Bayes, KNN, Lnear Regresson, Decson Tree, Rule Base Classfer (PART) to be appled n clearly dfferent data, for example all dscrete data based classfer of mushroom, contnuous data classfer of glass and dscrete and hdden-order data of balance scale. Ths research utlzed errors for comparson, namely MAE and RMSE as they helped to know on dfference between mputed data and actual data. From the research, t s found that when the mushroom data set was mputed wth mssng value by Decson Tree Technque, t s sutable wth Complete Dscrete and clustered n each group; however, ths technque wll gve low errors results when there are a large number of rules n mputaton of mssng value. When glass dentfcaton data set s mputed wth mssng value and conducted errors resultng n KNN to be sutable for numercal data (Contnuous ) and necessary to tune neghborhood value to be low as the data clusters really near. When balance scale data set s s mputed wth mssng value and conducted errors resultng 308

5 Journal of Industral and Intellgent Informaton Vol. 3, No. 4, December 2015 n Naïve Bayesan to be sutable n mputng mssng value as the property of balance scale data comprsed dscrete data and ordnal scale. Moreover, t s found that all 3 data sets are not sutable for lnear regresson technque as forecastng model gvng value as contnuous number whch are not batch data, makng hgh rate n errors. The matter that should be developed further s the case of each record havng mssng value more than 1 value, what sutable technque should be used to solve the problem and whch attrbutes should be mputed before, whch wll gve the lowest errors. ACKNOWLEDGMENT I would lke to thank Insttute for Research and Development Suan Sunandha Rajabhat Unversty s fund n supportng ths research study and sponsorshp. REFERENCES [1] K. Kularbphettong and C. Tongsr, Mnng educatonal data to analyze the student motvaton behavor, World Academy of Scence, Engneerng and Technology, Internatonal Scence Index 68, vol. 6, no. 8, pp , [2] N. Kaewrattanapat and W. Kunnu, The automatc classfcaton of tha news by smlarty method, n Proc. 2th Internatonal Symposum on Busness and Socal Scences: ISBSS, Osaka Japan, [3] N. Kaewrattanapat, Recrutment system based on SOAP and XML web servces, n Proc. Human Resource Natonal Conference, Chulalongkorn Unversty, [4] T. Bunyang, A. Roapchan, and N. Pongam, Comparson of data classfcaton n k nearest neghbor, naïve bayesan, decson tree and rule based classfer, n Proc. 7th Natonal Conference on Computng and Informaton Technology, pp , [5] K. Sron, Forcastn of cause n electrc current falure by utlzng data mnng technque n electrc dstrbuton of regonal electrcty authorty, Area 1, Central Regon. Master of Engneerng (Electrcal Engneerng), Faculty of Engneerng, Kasetsart Unversty, [6] K. Heng-praphrom and P. Mesut, Selecton of Property by Nearest Member Technque for Imputng Mssng Value by Weghted Nearest Technque of Mcro Array n NCCIT, pp , [7] S. Jusu and T. Jansutvechakul, Comparson of book borrowng for nabon school's lbrary by regresson analyss and artfcal neural network, n Proc. 6th NCCIT, 2010, pp [8] N. Pot, S. Prakancharoen, D. Thammasr, P. Ammaruekarat, and V. Nupan, A study and comparson of mssng value estmaton by statstcal method n Proc. 5th Natonal Conference on Computng and Informaton Technology, 2009, pp Jarumon Nookhong s Lecturer of Informaton Management, Faculty of Humantes and Socal Scences at Suan Sunandha Rajabhat Unversty, Bangkok (SSRU), Thaland She got her Bachelor degree of Informaton Scence (Informaton Management) Walalak Unversty. Also, She graduated her Master of Scences (Management Informaton System) Kng Mongkut's Unversty of Technology North Bangkok and Ph.d. canddate n Informaton and Communcaton Technology for Educaton from Kng Mongkut s Unversty of Technology North Bangkok, Thaland. She s research focuses on Mnng, Knowledge Management, Artfcal Neural Networks, Green ICT, Cloud Computng and Green Cloud System. Nutthapat Kaewrattanapat s Lecturer of Informaton Management, Faculty of Humantes and Socal Scences at Suan Sunandha Rajabhat Unversty, Bangkok (SSRU), Thaland Nutthapat Kaewrattanapat was born on 8 March 1983 n Thaland and receved hs B.S. degree wth 1st class honors n Computer Scence and M.S. degree n Management Informaton System and Ph.d. canddate n Informaton Technology (Royal Tha government scholarshp) from Kng Mongkut s Unversty of Technology North Bangkok, Thaland. He s currently a faculty member of Informaton Management Program, Suan Sunandha Rajabhat Unversty snce Hs research focuses on Informaton Technology, Computatonal Lngustc, Natural Language Processng and Mnng. 309