A Comparison of the Discretization Approach for CST and Discretization Approach for VDM

Transcription

1 Interntionl Journl of Innovtive Reserch in Advnced Engineering (IJIRAE) Volume1 Issue1 (Mrch 2014) A Comprison of the Discretiztion Approch for CST nd Discretiztion Approch for VDM Omr A. A. Shib Fculty of Science, Sebh University- Liby E-mil: bumod99@gmil.com Abstrct- The preprocessing dt is the most importnt tsk in dt mining steps. Discretiztion process is known to be one of the most importnt dt preprocessing tsks in dt mining. Presently, mny discretiztion methods re vilble. Discretiztion methods re used to reduce the number of vlues for given continuous ttributes by dividing the rnge of the ttribute into intervls. Discretiztion mkes lerning more ccurte nd fster. This pper will investigte the effects of discretiztion of continuous ttributes in dtset to the performnce of Cse Slicing Technique s clssifiction method nd compred it with nother clssifiction pproch which will be generic version of the VDM lgorithm the discretized vlue difference metric (DVDM). Keywords- dt mining, discretiztion, clssifiction, slicing, preprocessing. I. INTRODUCTION Dt mining clssifiction lgorithms developed to produce solution to improving the efficiency of systems includes lot of dt. Clssifiction is widely used technique in vrious fields, including dt mining whose gol is clssify lrge dtset into predefined clsses. In dtset the ttributes my be continuous, ctegoricl nd binry. However, most ttributes in rel world re in continuous form [1]. This pper will investigte the effects of discretiztion of continuous ttributes in dtset to the performnce of two clssifiction methods, generic version of the VDM lgorithm the discretized vlue difference metric (DVDM) nd the improved discretiztion pproch of Cse Slicing Technique (CST) [2]. The im is to show tht discretiztion will improve the ccurcy of CST clssifier more thn the VDM clssifier. The dtsets to be used re the Iris Plnts Dtset (IRIS), Credit Crd Applictions (CRX), Heptitis Domin (HEPA) nd Clevelnd Hert Disese (CLEV) from the UCI Mchine Lerning Repository [3]. All dtsets hve been discretized nd feed to clssifiction lgorithms. The rest of this pper is orgnized s follows: the next section presents some preprocessing dt tsks. Section 3 presents experimentl results tht compre the performnce of the lgorithms. Finlly the conclusion is presented in section 4. II. DATA PREPROCESSING In this section some preprocessing dt tsks such s discretiztion, weighting ttributes, similrity computtion etc. tht needed by ttribute selection pproch will be discussed. A. Identify ttributes Types Attributes cn be continuous or discrete. A continuous (or continuously-vlued) ttribute uses rel vlues. A discrete ttribute is one tht cn hve only fixed set of vlues, nd cn be either symbolic or liner. A liner discrete ttribute cn hve only discrete set of liner vlues. It cn be rgued tht ny vlue stored in computer is discrete t some levels. The reson continuous ttributes re treted differently is tht they cn hve so mny different vlues tht ech vlue my pper only rrely (perhps only once). This cuses problems for lgorithms such s Vlue Difference Metric (VDM) nd Cse Slicing Technique (CST) discussed in this Pper. These lgorithms depend on testing two vlues for equlity but these vlues will rrely be equl, though they my be quite close to ech other. A symbolic (or nominl) ttribute is discrete ttribute whose vlues re not in ny liner order. So, the continuous ttributes hve to be trnsformed into discrete vlues. This process is clled discretiztion, which will be discussed in subsection C. B. Weighting ttributes Most of the clssifiction lgorithms developed until now somehow try to mesure the reltive importnce of n ttribute in clssifiction compred to the others, nd use this knowledge while clssifying objects. IJIRAE , IJIRAE All Rights Reserved Pge - 1

2 Interntionl Journl of Innovtive Reserch in Advnced Engineering (IJIRAE) Volume1 Issue1 (Mrch 2014) Some of them try to determine the relevnt nd irrelevnt ttributes in set of ttributes in order to tke relevnt ttributes into ccount more thn the irrelevnt ones, nd some others ttempt to ssign weights ccording to the degree of relevnce on the clssifiction of instnces. In this stge of the Cse Slicing Technique, the bsic version of conditionl probbilities hs been used to mesure the importnce of ech ttribute in the clssifiction. The weight of ech ttribute hs been clculted to clssify the new cse by using this sttisticl pproch shown in Eq.1 nd Eq. 2. High weight vlues re ssigned to ttributes tht re highly correlted with the given clss. The conditionl probbilities hve been used becuse it gives rel weight for ech ttribute bsed on its repetition in the dtset nd bsed on the clss ttribute for ech cse. w i ) P( C i ) (1) ( P C i instnces contining i clss C (2) instnces contining i where the weight for ttribute of clss c is the conditionl probbility tht cse is member of c given the vlue of. Fig.1 gives the pseudo code for doing this. Let c be the output clss of cse i N,v,c = N,v,c + 1 {# of vlue v of ttribute with output clss c} N,v = N,v + 1 {# of vlue v of ttribute } For ech vlue v (of ttribute ) do For ech clss c do If N,v = 0 P,v,c = 0 Else p,v,c = N,v,c/N,v Endfor Endfor C. Discretiztion Fig. 1: Pseudo code for ssign weights to ttributes {using conditionl probbility} P(C i ) In dt mining, discretiztion process is known to be one of the most importnt dt preprocessing tsks. Most of the existing mchine lerning lgorithms is cpble of extrcting knowledge from dtbses tht store discrete ttributes. If the ttribute re continuous, the lgorithms cn be integrted with discretiztion lgorithms which trnsform them into discrete ttribute. Discretiztion methods re used to reduce the number of vlues for given continuous ttributes by dividing the rnge of the ttribute into intervls [4],[5]. Discretiztion mkes lerning more ccurte nd fster. Discretiztion s used in this pper nd in the mchine lerning literture in generl is process of trnsforming continuous ttribute vlue into finite number of intervls nd ssociting ech intervl with discrete, numericl vlue. The usul pproch for lerning tsks tht use mixed-mode (continuous nd discrete) dt is to perform discretiztion prior to the lerning process [6],[7],[8]. The discretiztion process first finds the number of discrete intervls, nd then the width, or the boundries for the intervls, given the rnge of vlues of continuous ttribute. More often thn not, the user must specify the number of intervls, or provide some heuristic rules to be used [9]. A vriety of discretiztion methods hve been developed in recent yers. Some models tht hve used the Vlue Difference Metrics (VDM) or vrints of it [10],[11],[12] hve discretized continuous ttributes into somewht rbitrry number of discrete rnges, nd then treted these vlues s nominl (discrete unordered) vlues. The discretiztion method proposed by Rndll nd Tony [13] nd hs been extended by Pyne nd Edwrds [14] s shown in Eq.3. IJIRAE , IJIRAE All Rights Reserved Pge - 2

3 Interntionl Journl of Innovtive Reserch in Advnced Engineering (IJIRAE) Volume1 Issue1 (Mrch 2014) x, if is discrete s, if x mx, else v disc ( x ) 1, if x min, else (3) ( x min ) / w 1 where w mx min (4) s where mx nd min re the mximum nd minimum vlues, respectively which occur in the trining set for ttribute nd s is n integer supplied by the user. Unfortuntely, there is currently little guidnce s to wht vlue should be ssigned to s. Current reserch is exmining more sophisticted techniques for determining good vlues of s, such s cross-vlidtion, or other sttisticl methods [13]. When using the slicing pproch, continuous vlues re discretized into s equl-width intervls (though the continuous vlues re lso retined for lter use), where s is n integer supplied by the user. The width w of discretized intervl for ttribute is given by Eq. 4. In this Pper, we propose new lterntive, which is to use discretiztion in order to collect sttistics nd determine good vlues of P(C i ) for continuous vlues occurring in the trining set cses, but then retin the continuous vlues for lter use. A generic version of the VDM lgorithm, clled the discretized vlue difference metric (DVDM) will be used for comprisons with extensions proposed in this pper. Thus, differences between DVDM nd the new function cn be pplied to the originl VDM lgorithm or other extensions. The discretized vlue v of continuous vlue x for ttribute is n integer from 1 to s, nd is given by Eq. 5. Fig.2 shows the pseudo code for the discretiztion of continuous vlues in the proposed pproch. ( x min ) if ttribute is continuous v disc ( x ) w (5) x if ttribute is discrete Let x be the input vlue for ttribute of cse i v disc (x) {which is just x if is discrete} w bs mx min / s {The width of discretized intervl for ttribute } {where mx nd min re the mximum nd minimum vlue, respectively, which occur in the trining set for ttribute }, {The discretized vlue v of continuous vlue x for ttribute is n integer from 1 to s nd s is determine by the user.} If is continuous then Else Endif x min w v disc ( x) / v disc ( x) x Fig. 2: Pseudo code for discretize continuous vlues III. EXPERIMENTAL RESULTS IJIRAE , IJIRAE All Rights Reserved Pge - 3

4 Interntionl Journl of Innovtive Reserch in Advnced Engineering (IJIRAE) Volume1 Issue1 (Mrch 2014) In this section the results of severl prcticl experiments re presented to compre the proposed pproch with DVDM pproch. In the experiments, we pplied the discretiztion lgorithm s the preprocessing step of CST clssifier on some selected rel world problems. A. Empiricl Results In this section the comprison of the discretiztion pproch of the DVDM nd the improved discretiztion pproch of Cse Slicing Technique is presented on some selected dtsets. In this comprison four dtsets hs been selected tht re Iris Plnts Dtset (IRIS), Credit Crd Applictions (CRX), Heptitis Domin (HEPA) nd Clevelnd Hert Disese (CLEV). All dtsets hve been discretized nd feed to clssifiction lgorithms, the results of the comprison re shown in Tble 1. Tble 1: Clssifiction ccurcy of CST ginst DVDM bsed on discretiztion pproch DVDM Methods CST Dtsets IRIS % % CRX % % HEPA % % CLEV % % In Tble 1 the comprison between the proposed discretiztion pproch for Cse Slicing Technique nd discretiztion pproch for VDM hs been done on four types of dtsets. The discretiztion result of CST nd VDM hve been forwrded into CST clssifier for clssifiction to mke sure tht the result obtined from ech technique is bsed on discretiztion pproch used during clssifiction. List of the result of clssifiction ccurcy shown in Tble 1. CST gve better clssifiction ccurcy compred with DVDM clssifiction ccurcy. Fig.3 shows the difference in clssifiction ccurcy of CST ginst DVDM bsed on discretiztion pproches. Fig. 3 : Difference in clssifiction ccurcy of CST ginst DVDM VI. CONCLUSION This pper hs presented comprison between the proposed discretiztion pproch for Cse Slicing Technique (CST) nd discretiztion pproch for VDM. The discretiztion result of CST nd VDM hve been forwrded into CST clssifier for clssifiction to mke sure tht the result obtined from ech technique is bsed on discretiztion pproch IJIRAE , IJIRAE All Rights Reserved Pge - 4

5 Interntionl Journl of Innovtive Reserch in Advnced Engineering (IJIRAE) Volume1 Issue1 (Mrch 2014) used during clssifiction. The experiments show tht using the proposed discretiztion pproch for (CST) indeed improves the ccurcy of clssifiction. It gve very high percentge of clssifiction ccurcy. REFERENCES [1]. Mehmet Hcibeyoglu, Ahmet Arsln, Sirzt Khrmnli. Improving Clssifiction Accurcy with Discretiztion on Dtsets Including Continuous Vlued Fetures, World Acdemy of Science, Engineering nd Technology, 2011, 54. [2]. Shib, O., Sulimn, M.N., Mmt, A. & Ahmd, F. An Efficient nd Effective Cse Clssifiction Method Bsed On Slicing, Interntionl Journl of The Computer, the Internet nd Mngement Vol. 14.No.2 (My - August, 2006) pp15-23 [3]. Murphy, P.M. UCI Repositories of Mchine Lerning nd Domin Theories,1997,URL: / (Accessed on 10 Jn. 2012). [4]. Kurgn. L nd Cios, K.J.: Discretiztion Algorithm tht Uses Clss-Attribute Interdependence Mximiztion, Proceedings of the Interntionl Conference on Artificil Intelligence (IC-AI 2001),pp , Ls Vegs, Nevd. [5]. Rtnmhtn, C. A. CloNI: Clustering of N-Intervl Discretiztion, Proceedings of the 4 th Interntionl Conference on Dt Mining Including Building Appliction for CRM & Competitive Intelligence, Rio de Jneiro, Brzil [6]. Ctlett, J. On Chnging Continuous Attributes into Ordered Discrete Attributes. Proceedings of the Europen Working Session on Lerning,1991 (pp ), Berlin: Springer-Verlg. [7]. Dougherty, J., Kohvi, R. & Shmi, M. Supervised nd Unsupervised Discretiztion of Continuous Fetures. Proc. of the 12 th Interntionl Conference on Mchine Lerning, 1995, (pp ), Sn Frncisco, CA: Morgn Kufmnn. [8]. Liu, H., Hussin, F., Tn, C. & Dsh, M. Discretiztion: An Enbling Technique. Journl of Dt Mining nd Knowledge Discovery, 6 (4), [9]. Ching, J., Wong, A. & Chn, K. Clss-Dependent Discretiztion for Inductive Lerning from Continuous nd Mixed Mode Dt. IEEE Trnsctions on Pttern Anlysis nd Mchine Intelligence, 17 (7), [10]. Cost, S. & Slzberg, S. A Weighted Nerest Neighbor Algorithm for Lerning With Symbolic Fetures. Mchine Lerning, 10, [11]. Rchlin, J., Simon, K., Slzberg, S. & Dvid, W. Towrds Better Understnding of Memory-Bsed nd Byesin Clssifiers. In Proceedings of the 11 th Interntionl Mchine Lerning Conference, 1994, (pp ), New Brunswick, NJ: Morgn Kufmnn. [12]. Mohri, T. & Tnk, H. An Optiml Weighting Criterion of Cse Indexing for Both Numeric nd Symbolic Attributes. Workshop, (Technicl Report ws-94-01), 01, 1994, (pp ), Menlo Prk, CA: AIII press. [13]. Rndll, D.W. & Tony, R.M. Vlue Difference Metrics for Continuously Vlued Attributes. In Proceedings of the Interntionl Conference on Artificil Intelligence, Expert Systems nd Neurl Networks, 1996 (pp ). [14]. Pyne, T.R. & Edwrds, P. Implicit Feture Selection With the Vlue Difference Metric. ECAI. 13 th Europen Conference on Artificil Intelligence Edited by Henri Prde, 1998, (pp ), Brighton, UK: Published by John Wiley & Sons, Ltd. IJIRAE , IJIRAE All Rights Reserved Pge - 5