Using Decision Boundary to Analyze Classifiers

 Michael Cunningham
 8 months ago
 Views:
Transcription
1 Using Decision Boundary to Analyze Classifiers Zhiyong Yan Congfu Xu College of Computer Science, Zhejiang University, Hangzhou, China Abstract In this paper we propose to use decision boundary to analyze classifiers. Two algorithms called decision boundary point set (DBPS) and decision boundary neuron set (DBNS) are proposed to obtain the data on the decision boundary. Based on DBNS, a visualization algorithm called SOM based decision boundary visualization (SOMDBV) is proposed to visualize the highdimensional classifiers. The decision boundary can give an insight into classifiers, which cannot be supplied by accuracy. And it can be applied to select proper classifier, to analyze the tradeoff between accuracy and comprehensibility, to discovery the chance of overfitting, to calculate the similarity of models generated by different classifiers. Experimental results demonstrate the usefulness of the method. 1. Introduction Classification is an important problem in machine learning, and it has many applications in real world. There are many classifiers now [1], whose performance is usually estimated by accuracy. Accuracy is the proportion of correct predictions to all predictions [1]. But accuracy is a raw performance score, and it cannot give much insight to classifiers [2]. Accuracy cannot tell which of the data are classified right and which are not, and it is unable to reveal the relative positions of the data predicted correct and incorrect. Data sets in real world are mostly highdimensional. Users usually get an intuitive insight by highdimensional data visualization algorithms, which are unsupervised. The class boundary cannot be clearly visualized by these algorithms [3]. Because there is no powerful tools users cannot understand classifiers very well. [2] proposes to use decision region connectivity to analyze highdimensional classifiers, which can be used to analyze convexity of decision region. The algorithm is independent of the dimension of the data set. [3] proposes an algorithm named SVMV to visualize classification results of Support Vector Machine(SVM) [4] using selforganizing mapping(som) [5]. The algorithm can clearly visualize SVM classification boundary and the distance between data and classification boundary in a 2D map. But the algorithm substitutes weight matrix of SOM for input data of SVM decision function, which limits its application to other classifiers. The method for using decision boundary to analyze classifiers is proposed in this paper. Decision boundary is the distinguishing boundary classifier uses to predict data, so the predicting labels of the data on the two sides of boundary are different. Two algorithms are provided to obtain the data on the classifier s decision boundary. The first algorithm named decision boundary point set (DBPS) is used to get the points near the decision boundary of classifiers. The second algorithm named decision boundary neuron set (DBNS) is used to get the neurons of SOM near the decision boundary of classifiers. Based on DBNS an algorithm named SOM based decision boundary visualization (SOMDBV) is proposed to visualize the decision boundary of highdimensional classifiers in a 2D SOM map. In the next section, the procedures of DBPS, DBNS and SOMDBV are described. In section 3, analysis of classifiers using decision boundary is given. In section 4, experiments are performed to demonstrate the usefulness of the algorithms and analysis proposed. Conclusion is drawn in section 5. We assume the output of classifiers is discrete class label, not the probability of input data to belong to some class, although the latter one can be transformed to the former easily. 2. Decision boundary algorithms In this section, we describe the details of following three decision boundary algorithms, DBPS algorithm, DBNS algorithm and SOMDBV algorithm.
2 A model will be obtained after a classifier is trained on the training data set. When the new data is coming, the model will be used to predict the labels of the data. That is the normal usage of classifiers. Some classifiers behave like a white box, and can provide users with comprehensible results. For example, RIPPER [6], a wellknown rulebased algorithm, learns set of rules, and the obtained rules give users a good understanding. But some other classifiers behave like a black box, and users are unable to understand what they have learned! SVM is an example of this kind of classifiers. The knowledge obtained by the trained SVM model is hidden in the decision function, which is complicated and abstract for user to understand. Users even do not know what has happened in the latter case. However, every classifier predicts the labels of data according to some guide lines. For example, RIPPER predicts the labels according to the rule set it has learned, while SVM predicts the labels according to the decision function it has trained. The guide lines, in spite of their forms, form decision boundaries in the input data space. The procedure of prediction can be seen as finding the relation between input data and boundaries. Using a trained classifier to classify data is equal to using the decision boundary of the same classifier to partition data. If the decision boundary of the classifier is obtained and visualized, users will have an insight into the classifier, which will help users to select proper classifier. The forms of the knowledge which classifiers adopt to construct decision boundary are diverse, so acquiring the analytical equations of the decision boundary is an exhausting task. Instead we obtain some sample points on the decision boundary to analyze classifiers. DBPS algorithm is used to obtain the sample point set near the decision boundary in the input space, while DBNS algorithm is used to obtain the sample neurons near the decision boundary on the 2D SOM map, and SOMDBV makes use of DBNS to visualize the decision boundary on the 2D SOM map DBPS algorithm There are two methods for obtaining the points on the decision boundary. The first one is internal method, which uses classifiers internal forms of knowledge to obtain the points on the boundary. For example, using the decision function of SVM we can compute the points on the boundary. The second one is external method, which uses some approximation methods to get the points near the boundary. The first method s advantage is that it can generate accurate points on the boundary, and its disadvantage is that every classifier needs its own implementation of this method because classifiers forms of knowledge are diverse. While the second method can be applied to more classifiers, but the points generated are not as accurate as the first one. DBPS algorithm adopts the second method, and generates the points near the boundary. DBPS uses binary search to calculate the points of intersection of decision boundary and connection between two data points predicted as different by classifiers. The detail of DBPS can be seen in Algorithm 1. Users can control the precision of the points by adjusting iter_no and toler. Algorithm 1* The Decision Boundary Point Set Algorithm generates the point set near the boundary X is the set of sample points. B is the set of decision boundary points. c(x) is the classifier function. iter_no is the limit of iterations numbers. toler is the tolerance of the boundary. for all x X do if c(x) == a then Xa Xa {x} else Xb Xb {x} for all xa Xa do for all xb Xb do B B {DBP (xa, xb, c, iter_no, toler)} function DBP (x1, x2, c, iter_no, toler) x_bound (x1 + x2) / 2 for i = 1 : iter_no do if distance(x1, x2) / 2 < toler then break if c(x_bound) == c(x1) then x1 x_bound else x2 x_bound x_bound (x1 + x2) / 2 return x_bound * The function distance(x1, x2) is trivial, so we do not describe the procedure of this function here.
3 If the classifier is highdimensional, it needs highdimensional data visualization algorithm to visualize the obtained decision boundary point set DBNS algorithm There are two methods for visualizing the decision boundary of a highdimensional classifier. The first one is to use DBPS algorithm to obtain the decision boundary point set in input space which is visualized by some highdimensional data visualization method. The second one is to project the input data onto some lowdimensional map and calculate the point set on the decision boundary in the map space. SOMDBV algorithm adopts the second method, and DBNS algorithm is used to obtain the neurons near the decision boundary on the 2D SOM map. SVMV algorithm uses the decision function of SVM to calculate the distance between the neuron and the classification boundary [3]. In DBNS algorithm, classifiers employ the weights of neurons as input to predict the labels of the data projected to these neurons. [7] adopts interpolation to get extended weight matrix, which avoids high computational complexity. We adopted the same process to obtain the neurons near the decision boundary. The method used to get the neurons near the boundary is the external method in 2.1, which is the same as DBPS algorithm. The topology of SOM used in this paper is rectangle grid, the algorithm can be applied to other topologies easily, however. As seen in Figure 1(a), if four neurons of the rectangle are predicted the same labels, then we suppose there is no neurons inside the rectangle which are near the boundary. Otherwise we use the interpolation to get neurons e, f, g, h, i, then partition the rectangle to 4 smaller ones. And we continue partitioning the small rectangles whose labels are not the same until the times achieve the number user given (Figure 1(b)). At last the center neuron of the rectangle is selected as the one near the decision boundary (Figure 1(c)). (a) (b) (c) Figure 1. Three cases of finding the neurons near the decision boundary. (a) predictions are the same; (b) predictions are not the same; (c) the last step. The detail procedure of DBNS algorithm can be seen in Algorithm2. Algorithm 2* The Decision Boundary Neuron Set Algorithm finds the neuron set near the boundary N is the neurons of the SOM, whose size is m n. B is the set of neurons near the decision boundary. c(x) is the classification model. iter_no is the limit of iterations numbers. for i = 1 : m1 do for j = 1 : n1 do N[] {N(i,j), N(i+1,j), N(i+1,j+1), N(i,j+1)} B B GetDBNeuron(N[], c, iter_no) function GetDBNeuron(N[], c, iter_no) dbn = {} if c(n[]) are not the same then if iter_no == 1 then dbn {GetCenterNeuron(N[])} else N2[][] Partition(N[]) for i = 1:4 do dbn dbn GetDBNeuron(N2[i], c, iter_no1) return dbn * The function GetCenterNeuron(N[]) and function Partition(N[]) are trivial, so we do not describe procedures of these two functions here SOMDBV algorithm The SOMDBV algorithm adopts the second method in 2.2. It first projects the data onto 2D SOM map, then uses DBNS algorithm to obtain the neurons near the decision boundary, and at last display the labels of the data, classifier s predictions of each neuron and the neurons near the decision boundary. The procedure of SOMDBV algorithm is as follows: 1) Classifier is trained on data set X to get the classification model C. 2) SOM algorithm is trained on the same data set X to get the weights W. 3) C is used to classify the W, and gives predictions L. 4) DBNS algorithm is used to get the neuron set N near the decision boundary.
4 5) Input data set X, classifier s predictions L and decision boundary neuron set N are displayed on the 2 D SOM map. 3. Applications of decision boundary What decision boundary can be used to analyze is as follows: 1) The distance between the data and decision boundary is clearly understood by users, which cannot be provided by accuracy. This will help user to select proper classifier. The classifier with boundary in the middle of the data belonging to different class is usually better than the classifier with boundary near data of one class and far from data of other class. It is also able to tell users in which region of the data space the classifier makes incorrect predictions. If users know the region which the new data is likely to fall into and there are several classifiers, they may be able to choose the proper classifier. 2) There is the tradeoff between accuracy and comprehensibility in data mining models [8]. The visualization of decision boundary is able to give an insight into the classifiers with high accuracy which usually results in low comprehensibility. The visualization will help users to analyze the tradeoff between accuracy and comprehensibility. 3) Overfitting is struggled to avoid by classier users. Visualization of decision boundary can give insight to overfitting. Given the same accuracy, the generalization of classifiers with complicated decision boundary is usually not as good as the ones with simpler decision boundary. This can help users to select the classifier with higher generalization, or set the proper parameters for classifier to obtain a more general model. 4) Decision boundary can be adopted to define the similarity of two models obtained by different classifiers. For example, the proportion of the region two classifiers predict the same labels to the whole region the data fall into may be a measure of models similarity. Then we can conclude two models are the same in the case of some given similarity. Given the similarity, one model may be able to be transformed into the model trained by another classifier, which can overcome the drawback of some classifiers. For example, trained artificial neural network (ANN) can be transformed into rule set by extracting rules from ANN, which improves the comprehensibility of the trained ANN with high accuracy [9]. The method for calculating similarity can be used to calculate the fidelity for extracting rules for ANN [9]. 5) Diversity among the base classifiers is important when constructing a classifier ensemble [10]. The decision boundary can be used to calculate the diversity. For example, the integral of the difference between two classifiers decision boundaries may be a measure of diversity, which reflects the difference of partitions of the data space between two classifiers. 4. Experimental results In this section, two experiments are performed to demonstrate the usefulness of proposed algorithms. Classifiers we used are RIPPER and SVM, and WEKA [11] is used for the implementation of these two classifiers. Gaussian kernel with parameter gamma is used as the kernel function of SVM. Implementation of SOM is from MATLAB SOM Toolbox. For SOM, the total iteration numbers are 1000, and the topology is grid topology. The size of SOM is Experimental results of DBPS DBPS algorithm is used to generate the decision boundary point sets of RIPPER and SVM for diamond data. Diamond data is twoclass simulation data with 2 dimensions, whose boundary is diamond, and its two diagonals length is Each class has 100 data points generated randomly. The results are shown in Figure 2, where cross symbols denote the data inside the diamond, while star symbols denote the data outside the diamond, and line between data of different classes is decision boundary. The decision boundary generated by RIPPER is like a cross, while the one generated by SVM is like a diamond as seen in Figure 2. The decision boundary generated by SVM is almost in the middle of the data of different class, while the decision boundary by RIPPER is close to data of one class and far from the data of other one. The position of decision boundary by SVM is more proper than that by RIPPER, so SVM will be the proper one for diamond data. At the same time, the shape of decision boundary generated by RIPPER is more regular, which can be understood better by users. The decision boundary by SVM is more complicated, and it is more difficult for user to understand it. In this experiment, there is the tradeoff between a powerful model with high accuracy and a transparent model with high comprehensibility.
5 (a) (a) (b) Figure 2. Decision boundary point set (a) by RIPPER; (b) by SVM using Gaussian kernel with gamma = Experimental results of SOMDBV The data set for SOMDBV algorithm is Johns Hopkins University Ionosphere database, which is from UCI machine learning repository [12]. The data set contains 351 records with 34 dimensions, of which 225 records are labeled Good, and 126 are labeled Bad. The results are shown in Figure 3, where square symbols denote the data of class Bad, and triangle symbols denote the data of class Good. Cross symbols denote the neurons predicted Bad, and dot symbols denote the neurons predicted Good. The line of Figure 3 denotes the decision boundary. As the analysis of 4.1, SVM in Figure 3(b) is more proper than RIPPER. (b) (c) Figure 3. Visualization of the Ionosphere data set (a) by RIPPER; (b) by SVM using Gaussian kernel with gamma = 2; (c) by SVM using Gaussian kernel with gamma = 20.
6 As seen in Figure 3(b) and Figure 3(c), the decision boundary generated by SVM using Gaussian kernel with gamma being 20 is more complicated than that generated by SVM with gamma being 2. So the SVM using Gaussian kernel with gamma being 20 is more likely to overfit the data. And this conclusion agrees with the experience and the common sense. The number of neurons predicted the same labels by RIPPER and SVM with gamma being 2 is larger than that by SVM with gamma being 20 and SVM with gamma being 2. So although the two SVM models are generated by the same classifier using different parameter, their similarity is less than that of SVM with gamma being 2 and RIPPER. 5. Conclusion and future work In this paper, a novel method for using decision boundary to analyze classifiers is proposed. Two algorithms are proposed to obtain data on decision boundary in different spaces. DBPS algorithm is used to obtain point set on decision boundary in input data space, while DBNS algorithm is used to obtain neuron set on decision boundary on 2D SOM map. SOMDBV algorithm using DBNS algorithm is proposed to visualize the decision boundary of highdimensional classifiers. With the help of decision boundary, users can get an insight into the classifiers. Decision boundary can be used to select proper classifier, to reveal the tradeoff between accuracy and comprehensibility, to detect overfitting, to calculate the similarity of classifiers and to calculate diversity in ensemble learning. This paper has not supplied calculation method for obtaining similarity and diversity. This work will be done in future, and the decision boundary will be used to analyze extracting rules from ANN and ensemble learning. Acknowledgements [3] X. Wang, S. Wu, and Q. Li, SVMV a novel algorithm for the visualization of SVM classification results, Advances in Neural Networks  ISNN 2006, SpringerVerlag, Berlin Heidelberg, 2006, pp [4] Vapnik, V.N., The nature of Statistical Learning Theory, Springer, Berlin Heidelberg, [5] Kohonen, T., SelfOrganizing Maps, Springer, Berlin Heidelberg, [6] W. Cohen, Fast effective rule induction, Proceedings of the 12th International Conference on Machine Learning, Morgan Kaufmann, Tahoe City, CA, 1995, pp [7] S. Wu, and W.S. Chow, Support vector visualization and clustering using selforganization map and support vector oneclass classification, Proceedings of IEEE International Joint Conference on Neural Networks, Portland, USA, 2003, pp [8] U. Johansson, L. Niklasson, and R. König, Accuracy vs. comprehensibility in data mining models, Proceedings of 7th International Conference on Information Fusion, Stockholm, Sweden, 2004, pp [9] R. Andrews, J. Diederich, and A.B. Tickle, Survey and critique of techniques for extracting rules from trained artificial neural networks, KnowledgeBased Systems, Elsevier, Amsterdam, 1995, pp [10] E.K. Tang, P.N. Suganthan, and X. Yao, An analysis of diversity measures, Machine Learning, Springer, Berlin Heidelberg, 2006, pp [11] Witten, I.H., and E. Frank, Data Mining: Practical machine learning tools and techniques, 2nd Edition, Morgan Kaufmann, San Francisco, [12] P.M. Murphy, and D.W. Aha, UCI Repository of machine learning databases [ Irvine, CA: University of California, Department of Information and Computer Science This paper is supported by 863 plan (No. 2007AA01Z197), and National Natural Science Foundation of China (No ). References [1] S.B. Kotsiantis, I.D. Zaharakis, and P.E. Pintelas, Machine learning: a review of classification and combining techniques, Artificial Intelligence Review, Springer, Berlin Heidelberg, 2006, pp [2] O. Melnik, Decision region connectivity analysis: a method for analyzing highdimensional classifiers, Machine Learning, Kluwer, Netherlands, 2002, pp
Concept Tree Based Clustering Visualization with Shaded Similarity Matrices
Syracuse University SURFACE School of Information Studies: Faculty Scholarship School of Information Studies (ischool) 122002 Concept Tree Based Clustering Visualization with Shaded Similarity Matrices
More informationFeatureweighted knearest Neighbor Classifier
Proceedings of the 27 IEEE Symposium on Foundations of Computational Intelligence (FOCI 27) Featureweighted knearest Neighbor Classifier Diego P. Vivencio vivencio@comp.uf scar.br Estevam R. Hruschka
More informationPerformance Analysis of Data Mining Classification Techniques
Performance Analysis of Data Mining Classification Techniques Tejas Mehta 1, Dr. Dhaval Kathiriya 2 Ph.D. Student, School of Computer Science, Dr. Babasaheb Ambedkar Open University, Gujarat, India 1 Principal
More informationGraph projection techniques for SelfOrganizing Maps
Graph projection techniques for SelfOrganizing Maps Georg Pölzlbauer 1, Andreas Rauber 1, Michael Dittenbach 2 1 Vienna University of Technology  Department of Software Technology Favoritenstr. 9 11
More informationDynamic Clustering of Data with Modified KMeans Algorithm
2012 International Conference on Information and Computer Networks (ICICN 2012) IPCSIT vol. 27 (2012) (2012) IACSIT Press, Singapore Dynamic Clustering of Data with Modified KMeans Algorithm Ahamed Shafeeq
More informationThe Role of Biomedical Dataset in Classification
The Role of Biomedical Dataset in Classification Ajay Kumar Tanwani and Muddassar Farooq Next Generation Intelligent Networks Research Center (nexgin RC) National University of Computer & Emerging Sciences
More informationCostsensitive C4.5 with postpruning and competition
Costsensitive C4.5 with postpruning and competition Zilong Xu, Fan Min, William Zhu Lab of Granular Computing, Zhangzhou Normal University, Zhangzhou 363, China Abstract Decision tree is an effective
More informationStability Assessment of Electric Power Systems using Growing Neural Gas and SelfOrganizing Maps
Stability Assessment of Electric Power Systems using Growing Gas and SelfOrganizing Maps Christian Rehtanz, Carsten Leder University of Dortmund, 44221 Dortmund, Germany Abstract. Liberalized competitive
More informationHALF&HALF BAGGING AND HARD BOUNDARY POINTS. Leo Breiman Statistics Department University of California Berkeley, CA
1 HALF&HALF BAGGING AND HARD BOUNDARY POINTS Leo Breiman Statistics Department University of California Berkeley, CA 94720 leo@stat.berkeley.edu Technical Report 534 Statistics Department September 1998
More informationUsing Analytic QP and Sparseness to Speed Training of Support Vector Machines
Using Analytic QP and Sparseness to Speed Training of Support Vector Machines John C. Platt Microsoft Research 1 Microsoft Way Redmond, WA 9805 jplatt@microsoft.com Abstract Training a Support Vector Machine
More informationPRIVACYPRESERVING MULTIPARTY DECISION TREE INDUCTION
PRIVACYPRESERVING MULTIPARTY DECISION TREE INDUCTION Justin Z. Zhan, LiWu Chang, Stan Matwin Abstract We propose a new scheme for multiple parties to conduct data mining computations without disclosing
More informationAnalyzing Outlier Detection Techniques with Hybrid Method
Analyzing Outlier Detection Techniques with Hybrid Method Shruti Aggarwal Assistant Professor Department of Computer Science and Engineering Sri Guru Granth Sahib World University. (SGGSWU) Fatehgarh Sahib,
More informationUnivariate Margin Tree
Univariate Margin Tree Olcay Taner Yıldız Department of Computer Engineering, Işık University, TR34980, Şile, Istanbul, Turkey, olcaytaner@isikun.edu.tr Abstract. In many pattern recognition applications,
More informationWeka ( )
Weka ( http://www.cs.waikato.ac.nz/ml/weka/ ) The phases in which classifier s design can be divided are reflected in WEKA s Explorer structure: Data preprocessing (filtering) and representation Supervised
More informationSOMSN: An Effective Self Organizing Map for Clustering of Social Networks
SOMSN: An Effective Self Organizing Map for Clustering of Social Networks Fatemeh Ghaemmaghami Research Scholar, CSE and IT Dept. Shiraz University, Shiraz, Iran Reza Manouchehri Sarhadi Research Scholar,
More information6. Dicretization methods 6.1 The purpose of discretization
6. Dicretization methods 6.1 The purpose of discretization Often data are given in the form of continuous values. If their number is huge, model building for such data can be difficult. Moreover, many
More informationData Mining and Knowledge Discovery Practice notes 2
Keywords Data Mining and Knowledge Discovery: Practice Notes Petra Kralj Novak Petra.Kralj.Novak@ijs.si Data Attribute, example, attributevalue data, target variable, class, discretization Algorithms
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationWEIGHTED K NEAREST NEIGHBOR CLASSIFICATION ON FEATURE PROJECTIONS 1
WEIGHTED K NEAREST NEIGHBOR CLASSIFICATION ON FEATURE PROJECTIONS 1 H. Altay Güvenir and Aynur Akkuş Department of Computer Engineering and Information Science Bilkent University, 06533, Ankara, Turkey
More informationVersion Space Support Vector Machines: An Extended Paper
Version Space Support Vector Machines: An Extended Paper E.N. Smirnov, I.G. SprinkhuizenKuyper, G.I. Nalbantov 2, and S. Vanderlooy Abstract. We argue to use version spaces as an approach to reliable
More informationCloNI: clustering of JN interval discretization
CloNI: clustering of JN interval discretization C. Ratanamahatana Department of Computer Science, University of California, Riverside, USA Abstract It is known that the naive Bayesian classifier typically
More informationTraffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers
Traffic Signs Recognition using HP and HOG Descriptors Combined to MLP and SVM Classifiers A. Salhi, B. Minaoui, M. Fakir, H. Chakib, H. Grimech Faculty of science and Technology Sultan Moulay Slimane
More informationImproving Classifier Performance by Imputing Missing Values using Discretization Method
Improving Classifier Performance by Imputing Missing Values using Discretization Method E. CHANDRA BLESSIE Assistant Professor, Department of Computer Science, D.J.Academy for Managerial Excellence, Coimbatore,
More informationMachine Learning : Clustering, SelfOrganizing Maps
Machine Learning Clustering, SelfOrganizing Maps 12/12/2013 Machine Learning : Clustering, SelfOrganizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The
More information6. NEURAL NETWORK BASED PATH PLANNING ALGORITHM 6.1 INTRODUCTION
6 NEURAL NETWORK BASED PATH PLANNING ALGORITHM 61 INTRODUCTION In previous chapters path planning algorithms such as trigonometry based path planning algorithm and direction based path planning algorithm
More informationORT EP R RCH A ESE R P A IDI! " #$$% &' (# $!"
R E S E A R C H R E P O R T IDIAP A Parallel Mixture of SVMs for Very Large Scale Problems Ronan Collobert a b Yoshua Bengio b IDIAP RR 0112 April 26, 2002 Samy Bengio a published in Neural Computation,
More informationCOMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS
COMBINED METHOD TO VISUALISE AND REDUCE DIMENSIONALITY OF THE FINANCIAL DATA SETS Toomas Kirt Supervisor: Leo Võhandu Tallinn Technical University Toomas.Kirt@mail.ee Abstract: Key words: For the visualisation
More informationCS 8520: Artificial Intelligence. Machine Learning 2. Paula Matuszek Fall, CSC 8520 Fall Paula Matuszek
CS 8520: Artificial Intelligence Machine Learning 2 Paula Matuszek Fall, 2015!1 Regression Classifiers We said earlier that the task of a supervised learning system can be viewed as learning a function
More informationLecture #11: The Perceptron
Lecture #11: The Perceptron Mat Kallada STAT2450  Introduction to Data Mining Outline for Today Welcome back! Assignment 3 The Perceptron Learning Method Perceptron Learning Rule Assignment 3 Will be
More informationLine Simplification Using SelfOrganizing Maps
Line Simplification Using SelfOrganizing Maps Bin Jiang Division of Geomatics, Dept. of Technology and Built Environment, University of Gävle, Sweden. Byron Nakos School of Rural and Surveying Engineering,
More informationEfficient Case Based Feature Construction
Efficient Case Based Feature Construction Ingo Mierswa and Michael Wurst Artificial Intelligence Unit,Department of Computer Science, University of Dortmund, Germany {mierswa, wurst}@ls8.cs.unidortmund.de
More informationA Rough Set Approach for Generation and Validation of Rules for Missing Attribute Values of a Data Set
A Rough Set Approach for Generation and Validation of Rules for Missing Attribute Values of a Data Set Renu Vashist School of Computer Science and Engineering Shri Mata Vaishno Devi University, Katra,
More informationA Cloud Framework for Big Data Analytics Workflows on Azure
A Cloud Framework for Big Data Analytics Workflows on Azure Fabrizio MAROZZO a, Domenico TALIA a,b and Paolo TRUNFIO a a DIMES, University of Calabria, Rende (CS), Italy b ICARCNR, Rende (CS), Italy Abstract.
More informationCONCEPT FORMATION AND DECISION TREE INDUCTION USING THE GENETIC PROGRAMMING PARADIGM
1 CONCEPT FORMATION AND DECISION TREE INDUCTION USING THE GENETIC PROGRAMMING PARADIGM John R. Koza Computer Science Department Stanford University Stanford, California 94305 USA EMAIL: Koza@Sunburn.Stanford.Edu
More informationSOM+EOF for Finding Missing Values
SOM+EOF for Finding Missing Values Antti Sorjamaa 1, Paul Merlin 2, Bertrand Maillet 2 and Amaury Lendasse 1 1 Helsinki University of Technology  CIS P.O. Box 5400, 02015 HUT  Finland 2 Variances and
More informationImproved DAG SVM: A New Method for MultiClass SVM Classification
548 Int'l Conf. Artificial Intelligence ICAI'09 Improved DAG SVM: A New Method for MultiClass SVM Classification Mostafa Sabzekar, Mohammad GhasemiGol, Mahmoud Naghibzadeh, Hadi Sadoghi Yazdi Department
More informationPATTERN RECOGNITION USING NEURAL NETWORKS
PATTERN RECOGNITION USING NEURAL NETWORKS Santaji Ghorpade 1, Jayshree Ghorpade 2 and Shamla Mantri 3 1 Department of Information Technology Engineering, Pune University, India santaji_11jan@yahoo.co.in,
More informationApproximate Discrete Probability Distribution Representation using a MultiResolution Binary Tree
Approximate Discrete Probability Distribution Representation using a MultiResolution Binary Tree David Bellot and Pierre Bessière GravirIMAG CNRS and INRIA RhôneAlpes Zirst  6 avenue de l Europe  Montbonnot
More informationForward Feature Selection Using Residual Mutual Information
Forward Feature Selection Using Residual Mutual Information Erik Schaffernicht, Christoph Möller, Klaus Debes and HorstMichael Gross Ilmenau University of Technology  Neuroinformatics and Cognitive Robotics
More informationUNSUPERVISED STATIC DISCRETIZATION METHODS IN DATA MINING. Daniela Joiţa Titu Maiorescu University, Bucharest, Romania
UNSUPERVISED STATIC DISCRETIZATION METHODS IN DATA MINING Daniela Joiţa Titu Maiorescu University, Bucharest, Romania danielajoita@utmro Abstract Discretization of realvalued data is often used as a preprocessing
More informationMetaData for Database Mining
MetaData for Database Mining John Cleary, Geoffrey Holmes, Sally Jo Cunningham, and Ian H. Witten Department of Computer Science University of Waikato Hamilton, New Zealand. Abstract: At present, a machine
More informationFlexible Lag Definition for Experimental Variogram Calculation
Flexible Lag Definition for Experimental Variogram Calculation Yupeng Li and Miguel Cuba The inference of the experimental variogram in geostatistics commonly relies on the method of moments approach.
More informationContents. ACE Presentation. Comparison with existing frameworks. Technical aspects. ACE 2.0 and future work. 24 October 2009 ACE 2
ACE Contents ACE Presentation Comparison with existing frameworks Technical aspects ACE 2.0 and future work 24 October 2009 ACE 2 ACE Presentation 24 October 2009 ACE 3 ACE Presentation Framework for using
More informationKernel Combination Versus Classifier Combination
Kernel Combination Versus Classifier Combination WanJui Lee 1, Sergey Verzakov 2, and Robert P.W. Duin 2 1 EE Department, National Sun YatSen University, Kaohsiung, Taiwan wrlee@water.ee.nsysu.edu.tw
More informationHybrid Models Using Unsupervised Clustering for Prediction of Customer Churn
Hybrid Models Using Unsupervised Clustering for Prediction of Customer Churn Indranil Bose and Xi Chen Abstract In this paper, we use twostage hybrid models consisting of unsupervised clustering techniques
More informationUsing Google s PageRank Algorithm to Identify Important Attributes of Genes
Using Google s PageRank Algorithm to Identify Important Attributes of Genes Golam Morshed Osmani Ph.D. Student in Software Engineering Dept. of Computer Science North Dakota State Univesity Fargo, ND 58105
More informationSemiSupervised Support Vector Machines for Unlabeled Data Classification
Optimization Methods and Software,, 1 14 (2001) c 2001 Kluwer Academic Publishers, Boston. Manufactured in The Netherlands. SemiSupervised Support Vector Machines for Unlabeled Data Classification GLENN
More informationFacial Expression Classification with Random Filters Feature Extraction
Facial Expression Classification with Random Filters Feature Extraction Mengye Ren Facial Monkey mren@cs.toronto.edu Zhi Hao Luo It s Me lzh@cs.toronto.edu I. ABSTRACT In our work, we attempted to tackle
More informationBoosting Algorithms for Parallel and Distributed Learning
Distributed and Parallel Databases, 11, 203 229, 2002 c 2002 Kluwer Academic Publishers. Manufactured in The Netherlands. Boosting Algorithms for Parallel and Distributed Learning ALEKSANDAR LAZAREVIC
More informationCombinatorial Approach of Associative Classification
Int. J. Advanced Networking and Applications 470 Combinatorial Approach of Associative Classification P. R. Pal Department of Computer Applications, Shri Vaishnav Institute of Management, Indore, M.P.
More informationFrom Ensemble Methods to Comprehensible Models
From Ensemble Methods to Comprehensible Models Cèsar Ferri, José HernándezOrallo, M.José RamírezQuintana {cferri, jorallo, mramirez}@dsic.upv.es Dep. de Sistemes Informàtics i Computació, Universitat
More informationWeb Data mininga Research area in Web usage mining
IOSR Journal of Computer Engineering (IOSRJCE) eissn: 22780661, p ISSN: 22788727Volume 13, Issue 1 (Jul.  Aug. 2013), PP 2226 Web Data mininga Research area in Web usage mining 1 V.S.Thiyagarajan,
More informationLecture 9: Support Vector Machines
Lecture 9: Support Vector Machines William Webber (william@williamwebber.com) COMP90042, 2014, Semester 1, Lecture 8 What we ll learn in this lecture Support Vector Machines (SVMs) a highly robust and
More informationLogical Decision Rules: Teaching C4.5 to Speak Prolog
Logical Decision Rules: Teaching C4.5 to Speak Prolog Kamran Karimi and Howard J. Hamilton Department of Computer Science University of Regina Regina, Saskatchewan Canada S4S 0A2 {karimi,hamilton}@cs.uregina.ca
More informationImage Segmentation Based on Watershed and Edge Detection Techniques
0 The International Arab Journal of Information Technology, Vol., No., April 00 Image Segmentation Based on Watershed and Edge Detection Techniques Nassir Salman Computer Science Department, Zarqa Private
More informationSpectral Coding of ThreeDimensional Mesh Geometry Information Using Dual Graph
Spectral Coding of ThreeDimensional Mesh Geometry Information Using Dual Graph SungYeol Kim, SeungUk Yoon, and YoSung Ho Gwangju Institute of Science and Technology (GIST) 1 Oryongdong, Bukgu, Gwangju,
More informationFeature Extraction Using ICA
Feature Extraction Using ICA Nojun Kwak, ChongHo Choi, and Jin Young Choi School of Electrical Eng., ASRI, Seoul National University San 561, Shinlimdong, Kwanakku, Seoul 151742, Korea {triplea,chchoi}@csl.snu.ac.kr,
More information4. Feedforward neural networks. 4.1 Feedforward neural network structure
4. Feedforward neural networks 4.1 Feedforward neural network structure Feedforward neural network is one of the most common network architectures. Its structure and some basic preprocessing issues required
More informationLecture 3: Art Gallery Problems and Polygon Triangulation
EECS 396/496: Computational Geometry Fall 2017 Lecture 3: Art Gallery Problems and Polygon Triangulation Lecturer: Huck Bennett In this lecture, we study the problem of guarding an art gallery (specified
More informationHandling Missing Values via Decomposition of the Conditioned Set
Handling Missing Values via Decomposition of the Conditioned Set MeiLing Shyu, Indika Priyantha KuruppuAppuhamilage Department of Electrical and Computer Engineering, University of Miami Coral Gables,
More informationSelfOrganizing Maps for Analysis of Expandable Polystyrene Batch Process
International Journal of Computers, Communications & Control Vol. II (2007), No. 2, pp. 143148 SelfOrganizing Maps for Analysis of Expandable Polystyrene Batch Process Mikko Heikkinen, Ville Nurminen,
More informationA Comparative Study of Locality Preserving Projection and Principle Component Analysis on Classification Performance Using Logistic Regression
Journal of Data Analysis and Information Processing, 2016, 4, 5563 Published Online May 2016 in SciRes. http://www.scirp.org/journal/jdaip http://dx.doi.org/10.4236/jdaip.2016.42005 A Comparative Study
More informationAn Empirical Investigation of the TradeOff Between Consistency and Coverage in Rule Learning Heuristics
An Empirical Investigation of the TradeOff Between Consistency and Coverage in Rule Learning Heuristics Frederik Janssen and Johannes Fürnkranz TU Darmstadt, Knowledge Engineering Group Hochschulstraße
More informationData Mining Technology Based on Bayesian Network Structure Applied in Learning
, pp.6771 http://dx.doi.org/10.14257/astl.2016.137.12 Data Mining Technology Based on Bayesian Network Structure Applied in Learning Chunhua Wang, Dong Han College of Information Engineering, Huanghuai
More informationData Mining: Classifier Evaluation. CSCIB490 Seminar in Computer Science (Data Mining)
Data Mining: Classifier Evaluation CSCIB490 Seminar in Computer Science (Data Mining) Predictor Evaluation 1. Question: how good is our algorithm? how will we estimate its performance? 2. Question: what
More informationUsing Analytic QP and Sparseness to Speed Training of Support Vector Machines
Using Analytic QP and Sparseness to Speed Training of Support Vector Machines John C. Platt Microsoft Research 1 Microsoft Way Redmond, WA 98052 jplatt@microsoft.com Abstract Training a Support Vector
More informationData Mining and Knowledge Discovery: Practice Notes
Data Mining and Knowledge Discovery: Practice Notes Petra Kralj Novak Petra.Kralj.Novak@ijs.si 2016/11/16 1 Keywords Data Attribute, example, attributevalue data, target variable, class, discretization
More informationUnsupervised Clustering of Web Sessions to Detect Malicious and Nonmalicious Website Users
Unsupervised Clustering of Web Sessions to Detect Malicious and Nonmalicious Website Users ANT 2011 Dusan Stevanovic York University, Toronto, Canada September 19 th, 2011 Outline DenialofService and
More informationNormalization based K means Clustering Algorithm
Normalization based K means Clustering Algorithm Deepali Virmani 1,Shweta Taneja 2,Geetika Malhotra 3 1 Department of Computer Science,Bhagwan Parshuram Institute of Technology,New Delhi Email:deepalivirmani@gmail.com
More informationCOMPUTATIONAL INTELLIGENCE
COMPUTATIONAL INTELLIGENCE Radial Basis Function Networks Adrian Horzyk Preface Radial Basis Function Networks (RBFN) are a kind of artificial neural networks that use radial basis functions (RBF) as activation
More informationData Mining. Introduction. Hamid Beigy. Sharif University of Technology. Fall 1394
Data Mining Introduction Hamid Beigy Sharif University of Technology Fall 1394 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1394 1 / 20 Table of contents 1 Introduction 2 Data mining
More informationHybrid Feature Selection for Modeling Intrusion Detection Systems
Hybrid Feature Selection for Modeling Intrusion Detection Systems Srilatha Chebrolu, Ajith Abraham and Johnson P Thomas Department of Computer Science, Oklahoma State University, USA ajith.abraham@ieee.org,
More informationA Network Intrusion Detection System Architecture Based on Snort and. Computational Intelligence
2nd International Conference on Electronics, Network and Computer Engineering (ICENCE 206) A Network Intrusion Detection System Architecture Based on Snort and Computational Intelligence Tao Liu, a, Da
More informationA NEW ALGORITHM FOR OPTIMIZING THE SELF ORGANIZING MAP
A NEW ALGORITHM FOR OPTIMIZING THE SELF ORGANIZING MAP BENHDECH Adil, GHANOU Youssef, EL QADI Abderrahim Team TIM, High School of Technology, Moulay Ismail University, Meknes, Morocco Email: adilbenhdech@gmail.com,
More informationSTUDYING OF CLASSIFYING CHINESE SMS MESSAGES
STUDYING OF CLASSIFYING CHINESE SMS MESSAGES BASED ON BAYESIAN CLASSIFICATION 1 LI FENG, 2 LI JIGANG 1,2 Computer Science Department, DongHua University, Shanghai, China Email: 1 Lifeng@dhu.edu.cn, 2
More informationWeb Usage Mining: A Research Area in Web Mining
Web Usage Mining: A Research Area in Web Mining Rajni Pamnani, Pramila Chawan Department of computer technology, VJTI University, Mumbai Abstract Web usage mining is a main research area in Web mining
More informationRule Based Learning Systems for Support Vector Machines
Rule Based Learning Systems for Support Vector Machines Haydemar Núñez Artificial Intelligence Laboratory, Central University of Venezuela, Caracas, Venezuela Cecilio Angulo (cecilio.angulo@upc.edu) and
More informationPrototyping DM Techniques with WEKA and YALE OpenSource Software
TIES443 Contents Tutorial 1 Prototyping DM Techniques with WEKA and YALE OpenSource Software Department of Mathematical Information Technology University of Jyväskylä Mykola Pechenizkiy Course webpage:
More informationA Systematic Overview of Data Mining Algorithms. Sargur Srihari University at Buffalo The State University of New York
A Systematic Overview of Data Mining Algorithms Sargur Srihari University at Buffalo The State University of New York 1 Topics Data Mining Algorithm Definition Example of CART Classification Iris, Wine
More informationParallel Evaluation of Hopfield Neural Networks
Parallel Evaluation of Hopfield Neural Networks Antoine Eiche, Daniel Chillet, Sebastien Pillement and Olivier Sentieys University of Rennes I / IRISA / INRIA 6 rue de Kerampont, BP 818 2232 LANNION,FRANCE
More informationLRLWLSI: An Improved Latent Semantic Indexing (LSI) Text Classifier
LRLWLSI: An Improved Latent Semantic Indexing (LSI) Text Classifier Wang Ding, Songnian Yu, Shanqing Yu, Wei Wei, and Qianfeng Wang School of Computer Engineering and Science, Shanghai University, 200072
More informationSYMBOLIC FEATURES IN NEURAL NETWORKS
SYMBOLIC FEATURES IN NEURAL NETWORKS Włodzisław Duch, Karol Grudziński and Grzegorz Stawski 1 Department of Computer Methods, Nicolaus Copernicus University ul. Grudziadzka 5, 87100 Toruń, Poland Abstract:
More informationA New Clustering Algorithm On Nominal Data Sets
A New Clustering Algorithm On Nominal Data Sets Bin Wang Abstract This paper presents a new clustering technique named as the Olary algorithm, which is suitable to cluster nominal data sets. This algorithm
More informationAnalytical model A structure and process for analyzing a dataset. For example, a decision tree is a model for the classification of a dataset.
Glossary of data mining terms: Accuracy Accuracy is an important factor in assessing the success of data mining. When applied to data, accuracy refers to the rate of correct values in the data. When applied
More informationCANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA. By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr.
CANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr. Michael Nechyba 1. Abstract The objective of this project is to apply well known
More informationIntroduction to Support Vector Machines
Introduction to Support Vector Machines CS 536: Machine Learning Littman (Wu, TA) Administration Slides borrowed from Martin Law (from the web). 1 Outline History of support vector machines (SVM) Two classes,
More informationIntroducing Partial Matching Approach in Association Rules for Better Treatment of Missing Values
Introducing Partial Matching Approach in Association Rules for Better Treatment of Missing Values SHARIQ BASHIR, SAAD RAZZAQ, UMER MAQBOOL, SONYA TAHIR, A. RAUF BAIG Department of Computer Science (Machine
More informationRPKM: The Rough Possibilistic KModes
RPKM: The Rough Possibilistic KModes Asma Ammar 1, Zied Elouedi 1, and Pawan Lingras 2 1 LARODEC, Institut Supérieur de Gestion de Tunis, Université de Tunis 41 Avenue de la Liberté, 2000 Le Bardo, Tunisie
More informationFace Recognition Based on LDA and Improved PairwiseConstrained Multiple Metric Learning Method
Journal of Information Hiding and Multimedia Signal Processing c 2016 ISSN 20734212 Ubiquitous International Volume 7, Number 5, September 2016 Face Recognition ased on LDA and Improved PairwiseConstrained
More informationEfficiency of kmeans and KMedoids Algorithms for Clustering Arbitrary Data Points
Efficiency of kmeans and KMedoids Algorithms for Clustering Arbitrary Data Points Dr. T. VELMURUGAN Associate professor, PG and Research Department of Computer Science, D.G.Vaishnav College, Chennai600106,
More informationAn Efficient Sequential Covering Algorithm for Explaining Subsets of Data
An Efficient Sequential Covering Algorithm for Explaining Subsets of Data Matthew Michelson 1 and Sofus Macskassy 1 1 Fetch Technologies, 841 Apollo St., Ste. 400, El Segundo, CA, USA Abstract Given a
More informationApplying Kohonen Network in Organising Unstructured Data for Talus Bone
212 Third International Conference on Theoretical and Mathematical Foundations of Computer Science Lecture Notes in Information Technology, Vol.38 Applying Kohonen Network in Organising Unstructured Data
More informationSELFORGANIZING methods such as the Self
Proceedings of International Joint Conference on Neural Networks, Dallas, Texas, USA, August 49, 2013 Maximal Margin Learning Vector Quantisation Trung Le, Dat Tran, Van Nguyen, and Wanli Ma Abstract
More informationComparison of Various Feature Selection Methods in Application to Prototype Best Rules
Comparison of Various Feature Selection Methods in Application to Prototype Best Rules Marcin Blachnik Silesian University of Technology, Electrotechnology Department,Katowice Krasinskiego 8, Poland marcin.blachnik@polsl.pl
More informationFace Detection using Hierarchical SVM
Face Detection using Hierarchical SVM ECE 795 Pattern Recognition Christos Kyrkou Fall Semester 2010 1. Introduction Face detection in video is the process of detecting and classifying small images extracted
More informationData mining with sparse grids
Data mining with sparse grids Jochen Garcke and Michael Griebel Institut für Angewandte Mathematik Universität Bonn Data mining with sparse grids p.1/40 Overview What is Data mining? Regularization networks
More informationA Dendrogram. Bioinformatics (Lec 17)
A Dendrogram 3/15/05 1 Hierarchical Clustering [Johnson, SC, 1967] Given n points in R d, compute the distance between every pair of points While (not done) Pick closest pair of points s i and s j and
More informationRough Set Approaches to Rule Induction from Incomplete Data
Proceedings of the IPMU'2004, the 10th International Conference on Information Processing and Management of Uncertainty in KnowledgeBased Systems, Perugia, Italy, July 4 9, 2004, vol. 2, 923 930 Rough
More informationFuzzy Partitioning with FID3.1
Fuzzy Partitioning with FID3.1 Cezary Z. Janikow Dept. of Mathematics and Computer Science University of Missouri St. Louis St. Louis, Missouri 63121 janikow@umsl.edu Maciej Fajfer Institute of Computing
More informationA Survey on Postive and Unlabelled Learning
A Survey on Postive and Unlabelled Learning Gang Li Computer & Information Sciences University of Delaware ligang@udel.edu Abstract In this paper we survey the main algorithms used in positive and unlabeled
More information