AMOL MUKUND LONDHE, DR.CHELPA LINGAM

Transcription

1 International Journal of Advances in Applied Science and Engineering (IJAEAS) ISSN (P): ; ISSN (E): X Vol. 2, Issue 4, Dec 2015, IIST COMPARATIVE ANALYSIS OF ANN WITH TRADITIONAL DATA MINING TECHNIQUES AMOL MUKUND LONDHE, DR.CHELPA LINGAM Department of Computer Engineering, Pillai HOC College of Engineering and Technology,(Rasayani) Raigad, India.,University of Mumbai amol.londhe20@gmail.com, chelpa.lingam@gmail.com ABSTRACT- A bulk amount of data generated in various fields such as science, business, industry and many more areas due to advance data digitalization and computerization techniques and these data will hold valuable information. Knowledge discovery and decision support processes use these generated data as resources for its operations. Data mining is the important approach to realize knowledge discovery. It is the process of extracting patterns or predicting previously unknown and useful trends from large quantities of data by using the knowledge of multidisciplinary fields such as statistics, artificial intelligence, machine learning, database, management information system and so on. There are many traditional data mining classification techniques such as Naive Bayesian classification, Decision tables, ID3 tree, C4.5 tree, CART etc. An Artificial Neural Network (ANN) is an information processing model that is inspired by the way biological nervous systems, such as the brain, process information.it can make network model from analyzing the mode in the data and discover the unknown knowledge. The aim of this paper is to investigate the performance of different data classification methods for a various set of datasets. KEYWORDS Knowledge Discovery, Data Mining, Artificial Neural Networks, Data Classification, Decision Tree, Back-propagation learning. known results found from sample one or more data sets. I. INTRODUCTION There is valuable information hidden in data. Since the underlying data is generated much faster than it can be processed and made sense of, this information often remains buried and untapped. It is very difficult task for individuals or groups to find out further detail about data with limited resources specifically technological resources. Data mining is a process which is extension of traditional data analysis and statistical approaches and it incorporates analytical techniques drawn from various disciplines like AI, machine learning, OLAP, data visualization, etc. Data mining is a stepwise process which involves steps such as defining problem, collecting related data, preprocessing data, estimating model, interpreting model and finally drawing conclusions from this model. The selection and implementation of the appropriate data-mining technique is the main task and is not straightforward. Data mining functionalities are used to describe the various patterns to be found in data mining tasks. In general, data mining tasks can be classified as descriptive and predictive. Descriptive mining tasks describe the general properties of the existing data in the database. Identification of patterns or relationships in data is task of descriptive model. If there is sample training data available for which values of that attribute are known and goal is to calculate approximately value of a particular target attribute. Predictive model makes prediction about values of new data set using II. DATA CLASSIFICATION Data classification is the forms of data analysis which can be used to extract models describing important data classes or to predict future data trends. The derived model is based on the analysis of a set of training data. Derived model represented with the help of various forms, such as classification (IF-THEN) rules, decision trees, and mathematical formulae. The generated model used to classify the new objects. Classification rules can be easily generated from decision trees.estimation and prediction can be viewed as types of classification. Data classification can be viewed a mapping from the database to the set of classes. All classes are predefined, no overlapping and partition the entire database. Each tuple in the database is assigned to exactly one class.normally; classification problem is implemented in two phases: 1) Create a specific model by evaluating the training data. Input the training data and as output a model developed by this step. 2) Tuples from the target Database can be classified by applying model developed in step 1. Classification results of two classification techniques are never same. Accuracy of classification is the most important 53

2 performance measure for classification problem. Classification accuracy determined by considering percentage of tuples placed in correct class. Confusion matrix can be used for calculating accuracy of classification. Various types of data classification algorithms based on categorization are as follows: Statistical-Based algorithms Distance-Based algorithms Decision Tree-Based algorithms Neural Network-Based algorithms Rule-Based Algorithms A. Naive Bayesian Classification Bayesian classifiers are statistical classifiers. This classifier offers a simple yet powerful supervised classification technique. A Naive Bayes classifier is a simple probabilistic classifier based on applying Bayes' theorem (from Bayesian statistics) with strong (naive) independence assumptions which works on the probability theory. In simple terms, a naive Bayes classifier assumes that there is no dependency between two or more features of class for their present or absent in class. For example, a fruit may be considered to be an orange if it is red, round, and about 3" in diameter. These properties depend on each other or upon the existence of the other features. A naive Bayes classifier considers all of these features independently and makes probability independently that this fruit is a apple. B. Decision Trees The decision tree is well known classification method due to its tree like structure. A decision tree is a tree structure that consists of elements as root node, branches, and leaf nodes. Test on an attribute represented by internal node, each branch represent the outcome of a test, and various class label hold by leaf node. The root node is the node present at the topmost level. Decision tree is a model which is not only predictive but also descriptive. ID3, C4.5, CART are the examples of decision tree algorithms. Classification problem using decision tree is a two-step process: a) Decision tree induction: Construct a decision tree using training data. b) For each tuple in dataset, apply the decision tree to determine its class. The ID3 technique to building a decision tree is based on information theory and attempts to minimize the expected number of comparisons. The concept used to quantify information is called entropy. Entropy is the measure of amount of uncertainty or surprise or randomness in a set of data. The decision tree algorithm C4.5 improves ID3. In C4.5, when tree is built, missing data are simply ignoredtree pruning strategies are also proposed in C4.5 such as subtree replacement or subtree raising [6]. CART (Classification and regression tree) is a technique that generates binary tree. As with ID3, entropy is used as a measure to choose the best splitting attribute. The tree stops growing when no split will improve the performance. III. ARTIFICIAL NEURAL NETWORKS Artificial Neural Networks are relatively simple electronic models based on same as neural structure of the brain. The brain basically learns from experience day to day. It is natural proof that some problems that are beyond the scope of current computers are indeed solvable by current computation methods. This new approach to computing also provides a more graceful degradation during system overload than its more traditional factors consider. In ANN simple computation elements used is called as neurons or node. All these nodes organized hierarchically in layers and are interconnected with each other like biological nervous system. Fig. 1 A Multilayer feedforward artificial neural network Artificial neural network is made up of interconnecting structure of neurons. A neuron is an information-processing unit that is fundamental to the operation of a neural network. Fig. 1 shows one such ANN. The source nodes in the input layer of the network supply respective elements of the activation pattern (input vector), which constitute the input signals applied to the neurons (computation nodes) in the second layer (i.e. the first hidden layer). The output signals of the second layer are used as inputs to the third layer, and so on for the rest of the network. Advantages of Neural Networks: High Accuracy Noise Tolerance Independence from prior assumptions Ease of maintenance In layered feed forward artificial neural network s back propagation algorithm is used. Meaning of layered network is nothing but all neuron arranged in layers. In network neurons send their signals forward and then the errors of algorithm are propagated backwards. Supervised learning technique used in back propagation algorithm, which means that we are providing not only examples of input but also expected output to the algorithm. Error of network is calculated and error is nothing but difference between actual results generated and expected result. The idea behind back propagation algorithm is until training data learn by ANN reduce error continuously in step by step manner. 54

3 A Back-propagation neural network is a connection of set of connected input/output units (neurons) and each connection has been allocating specific weight. By adjusting weights at the time of learning phase network will able to predict the correct class label of the input samples. Backpropagation Algorithm: 1) In network initialize the weights (frequently randomly) 2) Repeat * For each example e in the training set do (a) O = neural_net_output(network, e); Forward pass (b) T = teacher output for e (c) Calculate error (T O) at the output units (d) Compute delta w i for all weights from hidden layer to output layer; Backward pass (e) Compute delta w i for all weights from input layer to hidden layer; Backward pass continued (f) Update the weights in the network * End 3) Until all examples classified correctly or stopping criterion Satisfied 4) return (network); IV. EXPERIMENTAL RESULTS All the experiments and analysis are done with the help of WEKA tool. Weka is a software platform used for the various Data Mining tasks. It provides various learning algorithms for data analysis and study. Weka, developed at the University of Waikato in New Zealand. "WEKA" stands for the Waikato Environment for Knowledge Analysis. The system is written in Java, an object oriented programming language that is widely available for all major computer platforms, and Weka has been tested under Linux, Windows, and Macintosh operating systems. The various datasets used for this experiment are Iris, Weather, car, vowel, Diabetes and segment-test [10]. Glass dataset includes glass identification database with 10 attributes and classification attribute as type of glass. Segment-test dataset has image segmentation data with 19 continuous attributes and type of segment as classification attribute. Iris is the perhaps the best known database to be found in the pattern recognition literature. It has 4 numeric attributes and classification attribute as class of the plant. Weather dataset has data about play possibility of cricket match depending upon weather conditions.. Diabetes dataset has Pima Indians diabetes data with 8 attributes plus one class attribute. For comparing performance of the various data classification algorithms accuracy is the most critical criteria. Accuracy refers to the ability of the algorithm to correctly predict the class label of new or previously unseen data. Other performance evaluation criteria are speed, robustness, scalability and interpretability. Cross Validation: Normally, training data is firstly used to derive a classifier and then later to estimate the accuracy of the classifier. This can result in deceptive over-optimistic estimates. Cross validation randomly partitions the given data in k-folds. To estimate classifier s accuracy calculated by using cross validation techniques. Yet cross validation techniques useful for selecting among several classifiers even if it will increases the overall computation time 10-fold cross validation is calculated as follows. First, the dataset is divided in 2 sections randomly. One section is included 90% of all dataset and called as learning dataset. Another section is included 10% of all dataset and called as validation datasets. Second, the learning dataset is used for acquiring rules, and the validation dataset is used for validating the rules externally. Third, the process is repeated 10 times. The accuracy estimate is the overall number of correct classifications from the 10 iterations, divided by the total number of samples in the initial data [7]. Fig.2 Cross Validation technique (10 fold). Naïve Bayesian, C4.5 tree, Multilayer Perceptron Back propagation, CART and Decision tables are the algorithms used for comparing predictive accuracy. Following figures (from Fig. 3 to Fig. 7) shows the result of the comparison. Fig. 2 Percentage accuracy of five data classification algorithms for Iris 55

4 Fig. 3 Percentage accuracy of five data classification algorithms for Weather Fig. 4 Percentage accuracy of five data classification algorithms for Diabetes Figure 3 shows comparison of five data classification algorithms in terms of predictive accuracy for iris dataset. First, iris dataset is used as it is for training, then 5% cross validation is applied to iris dataset and used for training. Then after 10% cross validated iris data is applied for training. As cross validation randomly partitions data, classifiers accuracy results from 10% cross validated data are more appealing and useful for selecting among several classifiers. Similarly, Figure 4 to Figure 7 shows classifier s accuracy comparison for weather, Diabetes, Glass, Segment-test datasets respectively. Even though all five classifiers show higher accuracy for datasets without cross validation, results could be misleading. Cross validation provides natural randomness to training data. It was observed that Naive Bayesian classifier has lowest accuracy of data classification. This is due to assumption of attribute independence while classifying. Decision tree-based algorithms have better performance in case of traditional data mining techniques. But, neural network based backpropagation learning (MLP) gives best classifier s accuracy for most of the datasets. For backpropagation algorithm, classifier s accuracy was not much affected by n-folds of cross validation. It was seen that, backpropagation algorithm takes much time to learn and build model as that of other algorithms. As neural networks are fault tolerant structures, they do not get much affected by missing or noisy data. V. CONCLUSIONS Fig. 5 percentage accuracy of five data classification algorithms for Glass Fig. 6 percentage accuracy of five data classification algorithms for Segment As compare with other data mining algorithms Multilayer Perceptron backpropagation algorithm consumes relatively more time for learning than other algorithms but outcome (accuracy) is appreciable. The interpretability of Decision tree is simple due to tree based rules generation. Neural Networks produces high degree of accuracy, but the interpretation is difficult. In general, both Decision Trees and Neural Networks have advantages and drawbacks. If proper rule extraction methods are used for interpretation of neural networks, they are the best classifiers for data classification problems. Artificial Neural Networks are the promising substitutes for traditional data mining techniques due to their characteristics such as robustness, self-organizing structure, parallel processing, distributed storage and high degree of fault tolerance. REFERENCES [1] Prof. Sonal Kadu and Prof. Sheetal Dhande, Effective Data Mining Through Neural Networks", IJARCSSE - Volume 2, Issue 3, March [2] Dr. Yashpal Singh and Alok Singh Chauhan, Neural Networks in Data Mining,Journal of Theoretical and Applied Information Technology, [3] Haykin S., Neural Networks, Prentice Hall International Inc.,

5 [4] Yasogha, P and M. Kannan, Analysis of a Population of Diabetics Patients Databases in Weka Tool, International Journal of Science& Engineering Research, Vol. 2, Issue 5, May [5] Kaushik H. Raviya and Biren Gajjar, Performance Evaluation of Different Data Mining Classification Algorithm Using WEKA. [6] M.H. Dunham. and S.Sridhar Data Mining Introductory and Advanced Topics", Pearson Education 2007 ISBN [7] Han J, Kamber, Data Mining Concepts and Techniques, Morgan Kaufmann, March [8] Trilok Chand Sharma and Manoj Jain, WEKA Approach for Comparative Study of Classification Algorithm. [9] Mrs. Bharati M. Ramageri, Data Mining Techniques and Applications",IJCSE Vol. 1 No [10] C. Lakshmi Devasena, T. Sumathi, V.V. Gomathi and M. Hemalatha, Effectiveness Evaluation of Rule Based Classifiers for the Classification of Iris Data Set. 57