Preprocessing of Stream Data using Attribute Selection based on Survival of the Fittest

Transcription

1 Preprocessing of Stream Data using Attribute Selection based on Survival of the Fittest Bhakti V. Gavali 1, Prof. Vivekanand Reddy 2 1 Department of Computer Science and Engineering, Visvesvaraya Technological University, Belagavi, Karnataka, India 2 Faculty in Department of Computer Science and Engineering, Visvesvaraya Technological University, Belagavi, Karnataka, India Abstract: In these days data is quickly and constantly expanding and is not consistent in nature. Such a kind of data is practically impossible to store and process so we face a problem to deal with this evolving data. Further, in real applications, the data used is usually noisy, contains missing values, unnecessary features, and therefore much of the time is wasted in pre-processing the data. This time complexity can be reduced by choosing only relevant features to construct model for classification. The system proposed here addresses the issue of adaptive preprocessing for stream data. In this we use Genetic Algorithm as a search method for selecting the attributes which can then be used in learning model. The proposed system is tested on different stream datasets and resulted in significant increment in classification accuracy. Keywords: Concept drift, Adaptive pre-processing, Streaming data, Genetic algorithms, Feature selection. I. INTRODUCTION The term Data mining is known as Knowledge Discovery in Databases. It is the method of getting useful information from the data that is available and it is connected to many other fields such as statistics, IR, machine learning, pattern recognition etc., [4]. These days we have lots of amount of data which is originated from computers and machines. Because of this continuous generation of data we come across the problems such as limitation of storage, computation and communication capabilities in computing systems. To control these challenges, many methods and different techniques have been presented in the past years, which are known as adaptive learning. Adaptive learning model are of 2 types incremental or replacement. Incremental learning can be at the instance level, batch or ensemble level [2, 3]. Replacement can be full or partial. In real applications as and when data is evolves over time learning models need to update or retrain themselves. Moreover, the classification task would be less accurate or incorrect if adaptive learning is not possible. Many of the supervised learning approaches assumes that data is already pre-processed and therefore too much effort is not being spent on adapting the preprocessing along with adapting the learning model [7]. Many adaptive learning systems have been proposed to address this problem. Some of the systems adapt the preprocessor module as well as classification module or they adapt the modules individually according to the requirements. It has been showed that both the methods faces some limitations because learning model produces additional computation costs when it adapts to changes in data distribution. To handle this limitation, a feedback mechanism is introduced between the preprocessor and the classifier module in [1]. Further to improve performance we use minimum number of features to describe a model which can then be used to correctly distinguish normal from anomalous behaviors. Feature selection (subset selection or variable selection) is a method generally used in machine learning, wherein a subset of the features from the available data is selected and used in a learning algorithm. Feature selection is an essential step because while training the All Rights Reserved 257

2 it is computationally not possible to use all available features. In this paper, work is committed on upgrading adaptive preprocessing task in order to get the best output from adaptive learning. Feature selection using genetic algorithm is used as a preprocessing technique in an adaptive preprocessing model which provides relevant feature set. II. FRAMEWORK FOR ADAPTIVE PREPROCESSING WIH FEATURE SELECTION The goal of adaptive preprocessing is to decouple the adaptively of preprocessing and classification by introducing a feedback mechanism between the two modules. When to adapt preprocessing and when to adapt classification is decided with the help a selection strategy. To decide upon decoupling adaptively of the learning model the selection strategy considers the following four situations. In situation S0 there is no need to adapt the data is stationary. Hence the newly arrived data adds to the existing model. In situation S1, only the predictor needs to be adapted. This situation might occur when there is no change in the input data, but there is a change in the relation between the input and the target variables. In situation S2, only the preprocessor needs to be adapted. Such situation may takes place, when there is no change in the distribution of data, but there is a change in the noise on data. In situation S3, we need to adapt both the preprocessor and the predictor. Such condition may occur when the input data distribution changes [1]. Table 1.1: Situations to decide upon decoupling the adaptivity of learning model The proposed system is depicted with the goal to upgrade the performance of adaptive learning with the use of adaptive preprocessing. Most of supervised learning methods assume that the data comes already pre-processed or that preprocessing is an integral part of a learning algorithm. But in real applications, data which is received from various sources is generally improper which contain missing values, repeated attributes. Hence much part of model development is used for data preprocessing. As and when data changes takes place, learning models also need to be able to adapt to changes dynamically. 2.1 Preprocessing In real applications data is often unpredictable, inadequate and noisy. While preprocessing the data an essential step is to complete the missing values smooth out noise and fix inconsistencies [7]. There are several methods which can be used while handling the missing data. The choice of selecting the right method t depends mostly on the problem domain and the aim of the data mining process. Some of these methods are avoiding the tuple with missing values, manually filling in the missing values, global constant is used to fill in the missing value, using the attribute mean to fill in the missing value, attribute mean for every sample belonging to the identical class as the given tuple is used and to use most feasible value in order to fill in the missing value Classification Classification is a data mining (machine learning) technique used to predict group membership for data instances. For example, you may wish to use classification to predict whether the weather on a particular day will be sunny, rainy or cloudy. There are many conventional classification methods such as fuzzy logic techniques, genetic algorithm, case-based reasoning, Bayesian networks, decision tree induction, k-nearest neighbor classifier, rule based classification, support vector machines, rough set method etc. The only difference between the algorithms is whether they are lazy learners or eager learners. Eager learners are learners that use training tuples to construct the data model. All Rights Reserved 258

3 include: The decision tree classifiers, Bayesian classifier, support vector classifier etc. Lazy learners are learners that wait until a test tuple arrives for classification to perform generalization. Examples include nearest neighbor classifiers, locally weighted regression etc Genetic Algorithm GAs simulates the survival of the fittest among individuals over consecutive generation for solving a problem. Each generation consists of a population of character strings that are analogous to the chromosome that we see in our DNA. Each individual represents a point in a search space and a possible solution. The individuals in the population are then made to go through a process of evolution [6].GAs are based on an analogy with the genetic structure and behaviors of chromosomes within a population of individuals using the following foundations: Individuals in a population compete for resources and mates. Those individuals most successful in each 'competition' will produce more offspring than those individuals that perform poorly. Genes from `good' individuals propagate throughout the population so that two good parents will sometimes produce offspring that are better than either parent. Thus each successive generation will become more suited to their environment. Steps for GA: 1. Initialize population: P = Generate p hypotheses at random 2. Evaluate: For each h in P, compute Fitness(h)' 3. While [max Fitness(h)] < Fitness threshold do Create a new generation, Ps: i. Select: Probabilistically select (1 - r)p members of P to add to Ps. ii. Crossover: Probabilistically select random pairs of hypotheses from P. For each pair, (hl, h2), produce two offspring by applying the Crossover operator. Add all offspring to Ps iii. Mutate: Choose m percent of the members of P, with uniform probability. For each, invert one randomly selected bit in its representation. iv. Update: P =Ps v. Evaluate: for each h in P, compute Fitness(h) 4. Return the hypothesis from P that has the highest fitness. III. PROPOSED SYSTEM The framework for the proposed approach is as shown in fig. 1. A new method of using genetic algorithm as evolutionary search method when selecting features is being described in this paper. This proposed system is applied on two streaming datasets and finally its results are tested. In this approach, several times Genetic Algorithm is run while selecting features, by changing its parameters such as population size and maximum generations. Experiment is carried out on three different generations for a certain population size. In all iteration mutation rate and crossover rate is kept stationary. Finally, interconnection of the results of all experiment is taken. This result provides us with only those features that are chosen by each experiment. The chromosome number in one generation is indicated by the population size. The population size degrades the performance after the specific limit and so it is useless to increase it. The probability of how many chromosomes shall be used in reproduction is defined to be Crossover rate [10]. The mutation rate is the probability of how frequently individuals will get All Rights Reserved 259

4 Fig 1: Proposed approach IV. EXPERIMENTAL RESULTS The performance evaluation is done by comparing the proposed system with the existing system. For experimentation we use credit-g dataset and KDD dataset. The aim of these dataset classifications is usually subject to change over time (concept drift). Therefore the proposed system can be evaluated using these two dataset. The first step here is to import training data i.e. load the appropriate dataset and then train the pre-processor and the classifier. For experimental purpose here we use mean as the pre-processor and we use two classifiers i.e. IBK and Hoeffding. First the data is cleaned by training the preprocessor. Later the model for the training data is built by giving the data to the appropriate classifier. 4.1 Accuracy Accuracy means how much our system is correct enough to distinguish between normal and anomalous behavior. Accuracy for KDD Train and credit-g datasets is as shown in table 2 &3. The accuracy of the IBK and Hoeffding classifier is increased for both the datasets when feature selection along with the proposed approach is applied. Table 2: Comparison of accuracies for credit-g dataset Table 3: Comparison of accuracies for KDD train All Rights Reserved 260

5 4.1 Number of features selected Genetic algorithm is used as a search method to select the relevant features for classification. After applying the proposed approach the number of features selected for credit-g and KDD dataset is as shown in the table 4 & 5. As shown in table after applying the proposed approach the number of features selected is less thus decreasing the time required for classification. Table 4: Number of features selected for credit-g dataset Table 5: Number of features selected for KDD train dataset V. CONCLUSION This paper deals with the need for adaptive pre-processing in changing data and depicts how it helps to upgrade prediction accuracy. An online selection strategy that separately manages adaptivity of pre-processing and adaptivity of predictor is executed. Genetic algorithm is used as a search method while selecting the relevant features. The proposed system is executed and tested using the streaming credit-g and KDD train dataset. It can be concluded from the experimental results that the proposed system helps in choosing the minimum number of features from both the datasets and results in increase accuracy of the Hoeffding and IBK classifiers. ACKNOWLEDGMENT The authors would like to acknowledge the reviewers for their valuable comments, which contributed to the clarity of the research and in particular for their suggestions for the statements of applications. REFERENCES 1. IndreZliobaite, BogdanGabrys, Adaptive Preprocessing for Streaming Data, IEEE Trans. Knowledge and Data Engg., vol. 26, no. 2, pp , Feb Data Mining Concepts and Techniques Second edition, Jaiwei Han and MichelineKamber. 3. Mohammad M. Masud, Charu C. Aggarwal, Classification and Adaptive Novel Class Detection of Feature-Evolving Data Streams IEEE transactions on knowledge and data Engineering, VOL. 25, NO. 7, July KetanDesale, Roshani Ade Preprocessing of Streaming Data using Genetic Algorithm, International Journal of Computer Applications Volume 120 No.17, June Rutkowski, L.; Jaworski, M. ; Pietruczuk, L. ; Duda, P., Decision Trees for Mining Data Streams Based on the Gaussian Approximation - Knowledge and Data Engineering, IEEE Transactions on (Volume:26, Issue: 1 ) Jan Wei Li 2004 Using Genetic Algorithm for Network Intrusion Detection, Proceedings of the United States Department of Energy Cyber Security Grou, Training Conference, Vol SaranyaVani.M, Dr. S. Uma, Sherin. A, Efficient adaptive preprocessing with dimensionality reduction for streaming data IJRCAR vol.3 Issue 6, June All Rights Reserved 261