A Text Classification Model Using Convolution Neural Network and Recurrent Neural Network

Transcription

1 Volume 119 No , ISSN: (on-line version) url: A Text Classification Model Using Convolution Neural Network and Recurrent Neural Network Radhika K. 1, Bindu K.R. 2*, Latha Parameswaran 3 Department of Computer Science and Engineering Amrita School of Engineering, Coimbatore Amrita Vishwa Vidyapeetham, India 1 cb.en.p2cse16016@cb.students.amrita.edu 2* j_bindu@cb.amrita.edu 3 p_latha@cb.amrita.edu Abstract Text classification is defined as categorizing document into one of the category in which the text belongs to. Neural Networks are used for classification. Collection of documents is trained and tested using neural networks. In this paper we build a text classification model using Convolution Neural Network and Recurrent Neural Network. We train and test both CNN and RNN model with our dataset. The dataset we used is collection of essays.from the train and test accuracy obtained we reach a conclusion that RNN performs better than CNN for our essay dataset. Keywords- Convolution Neural Network, Recurrent Neural Network, Long short-term memory. I. Introduction Many researches on text classification are going onbecause of its significant role in areas like sentiment analysis, searching, filtering. Text classification is a method of assigning pre-defined classes to the document. Given a set of categories and a collection of dataset the process of finding correct category for each document in the dataset is done by a text classifier. Text classification deals with the collection, analysis, classification, manipulation, retrieval, storage and propagation of information in that database.feature extraction in text classification is mostly done by using bag of words (BoW) model. In that unigram bigram or n-gram are extracted as features. One of the important areas where text classification is used is web searching where classification of documents is done before ranking [6, 10]. A typical web searching model is shown in figure 1. Fig.1. Architecture of question answering system. In this paper a text classification model for CNN and RNN is created by using keras. Experiment is done for CNN and RNN with different ratios of test and train data to check the accuracy. In CNN and RNN models two hidden layers are used. 2. Related Work As the amount of data increasing day by day it is difficult to extract relevant data from the huge amount of unstructured data. The best method used to find the relevant data which is searching is through Topic Modeling. It automatically identifies the topic from the text and helps to retrieve data quickly. It s an unsupervised way for finding necessary text from a large amount data. For document clustering Topic Modeling is mainly used [3, 7].In deep learning CNN is one of the neural network which shows best performance in feature extraction and 1549

2 classification of text even from character level representation of text to train classifiers CNN showing good result because of its high ability to automatically detect features from the text.this helps widely to reduce memory and time for training and testing [1, 2]. 3.CNN Convolution is a process of taking input data and selecting features matrix from it. It consists of a sequence of layers. It includes data is preprocessed and converted to vector format then the result is transferred to convolution layer. Output from the convolution layer undergoes pooling, mainly used pooling method is max pooling after pooling dropout is done to improve the accuracy of the network [8, 9]. Dataset used is essay dataset, which consist of more than 2000 essays of different authors. Divide the data into different ratio of train and test data.assumes that some data as negative and some data as positive so the train and test data includes negative and positive data.specifications used to create CNN model are activation function used is softmax, filters 32, kernel size 4, dropout 0.25, max pooling pool size is 2, loss is calculated using categorical_crossentropy and optimizer used is adam. A model is created with one layer then increased to two. Figure 2 shows the CNN model. 4. RNN Fig.2. Text classification model created for CNN. Recurrent Neural Networks are a form of neural networks which have their own output fed to their own input. This practice of using output as input creates loop in the Networks and helps in preserving some memory.previous input or output has the very good weightage in predicting the output. Before passing to the neural network data is preprocessed and converted to vector format. Dropout is done to reduce the loss in the neural network [4, 5 and 11].Dataset used is essay dataset, divide the data in different ratios of train and test data.assumes that some data as negative and some data as positive so the train and test data includes negative 1550

3 and positive data.specifications used to create RNN model are activation function used is softmax, filters 32, kernel size 4, dropout 0.25, max pooling pool size 2, LSTM of, loss calculated using categorical_crossentropy and optimizer used is adam.a model is created with one layer then increased to two. Figure 3 shows the RNN model with two layers. 5. Results Fig.3.Text classification model created for RNN. Essay dataset is trained and tested with CNN model and obtained the accuracy as shown in Table 1. Table.1. Train and test accuracy obtained for CNN. CNN Performance for Essay Dataset Sl No Size of Train and Test Data Accuracy Train Data Test Data Train Accuracy Test Accuracy Fig.4. Accuracy graph obtained for CNN. 1551

4 In fig 4 X axis consist of the ratio number in which train and test data taken which is mentioned in Table: 1[1: train data and test data 500, 2: train data and test data, 3: train data 2000 and test data 400].Y axis consist of accuracy. For the three experiments got train accuracy as and test accuracy as 50. Essay dataset is trained and tested with RNN model and got the accuracy as shown in the Table 2. Table.2. Train and test accuracy obtained for RNN. RNN Performance for Essay Dataset Sl No Size of Train and Test Data Accuracy Train Data Test Data Train Accuracy Test Accuracy Fig.5. Accuracy graph obtained for RNN 6. Conclusion In this paperwe have created CNN and RNN model using keras for the essay dataset and evaluated their performance based on the test accuracy obtained. CNN we got test accuracy as 50 and for RNN we got test accuracy as 55.Based on the result reached a conclusion that RNN model performed better than CNN model for essay dataset. In future we can test the model using RCNN(combination of CNN and RNN) and evaluate it s performance. References [1] Prusa, Joseph D., and Taghi M. Khoshgoftaar. "Designing a Better Data Representation for Deep Neural Networks and Text Classification."Information Reuse and Integration (IRI), 2016 IEEE 17th International Conference on [2] Wen, Ying, et al. "Learning text representation using recurrent convolutional neural network with highway layers."arxiv preprint arxiv: , [3] Rajasundari T., Subathra P., Kumar P.N. Performance analysis of topic modeling algorithms for news articles Journal of Advanced Research in Dynamical and Control Systems [4] Liu, Pengfei, XipengQiu, and Xuanjing Huang. "Recurrent neural network for text classification with multi-task learning."arxiv: [5] Lai, Siwei, et al. "RecurrentConvolutional Neural Networks for Text Classification." AAAI. Vol

5 [6] Fauzi, M. Ali, AgusZainalArifin, and Sonny ChristianoGosaria. "Indonesian News Classification Using Naïve Bayes and Two-Phase Feature Selection Model." Indonesian Journal of Electrical Engineering and Computer Science : [7] K. R. Bindu, L. Parameswaran, K. V. Soumya, Performance Evaluation of Topic Modelling Algorithms with an application of Q & A Dataset International Journal of Applied Engineering Research, vol. 10, pp , [8] Zhang, Xiang, Junbo Zhao, and YannLeCun. "Character-level convolutional networks for text classification." Advances in neural information processing systems [9] Kim, Yoon. "Convolutional neural network for sentence classification arxiv preprint arxiv: ,2014. [10] Vikas K Vijayan; K. R. Bindu; Latha Parameswaran A comprehensive study of text classification algorithms 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Sept. 2017, Copyright 2017, IEEE. [11] Mittal, Nikita, and AkashSaxena. "Layer Recurrent Neural Network Based Power System Load Forecasting." Indonesian Journal of Electrical Engineering and Computer Science 16.3 (2015):

6 1554