Research of Classification based on Deep Neural Network

Transcription

1 2018 Internationa Conference on Sensor Network and Computer Engineering (ICSNCE 2018) Research of Emai Cassification based on Deep Neura Network Wang Yawen Schoo of Computer Science and Engineering Xi an Technoogica University Xi an, , China e-mai: Yu Fan a, Wei Yanxi b Schoo of Computer Science and Engineering Xi an Technoogica University Xi an, , China e-mai: a @qq.com b @qq.com Abstract The effective distinction between norma emai and spam, so as to maximize the possibe of fitering spam has become a research hotspot currenty. Naive bayes agorithm is a kind of frequenty-used emai cassification and it is a statistica-based cassification agorithm. It assumes that the attributes are independent of each other when given the target vaue. This hypothesis is apparenty impossibe in the emai cassification, so the accuracy of emai cassification based on naive bayes agorithm is ow. In ausion to the probem of poor accuracy of emai cassification based on naive bayes agorithm, schoars have proposed some new emai cassification agorithms. The emai cassification agorithm based on deep neura network is one kind of them. The deep neura network is an artificia neura network with fu connection between ayer and ayer. The agorithm extracted the emai feature from the training emai sampes and constructed a DNN with mutipe hidden ayers, the DNN cassifier was generated by training sampes, and finay the testing emais were cassified, and they were marked whether they were spam or not. In order to verify the effect of the emai cassification agorithm based on DNN, in this paper we constructed a DNN with 2 hidden ayers. The number of nodes in each hidden ayer was 30. When the training set was trained, we set up 2000 batches, and each batch has 3 trained data. We used the famous Spam Base dataset as the data set. The experiment resut showed that DNN was higher than naive Bayes in the accuracy of emai cassification when the proportion of the training set was 10%, 20%, 30%, 40% and 50% respectivey, and DNN showed a good cassification effect. With the deveopment of science and technoogy, spam manifests in many forms and the damage of it is more serious, this puts forward higher requirements for the accuracy of spam recognition. The focus of next research wi be combining various agorithms to further improve the effect of emai cassification. Keywords-Deep Neura Networks; Spam Emai; Cassification; Naive Bayes; SpamBase Data Set I. INTRODUCTION Emai has become a major way of communication for peope at present, but the probem of spam comes behind. The harm of spam is mainy manifested as the foowing aspects: occupying bandwidth, eading to the congestion of the emai server and reducing the efficiency of the network; consuming the time of the user and affecting the work efficiency. Therefore, the effective distinction between norma emai and spam, so as to maximize the possibe of fitering spam has become a research hotspot currenty. Naive bayes agorithm is a kind of frequenty-used emai cassification and it is a statistica-based cassification agorithm[1-3], which has the characteristics of simpe reaization and fast cassification. However, it assumes that the attributes are independent of each other when given the target vaue[4].this hypothesis is apparenty impossibe in the emai cassification, so the accuracy of emai cassification based on naive bayes agorithm is ow. In ausion to the probem of poor accuracy of emai cassification based on naive bayes agorithm, schoars have proposed some new emai cassification agorithms. The 17

2 2018 Internationa Conference on Sensor Network and Computer Engineering (ICSNCE 2018) emai cassification agorithm based on deep neura network (DNN) is one kind of them. ayer, a is the activation vaue of the ayer, z is the II. THEORETICAL BASIS The basic concept of artificia neura network is based on the hypothesis and mode construction of how the human brain responds to compex probems[4-6]. The deep neura network is an artificia neura network with fu connection between ayer and ayer, and its structure is shown in figure 1.The fu connection between ayer and ayer means that any neuron in the neurons in the ayer must be connected to any of the ayer.athough the deep neura network ooks compex, it is sti the same as the perceptron from a sma oca mode. Input ayer ayer1 ayer2 ayern Output ayer weighted input of a neurons in the w ayer, Then the weight coefficient of row j, coumn k. The reationship between the activation vaue of the activation vaue of the foowing matrix reationship: jk is ayer and the ayer can be expressed by the Here respresents the non-inear activation function of the nodes on the hidden ayers, and the traditiona DNN uses sigmoid function usuay, as shown in expression (3).Because the sigmoid function has properties such as monotone increasing and its inverse function has the property of monotone increasing, it is often used as a threshod function of neura networks, It maps the variabes between 0 and 1.The sigmoid function curve is shown in figure 2: Figure 1. Structure diagram of deep neura network We use w jk to respresent the weight coefficient between the neuron in the ayer and the neuron in the ayer, b j to represent the bias of the neuron in the ayer, to represent the activation vaue of the neuron in the ayer. We can get the foowing reationship between the activation vaue of the neuron in the ayerand the activation vaue of a neuron sin the ayer: Figure 2. The sigmoid function curve III. ALGORITHM DESCRIPTION Impementation process of mai cassification agorithm based on deep neura network was shown in Figure 3. We assume that w is the weight coefficient matrix of a the neurons in the ayer, b is the bias matrix of the 18

3 2018 Internationa Conference on Sensor Network and Computer Engineering (ICSNCE 2018) Begin Read the contents of the emai for training and extract the emai features for training Construct a DNN containing mutipe hidden ayers, set the number of hidden ayers,set the number of nodes on each ayer, set training batches and the number of training data for each batch Train DNN to generate DNN cassifier Read the contents of the emai for testing and extract the emai features for testing Use the DNN cassifier to cassify the testing emai and mark whether they are spam Compare the cassification resut of the emai with the actua tag, verify the correctness of the agorithm Figure 3. End Agorithm execution process Step 1: Read the contents of the emai for training from the Spam Base dataset and extract the emai features for training such as word_freq_, char_freq_, capita_run_ength_average, capita_run_ength_tota, and so on. capita_run_ength_ongest, Step 2: Construct a DNN containing mutipe hidden ayers, set the number of hidden ayers(n_casses),set the number of nodes on each ayer (hidden_units),set training batches( steps) and the number of training data for each batch (batch_size). Step 3: Train DNN to generate DNN cassifier. Step 4: Read the contents of the emai for testing from the SpamBase data set and extract the emai features for testing such as word_freq_, char_freq_, capita_run_ength_average, capita_run_ength_ongest, capita_run_ength_tota, and so on. Step 5: Use the DNN cassifier to cassify the testing emai and mark whether they are spam(1 or 0). Step 6: Compare the cassification resut of the emai (y_predict) with the actua tag(y_test), cacuate the accuracy of the agorithm in the emai cassification(accuracy_score) and verify the correctness of the agorithm. IV. EXPERIMENTAL RESULTS AND ANALYSIS In order to verify the effect of the emai cassification agorithm based on DNN, in this paper we constructeda DNN with 2 hidden ayers. The number of nodes in each hidden ayer was 30. When the training set was trained, we set up 2000 batches, and each batch has 3 trained data. We used the famous SpamBase dataset as the data set, which was from the UCI machine earning ibrary at the University of Caifornia, USA. The specific situation is shown in tabe I. We compared the two kinds of emai fitering agorithms of DNN and naive Bayes with accuracy, which is the main evauation standard of emai fitering technoogy. The accuracy is defined as foows: Number of correcty identified emai Accuracy Tota number of emais We did five groups of experiments in this paper.the seection case of training set and testing set in each experiment is shown in tabe II. 19

4 2018 Internationa Conference on Sensor Network and Computer Engineering (ICSNCE 2018) TABLE I. SPAMBASE DATA SET Index Vaue Index Vaue Tota number of the emai 4601 Number of attributes 57 Number of emai category abes 2 Emai category Vaidemai,spam emai Number of the spam emai 1813 The proportion of spam emai 39.4% Number of the vaid mai 2788 The proportion of vaidemai 60.6% TABLE II. THE SELECTION CASE OF TRAINING SET AND TESTING SET group number The proportion of the training set in a data The number of emai in training set The proportion of the testing set in a data The number of emai in testing set 1 90% % % % % % % % % % 2301 The experimenta resuts were shown in Figure 4. V. CONCLUSION Figure 4. The comparison of accuracy of the two agorithms The appication of emai cassification agorithm based on deep neura network is studied in this paper. The agorithm constructed mutipe hidden ayers and generated DNN cassifiers through training. The experiment resuts showed that the accuracy of the agorithm is obviousy higher than the naive Bayes agorithm. With the deveopment of science and technoogy, spam manifests in many forms and the damage of it is more serious, this puts forward higher requirements for the accuracy of spam recognition. The focus of next research wi becombining various agorithms to further improve the effect of emai cassification. The experiment resut showed that DNN was higher than naive Bayes in the accuracy of emai cassification when the proportion of the training set was 10%, 20%, 30%, 40% and 50% respectivey, and DNN showed a good cassification effect. ACKNOWLEDGMENTS New network and detection contro nationa joint engineering aboratory fund program (GSYSJ ). Xi an Technoogica University Principa Scientific Research Fund Project: XAGDXJJ

5 2018 Internationa Conference on Sensor Network and Computer Engineering (ICSNCE 2018) REFERENCE [1] Cao Cuiing, Wang Yuanyuan and Yuan Ye, Research of a spam fiter based on improved naive Bayes agorithm, Chinese Journa of Network and Information Security, Vo.3No.3, pp , March [2] Wang Zhiyong and Liu Hongmei, DESIGN AND IMPLEMENTATION OF BAYESIAN SPAM FILTE R ING SYSTEM, Journa of Inner Mongoia Agricutura University(Natura Science Edition), Vo.38 No.3, pp.82-86, May [3] Wang QingSong and Wei Ruyu, Bayesian Chinese Spam Fitering Method Based on Phrases, Computer Science, Vo. 43 No. 4, pp , Apr [4] Neura Networks and Deep Learning [EB/OL]. [5] Li Kun, CHai Yumei and Zhao Honging, Estimation of Feta Weight Based on Deep Neura Network, Computer Science, Vo. 43 No. 11A, pp , Nov [6] Cao Meng, Li Hongyan and Zhao Rongrong, A Pitch Detection Method Based on Deep NeuraNetwork, Microeectronics & Computer, Vo1.33 No.6, pp , June [7] Ren Rongrong, Zhou Mingquan and Geng Guohua, The muti-scae features extraction method based on deep neura network, Journa of Northwest University(Natura Science Edition), Vo. 47No.2, pp , Apr2017. [8] S.L.Zhang Research on Deep Neura Networks based Modes for Speech Recognition(Ph.D., University of Science and Technoogy of China, China 2017), p.35 21