GPU Implementation of Parallel SVM as Applied to Intrusion Detection System

Transcription

1 GPU Impementation of Parae SVM as Appied to Intrusion Detection System Sudarshan Hiray Research Schoar, Department of Computer Engineering, Vishwakarma Institute of Technoogy, Pune, India Noshir Tarapore Assistant Professor, Department of Computer Engineering, Vishwakarma Institute of Technoogy, Pune, India Abstract--Intrusion Detection System (IDS) is an important and necessary component in ensuring network security that invoves separation of attacks and Non-attacks. Support Vector Machine (SVM) is one of the most popuar toos for soving genera cassification probems because of its high prediction accuracy. However, the training phase of noninear kerne based SVM agorithm is a computationay expensive task, especiay for arge datasets. In this paper, we propose an agorithm to sove arge cassification probems based on parae SVM. The system utiizes powerfu GPU device and Mutithreading to improve the speed performance of SVM training phases. Initia experiments done with KDD CUP 1999 dataset show that the training time of SVM is shortened and the detection accuracy obtained by our method is exacty same or more than as that obtained by seria SVM. Keywords--Intrusion Detection System (IDS), Support Vector Machine (SVM), Graphics Processor Unit(GPU), Parae, Cascade, Mutithreading I. INTRODUCTION With the rapidy growing connectivity of the Internet, networked computer systems are increasingy paying vita roes in our modern society. Whie the Internet has brought great benefits to this society, it has aso made critica systems vunerabe to maicious attacks. Since a preventive approach such as firewa is not sufficient to provide sufficient security for a computer system, intrusion detection systems (IDS) are introduced as a second ine of defense. IDS can detect and identify intrusion behavior or intrusion attempts in a computer system by monitoring and anayzing network packet or system audit og, and then send intrusion aerts to system administrators in rea time. This anaysis of network packet is basicay cassification task to separate out attacks and non-attacks. Mosty machine earning tasks are used for this cassification process. The goa of machine earning is discovering, earning, and then adapting to the situations that may change, thus, making improvements in the machine performance. In the intrusion detection fied, the input of the references is appied in the machine earning agorithms so that they can earn the attack patterns. After that, the agorithms are used in the unknown input attacks for performing the actua detection process [1]. Some of the known approaches of machine earning are Artificia Neura Network (ANN), Support Vector Machine (SVM) and Genetic Agorithm (GA) [2]. Out of many avaiabe machine earning approaches SVM is most popuar because of its high prediction accuracy. SVM is based on statistica earning theory, the earning agorithm can sove the sma sampe earning probems. Using characteristics of SVM, we can extract features and cass data, distinguish norma and abnorma data in order to take effective steps to prevent the system from further infringement. When earning sampes set are sma, SVM appears to be best approach avaiabe but when datasets are arger SVM training consumes arge amount of time. This imitation of SVM can affect IDS to great extent [3]. To overcome this imitation use of GPU is proposed which can process SVM training agorithm on different cores simutaneousy. GPUs have become faster and more efficient than traditiona CPUs in certain types of computations because of its abiity to make parae operations [4]. Aso Mutithreading can significanty improve training process by dividing dataset into smaer chunks and processing them individuay. In this paper we propose an agorithm which divides arge datasets into smaer chunks and then processes these smaer chunks in parae. We are trying to minimize time required for SVM training using GPU and mutithreading approach. In this paper we take KDD CUP 1999 dataset which is a arge dataset containing different types of intrusion types and their features. We appy Cascade SVM approach to process it in parae in order to get minimum training time for training SVM. II. RELATED BACKGROUND A. Support Vector Machine Intrusion detection is a cassification probem in essentia. It aims to distinguish norma and abnorma exampes with detection agorithm. Support Vector 1559

2 Machine tries to find an optimization hyper-pane, which can cassify the exampes into two categories in the feature space [4]. Given training exampe: (x 1, y 1),...,(x, y ), x ϵ R d, y ϵ {+,-}, is the number of training exampe, d is the dimension of the exampe feature, X= x 1, x 2,...,x d, y is the cass abe. If y equa to 1, the corresponding exampe is norma, ese if the abe is -1, the corresponding exampe is abnorma. According to the SVM agorithm, for inear separabe probem, there is an optima separating hyper-pane (decision boundary) to cassify two cass exampes. The optima separating hyper-pane can be described as: ω. x + b = 0 Here ω is weight and the norma direction of the hyperpane, b is a bias. The process of soving this hyper-pane, can be considered as, for a given training dataset, seeking weight ω and bias b under some constrains condition to get a minima cost function: min 1 2 ω 2 + C ξ i (1) s.t. y i(ω. x + b) 1 >= 0 Where, C is a penaty coefficient, ξ is a reax variabe, ξ i > 0 [5]. Let a = a 1,a 2,..,a, be the nonnegative Lagrange mutipiers, one for each inequaity constraints in Eq.(1) The soution to Eq.(1) is equa to the soution to the constrained quadratic optimization probem using the Wofe dua theory as: max W(α) = α i s. t. y α i = 0 n 1 2 j=1 α i α j y i y j x i. x j 0 α i C i = 1,2,.. Where α i are the Lagrange mutipiers. If α i > 0, the corresponding exampe is caed support vector (SV). The decision function is: f(x) = sgn{ α y i (x. x i + b ) sgn() is a sign function, if f(x) > 0, the exampe beongs to norma cass, if f(x) < 0, the exampe beongs to abnorma cass and if f(x) = 0, user can cassify it freey. For inear non-separabe probem, the training exampes in input space X can be mapped into a high dimension space H by kerne function: K(x i, x j) = ϕ(x). ϕ(y) then inear cassifier in this high dimension space H can be constructed [5]. A number of CPU-based impementations of the SVM methodoogy are pubicy avaiabe today. LIBSVM, SVM ight, mysvm and TinySVM are ony some exampes. LIBSVM is one of the most compete impementations, providing many options and state-of-theart performance [6]. Therefore, it has become the SVM impementation of choice for a arge part of the research community. Thus, it was chosen as the basis of our work on GPU impementation of parae SVMs. B. Graphics Processor Unit The Graphics Processing Unit (GPU) is a massivey parae device. It is proved to have exceent performance on foating point cacuations, which is even more efficient than muti-core CPUs. The GPU is capabe of processing the arge amounts of data required to perform the cassification and detection tasks very quicky. Many schoars have used GPU to speed up appications. NVIDIA has introduced a programming framework for their GPUs, which is Compute Unified Device Architecture (CUDA) [7]. We can use it to take the advantage of the NVIDIA GPUs architecture. There are some SVM impementations offering many functions incuding different kernes, muticass cassification and cross-vaidation. LIBSVM is one of the famous SVM impementations which provides different SVM types of cassification and regression. It aso contains the most popuar kerne function such as the inear function, the poynomia function and the radia basis function. Many researches are conducted using LIBSVM. It aso provides GPU-acceerated LIBSVM which is a modification of the origina LIBSVM that expoits the CUDA framework to significanty reduce processing time whie producing identica resuts [8]. III. RELATED WORK In this section we provide currenty avaiabe methods to process parae SVM, use of GPU for training SVM to reduce time required. We aso focus on approaches that can be used to modify currenty avaiabe methods to get better resuts. Xueqin Zhang, Yifeng Zhang, Chunhua Gu have worked on GPU impementation of parae SVM [9]. They have proposed agorithm GSVM that makes use of GPU to reduce training time. Sequentia Minima Optimization (SMO) ony aows a subset containing two sampes to be optimized at each step hence training time increases. They have come up with parae SMO agorithm to improve training time. They have used Scatter-Parae-Reduce- Gather (SPRG) parae reduction method to get resuts from different memory bocks of GPU into one fina bock. They have achieved great resuts in terms of speed up and accuracy. Hans Peter Graf, Eric Cosatto, Leon Bottou, Igor Durdanovic, Vadimir Vapnik have proposed an agorithm where Instead of anayzing the whoe training set in one optimization step, the data are spit into subsets and optimized separatey with mutipe SVMs [10]. The partia resuts are combined and fitered again in a next set of SVMs, unti best resuts are achieved. Qi Li, Raied Saman, Vojisav Kecman they propose an inteigent system to sove arge cassification probems based on parae SVM [3]. The system utiizes the atest powerfu GPU device to improve the speed performance of SVM training and predicting phases. A these techniques have great agorithms to paraeize SVM and achieve speed up using GPU. Drawback in these methods is either using ony GPU or ony parae agorithm but not combining these two approaches where combination of these two approaches can ead to even higher speedup. 1560

3 IV. METHODOLOGY Proposed methodoogy uses parae SVM agorithm which divides arge dataset into smaer chunks to be processed in parae. It uses ayered approach towards processing such arge datasets where ony support vectors from previous ayer are passed onto next ayer thus discarding unnecessary data points to get accurate resuts whie buiding SVM mode required for prediction. It uses mutithreading to process data faster on seria machine so even if we don t use GPU it gives us significant speedup. Though we have combined this mutithreading approach with GPU impementation to benefit from both approaches and achieve even higher speedup. Exampe depicted in Figure 1 1. n = 2 therefore D T is divided into 4 subsets namey D T1, D T2, D T3, D T4 and appy it to SVM 1, SVM 2, SVM 3, SVM n subsets of support vectors SVs 1, SVs 2, SVs 3, SVs 4 are generated 3. SVM 1 and SVM 2, SVM 3 and SVM 4 are combined n-1 new resuts and support vector sets are generated SVM 5 and SVM 6 5. Ony one SVs SVs 7 is remaining 6. Continue iterations unti desired resuts are achieved i.e. if ɛ = 50 then SVs ' SVs shoud be greater than 50 to go for another iteration. Figure 1: Schematics of Parae SVM Architecture If we see the Figure 1 there are different eves in SVM training invoved where two SVMs from upper eve pass on their Support Vectors to the next eve and so on up to ast eve where ony support vectors are present and to improve accuracy of SVM we again send these support vectors to first eve and repeat the process unti desired accuracy is obtained. A. Proposed Agorithm of Parae SVM Here, Training Data D T, Support Vector Machine SVM i, Support Vectors Set in current iteration SVs, Support Vector Set in previous iteration SVs ' and Threshod ɛ denotes minimum change in no. of Support Vectors 1. The origina compete training set D T is divided into 2 n subsets where n = 1, 2,.., n 2. Origina SVM training probem is decomposed into 2 n sub-probems. 2 n sub-probems are resoved with parae SVM. 2 n sub-cassifiers and 2 n subsets of support vectors are generated n support vector sets are combined in groups of two for next step 4. New cassifiers are obtained through SVM agorithm with 2 n-1 new training sets. 2 n-1 new resuts and support vector sets are generated. 5. Repeat step 2 unti ony remaining one support vector set SVs. 6. If, SVs ' SVs <= ɛ Terminate Ese Return to step 2 B. To train and test the parae SVM mode generated by the proposed agorithm, a standard data set is required. We have used KDD CUP 99 and (improved version of KDD CUP 99) dataset [12]. Both these datasets are standardized and freey avaiabe on the internet. This dataset contains information about 22 types of network attacks. Each record in this dataset has 41 features and a cass abe depicting type of attack. We have used this dataset since it has a arge number of records, neary haf a miion records, and so we can verify if the proposed agorithm can create an efficient mode and is fast enough for a arge training dataset aso it can fuy utiize potentia of GPU if the dataset is arge enough. V. EXPERIMENT AND RESULTS A. Experiment Environment For Linear SVM we have used LIBSVM version 3.21 which happens to be most widey used SVM ibrary. For making use of GPU to train SVM we have used GPUacceerated LIBSVM which has LIBSVM acceerated with GPU using the CUDA Framework. For parae impementation of Parae agorithm we have taken mutithreading approach where mutipe threads are assigned to different SVMs for training them simutaneousy. In our experiment, the CPU is Inte i3-core E5500 with a frequency of 3.30GHz. The GPU is NVIDIA GeForce GTX 750 ti. The memory capacity is 2 GB, the number of mutiprocessors is 16, the number of cores (stream processors) is 640, mutiprocessors shared memory size is 16KB, and memory bandwidth is 86.4 GB/S. The integrated deveopment environment is Microsoft Visua Studio 2010 with CUDA SDK 6.5. Tabe 1: Training Time (in sec) for different datasets for seria execution CPU Training Time (in sec) KDD CUP KDD CUP 99 10%

4 Tabe 2: Training Time (in sec) for different datasets for varying partitions using parae SVM approach Parae SVM (Mutithreading) (time in sec) Tabe 4: Speed Up for different datasets for varying partitions using parae SVM approach Parae SVM (Mutithreading) (Speed Up) KDD KDD 10% KDD KDD 10% Tabe 3: Training Time (in sec) for different datasets for varying partitions using parae SVM and GPU approach Parae SVM (Mutithreading) + GPU (time in sec) KDD KDD 10% Figure 4: Speed Up for different datasets for varying size of partitions (Parae SVM + GPU) Tabe 5: Speed Up for different datasets for varying partitions using parae SVM and GPU approach Parae SVM (Mutithreading) (Speed Up) KDD Figure 2: Training time (in sec) for different datasets for varying size of partitions (Parae SVM) KDD 10% Figure 5: Speed Up for different datasets for size of partitions (Parae SVM + GPU) varying Figure 3: Training time (in sec) for different datasets for varying size of partitions (Parae SVM + GPU) 1562

5 B. Anaysis of Resuts Figure 2 indicates reationship between No. of Partitions and Time eapsed when we use ony Parae SVM agorithm(mutithreading), we can observe that time required for arger dataset is significanty greater than that of smaer dataset but as the no. of partitions grow time required for training SVM has no significant difference regardess of the size of dataset. Figure 3 is simiar to that of Figure 2 ony itte ess time is required for training because GPU pays important roe in making training faster. Figure 4 and Figure 5 indicate reationship between No. of Partitions and Speed Up, we can easiy see that speed up for two datasets which contains arge amount of records is better as No. of partitions grow but for smaest and argest dataset speedup is esser. Accuracy remains amost same for a of the datasets and aso for a number of partitions. VI. CONCLUSION In this proposed approach we have appied Parae SVM agorithm aong with GPU acceeration for training SVM for Intrusion Detection. Both these approaches provide great resuts. Parae SVM agorithm uses Mutithreading approach which trains different SVMs simutaneousy. Whie training Parae SVM and its GPU impementation we found that for moderatey arger datasets time required for training decreases as No. of partitions are increased. Speed Up achieved in comparison with sequentia SVM can reach to higher eves. In future this technique can be used for various appications where cassification is major task. REFERENCES [1] V. T. Goh, J. Zimmermann, and M. Looi, "Towards intrusion detection for encrypted networks," in Avaiabiity, Reiabiity and Security, ARES'09. Internationa Conference on, 2009, pp [2] S. Mukkamaa, G. Janoski and A.Sung, Intrusion detection: support vector machines and neura networks, In proceedings of the IEEE Internationa Joint Conference on Neura Networks (ANNIE), St. Louis, MO, pp , [3] T. Joachims, Making arge-scae SVM earning practica, In B. Schokopf, C. J. C. B-wges, and A. J. Smoa eds. Advances in Keme Methods Support Vector Leaming, Cambridge, MA: MIT press, [4] S.Theodoridis and K.Koutroumbas, Pattern Recognition (Second Edition), Academic Press, USA, [5] Edgar E. Osuna, Robert Freund and Federico Girosi, Support Vector Machines: Training and Appications, Massachusetts Institute of Technoogy, Artificia Inteigence Laboratory, A.I. Memo No March, C.B.C.L Paper No. 144, [6] C. C. Chang and C. J. Lin, LIBSVM: a ibrary for support vector machines, ACM Transactions on Inteigent Systems and Technoogy (TIST), vo. 2, no. 3, 2011, pp. 27. [7] NVIDIA, CUDA Compute Unified Device Architecture Programming Guide, June [8] Athanasopouos, A. Dimou, V. Mezaris, I. Kompatsiaris, "GPU Acceeration for Support Vector Machines", Proc. 12th Internationa Workshop on Image Anaysis for Mutimedia Interactive Services (WIAMIS 2011), Deft, The Netherands, Apri [9] X. Zhang, Y. Zhang, C. Gu., GPU Impementation of Parae Support Vector Machine Agorithm with Appications to Intruder Detection, JOURNAL OF COMPUTERS, VOL. 9, NO. 5, MAY 2014 [10] Graf H. P., Cosatto E., Bottou L., Durdanovic I., Vapnik V.; "Parae Support Vector Machines: The Cascade SVM" in Advances in Neura Information Processing Systems no. 17, , [11] Q. Li, R. Saman, V. Kecman, An Inteigent System for Acceerating Parae SVM Cassification Probems on Large s Using GPU. [12] KDD Authors Profie: Sudarshan Hiray is student pursuing Master of Technoogy, in Computer Science & Engineering from Department of Computer Engineering at Vishwakarma Institute of Technoogy, Pune. Noshir Tarapore is Assistant Professor at Department of Computer Engineering in Vishwakarma Institute of Technoogy, Pune. 1563