A Bayes Learning-based Anomaly Detection Approach in Large-scale Networks. Wei-song HE a*

Transcription

1 17 nd International Conference on Computer Science and Technology (CST 17) ISBN: A Bayes Learng-based Anomaly Detection Approach Large-scale Networks Wei-song HE a* Department of Electronic Information Engeerg, Liangjiang International College, Chongqg University of Technology a heweisong@gmail.com *Correspondg author Keywords: Anomaly detection, Network traffic, Bayes learng, Feature extraction, Detection algorithm. Abstract. With quick developments of new application patterns and network technologies, network anomalies have an important impact on network operations. How to accurately detect network anomalies has become the hot topic current communication networks. This paper proposes a new detection approach to diagnose the anomaly network traffic. Firstly, we use the Bayes learng theory to describe network traffic properties. By the learng process, normal network traffic can correctly be modeled. Secondly, the feature extraction is used to differentiate abnormal network traffic from normal huge traffic. Thirdly, the detail detection algorithm is presented to fd the anomalous component network traffic. Fally, we carry out the detailed simulation experiments. Simulation results dicate that our approach is effective. Introduction With quick advance of new communication technologies and application patterns, new network anomalies have the creasg appearance current networks. Particularly, network traffic has quickly creased communication networks. Network anomalies have impact on network performance and normal operations [1-]. How to effectively detect and fd abnormal components networks has become a larger challenge [3-4]. Moreover, network anomalies imply users' and network devices' abnormal behaviors their operation process. These behaviors turn degrade network performance and user experience. Accordgly, this further decreases the maximum profit of network providers and adds user costs. Specially, anomalous network traffic has produced the heavy damage to current networks. Therefore, anomaly detections for network traffic have become an important topic current academic and dustries [-8]. Some anomaly detection technologies have been proposed to diagnose network traffic. The transformation doma-based analysis methods, such as wavelet analysis, time doma transformation, frequency transformation, and so on, are used to fd anomalous network traffic [, ]. These methods achieved the fairly accurate detections for abnormal network traffic. Information metrics [3] and parametric methods [9] were aloes exploited to detect network traffic anomalies. By such this way, network traffic can be characterized and extracted different metrics or mode functions. Additionally, the parameter-based detection method could diagnose the abnormal part network traffic [1-11]. Periodicity features were characterized and modeled to fd out traffic 4

2 anomalies communication networks [7]. Accordgly, a period signal model could accurately to detection and diagnose abnormal network traffic. Additionally, the detection method for multimedia traffic was also presented to guarantee network performance [8]. A new detection approach was proposed to fd out abnormal network events [1-1]. To effectively detect abnormal network traffic, the spectral kurtosis theory was used to diagnose abnormal network traffic [16]. Dynamic detection technologies were presented to fd and identify abnormal dynamic traffic flow communication networks [4]. The formation theory analysis method was effectively used to extract anomalous network traffic components [17]. However, these approaches still hold the larger detection errors. This motivates us to research the new method to detect network traffic anomalies. This paper presents a new method to detect the anomaly parts network traffic. Our approach uses the Bayes learng theory to describe the normal network traffic and build the correspondg normal traffic model. Firstly, we regard network traffic as a dynamic Bayes random process. The correspondg Bayes theory model is used to characterize network traffic. Secondly, to accurately model network traffic, we describe the network traffic modelg problem as a Bayes learng process. By the Bayes learng, we can effectively capture the dynamic nature normal network traffic. In such a case, we can accurately build the correct network traffic model for normal situation. Accordgly, we simplify the complex network traffic modelg issue to the simple Bayes learng process. To describe network traffic correlations time doma, we transform network traffic sequence to a dynamic matrix accordg to the common destation of traffic flows. Then each row of dynamic traffic is used to perform the Bayes learng the distribution way. The correspondg extractg process is carried out the Bayes learng process. Thirdly, an appropriate detection threshold identification method is presented to fd out the abnormal network traffic. At the same time, we also propose an anomaly detection algorithm to perform the accurate and effective identification of anomalous network traffic. Simulation results show that our approach is effective and feasible The remag of this paper is arranged as follows. We proposed our detection method Section. In Section 3, we present the simulation results and analysis. Fally, we conclude our work Section 4. Problem Statement For any network traffic, we can refer it as a random process the time doma, namely x = { x(1), x(),...} where x() i (wherei = 1,,... ) represents the value of traffic flow x at time sloti. Network traffic x follows the normal process distribution as follows: 1 ( x μ) PX ( = x) = exp( ) (1) πδ δ where μ and δ are the mean value and variance of the normal process X. Generally, network traffic x() t at time slot t is correlated with network traffic x( z ) (where z = t 1, t,..., t n) before time slot t. In such a case, the Bayes theory is used to describe and model network traffic at time slot k as follows: PXk ( ( ), Xk ( 1), Xk ( ),...) PX ˆ( ) = PXk ( ( ) Xk ( 1), Xk ( ),...) = PXk ( ( 1), Xk ( ),...) ()

3 Without loss of generality, we only take to account the correlations of network traffic of - before with the current network traffic. This is reasonable because network traffic holds strongest correlations with network traffic of several closest to current time slot. In contrast, selectg network traffic of the - is appropriate and computationally simple. Therefore, Equation () can be converted to: PXk ( ( ), Xk ( 1), Xk ( ),..., Xk ( )) PXk ( ( ) Xk ( 1), Xk ( ),..., Xk ( ),) = PXk ( ( 1), Xk ( ),..., Xk ( )) Equation (3) can effectively describe the dynamic traffic of each network flow. However, different network flows hold the larger correlation space. Particularly, network traffic of flows with common destations exhibits larger space correlations. Thereby, we arrange network traffic with common destations to a row to construct an formation matrix about network traffic. Network traffic of all flows the network can be denoted as u= { u( t) i, j = 1,,..., n} and n is an teger. Then we atta a new matrix as follows: u11( t), u1( t),..., u1n ( t) u ( t), u ( t),..., u ( t)... un1( t), un( t),..., unn( t) 1 n = n n = Ut () { u()} t Each low Equation (4) holds the common destation. The traffic of these flows has the stronger space correlations, while they are dependent time. Therefore, we jotly analyze the traffic of each low the matrix Equation (4). The followg equation can be obtaed: uˆ ( t) = E( u( t)) [ ˆ( ( ))... ˆ( ( ))... ˆ = P u t P u t P( u ( t))] du ( t)... du ( t) () i i1 ik i1 k j uˆ () t Equation () is a function with respect to μ and δ for network flows from f i1 to f. Equation () can be further denoted as: uˆ ( t) = E( ui ( t) μi1, μi,..., μ, δi1, δi,..., δ) [ Pˆ ( u (), t μ, δ )... Pˆ ( u (), t μ, δ )... Pˆ ( u (), t μ, δ )] du ()... t du () t = k j i1 i1 i1 ik i i i1 Accordg to Equation (6), by the Bayes learng process N network traffic samples, we can decide the value of μi1, μi,..., μ, δi1, δi,..., δ. In such a case, we build the model about normal network traffic. Accordgly, we construct the below detection process for the traffic of each network flow u () t as follows: Δ u () t = u () t uˆ () t (7) Compute the variance of Δ u () t Equation (6) as follows: Δ d = var( Δ u ( t)) (8) We use the followg anomaly identification method: (3) (4) (6) 6

4 u () t is anomalous, if Δ u () t >Δd u () t is not anomalous, if Δu () t Δd Accordg to Equation (9), we can perform the detection process for network traffic time. Equations ()-(9) show our detection approach for the dynamic network traffic. Now, we propose our detection algorithm. The steps of our algorithm are as follows: Step 1: Give network traffic u= { u ( t) i, j = 1,..., n}, the number N of traffic samples. Step : Accordg to Equation (4), construct the formation matrix Ut () about network traffic. Let j = 1. Step 3: For row i the formation matrix Ut, () accordg to Equations (1)-(3) and the Bayes learng method, use the N network traffic samples to decide the parameters μ, δ of the traffic of network flow u () t. Step 4: Build the model about the traffic of network flow u () t usg the parameters μ, δ, namely attag the followg equation: 1 ( u( t) μ ) PU ( ( t) = u( t)) = exp( ) (1) πδ δ Step : If i < n, let j = j+ 1 and go to back Step 3. Step 6: Build the below jot distribution density function of u( t) = { u ( t),..., u ( t)} : i i1 (9) 1 x 16 (a) Bayes learng results 1 x 16 (a) background traffic (b) Bayes learng relative errors x 16 (b) anormal traffic Figure 1. Bayes learng results and relative errors for network traffic Figure. Network traffic without and with anomalies. pu ( ( t)) = N( μ, μ,..., μ δ, δ,..., δ ) (11) i i1 i, i1 i Step 7: Accordg to Equation (6), atta the estimation uˆ () t of network traffic u() t. 7

5 Step 8: Through Equation (7), compute the deviation Δ u () t of network traffic u() t. Step 9: By Equation (8), calculate the variance d Δ for network traffic u () t. Step 1: Then we use Equation (9) to perform the makg-decision process. If Δ u ( t) > Δ d, network traffic is anomalous. Step 11: If the detection process is fished, save the detection results to the file and exit. Or otherwise go back to Step 7 to contue the detection process. Simulation Result and Analysis Now we carry out the simulations to validate our detection approach for network traffic anomalies. In our test process, we ject anomalous network traffic to normal background network traffic at four different of 4, 8, 1, and 14 with the duration of, respectively. We run 1 times simulation to atta the average detection performance for our detection method. The detection threshold Equation (8) is established accordg to our detection algorithm. In our simulations, we discuss the model performance of network traffic built accordg to the Bayes learng process, and anomaly detection ability. We use the data from the real network to carry out our simulation process. In our experiment, we select the first 3 samples to build our traffic model. We construct a formation matrix about network traffic to carry out jot analysis process. Figure 1 shows the Bayes learng results and relative errors for network traffic. Figure 1 (a) shows that our method can effectively estimate network traffic usg the Bayes learng process, where the estimations follow the dynamic change of the real network traffic. Figure 1 (b) denotes that the estimation results of our method hold the small estimation errors. This dicates that our method can capture the network traffic nature and change. More importantly, we only use the first 3 to build the model about network traffic, while we can atta the accurate estimations. This states that our method can effectively perform the feature characterization of network traffic. Figure plots the network traffic without anomalies and with anomalies. From Figure, we are able to fd that there are no difference between normal network traffic and abnormal network traffic. The anomalous network traffic components hide the larger normal network traffic, which is very difficult to fd and diagnose. 1 x 16 (a) Bayes learng results for network traffic with anomalies x 1 7 (b) Bayes learng errors for network traffic with anomalies Figure 3. Bayes learng results for network traffic with anomalies. 8

6 1 (a) Smoothg deviations (b) Detectg anomaly Figure 4. Deviation smoothg and anomaly detection. Figure 3 plots the Bayes learng results and relative errors for network traffic with anomalies. Different Figure 1, we can fd that for network traffic with anomalies, our method can effectively exhibit the larger deviations for the anomalous network traffic. Figure 4 shows the traffic anomaly detection usg our method. To detection the anomalies, we smooth the deviations of network traffic, which can effectively embody network traffic deviation from normal components. Then we exploit the appropriate detection threshold to fd out the anomalous network traffic. Figure 4 shows that our method can effectively detect diagnose the anomalous traffic. Conclusions This paper proposes a new detection approach to detect the anomaly components network traffic, usg the Bayes learng theory. By Bayes learng, we can describe the normal network traffic and build the correspondg normal traffic model. Through Bayes learng, we describe the network traffic modelg problem as a Bayes learng process. Then we can effectively capture the dynamic nature normal network traffic. We transform network traffic sequence to a dynamic matrix accordg to the common destation of traffic flows so that we can describe network traffic correlations time doma. Then each row of dynamic traffic is used to perform the Bayes learng the distribution way. The correspondg extractg process is carried out the Bayes learng process. We also propose an anomaly detection algorithm to perform the accurate anomalous detection for network traffic. Fally, simulations show that our method is feasible and effective for detectg anomalous components network traffic. Acknowledgement This paper is supported by the Scientific and Technological Research Program of Chongqg Municipal Education Commission (Grant No. KJ18). References [1] M. Ahmed, A. N. Mahmood, J. Hu. A survey of network anomaly detection techniques. Journal of Network and Computer Applications, 16, 6(1):

7 [] D. Jiang, Z. Xu, P. Zhang, et al. A transform doma-based anomaly detection approach to network-wide traffic. Journal of Network and Computer Applications, 14, 4(): [3] Y. Xiang, K. Li, and W. Zhou. "Low-rate DDoS attacks detection and trace back by usg new formation metrics," IEEE Transactions on Information Forensics and Security, 11, 6(): [4] G. Thatte, U. Mitra, and J. Heidemann. "Parametric methods for anomaly detection aggregate traffic," IEEE/ACM Transactions on Networkg, vol. 19, no., pp. 1-, 11. [] D. Jiang, Y. Han, Z. Xu, et al, A time-frequency detectg method for network traffic anomalies, Proc. ICCP 1, 1, pp [6] W. Xiong, H. Hu, N. Xiong, et al. Anomaly secure detection methods by analyzg dynamic characteristics of the network traffic cloud communications. Information Sciences, 14, 8(14): [7] T. Akgül, S. Baykut, M. E. Kantarci, et al. Periodicity-based anomalies self-similar network traffic flow measurements. IEEE Transactions on Instrumentation and Measurement, 11, 6(4): [8] D. Jiang, Z. Yuan, P. Zhang, et al. A traffic anomaly detection approach communication networks for applications of multimedia medical devices. Multimedia Tools and Applications, 16, onle available, pp. 1-. [9] G. Thatte, U. Mitra, J. Heidemann. Parametric methods for anomaly detection aggregate traffic. IEEE Transactions on Networkg, 11, 19(): 1-. [1] D. Jiang, W. Q, L. Nie, et al. Time-frequency detection algorithm of network traffic anomalies, Proc. ICIIM'1, 1, pp [11] G. Thatte, U. Mitra, and J. Heidemann. "Parametric methods for anomaly detection aggregate traffic," IEEE/ACM Transactions on Networkg, vol. 19, no., pp. 1-, 11. [1] B. Eriksson, P. Barford, R. Bowden, et al. Basisdetect: A model-based network event detection framework, Proc. IMC, pp , 1. [13] D. Jiang, C. Yao, Z. Xu, et al. Multi-scale anomaly detection for high-speed network traffic. Transactions on Emergg Telecommunications Technologies, 1, 6(3): [14] M. H. Bhuyan, D. K. Bhattacharyya, J. K. Kalita. A multi-step outlier-based anomaly detection approach to network-wide traffic. Information Sciences, 16, 348(): [1] J. Kevric, S. Jukic, A. Subasi. An effective combg classifier approach usg tree algorithms for network trusion detection. Neural Computg and Applications, 16, onle available. [16] D. Jiang, C. Yao, W. Zhang, et al. A detection algorithm to anomaly network traffic based on spectral kurtosis analysis, Proc. ITSim 13, 13, pp [17] P. Tune, D. Veitch. Samplg vs sketchg: An formation theoretic comparison, Proc. of INFOCOM, 11, pp: