Anomaly Detection in Network Traffic: A Statistical Approach

Transcription

1 16 Anomaly Detection in Network Traffic: A Statistical Approach Manmeet Kaur Marhas, M.tech Scholar, Dept. of CSE, CMJ University, Shilong, Meghalaya, India Anup Bhange, Asst.Prof, Dept. of IT, KDK College of Eng, Nagpur Piyush Ajankar, Asst.Prof, Dept. of IT, KDK College of Eng, Nagpur ABSTRACT A global Internet usage enlarge rate of 380% superior than the period from 2000, the year of the dot-com bubble burst, until present select that Internet technology has become a stand of our daily life. In the similar period, cyber-crime has seen an unbelievable to facilitate to create sophisticated protection device for computers and networks a complete necessity. Firewalls as the major protection of the final decade do not give adequate protection anymore. This information has given increase to the expansion of intrusion detection and prevention systems. usual intrusion detection systems are hasty in the sense that they use a set of signatures, which lift at the same rate as new technique are exposed, to identify malicious traffic patterns. Anomaly detection systems are one more branch of intrusion detection systems that operate more proactively. They get a model of the normal system presentation and issue alerts whenever the behavior changes; making a suitable assumption that such changes are frequently caused by malicious or disrupting events. Anomaly detection has been a ground of comprehensive research over the last years as it pretense several challenging problems. In this Paper we present a statistical approach to analysis the allocation of network traffic to recognize the normal network traffic behavior. The Research proposals in anomaly detection naturally follow a fourstage approach, in which the initial three stages define the detection method, while the final stage is dedicated to authenticate the approach method to detect anomalies in network traffic, based on a α -stable model and statistical hypothesis testing. Here we focus on detecting and preventing two anomaly types, namely floods and flash-crowd.here we use NS2 simulator to calculate result. 1.1 INTRODUCTION TO ANOMALY: Network traffic amount provide fundamental traffic characteristics, provisions information for the control of the network, permit modeling and give a probability to develop and plan the use of network resources. It also permits developers to manage the brilliance of network service operations. While network traffic dimension is a well-known and appropriate area, a general method for detecting anomalies in network traffic is an important, unsolved problem (Denning 1986). Anomaly detection aspires at finding the presence of anomalous patterns in network traffic. Usual detection of such outline can provide network administrators with an extra source of information to identify network behaviour or finding the root cause of network faults. [1] 1.2 NATURE OF ANOMALY: A momentous characteristic of an anomaly detection method is the temperament of the preferred anomalies. Anomaly can be classified into following three categories: [2] POINT ANOMALY: If a distinct data occurrence can be calculated as anomalous with admiration to the rest of data, then the case is termed a point anomaly. This is the easiest type of anomaly and is the center of preponderance of examine on anomaly detection RELATIVE ANOMALY: If an information occurrence is anomalous in a precise context, but not or else, then it is characterizing a related anomaly. The notion of a context is entice by the structure in the data set and has to be count as a part of the problem formulation CONTEXTUAL ATTRIBUTES: The contextual attributes are use to control the context (or neighborhood) for that example. For example, in spatial data sets, the longitude and autonomy of a location are the related attributes. In time-series data, time is a contextual attribute that decide the position of an instance on the entire sequence BEHAVIORAL ATTRIBUTES: The behavioral attributes describe the no related characteristics of an instance. For example, in a spatial data set recitation the average rainfall of the entire world, the amount of rainfall at any location is a behavioral attribute. The anomalous performance is indomitable using the values for the behavioral attributes within a specific context. Data instance strength is a contextual anomaly in a given context, but an identical data instance (in terms of behavioral

2 17 attributes) could be measured normal in a different context. This property is key in recognize contextual and behavioral attributes for a contextual anomaly detection technique COLLECTIVE ANOMALY: If a collection of related data instances is anomalous with respect to the entire data set, it is termed a collective anomaly. The individual data instances in a collective anomaly may not be anomalies by themselves, but their occurrence together as a collection is anomalous. The highlighted sequence of events (buffer-overflow, ssh, ftp) correspond to a typical Web-based attack by a remote machine followed by copying of data from the host computer to a remote destination via ftp. It should be noted that this collection of events is an anomaly, but the individual events are not anomalies when they occur in other locations in the sequence. Intrusion detection comes in two flavors: Deterministic systems that rely on matching received traffic with predefined patterns of malicious traffic and statistical systems that derive models of system properties under normal conditions and compare predictions based on them with actual measurements. Research proposal in anomaly detection typically follow a four-stage process, in which the first three phases describe the detection method, while the last phase is enthusiastic to legalize the approach. So, in the first phase, traffic Information are collected from the network (simulated Information by using NS2) (Information collection). Second, Information are analyzed to mine its most relevant features (Information analysis). Third, traffic is classified as normal1 or abnormal (Conclusion) and fourth, the whole approach is validated with various types of traffic anomalies (Justification). 1) Information Collection. 2) Information analysis (feature extraction). 3) Conclusion (classifying normal1 vs. anomalous traffic). 4) Rationalization. Information Collection is classically carried out by polling one or more routers periodically, so that traffic data are collected and stored for posterior analysis in the second stage. Some authors sample data at the packet level, gathering information from headers, latencies, etc., while others prefer to use aggregated traffic as the source of information, often through the use of the Simple Network Management Protocol (SNMP). Sampling data at the packet level provides more information, but at the cost of a higher computational load and dedicated hardware must be employed. Aggregated traffic, on the other hand, gives less information from which to decide for the presence or absence of anomalies, but is a simpler approach and does not need any special hardware. In the Information analysis phase, several techniques can be applied to extract interesting features from present traffic. Some of them contain information theory, wavelets, Information-based measurements, and statistical models of these techniques, the use of statistical models as a way to mine significant features for Information analysis has been found to be very promising, since they concur to for a robust analysis even with small sample sizes (provided that the model is sufficient for real data). Several approaches have been used in the Conclusion stage as well. Classification methods based on neural networks [3], statistical tests, information theory to cite a few, can be found in anomaly detection literature. There seems to be a common point in all of them, though. The conclusion phase bases its decisions on the existence of a reference traffic window, which permit the classification method to assess whether the current traffic window is normal (i.e., it is sufficiently similar to the reference window) or abnormal (i.e., significantly different from the reference window). [4] How the reference window is chosen not only has an impact on the final normal versus abnormal classification rate, but it also decides the exact definition of a traffic anomaly. In the Justification stage, researchers give authority measures about the detection ability of their method according to chosen criterion, which is typically the detection rate in terms of false positives and false negatives (i.e., the fraction of normal traffic patterns incorrectly classified as anomalous and the fraction of anomalous traffic patterns incorrectly classified as normal, respectively). In this Paper, we proposed an anomaly detection and Prevention method based on α-stable distributions which does not need network administrators choose reference traffic windows and it is able to detect and prevent flood and flash crowd anomalies regardless of the presence or absence of abrupt changes in network traffic. 2. LITERATURE SURVEY 2.1 ANOMALY DETECTION: Detecting uncharacteristic traffic is a research theme that had recently conventional a lot of attention. We assembly this issue into two area; network intrusion detection and Internet traffic anomaly detection. The aim of intrusion detection is to watch a network from remote threats, thus, the detection method is examine the traffic at the edge of the protected network where total flows and packet load are usually available. In distinguish, Internet traffic anomaly detection aims at categorize anomalous traffic that is transiting in the core of the Internet where the check traffic is asymmetric due to routing policies, thus, flows are incomplete. Our

3 18 work is dedicated exclusively to Internet traffic anomaly detection, thus, in this paper anomaly detection refers only to this specific domain. For the last decade researchers have taken a strong interest in anomaly detection and proposed different detection methods that are basically monitoring traffic characteristics and discerning outliers. We differentiate different categories of anomaly detection method; the methods monitoring the traffic volume and those monitoring the distribution of traffic features. 2.2 VOLUME BASED ANOMALY DETECTORS: Volume based approaches are watch the number of bytes, packets or flows transmit over time and aims at detecting abnormal variances that represent rude usages of network resources or resource failures. Different methods have been proposed to successfully identify local and global traffic volume variances that stand for respectively short and long lasting anomalies. For example, Barford et al. [5] proposed a method depend on wavelet at scrutinize the traffic volume at different time span. Their technique makes employ of the wavelet analysis to analyze the traffic into three distinct signals presenting local, normal and global variances of the traffic. The crumbling signals are analyzed by a detection process that finds the irregularities and reports the period of time they occur. Since the three signals represent the traffic at different time scales this approach is able to report short and long lasting anomalies. Nevertheless, as the whole traffic is collected into a single signal diagnosing the detected anomalies is Challenging and anomalous flows or IP addresses are left unknown. Soule et al. [6] discuss another detection method that also scrutinizes the traffic volume in matrices. The main idea fundamental of their approach is to represent in a matrix the traffic between nodes of a large network and remove the normal traffic using a Kalman filter. The residual traffic is analyzed with a statistical method that detects anomalous traffic and reports the pair of nodes affected by the anomalous traffic. These volume-based anomaly detectors effectively report volume anomalies while their false positive rate is low. 2.3 TRAFFIC FEATURES BASED ANOMALY DETECTORS: In order to overcome the disadvantage of volume-based anomaly detectors researchers proposed to purify the traffic features that are examine by the anomaly detectors. For example, as numerous anomalies cause abnormal utilization of ports or addresses, examine the distribution of the traffic into the port and address spaces permits to identify anomalous traffic that is not reported by volume-based detectors (e.g., port scan). However, due to the size of analyzed traffic inspecting detailed traffic features is costly and imposes researchers to intricate effective traffic aggregation schemes the main challenge in aggregating network traffic is the tradeoff between preserve a crisp represent ion of the traffic and conserves its motivating characteristics. We differentiate four groups of detection technique in regard to their traffic aggregation scheme; namely, detection methods aggregating the traffic in a single signal, those aggregating the traffic in traffic matrices, methods aggregating traffic in histograms, and the other methods SIGNALS: A signal provides an intuitive and coarse view of the traffic by representing the time evolution of a single characteristic of the traffic. Contrarily to volume based method, here the analyzed signals are obtained from fine-grained measures providing details traffic characteristics. The measure that probably has received the most attention in this research domain is the entropy (i.e., Shannon entropy) TRAFFIC MATRIX: A traffic matrix represents a time series of flows aggregated according to the ingress and egress routers they passed to transit on the network, also called, origin-destination flow (or OD flow). The effectiveness of aggregating traffic into traffic matrices have been validated in a comparative study. Perhaps the most famous anomaly detection method using traffic matrices is the PCA-based detector firstly proposed. Similarly to their volume-based anomaly detector they proposed an anomaly detector relying on PCA but analyzing the distribution of traffic features HISTROGRAM: In Information the distributions of data is commonly studied in the form of histograms. Several works using histograms have been carried out in anomaly detection, for example Dewaele et al. [7] proposed to model flows in histograms and evaluate their geometry using the Gamma distribution model. The normal behavior of the traffic is computed from the distributions of the traffic majority and outliers are reported as anomalous. 3. PROPOSED WORK AND METHODOLOGY The methodical work that is followed to differentiate network traffic and to get anomaly information connected with the traffic examines. The method occupies the steps followed to produce anomaly result. The steps start with examining of the simulated data by

4 19 using (NS2) and ends with a graph representing the abnormal traffic and normal traffic in a time interval. In research proposed method to detect and prevent the anomaly in network traffic, by using the statistical approach and α-stable model. 3.1 STATISTICAL ANOMALY DETECTION: The potential to detect unknown attacks is the strength of statistical anomaly detection systems. Anomaly detection systems derive a model of the normal behavior of a network or system and detect divergence from this normal profile. This enables them to detect known and unknown malicious activities likewise. The normal profile has been derived based on different Information such as system calls on a single host, payload byte patterns in received traffic, or volume and entropy Information over the traffic in a whole network. 3.2 STATISTICAL ANOMALY DETECTION ALGORITHM: STEP: 1 STEP: 2 STEP: 3 Node Initialization I =1 to 10 Initialize Threshold =value Transfer Packets in Sequential Node For I=1 to 10 Xmt (node [i], node (i+1)) If (xmt (node (i), node (i+1)!) Display Anomaly Detected Then, If (Threshold==n) (a) Count the Packet on each Node= Counter Threshold Counter (b) DDoS attack Detected i.e. Flood anomaly detected Else, Display No anomaly found Packet Received (node (i), node (i+1) (c) Display Counter on Node [i] If (i==10) Xmt (node [i-(i-1)], node [i]) Display Flash Anomaly Detected, go to call (b)

5 20 DESIGN OF NETWORK: In this section, Paper presents the Design of our research work. As mentioned we use the NS2 to calculate the result. Here we focus on to detecting and flood and flash crowd anomaly in wireless network. Here we consider the 10 nodes in network and sending the packet at regular interval of time and finding out the behavior of network and providing the proper threshold to calculate the flood anomaly in network [5] P. Barford, J. Kline, D. Plonka, and A. Ron. A signal analysis of network traffic anomalies. IMW '02, pages 71{82, (Cited on pages 11, 25, 32, 57, 91 and 95.) [6] A. Soule, K. Salamatian, and N. Taft. Combining filtering and statistical methods for anomaly detection. IMC '05, pages 331{344, (Cited on page 12.) [7] G. Dewaele, K. Fukuda, P. Borgnat, P. Abry, and K. Cho. Extracting hidden anomalies using sketch and non gaussian multiresolution statistical detection procedures. SIGCOMM LSAD '07, pages 145{152, (Cited on pages 3, 5, 13, 17, 22, 25, 32, 35, 40, 48, 57, 91 and 104. Figure 4.1 Nam output showing nodes in wireless networks 5.CONCLUSION: This Paper Presents the idea about the anomaly Detection in network Traffic, and also discusses statistical approach for anomaly Detection in Network Traffic. Ns2 is used for Design of Network and calculating the simulating Result. 6.REFERENCES: [1] D. E. Denning, An intrusion detection model, in Seventh IEEE Symposium on Security and Privacy, 1987, pp [2] Y. Gu, A. McCallum, and D. Towsley, Detecting Anomalies in Network Traffic Using Maximum Entropy Estimation, Proc. Internet Measurement Conf., Oct [3] M. Ramadas, S. Ostermann, and B. Tjaden, Detecting Anomalous Network Traffic with Self- Organizing Maps, Proc. Sixth Int l Symp. Recent Advances in Intrusion Detection, pp , [4] J.Brutlag, Aberrant Behavior Detection in Time Series for Network Monitoring, Proc. USENIX 14th System Administration Conf. (LISA), pp , Dec