Outline of Presentation. Data Mining and Intrusion Detection. Intrusion Detection Systems(IDS) IDS Where, How and Why? Issues and Problems

Transcription

1 Data Mining and Intrusion Detection A short presentation focusing on cost-sensitive issues submitted in partial fulfillment of CSE 8331 Data Mining, Spring Abhishek D. Sanwal Outline of Presentation Intrusion Detection Systems Where? How? Why? Issues and Problems Big Issue Cost Cost Sensitive Modeling Past, present and future Cost Factors Cost Metrics Cost Models Reducing Costs Implementation overview Summary Conclusions Future Directions Project1: Real-time DM IDS Analysis Architecture Q & A? 2 Intrusion Detection Systems(IDS) Why do we need them despite all other security measures? If we can make a network secure enough using a firewall and other precautions why is it not enough? Because A firewall is simply a fence around the network, with a couple of well-chosen gates A fence has no capability in detecting somebody trying to break-in (such as digging a hole/tunnel under it) Nor does a fence know if somebody coming through the gate is supposed to be allowed in it An IDS is analogous to having security guards, surveillance, and countermeasures, in addition to barbed wire fencing, high- tough to scale walls etc. Firewall Active Filtering. IDS Passive Monitoring IDS Where, How and Why? The sensors of an IDS are intuitively placed at locations close to the perimeter (though a secure perimeter it may be breached) adjoining the perimeter; inside and outside the enclave IDS could have Single Sensor Distributed Sensors (with) Central Processing Distributed Processing Co-related Decision Making 3 4 Issues and Problems Deployment Geographical Dispersal Installation Setup and customization Management Upgrading Modification as per change in network architecture Maintaining the sensors Data overload Changes in the Security World Integration Disparate Data Formats Lack of single cross-reference between data sources Time synchronization of sensors 5 What do we need to add to an IDS? Universal Data format for IDS data IDEF (Intrusion Detection Exchange Format) CIDF (Common Intrusion Detection Framework) (Intrusion Detection Working Group) Common cross-reference for different IDS for interoperability Time synchronization built into IDS Data reduction for analysis Reduce false alarms Updating of attack signatures on timely basis Automated encoding of intrusions and traffic into Models Previous systems - all knowledge-engineered 6 1

2 Tackling some Issues for now Cost Sensitive IDS Maybe later in the paper but not this presentation DM with Unlabeled Data DM with co-relation framework Detecting Novel Attacks Cost Sensitive Modeling for IDS-Why? Need to evaluate technical effectiveness IDSs fail to detect new and stealthy attacks Real-Time IDS exposed to Overload Attacks Intended Intrusion Successful Overload of Intrusion Reports forces admin to raise detection and response thresholds real attacks get ignored / overlooked 7 8 What is needed? Cost Effectiveness of the IDS As any investor in a business would say R O I (Return on Investment ) Motivation: To prevent losses Cost Sensitive Modeling - Why not in use? Complexity of IDS-cost analysis on a low priority Site-specific cost factors Addition of features Improvement of detection Rules Not all factors reduce to Discrete dollars Units of measurement Probabilities Solution: Qualitative Analysis Measure relative magnitudes of Cost Factors But how? Where do we go now? 9 10 there is hope Eureka! IDS Framework proposed by Wenke Lee [PhD Thesis] Automate development and lower development cost DM Algorithms are applied towards Computing Activity Patterns Extract Predictive Features Generate Detection Rules (using m/c Learning Algorithms) It will help in reducing the complexity in adding cost sensitive modeling due the adaptability of the framework 11 Cost Sensitive Modeling Cost Factors Cost Metrics Cost Models Cost Breakdown Damage Cost Response Cost Operational Cost The above costs are Site specific and depend on Security policies Information Assets Risk Factors 12 2

3 To produce meaningful cost metrics? Attack Type U2R R2L DOS PROBE Essentially the Attack Taxonomy group intrusions into types Categorized / analyzed from different perspectives Description User->Illegal Root Outside->Inside DOS of Target Target Info Sub- Category Local Remote Single Multiple Crashing Consumption Simple Stealth Cause -More complex attacks Various Buffer Overflow Attacks 1-event Multiple Events, hosts, days etc. Malicious Event (e.g. teardrop) Repeated Malicious Events (e.g.synflood) Probe events over a short time(e.g.portscan) Probe events distributed sparsely over a long time window (e.g.slow stealthy portscan) 13 DCost RCost Highest Lowest Damage Cost Criticality Importance or Value depending on functional role in the organization or Relative cost of replacement/unavailability/disclosure Lethality Degree of Damage that could potentially be caused by the attack Progress Is a measure of how successful the attack is in achieving its goal i.e. percentage of max. DCost that should be accounted for For each attack category Define a relativit lethality scale Use it as the base damage cost (based) Hence, DCost = Criticality x based i.e. Dcost = Criticality x Lethality Actual Cost = Progress x Criticality x based DCost = Criticality of Target x Lethality of Attack x Attack Progress 14 Response Cost Response Mechanisms depend on IDS Capabilities Site-specific policies Attack type, target resource Response Cost - Repair downtime Is greater for more complex attacks marked orange in the attack taxonomy RCost = criticality x baser [base response cost]..to respond or.. not to respond.. is the question.. For an Intrusion Response Cost > Damage Cost => Simple Logging Response Cost Damage Cost => Take appropriate response such that RCost remains less than Dcost Responses Termination of network connection/session Rebooting targeted system Notify admin of offending machine(s) Recording session for evidence and further investigation (to avoid action against false positives) Identification, Containment, Eradication, Recovery Operational Cost Inherent Cost of running an IDS Amount of Time & Computing Resources required to extract and test features OpCost is associated with time because a real-time IDS must detect an attack while it is in progress and generate an alarm as quickly as possible so that the damage can be minimized A slower IDS which uses features with higher computational costs should be penalized Even the cost of computational resources is a factor Since, a resource in use for an IDS may not be used by another task at the same time 17 Operational Cost (contd..) More features => Operational Cost Increases e.g. co-relation analysis to detect extended-coordinated attacks Features are classified relatively on their Computational Costs Level 1- can be computed from the 1 st packet itself (e.g. service) Level 2 - can be computed at any point during the life of the connection (e.g. connection state) Level 3 - can be computed at the end of the connection using information only about the connection being examined (e.g. bytes sent during the connection) Level 4 - can be computed at the end of the connection, but require access to data of potentially many other prior connections (e.g. temporal and statistical features and are most costly to compute) 18 3

4 Cost Models Operational Costs (OpCost) Inherent cost of running an IDS In the absence of an IDS the OpCost will decrease (costs will be only the other non-ids security measures) But, that will be sensible only if the value or damage cost to the system, its downtime and repair cost and is acceptable and does not make it feasible to install and run an IDS with higher costs Consequential Costs (CCost) Consequential Costs are incurred as a consequence of a predictions made for an event and response(s) generated, if any 19 Cost Models (contd..) For an event e = (a,p,r) under observation described by.. attack type a progress p of the attack and, the target resource r the detection outcome of event e can be one of the following: False Negative (FN) cost of not detecting an attack False Positive (FP) cost of responding to and the penalty incurred while treating a legitimate event as an intrusion True Positive (TP) cost of detecting attack & possibly responding to it True Negative (TN) the cost of finding events to be legitimate Misclassified Hit (MCH) the cost of mis-identifying an attack, responding to it incorrectly (if RcostDcost) and also incurring the cost of damage due to the true attack & the effect of its progress until an appropriate response is generated and takes effect 20 Cost Models (contd..) Model for Consequential Cost [Wenke Cost ACM 2001] Outcome False Negative (FN) Miss False Positive (FP) False Alarm True Positive (TP) Hit True Negative (TN) Normal Activity Misclassified Hit Mistaken Attack Consequential Cost - CCost(e) DCost(e) RCost(e ) + PCost(e) 0 RCost(e) + 1 DCost(e), DCost(e) 0 1 is a function of the progress of the attack RCost(e ) + 2 DCost(e), DCost(e) 2 is a function of the progress and the effect of the incorrect response Condition If RCost(e ) DCost(e ) If RCost(e ) > DCost(e ) If RCost(e) DCost(e) If RCost(e) > DCost(e) If RCost(e ) DCost(e ) If RCost(e ) > DCost(e) Cost Models (contd..) Therefore, for a labeled test set E, where each e E has a label of normal or one of the intrusions We define the Cumulative Cost of the IDS as: Cumulative Cost(E) = ( CCost ( e) + OpCost( e)) e E It may not always be possible to put damage costs into the same measurement units Hence each of it is analyzed on its own relative scale We have to thus compare and then combine the two so that we can compute CCost(e) for use in calculation of relevant cost Cost Sensitive Modeling Cost-sensitive modeling must be performed periodically because cost metrics must take into account changes in Information assets Security policies Therefore it is important to develop tools that can automatically produce cost-sensitive models for given cost metrics Machine learning methods for reducing the Cumulative Cost of intrusion detection have been developed and evaluated We shall describe the particular methods which have proven most effective in building cost-sensitive models. 23 Cost Sensitive Modeling: Reducing Costs Reducing OpCost Use the cheaper features in the ID models Less computationally intensive Lesser delay Reducing CCost If for an intrusion RCost > DCost Then Do not respond to the attack 24 4

5 Reducing Operational Costs A multiple model approach Build multiple rule-sets, each with features of different cost levels Originally computational costs of features were not considered as accuracy was the only concern Use cheaper rule-sets first (less accurate), costlier ones later only for required higher accuracy A simple way to incorporate cost-sensitive rule induction in RIPPER (i.e. make it more sensitive to feature costs) Modify search heuristic s.t. it considers information gain & feature cost Different types of intrusions can enforce a notion of a different per-class sensitivity e.g. for very damaging intrusions, it is desirable to produce a model that is as accurate as possible, without much consideration to the cost of features for intrusions that cause little harm, it is desirable to have a model that is very cheap even if accuracy is sacrificed 25 Reducing Consequential Costs MetaCost Purposely re-label intrusions with RCost > DCost as normal The main idea behind metacost is to first re-label each record in the training dataset using the record s optimal class, which may not be the same as its actual class then apply a standard machine learning algorithm to the relabeled training set Post-Detection decision module Action depends on comparison of RCost and DCost Eliminates the time consuming need to retrain models when cost factors change Detection Module Intrusion Report - Attack Names - Targets Compute -Dcost -RCOst Decision Module Log: Intrusion Report Response Module RDBMS/Files 26 Implementation Baseline(Flat), NC and FSM Rulesets Ordered Ruleset Unordered Ruleset -Evaluation of an ordered -Here there is atleast one rule ruleset does not require each for each class and many rules rule to be tested, but proceeds for frequently occuring from the top of the ruleset to classes the bottom until any rule -There is also a default class evaluates to true. which is used for prediction -The OpCost to evaluate an OR when none of the rules are for a given event is the total satisfied cost of computing unique -The advatage : all rulesare features until a prediction is evaluated during prediction made and conflicts are broken by -For ID a freq. Ruleset is using the more accurate rule usually lowest in OpCost -Unordered Rulesets ie.accurately classifies normal generally contain more rules events (the1st rules identify and are less efficient in normal, which is usually the execution than freq & + freq most frequently occuring class) ordered rulesets but, -Otherside, a +freq ruleset will -There are usually several mostlikely be higher in OpCost rules of high precision for the but more accurate in most frequent class, resulting classifying intrusions in accurate classification of normal events FSM Approach -This is the hand coded attempt to model the decision tree approach used in many IDS applications -At the top level, packets are classified by protocol and then proceed in the FSM -All the rules ina catgory are grouped into a single decision tree, with leaves of the tree classified as attacks or normal connections -Features are computed at each point in the tree, cheapest first Conclusions.. Trade-offs exist between various factors The total expected Cost of an IDS is the sum of its Operational Cost and Consequential Costs A multiple model machine learning approach for reducing the operational cost A post-detection module for reducing consequential cost Future Directions Until now the cost and the attack analysis was performed for individual attacks In the real world, it is required to study cost-sensitive analysis and modeling for detection of complex attack scenarios Also, incorporating uncertainity of cost analysis due to incomplete or imprecise estimation esp. in anomaly detection systems into the process of cost-sensitive modeling Project 1: Some Real DM Analysis in an IDS Global Data Guard(GDG) developed Empirical Surveillance Program (ESP) Application of behavioral techniques to IDS for 10 years assembled into one Next generation IDS with continual adaptive data collection, analysis and correlation engine ESP is believed to be the most advanced behavioral IDS due to Adaptive adapts to changing and evolving networks Behavioral Analyzes raw packet information, along with firewall and router logs, to find anomalous behavior Historical Discovers slow/stealth attacks by leveraging historical data Captures forensics data Scalable Runs on a multi-server system which facilities distributed processing Easy-to-use Proven Layered Defense combining behavioral with the traditional Knowledge-based IDS Database Server with Host Intrusion Detection ESP Analysis System Private Network ESP Sensor & Network Analysis Stage 2 Private Zone FIREWALL Server Stage 1 RDBMS Public Zone Web Server Internet FIREWALL Stage 3 FIREWALL 768-Bit Encrypted Tunnel Perimeter Device ESP is compromised of: Analysis and correlation engine Supports customer specific and global correlation Unified Console e-tunnel (in development) Ability to plug into any knowledgebased IDS Includes knowledge-based, behavioral-based and host-based agents ESP is a passive network device, therefore no impact on customer s network ESP supports threat deflection via managed honeypot 29 Internal Sensor External Sensor 30 5

6 Case Study Case Studies performed Will be a part of the project report Combined project 1 and 2 31 Fini The term paper will incorporate a complete look at Data Mining for IDS and will not be simply focussed on the Cost sensitive Modeling References ( will be in the term paper ) Any Questions? My other interests include: Mobile Ad Hoc Networks ( Security and QoS) Security and Intrusion Detection If you really liked what I have here.. I am considering employment opportunities Contact:: Abhishek Sanwal asanwal@engr.smu.edu asanwal@graffiti.net