Anatomy of a Real-Time Intrusion Prevention System

Transcription

1 Anatomy of a Real-Time Intrusion Prevention System Koller, R.; Rangaswami, R.; Marrero, J.; Hernandez, I.; Smith, G.; Barsilai, M.; Necula, S.; Sadjadi, S.M.; Tao Li; Merrill, K., Autonomic Computing, ICAC '08. International Conference on, vol., no., pp , 2-6 June CS895 Firas Alomari

2 Introduction Intrusion Detection Systems (IDS):IDS monitors a set of variables in the environment to trigger an alert: Exploit based: signatures of known attacks. False negative: attacks with no alarms triggered. Signatures development, updates and size. Consideration performance and design. Anomaly based: statistical (behavioral) patterns of normal systems. False positive: alarms are triggered with no actual attack. Training data: offline, online, and size. Considerations: Performance, design.

3 Introduction Challenges Accuracy: False negative. False positive. Examples: TOCTTOU: between the check operation and use operation an attacker can remove and replace a file to be executed as root (requires correlation between different processes). Root Kits: changes the root files to conceal their presence. Performance: Tasks response time distribution. Overhead time (events monitoring, collection, correlation and matching). IDS need to consider booth factors Dual approach. Configurable performance parameters.

4 Introduction Rootsense Holistic Mentoring: In TOCTTOU we need a time sensitive correlation between different processes and users. Root only intrusions: Root is more important and more feasible to detect. Dual approach: In root intrusions both approaches are complementary and there is a need to have both. Real-time response: efficient design and tunable parameters: Usability and adaptability.

5 System Architecture Five layers Monitored Subsystems Sensors Monitoring Modules Analyzing model Response Mechanism Events [event ID] [timestamp] [subsystem] [pid] [uid] [euid] [event type] [arguments] P exec / usr/bin/yum F write / bin/ls 211 Signatures Generic events doesn t include timestamp information and can use wild cards P p u 0 exec :U * * Ordered signatures Malicious activity signatures Anomalous activity signatures Benign activity signatures

6 Detection and Response Two independent detectors: Root penetration detector: multiple users/processes. Root misbehavior detector: only cover the root (EUID) Activity state machine Create a state machine for every system events that matches a first event in a signature and populate the state with the event information. The state machine is set to waiting for event status. For every waiting for event status state matching the observed system event, populate the state info and transition to the next state. If the next state terminates then invoke response mechanism. If the event belongs to a terminating class of events terminate and abort of the state machine.

7 Detection and Response Root misbehavior Same as penetration detector and at the same time monitor benign activity signatures. If the benign activity signatures terminates before the anomalous signatures trigger an alarm. Signatures Databases: Three separate sets to keep the number manageable. Hard coded based on known exploits, and root misbehavior and benign activity. Three hash tables to speedup the detection: Signatures hash-table State machine hash-table Anomalous signature hash-table

8 Overhead The kernel component is placed between the syscall interface and the implementation. Invocation of a syscall will generate an event that will be placed in a circular buffer and the process (sleep) in the waiting queue. The event reader reads g events and analyze them. When next g events are read the sleeping processes are woken up. The response mechanism is invoked if the analyzer finds a malicious event. Depending on the policy, malicious processes are either terminated or denied and an alarm is triggered. Analyzing each system call before it s executed introduces overhead due to context switching between rootsense and the kernel

9 Overhead IDS overhead characteristics can be captured by the response time distribution. Average response time: system interactivity. Standard deviation value: Jitter. G control the number of switches between user space and the kernel. G is a configurable control parameter. Bigger G improves average response time but introduces jitter time for the users. Web server - larger G Desktop - smaller G

10 Accuracy Evaluation Accuracy Known attacks and benign activity.

11 Evaluation Performance Three benchmarks Lmbench, Unixbench, and Iozone. Interceptor only incur a small overhead

12 Evaluation Response time Maximum response time increase with G. Very small G degrade performance exponentially.

13 Evaluation Signature database size Signature numbers decrease performance. The number of state per signature effects is minimal.

14 Conclusion IDS design with performance consideration. Root intrusions only. Dual detection approach. Controllable response time distribution metrics. Future work Includes a signature language. Automatic backtracking of misbehavior to generate an exploit signature.