Statistical Aspects of Intrusion Detection

Transcription

1 Statistical Aspects of Intrusion Detection Mgr. Rudolf B. Blažek, Ph.D. Department of Computer Systems Faculty of Information Technologies Czech Technical University in Prague Rudolf Blažek Network Security MI-SIB, ZS 2011/12, Lecture 8 The European Social Fund Prague & EU: We Invest in Your Future

2 Statistické aspekty detekce síťových útoků Mgr. Rudolf B. Blažek, Ph.D. Katedra počítačových systémů Fakulta informačních technologií České vysoké učení technické v Praze Rudolf Blažek Síťová bezpečnost MI-SIB, ZS 2011/12, Přednáška 8 Evropský sociální fond Praha & EU: Investujeme do vaší budoucnosf

3 Intrusion Detection Detection of Intrusions 3

4 Intrusion Detection Introduction Intrusion Detection And Prevention Several required tasks Collect relevant data about network traffic Store it efficiently and fast (at a central location) Analyze the collected data Fast online processing (real-time detection) Offline processing (longer-term analysis) Report detected intrusions and/or identify behavior patterns process false alerts feedback React to intrusions (passive / active / prevention) 4

5 Intrusion Detection Introduction Data Collection Network Monitoring Approach Depends on the Goals Fast online processing (real-time detection) May require processing of each packet data Packet header info (type, flags, seq. number), payload size State-less or state-full analysis (may require app. layer info) Offline processing (longer-term analysis) It is often enough to collect information about network flows Flow type, end-points, start-time, duration, amount of data... 5

6 Intrusion Detection Real-time Data Capture Collecting Packet Information pcap Packet Capture a commonly used API for network traffic monitoring open source C/C++ library libpcap unix implementation ( WinPcap Windows library ( tcpdump a traffic capture/monitoring tool for unix, uses libpcap developed in the same project as libpcap WindDump for Windows (part of the WinPcap project) 6

7 Intrusion Detection Real-time Data Capture Collecting Packet Information Example: tcpdump usage tcpdump -w captureddata.pcap -i eth0 tcp port 8083 or udp \( or \) Captures data to a binary file captureddata.pcap tcp traffic on port 8083, or udp traffic on ports or careful: we used or (union), not and (empty inter Intrusion Detection) The data can be analyzed later by tcpdump 7

8 Intrusion Detection Real-time Data Capture Collecting Packet Information Example: tcpdump usage tcpdump -w captureddata.pcap dst and tcp port 22 Captures data to a binary file captureddata.pcap tcp traffic on port 22 of host with IP ssh connections to host

9 Intrusion Detection Real-time Data Capture Collecting Packet Information Example: tcpdump usage Read and display packets from a file tcpdump -nnr captureddata.pcap tcpdump -tttnnr captureddata.pcap -r(( -nn( -ttt(... read packets from a binary file... do not resolve names for IP and ports (DNS)... differential timestamps 19:57: IP > : Flags [.], ack 118, win 33264, options [nop,nop,ts val ecr ], length 0 00:00: IP > : Flags [.], ack 178, win 33301, options [nop,nop,ts val ecr ], length 0 9

10 Intrusion Detection Real-time Data Capture Collecting Packet Information Example: tcpreplay usage tcpreplay --mbps=10.0 -i eth0 captureddata.pcap Replays data from a binary file captureddata.pcap interface eth0 rate 10Mbps 10

11 Intrusion Detection Real-time Data Capture Collecting Packet Information Example: tcpdump usage Capture messages for the ARP and RARP protocols tcpdump -i eth0 arp or rarp Capture all packets other than ARP and RARP tcpdump -i eth0 not arp and not rarp The first example: Can be used for monitoring ARP cache poisoning (MiM attack by ettercap that we discussed) 20:25: ARP, Request who-has tell , length 28 20:25: ARP, Reply is-at fa:ab:38:a0:22:1a (oui Unknown), length 28 11

12 Intrusion Detection Real-time Data Capture Tools using the pcap API Various traffic monitoring tools tcpdump and WindDump WireShark (formerly ethereal) Intrusion detection systems Snort, Bro, etc... Wrappers for the pcap API e.g scapy for Python 12

13 Intrusion Detection Subsection Hierarchical Distributed IDS IDS Agent GSM/GPRS IDS Messages IDS Agent GSM/GPRS BSS IDS Fusion IDS Agent GSM/GPRS IDS Agent GSM/GPRS IDS Messages IDS Agent GSM/GPRS BSS IDS Fusion IDS Agent GSM/GPRS IDS Messages IDS Agent WiFi/WiMax WLAN WMAN IDS Fusion IDS Agent WiFi/WiMax IDS Agent Remote LAN Remote Subnet IDS Messages Firewall LAN Fusion Center IDS Messages Service Provider Central LAN Top Level IDS Fusion Center IDS Messages Firewall IDS Monitoring Center Firewall Server Farm Firewall IDS Sensor Firewall IDS Sensor Firewall IDS Sensor Firewall IDS Sensor Internet Multiple backbone connections 13

14 Intrusion Detection Distributed Detection Systems Hierarchical Distributed IDS IDS Agent IDS Agent Intermediate Fusion Center IDS Agent IDS Agent IDS Agent IDS Agent Subnet Sensor Intermediate Fusion Center IDS Monitoring IDS Fusion Center Intranet Sensor IDS Monitoring 14

15 Intrusion Detection Distributed Detection Systems Hierarchical Distributed IDS GSM/GPRS GSM/GPRS GSM/GPRS BSS WiFi/WiMax WiFi/WiMax WiFi/WiMax YZU Dorm Hub Optical Fiber Wireless LMDS Copper WLAN WMAN HLR/DNS DHCP/MS SGSN Emulator Nei-Li Central Office EI/TI Modem Ban-Ciao MSC/SGSN Cisco Router EGGSN Intranet Router LAN 15

16 Intrusion Detection Distributed Detection Systems Hierarchical Distributed IDS IDS Agent GSM/GPRS IDS Agent GSM/GPRS BSS Intermediate GSM/GPRS Fusion Center IDS Agent IDS Agent WiFi/WiMax WiFi/WiMax IDS Agent IDS Agent Subnet Sensor YZU Dorm Hub Optical Fiber Wireless LMDS Copper WLAN / WMAN Intermediate WiFi/WiMax Fusion Center HLR/DNS DHCP/MS SGSN Emulator Nei-Li Central Office Ban-Ciao MSC/SGSN IDS Monitoring IDS Monitoring EI/TI Modem Cisco Router IDS Fusion Center EGGSN Intranet Sensor Intranet Router 16

17 Statistical Aspects of Intrusion Detection Statistical Performance Metrics for Intrusion Detection Systems 17

18 Statistical Aspects of Intrusion Detection Performance Metrics Statistical Aspects of Intrusion Detection Non-statistical features of network intrusions: Network protocols are deterministic and well understood Protocol anomalies can be detected by stateful analysis Many ad-hoc methods work well to detect various attacks Example of a good deterministic ad-hoc detection rule Suspect a host is using P2P transfers if: it uses network flows via port 6881 uses ports above 50000, with many port changes connects to many IPs, most of them inaccessible at the end many connection finish at the same time 18

19 Statistical Aspects of Intrusion Detection Performance Metrics Statistical Aspects of Intrusion Detection Statistical features of network intrusions: Network intrusions occur randomly Intrusions occur at unknown points in time Intrusions lead to changes of statistical properties of some observable characteristics Attack detection viewed as a change-point detection (CPD): Detect changes in the distributions (models, parameters) With fixed delays (batch-sequential approach) Or with minimal average delays (sequential approach) While maintaining the false alarm rate at a given level 19

20 Statistical Aspects of Intrusion Detection Performance Metrics CPD Methodology Observed sequence of random variables (or vectors): X1, X2,... X1, X2,..., Xn represent some network characteristics observed at times t1, t2,... Examples: numbers of deauthentication frames, numbers of failed connections, levels of link saturation etc. A change in distribution occurs at an unknown index λ The change corresponds to a network traffic anomaly at time tλ Pk and Ek denote the probability and the expectation when λ=k P0 and E0 correspond to the pre-change and the no-change distribution 20

21 Statistical Aspects of Intrusion Detection Performance Metrics Common Performance Metrics in Sequential Intrusion Detection The performance of the sequential detection procedures can be measured using various of criteria: Test Power and Probability of False Alert: We desire high power and low probability of false alarms. For a fixed decision time k we can define: PWR rms. k For = a P(τ fixed k decision λ k) time k, the ual PFA network k = P(τ monitoring, k λ > k) h= P 0 (τ k) In long-term network ver, the monitoring change-point the change-point λ (i.e. intruλ (i.e. an intrusion) may occur very late. For large k, any detection procedure will have PFA k nearly 1 or at least relatively high. 21

22 Statistical Aspects of Intrusion Detection Performance Metrics Common Performance Metrics in Sequential Intrusion Detection Test Power and Probability of False Alert: It is therefore more practical to consider conditional PFA for a sliding time interval of length T PFA T k = P(τ < k + T τ k,λ k + T) = P 0 (τ < k + T τ k), PFA T = sup 1 k P 0 (τ < k + T τ k) The condition τ k corresponds to false alarms shortly after (within period T) The IDS was inspected and found OK at time k The IDS was started or restarted after an alert (k = 0) An upper bound on PFAT works for all of these situations 22

23 Statistical Aspects of Intrusion Detection Performance Metrics Common Performance Metrics in Sequential Intrusion Detection Run Length and Rate of False Errors: The average run th length of the before det the change ARL 0 =E 0 τ monitoring The average run length after the change ARL 1 =E 1 τ, uick detecti We desire quick detection (low ARL 1 ) and infrequent false alarms (high ARL0). The average false alarm rate FAR(τ) = 1 / E 0 τ ( ( ( ( ( ( ( ( ( is often used instead of ARL0 Low FAR is a very important and practical requirement!!! 23

24 Statistical Aspects of Intrusion Detection Performance Metrics Common Performance Metrics in Sequential Intrusion Detection Detection Delay: Average detection delay, assuming a change at a fixed λ = k E k (τ-k) + (τ-k) + ) c As k increases the delay often approaches 0. For a random λ we can summarize: Conditional average detection delay ADD λ (τ) = E λ (τ λ τ λ). Very important in continual surveillance k, is E k (τ-k) E(E λ (τ-k) + ) nt role in co Often approaches a constant for increasing λ = k (stable detection system) 24

25 Statistical Aspects of Intrusion Detection Performance Metrics Common Performance Metrics in Sequential Intrusion Detection Utility Function: There is often cost associated with false alerts and detection delays communication overhead in network security. If the cost (utility function) is a linear function of FAR and delay, an optimal procedure can in some cases be obtained by fixing the FAR and minimizing the ADD 25

26 Statistical Aspects of Intrusion Detection Performance Metrics Detection Performance Metrics Tradeoffs Tradeoffs in Continual Surveillance Tuning Adjustment Higher Threshold Lower Threshold Optimization Strategy A Optimization Strategy B Effects on FAR and ADD Smaller FAR Longer ADD Higher FAR Shorter ADD Limit FAR γ Minimize ADD Limit ADD K Minimize FAR PFA and PWR in a time interval of given size Smaller PFA Lower Power Higher PFA Higher Power Limit PFA α Maximize Power Guarantee Power β Minimize PDA Effects on overall PFA PFA approaches 1 PFA approaches 1 Not applicable Not applicable 26

27 Statistical Aspects of Intrusion Detection Performance Metrics Detection Performance Metrics Tradeoffs Algorithms cannot maintain all metrics at prescribed levels Optimization of detection procedures balances the tradeoffs A standard optimization strategy: Prescribe the bounds for some metrics Use some other metrics as optimization criteria The selection of these metrics has practical considerations Classical approach: maximize the test power among all tests with a fixed prescribed low level of PFA Continual surveillance: minimizing the detection delay for a prescribed low FAR 27

28 Statistical Aspects of Intrusion Detection Performance Metrics Detection Performance Metrics Tradeoffs Tradeoffs in Continual Surveillance Tuning Adjustment Higher Threshold Lower Threshold Optimization Strategy A Optimization Strategy B Effects on FAR and ADD Smaller FAR Longer ADD Higher FAR Shorter ADD Limit FAR γ Minimize ADD Limit ADD K Minimize FAR PFA and PWR in a time interval of given size Smaller PFA Lower Power Higher PFA Higher Power Limit PFA α Maximize Power Guarantee Power β Minimize PDA Effects on overall PFA PFA approaches 1 PFA approaches 1 Not applicable Not applicable 28

29 Statistical Aspects of Intrusion Detection Performance Metrics The Advantages of Controlling False Alert Rates Continual surveillance: minimizing the detection delay for a prescribed low FAR Another possible approach prescribe a bound for the ADD and minimize the FAR useful for mission-critical applications (finance, defense) The main argument against controlling FAR = 1/E0τ Low value of FAR does not imply low PFAk = P0(τ k) Experiments show that for our procedure the pre-change P0 distribution of τ is approximately geometric Then low FAR guarantees low PFAk T 29