Introduction to Network Security Missouri S&T University CPE 5420 Anomaly Detection

Transcription

1 Introduction to Network Security Missouri S&T University CPE 5420 Anomaly Detection Egemen K. Çetinkaya Egemen K. Çetinkaya Department of Electrical & Computer Engineering Missouri University of Science and Technology 31 October 2016 rev Egemen K. Çetinkaya

2 Anomaly Detection in Networks Outline Introduction and motivation Anomaly detection taxonomy Detection in wireless networks Signal analysis of anomalies Conclusions and summary 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 2

3 Anomaly Detection in Networks Introduction and Motivation Introduction and motivation Anomaly detection taxonomy Detection in wireless networks Signal analysis of anomalies Conclusions and summary 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 3

4 ResiliNets Strategy D 2 R 2 + DR Real time control loop: D 2 R 2 defend against challenges to normal ops passive active detect when defenses fail remediate to do the best possible recover to original state Background loop: DR diagnose fault that lead to failure refine future D 2 R 2 behavior Diagnose Defend Refine [SHÇ+2010] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 4

5 Anomaly Detection in Networks Introduction and Motivation Detection is required once the defenses fail Threat model is essential for successful detection Primarily work in literature is about security flash crowds don t follow this Different domains need different detection algorithms wired vs. wireless Several types of detection mechanisms exist Historical progress: console monitoring for user activity reviewing logs: tedious, takes long time, after the attack? real-time systems: new attack recognition is not easy 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 5

6 Challenges to Normal Operation Perturbations Unintentional misconfiguration, operational mistakes random node or random link failures Large scale natural disasters (geo-correlated failures) natural: hurricanes, tsunami, floods, earthquakes, etc. man-made: fire, explosions, etc. Attacks from an intelligent adversary Environmental challenges primarily wireless environments Unusual but legitimate traffic (e.g. flash crowd) Dependent failures Social, political, economical, and business factors 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 6

7 P(t) 1 S a +S r T1 Model Survivability Attributes S u S r Remediate Egemen K. Çetinkaya Recover S a 0 t r t R t 0 t 1 t 2 t S u fraction unservable after failure at t 0 S a fraction available at t 0 S r fraction restored at t 1 full restoration at t 2 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 7

8 Challenge Evaluation ANSA Model V v 2 v 2 o 1 e 1 e 2 o 2 e 3 o 3 t 2 Correct operation all events occur within expectations e i E o i O : val(o i ) (t i, t i ) t 2 t [ER1994] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 8

9 Anomaly Detection in Networks Anomaly Detection Taxonomy Introduction and motivation Anomaly detection taxonomy Detection in wireless networks Signal analysis of anomalies Conclusions and summary 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 9

10 Anomaly Detection in Networks Detection Model = M, D Anomaly detection system: Model of normal behavior: M Degree of deviation: D [TTV2004] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 10

11 Anomaly Detection in Networks Classification Model Learn a model classifier From a set of labeled data instances training Classify using learned model testing [CBK2009] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 11

12 Anomaly Detection in Networks Classification Boundaries Objects might classified based on attributes: x 1 & x 2 Linear classification finds a line between classes Classification boundaries may be non-linear [BBK2014] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 12

13 Detection normal anomalous Egemen K. Çetinkaya Anomaly Detection in Networks Correct Detection of Events Also known as false alarm Successful detection false positive true negative true positive false negative Harmless event labeled normal Attacks not detected, similar to normal behavior harmless attack Event Nature [TTV2004] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 13

14 Anomaly Detection Taxonomy Overview Network feature analyzed network traffic flow analysis protocol analysis network elements and topology Behavior model learnt models specification-based models Analysis scale microscale mesoscale macroscale 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 14

15 Anomaly Detection Taxonomy Network Feature Analyzed Network traffic flow analysis: utilizes temporal evolution of traffic flow stochastic process and signal analysis are main methods protocol analysis data link network transport application Network elements and topology 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 15

16 Anomaly Detection Taxonomy Behavior Models Constructing the normal behavior of the system by machine learning techniques manually providing specifications Learnt models statistical (e.g. estimators, Markov chains, etc.) rule-based artificial neural networks high false alarm rate since data is trained Specification-based models (signature or misuse) difficult, time consuming, only done once low false alarm rate 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 16

17 Anomaly Detection Taxonomy Analysis Scale Several dimensions of analysis exist microscale mesoscale macroscale Functional perspective: service, host, network Time dimension: hourly, daily/weekly, seasonal Protocol analysis: packets, streams, simultaneous analysis of connections 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 17

18 Anomaly Detection in Networks Detection in Wireless Networks Introduction and motivation Anomaly detection taxonomy Detection in wireless networks Signal analysis of anomalies Conclusions and summary 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 18

19 IDS in Wireless Networks Overview Mobile and wireless networks differ than wired nets links susceptible to attacks nodes do not have enough physical protection dynamic topology IDS agents in each mobile host IDS agents can detect and decide locally Collaborative among the nodes if node detects anomaly with weak evidence detection state information is propagated to neighbor nodes 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 19

20 IDS in Wireless Networks Detection State Information Use of level-of-confidence value: p % Node A concludes from local data there is intrusion Node A concludes from local and neighbor states Node A, B, C, collectively concludes about intrusion Weights can be included in computation nearby nodes can have more weight then distant nodes 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 20

21 IDS in Wireless Networks Detection Procedure Node sends intrusion state request to neighbor Each node propagates state information likelihood of intrusion to its immediate neighbors Each node determines whether majority of reports indicate intrusion if yes; node concludes network is under attack Any node detecting intrusion can initiate response e.g. initiating re-authentication to exclude malicious nodes 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 21

22 IDS in Wireless Networks Anomalies of MANET Routing Protocols Training of networks required for simulation this entails aggregation of trace files Routing intrusion metrics: percentage of changed routes it can also include: bad routes stale routes updated routes percentage of changes in the sum of hops of all routes Traffic flow, routing activities, topological patterns preferred in correlating change of information 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 22

23 Anomaly Detection in Networks Signal Analysis of Anomalies Introduction and motivation Anomaly detection taxonomy Detection in wireless networks Signal analysis of anomalies Conclusions and summary 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 23

24 Signal Analysis of Anomalies Overview Wavelets describes time series in frequency and time useful for characterizing data with spikes and discontinues Fourier transform is good for frequency analysis Following results show IP flow and SNMP data flow is end-to-end association: by src/dst address, port SNMP: Simple Network Management Protocol [RFC 1157] defines a set of MIB (Management Information Base) From Univ. of Wisconsin Madison gateway router data collected over 6 months analysis includes high, medium, low band data 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 24

25 Signal Analysis of Anomalies Ambient IP Flow Traffic Egemen K. Çetinkaya Baseline IP flow traffic [BKPR2002] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 25

26 Signal Analysis of Anomalies Ambient SNMP Traffic Egemen K. Çetinkaya Baseline SNMP traffic [BKPR2002] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 26

27 Signal Analysis of Anomalies Byte Traffic for Flash Crowd Long-lived events can be captured by: low-band and mid-band filters [BKPR2002] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 27

28 Signal Analysis of Anomalies Average Packet Size for Flash Crowd Long-lived events can be captured by: low-band and mid-band filters [BKPR2002] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 28

29 Signal Analysis of Anomalies Flow Traffic During DoS Attacks Short-lived events can be captured by: high-band and mid-band filters [BKPR2002] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 29

30 Signal Analysis of Anomalies Byte Traffic During Measurement Analysis Short-lived events can be captured by: high-band and mid-band filters [BKPR2002] 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 30

31 Anomaly Detection in Networks Conclusions and Summary Introduction and motivation Anomaly detection taxonomy Detection in wireless networks Signal analysis of anomalies Conclusions and summary 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 31

32 Anomaly Detection Taxonomy Open Challenges [PP2007] Mechanisms to keep up with the high-speed nets Reduction of false alarm rate: 1/10 5 Evaluation of detection mechanisms/algorithms Defending detection systems from attacks attacks generating false alarms Better understanding of anomalies taxonomy of challenges IDS and encryption does not work IDS for internal attacks different access requirements for different users 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 32

33 Anomaly Detection Taxonomy Conclusions and Summary Detection is essential part of a resilient network Anomaly detection primarily applied in security area Wired/wireless domains have different mechanisms 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 33

34 References and Further Reading [KPS2002] Charlie Kaufman, Radia Perlman, and Mike Speciner, Network Security: Private Communication in a Public World, 2nd edition, Prentice Hall, [CBK2009] Varun Chandola, Arindam Banerjee, and Vipin Kumar, Anomaly Detection: A Survey, ACM Computing Surveys, Volume 41, Issue 3, pp. 15:1 15:58, July [ZLH2003] Yongguang Zhang, Wenke Lee, and Yi-An Huang, Intrusion Detection Techniques for Mobile Wireless Networks, Wireless Networks, Volume 9, Issues 5, pp , September [R1999] Martin Roesch, Snort Lightweight Intrusion Detection for Networks, in Proceedings of the 13th USENIX Conference on System Administration (LISA), Seattle, WA, November 1999, pp October 2016 MST CPE 5420 Anomaly Detection in Networks 34

35 References and Further Reading [SHÇ+2010] James P.G. Sterbenz, David Hutchison, Egemen K. Çetinkaya, Abdul Jabbar, Justin P. Rohrer, Marcus Schöller, and Paul Smith, Resilience and Survivability in Communication Networks: Strategies, Principles, and Survey of Disciplines, Computer Networks, Vol. 54, No. 8, pp , June [ANSA] Nigel Edwards and Owen Rees, A Model for Failures in Dependable Systems, ANSA Technical Report, March [ATIS] ATIS Technical Report 24, Network Survivability Performance, Nov October 2016 MST CPE 5420 Anomaly Detection in Networks 35

36 References and Further Reading [TTV2004] J. M. Estevez-Tapiador, P. Garcia-Teodoro, J. E. Diaz- Verdejo, Anomaly detection methods in wired networks: a survey and taxonomy, Computer Communications, Vol. 27, No. 16, October 2004, pp [BKPR2002] P. Barford, J. Kline, D. Plonka, A. Ron, A signal analysis of network traffic anomalies, ACM IMW, Marseille, 2002, pp [PP2007] A. Patcha and J. Park, An overview of anomaly detection techniques: Existing solutions and latest technological trends, Computer Networks, Vol. 51, No. 12, August 2007, pp [BBK2014] M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, Network Anomaly Detection: Methods, Systems and Tools, IEEE Comm. Surveys & Tutorials, Vol. 16, No. 1, 2014, pp October 2016 MST CPE 5420 Anomaly Detection in Networks 36

37 End of Foils 31 October 2016 MST CPE 5420 Anomaly Detection in Networks 37