An empirical approach to modeling uncertainty in Intrusion Analysis

Size: px

Start display at page:

Download "An empirical approach to modeling uncertainty in Intrusion Analysis"

Luke Bryant
5 years ago
Views:

1 An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University Joint work with S. Raj Rajagopalan at HP Labs and Sakthi Sakthivelmurugan Departmental seminar, Computer Science at Virginia Tech March 19, 2010

2 A day in the life of a real SA Abnormally high traffic Network Monitoring Tools TrendMicro server communicating with known BotNet controllers netflow dump system administrator Seemingly malicious code modules Key challenge: uncertainty in data. memory dump Found open IRC sockets with other TrendMicro servers These TrendMicro Servers are certainly compromised! 2

3 An empirical approach In spite of the lack of theory or good tools, sysadmins are coping with attacks. Can we build a system that mimics what they do (for a start): An empirical approach to Intrusion Analysis using existing reality Our goal: Provide some degree of automation in the process 3

4 High-confidence Conclusions with Evidence Internal model Reasoning Engine Mapping observations to their semantics Targeting subsequent observations Observations IDS alerts, netflow dump, syslog, server log 4

5 Capture Uncertainty Qualitatively Confidence level Uncertainty Modes Low Possible p Moderate Likely l High Certain c Arbitrarily precise quantitative measures are not meaningful in practice Roughly matches confidence levels practically used by practitioners 5

6 High-confidence Conclusions with Evidence Internal model Reasoning Engine Mapping observations to their semantics Targeting subsequent observations Observations IDS alerts, netflow dump, syslog, server log 6

7 Observation Correspondence Observations what you can see mode Internal conditions what you want to know obs(anomalyhightraffic) p int(attackernetactivity) obs(netflowblacklistfilter(h, BlackListedIP)) l int(compromised(h)) obs(memorydumpmaliciouscode(h)) obs(memorydumpircsocket(h1,h2)) l l int(compromised(h)) int(exchangectlmessage(h1,h2)) 7

8 High-confidence Conclusions with Evidence Internal model Reasoning Engine Mapping observations to their semantics Targeting subsequent observations Observations IDS alerts, netflow dump, syslog, server log 8

9 Internal Model Logical relation among internal conditions. direction of, inference mode Condition 1 Condition 2 Condition 1 infers Condition 2 int(compromised(h1)) int(sendexploit(h1,h2)) int(compromised(h2)) int(probeothermachine(h1,h2)) f, p f, l b, p b, c int(probeothermachine(h1,h2)) int(compromised(h2)) int(sendexploit(h1,h2)) int(compromised(h1)) 9

10 High-confidence Conclusions with Evidence Internal model Reasoning Engine Mapping observations to their semantics Targeting subsequent observations Observations IDS alerts, netflow dump, syslog, server log 10

11 Reasoning Methodology Simple reasoning Observation correspondence and internal model are inference rules Use inference rules on input observations to derive assertions with various levels of uncertainty Proof strengthening Derive high-confidence proofs from assertions derived from low-confidence observations 11

12 Example 1 Inference through Observation Correspondence obs(memorydumpircsocket(h1,h2)) l int(exchangectlmessage(h1,h2)) int(exchangectlmsg( , ), l ) obsmap obs(memorydumpircsocket( , )) 12

13 Example 2 Inference through Internal Model int(compromised( ), l ) Int rule int(exchangectlmsg( , ), ) l obsmap obs(memorydumpircsocket( , )) 13

14 Proof Strengthening f is likely true f is certainly true proof strengthening f is likely true Observations: O 1 O 2 O 3 14

15 Proof Strengthening Example c int(compromised( ), ) int(compromised( ), l ) strengthenedpf intr int(exchangectlmsg( , ), l ) obsmap obs(memorydumpircsocket( , )) int(compromised( ), l ) obsmap obs(memorydumpmaliciouscode( )) strengthen( l, l ) = c 15

16 Evaluation Test if the empirically developed model can derive similar high-confidence trace when applied on different scenarios Keep the model unchanged and apply the tool to different data sets 16

Done only once SnIPS (Snort Intrusion Analysis using Proof Strengthening) Architecture Snort Rule Repository Observation Correspondence Internal Model

17 Done only once SnIPS (Snort Intrusion Analysis using Proof Strengthening) Architecture Snort Rule Repository Observation Correspondence Internal Model High-confidence answers with evidence Reasoning Engine User query, e.g. which machines are certainly compromised? (convert to tuples) pre-processing Snort alerts 17

18 Snort rule class type Internal predicate mapped from classtype alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"web-misc guestbook.pl access ;uricontent:"/guestbook.pl ; classtype:attempted-recon; sid:1140;) obsmap(obsruleid_3615, obs(snort( 1:1140, FromHost, ToHost)), int(probeothermachine(fromhost, ToHost)),?). 18

19 Snort rule documents Hints from natural-language description of Snort rules Impact: Information gathering and system integrity compromise. Possible unauthorized administrative access to the server. Possible execution of arbitrary code of the attackers choosing in some cases. Ease of Attack: Exploit exists obsmap(obsruleid_3615, obs(snort( 1:1140, FromHost, ToHost)), int(probeothermachine(fromhost, ToHost)), ). obsmap(obsruleid_3614, obs(snort( 1:1140, FromHost, ToHost)), int(compromised(tohost)), p)? l 19

20 Automatically deriving Observation Correspondence Internal Predicate % of rules Mapped automatically 59% Not mapped automatically 41% Snort has about 9000 rules. This is just a base-line and needs to be fine-tuned. Would make more sense for the rule writer to define the observation correspondence relation when writing a rule. 20

21 Data set description Treasure Hunt (UCSB 2002) 4hrs Collected during a graduate class experiment Large variety of system monitoring data: tcpdump, sys log, apache server log etc. Honeypot (Purdue, 2008) 2hrs/day over 2 months Collected for spam analysis project Single host running misconfigured Squid proxy KSU CIS department network days 200 machines including servers and workstations. 21

Some result from Treasure Hunt data set?- show_trace(int(compromised(h), c)).

int(compromised( 192.168.10.90 ), p) intrule_1 A probe was sent from int(probeothermachine( 192.

168.10.90 ),l) intrule_3 An exploit was sent to int(sendexploit( 128.111.49.46, 192.168.10.90 ), l) obsruleid_3749 192.

22 Some result from Treasure Hunt data set?- show_trace(int(compromised(h), c)). int(compromised( ),c) strengthenedpf was certainly compromised! int(compromised( ), p) intrule_1 A probe was sent from int(probeothermachine( , ), p) obsrulepre_ obs(snort( 122:1, , ,_h272)) int(compromised( ),l) intrule_3 An exploit was sent to int(sendexploit( , ), l) obsruleid_ obs(snort( 1:1807, , ,_h336)) 22

23 Data Reduction Data set Duration of Network traffic Snort alerts pre-processed alerts Highconfidence proofs Treasure Hunt 4 hours 4,849, Honeypot 2 hrs/day for 2 months 637, CIS Network 3 days 1,138,

24 Future Work Continue the empirical study and improve the reasoning model Establish a theoretical foundation for the empirically-developed method Modal Logic Bayes Theory Dempster-Shafer Theory 25

25 Related work Y. Zhai et al. Reasoning about complementary intrusion evidence, ACSAC 2004 F. Valeur et al., A Comprehensive Approach to Intrusion Detection Alert Correlation, 2004 R. Goldman and S. Harp, "Model-based Intrusion Assessment in Common Lisp", 2009 C. Thomas and N. Balakrishnan, Modified Evidence Theory for Performance Enhancement of Intrusion Detection Systems,

26 Summary Based on a true-life incident we empirically developed a logical model for handling uncertainty in intrusion analysis Experimental results show Model simulates human thinking and was able to extract high-confidence intrusion Model empirically developed from one incident was applicable to completely different data/scenarios Reduction in search space for analysis 27

27 Thank you Questions? 28

28 Summarization Compact the information entering reasoning engine Group similar internal condition into a single summarized internal condition 29

29 Comparison of the three data sets attempted-admin attempted-dos attempted-recon attempted-user bad-unknown default-login-attempt misc-activity misc-attack not-suspicious policy-violation non-standard-protocol protocol-command-decode rpc-portmap-decode shellcode-detect Department Treasure Hunt Honeypot successful-admin successful-recon-limited suspicious-filename-detect system-call-detect trojan-activity unknown web-application-activity web-application-attack 30

Anomaly Detection in Communication Networks

Anomaly Detection in Communication Networks Prof. D. J. Parish High Speed networks Group Department of Electronic and Electrical Engineering D.J.Parish@lboro.ac.uk Loughborough University Overview u u