Can the Best Defense be to Attack?

Transcription

1 Can the Best Defense be to Attack? MITACS Digital Security Seminar Series at Carleton University Presenter: Dr. Nur Zincir-Heywood Dalhousie University, Faculty of Computer Science

2 Arms Race Security engineers vs black hat attackers Attackers to evade defense systems Security research/technology to patch/new systems Even in research environments Defender systems White hat attackers (vulnerability testing and evasion) MITACS Seminar, 2007 Zincir-Heywood 2

3 Bad guys Intelligent Malicious intentions Automated tools Botnets MITACS Seminar, 2007 Zincir-Heywood 3

4 Good guys Confidentiality, integrity, authentication cryptography Policies Access Control Firewalls Virus checkers Worm detectors MITACS Seminar, 2007 Zincir-Heywood 4

5 How do we know we are attacked? Alerts from security tools used Usually signature based Poor in new attacks Low FP, but Monitoring traffic Expert(s) perform forensic analysis after the event Deep packet inspection Results in patches and new releases MITACS Seminar, 2007 Zincir-Heywood 5

6 First egg or chicken? Do we need to be attacked first to understand that there is a new attack? What about Penetration and Vulnerability checking Blind-spot analysis (evasion) MITACS Seminar, 2007 Zincir-Heywood 6

7 Mimicry Attacks Assume a core attack Modify it to look different but actually does the same/similar damage Hiding in normal behavior Hiding in blind-spot of the detector Hiding it in a less harmful attack MITACS Seminar, 2007 Zincir-Heywood 7

8 Two Sides of the Arms Race Defender Signature based IDSs Anomaly based IDSs Attacker Mimicry attacks Evasion attacks MITACS Seminar, 2007 Zincir-Heywood 8

9 Defender Anomaly based Stide Improved versions Signature based Snort (Mutz( et al, 03; Vigna et al, 04; Kayacik et al, 05) ISS RealSecure (Mutz et al, 03) Symantec Net prowler (Vigna( et al, 04) MITACS Seminar, 2007 Zincir-Heywood 9

10 Anomaly Based Detectors: Host based Three categories: Black Box Black box, gray box and white box Black-box Techniques Extract info only from system calls Fixed length - Stide (Forrest et al, 96) Alternative data models (Warrender( et al, 99) Variable length (Wespi( et al, 00) MITACS Seminar, 2007 Zincir-Heywood 10

11 Anomaly Based Detectors: Gray-Box Extract info from system calls + run-time process execution state Utilize a Finite State Automata (FSA) to characterize normal behavior (Sekar( et al, 01) Instead of FSA utilize a virtual path table details not only where the system call is executed from, but also describes the point where the execution is returned (Feng( et al, 03) Generate an execution graph to understand the max subset of program control flow graph (Gao( et al, 04) MITACS Seminar, 2007 Zincir-Heywood 11

12 Anomaly Based Detectors: White-box Extract info from the monitored program Static analysis of source code or binary image System calls represented by a state machine extracted from control-flow graph (Wagner et al, 01; Giffin et al, 02 and 04) System call inlining and notify calls are introduced (Lam et al, 04) Static analysis to extract an automaton with call stack information is introduced (Feng( et al, 04) multiple detection models are applied to system call arguments overall aggregate score of these models is introduced (Mutz et al, 06) MITACS Seminar, 2007 Zincir-Heywood 12

13 Attacker: Mimicry Attacks Modify system call sequence of an exploit rendering it undetectable to a specific IDS (Wagner et al, 02) - wb,, manual Similar approach based on modifying the exploit code (Tan et al, 02) - wb,, manual Generate variations of signatures to test the quality of detection against Snort, ISS RealSecure, Symantec Net Prowler (Mutz( et al, 03; Vigna et al, 04) - bb (!), automatic Generate attack against gray-box detectors (Gao( et al, 04) - gb,, manual Evolve mimicry attack against Snort (Kayacik( et al, 05) - bb, automatic MITACS Seminar, 2007 Zincir-Heywood 13

14 Mimicry Attacks: Kruegel et al, 05 Automatic attack generation White box testing Assumes vulnerable application is known Assumes a core attack is known Against a gray-box (Sekar( et al, 01) and a white-box detector (Feng et al, 04) Utilize the details of how the detector works in attack generation Statically analyze victim x86 binaries Approach employs symbolic execution Objective is to identify code pointers that can be modified to point to the attacker code Tested on 3 sample programs protected by the above IDSs MITACS Seminar, 2007 Zincir-Heywood 14

15 Symbolic execution (Kruegel et al, 05) MITACS Seminar, 2007 Zincir-Heywood 15

16 Deriving an Appropriate Configuration (Kruegel et al, 05) MITACS Seminar, 2007 Zincir-Heywood 16

17 Results for Real World Applications (Kruegel et al, 05) MITACS Seminar, 2007 Zincir-Heywood 17

18 Execution Steps and Time (Kruegel et al, 05) MITACS Seminar, 2007 Zincir-Heywood 18

19 Design Requirements Developing a static analysis tool for each binary system White box testing approach Expensive (knowledge) Limited semantic coverage Exhaustive search (constrained by the above item) using symbolic execution Solving a linear constraint can be exponential in the number of inequalities Assumes that each symbolic expression refer to different memory location Not all symbolic expressions can be resolved (see the above item) MITACS Seminar, 2007 Zincir-Heywood 19

20 Mimicry Attacks: Giffin et al, 06 Automatic attack generation White box testing Assumes vulnerable application is known Assumes a core attack is known Develop a model of OS wrt security critical state Manually construct the OS model Manually construct the malicious OS state Apply model checking to prove that no reachable OS configuration corresponds to the effect of an attack Test it against Stide IDS using wu-ftpd ftpd,, restore, traceroute, passwd applications MITACS Seminar, 2007 Zincir-Heywood 20

21 OS Model Manually identify what OS state variables constitute security relevant states Initial assignment of values to OS state variables encode the OS state configuration before execution of a process For each system call a relation is provided for how it changes state based upon the previous state (pre- and post- conditions) MITACS Seminar, 2007 Zincir-Heywood 21

22 Architecture (Giffin et al, 06) MITACS Seminar, 2007 Zincir-Heywood 22

23 Describing Stide Model for Each Application (Giffin et al, 06) MITACS Seminar, 2007 Zincir-Heywood 23

24 Evaluation of the Stide Model to Detect Attacks Yes - indicates Detected, No - indicates Undetected (Giffin et al, 06) MITACS Seminar, 2007 Zincir-Heywood 24

25 Model Checking Running Times (Giffin et al, 06) MITACS Seminar, 2007 Zincir-Heywood 25

26 Design Requirements Developing OS model manually White box testing approach Expensive (knowledge) Limited semantic coverage Exhaustive search (constrained by the above item) using model checking What if the OS model abstraction is wrong? MITACS Seminar, 2007 Zincir-Heywood 26

27 Mimicry Attacks: Kayacik et al, 07a Automatic attack generation Black box testing Assumes vulnerable application is known Assumes a core attack is known Against Stide,, using traceroute application Search space too large to deploy exhaustive methods Genetic Programming employed MITACS Seminar, 2007 Zincir-Heywood 27

28 Methodology Motivations for using GP Goal based objectives Representation Intron Code Training data for traceroute Previous work employ: Traceroute nis.nsf nsf.net Fitness function for GP MITACS Seminar, 2007 Zincir-Heywood 28

29 Occurrence of System Calls (Kayacik et al, 07a) MITACS Seminar, 2007 Zincir-Heywood 29

30 Parameter Types (Kayacik et al, 07a) MITACS Seminar, 2007 Zincir-Heywood 30

31 Fitness Function (Kayacik et al, 07a) MITACS Seminar, 2007 Zincir-Heywood 31

32 Training Data (Kayacik et al, 07a) MITACS Seminar, 2007 Zincir-Heywood 32

33 Stide Anomaly Rates against Training Data (Kayacik et al, 07a) MITACS Seminar, 2007 Zincir-Heywood 33

34 Stide Anomaly Rates against Exploits (Kayacik et al, 07a) MITACS Seminar, 2007 Zincir-Heywood 34

35 Mismatch rates (%) Reported by Stide (Kayacik et al, 07b) MITACS Seminar, 2007 Zincir-Heywood 35

36 Contribution of Preamble (Kayacik et al, 07b) MITACS Seminar, 2007 Zincir-Heywood 36

37 Design Requirements Attack = preamble + exploit Anomaly rate should be calculated over both There is no attack with 0% anomaly even for the previous work when we analyze the whole attack Can work with any IDS - bb Evolutionary computation Efficient sampling of large search space Longer training times MITACS Seminar, 2007 Zincir-Heywood 37

38 Van Oorschot et al, 05 Hardware assisted circumvention of self- hashing software tamper resistance Attack generation against self-hashing technique on many modern processors (x86, UltraSparc,, AMD64, ARM ) White-box Assumes vulnerable application is known Manual generation MITACS Seminar, 2007 Zincir-Heywood 38

39 What s s next? White box vs black box testing Preamble vs exploit generation Dynamic vs static Deterministic vs stochastic Allergy attacks Co-evolution of attackers & detectors Theoretical modeling of the arms race Experimental results to explore the models MITACS Seminar, 2007 Zincir-Heywood 39

40 Why bother? To be able to predict To be a step a head if possible To understand attacker behavior To test defense systems before attackers To improve defense systems Automatic signature generation Automatic attack training data generation To generate anti-botnet teams :-) MITACS Seminar, 2007 Zincir-Heywood 40

41 One final thought When asked Vint Cerf told that there are 2 important events that started Internet s evolution: Launch of Sputnik Breakup of AT&T What about security: Bombing of 9/11?? MITACS Seminar, 2007 Zincir-Heywood 41

42 References Mutz D., Vigna G., Kemmerer R., An Experience Developing an IDS Stimulator for the Black-Box Testing of Network Intrusion Detection Systems, ACSAC, Vigna,, G., Robertson, W., Balzarotti D., Testing Network Based Intrusion Detection Signatures Using Mutant Exploits, ACM CCS, Kayacik H. G., Zincir-Heywood A. N., Heywood M. I., Evolving Successful Stack Overflow Attacks for Vulnerability Testing, ACSAC, Forrest S., Hofmeyr S. A., Somayaji A., Longstaff T. A., A sense of self for Unix processes, IEEE SP, Warrender C., Forrest S., Pearlmutter BA, Detecting intrusions using system calls: Alternative data models, IEEE SP, Wespi,, A., Dacier,, M., and Debar, H., Intrusion Detection Using Variable-Length Audit Trail Patterns, RAID, Sekar R., Bendre M., Dhurjati D., Bollineni P., A Fast Automation-based Method for Detecting Anomalous Program Behavior, IEEE SP, Feng H., Kolesnikov O., Fogla P., Lee W., Gong W., Anomaly detection using call stack information, IEEE SP, Gao D., Reiter M., Song D., Gray box extraction of execution graphs for anomaly detection, ACM CCS, Wagner D., Dean D., Intrusion detection via static analysis, IEEE SP, J. Giffin,, S. Jha,, and B. Miller. Detecting Manipulated Remote Call Streams, Usenix Security, MITACS Seminar, 2007 Zincir-Heywood 42

43 References J. Giffin,, S. Jha,, and B.P. Miller. Efficient context sensitive intrusion detection, NDSS, H. Feng,, J. Giffin,, Y. Huang, S. Jha,, W. Lee, B. Miller. Formalizing sensitivity in static analysis for intrusion detection, IEEE SP, L. Lam and T. Chiueh.. Automatic Extraction of Accurate Application-Specific Sandboxing Policy, RAID, Mutz D., Valeur F., Vigna G., Kruegel C., Anomalous System Call Detection, ACM Transactions on Information system and Security, Wagner D., Soto P., Mimicry attacks on host based intrusion detection systems, ACM CCS, Tan, K. M. C., Killourhy,, K. S., Maxion,, R. A., Undermining an Anomaly-based Intrusion Detection System using Common Exploits, RAID, D. Gao,, M. Reiter, and D. Song. On Gray-Box Program Tracking for Anomaly Detection, Usenix Security, Kruegel C., Kirda E., Mutz D., Robertson W., Vigna G., Automating mimicry attacks using static binary analysis, USENIX Security Symposium, Giffin J. T., Jha S., Miller BP, Autoated Discovery of Mimicry Attacks, RAID, Kayacik HG, Zincir-Heywood AN, Heywood MI, Automatically Evading IDS Using GP Authored Attacks, IEEE CISDA, 2007a. Kayacik HG, Zincir-Heywood AN, On the Contribution of Preamble to Information Hiding in Mimicry Attacks, IEEE SSNDS, 2007b. Van Oorschot PC, Somayaji A., Wurster G., Hardware Assisted circumvention of self hashing software tamper resistance, IEEE Transactions on Dependable and Secure Computing, MITACS Seminar, 2007 Zincir-Heywood 43

44 THANKS A LOT! ANY QUESTIONS? COMMENTS? MITACS Seminar, 2007 Zincir-Heywood 44