The Importance of Interpretability in Cyber Security Analytics

Transcription

1 The Importance of Interpretability in Cyber Security Analytics Juston Moore Advanced Research in Cyber Systems Group October 4, 2017 Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA LA-UR

2 Advanced Research in Cyber Systems, LANL Cyber research group at large DOE national research laboratory Identify gaps, emerging threats Data collection and sensors Modelling and simulation Tool/product development Our Prospective: We assume an initial compromise will inevitably happen. Can we detect and mitigate attacks within the network perimeter effectively? Oct. 4, 2017 #

3 Data Sources for Cyber Research To motivate a larger research effort focused on operational cyber data LANL have released three publicly available datasets: 9 month time-series user/computer bipartite, day comprehensive, day Netflow and Window Event Logs, Caveat: Human subject research implications 10/20/17 3

4 2017 Dataset Overview 10/20/17 4

5 Modeling challenges Where collected, not necessarily for cybersecurity purposes Debugging Where collected, not intended for analytics Primary purpose is to support human operations Not useful unless combined with other data sets Collection is often incomplete 10/20/17 5

6 Modeling Periodic Activity in Network Data Raw authentication events for one user Changepoints for one user Oct. 4, 2017 #

7 Relational User Behavior Analytics REDUCE: Statistically-Guided Reverse Engineering MalNet: Deep Learning for Cross-Compiler Code Similarity Search

8 Relational User Behavior Analytics REDUCE: Statistically-Guided Reverse Engineering MalNet: Deep Learning for Cross-Compiler Code Similarity Search

9 LANL Cyber Data Data is collected for security purposes from all LANL Windows machines LANL requires methods to monitor the network for malicious intrusions Important events include authentications and processrunning activities 10/20/17 9

10 Internal Red Team Identification Oct. 4, 2017 #

11 Compromised Account Detection 10/20/17 11

12 User Behavior Analysis Human Resources Employee Records Project Management Authentication SharePoint Software Engineering Development Oct. 4, 2017 #

13 Behavior-Based User Profiling user activity item popularity preference attribute Cluster 1 Cluster 2 Storage counts Cluster 3 Stats Wiki Ext. Chat Emai l Vide o Chat Oct. 4, 2017 #

14 Red Team Identification Peer-based anomaly detection via collaborative filtering Oct. 4, 2017 #

15 Relational User Behavior Analytics REDUCE: Statistically-Guided Reverse Engineering MalNet: Deep Learning for Cross-Compiler Code Similarity Search

16 Detection Eventually Fails for Malware Endpoint Agents Firewalls Missed Samples Large-Scale Processing & Detection AntiVirus Machine Learning Detectors Missed Sample 1 Manual Malware Analysis Missed Sample 2 Missed Sample 3 Oct. 4, 2017 #

17 Tools for Malware Similarity Analysis Tools exist for: Large-scale sample-level similarity search Kam1n0, VXClass, multiple tools from DARPA Cyber Genome Comparison between two samples BinDiff, Diaphora Sharing information between malware analysts regarding one sample CollabREate There is currently a lack of tools to identify shared code within a known malware family, and to share this information among a team 10/20/17 17

18 REDUCE System Overview REDUCE User Interface New Samples YARA Signatures Statistically Guided Signature Generation Malware Samples REDUCE Static Code Analysis Code Similarity Search Visual Similarity Exploration Clean Samples Collaborative Code Commenting 10/20/17 18

19 Yara Signatures Efficient signatures for large-scale deployment Malware repository support: VirusTotal, LANL s CodeVision, Sandia s FARM Network-based defenses: bro, PaloAlto, Host-based defenses: CarbonBlack, Tanium, Translation for MIR, Encase, and other IOC formats Especially effective for in-memory search 10/20/17 19

20 Yara Signature Compilation 10/20/17 20

21 REDUCE Increases Situational Awareness REDUCE developed at Los Alamos Jan-13 Jul-13 Jan-14 Jul-14 Jan-15 Jul-15 10/20/17 21

22 Finding Unique APT Code REDUCE takes a simple approach and looks for function-level similarity Remove memory references, registers Match assembly opcodes Identify unique functions using Mutual Information This basic technique works surprisingly well for a large range of APTs, but it is easily defeated by simple code modifications We re now deploying Locality Sensitive Hash random projections to identify similar functions with a few assembly-level modifications We ll look at obfuscating compilers in just a few seconds 10/20/17 22

23 Interactive Signature Generation Prioritized list of indicators Signature effectiveness is displayed in realtime 10/20/17 23

24 Seeing Selected Features in Context 10/20/17 24

25 Collaborative Code Commenting 10/20/17 25

26 DHS Transition to Practice Program We are actively seeking partners to pilot REDUCE! 10/20/17 26

27 Relational User Behavior Analytics REDUCE: Statistically-Guided Reverse Engineering MalNet: Deep Learning for Cross-Compiler Code Similarity Search

28 APT6 Compiler Change mov bl, [eax] mov cl, [eax] movzx edx, byte ptr [eax] movzx edx, byte ptr [eax] mov dl, [eax-1] mov dl, [eax-1] movzx ecx, byte ptr [eax-1] movzx ecx, byte ptr [eax-1] shl dl, 2 shl dl, 2 sar dl, 4 sar dl, 4 sar bl, 4 sar cl, 4 and dl, 3 and dl, 3 and bl, 3 and cl, 3 add cl, cl add cl, cl add dl, bl add dl, cl add cl, cl add cl, cl mov [esi-1], dl mov [esi-1], dl add dl, cl add dl, cl mov dl, [eax+1] mov dl, [eax+1] mov [esi-1], dl mov [esi-1], dl mov bl, [eax] mov cl, [eax] movzx edx, byte ptr [eax+1] movzx edx, byte ptr [eax+1] sar dl, 2 sar dl, 2 movzx ecx, byte ptr [eax] movzx ecx, byte ptr [eax] and dl, 0Fh and dl, 0Fh sar dl, 2 sar dl, 2 shl bl, 4 shl cl, 4 and dl, 0Fh and dl, 0Fh xor dl, bl xor dl, cl shl cl, 4 shl cl, 4 mov [esi], dl mov [esi], dl xor dl, cl xor dl, cl Oct. 4, 2017 #

29 APT6 Compiler Change Timeline Oct. 4, 2017 #

30 Approach Goal: A code index will allow analysts to find similarities and differences across vast malware collections at the source code level We learn function embeddings using recurrent neural networks, which are are highly effective for human language translation We apply a bidirectional recurrent network to machine code found in compiled binaries Input Layer Output Backward Pass LSTM Cell Forward Level 2 LSTM Cell Forward Level 1 Inst 1 LSTM Cell Backward Level 1 LSTM Cell Backward Level 2 LSTM Cell Forward Level 2 LSTM Cell Forward Level 1 Inst 2 LSTM Cell Backward Level 1 LSTM Cell Backward Level 2 LSTM Cell Forward Level 2 LSTM Cell Forward Level 1 Inst n-1 LSTM Cell Backward Level 1 LSTM Cell Backward Level 2 LSTM Cell Forward Level 2 LSTM Cell Forward Level 1 Inst n LSTM Cell Backward Level 1 LSTM Cell Backward Level 2 Output Forward Pass Input Layer 10/20/17 30

31 Model We train our embeddings to capture similarity between malware functions using a Siamese network design Concatenate to vector Concatenate to vector Function 1 recurrent network Output Forward Pass Output Backward Pass Output Forward Pass Output Backward Pass Function 2 recurrent network Left and right networks are identical Function 1 Embedding Function 2 Embedding Compute Embedding Vector Distance Similar Functions: Minimize distance Dissimilar Functions: Maximize distance Oct. 4, 2017 #

32 Obfuscator-LLVM Instruction Substitution is a serious pain for distance metrics that count the usage of different assembly instructions! a = b + c Ex: r = rand(); a = b + r; a = a + c; a = a - r a = b & c (b ^ ~c) & b a = b c (b & c) (b ^ c) a = b ^ c (~b & c) (b & ~c) 10/20/17 32

33 Obfuscator-LLVM Control Flow Flattening Oct. 4, 2017 #

34 Obfuscator-LLVM Bogus Control Flow 10/20/17 34

35 Experimental Dataset We compiled several open-source software libraries with various compilers, optimizations, and code obfuscation techniques 10/20/17 35

36 Performance: Accuracy We significantly outperformed simple distance-based similarity search for identifying similar source codes 10/20/17 36

37 Performance by Query Compiler 10/20/17 37

38 Challenge Problem: the movfuscator! mov s are Turing-complete. So let s compile EVERYTHING to mov s! This certainly breaks existing instruction-based similarity search 10/20/17 38

39 Challenge Problem: the movfuscator! mov s are Turing-complete. So let s compile EVERYTHING to mov s! Conditional execution also breaks current static control flow analysis 10/20/17 39

40 Thank You! Cyber Physical Protect Infrastructure Modification Signature Deployment PHYSICAL WORLD SENSE Data Analytics Network Security Analyze Human Analyst Statistical Exploration Detect Rule-Based Systems Machine Learning Monitor Network Host A-4 Hierarchy of Security DIGITAL WORLD INPUT COMPUTE PHYSICAL WORLD ACT DIGITAL WORLD OUTPUT Juston Moore