Network Security Today: Finding Complex Attacks at 100Gb/s

Transcription

1 : Finding Complex Attacks at 100Gb/s Robin Sommer International Computer Science Institute, & Lawrence Berkeley National Laboratory

2 The Old Days Border Traffic Lawrence Berkeley National Lab (Today) 10GE upstream, 4,000 user, 12,000 hosts Total connections 2

3 The Old Days Border Traffic Lawrence Berkeley National Lab (Today) 10GE upstream, 4,000 user, 12,000 hosts Total connections Successful connections Attempted connections 2

4 The Old Days Border Traffic Lawrence Berkeley National Lab (Today) 10GE upstream, 4,000 user, 12,000 hosts Total connections Successful connections Attempted connections 2

5 Today s Threats 3

6 Today s Threats Trend 1: Commercialization of attacks Thriving underground economy ( Crime-as-a-Service ). Bear Race: Attack is good enough if it pays. Source: Gary Larson 3

7 Today s Threats Trend 1: Commercialization of attacks Thriving underground economy ( Crime-as-a-Service ). Bear Race: Attack is good enough if it pays. Trend 2: High-skill / high-resource attacks. Activist Hacking. Advanced Persistent Threats / Nation-states. Source: Wikimedia Commons Source: Computer Security Articles Source: EFF 3

8 Today s Threats Trend 1: Commercialization of attacks Thriving underground economy ( Crime-as-a-Service ). Bear Race: Attack is good enough if it pays. Trend 2: High-skill / high-resource attacks. Activist Hacking. Advanced Persistent Threats / Nation-states. Trend 3: Insider Attacks Exfiltration Sabotage 3

9 Defender Challenges Varying threat models. No ring rules them all. 4

10 Defender Challenges Varying threat models. No ring rules them all. Semantic complexity. The action is really at the application-layer. 4

11 Defender Challenges Varying threat models. No ring rules them all. Semantic complexity. The action is really at the application-layer. Volume and variability. Network traffic is an enormous haystack. 4

12 Deep Packet Inspection at High Speed 5

13 Analyzing Semantics 6

14 Analyzing Semantics Internet Tap Internal Network IDS Example: Finding downloads of known malware. 6

15 Analyzing Semantics Internet Tap Internal Network IDS Example: Finding downloads of known malware. 1. Find and parse all Web traffic. 2. Find and extract binaries. 3. Compute hash and compare with database. 4. Report, and potentially kill, if found. 6

16 Back in 2005 TBytes/month Munich Scientific Network (2005) 3 major Total universities, bytes 1 GE upstream Incoming bytes ~100,000 Users ~50,000 Hosts Total upstream bytes Incoming bytes Data: Leibniz-Rechenzentrum, München 7

17 Back in 2005 TBytes/month Munich Scientific Network (Today) Total bytes Incoming bytes 3 major universities, 2x10GE upstream ~100,000 Users ~65,000 Hosts Total upstream bytes Incoming bytes Oct Data: Leibniz-Rechenzentrum, München 8

18 Traditional Gap: Research vs. Operations Conceptually simple tasks can be hard in practice. Academic research often neglects operational constraints. Operations cannot leverage academic results. We focus on working with operations. Close collaborations with several large sites. Extremely fruitful for both sides. 9

19 Research Platform: Bro 10

20 Research Platform: Bro Originally developed by Vern Paxson in Open-source, BSD-license, maintained at ICSI and NCSA. In operational use since the beginning. Conceptually very different from other IDS. 10

21 Architecture Packets Network 11

22 Architecture Events Protocol Decoding Event Engine Packets Network 11

23 Architecture Logs Notification Analysis Logic Script Interpreter Events Protocol Decoding Event Engine Packets Network 11

24 Architecture Logs Notification User Interface Analysis Logic Script Interpreter Events Protocol Decoding Event Engine Packets Network 11

25 Script Example: Matching URLs Task: Report all Web requests for a file passwd 12

26 Script Example: Matching URLs Task: Report all Web requests for a file passwd event http_request(c: connection, # Connection. method: string, # HTTP method. original_uri: string, # Requested URL. unescaped_uri: string, # Decoded URL. version: string) # HTTP version. { if ( method == "GET" && unescaped_uri == /.*passwd/) NOTICE(...); # Alarm. } 12

27 Script Example: Scan Detector Task: Count failed connection attempts per source address. 13

28 Script Example: Scan Detector Task: Count failed connection attempts per source address. global attempts: table[addr] of count &default=0; event connection_rejected(c: connection) { local orig = c$id$orig_h; # Get originator address. local n = ++attempts[orig]; # Increase counter. if ( n == SOME_THRESHOLD ) # Check for threshold. NOTICE(...); # Alarm. } 13

29 Who s Using It? Diverse Deployment Base Universities Research Labs Supercomputer Centers Government Organizations Fortune 20 Enterprises Examples Lawrence Berkeley National Lab National Center for Supercomputing Applications National Center for Atmospheric Research Indiana University... and many more sites Fully integrated into Security Onion Popular security-oriented Linux distribution Recent User Meetings Bro Workshops 2011/13 at NCSA Bro Exchange 2012 at NCAR Attended by about operators from from organizations 14

30 Bro History Vern writes 1st line of code

31 Bro History Vern writes 1st line of code LBNL starts using Bro operationally v0.2 1st CHANGES entry v0.4 HTTP analysis Scan detector IP fragments Linux support v0.6 RegExps Login analysis v0.7a90 Profiling State Mgmt v0.7a175/0.8ax Signatures SMTP IPv6 support User manual v0.8ax/0.9ax SSL/SMB STABLE releases BroLite v1.1/v1.2 when Stmt Resource tuning Broccoli DPD v1.0 BinPAC IRC/RPC analyzers 64-bit support Sane version numbers v1.5 BroControl v1.4 DHCP/BitTorrent HTTP entities NetFlow Bro Lite Deprecated Bro SDCI v2.0 New Scripts v2.2 File Analysis Summary Stat. v2.1 IPv6 Input Framew. Bro Center v0.7a48 Consistent CHANGES 0.8a37 Communication Persistence Namespaces Log Rotation v1.3 Ctor expressions GeoIP Conn Compressor

32 Bro History Host Context Time Machine Enterprise Traffic TRW State Mgmt. Independ. State Bro Cluster Shunt Academic Publications USENIX Paper Stepping Stone Detector Anonymizer Active Mapping Context Signat. BinPAC DPD 2nd Path Parallel Prototype Autotuning Input Framework Vern writes 1st line of code LBNL starts using Bro operationally v0.2 1st CHANGES entry v0.4 HTTP analysis Scan detector IP fragments Linux support v0.6 RegExps Login analysis v0.7a90 Profiling State Mgmt v0.7a175/0.8ax Signatures SMTP IPv6 support User manual v0.8ax/0.9ax SSL/SMB STABLE releases BroLite v1.1/v1.2 when Stmt Resource tuning Broccoli DPD v1.0 BinPAC IRC/RPC analyzers 64-bit support Sane version numbers v1.5 BroControl v1.4 DHCP/BitTorrent HTTP entities NetFlow Bro Lite Deprecated Bro SDCI v2.0 New Scripts v2.2 File Analysis Summary Stat. v2.1 IPv6 Input Framew. Bro Center v0.7a48 Consistent CHANGES 0.8a37 Communication Persistence Namespaces Log Rotation v1.3 Ctor expressions GeoIP Conn Compressor

33 Bro History Host Context Time Machine Enterprise Traffic TRW State Mgmt. Independ. State Bro Cluster Shunt Academic Publications Stepping Stone Example: USENIX Processing Paper Detector performance 1995 Anonymizer Active Mapping Context Signat. BinPAC DPD 2nd Path LBNL operations had trouble keeping up Research question: How can Bro scale up? Parallel Prototype Autotuning Input Framework Vern writes 1st line of code LBNL starts using Bro operationally v0.2 1st CHANGES entry v0.4 HTTP analysis Scan detector IP fragments Linux support v0.6 RegExps Login analysis v0.7a90 Profiling State Mgmt v0.7a175/0.8ax Signatures SMTP IPv6 support User manual v0.8ax/0.9ax SSL/SMB STABLE releases BroLite v1.1/v1.2 when Stmt Resource tuning Broccoli DPD v1.0 BinPAC IRC/RPC analyzers 64-bit support Sane version numbers v1.5 BroControl v1.4 DHCP/BitTorrent HTTP entities NetFlow Bro Lite Deprecated Bro SDCI v2.0 New Scripts v2.2 File Analysis Summary Stat. v2.1 IPv6 Input Framew. Bro Center v0.7a48 Consistent CHANGES 0.8a37 Communication Persistence Namespaces Log Rotation v1.3 Ctor expressions GeoIP Conn Compressor

34 Load-balancing Architecture 16

35 Load-balancing Architecture NIDS 10G Packet Analysis Detection Logic 16

36 External Packet Load-Balancer Load-balancing Architecture Flows NIDS 1 1G Packet Analysis Detection Logic NIDS 2 10G 1G Packet Analysis Detection Logic NIDS 3 1G Packet Analysis Detection Logic 16

37 External Packet Load-Balancer Load-balancing Architecture Flows NIDS 1 1G Packet Analysis Detection Logic Communication NIDS 2 10G 1G Packet Analysis Detection Logic Communication NIDS 3 1G Packet Analysis Detection Logic 16

38 External Packet Load-Balancer Load-balancing Architecture Flows NIDS 1 Bro Cluster 1G Packet Analysis Detection Logic Communication NIDS 2 10G 1G Packet Analysis Detection Logic Communication NIDS 3 1G Packet Analysis Detection Logic 16

39 A Production Load-Balancer 17

40 A Production Load-Balancer cflow: 10GE line-rate, stand-alone load-balancer 10 Gb/s in/out Web & CLI Filtering capabilities 17

41 A Production Load-Balancer cflow: 10GE line-rate, stand-alone load-balancer 10 Gb/s in/out Web & CLI Filtering capabilities 17

42 Next Stop: 100 Gb/s 2011 Now these sites need a monitoring solution... Working with cpacket on a 100GE loadbalancer DOE/ESNet 100G Advanced Networking Initiative Source: ESNet Source: ESNet 18

43 Next Stop: 100 Gb/s 2014 Source: ESNet 19

44 On Deck: 400G Connectivity Berkeley National Laboratory File System Links 2 x 100G Inter-site Traffic 100G Computational Research and Theory Building. WAN 100G WAN 100G Oakland Scientific Facility. Sources: ESNet/LBNL/NERSC 20

45 Science DMZ Internet 10G 10G 10G Campus LAN 21

46 Science DMZ Internet 100G 100G 100G Campus LAN 21

47 Science DMZ Internet 100G 10G 10G Campus LAN 21

48 Science DMZ Internet 100G 10G 10G 100G Campus LAN Science DMZ Switch 100G Transfer/Storage Nodes 21

49 Science DMZ Internet 100G 10G 10G Clean, highbandwith path 100G Low-bandwidth campus access Campus LAN Science DMZ Switch 100G Transfer/Storage Nodes 21

50 Science DMZ Internet 100G 10G 10G 10G 100G Campus LAN Science DMZ Switch 100G Transfer/Storage Nodes 22

51 Science DMZ Internet 100G 10G 10G 10G 100G Campus LAN Science DMZ Switch 100G 100G Transfer/Storage Nodes 22

52 100G Bro Cluster Science DMZ Switch 100G 23

53 100G Bro Cluster Science DMZ Switch 100G 100G Load-balancer 23

54 100G Bro Cluster Science DMZ Switch 100G 100G Load-balancer 10G 23

55 100G Bro Cluster Science DMZ Switch 100G 100G Load-balancer 10G Bro Cluster 23

56 100G Bro Cluster Science DMZ Switch 100G 100G Load-balancer API 10G Control Bro Cluster 23

57 100G Bro Cluster Science DMZ Switch API 100G 100G Load-balancer API 10G Control Control Bro Cluster 23

58 100G Bro Cluster Science DMZ Switch API 100G 100G Load-balancer API 10G Control Control Bro Cluster 23

59 Parallelizing DPI on Multi-core Systems 24

60 Going Multi-Core Bro is single-threaded Cluster backends have muitple cores, mostly idle. Work-around: Cluster in a box We really want multi-threading, though. Needs to scale well with increasing numbers of cores. Needs to be transparent to the operator. For some IDS, that s not so hard. For others, it is... 25

61 Concurrent Analysis Logs Notification Analysis Logic Script Interpreter Events Protocol Decoding Event Engine Packets Network 26

62 Concurrent Analysis Logs Notification Single Thread Analysis Logic Script Interpreter Events Protocol Decoding Event Engine Packets Network 26

63 Concurrent Analysis Notification Detection Logic Scripting Language Script Threads Events Packet Analysis Event Engine Event Engine Threads Packets Dispatcher Kernel or NIC Network 27

64 Concurrent Analysis Notification Detection Logic Scripting Language Script Threads Events Packet Analysis Event Engine Event Engine Threads Cluster in a Box Packets Dispatcher Kernel or NIC Network 27

65 Concurrent Analysis How to parallelize a scripting language? Notification Detection Logic Scripting Language Script Threads Events Packet Analysis Cluster in a Box Event Engine Packets Dispatcher Kernel or NIC Event Engine Threads Network 27

66 How to Parallelize Event Handlers? Simple: State-less Analysis 28

67 How to Parallelize Event Handlers? Simple: State-less Analysis event http_request(c: connection, # Connection. method: string, # HTTP method. original_uri: string, # Requested URL. unescaped_uri: string, # Decoded URL. version: string) # HTTP version. { if ( method == "GET" && unescaped_uri == /.*passwd/) NOTICE(...); # Alarm. } 28

68 How to Parallelize Event Handlers? (2) Challenging: Analysis that keeps global state. 29

69 How to Parallelize Event Handlers? (2) Challenging: Analysis that keeps global state. global attempts: table[addr] of count &default=0; event connection_rejected(c: connection) { local orig = c$id$orig_h; # Get originator address. local n = ++attempts[orig]; # Increase counter. if ( n == SOME_THRESHOLD ) # Check for threshold. NOTICE(...); # Alarm. } 29

70 Parallelizing Event Execution attempts[addr] of count addr count connection_rejected(c): s = c.originator ++attempts[s] 30

71 Parallelizing Event Execution attempts[addr] of count addr count connection_rejected(c): # s = c.originator ++attempts[s] 30

72 Parallelizing Event Execution attempts[addr] of count addr count Thread 1 connection_rejected(c): # s = c.originator ++attempts[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts[s] Thread 3 connection_rejected(c): # s = c.originator ++attempts[s] 30

73 Parallelizing Event Execution attempts[addr] of count addr count Thread 1 connection_rejected(c): # s = c.originator LOCK(attempts) ++attempts[s] UNLOCK(attempts) Thread 2 connection_rejected(c): # s = c.originator LOCK(attempts) ++attempts[s] UNLOCK(attempts) Thread 3 connection_rejected(c): # s = c.originator LOCK(attempts) ++attempts[s] UNLOCK(attempts) 30

74 Parallelizing Event Execution attempts[addr] of count addr count Thread 1 connection_rejected(c): # s = c.originator ++attempts[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts[s] Thread 3 connection_rejected(c): # s = c.originator ++attempts[s] 30

75 Parallelizing Event Execution attempts_1 attempts_2 attempts_3 attempts[addr] of count addr count Thread 1 connection_rejected(c): # s = c.originator ++attempts[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts[s] Thread 3 connection_rejected(c): # s = c.originator ++attempts[s] 30

76 Parallelizing Event Execution attempts_1 attempts_2 attempts_3 attempts[addr] of count addr count Thread 1 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_1[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_2[s] Thread 3 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_3[s] 30

77 Parallelizing Event Execution attempts_1 attempts_2 attempts_3 attempts[addr] of count addr count hash: addr -> {1, 2,3} hash(addr) Thread 1 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_1[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_2[s] Thread 3 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_3[s] 30

78 Parallelizing Event Execution attempts_1 attempts_2 attempts_3 attempts[addr] of count addr count hash: addr -> {1, 2,3} hash(addr) Thread 1 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_1[s] ++attempts_(hash(s))[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_2[s] ++attempts_(hash(s))[s] Thread 3 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_3[s] ++attempts_(hash(s))[s] 30

79 Parallelizing Event Execution attempts_1 attempts_2 attempts_3 attempts[addr] of count addr count hash: addr -> {1, 2,3} hash(addr) Thread 1 connection_rejected(c): Thread hash(s) # s = c.originator ++attempts[s] ++attempts_1[s] ++attempts_(hash(s))[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_2[s] ++attempts_(hash(s))[s] Thread hash(s) Thread hash(s) Thread 3 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_3[s] ++attempts_(hash(s))[s] 30

80 Parallelizing Event Execution Thread 1 s attempts Thread 2 s attempts Thread 3 s attempts attempts[addr] of count addr count hash: addr -> {1, 2,3} hash(addr) Thread 1 connection_rejected(c): Thread hash(s) # s = c.originator ++attempts[s] ++attempts_1[s] ++attempts_(hash(s))[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_2[s] ++attempts_(hash(s))[s] Thread hash(s) Thread hash(s) Thread 3 connection_rejected(c): # s = c.originator ++attempts[s] ++attempts_3[s] ++attempts_(hash(s))[s] 30

81 Parallelizing Event Execution Thread 1 s attempts Thread 2 s attempts Thread 3 s attempts attempts[addr] of count addr count hash: addr -> {1, 2,3} hash(addr) Thread 1 connection_rejected(c): Thread hash(s) # s = c.originator ++attempts_1[s] ++attempts_(hash(s))[s] ++attempts[s] Thread 2 connection_rejected(c): # s = c.originator ++attempts_2[s] ++attempts_(hash(s))[s] ++attempts[s] Thread hash(s) Thread hash(s) Thread 3 connection_rejected(c): # s = c.originator ++attempts_3[s] ++attempts_(hash(s))[s] ++attempts[s] 30

82 Parallel Event Scheduling 31

83 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n 31

84 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A conn_rejected 31

85 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A conn_rejected Orig A conn_rejected 31

86 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A conn_rejected Orig A conn_rejected Orig B conn_rejected 31

87 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A Orig A Orig B Conn X conn_rejected conn_rejected conn_rejected http_request 31

88 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A Orig A Orig B Conn X Conn Y conn_rejected conn_rejected conn_rejected http_request http_request 31

89 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A Orig A Orig B Conn X Conn Y Conn conn_rejected conn_rejected conn_rejected http_request http_request http_reply 31

90 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A Orig A Orig B Conn X Conn Y Conn Conn Y conn_rejected conn_rejected conn_rejected http_request http_request http_reply http_reply 31

91 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A Orig A Orig B Conn X Conn Y Conn Conn Y Orig A conn_rejected conn_rejected conn_rejected http_request http_request http_reply http_reply conn_rejected 31

92 Parallel Event Scheduling Thread 1 Thread 2 Thread 3 Thread 4 Thread n Orig A Orig A Orig B Conn X Conn Y Conn Conn Y Orig A conn_rejected conn_rejected conn_rejected http_request http_request http_reply http_reply conn_rejected Challenge: Implementing this 31

93 New Platform: Abstract Machine A High-Level Intermediary Language for Traffic Inspection 32

94 New Platform: Abstract Machine A High-Level Intermediary Language for Traffic Inspection Domain-specific Data Types State Management Concurrent Analysis Real-time Performance Robust/Secure Execution High-level Standard Components First-class networking types built-in Containers with state management support Domain-specific concurrency model Scalability through parallelization Well-defined, contained execution environment Platform for building high-level, reusable functionality on Timers can drive execution Support for incremental processing Compilation to native code Static type-system, and robust error handling Extensive optimization potential 32

95 New Platform: Abstract Machine A High-Level Intermediary Language for Traffic Inspection Domain-specific Data Types State Management Concurrent Analysis Real-time Performance Robust/Secure Execution High-level Standard Components First-class networking types built-in Containers with state management support Domain-specific concurrency model Scalability through parallelization Well-defined, contained execution environment Platform for building high-level, reusable functionality on Timers can drive execution Support for incremental processing Compilation to native code Static type-system, and robust error handling Extensive optimization potential 32

96 Summary 33

97 Conclusions Threats have changed. Detection requires deep, flexible, semantic analysis. Working to push the limits. Leverage capabilities of modern network hardware. Exploit parallelism inherent in network traffic analysis. Bro is an ideal platform for such work. Operationally deployed across the country. Bridges traditional gap between academia and operations. 34

98 Thanks for you attention Robin Sommer International Computer Science Institute, & Lawrence Berkeley National Laboratory 35