On the Robustness of Random Walk Algorithms for the Detection of Unstructured P2P Botnets

Transcription

1 On the Robustness of Random Walk Algorithms for the Detection of Unstructured P2P Botnets Dominik Muhs 1 1 Stefen Haas 2 Technische Universität Dresden Dresden, Germany firstllast@tuddresdenlde Thorsten Strufe 1 Mathias Fischer 2 Universität Hamburg Hamburg, Germany firstllast@informatislunidhamburglde 2

2 Outline [7] 2

3 Outline Il Motivation [7] 3

4 Outline Il Motivation IIl Botnets 1l Definition 2l Graph Model [7] 4

5 Outline Il Motivation IIl Botnets 1l Definition 2l Graph Model IIIl Random Walss [7] 5

6 Outline Il Motivation IIl Botnets 1l Definition 2l Graph Model IIIl Random Walss IVlAnalysis and Detection [7] 6

7 Outline Il Motivation IIl Botnets 1l Definition 2l Graph Model IIIl Random Walss IVlAnalysis and Detection Vl Limiting Knowledge [7] 7

8 Outline Il Motivation IIl Botnets 1l Definition 2l Graph Model IIIl Random Walss IVlAnalysis and Detection Vl Limiting Knowledge VIlResults [7] 8

9 Outline Il Motivation IIl Botnets 1l Definition 2l Graph Model IIIl Random Walss IVlAnalysis and Detection Vl Limiting Knowledge VIlResults [7] VIIlConclusion 9

10 [1] 10

11 [2] 11

12 [3] 12

13 [4] 13

14 [5] 14

15 [6] 15

16 What are Botnets? [9] 16

17 What are Botnets? Device collection [9] 17

18 What are Botnets? Device collection Internetdconnected [9] 18

19 What are Botnets? Device collection Internetdconnected Malwaredinfected [9] 19

20 What are Botnets? Device collection Internetdconnected Malwaredinfected Remotely controlled (usually centralized) [9] 20

21 Why are Botnets bad? [9] 21

22 Why are Botnets bad? Clicsfraud [9] 22

23 Why are Botnets bad? Clicsfraud Spam [9] 23

24 Why are Botnets bad? Clicsfraud Spam DDoS attacss [9] 24

25 Why are Botnets bad? Clicsfraud Spam DDoS attacss Cryptocurrency mining [9] 25

26 Why are Botnets bad? Clicsfraud Spam DDoS attacss Cryptocurrency mining Intellectual property theft [9] 26

27 Topological Categories Centralized [8] 27

28 Topological Categories Centralized Decentralized [8] 28

29 Topological Categories Centralized Decentralized Structured Unstructured [8] 29

30 Centralized Botnets 30

31 Centralized Botnets Central C2 server 31

32 Centralized Botnets Central C2 server Star topology 32

33 Centralized Botnets Central C2 server Star topology IRC/HTTP/ 33

34 Centralized Botnets Central C2 server Star topology IRC/HTTP/ Single point of failure 34

35 Structured P2P Botnets 35

36 Structured P2P Botnets No C2 server 36

37 Structured P2P Botnets No C2 server Hard to tase down 37

38 Structured P2P Botnets No C2 server Hard to tase down Specific rule set 38

39 Structured P2P Botnets No C2 server Hard to tase down Specific rule set Kademlia, Chord 39

40 Unstructured P2P Botnets 40

41 Unstructured P2P Botnets Randomized 41

42 Unstructured P2P Botnets Randomized Evade topological matching 42

43 Unstructured P2P Botnets Randomized Evade topological matching Statistical methods necessary 43

44 Existing Approaches [7] 44

45 Existing Approaches Leverage graph models [7] 45

46 Existing Approaches Leverage graph models and random walss [7] 46

47 Existing Approaches Leverage graph models and random walss [7] 47

48 Existing Approaches Leverage graph models and random walss Focus on structured botnets [10, 11, 12] [7] 48

49 Existing Approaches Leverage graph models and random walss Focus on structured botnets [10, 11, 12] Do not use open technologies [7] 49

50 Existing Approaches Leverage graph models and random walss Focus on structured botnets [10, 11, 12] Do not use open technologies Often assume complete snowledge on botnet communication [7] 50

51 Our Approach 51

52 Our Approach Leverages random walss 52

53 Our Approach Leverages random walss Uses opendsource technologies 53

54 Our Approach Leverages random walss Uses opendsource technologies Tested on unstructured botnets 54

55 Our Approach Leverages random walss Uses opendsource technologies Tested on unstructured botnets Precise when information is limited 55

56 Our Approach Leverages random walss Uses opendsource technologies Tested on unstructured botnets Precise when information is limited Can be combined with other approaches 56

57 Communication Graph 57

58 Communication Graph No payload data needed 58

59 Communication Graph No payload data needed Networs operator s view 59

60 Communication Graph No payload data needed Networs operator s view Aggregated NetFlow data 60

61 Communication Graph No payload data needed Networs operator s view Aggregated NetFlow data Idea: extract welld connected subgraph 61

62 Communication Graph No payload data needed Networs operator s view Aggregated NetFlow data Idea: extract welld connected subgraph Approach: Random Walss 62

63 G L =(V L, E L ) 63

64 k=0 64

65 k=1 65

66 k=2 66

67 k=3 67

68 k=4 68

69 Probability Distribution 69

70 Probability Distribution n=10,000 walss 70

71 Probability Distribution n=10,000 walss Of length k=3 71

72 Probability Distribution n=10,000 walss Of length k=3 With loss l=0.5 72

73 Probability Distribution n=10,000 walss Of length k=3 With loss l=0.5 Fastdmixing artifact 73

74 The Analysis Pipeline 74

75 The Analysis Pipeline Aggregate NetFlow data (Python 3l6, networkx) 75

76 The Analysis Pipeline Aggregate NetFlow data (Python 3l6, networkx) Evaluation steps: Botnet node mapping 76

77 The Analysis Pipeline Aggregate NetFlow data (Python 3l6, networkx) Evaluation steps: Botnet node mapping Apply loss functions 77

78 The Analysis Pipeline Aggregate NetFlow data (Python 3l6, networkx) Evaluation steps: Botnet node mapping Apply loss functions Execute random walss (numpy) 78

79 The Analysis Pipeline Aggregate NetFlow data (Python 3l6, networkx) Evaluation steps: Botnet node mapping Apply loss functions Execute random walss (numpy) Normalize resulting probability distribution 79

80 The Analysis Pipeline Aggregate NetFlow data (Python 3l6, networkx) Evaluation steps: Botnet node mapping Apply loss functions Execute random walss (numpy) Normalize resulting probability distribution Cluster wals destinations (DBSCAN) 80

81 The Test Dataset 81

82 The Test Dataset CTU11 from Czech Technical University 82

83 The Test Dataset CTU11 from Czech Technical University ZA24 ZeroAccess communication graph 83

84 Loss Strategies

85 Loss Strategies Other approaches do not evaluate limited networs view

86 Loss Strategies Other approaches do not evaluate limited networs view Unrealistic assumptions: All communication relationships captured

87 Loss Strategies Other approaches do not evaluate limited networs view All communication relationships captured Complete botnet in snown networs 0 4 Unrealistic assumptions:

88 Loss Strategies Other approaches do not evaluate limited networs view 15 All communication relationships captured Complete botnet in snown networs Solution: Simulate loss on communication graph Unrealistic assumptions:

89 Random Botnet Edge Deletion

90 Random Botnet Edge Deletion Random subset of botnet edges

91 Random Botnet Edge Deletion Random subset of botnet edges

92 Random Botnet Edge Deletion Random subset of botnet edges 16 Outdofdview connections

93 Random Botnet Edge Deletion Random subset of botnet edges Outdofdview connections ISPdrelated loss (elgl 1:256 sampling)

94 RBED Robustness 94

95 RBED Robustness Random Botnet Edge Deletion 95

96 RBED Robustness Random Botnet Edge Deletion 96

97 RBED Robustness Random Botnet Edge Deletion 97

98 RBED Robustness Random Botnet Edge Deletion 90% loss 83% precision 98

99 Host-based Visibility

100 Host-based Visibility Sensor deployment

101 Host-based Visibility Sensor deployment 16 Randomly chosen

102 Host-based Visibility Sensor deployment 16 Randomly chosen

103 Host-based Visibility Sensor deployment 16 Randomly chosen 0 4 No communication between unmonitored hosts

104 Host-based Visibility Sensor deployment 16 Randomly chosen 0 4 No communication between unmonitored hosts Honeypot scenario

105 Sensor-Network Robustness Sensor deployment 105

106 Sensor-Network Robustness Sensor deployment 106

107 Sensor-Network Robustness Sensor deployment 107

108 Sensor-Network Robustness Sensor deployment 25 sensors 90% precision 108

109 Conclusion [7] 109

110 Conclusion Structured and unstructured botnets: fastdmixing [7] 110

111 Conclusion Structured and unstructured botnets: fastdmixing Highdprecision detection 83% precision [7] 111

112 Conclusion Structured and unstructured botnets: fastdmixing Highdprecision detection 83% precision With 90% missing edges [7] 112

113 Conclusion Structured and unstructured botnets: fastdmixing Highdprecision detection 83% precision With 90% missing edges Simple architecture [7] 113

114 Conclusion Structured and unstructured botnets: fastdmixing Highdprecision detection 83% precision With 90% missing edges Simple architecture Only opendsource algorithms [7] 114

115 [7] Thanss! Questions? 115

116 References [1] [2] [3] [4] [5] [6] [7] Icon made by Freepik from [8] Icon made by ddara from [9] Icon made by Kiranshastry from [10] Shishir Nagaraja et al. BotGrep: fnding P2P bots with structured graph analysis. In: USENIX Security Symposium. 2010, p. 7. [11] Pratik Narang et al. PeerShark: Detecting peer-to-peer botnets by tracking conversations. In: Proceedings IEEE Symposium on Security and Privacy. Vol. January , pp [12] Guofei Gu, Junjie Zhang, and Wenke Lee. BotSnifer : Detecting Botnet Command and Control Channels in Network Trafc. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium (2008), pp