Spamming Botnets: Signatures and Characteristics

Transcription

1 Spamming Botnets: Signatures and Characteristics Himanshu Jethawa Department of Computer Science Missouri University of Science and Technology October/ pdf 9 Nov 2016 Network Security Presentation 2016 Himanshu Jethawa

2 Introduction AutoRE Presentation Overview Outline Automatic URL Regular Expression Generation Results Botnet Validation Conclusion 9 Nov 2016 Network Security Presentation 2

3 Introduction AutoRE Presentation Overview Introduction Automatic URL Regular Expression Generation Results Botnet Validation Conclusion 9 Nov 2016 Network Security Presentation 3

4 Introduction Botnets have been widely used for sending spam s Goal Perform large scale analysis of spamming botnet characteristics Identify trends that can benefit future botnet detection and defense mechanisms Detect botnet membership Track sending behavior 9 Nov 2016 Network Security Presentation 4

5 Knowing botnet members allows to know future nefarious activities such as DDoS Attacks Phishing Spam s often contain multiple URLS, some are legitimate Use of AutoRE framework Identifies botnet hosts Introduction Continued Generates botnet spam signatures from s 9 Nov 2016 Network Security Presentation 5

6 Introduction AutoRE Presentation Overview Outline Automatic URL Regular Expression Generation Results Botnet Validation Conclusion 9 Nov 2016 Network Security Presentation 6

7 AutoRE Figure 9 Nov 2016 Network Security Presentation 7

8 AutoRE URL Pre-Processing Input: Untagged s(spam/not spam) Extracts URL string, source server IP address and sending time URLs are then grouped based on their web domains 9 Nov 2016 Network Security Presentation 8

9 AutoRE URL Group Selection Which group characterizes an underlying spam campaign? Selection made on bursty property Si(n)- number of IP addresses which send atleast one URL in ith group during nth-window Sharp spikes in Si(n) used to rank URL groups 9 Nov 2016 Network Security Presentation 9

10 AutoRE Signature Generation and Botnet Identification For a group of URLs from a domain, two types of signatures generated: Complete URL based signature Regular expression signature In both cases, signatures are required to satisfy properties: Bursty (5 days, no discarding) Distributed (20 ASes) Specific (information entropy metric for matching for random URL with a signature) 9 Nov 2016 Network Security Presentation 10

11 AutoRE Regular Expression Generator Used for generation of regular expression from polymorphic URLs Input: Set of polymorphic URLs Modules: Signature Tree Construction Regular Expression Generation Signature Quality Evaluation 9 Nov 2016 Network Security Presentation 11

12 AutoRE Regular Expression Generator Signature Tree Construction 9 Nov 2016 Network Security Presentation 12

13 AutoRE Regular Expression Generator Detailing: Domain specific regular expression Generalization: Conversion of domain-specific regular expression to domain-agnostic one 9 Nov 2016 Network Security Presentation 13

14 AutoRE Regular Expression Generator Signature Quality Evaluation Measure quality of signature Discard too general signatures Entropy reduction Quantify probability of random string matching signature B(u) Be(u) AutoRE discards signatures whose entropy reductions are smaller than a preset threshold 9 Nov 2016 Network Security Presentation 14

15 Introduction AutoRE Presentation Overview Outline Automatic URL Regular Expression Generation Results Botnet Validation Conclusion 9 Nov 2016 Network Security Presentation 15

16 Results Dataset Hotmail messages but not from those Ips blacklisted by SpamHaus Identified 7,721 botnet-based spam campaigns 580,466 spam messages sent from 340,050 distinct botnet host IP addresses spanning 5,916 ASes. 9 Nov 2016 Network Security Presentation 16

17 Results Graphs Figure shows comparison of number of regular expressions before generalization and after generalization for three different months It is evident and obvious that after generalization, number declines Conversion from domain-specific to domain-agnostic regular expressions 9 Nov 2016 Network Security Presentation 17

18 Results Graphs Percentage of spam generated by botnets out of the of total spams is shown in the figure Simple URLs are majority in the total number of spams by botnets around 80% Spams based on regular expressions are around 20% Percentage of spam captured by AutoRE signatures 9 Nov 2016 Network Security Presentation 18

19 Introduction AutoRE Presentation Overview Outline Automatic URL Regular Expression Generation Results Botnet Validation Conclusion 9 Nov 2016 Network Security Presentation 19

20 Validation Hotmail s are already tagged by users as spam/non-spam The labels generated by AutoRE are compared with the labels by users In the figure: CU- Complete URL RE- Regular expression Data for three different months was taken False positive rate of AutoRE signatures- fraction of non-spam s matching signature out of total non-spam s 9 Nov 2016 Network Security Presentation 20

21 Validation It is evident from the figure that False Positive Rate(Regular Expression) << False Positive Rate(Conjunction of frequent keywords) Comparison of approaches 9 Nov 2016 Network Security Presentation 21

22 Validation It is evident that after generalization, number of spam s captured were more compared to before Number of spams captured by RE signatures 9 Nov 2016 Network Security Presentation 22

23 Conclusion Botnet hosts are widespread across the internet detecting and blacklisting individual botnet host will continue to remain a challenging task Demonstration of feasibility of detecting botnet hosts using botnet spam signatures Comparison of data from different time period concludes that botnets are getting increasingly sophisticated 9 Nov 2016 Network Security Presentation 23

24 References Xie, Yinglian, et al. "Spamming botnets: signatures and characteristics." ACM SIGCOMM Computer Communication Review 38.4 (2008): Nov 2016 Network Security Presentation 24

25 THANK YOU 9 Nov 2016 Network Security Presentation 25