Analyzing and Modeling Longitudinal Security Data: Promises and Pitfalls. Benjamin Edwards, Steven Hofmeyr, Stephanie Forrest, and Michel van Eeten

Transcription

1 Analyzing and Modeling Longitudinal Security Data: Promises and Pitfalls Benjamin Edwards, Steven Hofmeyr, Stephanie Forrest, and Michel van Eeten

2 What security interventions have a long-term impact on malicious activity?

3 Security Research Vulnerabilities, Defenses, and Measurement ACSAC Vulnerabilities 34 Defenses 13 Measurement

4 Analysis of a Long-term Problem: Spam Nearly 40 year old problem $20B-$50B cost to US companies 1 > 50% of messages sent in Major vector for malicious activity Malware Phishing (Target breach) 1 2 Rao and Reiley. The Economics of Spam 2012 Kaspersky 2015 Q3 Spam Report

5 Spam Interventions Botnet Takedowns Real-time Adaptive Blacklisting Value Chain Interventions

6 Global Data: The Spam Trap 10 Years Data 2 Billion Messages 440 Million Source IPs 659 Autonomous Systems 260 ISPs 80% of Broadband Market 60 Countries OECD and EU

7 Mapping IP Addresses to Autonomous Systems to ISPs 440 million IP addresses

8 Mapping IP Addresses to Autonomous Systems to ISPs 440 million IP addresses Historical BGP lookup (pyasn) 659 Autonomous Systems

9 Mapping IP Addresses to Autonomous Systems to ISPs 440 million IP addresses Historical BGP lookup (pyasn) 659 Autonomous Systems Manual inspection of historical WHOIS Administering Entity

10 Mapping IP Addresses to Autonomous Systems to ISPs 440 million IP addresses Historical BGP lookup (pyasn) 260 ISPs 659 Autonomous Systems Manual inspection of historical WHOIS Manual inspection of media reports Administering Entity

11 Mapping IP Addresses to Autonomous Systems to ISPs 440 million IP addresses Historical BGP lookup (pyasn) 659 Autonomous Systems Manual inspection of historical WHOIS 260 ISPs Manual inspection of media reports Administering Entity Telegeography GlobalComms Database Quarterly Customers

12 Mapping IP Addresses to Autonomous Systems to ISPs 440 million IP addresses Historical MaxMind GeoIP Database Historical BGP lookup (pyasn) 659 Autonomous Systems Manual inspection of historical WHOIS 260 ISPs Manual inspection of media reports Administering Entity Telegeography GlobalComms Database Quarterly Customers 60 Countries

13 Wickedness: A Comparable Metric Wi(t)= Ai(t)/Ci(t) Ai(t): IP Addresses for ISP i at time t Ci(t): Customers for ISP i at time t

14 Global Wickedness

15 Features of Global Wickedness Highly Autoregressive High Variance 800x increase from 1 week to the next (April 2011, Pakistan) 0 to 80% in individual ISPs % to 0.33% Globally Pitfall: Difficult to distinguish significant changes from typical changes.

16 Controlling for Wickedness Risk Factors Demographic Factors GDP Unemployment Education Geographic Factors Regional Clustering ISP Connectivity Topological Features Traffic Dynamics

17 ISP Network Construction

18 ISP Network Construction

19 ISP Network Construction

20 ISP Network Construction

21 A sample of topological measures correlating with wickedness.

22 Traffic Dynamics and Geographic Homogeneity Wicked Traffic Gravity Model of Traffic Flow ri,j = CiCj / di,j2 Ci Regional Clustering Average Regional Wickedness Cj

23 Surprisingly strong correlations throughout time

24 A Simple Model of Wickedness For each ISP i at time t Autoregression Risk Factors Different Qualitative Eras ln(wi,y(t)) = 0,y 3,y ln(wi(t-1)) + ln(ei(t)) + 1,y ln(ri(t-1)) + P (t) + 4,y i 5,y 2,y ln(gi(t-1)) + ln(di(t))

25 Qualitative Changes in Wickedness

26 Identifying Changes in Spam Dynamics For all possible divisions of the data into 1, 2 or 3 eras Fit maximum likelihood estimate (MLE) for i for each era Calculate Akaike Information Criteria (AIC) of the model Select model with the minimum AIC

27 Spam in 3 Eras Era 2 Era 3 Era 1

28 Autoregressive Model: MLE Fits Era 1 Era 2 Era 3 Jan Dec 2010 Dec 2010-Jun 2012 Jun 2012-Dec 2014 log Prev Wickedness log Prev Wicked Traffic log Prev Region Wickedness log GDP per capita Shortest Path Length Variable Weighted Degree R2

29 Botnet Takedowns For each takedown k Bk(t-j) = 1 takedown occurred j weeks ago 0 otherwise { ln(wi,y(t)) = 0,y ln(wi(t-1)) + 1,y ln(ri(t-1)) + ln(ei(t)) + 4,y Pi(t) + k j k,j Bk(t-j) 3,y 5,y 2,y ln(gi(t-1)) + ln(di(t)) +

30 Takedown Effectiveness: Era 1 Botnet Takdown Date 1 Week Change 6 Week Change McColo Nov 11, Mariposa Dec 24, Waledac Mar 5, 2010 NS -3.5 Spamit Oct 1, 2010 NS 6.1 Bredolab/Spamit Oct 29,

31 Takedown Effectiveness: Eras 2 and 3 Botnet Date 1 Week Change 6 Week Change Rustock Mar 19, Coreflood/Rustock April 16, Kelihos Sep 17, Kelihos v2 Apr 1, 2012 NS 30.1 Hermes Jun 24, Grum/Hermes Jul 22, Virut Jan 22,

32 Global vs. Regional Effects Bots are not distributed uniformly. Economics might change between infection and takedown. Bots may migrate after a takedown.

33 Country Takedown Effect

34 Country Takedown Effect

35 Conclusions Comparable metrics establish a uniform measure of the problem Careful analysis of nonlinear, high variance data is needed. Rigorous models can identify meaningful relationships between interventions and their effects. Many factors correlate with spam levels. Takedowns are not always effective. Interventions may have different effects in different regions.

36 Future Work Corroborate results with other data sources. What makes a good takedown? Identify at-risk countries for cyber capacity building.

37 Thank You. Hadi Asghari Drew Levin George Stelle Robert Axelrod Jana Hartman Cari Martinez David Mohr Patrick Gage Kelley I ll take your questions.