Predicting Vulnerable Software Components

Transcription

1 Predicting Vulnerable Software Components Stephan Neuhaus, et. al. 10/29/2008 Stuart A Jaskowiak, CSC 682 1

2 What's in the paper? Introduction Scope of this Work Components and Vulnerabilities Imports and Functions Matter Predicting Vulnerabilities from Features Case Study: Mozilla Related Work Conclusions and Future Work 10/29/2008 Stuart A Jaskowiak, CSC 682 2

3 Introduction Designed Vulture to mine vulnerability database to associate vulnerabilities with individual components 10/29/2008 Stuart A Jaskowiak, CSC 682 3

4 Why Vulture? Vulnerable components tend to share function calls In Mozilla, 14 components import nsnodeutils.h, 13 components (93%) had to be patched because of security leaks All 15 that import nsicontent.h, nsiinterface- RequestorUtils.h and nscontentutils.h together are vulnerable! 10/29/2008 Stuart A Jaskowiak, CSC 682 4

5 Scope Author's Hypothesis is not the import or function but the cause of the vulnerability is the import s or function s domain 10/29/2008 Stuart A Jaskowiak, CSC 682 5

6 Upfront Validity Concerns Study Size Bugs in the database or the code Bugs in R Library Wrong or noisy input data Yet unknown vulnerabilites 10/29/2008 Stuart A Jaskowiak, CSC 682 6

7 Components and Vulnerabilities Component an entity in a software product that may have issues Vulnerability - a defect in one or more components that manifests itself as some violation of a security policy Use Bugzilla to map bug ids to components 10/29/2008 Stuart A Jaskowiak, CSC 682 7

8 Mozilla Components and Vulnerabilities As of 1/4/2007, Mozilla has 13,111 C/C++ files 10,452 components 134 vulnerability advisories 302 bug reports Of the 10,452 components, 424 (4.05%) were vulnerable 10/29/2008 Stuart A Jaskowiak, CSC 682 8

9 Top Ten Mozilla Vulnerabilities First 4 are all Javascript related 10/29/2008 Stuart A Jaskowiak, CSC 682 9

10 Imports and Functions Matter Imports #include <name> #include name #include NAME Function Calls are harder C/C++ need preprocessing to determine if calls are made 10/29/2008 Stuart A Jaskowiak, CSC

11 Features in Mozilla In Mozilla 79,494 of form component x imports import y 9,481 distinct imports 324,822 of form component x calls function y 92,265 distinct function names 10/29/2008 Stuart A Jaskowiak, CSC

12 Predicting Vulnerabilities from Features How vulnerable is a new component? 10/29/2008 Stuart A Jaskowiak, CSC

13 Precision and Recall for Vulture Precision is TP/(TP + FP), Recall is TP/ (TP+FN) 10/29/2008 Stuart A Jaskowiak, CSC

14 Case Study: Mozilla How fast? 10/29/2008 Stuart A Jaskowiak, CSC

15 Case Study: Mozilla (cont) How accurate? For imports: Of all vulnerable components, Vulture flags 45% as vulnerable For function calls: Of all components flagged as vulnerable, 70% actually are vulnerable. Vulture is much better than random selection. Among the top 30 predicted components, Vulture finds 82% of all vulnerabilities. 10/29/2008 Stuart A Jaskowiak, CSC

16 Case Study: Mozilla (cont) Vulture vs Actual Bugs 10/29/2008 Stuart A Jaskowiak, CSC

17 Related Work that lead to Vulture Looking at components histories Evolution of defect numbers Estimating the number of vulnerabilities Testing the binary (Statically) examining the source Hardening the source or runtime enviroment 10/29/2008 Stuart A Jaskowiak, CSC

18 Conclusions A technique for mapping past vulnerabilities by mining and combining vulnerability databases with version archives. Empirical evidence that contradicts popular wisdom saying that vulnerable components will generally have more vulnerabilities in the future. Evidence that features correlate with vulnerabilities. 10/29/2008 Stuart A Jaskowiak, CSC

19 Conclusions (cont) A tool that learns from the locations of past vulnerabilities to predict future ones with reasonable accuracy. An approach for identifying vulnerabilities that automatically adapts to specific projects and products. A predictor for vulnerabilities that only needs a set of suitable features, and thus can be applied before the component is fully implemented. 10/29/2008 Stuart A Jaskowiak, CSC