Michel Aubizzierre INFILTRATE Jan 12th 2012

Transcription

1 Michel Aubizzierre INFILTRATE Jan 12th 2012

2

3

4 Seagulls are the security researchers of the sea

5 Unearthing the world s greatest bugs

6

7

8

9

10

11

12

13

14

15

16

17 When I say: Automated testing <div id= box style= width: 1000px; padding-left: 337px; > </div> I want: document.getelementbyid( box ).offsetwidth == 1337px; PASS / FAIL

18 Merging master release security patch

19 WebKit It s different everywhere: Browsers, applications, embedded devices Architectures: x86, x64, ARM, MIPS Features: SVG, AudioContext, CSS Regions Heap allocator Performance: set-top-box, phone, laptop

20 WebKit had about 300 security bugs in 2011 Enough data for meaningful machine learning exercise

21 Also see How Open Should Open Source Be? TechRpts/2011/EECS pdf They used libsvm on Firefox version control and: committer invisible bug

22

23 Bug halflife Mean time between fix published in trunk and stable browser released: Chrome: 22 days Safari: 92 Days ios: 107 days Blackberry: Unknown, slower than ios itunes: Similar to ios and Safari webkitgtk: depends on vendor Other platforms don t have a half-life

24 Fastest fix chrome: 1 day (pwn2own), 7 days for regular bugs safari: 16 days ios: 34 days

25 Bugs remaining

26 Exploit TCO Shellcode Minimized Crash Get PC Stage 2./win { today we are optimizing this Not to scale

27 Is that browser vulnerable to this? Is there a stable browser out there vulnerable to this bug? Does Safari have the vulnerable code? Is it reachable in Safari? Is it exploitable in build 5.1A123 of Safari?

28 Is that browser vulnerable to this? For Chrome and Safari, there is some data available Relevant data is not present in version control This method will therefore not find it

29

30 Artificial Intelligence What is beauty? SMT solvers on a moonlit beach

31 Machine Learning If 1,1,0 1 and 1,0,1 1 and 0,1,1 0 1,0,0? one

32 Plan 0x00 Tell machine learning software what to do 0x04 Execute magical machine learning 0x08 Check results 0x0c Improve inputs 0x10 jmp 0x04

33 SVM Support Vector Machines you can consider it a black box expects inputs to be lists of numbers gives back numbers (almost) no parameters

34 SVM My question: Is commit security related? Expected answer: 1 (or 0) Commit must be modeled as list of numbers

35 Features Single attribute of an entity: References invisible bug? Message contains the word crash?

36 Enumerations Committer is Committer is Committer is Split into three attributes, is inferno, is cevans, is abarth [1, 0, 0], [0, 1, 0], [0, 0, 1] Expressed as sparse matrix: Only list attributes which aren t 0, e.g. 2:1

37 Training data Known correct answers Both positive and negative Commits 123, 456 are security fixes Commits abc, def are not security fixes

38 Cross validation Split training data n-ways For every set of (n-1) groups, do they correctly predict the remaining n

39 Cross validation Training data: A B C D E F Does A B C D predict E F Does A B E F predict C D Does C D E F predict A B

40 Features of security related commits Authored by member of the security team Reviewed by member of the security team Mentions a member of the security team Mentions a restricted bug The patch contains the word crash

41 Features of security related commits 2 Merged to a branch Merged by a member of the security team Merge reviewed by a member of the security team Message mentions keyword: crash, CVE, out of bounds, use after free, security

42 Features of boring commits Mentions keyword: build, flakiness, rebaseline, unreviewed, rolling out, null

43 Restricted bugs There are about bugs Curl them all Check for /You are not authorized/ Takes about a day

44 Going through the repo Git Master branch available on WebKit git Chromium branches through git-svn see

45 Going through the repo Grit ruby gem by GitHub Monkey patched: def by_security_team? WebKit::SECURITY_TEAM.include?(committer.to_s) end def reviewed_by_security_team?!!(review=~message) end

46 JSONize it { } "svn_rev":"95749", "committer":"andersca@apple.com", "by_security_team":false, "reviewed_by_security_team":false, "mentions_security_team":false, "restricted_bug":false, "keywords":["origin","crash","broke"], "crash_in_patch":true, "bug":68570

47 libsvmize it 1 1:1 2:0 3:0 4:1 5:0 6:1 7:1 8:0 18:1 71:1 #94857 merged 0 1:0 2:0 3:0 4:1 5:1 6:1 7:1 8:0 17:1 71:1 72:1 80:1 #94864 crash merged 1 1:0 2:0 3:0 4:1 5:1 6:1 7:1 8:0 17:1 71:1 72:1 79:1 80:1 #94905 crash, build merged -1 1:0 2:0 3:0 4:0 5:1 6:0 7:0 8:0 64:1 80:1 #94955 crash -1 1:0 2:0 3:0 4:0 5:1 6:1 7:0 8:0 17:1 73:1 80:1 #94982 crash merged 1 1:0 2:0 3:0 4:1 5:0 6:1 7:0 8:0 65:1 66:1 88:1 #95010 out-of-bounds merged -1 1:0 2:0 3:0 4:0 5:1 6:0 7:0 8:0 17:1 80:1 #95017 crash -1 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 39:1 94:1 #95785 unreviewed -1 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 60:1 # :0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 42:1 94:1 #95787 unreviewed

48 ??? despite the name, libsvm is a set of command line tools for me:./svm-train -c 64 -nu inf.t &&./svm-predict inf inf.model inf.out && paste inf inf.out grep -v "\-1$"

49 Check results 0 1:0 2:0 3:0 4:1 5:1 6:1 7:1 8:0 17:1 67:1 71:1 80:1 95:1 #95791 use after free, crash merged 1 0 1:0 2:0 3:0 4:0 5:1 6:0 7:0 8:0 26:1 80:1 86:1 #95673 origin, crash 1 0 1:0 2:0 3:0 4:1 5:0 6:0 7:0 8:0 20:1 89:1 #95679 policy 1 0 1:0 2:0 3:0 4:1 5:1 6:1 7:1 8:0 17:1 71:1 #95689 merged 1 0 1:0 2:0 3:0 4:1 5:1 6:0 7:0 8:0 20:1 85:1 94:1 #95690 unreviewed, null 1 0 1:0 2:0 3:0 4:1 5:1 6:1 7:1 8:0 17:1 71:1 #95728 merged 1 0 1:0 2:0 3:0 4:0 5:1 6:0 7:0 8:0 86:1 #95729 origin 1 0 1:0 2:0 3:0 4:1 5:0 6:0 7:0 8:0 20:1 # :0 2:0 3:0 4:0 5:1 6:0 7:0 8:0 86:1 92:1 #95845 origin, security 1 0 1:1 2:0 3:0 4:1 5:0 6:1 7:1 8:0 18:1 71:1 #95857 merged 1 0 1:0 2:0 3:0 4:1 5:0 6:0 7:0 8:0 17:1 92:1 #95880 security 1 0 1:0 2:0 3:0 4:0 5:0 6:1 7:1 8:0 71:1 #95924 merged 1 0 1:0 2:0 3:0 4:0 5:0 6:1 7:1 8:0 71:1 #95959 merged 1 0 1:0 2:0 3:0 4:0 5:0 6:1 7:0 8:0 67:1 #96020 merged 1

50 It works improving the training set improves results commit message of false negatives & false positives give hints for new keyword features 80-90% success rate during cross validation

51 Bugs found through fuzzing source code review insider expertise

52 Types of bugs JIT bugs crypto bugs policy errors memory corruption

53

54 Photo thanks CC attribution / /

55 Thank