Analysis/Bug-finding/Verification for Security

Transcription

1 Analysis/Bug-finding/Verification for Security VIJAY GANESH University of Waterloo Winter 2013

2 Analysis/Test/Verify for Security Instrument code for testing Heap memory: Purify Perl tainting (information flow) Java race condition checking Black-box testing Fuzzing and penetration testing Black-box web application security analysis Static code analysis FindBugs, Fortify, Coverity, MS tools, Model Checking tools NuSMV to verify program properties Slide courtesy John Mitchell 2

3 Manual Testing Doesn t Scale Entry Exit Software Slides courtesy John Mitchel 3

4 Manual Testing Doesn t Scale Entry Exit Software Behaviors Slides courtesy John Mitchel 3

5 Manual Testing Doesn t Scale Entry Exit Software Behaviors Slides courtesy John Mitchel 3

6 Manual Testing Doesn t Scale Entry Exit Software Behaviors Slides courtesy John Mitchel 3

7 Manual Testing Doesn t Scale Entry Exit Software Behaviors Slides courtesy John Mitchel 3

8 Manual Testing Doesn t Scale Entry Exit Software Behaviors Slides courtesy John Mitchel 3

9 Manual Testing Doesn t Scale Entry Manual testing only examines small subset of behaviors Exit Software Behaviors Slides courtesy John Mitchel 3

10 Bug in Swfdec Adobe Flash Movie Player clipconv8x8_u8_s16_c(ptr...){... for (i = 0; i < 8; i++) { for (j = 0; j < 8; j++) { x = BLOCK8x8_S16 (src,sstr,i,j); if (x < 0) x = 0; ptr comes from caller ptr references unallocated memory if (x > 255) x = 255; (*((uint8_t *)((void *) ptr + stride*row) + column)) = x; Code From LibOIL Library version 0.3.x (GNOME Windowing System)

11 Bug in Swfdec Adobe Flash Movie Player jpeg_decoder(jpegdecoder* dec) { dec->width_blocks = (dec->width + 8*max_h_sample - 1)/(8*max_h_sample); dec->height_blocks = (dec->height + 8*max_v_sample - 1)/(8*max_v_sample); int rowstride, image_size;... rowstride = dec->width_blocks * 8*max_h_sample / dec->comps[i].h_subsample; imagesize = rowstride * (dec->height_blocks * 8*max_v_sample/dec->comps[i].v_subsample); malloc OK But imagesize 0 dec->c[i].image=malloc(imagesize);... //LibOIL API function call clipconv8x8_u8_s16_c(dec->c[i].image...);... Code from Swfdec Shockwave Flash Movie Player

12 Bug in Swfdec Adobe Flash Movie Player jpeg_decoder(jpegdecoder* dec) { from input movie dec->width_blocks = (dec->width + 8*max_h_sample - 1)/(8*max_h_sample); dec->height_blocks = (dec->height + 8*max_v_sample - 1)/(8*max_v_sample); int rowstride, image_size;... rowstride = dec->width_blocks * 8*max_h_sample / dec->comps[i].h_subsample; imagesize = rowstride * (dec->height_blocks * 8*max_v_sample/dec->comps[i].v_subsample); malloc OK But imagesize 0 dec->c[i].image=malloc(imagesize);... //LibOIL API function call clipconv8x8_u8_s16_c(dec->c[i].image...);... Code from Swfdec Shockwave Flash Movie Player

13 Essence of this Bug Overflow in imagesize computation jpegdecode(image) { imagesize = f(image->height)*g(image->width); ptr = malloc(imagesize); LibraryCall(ptr, );

14 Difficulty of finding this Bug Deep in the program Stack depth 50 Number of instructions in path: ~ 7 million Program source (excluding libraries) ~70 KLOC Complex input format Movie file, arbitrarily large and complex Few regions of input (height, width) Construct a test input to find bug automatically

15 Random Fuzzing Input Movie File Multiple Fuzzed Movie Files The Fuzzer randomly mutates various fields in the file It can work sometimes However, often produces mal-formed inputs rejected by parser

16 Problem with Random Fuzzing Fuzzed Input Movie File Parsing And Syntactic Check Semantic Core of the Program Program under test

17 Problem with Random Fuzzing Fuzzed Input Movie File Parsing And Syntactic Check Semantic Core of the Program Random Fuzzed inputs often Rejected by Parser Program under test

18 Problem with Random Fuzzing Fuzzed Input Movie File Parsing And Syntactic Check Semantic Core of the Program Random Fuzzed inputs often Rejected by Parser Program under test Rejected In Parsing

19 Information-flow based Fuzzer Program Source Valid Seed Input BuzzFuzz Bug-revealing Input

20 Information-flow based Execution Instrument source for information-flow analysis (tainttracking) Track input regions to sinks Dynamic data dependency analysis Map from values to set of input bytes

21 Information-flow based Execution jpeg_decoder(jpeg* dec) { dec->width_blocks = (dec->width + 8*max_h_sample - 1)/(8*max_h_sample); dec->height_blocks = (dec->height + 8*max_h_sample - 1)/(8*max_h_sample);... rowstride = dec->width_blocks * 8*max_h_sample / dec->comps[i].h_subsample; image_size = rowstride * (dec->height_blocks * 8*max_v_sample/dec->comps[i].v_subsample); dec->c[i].image=malloc(image_size);... //LibOIL API function call clipconv8x8_u8_s16_c(dec->c[i].image...);... //End of Movie Player Code Input Movie File Data Dependency from input bytes to computed values

22 Information-flow based Fuzzer Selecting Attack Points Library API as Sinks or Attack Points Program Source Valid Seed Input BuzzFuzz List of Program Attackpoints

23 Information-flow based Fuzzer Fuzzing with Extremal Values Extremal values for integers: 0,-1,int_max Valid Seed Input Fuzzer 0xffff Fuzzed Input 0xffff Height:1862,1863 Width: 1864,1865 Generated Taint Report

24 Information-flow based Fuzzer Smarter way to Fuzz 0xffff Fuzzed Input 0xffff Program under test CRASH!!

25 Program Analyzers Code Spec

26 Program Analyzers Code Program Analyzer Spec

27 Program Analyzers Code Report Type Line 1 mem leak 324 Program Analyzer 2 buffer oflow 4,353,245 3 sql injection 23,212 4 stack oflow 86,923 Spec 5 dang ptr 8,491 10,502 info leak 10,921

28 Program Analyzers analyze large code bases Code Report Type Line 1 mem leak 324 Program Analyzer 2 buffer oflow 4,353,245 3 sql injection 23,212 4 stack oflow 86,923 Spec 5 dang ptr 8,491 10,502 info leak 10,921

29 Program Analyzers analyze large code bases Code Report Type Line 1 mem leak 324 Program Analyzer 2 buffer oflow 4,353,245 3 sql injection 23,212 4 stack oflow 86,923 Spec 5 dang ptr 8,491 10,502 info leak 10,921 potentially reports many warnings

30 Program Analyzers analyze large code bases Code Report Type Line 1 mem leak 324 Spec Program Analyzer 2 buffer oflow 4,353,245 3 sql injection 23,212 4 stack oflow 86,923 5 dang ptr 8,491 false alarm false alarm 10,502 info leak 10,921 potentially reports many warnings

31 Program Analyzers analyze large code bases Code Report Type Line 1 mem leak 324 Spec Program Analyzer 2 buffer oflow 4,353,245 3 sql injection 23,212 4 stack oflow 86,923 5 dang ptr 8,491 false alarm false alarm 10,502 info leak 10,921 potentially reports many warnings may emit false alarms

32 Static Analysis Goals Bug finding Identify code that the programmer wishes to modify or improve Correctness Verify the absence of certain classes of errors Note: some fundamental limitations Slides courtesy John Mitchell

33 Soundness and Completeness Property Soundness Definition If the program contains an error, the analysis will report a warning. Sound for reporting correctness Completeness If the analysis reports an error, the program will contain an error. Complete for reporting correctness Slides courtesy John Mitchell

34 Complete Incomplete Unsound Sound Slides courtesy John Mitchell

35 Complete Incomplete Sound Reports all errors Reports no false alarms Unsound Slides courtesy John Mitchell

36 Complete Incomplete Sound Reports all errors Reports no false alarms Undecidable Unsound Slides courtesy John Mitchell

37 Complete Incomplete Sound Reports all errors Reports no false alarms Undecidable Reports all errors May report false alarms Unsound Slides courtesy John Mitchell

38 Complete Incomplete Sound Reports all errors Reports no false alarms Undecidable Reports all errors May report false alarms Decidable Unsound Slides courtesy John Mitchell

39 Complete Incomplete Sound Reports all errors Reports no false alarms Undecidable Reports all errors May report false alarms Decidable Unsound May not report all errors Reports no false alarms Slides courtesy John Mitchell

40 Complete Incomplete Sound Reports all errors Reports no false alarms Undecidable Reports all errors May report false alarms Decidable Unsound May not report all errors Reports no false alarms Decidable Slides courtesy John Mitchell

41 Complete Incomplete Sound Reports all errors Reports no false alarms Undecidable Reports all errors May report false alarms Decidable Unsound May not report all errors Reports no false alarms Decidable May not report all errors May report false alarms Slides courtesy John Mitchell

42 Complete Incomplete Sound Reports all errors Reports no false alarms Undecidable Reports all errors May report false alarms Decidable Unsound May not report all errors Reports no false alarms Decidable May not report all errors May report false alarms Decidable Slides courtesy John Mitchell

43 Static Analysis 20 Slides courtesy Andy Chou

44 Static Analysis Source Code char *p; if(x == 0) p = foo(); else p = 0; if(x!= 0) s=*p; else...; return; 20 Slides courtesy Andy Chou

45 Static Analysis Source Code char *p; if(x == 0) p = foo(); else p = 0; if(x!= 0) s=*p; else...; return; true p = 0 true... char *p if (x == 0) if(x!= 0) false p = foo() false s=*p 20 return Slides courtesy Andy Chou

46 Static Analysis Source Code Symbolic CFG Analysis Defects detected char *p; if(x == 0) p = foo(); else p = 0; if(x!= 0) s=*p; else...; return; true p = 0 true... char *p if (x == 0) if(x!= 0) false p = foo() false s=*p Assigning: p=0 x!=0 taking true branch Dereferencing null pointer p return Slides courtesy Andy Chou 20

47 Example int double (int v) { return 2*v; void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { ERROR;

48 Example int double (int v) { return 2*v; N 2*y == x Y void testme (int x, int y) { z = double (y); if (z == x) { N x > y+10 Y if (x > y+10) { ERROR; ERROR

49 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { x = 22, y = 7 x = x 0, y = y 0 if (x > y+10) { ERROR;

50 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { ERROR; x = 22, y = 7, z = 14 x = x 0, y = y 0, z = 2*y 0

51 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { 2*y 0!= x 0 if (x > y+10) { ERROR; x = 22, y = 7, z = 14 x = x 0, y = y 0, z = 2*y 0

52 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { ERROR; Solve: 2*y 0 == x 0 Solution: x 0 = 2, y 0 = 1 x = 22, y = 7, z = 14 x = x 0, y = y 0, z = 2*y 0 2*y 0!= x 0

53 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { ERROR; x = 2, y = 1, z = 2 x = x 0, y = y 0, z = 2*y 0 2*y 0 == x 0

54 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { ERROR; 2*y 0 == x 0 x 0 > y x = 2, y = 1, z = 2 x = x 0, y = y 0, z = 2*y 0

55 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { ERROR; Solve: (2*y 0 == x 0 ) AND (x 0 > y ) Solution: x 0 = 30, y 0 = 15 2*y 0 == x 0 x 0 y x = 2, y = 1, z = 2 x = x 0, y = y 0, z = 2*y 0

56 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { x = 30, y = 15 x = x 0, y = y 0 if (x > y+10) { ERROR;

57 Concolic Testing Approach int double (int v) { return 2*v; concrete state Concrete Execution symbolic state Symbolic Execution path condition void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { ERROR; Program Error x = 30, y = 15 x = x 0, y = y 0 2*y 0 == x 0 x 0 > y 0 +10

58 Explicit Path (not State) Model Traverse all execution paths one by one to F T detect errors assertion violations F T F T program crash uncaught exceptions T F T combine with valgrind to discover memory errors T F T

59 Reliability through Logical Reasoning Engineering, Usability, Novelty Program Specification Logic Formulas Program Reasoning Tool Solver SAT/UNSAT Program is Correct? or Generate Counterexamples (Test cases) 33

60 What is at the Core? The SAT/SMT Problem Logic Formula (q p r) (q p r)... Solver SAT UNSAT Rich logics (Modular arithmetic, Arrays, Strings,...) NP-complete, PSPACE-complete,... Practical, scalable, usable, automatic Enable novel software reliability approaches 34

61 Programs Reasoning & STP Why Bit-vectors and Arrays STP logic tailored for software reliability applications Support symbolic execution/program analysis Int Var Char Var Arithmetic operation (x+y, x-y, x*y, x/y,...) assignments x = expr; C/C++/Java/... if conditional if(cond) x = expr 1 else x = expr 2 inequality Memory read/write x = *ptr + i; Structure/Class Function Loops 32 bit variable 8 bit variable Bit-vectors and Arrays Arithmetic function (x+y,x-y,x*y,x/y,...) equality x = expr; if-then-else construct x = if(cond) expr 1 else expr 2 inequality predicate Array read/write ptr[]; x = Read(ptr,i); Serialized bit-vector expressions Symbolic execution Bounding 35

62 How to Automatically Crash Programs? Concolic Execution & STP Problem: Automatically generate crashing tests given only the code Program Automatic Tester Symbolic Execution Engine with Implicit Spec Formulas SAT/UNSAT STP Crashing Tests 36

63 How to Automate Testing? Concolic Execution & STP Structured input processing code: PDF Reader, Movie Player,... Buggy_C_Program(int* data_field, int len_field) { int * ptr = malloc(len_field*sizeof(int)); int i; //uninitialized while (i++ < process(len_field)) { //1. Integer overflow causing NULL deref //2. Buffer overflow *(ptr+i) = process_data(*(data_field+i)); Formula captures computation Tester attaches formula to capture spec 37

64 How to Automate Testing? Concolic Execution & STP Structured input processing code: PDF Reader, Movie Player,... Buggy_C_Program(int* data_field, int len_field) { int * ptr = malloc(len_field*sizeof(int)); int i; //uninitialized while (i++ < process(len_field)) { //1. Integer overflow causing NULL deref //2. Buffer overflow *(ptr+i) = process_data(*(data_field+i)); Equivalent Logic Formula derived using symbolic execution data_field, mem_ptr : ARRAY; len_field : BITVECTOR(32); //symbolic i, j, ptr : BITVECTOR(32);//symbolic.. mem_ptr[ptr+i] = process_data(data_field[i]); mem_ptr[ptr+i+1] = process_data(data_field[i+1]);.. Formula captures computation Tester attaches formula to capture spec 37

65 How to Automate Testing? Concolic Execution & STP Structured input processing code: PDF Reader, Movie Player,... Buggy_C_Program(int* data_field, int len_field) { int * ptr = malloc(len_field*sizeof(int)); int i; //uninitialized while (i++ < process(len_field)) { //1. Integer overflow causing NULL deref //2. Buffer overflow *(ptr+i) = process_data(*(data_field+i)); Equivalent Logic Formula derived using symbolic execution data_field, mem_ptr : ARRAY; len_field : BITVECTOR(32); //symbolic i, j, ptr : BITVECTOR(32);//symbolic.. mem_ptr[ptr+i] = process_data(data_field[i]); mem_ptr[ptr+i+1] = process_data(data_field[i+1]);.. Formula captures computation Tester attaches formula to capture spec 37

66 How to Automate Testing? Concolic Execution & STP Structured input processing code: PDF Reader, Movie Player,... Buggy_C_Program(int* data_field, int len_field) { int * ptr = malloc(len_field*sizeof(int)); int i; //uninitialized while (i++ < process(len_field)) { //1. Integer overflow causing NULL deref //2. Buffer overflow *(ptr+i) = process_data(*(data_field+i)); Equivalent Logic Formula derived using symbolic execution data_field, mem_ptr : ARRAY; len_field : BITVECTOR(32); //symbolic i, j, ptr : BITVECTOR(32);//symbolic.. mem_ptr[ptr+i] = process_data(data_field[i]); mem_ptr[ptr+i+1] = process_data(data_field[i+1]);.. //INTEGER OVERFLOW QUERY 0 <= j <= process(len_field); ptr + i + j = 0? Formula captures computation Tester attaches formula to capture spec 37