VUzzer: Application-Aware Evolutionary Fuzzing

Transcription

1 VUzzer: Application-Aware Evolutionary Fuzzing Sanjay Rawat, Vivek Jain, Ashish Kumar, Lucian Cocojar, Cristiano Giuffrida, Herbert Bos (Presenter: Dennis Andriesse ) Vrije Universiteit Amsterdam IIIT Hyderabad NDSS 2017

2 Introduction Fuzzing is a powerful testing technique Generate/transform inputs to trigger bugs Fuzzers like AFL very successful in finding (low-hanging) bugs Deep bugs are harder! read(fd, buf, size); 3 if(buf[5] == 0xD8 && buf[4] == 0xFF) // notice the order of CMPs 4... some useful code... 5 else 6 EXIT_ERROR("Invalid file\n") AFL spends hours trying to enter the if branch Must guess which offsets to mutate Must guess the exact sequence 0xFFD8 VUzzer: Application-Aware Evolutionary Fuzzing 1 of 11

3 Introduction 1 int main(int argc, char **argv) { 2 unsigned char buf[1000]; 3 int i, fd, size, val; 4 if((fd = open(argv[1], O_RDONLY)) == 1) 5 exit(0); 6 fstat(fd, &s); 7 size = s.st_size; 8 if(size > 1000) 9 return 1; 10 read(fd, buf, size); 11 if(buf[1] == 0xEF && buf[0] == 0xFD) // notice the order of CMPs 12 printf("magic bytes matched!\n"); 13 else 14 EXIT_ERROR("Invalid file\n"); 15 if(buf[10] == % && buf[11] ) { 16 printf("2nd stop: on the way...\n"); 17 if(strncmp(&buf[15], "MAZE", 4) == 0) // nested IF some bug here else { 20 printf("you just missed me...\n"); some other task close(fd); return 0; 23 } 24 } else { 25 ERROR("Invalid bytes"); some other task close(fd); return 0; 28 } 29 close(fd); return 0; 30 } Finding deeply nested bugs infeasible with current approaches VUzzer: Application-Aware Evolutionary Fuzzing 2 of 11

4 Overview We introduce VUzzer, a mutation-based graybox fuzzer Key idea: find out where to mutate, and what to insert Evolutionary fuzzing: mutate/select most promising paths Magic byte detection: find magic bytes in inputs (JPG, ELF, PE, BMP,...) to delve deeper into bug-prone code like parsers Limited input type detection: detect that parts of input are integers, to aid mutation Find deeply nested bugs without wasting time on boring paths Limit usage of non-scalable techniques (DTA, symbex) VUzzer: Application-Aware Evolutionary Fuzzing 3 of 11

5 The VUzzer Approach Evolutionary Fuzzing Loop BB monitoring DTA evolutionary fuzzing loop close Test case Application binary Error BBs Executed BBs Append input Static analysis BB weights CMP imm Fitness function Fitness list Interesting offsets detection Magic bytes LEA offsets Crossover Mutation 1. Lightweight intraprocedural static analysis to find magic immediates and calculate BB weights (for path prioritization) Instrument cmp to detect comparisons against magic bytes Instrument lea to find input bytes used as index Model CFG as Markov model to compute probability of reaching each BB Unlikely BBs get higher weight VUzzer: Application-Aware Evolutionary Fuzzing 4 of 11

6 The VUzzer Approach Evolutionary Fuzzing Loop BB monitoring DTA evolutionary fuzzing loop close Test case Application binary Error BBs Executed BBs Append input Static analysis BB weights CMP imm Fitness function Fitness list Interesting offsets detection Magic bytes LEA offsets Crossover Mutation 2. Run application with seed inputs to infer initial control-/data-flow features Taint analysis used to find magic bytes and integer fields in input Use this knowledge for mutating input in later steps Detect suspected error-handling BBs Covered by totally random inputs, but not by known valid inputs Given a negative weight (avoid dead ends ) VUzzer: Application-Aware Evolutionary Fuzzing 5 of 11

7 The VUzzer Approach Evolutionary Fuzzing Loop BB monitoring DTA evolutionary fuzzing loop close Test case Application binary Error BBs Executed BBs Append input Static analysis BB weights CMP imm Fitness function Fitness list Interesting offsets detection Magic bytes LEA offsets Crossover Mutation 3. Main evolutionary loop: 1 Recombine/mutate inputs to form new inputs 2 Run application with new inputs 3 New BBs seen? Use DTA to infer more structural properties 4 Calculate fitness of inputs based on likelihood of paths covered 5 Select fittest inputs for next iteration VUzzer: Application-Aware Evolutionary Fuzzing 6 of 11

8 Evaluation Relative number of inputs executed (%) VUzzer AFLPIN Compared VUzzer against AFL for DARPA CGC dataset VUzzer requires far fewer inputs than AFL to find bugs VUzzer: Application-Aware Evolutionary Fuzzing 7 of 11

9 Evaluation LAVA-M DATASET : PERFORMANCE OF VUZZER COMPARED TO PRIOR APPROACHES. Program Total bugs FUZZER SES VUzzer (unique bugs, total inputs) uniq (27K) base (14k) md5sum who (5.8K) LAVA-M dataset: VUzzer finds more bugs than existing coverage-based (FUZZER) and symbex/sat-based (SES) approaches VUzzer: Application-Aware Evolutionary Fuzzing 8 of 11

10 Evaluation VA DATASET : PERFORMANCE OF VUZZER VS. AFL. Application VUzzer AFL #Unique crashes #Inputs #Unique crashes #Inputs mpg K K gif2png+libpng K M pdf2svg+libpoppler 13 5K 0 923K tcpdump+libpcap K M tcptrace+libpcap K M djpeg+libjpeg 1 90K M Found more bugs with fewer inputs compared to AFL for VA dataset (various applications) VUzzer: Application-Aware Evolutionary Fuzzing 9 of 11

11 Evaluation Number of crashes time (hours) mpg time (hours) gif2png VUzzer AFL Time taken by VUzzer to find all crashes found by AFL time (hours) tcpttrace Crashes found faster, with more consistent forward progress VUzzer: Application-Aware Evolutionary Fuzzing 10 of 11

12 Conclusion Designed novel fuzzing approach based on evolutionary approach Effective DTA-based heuristics for identifying magic bytes and input structure More effective mutation of inputs BB prioritization to avoid dead ends Find more bugs with orders of magnitude fewer inputs than existing approaches VUzzer released open source VUzzer: Application-Aware Evolutionary Fuzzing 11 of 11