6 ways to statically find software bugs. Gang Fan

Transcription

1 6 ways to statically find software bugs Gang Fan S

2 Outline S What bugs? S Why statically? S Six ways to find bugs: S Pattern matching S Inconsistency S IFDS S Value flow graph S Backward symbolic analysis S Symbolic execution S Summary Sweet spot Graph Reachability Symbolic + Solver

3 What bugs? S Cause system failure: S Null pointer dereference, double free, use after free S Waste resources. S Resource leak, unused code, S Do not express the genuine intention of developers. S Logical error, misuse of APIs, other mistakes

4 Why statically?

5 No need to run a program

6 Earlier in SDLC

7 Cover more cases

8 Can work overnight

9 Outline S What bugs? S Why statically? S Six ways to find bugs: S Pattern matching S Inconsistency S IFDS S Value flow graph S Backward symbolic analysis S Symbolic execution S Summary Sweet spot Graph Reachability Symbolic + Solver

10 Benefits of Pattern Matching & Inconsistency S Easy to implement S Very consistent. S Scale to large projects. S Very efficient and fast. S Supporting large classes of program problems. S Bug, Style problem, performance down point.

11 Pattern matching S FindBugs: S More than 1 million downloads. S David Hovemeyer, Bill Pugh. S Plugin for Eclipse, Netbeans and more.

12 An bug example

13 Media Findbugs can work with S Class structure and inheritance hierarchy S Linear code scan S Control sensitive S Dataflow

14 Publications: Pattern Matching S David Hovemeyer and William Pugh. Finding bugs is easy, 2004 S Nick Rutar, et al A comparison of Bug Finding Tools for Java, 2004 S Nathaniel Ayewah, et al The Google Findbugs fixit. 2010

15 Pattern: Cloneable Not implemented Correctly

16 Pattern: Dropped Exceptions try { file_operation(); }catch(exception e){ // simply ignored }

17 Pattern: Nullcheck of value previously dereferenced public void func(int a[]){ a[1] = 2; if(a!= NULL){ a[2] = 3; } }

18 Pattern: Equal Objects Must Have equal hashcodes.

19 Outline S What bugs? S Why statically? S Six ways to find bugs: S Pattern matching S Inconsistency S IFDS S Value flow graph S Backward symbolic analysis S Symbolic execution S Summary Sweet spot Graph Reachability Symbolic + Solver

20 Publication: Program Inconsistency S Dawson Engler, David Yu Chen, Seth Hallem, Andy Chou, and Benjamin Chelf. Bugs as deviant behavior: 2001 S Isil Dillig, Thomas Dillig, and Alex Aiken. Static error detection using semantic inconsistency inference, 2007

21 Program inconsistency S Program beliefs are facts implied by code. S Must belief S Extracted from code. S There is no doubt that the programmer hold such belief S Ex: *p è p is non-null S May belief S Observed from code. S May be coincident. S Ex: B() is always called after A()

22 Must belief S Philosophy: If two beliefs contradict, we know that one is an error without knowing which one is the correct belief:

23 S Statistic analysis. May belief

24 Outline S What bugs? S Why statically? S Six ways to find bugs: S Pattern matching S Inconsistency S IFDS S Value flow graph S Backward symbolic analysis S Symbolic execution S Summary Sweet spot Graph Reachability Symbolic + Solver

25 Benefits of IFDS S IFDS is a general framework that can model large classes of problems. S For a distributive problems, the IFDS solution is always precise. S Inter-procedural Context sensitive. S Pure data flow analysis. S Efficient and the performance increases with the development of underlying graph reachability.

26 Publication: IFDS S Thomas Reps, Susan Horwitz, and Mooly Sagiv. Precise interprocedural dataflow analysis via graph reachability, 1995 S Mooly Sagiv, Thomas Reps, and Susan Horwitz. Precise interprocedural dataflow analysis with applications to constant propagation, 1996 S Bodden, E. Inter-procedural data-flow analysis with IFDS/IDE and soot. 2-12

27 IFDS S Inter-procedural Finite Distributive Subset(IFDS) S Originally proposed by Thomas Reps in 1995 S Extended to IDE S Meet-over-all-valid-paths. S CFL-reachability on Exploded Super Graph

28 Valid path

29 CFL-language for valid path S matched ::= matched ( i matched ) i ε S valid ::= valid ( i matched matched S A valid path can be a open path

30 Example: possibly uninitialized variables main() { x = 3; p(x,y); print(y); } Bug? p(a,b){ if(...){ } } return; b =a; p(a,b); printf(b); Bug?

31 Super Graph

32 Transfer function to graph

33

34

35 Outline S What bugs? S Why statically? S Six ways to find bugs: S Pattern matching S Inconsistency S IFDS S Value flow graph S Backward symbolic analysis S Symbolic execution S Summary Sweet spot Graph Reachability Symbolic + Solver

36 Publication: Value flow graph S Bernhard Steffen, Jens Knoop, Oliver Rüthing, The value flow graph: A program representation for optimal program transformations 1990 S Sigmund Cherem, Lonnie Princehouse, and Radu Rugina. Practical memory leak detection using guarded value-flow analysis S Yulei Sui, Ding Ye, and Jingling Xue. Detecting memory leaks statically with full-sparse value-flow analysis, 2014

37 Benefits of Value flow analysis S Sparse representation of program data dependency. S Support infinite value domain. S Compare to IFDS, which only supports Finite domain. S Efficient and the performance increases with the development of underlying graph reachability. S Assisted with SMT solver, it could support even more classes problems.

38 Definition and Uses S Definitions: S A = 1 S B = 2 S Uses: S A*A S B+A int x = 0; /* A */ x = x + y; /* B */ /* 1, some uses of x */ x = 35; /* C */ /* 2, some uses of x2 */ S Static Single Assignment(SSA) S Express def-use relationship explicitly in program code.

39 Use-def / def-use chain S A list structure: def-use (DU) chain S For each definition d compute a chain of uses that d may reach S Is a sparse representation of data flow. S Compute information only at the program points where it is actually used. S Use-def chain S A counterpart of def-use chain S Static Single Assignment(SSA) S Express def-use relationship explicitly in program code.

40 DU chain

41 Merge points

42 SSA

43 DU chain over SSA

44 Value flow graph S Trace value equivalences over the whole program. S VFG: S a = b S a = φ (b,c,d) S r = func(arg1,arg2) where func(p1, p2) return ret.

45 Find bugs with Value flow graph void foo(){ int *p = NULL; other_statements; bar(p); } void bar(int *p){ *p = 2; }

46 Memory leak cannot be detected with a simple VFG int func() { int *p = malloc(); if (p == NULL) return -1; int *q = malloc(); if (q == NULL) return -1; // p is leaked... other code... } free(p); free(q); return 0; Cannot detect this memory leak bug!!!

47 Guarded Value flow graph S Sigmund Cherem, Lonnie Princehouse, and Radu Rugina. Practical memory leak detection using guarded value-flow analysis S Yulei Sui, Ding Ye, and Jingling Xue. Detecting memory leaks statically with full-sparse value-flow analysis, 2014

48 Guarded Value flow graph int func() { int *p = malloc(); if (p == NULL) return -1; int *q = malloc(); if (q == NULL) return -1; // p is leaked... other code... } free(p); free(q); return 0; p!= null && q == null triggers a Memory Leak!!!

49 Outline S What bugs? S Why statically? S Six ways to find bugs: S Pattern matching S Inconsistency S IFDS S Value flow graph S Backward symbolic analysis S Symbolic execution S Summary Sweet spot Graph Reachability Symbolic + Solver

50 Benefits of Symbolic + Solver S Generally path sensitive. S Very precise. S Leverage the power of SMT Solvers and the performance grows with the development of those Solvers. S Could be extended to support inter-procedural analysis

51 Weakest Pre-condition(WP) S Pre-condition Post-condition: S {?}S{R} S A condition P 1 is weaker than P 2 if P 2 à P 1 S Weakest Pre-condition S S: y = x * x, R: y >= 4, S P: x = 100, x = 200, x = 300 S Weaker P: x >= 100: S WP: (x <= 2) U (x >= 2)

52 Backward WP

53 Backward symbolic analysis S Demand-driven S Computes WP backwardly S Explore all paths backwardly

54 Function boundary problem int func(int *p){ /* false */ if(p){ /* p is null */ *p; // a potential NPD /* p is null */ } } int func(int *p){ /* p is null */ *p; /* p is null */ } // a potential NPD

55 Annotation S ESC/Java S LCLint S David Hovemeyer and William Pugh. Finding bugs is easy S David Evans. Static detection of dynamic memory errors S David Evans. Using specifications to check source code S Bag(/* non_null */ int [] input );

56 Other approaches S S S Assisted annotation S Houdini S ICFG S Cormac Flanagan and K. Rustan M. Leino. Houdini, an annotation assistant for esc/java. Marple S Wei Le, Wei Le, and Mary Lou" Soffa. Marple: A demand-driven path-sensitive buffer overflow detector Function summary: S ESP, Snugglebug S S Manuvir Das, Sorin Lerner, and Mark Seigle. Esp: Path-sensitive program verification in polynomial time Satish Chandra, Stephen J. Fink, Manu Sridharan, Snugglebug: A Powerful Approach To Weakest Preconditions

57 Outline S What bugs? S Why statically? S Six ways to find bugs: S Pattern matching S Inconsistency S IFDS S Value flow graph S Backward symbolic analysis S Symbolic execution S Summary Sweet spot Graph Reachability Symbolic + Solver

58 Symbolic Execution S First introduced in 1975 S Very active area of research. S Cristian Cadar, Daniel Dunbar, and Dawson Engler. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs S P Godefroid, DART: Directed Automated Random Testing S K Sen, CUTE: A Concolic Unit Testing Engine for C S P Godefroid, SAGE: Whitebox Fuzzing for Security Testing. S Corina Pasareanu, et al, Combining Unit-level Symbolic Execution and System-level Concrete Execution for Testing NASA Software

59 Symbolic Execution S Simulating program execution with symbols as arguments S Can take any feasible path S Path condition is a conjunct of constraints on the symbolic input values. S Solution of a path-condition is a test-input that covers the respective path.

60 An Example x = * int bad_abs(int x) { if (x < 0) return x; if (x == 1234) return x; return x; } TRUE x < 0 FALSE x < 0 x 0 return -x TRUE x = 1234 x = 1234 x = -2 test1.out return -x FALSE x 1234 return x 60 x = 1234 test2.out x = 3 test3.out

61 The KLEE project S Using symbolic execution to generate test cases. S Manages to find 10 fatal bugs in well programed and tested project COREUTILS S Better coverage than manual test cases. S Cristian Cadar, Daniel Dunbar, and Dawson Engler. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs

62 Outline S What bugs? S Why statically? S Six ways to find bugs: S Pattern matching S Inconsistency S IFDS S Value flow graph S Backward symbolic analysis S Symbolic execution S Summary Sweet spot Graph Reachability Symbolic + Solver

63 Summary SMT Solver S Six techniques: S Pattern matching, S Inconsistency, S IFDS, S VFG, S Backward Symbolic analysis, S Symbolic Execution. Implementation Difficulty Hard Complex Precision Precise Scalability 1M+ LoC

64 Inspirations for future research S Path explosion S Ex: Loop, recursive call.. Model Paths with Symbolic Analysis S Overwhelming Solver queries. Assisted constraint generation with DFA. Staged Analysis Framework S Lack of environment information S Ex: Network IO, File Combination of concrete execution with static analysis

65 Summary: 1. Path sensitivity is necessary. 2. Rely heavily on SMT solvers. 3. Should not ignore low hang fruits.