n HW7 due in about ten days n HW8 will be optional n No CLASS or office hours on Tuesday n I will catch up on grading next week!

Transcription

1 Announcements SMT Solvers, Symbolic Execution n HW7 due in about ten days n HW8 will be optional n No CLASS or office hours on Tuesday n I will catch up on grading next week! n Presentations n Some of you haven t sent me their selections n Please do! n Quiz 6 Spring 18 CSCI 4450/6450, A Milanova 2 Outline n (for HW7) n Symbolic Execution n Overview and applications n Challenges n Tools and techniques n is a language for specifying input to SMT solvers (e.g., Z3) n declare an integer constant x n (assert (> x 0)) add x>0 to known facts n checks if there exist an assignment that makes all known facts true; returns (sat) or (unsat) n (get-model) print this assignment Spring 18 CSCI 4450/6450, A Milanova 3 n Spring 18 CSCI 4450/6450, A Milanova 4 (declare-const a Int) (declare-fun f (Int Bool) Int) (assert (> a 10)) (assert (< (f a true) 100)) Spring 18 CSCI 4450/6450, A Milanova 5 n Your homework is to write a Tiny Dafny n Given an IMP program { P c { Q generate verification conditions in n Verify conditions with Z3 n Yet another programming language, OCaml! n Some pitfalls n Function calls: (f arg1 arg2) NOT f(arg1,arg2)! n == is reference equality. Use (String.equal s1 s2) 6 1

2 n Suppose we need to verify { P c { Q n Generate wp(c,q) n Program verifies when P => wp(c,q) is valid n A logical formula is valid when true for all inputs n Encoding n Duality of satisfiability and validity: F is valid iff!f is unsatisfiable n Ask: is!(p => wp(c,q)) satisfiable n If (unsat) program is correct n If (sat) our program is incorrect, we ll get model 7 Example requires: x == 1 x == -2 ensures: y == 0 { y = x + 4; if (x > 0) { y = x*x - 1; else { y = y + x; wp(,{y=0) =?? (x=1 or x=-2) => ((x>0 and x*x-1 = 0) or (x<=0 and x+4+x=0)) code: (assert (and (or (= x 1) (= x -2)) (not (or (and (<= x 0) (= (+ (+ x 4) x) 0)) (and (> x 0) (= (- (* x x) 1) 0)) )))) Spring 18 CSCI 4450/6450, A Milanova 8 Example requires: x == 1 x == -5 ensures: y == 0 { y = x + 4; if (x > 0) { y = x*x - 1; else { y = y + x; vc(,{y=0) = wp(,{y=0) =?? (x=1 or x=-5) => ((x>0 and x*x-1 = 0) or (x<=0 and x+4+x=0)) code: (assert (and (or (= x 1) (= x -5)) (not (or (and (<= x 0) (= (+ (+ x 4) x) 0)) (and (> x 0) (= (- (* x x) 1) 0)) )))) (get-model) Spring 18 CSCI 4450/6450, A Milanova 9 Another Example Is this formula valid? (x>0 and x+5 > 5) or (x<=0 and (x=0 => x + x + 5 = 5)) code: (assert (not (and (> x 0) (> (+ x 5) 5)))) (assert (not (and (<= x 0) (or (not (= x 0)) (= (+ (+ x x) 5) 5))))) Spring 18 CSCI 4450/6450, A Milanova (example from MIT 2015 Program Analysis OCW) 10 SMT Solvers n SAT Solvers are at the heard of SMT Solvers n In practice, optimizations on SMT expressions is crucial n Simple identities (x+0=x, x*0=0) n E.g., (simplify (> (+ x 5) 5)) yields (not (<= x 0)) n Theory of arrays: n E.g., (simplify (select (store a 42 x) 42)) n Cache solver queries n Remove useless variables Outline n (for HW7) n Symbolic Execution n Overview and applications n Challenges n Tools and techniques n Reading: A Survey of Symbolic Execution Techniques by Baldoni et al. Oct 2017 Spring 18 CSCI 4450/6450, A Milanova 11 Spring 18 CSCI 4450/6450, A Milanova 12 2

3 Classical References n Robert S. Boyer, Bernard Elspas, and Karl N. Levitt, Select: A Formal System for Testing and Debugging Programs by Symbolic Execution, ICRS 1975 n James C. King, Symbolic Execution and Program Testing, CACM, 19(7): , 1976 n William E. Howden, Symbolic Testing and the Dissect Symbolic Evaluation System IEEE TSE, 3(4): , Resurgence and Applications n More powerful computers lead to much more powerful reasoning tools (e.g., Z3) n Systems that started a resurgence n DART by Godefroid and Sen, PLDI 2005 n EXE by Cadar, Ganesh, Pawlowski, Dill and Engler, CCS 2006 Spring 18 CSCI 4450/6450, A Milanova 14 Symbolic Execution Example 1 void foobar (int a, int b) { int x = 1, y = 0; if (a!= 0) { y = 3+x; if (b == 0) x = 2*(a+b); { x-y!= 0 Symbolic variables: e.g., a: α, b: β State σ: map from variables to expressions (either symbolic or concrete) Evaluation of program statements Path condition (for path p): a logical formula F s.t. if F is true, execution takes path p Spring 18 CSCI 4450/6450, A Milanova (example due to Baldoni et al.) 15 T Example 1 σ = { aàα,bàβ,xà1,yà0 π = α!=0 4. y=3+x; σ = { aàα,bàβ,xà1,yà4 π = α!=0 5. if (b == 0) σ = { aàα,bàβ,xà1,yà4 π = α!=0 β=0 6. x=2*(a+b); T σ = { aàα,bàβ π = true 2. x=1; y=0; σ = { aàα,bàβ,xà1,yà0 π = true 3. if (a!= 0) σ = { aàα,bàβ,xà2*(α+β),yà4 π = α!=0 β=0 8. assert { x-y!= 0 NOT OK: 2*(α+β)-4 = 0 α!=0 β=0 F σ = { aàα,bàβ,xà1,yà0 π = α=0 8. assert { x-y!= 0 OK. σ = { aàα,bàβ,xà1,yà0 π = α!=0 β!=0 8. assert { x-y!= 0 OK. F 16 n Why? Motivation for Symbolic Execution n One symbolic execution path covers many actual inputs n Exactly the set of inputs that satisfy the path condition n Thus, we cover a lot more of the program input space than testing Spring 18 CSCI 4450/6450, A Milanova 17 VC Generation Works Too void foobar (int a, int b) { int x = 1, y = 0; if (a!= 0) { y = 3+x; if (b == 0) x = 2*(a+b); { x-y!= 0 Spring 18 CSCI 4450/6450, A Milanova 18 3

4 VC generation vs. Symbolic Execution? n VC generation = Backward reasoning n HW7 Challenges to Symbolic Execution? n State space explosion (Path explosion) n n conditionals generate 2 n paths n Symbolic execution = Forward reasoning n HW8 (one option): Add a symbolic execution engine as another interpreter of IMP programs Spring 18 CSCI 4450/6450, A Milanova 19 n Memory: how to handle pointers and arrays? n Constraint solving: are SMT solvers good enough to solve complex constraints? n Edge of program, i.e., libraries and binary code: how do we handle them, with no benefit of high-level static analysis? Spring 18 CSCI 4450/6450, A Milanova 20 n We can think of program execution as a DAG n Nodes n i represent states n Edges (n i,n j ) represent state transitions n We need strategies/heuristics for graph exploration n At each step, how do we chose which paths to explore and which paths to drop n There are many strategies and heuristics! n DFS n BFS n Advantages n Simplicity n Drawbacks n Generally, unguided by other knowledge n DFS can get stuck in one part of program n BFS considered the better one Spring 18 CSCI 4450/6450, A Milanova 21 Spring 18 CSCI 4450/6450, A Milanova 22 n Heuristic try to steer towards paths more likely to fail assertions n Run symbolic execution engine for a limited period of time n One big idea: randomness n At each step choose paths at random n Consensus: randomness works very well! n Any new heuristic must compare with random n A drawback: reproducibility Spring 18 CSCI 4450/6450, A Milanova 23 Run Different Searches at the Same Time n Advantages? n May achieves better coverage as it explores different strategies n Strategies target certain kinds of bugs better than others n Drawbacks? n As good as best search strategy but wastes time running other search strategies too Spring 18 CSCI 4450/6450, A Milanova 24 4

5 Libraries and Binary Code n Edges of the program n Libraries, binary code n One way n Pull in library code (libc, glibc) n Hard. Symbolic execution easily gets stuck n Another way n Summaries (stubs) for library code n Also hard. A lot of work and often unsound n Conclolic execution gets around these Spring 18 CSCI 4450/6450, A Milanova 25 Concolic Execution n Another big idea, due to Sen et al., FSE 2005 n Mixes concrete and symbolic execution n One variation: dynamic symbolic execution n Instrument program to do symbolic execution n Select some inputs n Run path from start to finish, maintaining concrete state and symbolic state n When finished, generate a new path condition by negating last path condition n Solve path condition and if satisfiable, generate input and run 26 Concolic Execution, Example σ = { aàα,bàβ π = true 2. x=1; y=0; σ = { aàα,bàβ,xà1,yà0 π = true 3. if (a!= 0) σ = { aàα,bàβ,xà1,yà0 π = α!=0 4. y=3+x; σ = { aàα,bàβ,xà1,yà4 π = α!=0 5. if (b == 0) σ = { aàα,bàβ,xà1,yà4 π = α!=0 β=0 6. x=2*(a+b); Suppose we chose inputs a=1, b=1. Concrete σ = { aà1,bà1,xà1,yà0 at 3 and 4 Concrete σ = { aà1,bà1,xà1,yà4 at 5 and 8 σ = { aàα,bàβ,xà1,yà4 π = α!=0 β!=0 8. assert { x-y!= 0 OK. σ = { aàα,bàβ,xà2*(α+β),yà4 π = α!=0 β=0 8. assert { x-y!= 0 NOT OK! 2*(α+β)-4 α!=0 β=0 = 0 is satisfiable 1. Now negate β!=0 2. Ask solver for new inputs, e.g., a=1, b=0 3. Run program again 27 Concolic Execution n Why this works? n Search is guided by a concrete path, therefore there are shadow concrete values for most symbolic variables n Thus, SMT formula becomes easier to solve Spring 18 CSCI 4450/6450, A Milanova 28 Recent Success n SAGE n Microsoft, concolic execution n Finds bugs in file parsers n Microsoft continuously runs SAGE! n Mayhem n Combines BFS and advanced search techniques n Runs on binary code n Automatically generates exploits when bug found n KLEE n Symbolically executes LLVM bitcode 29 Spring 18 CSCI 4450/6450, A Milanova 30 5