Semantics-Based Program Verifiers for All Languages

Transcription

1 Language-independent Semantics-Based Program Verifiers for All Languages Andrei Stefanescu Daejun Park Shijiao Yuwen Yilong Li Grigore Rosu Nov 2, OOPSLA 16

2 Problems with state-of-the-art verifiers Missing details of language behaviors e.g., VCC s false positives/negatives, undefinedness of SV-COMP benchmarks Fragmentation: specific to a fixed language

3 Missing details of language behaviors 1 unsigned x = UINT_MAX; 2 unsigned y = x + 1; 3 _(assert y == 0) VCC incorrectly reported an overflow error

4 Missing details of language behaviors 1 int assign(int *p, int x) 2 _(ensures *p == x) 3 _(writes p) 4 { 5 return (*p = x); 6 } 7 8 void main() { 9 int r; 10 assign(&r, 0) == assign(&r, 1); 11 _(assert r == 1) 12 } VCC incorrectly proved it, missing non-determinism

5 Missing details of language behaviors * Grigore Rosu,

6 Problems with state-of-the-art verifiers Missing details of language behaviors Fragmentation: specific to a fixed language e.g., KLEE (LLVM), JPF (JVM), Pex (.NET), CBMC (C), SAGE (x86), Implemented similar heuristics/optimizations: duplicating efforts

7 Our solution Clear separation, yet smooth integration, Between semantics reasoning and proof search, Using language-independent logic & proof system

8 Idea: separation of concerns Semantics Reasoning Proof Search Language semantics: C (c11, gcc, clang, ) Java (6, 7, 8, ) JavaScript (ES5, ES6, ) Verification techniques: Deductive verification Model checking Abstract interpretation Defined/implemented once, and reused for all others

9 Idea: separation of concerns Semantics Reasoning Proof Search Language semantics: C (c11, gcc, clang, ) Java (6, 7, 8, ) JavaScript (ES5, ES6, ) JPF VCC CBMC Verification techniques: Deductive verification Model checking Abstract interpretation Defined/implemented once, and reused for all others

10 Language-independent verification framework Semantics Program & Properties Language-independent uniform notation (logic) Language-independent proof systems ` Proof automation Provides a nice interface (logic) in which both language semantics and program properties can be described. Proof search in this logic becomes completely languageindependent.

11 Language-independent verification framework Operational semantics (C/Java/JavaScript semantics) Reachability properties (Functional correctness of heap manipulations) Language-independent uniform notation (Matching logic reachability) Language-independent proof systems (Matching logic reachability proof systems) Proof automation (Symbolic execution, SMT, Natural proofs, )

12 Operational semantics Easy to define and understand than axiomatic semantics Require little mathematical knowledge Similar to implement language interpreter Executable, thus testable Important when defining real large languages Shown to scale to defining full language semantics C, Java, JavaScript, Python, PHP,

13 Language-independent verification framework Operational semantics (C/Java/JavaScript semantics) Reachability properties (Functional correctness of heap manipulations) Language-independent uniform notation (Reachability logic) Language-independent proof systems (Reachability logic proof systems) Proof automation (Symbolic execution, SMT, Natural proofs, )

14 Reachability logic Unifying logic in which both language semantics and program correctness properties can be specified. reachability between patterns pattern formula representing a set of program states Pattern formula is FOL without predicate symbols. Similar to algebraic data types for pattern matching in functional languages such as OCaml and Haskell.

15 Expressiveness: semantics In OCaml: match e with ADD(x,y) => x + y SUB(x,y) => x - y MUL(x,y) => x * y DIV(x,y) when y!= 0 => x / y

16 Expressiveness: semantics In OCaml: match e with ADD(x,y) => x + y SUB(x,y) => x - y MUL(x,y) => x * y DIV(x,y) when y!= 0 => x / y In Reachability logic: ADD(x,y) => x + y SUB(x,y) => x - y MUL(x,y) => x * y DIV(x,y) /\ y!= 0 => x / y

17 Expressiveness: properties In Hoare logic: fun insert (v: elem, t: tree) return (t : bst(t ) and keys(t ) == keys(t) \union { v }

18 Expressiveness: properties In Hoare logic: fun insert (v: elem, t: tree) return (t : bst(t ) and keys(t ) == keys(t) \union { v } In Reachability logic: insert /\ bst(t) =>. /\ bst(t ) /\ keys(t ) == keys(t) \union { v }

19 Expressiveness Reachability formula can specify: Pre-/post-conditions Safety properties by augmenting semantics No liveness properties yet (ongoing work) Pattern formula can include: Recursive predicates Separation logic formula

20 Language-independent verification framework Operational semantics (C/Java/JavaScript semantics) Reachability properties (Functional correctness of heap manipulations) Language-independent uniform notation (Reachability logic) Language-independent proof systems (Reachability logic proof systems) Proof automation (Symbolic execution, SMT, Natural proofs, )

21 Proof system Language-independent proof system for deriving sequents of the form: Step : = '! W ' l ) 9 ' r 2 S 9FreeVars(' l ).' l = ((' ^ ' l ),? Cfg ) ^ ' r! ' 0 for each ' l ) 9 ' r 2 S S, A `C ' ) 8 ' 0 Axiom : ' ) Q ' 0 2 S [ A is FOL formula (logical frame) Reflexivity : S, A ` ' ) Q ' S, A `C ' ^ ) Q ' 0 ^ Transitivity : S, A `C ' 1 ) Q ' 2 S, A [ C ` ' 2 ) Q ' 3 S, A `C ' 1 ) Q ' 3 Consequence : = ' 1! ' 0 1 S, A `C ' 0 1 )Q ' 0 2 = ' 0 2! ' 2 S, A `C ' 1 ) Q ' 2 Case Analysis : S, A `C ' 1 ) Q ' S, A `C ' 2 ) Q ' S, A `C ' 1 _ ' 2 ) Q ' Abstraction : S, A `C ' ) Q ' 0 X \ FreeVars(' 0 ) = ; S, A `C 9X ' ) Q ' 0 Circularity : S, A `C[{') Q ' 0 } ' ) Q ' 0 S, A `C ' ) Q ' 0

22 Proof system Language-independent proof system for deriving sequents of the form: semantics ' 1 ) ' 0 1 ' 2 ) ' 0 2 ' 3 ) ' property ` ' ) ' 0 Step : = '! W ' l ) 9 ' r 2 S 9FreeVars(' l ).' l = ((' ^ ' l ),? Cfg ) ^ ' r! ' 0 for each ' l ) 9 ' r 2 S S, A `C ' ) 8 ' 0 Axiom : ' ) Q ' 0 2 S [ A is FOL formula (logical frame) Reflexivity : S, A ` ' ) Q ' S, A `C ' ^ ) Q ' 0 ^ Transitivity : S, A `C ' 1 ) Q ' 2 S, A [ C ` ' 2 ) Q ' 3 S, A `C ' 1 ) Q ' 3 Consequence : = ' 1! ' 0 1 S, A `C ' 0 1 )Q ' 0 2 = ' 0 2! ' 2 S, A `C ' 1 ) Q ' 2 Case Analysis : S, A `C ' 1 ) Q ' S, A `C ' 2 ) Q ' S, A `C ' 1 _ ' 2 ) Q ' Abstraction : S, A `C ' ) Q ' 0 X \ FreeVars(' 0 ) = ; S, A `C 9X ' ) Q ' 0 Circularity : S, A `C[{') Q ' 0 } ' ) Q ' 0 S, A `C ' ) Q ' 0

23 Proof system Language-independent proof system for deriving sequents of the form: semantics ' 1 ) ' 0 1 ' 2 ) ' 0 2 ' 3 ) ' property ` ' ) ' 0 Step : = '! W ' l ) 9 ' r 2 S 9FreeVars(' l ).' l = ((' ^ ' l ),? Cfg ) ^ ' r! ' 0 for each ' l ) 9 ' r 2 S S, A `C ' ) 8 ' 0 Axiom : ' ) Q ' 0 2 S [ A is FOL formula (logical frame) Reflexivity : S, A ` ' ) Q ' S, A `C ' ^ ) Q ' 0 ^ Transitivity : S, A `C ' 1 ) Q ' 2 S, A [ C ` ' 2 ) Q ' 3 S, A `C ' 1 ) Q ' 3 Consequence : = ' 1! ' 0 1 S, A `C ' 0 1 )Q ' 0 2 = ' 0 2! ' 2 ADD(x,y) => x + y SUB(x,y) => x - y MUL(x,y) => x * y... ` S, A `C ' 1 ) Q ' 2 insert /\ bst(t) Case Analysis : => S, A `C ' 1 ) Q ' S, A `C ' 2 ) Q '. /\ bst(t ) S, A `C ' 1 _ ' 2 ) Q ' /\ keys(t ) Abstraction == : keys(t) \union { v } S, A `C ' ) Q ' 0 X \ FreeVars(' 0 ) = ; S, A `C 9X ' ) Q ' 0 Circularity : S, A `C[{') Q ' 0 } ' ) Q ' 0 S, A `C ' ) Q ' 0

24 Language-independent verification framework Operational semantics (C/Java/JavaScript semantics) Reachability properties (Functional correctness of heap manipulations) Language-independent uniform notation (Reachability logic) Language-independent proof systems (Reachability logic proof systems) Proof automation (Symbolic execution, SMT, Natural proofs, )

25 Proof automation Deductive verification Symbolic execution for reachability space search Domain reasoning (e.g., integers, bit-vectors, floats, set, sequences, ) using SMT Natural proofs technique for quantifier instantiation for recursive heap predicates (e.g., list, tree, )

26 Language-independent verification framework Operational semantics (C/Java/JavaScript semantics) Reachability properties (Functional correctness of heap manipulations) Language-independent uniform notation (Reachability logic) Language-independent proof systems (Reachability logic proof systems) Proof automation (Symbolic execution, SMT, Natural proofs, ) Does it really work? Q1: How easy to instantiate the framework? Q2: Is performance OK?

27 Evaluation Instantiated framework by plugging-in three language semantics. Semantics of C [POPL 12, PLDI 15] Semantics of Java [POPL 15] Semantics of JavaScript [PLDI 15] Verification Framework C verifier Java verifier JavaScript verifier Verified challenging heap-manipulating programs implementing the same algorithms in all three languages.

28 Efforts C Java JavaScript Semantics development (months) Semantics size (LOC) 17,791 13,417 6,821 Language-specific e ort (days) Semantics changes size (#rules) Semantics changes size (LOC) Specifications Instantiating efforts include: Fixing bugs of semantics Specifying heap abstractions (e.g., lists and trees) instantiating framework (additional effort)

29 Efforts C Java JavaScript Semantics development (months) Semantics size (#rules) 2,572 1,587 1,378 Semantics size (LOC) 17,791 13,417 6,821 Language-specific e ort (days) Semantics size (LOC) 7 17, , ,821 Language-specific e ort (days) Semantics changes size (#rules) Semantics changes size (LOC) Specifications Instantiating efforts include: Fixing bugs of semantics Specifying heap abstractions (e.g., lists and trees) defining semantics (already given) instantiating framework (additional effort)

30 Experiments Time (secs) Programs C Java JS BST find BST insert BST delete AVL find AVL insert AVL delete RBT find RBT insert RBT delete 1, Programs C Java JS Treap find Treap insert Treap delete List reverse List append Bubble sort Insertion sort Quick sort Merge sort Total 4, Average

31 Experiments Time (secs) Programs C Java JS BST find BST insert BST delete Full functional correctness: AVL find Programs C Java JS Treap find Treap insert Treap delete List reverse AVL insert insert /\ bst(t) =>. /\ bst(t ) /\ keys(t ) == keys(t) \union { v } AVL delete RBT find List append Bubble sort Insertion sort RBT insert Quick sort RBT delete 1, Merge sort Total 4, Average

32 Experiments Time (secs) Programs C Java JS BST find BST insert BST delete AVL find AVL insert Programs C Java JS Treap find Treap insert Treap delete List reverse List append AVL delete Bubble sort Performance is comparable to a state-of-the-art Insertion sort verifier 61.9 for 31.1 C, 44.8 VCDryad [PLDI 14], based on a separation Quick sort logic 79.2 extension 47.1 of VCC: 48.1 e.g., AVL insert : 260s vs 280s (ours) RBT find RBT insert RBT delete 1, Merge sort Total 4, Average

33 Idea: separation of concerns Language-independent verification framework Semantics Semantics Reasoning Proof Search Program & Properties Semantics-Based Program Verifiers for All Languages Language-independent uniform notation (logic) Language semantics: Verification techniques: Andrei S, tefa nescu Daejun Park Deductive University of Illinois at verification University of Illinois at Urbana-Champaign, USA VCC C (c11, gcc, clang, ) Java (6, 7, 8, ) JavaScript (ES5, ES6, ) JPF Urbana-Champaign, USA Model checking CBMC stefane1@illinois.edu Abstract interpretation Defined/implemented once, and Yilong Li Runtime Verification, Inc., USA reused for all others yilong.li@runtimeverification.com Shijiaoproof Yuwen Language-independent systems University of Illinois at Urbana-Champaign, USA Proof automation yuwen2@illinois.edu ` Provides a nice interface (logic) in which both language Grigore Ros, u semantics and program properties can be described. University of Illinois at Urbana-Champaign, USA Proof search in grosu@illinois.edu this logic becomes completely languageindependent. Evaluation Abstract We present a language-independent verification framework that can be instantiated with an operational semantics to automatically generate verifier. The framework treats Instantiated framework by plugging-in threea program language semantics. both the operational semantics and the program correctness specifications as reachability rules between matching logic Semantics of C patterns, and uses the sound and relatively complete reachc verifier [POPL 12, PLDI 15] ability logic proof system to prove the specifications using the semantics. We instantiate the framework with the semansemantics of Java Verification Java verifier tics of one academic language, KernelC, as well as with [POPL 15] Framework three recent semantics of real-world languages, C, Java, and JavaScript, developed independently of our verification infrassemantics of JavaScript JavaScript tructure. We evaluate our approachverifier empirically and show that [PLDI 15] the generated program verifiers can check automatically the full functional correctness of challenging heap-manipulating programs implementing operations on list and tree data struc Verified challenging heap-manipulating programs tures, like AVL trees. This is the first approach that can implementing the same algorithms in all three languages. turn the operational semantics of real-world languages into correct-by-construction automatic verifiers. 1. Introduction Experiments Operational semantics are easy to define and understand, similarly to implementing an interpreter. They require little formal training, scale up well, and, being executable, can Time (secs) be tested. Thus, operational semanticsprograms are typicallycused asjava JS Programs C Java JS trusted reference models for the defined languages. Despite Treap find BST find these advantages, they are rarely used directly for program Treap insert BST insert verification, because proofs tend to be low-level, as they work Treap delete 90.4 or BST delete transition directly with71.7 the corresponding system. Hoare dynamic logics higher level reasoning cost of 4.1 List reverse at the AVL find 13.0 allow (re)defining281.3 the language as135.0 a set of abstract proof14.8 rules, 7.3 List append 5.3 AVL insert which are harder to understand and trust. The state-of-the-art Bubble sort 31.3 AVL delete in mechanical program verification is to develop and66.4 prove 38.8 Insertion sort 44.8 RBT find proof systems 6.8 such language-specific sound w.r.t. a 61.9 trusted 31.1 operational903.8 semantics [3, 26, ], but thatsort needs to be done 47.1 Quick RBT insert for each language separately and is labor intensive. Merge sort RBT delete 1, Defining multiple semantics for the same language and proving the soundness of one semantics in terms of antotal 4, other are highly uneconomical tasks when real languages are concerned, often taking several Average man-years to complete. Categories and Subject Descriptors D.2.4 [Software EngiFor these reasons, many program verification tools forgo neering]: Software/Program Verification correctness proofs; defining an operational or an axiomatic semantics altogether,