Formal Verification of Programs and Their Transformations

Transcription

1 Formal Verification of Programs and Their Transformations Guodong Li Advisor : Ganesh Gopalakrishnan June 25,

2 Dissertation statement Formal methods are essential for reliable software design. For maximum effectiveness, one should tailor it to the domain, the program representation, and the kind of analysis insights being sought. This dissertation takes three widely different domains and concretely illustrates what formal methods worked for each. 2

3 Outline Formal Model and Analysis Program semantics Reasoning mechanisms Three Case Studies A trusted compiler from logic specifications to assembly code A formalization of MPI 2.0 A symbolic checker for CUDA kernel programs 3

4 Meaning of Programs Euclid s s algorithm for computing GCD int gcd(int x, int y) { while (x!= y) do if x <= y then y = y - x; else x = x - y; } gcd(a,b) = if a=b then a else if a>b then gcd(a-b,b) else gcd(a,b-a) 4

5 Program Semantics Operational semantics: program behaviour: states and transitions granularity: big-step, small-step, Denotational semantics: programs as state transformers Axiomatic semantics: programs as predicate transformers properties of states as logic assertions 5

6 Examples of Operational Semantics Robin Milner's semantics for Standard ML Xavier Leory's semantics for a subset of C (Clight Clight) 6

7 Main Issues The correctness of the semantics Formal modeling of a language and its semantics Soundness checking, e.g. static and dynamic type checking The usability of the semantics It must be convenient (e.g. manually) to reason about the programs w.r.t the semantics The reasoning procedure should be as automated as possible 7

8 Example: reasoning about imperative intermediate languages of our compiler Use Hoare logic to reason about the language Synthesize a mathematical function from an imperative program 8

9 A language satisfying the two requirements (correctness and usability) HOL's specification language a logic language embedded in the higher order logic of HOL its semantics is ensured to be correct (w.r.t the logic) the semantics is tractable (i.e. extremely easy to reason about) since all reasoning mechanisms in HOL are built on it. 9

10 Example f1(x, a) = if (x = 0) then (x,a) else f1 (x - 1, x * a) f2 x = if (x = 0) then 1 else x * f2(x - 1) The semantics is transparent (defined by the logic of the theorem prover) no operational or denotational semantics The semantics is executable e.g. rewrite f2 3 to 6 The semantics is tractable e.g. induct on x to prove x. #2(f1(x,1)) = f2 x 10

11 This language is expressive A logic version of a large subset of Standard ML 11

12 Case Study I: compiling HOL specs A problem: programs run on real machines; compilers may introduce bugs when translating a high level program. Our solution Specify systems (like cryptographic algorithms) in the HOL logic Compile the specifications to assembly Automatically prove the correctness of the compilation 12

13 Example: Trusted Compiler source transformation proof of equivalence code generation 13

14 Compilation Front-end: removing high-level features such as polymorphism, higher-order functions, pattern matching and composite expressions. Mid-end: normalization, inline expansion, register allocation and optimizations (such as SSA conversion, dead code elimination, common expression elimination constant folding, code motion, ect.) Back-end HSL (Heap and Stack Level) CFL (Control Flow Level) ARM SAL (Structured Assembly Language) ARM formal semantics and reasoning mechanisms 14

15 Translation Validation (not verifying the compiler itself) Per-run based proof: the given source and its target is equivalent. We've implemented two register allocators offline: use traditional graph coloring algorithms online: use a small set of rewriting rules The result can be β-reduced to a form that is α-equivalent to original expression. Round ((y,z),(k0,k1,k2,k3),s) = let s' = s + DELTA in let y' = y + ShiftXor(z, s', k0, k1) in ((y', z + ShiftXor(y', s', k2, k3)), (k0,k1,k2,k3), s')` Round ((r0,r1),(r2,r3,m1,m2),m3) = let m4 = r2 in let r2 = m3 in let r2 = r2 + DELTA in let m3 = r3 in let r3 = ShiftXor (r1,r2,m4,m3) in let r0 = r0 + r3 in let r3 = ShiftXor (r0,r2,m1,m2) in let r1 = r1 + r3 in ((r0,r1),(m4,m3,m1,m2),r2) 15

16 Summary of the trusted compiler A proof-based compiler from a higher order logic to assembly (both abstract and realistic). specify programs as mathematical (and logic) functions translate to low-level executable format (e.g. assembly) by proof perform ad-hol translation validation rely heavily on term rewriting Verification Method: mechanical theorem proving translation validation (by comparing the source and target program) 16

17 Case Studies II and III Address concurrent programs Much more difficult to handle concurrent behaviors are tricky Not use (interactive) theorem proving apply more automated (yet less rigid) methods such as model checking and constraint solving 17

18 Example of CPU-GPU hardware/software OpenCL CPU / GPU Communication (image courtesy Khronos Group) 18

19 Parallel Programming Models Message Passing (e.g. MPI) Process 0 Process 1 Process 2 Process 3 SIMD (e.g. CUDA) Grid 1 Block (0, 0) Block (0, 1) Block (1, 0) Block (1, 1) Block (2, 0) Block (2, 1) Block (2, 0) Thread Thread (0, 0) (1, 0) Thread (0, 1) Thread (1, 1) Thread Thread (0, 2) (1, 2) Thread (2, 0) Thread (2, 1) Thread (2, 2) Thread (3, 0) Thread (3, 1) Thread (3, 2) Thread (4, 0) Thread (4, 1) Thread (4, 2) Global Memory Shared Memory 19

20 Concurrency introduces new challenges: Exponentially growing interleavings init: x = 0; y = 0; t0: x++; if (x > 1) assert(0); t1: y++; x++; Traditional testing is ineffective: Question : How can you shuffle the actions of threads t0 and t1 so as to cause the assert(0) assert(0) to be reached? (answer in next slide) 20

21 Reason why testing is ineffective: Exponentially growing interleavings and input values init: x = 0; y = 0; t0: x++; if (x > 1) assert(0); t1: y++; x++; Question : How can you shuffle the actions of threads t0 and t1 so as to cause the assert(0) assert(0) to be reached? Ans: See the arrows! BEGIN HERE! 21

22 Testing v.s Formal Analysis Ad-hoc Testing Incomplete analysis Formal Verification exhaustive analysis Why formal analysis works in practice: * It applies Exhaustive Analysis, as opposed to Incomplete Analysis * It may rely on Abstraction (both manual, and automated) 22

23 Case Study II: Formalizing concurrent APIs (MPI 2.0) A Simple MPI Program P0 MPI_Irecv(rcvbuf1 Irecv(rcvbuf1, *, req1); MPI_Irecv(rcvbuf2 Irecv(rcvbuf2, from 1, req2); MPI_Wait(req1); MPI_Wait(req2); MPI_Bcast(revbuf3 Bcast(revbuf3, root = 1); P1 sendbuf1 = 10; MPI_Bcast(sendbuf1 Bcast(sendbuf1, root = 1); MPI_Isend(sendbuf2 Isend(sendbuf2, to 0, req); MPI_Wait(req); P2 sendbuf2 = 20; MPI_Isend(sendbuf2 Isend(sendbuf2, to 0, req); MPI_Bcast(recvbuf2 Bcast(recvbuf2, root = 1); MPI_Wait(req); 23

24 Concurrent Behaviors P0 Irecv Irecv(from 1) Wait Wait Bcast P1 Bcast Isend Wait P2 Isend Bcast Wait schedule 1: OK P0 Irecv Irecv(from 1) Wait Wait Bcast P1 Bcast Isend Wait P2 Isend Bcast Wait schedule 2: deadlock! 24

25 Verification of MPI Programs 1.Formal Executable Spec of the operations of MPI written in TLA+ 2.Simple MPI / C programs are compiled into TLA+ models and linked with Formal Semantics all under Microsoft VisualStudio (by Robert Palmer and Micheal DeLisi) 3.Errors in Litmus Tests generate error traces that can step the Visual-Studio debugger 25

26 Operational Semantics A state consists of all the MPI objects A MPI function is modeled by a transition rule relating the old state and new state A scheduler to perform message passing and synchronization Low level details (e.g. topology) are abstracted away 26

27 Example: the transfer rule 27

28 The Specification Size of the MPI 2.0 specification (excluding comments and blank lines) 28

29 Summary of the MPI Verification concrete input C MPI program Microsoft Phoenix Compiler TLA+ code TLC Model Checker TLA+ model of MPI calls Problems: Require concrete configuration and input values; Need partial order reduction (and symmetry reduction ) to prune redundant interleavings 29

30 An Alternative: symbolic analysis t0: x++; if (x > 1) assert(0); t1: y++; x++; Symbolic methods can Cover all possible input values in one go Cover all possible interleavings in one go Can smartly avoid un-necessary interleavings Made possible by CONSTRAINT SOLVING ENGINES (Satisfiability Modulo Theory solvers (SMT solvers) which work FAR BETTER than exhaustive search in practice. 30

31 Case Study III: Symbolic Analysis of CUDA kernels CUDA kernel programs are a good fit They are extremely light-weight threaded programs They are SIMD (Single Instruction Multiple Data) programs with high symmetry. The syntax is simple (e.g. without dynamic memory allocation and recursive function calls) Only simple API calls (e.g. math functions) are supported; no need to interpret native API calls. 31

32 The Tool (PUG) Flow C Application Containing Multiple Kernels Analyzer Kernel Descriptions PUG Analyzer for Kernels Verification Results CPU / GPU Communication Codes Kernel Invocation Contexts CPU / GPU Communication Verifier (CGV) Verification Results 32

33 Internal Architecture of PUG Kernel Program In C Analyzer supported by LLNL Rose Verification Conditions in SMT format SMT checker for unsatisfiability UNSAT: The instance is OK OK (no bug is found ) SAT: The instance has bugs (the SAT instance gives a counter example) 33

34 Model programs as logical formulas (symbolic constraints) Program: Symbolic Constraint P: if i > 0 { j = i * 10; k = j i;} else k = i + k; ite (i0 > 0, j1 = i0 * 10 k1 = j1 i0, k1 1 = i0 0 + k0 j1 = j0 ) Claim: Input: i = x, k = y Output: k = x + y? P i0 = x k0 = y k1 x + y Satisfiable: the claim is false Executable: if x and y are concrete then k1's concrete value is returned by the solver 34

35 Concurrency: different interleavings (schedules) lead to different results v is shared; initially v = 0 thread t1 thread t2 v := v + 1; v := a; v := b; v s s possible value after the execution may be 1, a, b, a + 1 or b + 1 schedule 1: v := 0 v := a v := b v := schedule 2: v := b schedule 3: v := a v := a + 1 schedule 4: v := b v := b

36 Explore the interleavings (schedules) in a symbolic manner v is shared; initially v = 0 thread t1 thread t2 v := v + 1; v := a; v := b; with SIds (schedule ids) thread t1 thread t2 v[v t1 1 ] := v[v t1 0] ] + 1; v[v t2 1] ] := a; v[v t2 2] ] := b; Constraints (as logical formulas) v t1 0,, v t1 1,, v t2 1,, v t2 2 {1,2,3,4} v t1 0 < v t1 1 /\ v t2 1 < v t2 2 for i {2,3,4}, v[i] s s previous value is v[i-1] the solutions of these constraints give all the possible schedules A schedule corresponding to the a solution of the constraint t2 1 = 1 t1 0 = 2 t2 2 = 3 t1 1 = 4 v t2 v t1 v t2 v t1 v[v 1 ] := a v = a read v[v 0 ] v = a v[v 2] ] := b v = b v[v 1 ] := v[v 0] ] + 1 v = b

37 Checking specific properties (e.g. races) Can two threads ever conflict on the same memory location? void global kernel (int *a, int b, int N) { int idx = blockidx.x * blockdim.x + threadidx.x; if (idx < N) { a[idx] = a[idx] + b; } } A race occurs when (1) different threads access the same element; (2) at least one of the accesses is write. (and (and (/= t1.x t2.x) (bv-lt t1.x bdim.x) (bv-lt t2.x bdim.x)) (and (= idx1@t1 (bv-add (bv-mul bid.x bdim.x) t1.x)) (ite (bv-lt idx1@t1 N0) true (= a1 a0))) (and (= idx1@t2 (bv-add (bv-mul bid.x bdim.x) t2.x)) (ite (bv-lt idx1@t2 N0) true (= a1 a0))) true true (or (and (bv-lt idx1@t1 N0) (bv-lt idx1@t2 N0) (= idx1@t1 idx1@t2)) (and (bv-lt idx1@t1 N0) (bv-lt idx1@t2 N0) (= idx1@t1 idx1@t2)))) Unsatisfiable No races or synchronization errors are found. solving time = 0.01s 37

38 PUG in Action : Example with a race void global kernel (int *a, int b, int N) { int idx = blockidx.x * blockdim.x + threadidx.x; if (idx < N) a[idx] = a[idx-1] + b; } race occurs for t1 = 0 and t2 = 1. Witness Tells us Exactly how Race happens! (and (and (/= t1.x t2.x) (bv-lt t1.x bdim.x) (bv-lt t2.x bdim.x)) (and (= idx1@t1 (bv-add (bv-mul bid.x bdim.x) t1.x)) (ite (bv-lt idx1@t1 N0) true (= a1 a0))) (and (= idx1@t2 (bv-add (bv-mul bid.x bdim.x) t2.x)) (ite (bv-lt idx1@t2 N0) true (= a1 a0))) true true (or (and (bv-lt idx1@t1 N0) (bv-lt idx1@t2 N0) (= (bv-sub idx1@t1 0b ) idx1@t2)) (and (bv-lt idx1@t1 N0) (bv-lt idx1@t2 N0) (= idx1@t1 idx1@t2)))) satisfiable (= bid.x 0b ) (= bdim.x 0b ) (= t1.x 0b ) (= t2.x 0b ) (= idx1@t1 0b ) (= N0 0b ) (= idx1@t2 0b ) (= (a1 0b ) 0b ) (= (a0 0b ) 0b )!!!!!!! Found conflict: There exists a race on variable a. solving time = 0.01s 38

39 Other Details Modeling Barrier Intervals Handling Conditional Statements Reducing Checking Complexity through Serialization (Partial Order Reduction) Checking Consistent Barrier Usage Incremental Solving Handling Aliasing Incorporating Memory Consistency Models Handling Loops Efficiently (Loop Abstraction) Checking performance bugs (e.g. bank conflicts) 39

40 Experimental Results (Conflict Checking) Kernels (in loc +O +C +R B.C. Time CUDA SDK ) (pass) Bitonic Sort 65 LOW 2.2s MatrixMult 102 * * HIGH <1s Histogram LOW 2.9s Sobel 130 * HIGH 5.6s Reduction 315 HIGH 3.4s Scan 255 * * * LOW 3.5s Scan Large 237 * * LOW 5.7s Nbody 206 * HIGH 7.4s Bisect Large 1400 * * HIGH 44s Radix Sort 1150 * * * LOW 39s Eigenvalues 2300 * * * HIGH 68s + O: require bit-vector computations do not overflow +C: require constraints on the input values +R: require manual refinement B.C.: measure how serious the bank conflicts are Time: SMT solving time 40

41 Ongoing Work Parameterized verification (the analysis is independent of the number of threads) Equivalence checking (tells whether the optimized version of a kernel is equivalent to the original one) Combination of the run-time and the static method Automatic test generation; use the symbolic analyzer to generate inputs for the runtime checker 41

42 Review of this talk 1.A trusted compiler from logic specifications to assembly code 2.A formaliation of MPI programs 3.A symbolic checker for CUDA kernels Method Theorem Proving (1) Model Checking (2) Generality Good Fair Poor Rigidity Good Poor Fair Constraint Solving (3) Productivity Poor Fair Good Automation Poor Good Good 42

43 Thank You! Questions? 43