Verification of an ML compiler. Lecture 1: An introduction to compiler verification

Transcription

1 Verification of an ML compiler Lecture 1: An introduction to compiler verification Marktoberdorf Summer School MOD 2017 Magnus O. Myreen, Chalmers University of Technology

2 Introduction

3 Your program crashes. Where do you look for the fault? Do you look at your source code? Do look at the code for the compiler you used? users want to rely on compilers

4 Verified compilers What? A verified compiler is a compiler that comes with a machine-checker proof. The proof states that the compiler preserves the behaviour of source programs. Traditional compiler development relies on testing. Compiler verification is considered too costly. cost reduction?

5 All (unverified) compilers have bugs Every compiler we tested was found to crash and also to silently generate wrong code when presented with valid input. PLDI 11 Abstract Finding and Understanding Bugs in C Compilers Xuejun Yang Yang Chen Eric Eide John Regehr Compilers should be correct. To improve the quality of C compilers, we created Csmith, a randomized test-case generation tool, and spent three years using it to find compiler bugs. During this period University of Utah, School of Computing { jxyang, chenyang, eeide, regehr }@cs.utah.edu [The verified part of] CompCert is the only compiler we have tested for which Csmith cannot find wrong-code we reported more than 325 previously unknown bugs to compiler developers. Every compiler we tested was found to crash and also to silently generate wrong code when presented with valid input. In this paper we present our compiler-testing tool and the results of our bug-hunting study. Our first contribution is to advance the state of the art in compiler testing. Unlike previous tools, Csmith generates programs that cover a large subset of C while avoiding the s that would destroy its ability 1 int foo (void) { 2 signed char x = 1; 3 unsigned char y = 255; 4 return x > y; 5 } errors. This is not for lack of trying: we have devoted about six CPU-years to the task. Figure 1. We found a bug in the version of GCC that shipped with Ubuntu Linux for x86. At all optimization levels it compiles this function to return 1; the correct result is 0. The Ubuntu compiler was heavily patched; the base version of GCC did not have this bug. d Csmith, a randomized test-case generator that sup- -

6 Motivations Bugs in compilers are not tolerated by users Bugs can be hard to find by testing Verified compilers must be used for verification of source-level programs to imply guarantees at the level of verified machine code Research question: how easy (cheap) can we make compiler verification?

7 State of the art

8 CompCert CompCert C compiler Leroy et al. Source: Compiles C source code to assembly. Has good performance numbers Proved correct in Coq.

9 Values Languages Compiler transformations CakeML compiler Compiles CakeML concrete syntax to machine code. Proved correct in HOL4. Has mostly good performance numbers (later lecture) abstract values incl. closures and ref pointers abstract values incl. ref and code pointers source syntax source AST no modules no cons names no declarations exhaustive pat. matches no pat. match ClosLang: last language with closures (has multi-arg closures) BVL: functional language without closures only 1 global, handle in call DataLang: imperative language Parse concrete syntax Infer types, exit if fail Eliminate modules Replace constructor names with numbers Reduce declarations to exps; introduce global vars Make patterns exhaustive Move nullary constructor patterns upwards Compile pattern matches to nested Ifs and Lets Rephrase language Fuse function calls/apps into multi-arg calls/apps Track where closure values flow; annotate program Introduce C-style fast calls wherever possible Remove deadcode Prepare for closure conv. Perform closure conv. Inline small functions Fold constants and shrink Lets Split over-sized functions into many small functions Compile global vars into a dynamically resized array Optimise Let-expressions Switch to imperative style Reduce caller-saved vars Combine adjacent memory allocations Remove data abstraction Simplify program Known as the first verified compiler to be bootstrapped. I m one of the six developers behind version 2 (diagram to the right). machine words and code labels 32-bit words WordLang: imperative language with machine words, memory and a GC primitive StackLang: imperative language with array-like stack and optional GC LabLang: assembly lang. ARMv6 Select target instructions Perform SSA-like renaming Force two-reg code (if req.) Remove deadcode Allocate register names Concretise stack Implement GC primitive Turn stack access into memory acceses Rename registers to match arch registers/conventions Flatten code Delete no-ops (Tick, Skip) Encode program as concrete machine code later lecture zooms in 64-bit words ARMv8 x86-64 MIPS-64 RISC-V All languages communicate with the external world via a byte-array-based foreign-function interface.

10 A spectrum robust, inflexible more flexible, but can be fragile proved to always work correctly produces a proof for each run Verified compilers Proof-producing compilers Pilsner Fiat CompCert C compiler CakeML compiler Cogent Translation validation for a verified OS kernel CompCertTSO

11 These 4 lectures on Verification of an ML compiler will focus on key concepts rather than implementation details The lectures: Lecture 1: introduction to compiler verification Lecture 2: data representation and garbage collection Lecture 3: closures and call optimisations Lecture 4: compiler bootstrapping

12 but feel free to ask The lectures will not cover: Verification of parsing and ML-style type inference Optimisations (except call optimisations in lecture 3) Modelling and verification of I/O How to manage a large code base of proofs The lectures: Lecture 1: introduction to compiler verification Lecture 2: data representation and garbage collection Lecture 3: closures and call optimisations Lecture 4: compiler bootstrapping

13 Let s get started! Lecture 1: introduction to compiler verification

14 Questions Should we write a new compiler for the verification? yes, because then it has nice invariants What should we prove about it? that the generated programs behave the same as the source programs Do we need to use mechanised proof? yes

15 Proving a compiler correct Ingredients: a formal logic for the proofs accurate models of the source language the target language the compiler algorithm Tools: a proof assistant (software) does this correspond with reality? requires testing Method: prove simulation theorem using induction

16 Ingredient 1: Formal logic

17 Formal logic These lectures will use classical higher-order logic (HOL) HOL = lambda calculus with simple ML-like types Allows for quantification over functions and relations. ` () 8 Example 1: Skolem ` (8 x. 9 y. P x y) () 9 f. 8 x. P x (f x) ` 8 9 () 9 8 Example 2: complete induction over nat: ` (8 n. (8 m. m < n ) P m) ) P n) ) 8 n. P n Example 3: definition of list permutation: ` PERM L 1 L 2 () 8 x. FILTER ((=) x) L 1 = FILTER ((=) x) L 2

18 User-definitions in HOL Datatypes: datatype e = Var string Num int Add e e Assign string e datatype r = Rval int Rbreak Rfail Recursive ` 9 functions ^ (that terminate or are tail-recursive): FILTER P [] = [] FILTER P (h::t) = if P h then h::filter P t else FILTER P t Inductive relations (also co-inductive relations): (S1) (t 1,s) + t (Rval n 1,s 1 ) (t 2,s 1 ) + t r (Seq t 1 t 2,s) + t r (S2) (t 1,s) + t (r,s 1 ) is_rval r (Seq t 1 t 2,s) + t (r,s 1 )

19 Ingredient 2: Language models

20 Syntax The abstract syntax is defined as a datatype. A toy source language: t = Dec string t Exp e Break Seq t t If e t t For e e t e = Var string Num int Add e e Assign string e evaluates an expression Break aborts a loop For-loop expressions can have side-effects (c.f. ML)

21 Syntax The abstract syntax is defined as a datatype. A toy source language: t = Dec string t Exp e Break Seq t t If e t t For e e t e = Var string Num int Add e e Assign string e A toy target language: inst = Add reg reg reg Int reg int Jmp offset JmpIf reg offset an assembly program is a list of instructions

22 Operational Semantics the options There are 4 main variants one can choose from. used for concurrency relational functional common in ACL2 small-step big-step CakeML uses this style avoids treating diverging behaviours separately most common for compiler proofs requires defining: inductive rel. for terminating behaviours; co-inductive rel. for diverging behaviours.

23 Towards a functional big-step semantics (1) We define an interpreter for the source in Standard ML: fun lookup y [] = NONE lookup y ((x,v)::xs) = if y = x then SOME v else lookup y xs fun run_e s (Var x) = (case lookup x s of NONE => (Rfail,s) SOME v => (Rval v,s)) run_e s (Num i) = (Rval i,s) run_e s (Assign (x, e)) = (case run_e s e of (Rval n1, s1) => (Rval n1, (x,n1)::s1) r => r) run_e s (Add (e1, e2)) =... state state is a simple is an a-list assoc-list assignments update the state expression evaluation returns Rval or Rfail and new state

24 Towards a functional big-step semantics (2) and the evaluation of statements (type t): fun run_t s (Exp e) = run_e s e run_t s (Dec (x, t)) = run_t ((x,0)::s) t run_t s Break = (Rbreak, s) run_t s (Seq (t1, t2)) = Break causes an Rbreak (case run_t s t1 of (Rval _, s1) => run_t s1 t2 r => r) Rbreak skips Seq-code run_t s (If (e, t1, t2)) = (case run_e s e of (Rval n1, s1) => run_t s1 (if n1 = 0 then t2 else t1) r => r)

25 Towards a functional big-step semantics (3) so far everything we wrote in ML could be defined in HOL. r => r) run_t s (For (e1, e2, t)) = (case run_e s e1 of (Rval n1, s1) => if n1 = 0 then (Rval 0, s1) else (case run_t s1 t of (Rval _, s2) => (case run_e s2 e2 of but now this recursive call introduces potential non-termination (Rval _, s3) => run_t s3 (For (e1, e2, t)) r => r) (Rbreak, s2) => (Rval 0, s2) r => r) r => r)

26 Why is non-terimination an issue? Suppose HOL allowed definitions of non-terminating functions, e.g. f n = f n + 1 then f n - f n = f n f n and 0 = 1

27 Towards a functional big-step semantics (4) We can force our functions to terminate by inserting a clock. ML: run_t s (For (e1, e2, t)) =... (Rval _, s3) => run_t s3 (For (e1, e2, t)) HOL: eval_t s (For e 1 e 2 t) = in HOL, the state is a record... containing a clock (Rval _,s 3 ) ) if s 3.clock 6= 0 then which gets decremented eval_t (dec_clock s 3 ) (For e 1 e 2 t) else (Rtimeout,s 3 ) if it hits zero, then we abort with an uncatchable exception

28 Functional big-step semantics Other parts do not require clock clutter: HOL: eval_t s (Exp e) = eval_e s e eval_t s (Dec x t) = eval_t (store_var x 0 s) t eval_t s Break = (Rbreak,s) since the size of eval_t s (Seq t 1 t 2 ) = the input shrinks case eval_t s t 1 of (Rval _,s 1 ) ) eval_t s 1 t 2 r ) r eval_t s (If e t 1 t 2 ) = case eval_e s e of (Rval n 1,s 1 ) ) eval_t s 1 (if n 1 = 0 then t 2 else t 1 ) r ) r

29 Observational semantics Compiler proof is to relate different observational semantics. Without I/O, programs can only Terminate, Diverge or Crash. terminates if there is a clock that is sufficient semantics t = if 9 c v s. sem_t (s_with_clock c) t = (Rval v,s) then Terminate else if 8 c. 9 s. sem_t (s_with_clock c) t = (Rtimeout,s) then Diverge else Crash diverges if all clocks cause timeouts crashes otherwise, e.g. when Rbreak propagates to the top

30 Exercise Define a functional big-step semantics in your favourite prover (e.g. HOL4, Isabelle/HOL, Coq, ACL2) There is a trick to the termination proof. Ask me!

31 Ingredient 3: Compiler function

32 Ingredient 3: Compiler function Three phases: phase 1: rewrite For to something simpler phase 2: split expressions into instructions phase 3: flatten to list of assembly instructions

33 Ingredient 3: Compiler function Three phases: ) phase 1: rewrite For to something simpler phase1 (Exp e) = Exp e phase1 (Dec x t) = Seq (Exp (Assign x (Num 0))) (phase1 t) phase1 Break = Break phase1 (Seq t 1 t 2 ) = Seq (phase1 t 1 ) (phase1 t 2 ) phase1 (If e t 1 t 2 ) = If e (phase1 t 1 ) (phase1 t 2 ) phase1 (For e 1 e 2 t) = Loop (If e 1 (Seq (phase1 t) (Exp e 2 )) Break) where Loop t = For (Num 1) (Num 1) t. Makes all loops simple while-true loops.

34 Proving a compiler correct Ingredients: a formal logic for the proofs accurate models of the source language the target language the compiler algorithm Tools: a proof assistant (software) Method: prove simulation theorem using induction

35 Induction theorem ` 8P. (8 s e. P s (Exp e)) ^ (8 s x t. P (store_var x 0 s) t ) P s (Dec x t)) ^ (8 s. P s Break) ^ (8 s t 1 t 2. (8 v 2 s 1 v 5. (eval_t s t 1 = (v 2,s 1 )) ^ (v 2 = Rval v 5 ) ) P s 1 t 2 ) ^ P s t 1 ) P s (Seq t 1 t 2 )) ^ (8 s e t 1 t 2. (8 v 2 s 1 n 1. (eval_e s e = (v 2,s 1 )) ^ (v 2 = Rval n 1 ) ) P s 1 (if n 1 = 0 then t 2 else t 1 )) ) construct. P s (If e t 1 t 2 )) ^ (8 s e 1 e 2 t. (8 v 4 s 1 n 1 v 3 s 2 v 7 v 2 s 3 v 5. (eval_e s e 1 = (v 4,s 1 )) ^ (v 4 = Rval n 1 ) ^ n 1 6= 0 ^ (eval_t s 1 t = (v 3,s 2 )) ^ (v 3 = Rval v 7 ) ^ (eval_e s 2 e 2 = (v 2,s 3 )) ^ (v 2 = Rval v 5 ) ^ s 3.clock 6= 0 ) P (dec_clock s 3 ) (For e 1 e 2 t)) ^ (8 v 4 s 1 n 1. (eval_e s e 1 = (v 4,s 1 )) ^ (v 4 = Rval n 1 ) ^ n 1 6= 0 ) P s 1 t) ) P s (For e 1 e 2 t)) ) 8 v v 1. P v v 1 Has structure of functional big-step interpreter. One case for each program

36 Simulation proof Particularly simple for phase 1: ` 8s t. eval_t s (phase1 t) = eval_t s t Requires only 8 lines of HOL4 proof script (mostly rewriting). We can lift this to the observational semantics with a one-line proof by rewriting: ` 8t. semantics (phase1 t) = semantics t

37 Simulation theorems in general Usually, each compile phase requires a theorem of the form: non-failing evaluation of source intermediate language (IL) evaluate code s1 = (res,s2) res Rfail state_rel s1 t1 implies t2. evaluate (compile code) t1 = (res,t2) state_rel s2 t2 similar evaluation in target IL w.r.t. some relation between states (state_rel)

38 Simulation theorems in general Usually, each compile phase requires a theorem of the form: evaluate code s1 = (res,s2) res Rfail state_rel s1 t1 t2. evaluate (compile code) t1 = (res,t2) state_rel s2 t2 usually: state_rel keeps clocks in sync variant: target can consume more clock ticks

39 Simulation theorems in general Usually, each compile phase requires a theorem of the form: evaluate code s1 = (res,s2) res Rfail state_rel s1 t1 t2. evaluate (compile code) t1 = (res,t2) state_rel s2 t2 Sufficient to prove observational equivalence for both terminating and diverging runs.

40 Phase 2 simulation theorem 9 ^ ^ source IL ` 8s e t n res s 1. (eval_t s e = (res,s 1 )) ^ res 6= Rfail ^ phase2_subset e ^ possible_var_name n s.store ^ s.store v t.store ^ target IL not failing compiler (t.clock = s.clock) ^ t_max e < strlen n ) 9 t 1. (eval_t t (flatten_t e n) = (res,t 1 )) ^ s 1.store v t 1.store ^ (t 1.clock = s 1.clock) ^ possible_var_name n s 1.store ^ 8 k v. restrict language syntax following phase 1 state_rel relation possible_var_name k s.store ^ t_max e < strlen k ^ strlen k < strlen n ^ (lookup t.store k = SOME v) ) (lookup t 1.store k = SOME v) extra property

41 Composing top-level theorems Each phase maintains observational equivalence: semantics (phase1 t) = semantics t semantics t 6= Crash ^ phase2_subset t ) (semantics (phase2 t) = semantics t) semantics t 6= Crash ^ phase3_subset t ) (asm_semantics (phase3 0 0 t) = semantics t) observational semantics of target assembly Here: compile t = phase3 0 0 (phase2 (phase1 t))

42 Composing top-level theorems lemma: correct syntax implies no Crashes Result: for ML: type-correct program implies no Crash ` 8t. syntax_ok t ) (asm_semantics (compile t) = semantics t) where compile t = phase3 0 0 (phase2 (phase1 t))

43 What we learnt Ingredients: formal logic, compiler, language semantics Tools: proof assistant Method: using functional big-step semantics it suffices to prove theorems of the form: evaluate code s1 = (res,s2) res Rfail state_rel s1 t1 t2. evaluate (compile code) t1 = (res,t2) state_rel s2 t2 in order to prove observational equivalence, i.e. ` 8t. syntax_ok t ) (asm_semantics (compile t) = semantics t)

44 Extra slides

45 Comparing functional with relation big-step In the functional version, Seq was specified by: eval_t s Break = (Rbreak,s) eval_t s (Seq t 1 t 2 ) = case eval_t s t 1 of (Rval _,s 1 ) ) eval_t s 1 t 2 r ) r In the relational version, Seq is specified using four rules: (S1) (t 1,s) + t (Rval n 1,s 1 ) (t 2,s 1 ) + t r (Seq t 1 t 2,s) + t r (S2) (t 1,s) + t (r,s 1 ) is_rval r (Seq t 1 t 2,s) + t (r,s 1 ) + (S1 0 ) (t 1,s) * t (Seq t 1 t 2,s) * t (S2 0 ) (t 1,s) + t (Rval n 1,s 1 ) (t 2,s 1 ) * t (Seq t 1 t 2,s) * t

46 Induction from relational big-step `... ^... ^... ^... ^... ^... ^... ^... ^ (8 s s 1 e 1 e 2 t. (e 1,s) + e (Rval 0,s 1 ) ) P (For e 1 e 2 t,s) (Rval 0,s 1 )) ^ (8 s s 1 e 1 e 2 t r. (e 1,s) + e (r,s 1 ) ^ is_rval r ) P (For e 1 e 2 t,s) (r,s 1 )) ^ (8 s s 1 s 2 s 3 e 1 e 2 t n 1 n 2 n 3 r. (e 1,s) + e (Rval n 1,s 1 ) ^ n 1 6= 0 ^ P (t,s 1 ) (Rval n 2,s 2 ) ^ (e 2,s 2 ) + e (Rval n 3,s 3 ) ^ P (For e 1 e 2 t,s 3 ) r ) P (For e 1 e 2 t,s) r) ^ (8 s s 1 s 2 e 1 e 2 t n 1. (e 1,s) + e (Rval n 1,s 1 ) ^ n 1 6= 0 ^ P (t,s 1 ) (Rbreak,s 2 ) ) P (For e 1 e 2 t,s) (Rval 0,s 2 )) ^ (8 s s 1 s 2 s 3 e 1 e 2 t n 1 n 2 r. (e 1,s) + e (Rval n 1,s 1 ) ^ n 1 6= 0 ^ P (t,s 1 ) (Rval n 2,s 2 ) ^ (e 2,s 2 ) + e (r,s 3 ) ^ is_rval r ) P (For e 1 e 2 t,s) (r,s 3 )) ^ (8 s s 1 s 2 e 1 e 2 t n 1 r. (e 1,s) + e (Rval n 1,s 1 ) ^ n 1 6= 0 ^ P (t,s 1 ) (r,s 2 ) ^ is_rval r ^ r 6= Rbreak ) P (For e 1 e 2 t,s) (r,s 2 )) ) 8 ts rs. ts + t rs ) P ts rs It has one rule for each case in the relation Six cases for For!

47 Observational semantics with I/O Defining the observational semantics when there is I/O. semantics t input (Terminate io_trace) () 9 c nd i s. (sem_t (init_st c nd input) t = (Rval i,s)) ^ (FILTER ISL s.io_trace = io_trace) semantics t input Crash () 9 c nd r s. (sem_t (init_st c nd input) t = (r,s)) ^ ((r = Rbreak) _ (r = Rfail)) semantics t input (Diverge io_trace) () 9 nd. (8 c. 9 s. sem_t (init_st c nd input) t = (Rtimeout,s)) ^ (io_trace W = c. fromlist (FILTER ISL (SND (sem_t (init_st c nd input) t)).io_trace))