How Can You Trust Formally Verified Software?

Transcription

1 How Can You Trust Formally Verified Software? Alastair Reid Arm

2 Formal verification Of libraries and apps Of compilers Of operating systems 2

3 Fonseca et al., An Empirical Study on the Correctness of Formally Verified Distributed Systems, Eurosys 17 Formally Verified Software 3

4 Fonseca et al., An Empirical Study on the Correctness of Formally Verified Distributed Systems, Eurosys 17 Formally Verified Software Verification Tool Shim Code Formal Specifications 3

5 Takeaway #1: 3 key questions to ask 1. What specifications does your proof rely on? 2. Why do you trust those specifications? 3. Does anybody else use these specifications? 4

6 Takeaway #2: Specifications must have multiple uses 5

7 Takeaway #2: Specifications must have multiple uses 6

8 How can you trust formally verified software? How can you trust formally verified software? Specifications are part of the TCB 3 key questions Specifications must have multiple users How can you trust formal specifications? Testing specifications Verifying processors Verifying specifications How can you trust formally verified software? 7

9 Trustworthy Specifications of the ARM v8-a and v8-m architecture, FMCAD 2016 Creating trustworthy specifications

10 Arm Architecture Reference Manual (ARMARM) Pages bit / 64-bit Instructions Exceptions / Interrupts Privilege / Security Virtual Memory System registers Debug / Trace Profiling 9

11 English prose 10

12 Pseudocode 11

13 Arm Architecture Specification Language (ASL) Indentation-based syntax Imperative First-order Strongly typed (type inference, polymorphism, dependent types) Bit-vectors Unbounded integers Infinite precision reals Arrays, Records, Enumerations Exceptions 12

14 ASL Spec Lexer Parser Typechecker C Backend Interpreter 13

15 Architectural Conformance Suite Processor architectural compliance sign-off Large v8-a 11,000 test programs, > 2 billion instructions v8-m 3,500 test programs, > 250 million instructions Thorough Tests dark corners of specification 14

16 Tesfng Pass Rate (Arfsts Impression) ISA Supervisor Hypervisor/Security Time 15 15

17 v8-m 100% 75% 50% 25% 0% 16 16

18 Measuring architecture coverage of tests Untested: op1*op2 == -3.0, FPCR.RND=-Inf 17

19 18

20 End to End Verification of ARM processors with ISA Formal, CAV 2016 Formal verification of processors

21 Checking an instrucfon ADD ARMResearch 20

22 Checking an instrucfon CMP LDR ADD STR BNE Context ARMResearch 20

23 IF ID EX MEM WB Fetch Decode R0 - R15 R0 - R15 Memory 21

24 IF ID EX MEM WB Fetch Decode R0 - R15 R0 - R15 Memory πpost πpre 21

25 IF ID EX MEM WB Fetch Decode R0 - R15 R0 - R15 Memory πpost Post_cpu πpre Pre Post_spec Spec ==? 21

26 Architecture Specification ASL to Verilog Combinational Verilog Specialize Monomorphize Constant Propagation Width Analysis Exception Handling ARMResearch 22

27 Arm CPUs verified with ISA-Formal A-class Cortex-A53 Cortex-A32 Cortex-A35 Cortex-A55 Next generation R-class Cortex-R52 Next generation M-class Cortex-M4 Cortex-M7 Cortex-M33 Next generation Cambridge Projects Rolling out globally to other design centres 23 Sophia, France - Cortex-A75 (partial) Austin, USA - TBA Chandler, USA - TBA

28 24

29 Who guards the guards? Formal Validation of ARM v8-m Specifications OOPSLA 2017 Formal validation of specifications

30 Suppose Last year: audited all accesses to privileged registers Specification: Added missing privilege checks Testsuite: Added new tests to test every privilege check Formal testbench: Verify every check This year: add new instruction but accidentally omit privilege check How many tests in the test suite will fail on new specification? 26

31 Can we formally verify specification? Specification of the specification Disallowed behaviour Invariants Cross-cutting properties Tools that can prove properties of ASL specifications 27

32 28

33 State change 28

34 State change Event 28

35 29

36 Input Output State State 29

37 Input ExceptionEntry Output State State 29

38 Input ExceptionEntry Output State TakeReset State 29

39 rule lockup_exit assume Fell(LockedUp); Called(TakeColdReset) Called(TakeReset) Rose(InDebugState()) Called(ExceptionEntry); 30

40 Converting ASL to SMT Functions Local Variables Statements Assignments If-statements Exceptions Arithmetic operations Boolean operations Bit Vectors Arrays Functions Local Variables Statements Assignments If-statements Exceptions Arithmetic operations Boolean operations Bit Vectors Arrays 31

41 Formally Validating Specifications v8-m Spec Property Verification CEX Bug in Spec Proof 32

42 Formally Validating Specifications 12 Bugs v8-m Spec Verification CEX Found Bug in Spec so far Property Proof 32

43 33

44 33 rule lockup entry assume Rose(LockedUp); assume Called(TakeReset); property a HaveMainExt() CFSR!= 0; property b1 Stable(ExnPending); property b2 Stable(ExnActive); property c PC == 0xEFFFFFFE; property e HFSR.FORCED == 0;

45 33 rule lockup entry assume Rose(LockedUp); assume Called(TakeReset); property a HaveMainExt() CFSR!= 0; property b1 Stable(ExnPending); property b2 Stable(ExnActive); property c PC == 0xEFFFFFFE; property e HFSR.FORCED == 0; Stable(HFSR.FORCED);

46 33 Debug view of is not changed. rule lockup entry assume Rose(LockedUp); assume Called(TakeReset); property a HaveMainExt() CFSR!= 0; property b1 Stable(ExnPending); property b2 Stable(ExnActive); property c PC == 0xEFFFFFFE; property e HFSR.FORCED == 0; Stable(HFSR.FORCED);

47 34

48 ARM Test Suite C Backend Test Coverage ASL Spec Lexer Parser Typechecker Interpreter Verilog Backend Simulation Trace Bounded Model Checker Arm Processor Architecture Properties SMT Backend SMT Solver 35

49 Public release of machine readable Arm specification Enable formal verificafon of somware and tools Releases April 2017: v8.2 July 2017: v8.3 Working with Cambridge University REMS group to convert to SAIL Backends for HOL, OCaml, Memory model, (hopefully Coq too) Tools: hops://github.com/alastairreid/mra_tools Talk to me about how I can help you use it 36

50 Potential uses of processor specifications Verifying compilers Verifying OS page table / interrupt / boot code Verifying processor pipelines Verification and discovery of peephole optimizations Automatic generation of binary translators Automatic generation of test cases Decompilation of binaries Abstract interpretation of binaries etc. 37

51 How can you trust formally verified software?

52 How can you trust formal specifications? Test the specifications you depend on Ensure specifications have multiple uses Create meta-specifications 39

53 Thank You! Danke! 谢谢! ありがとう! Gracias! Kiitos! Trustworthy Specifications of the ARM v8-a and v8-m architecture, FMCAD 2016 End to End Verification of ARM processors with ISA Formal, CAV 2016 Who guards the guards? Formal Validation of ARM v8-m Specifications OOPSLA