How Can You Trust Formally Verified Software?

Transcription

1 How Can You Trust Formally Verified Software? Alastair Reid Arm

2 2

3 Buffer over-read vulnerabilities 3

4 Buffer over-read vulnerabilities Logic error vulnerabilities 3

5 Buffer over-read vulnerabilities Logic error vulnerabilities Null Buffer pointer over-read dereference vulnerabilities 3

6 Buffer over-read vulnerabilities Use after free Logic error vulnerabilities Null Buffer pointer over-read dereference vulnerabilities 3

7 Buffer over-read vulnerabilities Use after free s e i t i l i b a r e n l u v r o r r e Logic Buffer overflow vulnerabilities Null pointer dereference Buffer over-read vulnerabilities 3

8 Formal verification Of libraries and apps Of compilers Of operating systems 4

9 Fonseca et al., An Empirical Study on the Correctness of Formally Verified Distributed Systems, Eurosys 17 Formally Verified Software 5

10 Fonseca et al., An Empirical Study on the Correctness of Formally Verified Distributed Systems, Eurosys 17 Formally Verified Software Verification Tool Shim Code Formal Specifications 5

11 Takeaway #1: 3 key questions to ask 1. What specifications does your proof rely on? 2. Why do you trust those specifications? 3. Does anybody else use these specifications? 6

12 Takeaway #2: Specifications must have multiple uses 7

13 Takeaway #2: Specifications must have multiple uses 8

14 How can you trust formally verified software? How can you trust formally verified software? Specifications are part of the TCB 3 key questions Specifications must have multiple users How can you trust formal specifications? Testing specifications Verifying processors Verifying specifications How can you trust formally verified software? 9

15 Trustworthy Specifications of the ARM v8-a and v8-m architecture, FMCAD 2016 Creating trustworthy specifications

16 The state of most processor specifications Large (1000s of pages) Broad (10+ years of implementations, multiple manufacturers) Complex (exceptions, weak memory, ) Informal (mostly English prose) We are all just learning how to (retrospectively) formalize specifications 11

17 Arm Processor Specifications A-class (phones, tablets, servers, ) 6,000 pages 40,000 line formal specification Instructions (32/64-bit) Exceptions / Interrupts Memory protection Page tables Multiple privilege levels System control registers Debug / trace M-class (microcontrollers, IoT) 1,200 pages 15,000 line formal specification Instructions (32-bit) Exceptions / Interrupts Memory protection Page tables Multiple privilege levels System control registers Debug / trace 12

18 English prose 13

19 Pseudocode 14

20 Arm Architecture Specification Language (ASL) Indentation-based syntax Imperative First-order Strongly typed (type inference, polymorphism, dependent types) Bit-vectors Unbounded integers Infinite precision reals Arrays, Records, Enumerations Exceptions 15

21 ASL Spec Lexer Parser Typecheck C Backend Interpreter 16

22 Architectural Conformance Suite Processor architectural compliance sign-off Large v8-a 11,000 test programs, > 2 billion instructions v8-m 3,500 test programs, > 250 million instructions Thorough Tests dark corners of specification 17

23 Testing Pass Rate (Artists Impression) ISA Supervisor Hypervisor/Security Time 18 18

24 v8-m 100% 75% 50% 25% 0% 19 19

25 Measuring architecture coverage of tests Untested: op1*op2 == -3.0, FPCR.RND=-Inf 20

26 21

27 End to End Verification of ARM processors with ISA Formal, CAV 2016 Formal verification of processors

28 Checking an instruction ADD ARMResearch 23

29 Checking an instruction CMP LDR ADD STR BNE Context ARMResearch 23

30 IF ID EX MEM WB Fetch Decode R0 - R15 R0 - R15 Memory 24

31 IF ID EX MEM WB Fetch Decode R0 - R15 R0 - R15 Memory πpost πpre 24

32 IF ID EX MEM WB Fetch Decode R0 - R15 R0 - R15 Memory πpost Post_cp πpre Pre Post_spe Spec ==? 24

33 Architecture Specification ASL to Verilog Combinational Verilog Specialize Monomorphize Constant Propagation Width Analysis Exception Handling ARMResearch 25

34 Arm CPUs verified with ISA-Formal A-class Cortex-A53 Cortex-A32 Cortex-A35 Cortex-A55 Next generation R-class Cortex-R52 Next generation M-class Cortex-M4 Cortex-M7 Cortex-M33 Next generation Cambridge Projects Rolling out globally to other design centres 26 Sophia, France - Cortex-A75 (partial) Austin, USA - TBA Chandler, USA - TBA

35 27

36 Who guards the guards? Formal Validation of ARM v8-m Specifications OOPSLA 2017 Formal validation of specifications

37 Suppose Last year: audited all accesses to privileged registers Specification: Added missing privilege checks Testsuite: Added new tests to test every privilege check Formal testbench: Verify every check This year: add new instruction but accidentally omit privilege check How many tests in the test suite will fail on new specification? 29

38 Executable Specification Defines what is allowed Animation Check spec matches expectation Testable Compare spec against implementation 30

39 Executable Specification Defines what is allowed Animation Check spec matches expectation Testable Compare spec against implementation Does not define what is not allowed e.g., Impossible states, impossible actions/transitions, security properties No redundancy Problem when extending specification 30

40 Creating a specification of disallowed behaviour Where to get a list of disallowed behaviour? How to formalise this list? How to formally validate specification against spec of disallowed behaviour? 31

41 Rule JRJC Exit from lockup is by any of the following: A Cold reset. A Warm reset. Entry to Debug state. Preemption by a higher priority processor exception. 32

42 Rule R JRJC Exit State from Change lockup X is by any of the following: Event A Cold A reset. Event A Warm B reset. State Entry to Change Debug C state. Event Preemption D by a higher priority processor exception. 32

43 Rule R JRJC Exit State from Change lockup X is by any of the following: Event A Cold A reset. Event A Warm B reset. State Entry to Change Debug C state. Event Preemption D by a higher priority processor exception. And cannot happen any other way 32

44 Rule R JRJC Exit State from Change lockup X is by any of the following: Event A Cold A reset. Event A Warm B reset. State Entry to Change Debug C state. Event Preemption D by a higher priority processor exception. And cannot happen any other way Rule R: X A B C D 32

45 State Change X Exit from lockup Fell(LockedUp) Event A A Cold reset Called(TakeColdReset) Event B A Warm reset Called(TakeReset) State Change C Entry to Debug state Rose(Halted) Event D Preemption by a higher priority processor exception Called(ExceptionEntry) 33

46 Rule JRJC Exit from lockup is by any of the following: A Cold reset. A Warm reset. Entry to Debug state. Preemption by a higher priority processor exception. 34

47 Rule JRJC Exit from lockup is by any of the following: A Cold reset. A Warm reset. Entry to Debug state. Preemption by a higher priority processor exception. Fell(LockedUp) Called(TakeColdReset) Called(TakeReset) Rose(Halted) Called(ExceptionEntry) 34

48 Rule JRJC Exit from lockup is by any of the following: A Cold reset. A Warm reset. Entry to Debug state. Preemption by a higher priority processor exception. Fell(LockedUp) Called(TakeColdReset) Called(TakeReset) Rose(Halted) Called(ExceptionEntry) Called_TakeColdReset = FALSE; Called_TakeReset = FALSE; Called_TakeExceptionEntry = FALSE; Past_LockedUp = LockedUp; Past_Halted = Halted; 34 assert(( Past_LockedUp > LockedUp) ==> ( Called_TakeColdReset Called_TakeReset Past_Halted < Halted Called_ExceptionEntry));

49 Rule VGNW Entry to lockup from an exception causes Any Fault Status Registers associated with the exception to be updated. No update to the exception state, pending or active. The PC to be set to 0xEFFFFFFE. EPSR.IT to become UNKNOWN. In addition, HFSR.FORCED is not set to 1. 35

50 Rule VGNW Entry to lockup from an exception causes Any Fault Status Registers associated with the exception to be updated. Out of date No update to the exception state, pending or active. Misleading The PC to be set to 0xEFFFFFFE. Untestable EPSR.IT to become UNKNOWN. Ambiguous In addition, HFSR.FORCED is not set to 1. 35

51 Arm Specification Language Arithmetic operations Boolean operations Bit Vectors Arrays Functions Local Variables Statements Assignments If-statements Loops Exceptions SMT Arithmetic operations Boolean operations Bit Vectors Arrays Functions Local Variables Statements Assignments If-statements Loops Exceptions 36

52 Formally Validating Specifications v8-m Spec Property Verification CEX Bug in Spec Proof 37

53 Formally Validating Specifications 12 Bugs v8-m Spec Verification CEX Found Bug in Spec so far Property Proof 37

54 38

55 ARM Test Suite C Backend Test Coverage ASL Spec Lexer Parser Typechecker Interpreter Verilog Backend Arm Processor Simulation Trace Bounded Model Architecture Properties SMT Backend SMT Solver 39

56 Public release of machine readable Arm specification Enable formal verification of software and tools Releases April 2017: v8.2 July 2017: v8.3 Working with Cambridge University REMS group to convert to SAIL Backends for HOL, OCaml, Memory model, (hopefully Coq too) Tools: Talk to me about how I can help you use it 40

57 Potential uses of processor specifications Verifying compilers Verifying OS page table / interrupt / boot code Verifying processor pipelines Verification and discovery of peephole optimizations Automatic generation of binary translators Automatic generation of test cases Decompilation of binaries Abstract interpretation of binaries etc. 41

58 How can you trust formally verified software?

59 How can you trust formal specifications? Test the specifications you depend on Ensure specifications have multiple uses Create meta-specifications 43 Hiring in Security and Correctness group contact me

60 Thank You! Danke! Merci! ありがとう! Gracias! Kiitos! Trustworthy Specifications of the ARM v8-a and v8-m architecture, FMCAD 2016 End to End Verification of ARM processors with ISA Formal, CAV 2016 Who guards the guards? Formal Validation of ARM v8-m Specifications, OOPSLA