Model Checking. Automatic Verification Model Checking. Process A Process B. when not possible (not AI).

Transcription

1 Sérgio Campos Why? Imagine the implementation of a complex hardware or software system: A 100K gate ASIC perhaps 100 concurrent modules; A flight control system dozens of concurrent processes in multiple CPUs. Under test the system fails approximately every 3 days. Failures is not repeatable race conditions; Internal signals are hard to watch; Too much data; Heisenbug. The reason could be: x and y happen simultaneously every times Assumed mutual exclusion. 1 / 32 3 / 32 This course will be about: Not testing, not simulation... No manual proofs... Formal Guaranteed results whenever possible, knowing when not possible (not AI). Model checking. Mostly. Clarke, E., Grumberg, O., Peled, D.. The MIT Press, A trivial example: Process A Process B x++; x--; These errors can be prevented by good practice: semaphores, monitors, etc. But other errors can be more subtle... 2 / 32 4 / 32

2 The Analyzer does not see Sensor, and blocks Reporter indefinitely What if the Analyzer decides to run? Processes communicate using shared memory. Priority order: Sensor (most important); Analyzer; Reporter; Consider a air traffic control system: 6 / 32 8 / 32 7 / 32 It happened In Mars NASA Pathfinder, 1997: Verification Verification 5 / 32 Verification Sensor may never be blocked But the events below cause priority inversion: Verification

3 Pentium FDIV: SRT Division Circuit P0 = dividend r Pj 1 = r Pj qj 1 divisor Qj 1 = r Qj + qj 1 Radiotherapy Between 1985 and 1986 the radiotherapy machine massively overdosed 6 patients causing 2 deaths and worsening the other patients conditions severely. Two modes of operation: After entering patients data, it is possible to edit this data. In some cases this causes a change in mode of operation, from x-ray to electron mode. 9 / / 32 SRT Division Circuit Quotient Logic: Table was compressed to remove unreachable entries. reachable entries were considered unreachable and removed This caused an error only for operands that try to use those entries. Rare, but it sure happens And it cost them US$500M Error caused by: Race conditions in accessing shared variables. No hardware interlock: in this case, a pin holding the filter in place. The previous machine had a hardware interlock, but it was removed from the They used the same software, and they considered the software correct 10 / / 32

4 In June 4, 1996, the rocket was launched for its first flight test 37 seconds after, it veered off its trajectory and was self-destructed. The error: Software reused from Ariane 4 But the previous rocket was much less powerful Navigation software detected a much stronger change in course than expected And the software that handled it overflowed. Unknown Incident... define three 3... three++;... a = a + three; / / 32 One of the most expensive bugs in history: 10 years development at the cost of US$ ,00 (7 billion ) What can be done? Simulation and testing are problematic We will consider a mathematical approach. It requires: A model of the system as a mathematical object. The model represents formally: program state; transition relation. A specification method for expressing properties such as: A req is always followed by an ack A proof method to show that the model satisfies the properties. Proving can be done: by hand; semi-automatic: user suggests a proof that is machine checked; fully automatic. 14 / / 32

5 I object Proofs are about the models, not the systems; Specs are subject to error and incompleteness; The software that generates proofs can be buggy; Computer proofs are unreadable. Not really... Lets put aside mathematical certainty as our first goal: We can use formal methods as a methodology that helps produce more reliable systems. Even hand proofs are likely to produce better programs. Increased automation makes them even better. A Selective Early Program proving suggested by a few visionaries (McCarthy, Dijkstra). Late Software crisis declared at NATO conference on Software Engineering. Floyd s method for flow charts: Label each control point with invariant assertions Proof: all assertions preserved by program transitions. This proves partial correctness (if program terminates, result is correct). 17 / / 32 The Real Problem Real systems are several orders of magnitude larger than what can be handled. A simple program can have 10 integer variables: = 320 boolean variables; Thats = configurations is the number of atoms in the universe Can we do non trivial things? Automation: fully automated methods help. Efficiency: e.g. with BDDs weve gone from 10 5 to overnight. Compositional reasoning: divide into simpler subsystems and verify them.... Late 1960s Hoare logic a structured approach: Hoare triple P}S Q} if P holds and S terminates, then Q holds Axiom for assignment P x }x := f P} example y >= 0}x := y x >= 0} Compositional rules Sequential composition P1}S1 P2} P2}S2 P3} P1}S1; S2 P3} While loop P B}S P} P}whileB os P B} 18 / / 32

6 Late 1960s Hoare logic a structured approach: The proof follows the program structure: 1. Prove properties of program components; 2. Combine properties using inference rules. First example of compositional approach. 1970s Proofs of concurrent programs Owicki-Gries Difficult to identify control points: one for each process; Suggested a compositional system for concurrent programs: Data shared only through resources. Critical sections to access resources. Resource invariant I (r) Triple P}S Q} means that if P is true initially, then: 1. Finally (if S terminates) Q; 2. I (r) is preserved while S is in critical sections Coroutine rule: P1}S1 Q1} P2}S2 Q2} I (r) P1 P2}S1 S2 I (r) Q1 Q2} Aux. vars added to aid in stating invariants 21 / / s Large amount of work done on Axiomatizing various languages, constructs, e.g procedure calls, assignments to arrays, etc. Automating proofs: Notably, the Boyer-Moore prover can automatically prove correctness of some recursively defined LISP functions, (e.g. APPEND), without the user having to suggest an invariant. Late 1970s Temporal logic and reactive systems. Partial correctness and termination are not adequate abstractions of concurrent processes. Notion of a reactive system: interacts with its environment without terminating (Pnueli) Examples of reactive systems: control/operating systems communication protocols hardware 22 / / 32

7 Late 1970s Temporal logic and reactive systems. Need to be able to state and prove properties of execution sequences: If P is scheduled infinitely often, it eventually produces an output (liveness). If P sends a message, then it won t send another message until it receives an ack (safety). Late 1970s Protocol analysis and reachability analysis The model: Set of program states Transition relation between states Idea: instead of reasoning symbolically about the model, just construct it explicitly This is called reachability analysis. Advantage: Totally automated. Disadvantages: 1. Can verify only limited properties (non-compositional). 2. State explosion problem: Can handle only fairly small models For these reasons, this technique was more or less dismissed / / 32 Late 1970s Temporal Operators G p = Globally p p = Eventually p X p = At the next time p Example: G scheduled(p) terminated(p) if p is always eventually scheduled, it eventually terminates. Temporal logic allows us to reason compositionally about parallel programs As Hoare allows us to reason about sequential ones. However, this does not mean that it is easy. e.g., Hailpern provides a 15 page proof of the correctness of a nearly trivial protocol (the ABP). Symbolic X Explicit 26 / / 32

8 Model checking (Clarke/Emerson, Queille/Sifakis) 1. Specify properties in TL; 2. Build finite state model explicitly; 3. Check that model satisfies specifications; 4. Produce counterexample when false. This approach has a number of advantages: High expressiveness (> reachability analysis). Is compositional, since based on TL. Complexity is linear in formula size for CTL. Very good for disproving properties. For small but tricky asynchronous circuits: Showed previously published designs incorrect (embarrassing). Showed previously proved correct designs incorrect (highly embarrassing). Late and 1990s Partial order methods Often, state explosion is due to many (irrelevant) orderings of independent events. Symmetry reductions Consider only one state in each class of symmetrically related states. Often process id s, address and data values are equivalent under symmetry transformations. Real-Time Discrete: Verus Continuous: Timed automata 29 / / 32 Late and 1990s Success of model checking on small examples leads to various attacks on the state explosion problem. Modularity - abstracting system components Using temporal formulas Abstract FS models by other FS models. ally generated reductions Symbolic model checking techniques Use compact boolean forms to represent state sets and transition relations (e.g. BDD s) Can handle state spaces many orders of magnitude larger. Other domains: Software Bioinformatics Other representations Bounded s SAT solvers Statistical Used by: Intel, Motorola, IBM, Siemens,... Formal verification companies: Cadence Verplex Jasper 30 / / 32