CISC : Finite-State Verification

Transcription

1 CISC : Finite-State Verification Stephen F. Siegel Department of Computer and Information Sciences University of Delaware Fall

2 The Software Crisis The desire for formal software verification arose out of the acknowledgment of an (apparently permanent) software crisis. 2

3 The Software Crisis The desire for formal software verification arose out of the acknowledgment of an (apparently permanent) software crisis. From crisis: Notion of software crisis arose in late 1960s refers to the difficulty of writing correct, understandable and verifiable computer programs 2

4 The Software Crisis The desire for formal software verification arose out of the acknowledgment of an (apparently permanent) software crisis. From crisis: Notion of software crisis arose in late 1960s refers to the difficulty of writing correct, understandable and verifiable computer programs Specifically: software projects run over-budget software projects run over-time software is of low quality software often does not meet requirements projects are often unmanageable and code difficult to maintain 2

5 1968 NATO Conference on Software Engineering 3

6 1968 NATO Conference on Software Engineering convened by NATO Science Committee refers to software crisis and software gap 4

7 1968 NATO Conference on Software Engineering convened by NATO Science Committee refers to software crisis and software gap One of the major motivations for the organizing of the conference was an awareness of the rapidly increasing importance of computer software systems in many activities of society. 4

8 1968 NATO Conference on Software Engineering convened by NATO Science Committee refers to software crisis and software gap One of the major motivations for the organizing of the conference was an awareness of the rapidly increasing importance of computer software systems in many activities of society....the consequences of software failure in all its aspects are becoming increasingly serious. Particularly alarming is the seemingly unavoidable fallibility of large software, since a malfunction in an advanced hardware-software system can be a matter of life and death, not only for individuals, but also for vehicles carrying hundreds of people and ultimately for nations as well. David and Fraser 4

9 1968 NATO Conference on Software Engineering convened by NATO Science Committee refers to software crisis and software gap One of the major motivations for the organizing of the conference was an awareness of the rapidly increasing importance of computer software systems in many activities of society....the consequences of software failure in all its aspects are becoming increasingly serious. Particularly alarming is the seemingly unavoidable fallibility of large software, since a malfunction in an advanced hardware-software system can be a matter of life and death, not only for individuals, but also for vehicles carrying hundreds of people and ultimately for nations as well. David and Fraser coined the term software engineering the application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software 4

10 Edsger Dijkstra seminal contributions to programming languages and formal verification participated in the 1968 Conference The dissemination of knowledge is of obvious value the massive dissemination of error-loaded software is frightening. 5

11 Edsger Dijkstra seminal contributions to programming languages and formal verification participated in the 1968 Conference The dissemination of knowledge is of obvious value the massive dissemination of error-loaded software is frightening. 1972: ACM Turing Award Lecture discusses software crisis The Humble Programmer 5

12 Edsger Dijkstra seminal contributions to programming languages and formal verification participated in the 1968 Conference The dissemination of knowledge is of obvious value the massive dissemination of error-loaded software is frightening. 1972: ACM Turing Award Lecture discusses software crisis The Humble Programmer Program testing can be used to show the presence of bugs, but never to show their absence! 5

13 Edsger Dijkstra seminal contributions to programming languages and formal verification participated in the 1968 Conference The dissemination of knowledge is of obvious value the massive dissemination of error-loaded software is frightening. 1972: ACM Turing Award Lecture discusses software crisis The Humble Programmer Program testing can be used to show the presence of bugs, but never to show their absence! Computer Science is no more about computers than astronomy is about telescopes. 5

14 Edsger Dijkstra seminal contributions to programming languages and formal verification participated in the 1968 Conference The dissemination of knowledge is of obvious value the massive dissemination of error-loaded software is frightening. 1972: ACM Turing Award Lecture discusses software crisis The Humble Programmer Program testing can be used to show the presence of bugs, but never to show their absence! Computer Science is no more about computers than astronomy is about telescopes. Elegance is not a dispensable luxury but a quality that decides between success and failure. 5

15 Other sightings of the Software Crisis 1994: Scientific American, Software s Chronic Crisis, by W. Wayt Gibbs 1/4 large software development efforts canceled average development project overshoots schedule by 50% 3/4 of large systems are operating failures do not function as intended or are not used at all 6

16 Famous Software Failures: Ariane 5 Flight 501 References 5 Flight 501 Ariane 5: Flight 501 Failure: Report by the Inquiry Board, Prof. J. L. Lions, Chairman, July 19, Summary June 4, 1996: first launch of Ariane 5 $10 billion to develop carried four expensive (and uninsured) scientific satellites exploded 37 seconds after lift-off 7

17 Ariane 501: Software systems a computer controls the interial guidance system determines velocity, acceleration,... of rocket sends information to computer responsible for steering 8

18 Ariane 501: Software systems a computer controls the interial guidance system determines velocity, acceleration,... of rocket sends information to computer responsible for steering control code (Ada) reused from Ariane 4 used to align system before takeoff runs for 50 seconds after takeoff in case of launch countdown delays this feature not used in Ariane 5! 8

19 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 9

20 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 2. this is converted to a 16-bit signed integer 9

21 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 2. this is converted to a 16-bit signed integer 3. the value was too large and an Ada exception was thrown 9

22 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 2. this is converted to a 16-bit signed integer 3. the value was too large and an Ada exception was thrown 4. the exception was not caught by the code, causing computer to crash 9

23 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 2. this is converted to a 16-bit signed integer 3. the value was too large and an Ada exception was thrown 4. the exception was not caught by the code, causing computer to crash 5. the backup guidance computer also crashed, for same reason 9

24 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 2. this is converted to a 16-bit signed integer 3. the value was too large and an Ada exception was thrown 4. the exception was not caught by the code, causing computer to crash 5. the backup guidance computer also crashed, for same reason 6. guidance computer spewed diagnostic error messages to steering computer 9

25 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 2. this is converted to a 16-bit signed integer 3. the value was too large and an Ada exception was thrown 4. the exception was not caught by the code, causing computer to crash 5. the backup guidance computer also crashed, for same reason 6. guidance computer spewed diagnostic error messages to steering computer 7. steering computer interpreted these as guidance information and made radical adjustments to correct course 9

26 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 2. this is converted to a 16-bit signed integer 3. the value was too large and an Ada exception was thrown 4. the exception was not caught by the code, causing computer to crash 5. the backup guidance computer also crashed, for same reason 6. guidance computer spewed diagnostic error messages to steering computer 7. steering computer interpreted these as guidance information and made radical adjustments to correct course 8. rocket began to disintegrate due to extreme stresses from sharp attitude 9

27 Ariane 501: Failure sequence 1. a 64-bit floating-point variable is used to represent horizontal velocity 2. this is converted to a 16-bit signed integer 3. the value was too large and an Ada exception was thrown 4. the exception was not caught by the code, causing computer to crash 5. the backup guidance computer also crashed, for same reason 6. guidance computer spewed diagnostic error messages to steering computer 7. steering computer interpreted these as guidance information and made radical adjustments to correct course 8. rocket began to disintegrate due to extreme stresses from sharp attitude 9. rocket self-destructed 9

28 Ariane 501, cont. inertial guidance system code was extensively tested 10

29 Ariane 501, cont. inertial guidance system code was extensively tested... using parameters from Ariane 4 horizontal velocity increases 5 times faster in Ariane 5 10

30 Ariane 501, cont. inertial guidance system code was extensively tested... using parameters from Ariane 4 horizontal velocity increases 5 times faster in Ariane 5 recommendations of Inquiry Board...set up a team that will prepare the procedure for qualifying software, propose stringent rules for confirming such qualification, and ascertain that specification, verification and testing of software are of a consistently high quality... 10

31 Famous Software Failures: Therac-25 References Nancy Leveson, Clark S. Turner, An Investigation of the Therac-25 Accidents, IEEE Computer, Vol. 26, No. 7, July 1993, http: //courses.cs.vt.edu/ cs3604/lib/therac 25/Therac 1.html Nancy Leveson, Software: System Safety and Computers, Addison-Wesley

32 Therac-25 computerized radiation therapy machine : 6 known massive radiation overdoses at least 5 deaths and serious injuries programming errors have been reduced by extensive testing... programming errors also existed in Therac-20 but were never observed due to hardware safety interlocks that were removed in the Therac-25 12

33 Therac-25, cont. The equipment control task did not properly synchronize with the operator interface task, so that race conditions occurred if the operator changed the setup too quickly. This was evidently missed during testing, since it took some practice before operators were able to work quickly enough for the problem to occur. (Wikipedia) 13

34 Therac-25, cont. The equipment control task did not properly synchronize with the operator interface task, so that race conditions occurred if the operator changed the setup too quickly. This was evidently missed during testing, since it took some practice before operators were able to work quickly enough for the problem to occur. (Wikipedia) The software was highly reliable. It worked tens of thousands of times before overdosing anyone, and occurrences of erroneous behavior were few and far between. AECL assumed that their software was safe because it was reliable... Leveson 13

35 Therac-25, cont. The equipment control task did not properly synchronize with the operator interface task, so that race conditions occurred if the operator changed the setup too quickly. This was evidently missed during testing, since it took some practice before operators were able to work quickly enough for the problem to occur. (Wikipedia) The software was highly reliable. It worked tens of thousands of times before overdosing anyone, and occurrences of erroneous behavior were few and far between. AECL assumed that their software was safe because it was reliable... Leveson The software should be subjected to extensive testing and formal analysis at the module and software level; system testing alone is not adequate. Leveson 13

36 Other examples from the Software Crisis From Dependable Software by Design, Daniel Jackson, Scientific American, June 2006 Denver airport baggage handler software problems delayed airport s opening by 16 months hundreds of millions of dollars in cost overruns finally gave up IRS, FBI, and FAA systems 14

37 Scientific Computing Some applications of scientific computing: study of fluid turbulence study of cosmological structure formation study of atomic and molecular structure molecular biology climate modeling automobile, aircraft design 15

38 Scientific Computing Some applications of scientific computing: study of fluid turbulence study of cosmological structure formation study of atomic and molecular structure molecular biology climate modeling automobile, aircraft design A third path to scientific understanding empirical (experimentation) 2. deductive (math, logic) 3. simulation (using computers) 15

39 Scientific Computing, cont. codes are exceedingly complex typically parallel (using, e.g., MPI) many sources of nondeterminism aspects of program execution which are not specified by the program s code relative ordering of execution steps from the parallel processes can differ from execution to execution 16

40 Software Crisis hits Scientific Computing 17

41 Computational Science Demands a New Paradigm by Douglass E. Post and Lawrence G. Votta Physics Today, January 2005 The field has reached a new threshold.... New methods of verifying and validating complex codes are mandatory if computational science is to fulfill its promise for science and society....diligence and alertness are far from a guarantee that the code is free of defects. Better verification techniques are desperately needed. 18

42 Current Methods for Dealing with the Crisis: Testing But testing exhibits serious limitations: lack of coverage only a tiny fraction of inputs can be tested (Ariane 5) 19

43 Current Methods for Dealing with the Crisis: Testing But testing exhibits serious limitations: lack of coverage only a tiny fraction of inputs can be tested (Ariane 5) nondeterminism correct result on one execution does not even guarantee correct result on another execution with the same input (Therac-25) 19

44 Current Methods for Dealing with the Crisis: Testing But testing exhibits serious limitations: lack of coverage only a tiny fraction of inputs can be tested (Ariane 5) nondeterminism correct result on one execution does not even guarantee correct result on another execution with the same input (Therac-25) problem of oracles in scientific computation, often don t know correct result for a given test input, so can t tell if the observed result is correct 19

45 Validation approaches Validation Testing 20

46 Validation approaches Validation Testing Formal Verification Theorem Proving Theorem Proving (deductive reasoning) construct proof that program satisfies a property axioms, definitions, theorems may be partially automated provides a high level of confidence applies to systems of arbitrary size limitations theoretical limits requires enormous skill very time-consuming 20

47 Validation approaches Testing Validation Theorem Proving Formal Verification Finite-State Verification Finite-State Verification (FSV) 1. construct a model of the program using only a finite number of states 2. formalize correctness properties for the model 3. use automated algorithmic techniques to verify that all executions of the model satisfy the properties Model Checking 20

48 Finite-State Verification 1. construct a model of the program using only a finite number of states 2. formalize correctness properties for the model 3. use automated algorithmic techniques to verify that all executions of the model satisfy the properties 21

49 Finite-State Verification 1. construct a model of the program using only a finite number of states 2. formalize correctness properties for the model 3. use automated algorithmic techniques to verify that all executions of the model satisfy the properties what is a model? a simplified or abstract version of the program, often written in a modeling language for a particular FSV tool floating-point variables are usually not used in models 21

50 Finite-State Verification 1. construct a model of the program using only a finite number of states 2. formalize correctness properties for the model 3. use automated algorithmic techniques to verify that all executions of the model satisfy the properties what is a model? a simplified or abstract version of the program, often written in a modeling language for a particular FSV tool floating-point variables are usually not used in models what is a state of the model? a vector with one component for each variable in the model 21

51 Finite-State Verification 1. construct a model of the program using only a finite number of states 2. formalize correctness properties for the model 3. use automated algorithmic techniques to verify that all executions of the model satisfy the properties what is a model? a simplified or abstract version of the program, often written in a modeling language for a particular FSV tool floating-point variables are usually not used in models what is a state of the model? a vector with one component for each variable in the model what are typical properties of models? freedom from deadlock assertions about the state assert(x==y*z); assertions about the order of events (temporal logic) ((x==1) (y==1)) 21

52 Finite-State models the model can be tailor-made for a specific property example: parallel matrix multiplication program property: freedom from deadlock model: remove all floating-point variables remove all statements that assign to those variables replace all messages by 1 result: MPI communication skeleton of the program 22

53 Finite-State models the model can be tailor-made for a specific property example: parallel matrix multiplication program property: freedom from deadlock model: remove all floating-point variables remove all statements that assign to those variables replace all messages by 1 result: MPI communication skeleton of the program ideally, models should be conservative: for every execution of the original program there is a corresponding execution of the model if a property holds for all executions of a conservative model, it holds for all executions of the original program 22

54 Finite-State models the model can be tailor-made for a specific property example: parallel matrix multiplication program property: freedom from deadlock model: remove all floating-point variables remove all statements that assign to those variables replace all messages by 1 result: MPI communication skeleton of the program ideally, models should be conservative: for every execution of the original program there is a corresponding execution of the model if a property holds for all executions of a conservative model, it holds for all executions of the original program but sometimes we must compromise impose small bounds on configuration Small Configuration Hypothesis defects almost always manifest themselves in small configurations 22

55 Reachability state: a vector s with one component for every variable in the model 23

56 Reachability state: a vector s with one component for every variable in the model initial state: the state s 0 for the initial values of the variables 23

57 Reachability state: a vector s with one component for every variable in the model initial state: the state s 0 for the initial values of the variables next(s): set of all states reachable from s by a single execution step 23

58 Reachability state: a vector s with one component for every variable in the model initial state: the state s 0 for the initial values of the variables next(s): set of all states reachable from s by a single execution step state space: the directed graph with nodes: states edges: s t iff t next(s) 23

59 Reachability state: a vector s with one component for every variable in the model initial state: the state s 0 for the initial values of the variables next(s): set of all states reachable from s by a single execution step state space: the directed graph with nodes: states edges: s t iff t next(s) reachable state space: subgraph G of all states reachable from s 0 23

60 Reachability state: a vector s with one component for every variable in the model initial state: the state s 0 for the initial values of the variables next(s): set of all states reachable from s by a single execution step state space: the directed graph with nodes: states edges: s t iff t next(s) reachable state space: subgraph G of all states reachable from s 0 can be computed by starting with s 0, computing all next states, computing all next states of those states,... 23

61 Reachability state: a vector s with one component for every variable in the model initial state: the state s 0 for the initial values of the variables next(s): set of all states reachable from s by a single execution step state space: the directed graph with nodes: states edges: s t iff t next(s) reachable state space: subgraph G of all states reachable from s 0 can be computed by starting with s 0, computing all next states, computing all next states of those states,... paths through G correspond to executions of the model 23

62 Example: Shared Resource boolean x; proc rw0 { while (true) { x := 0; synch(); if (x == 0) use_resource(); proc rw1 { while (true) { x := 1; synch(); if (x == 1) use_resource();

63 Example: Shared Resource Property 1: Freedom from deadlock The program does not deadlock Property 2: Mutual exclusion It is never the case that both processes use the resource 032 at020 the same102 time. 123 Property 3: Liveness The resource 002will eventually 030 be 103used. 120 State: [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); if (x == 0) use_resource(); proc rw1 { while (true) { x := 1; synch(); if (x == 1) use_resource();

64 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); if (x == 0) use_resource(); proc rw1 { while (true) { x := 1; synch(); if (x == 1) use_resource();

65 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); if (x == 0) use_resource(); proc rw1 { while (true) { x := 1; synch(); if (x == 1) use_resource();

66 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

67 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

68 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

69 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

70 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

71 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

72 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

73 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

74 Example: Shared Resource [x, pc 0, pc 1 ] boolean x; proc rw0 { while (true) { x := 0; synch(); 2 if (x == 0) use_resource(); proc rw1 { while (true) { 0 x := 1; 1 synch(); if (x == 1) use_resource();

75 FSV: strengths and weaknesses strengths can prove things about all possible executions of a program can be (close to) fully automated produces a trace if property does not hold 25

76 FSV: strengths and weaknesses strengths can prove things about all possible executions of a program can be (close to) fully automated produces a trace if property does not hold weaknesses model construction problems the result is only as good as the model state space explosion problem the number of states typically grows exponentially with the number of processes 25

77 The current state of FSV progress in automatic model extraction Bandera and Bogor (Java) Java PathFinder (Java) Microsoft s SLAM toolset (C) BLAST (C) 26

78 The current state of FSV progress in automatic model extraction Bandera and Bogor (Java) Java PathFinder (Java) Microsoft s SLAM toolset (C) BLAST (C) development of techniques to combat state explosion partial order reductions (SPIN) use of BDDs to represent state space (SMV, NuSMV) symmetry abstraction counterexample-guided refinement 26

79 The current state of FSV progress in automatic model extraction Bandera and Bogor (Java) Java PathFinder (Java) Microsoft s SLAM toolset (C) BLAST (C) development of techniques to combat state explosion partial order reductions (SPIN) use of BDDs to represent state space (SMV, NuSMV) symmetry abstraction counterexample-guided refinement increasingly used in industry Intel, Motorola, Microsoft,... numerous conferences and workshops 26

80 Introduction to the SPIN Model Checker spinroot.com open source development started in the 1980s freely available since 1991 ACM Systems Software Award for 2001 geared towards software verification input language: Promela properties specified as assertions, or LTL formulas (converted to Büchi automata) Gerard Holzmann originally Bell Labs now NASA JPL 27

81 Promela encoding of Shared Resource Example bit x = 0; chan synch = [0] of {bit; proctype rw0() { do :: x = 0; synch!0; if :: x == 0 -> useresource: skip :: else fi od 28 proctype rw1() { do :: x = 1; synch?0; if :: x == 1 -> useresource: skip :: else fi od init { atomic { run rw0(); run rw1()

82 Property Specifications for Shared Resource 1. freedom from deadlock checked by default (improper endstates) 2. mutual exclusion 2.1 add line #define u0 2.2 add line #define u1 2.3 LTL formula for violation: <>(u0 && u1) 2.4 convert LTL formula to never claim spin -f <>(u0 && u1) 2.5 insert never claim into Promela model 3. liveness []!u0 29

83 Demo: Using SPIN on Shared Resource Example 1. verification of freedom from deadlock 1.1 spin -a shared-deadlock.prom 1.2 cc -o pan pan.c 1.3./pan 2. verification of mutual exclusion 2.1 spin -a shared-mutex.prom 2.2 cc -o pan pan.c 2.3./pan -a 3. verification of liveness 3.1 spin -a shared-live.prom 3.2 cc -o pan pan.c 3.3./pan -a 3.4 spin -t -p -g shared-live.prom 30