Coding and Testing. The Coding Phase. Coding Goal: Create code that can be executed on a computer! What makes a programming language good?

Transcription

1 Requirements Spec.! Design! Coding and Testing Computer Science 320! Fall 2010! Prof. Leon Osterweil! Characteristics of! System to be! built must! match required! characteristics! Test Results must! match required behavior! Hi level design must! show HOW requirements! can be met! Test plan! exercises! this code! Hi Level! consistent! views! Low! level! Code must! implement! design! Test Plan! Code! The Coding Phase relations to! test cases! Code-related artifacts executables! parse trees! Object! Source! relations! to low! level! design! Goal: Produce executable code in a coding language! Gets down to very specific details:! --Procedures/algorithms! --Data structures! --The interactions between them! --THE DEVIL IS IN THE DETAILS! Coding is usually 10-15% of the effort on a software!!development project: We will spend little time on it in!!this course! instrumented! source! Coding should follow closely the specifications resulting! from the final phases of design! --Modular structure of the code! --Object specifications (data modules) (Data abstractions)! Coding Goal: Create code that can be executed on a computer! Developer writes source code! Object code emitted from a compiler!!so, is code really just another model?! Executable results from loading object code with libraries,! utilities, etc.! What makes a programming language good? Important to keep all of these straight! Some designed to support specific design methodologies! Some are special-purpose, well adapted to application!!domains!

2 A good language is one that meets the needs of its stakeholders What makes a programming language good? If it meets the needs of its stakeholders Different kinds of projects Quality is super-important Rapid deployment is key Evolvability is paramount Emphasis on user interface Etc. Suggest languages with strengths like Readability Expressive power Low level (close to the machine) Dynamism and late binding Etc. On Languages Bad code can be written in any language But some languages encourage bad practices Good code can be written in any language But some languages encourage it/make it easier And discourage bad practices Most modern languages try to encourage good practices Like those we have been advocating (in discussing design) Modularity Information hiding Data abstraction Incorporation of design and requirements specification into code Support for testing and analysis Information Hiding in Implementation Implementation units should hide internal details as specified by a Modular design Superior procedure semantics support this better Implementation units should communicate through welldefined interfaces (not global variables). Some languages make global data easier than others Some languages make it hard to inspect internals of Modules. Others make it easier Different decisions are harder or easier to hide Algorithm Data representation Lower-level modules Policy Data Abstractions User's (client's)-eye view of the data types to be used! Essentially the same as Parnas notion of a "data module"! --and the notion of an "object"! Cluster of "accessing primitives" / "methods" whose! purpose is to provide the only mechanisms for! manipulating data of a given type! Problem: How to specify the semantics of these types! --without specifying their implementation! Being rigorous help separate (even slightly) different! notions of an ADT from each other! Assertion Languages Assert statements to define assertions Assertions defined by programmer Locations identified by programmer Reactions to violations defined by programmer Different assertion language semantics Usually Boolean logic Sometimes private data space

3 Tool Suites Better tools make languages more useful Better editors Better diagnostics Better testing aids More powerful libraries Etc. Ada Early language that supported information hiding Use of External and Internal part dichotomy Strict encapsulation Support for data abstraction Packages Very wordy Support for disciplined concurrency No type hierarchy Very static language C C++ Gets you down close to the machine Little restriction on use of pointers Little restriction (help with) dynamic storage allocation Little support for encapsulation Adds support for objects to C Thus, support for objects (encapsulation) Type hierarchies Still little discipline over pointers, storage allocation Java Lisp Early language with special attention paid to dynamism and the web Designed to facilitate distributed applications Host readily on various machines (across the web) Support by lots of tools Highly dynamic language Various sorts of late binding But more discipline than C (eg. over use of pointers) Very dynamic language Very little compilation done Mostly interpretation Create code on the fly and interpret it Excellent vehicle for rapid prototyping Virtually no concept of types Types with Lisp extensions Primitive flow of control structures Very hard to encapsulate

4 Ruby/Python/Perl Prolog Highly dynamic Interpreted, not compiled Sometimes used as a scripting language, sometimes as general-purpose programming language Object-orientedness varies Extensive libraries Supporting frameworks RoR for web applications Rule-based language No real procedural flow of control Emphasis on reaction Favorite language for trying to capture human knowledge Data is subordinated Structuring, modularization are difficult Patterns Higher level implementation constructs Idioms (Rich and Waters, ~1985) The Gang of Four book Inspiration from real architects (C. Alexander) Idioms in common use Suggest ways that humans think/human esthetics Transcend specific languages Some finding more direct support in newer languages Coding closely tied up with Testing The essence is dealing with Faults They manifest themselves as Failures Basic Definitions Failure: inconsistency between actual behavior of software and specification of intent Fault: software flaw whose execution caused the failure Error: human action that results in software containing a fault Testing: The systematic (?) search of a program's execution space for the occurrence of a failure Debugging: Searching for the fault that caused an observed failure More Definitions Testing: The systematic (?) search of a program's execution! space for the occurance of a failure! Debugging: Searching for the fault that caused an!!observed failure! Analysis: The static examination of a program's textual! representation for the purpose of inferring characteristics! Verification: Using analytic inferences to formally prove that! all executions of a program must be consistent with intent!

5 Validation and Verification (V&V) Informal Requirements! Formal Requirements! Software Implementation! Verification! Validation! Other Quality/Reliability Improvement Approaches Fault avoidance: software development techniques that reduce the incidence of faults (e.g., design principles and methods, formal specification, prototyping)! Fault elimination: software analysis techniques that detect and remove faults (e.g., reviews and inspections, static analysis, testing, formal verification)! Fault prediction: software analysis techniques that predict the occurence of faults and direct further efforts (e.g., reliability assessment, metrics)! Fault tolerance: software execution techniques that detect and correct errors before a failure occurs!(e.g., self-checking assertions, recovery blocks, n- version programming)! The Basic Fault Elimination Approach COMPARISON of! BEHAVIOR to! INTENT! INTENT:! - Originates with requirements! - Different types of intent (requirements)! - Each type relatively better captured with different set of! formalisms! BEHAVIOR! - Can be observed as software executes! - Can be inferred from execution model! - Different models support different sorts of inferences! Specification of " Intended Behavior " A Unifying Framework Development" (Synthesis)" Process" Evaluation" (Analysis) " Process" Specification of " Actual Behavior" COMPARISON! - Can be informal--done by human eyeballs! - Can be done by computers--eg. comparing text strings! - Can be done by formal machines (eg. FSM's)! - Can be done by rigorous mathematical reasoning! Results obtained will vary as a function of the above! Comparison of Behavior to Intent (constraint evaluation)" Testing/Analysis Results" Testing Behavior determined by examining test execution results! Intent derived (somehow) from (various) specifications! Comparison is done by textual examination! Testing must select test cases likely to reveal failures! Equivalence partitioning is the typical approach! a test of any value in a given class is equivalent to a test of any other value in that class! if a test case in a class reveals a failure, then any other test case in that class should reveal the failure! Two basic types of testing! functional testing is based on the functional specification (black box testing)! structural testing is based on the software structure (white box testing)! Black box vs. white box testing White box testing (vs. Black box testing) Black Box testing No knowledge of internals Testing relies on observing external behavior Driven by users, customers, etc. White Box testing Uses knowledge of program structure Uses monitoring of program internals Often driven by developers, based on implementation issues Complementary approaches Each produces results that stimulate the other

6 Black Box vs. White Box Testing Dynamic Testing SELECTED INPUTS RESULTANT OUTPUTS DESIRED OUTPUT Specification of! Intended Behavior! Required" Outputs" Test " Specification of Actual Execution" Behavior! Results" BLACK BOX TESTING SELECTED INPUTS RESULTANT OUTPUTS INTERNAL BEHAVIOR WHITE BOX TESTING (CLEAR BOX TESTING)! DESIRED OUTPUT SOFTWARE DESIGN Result " Comparator" (Human or" Machine)" Comparison of Behavior to Intent! Failure" Reporting" Testing Aims to Answer Jalote Chapter 10, starting on Page 465 covers this subject Does the software do what it is supposed to do? When might it fail? How fast does it run? How accurate are the results? What are its failure modes and characteristics? What can I count on? What should I worry about? What are its strengths and weaknesses? A Simplistic View A More Realistic View Input Space" Program " Program " Input Space Divided into Domains" Output Space"

7 Partitioning the Input Space Rationale: Test the program one function at a time The requirements specification is used to enumerate the functions Then test the different functions And the non-functional requirements Do this by sampling from the different partitions Testing is Sampling the Input Space Key problem: What is the input space? What is the software intended to do? Subproblem: The input space is large One dimension for each program input Each dimension can have as many elements as there are legal inputs (eg. 2**32 different integers) Each input really is different How different? Which differences matter? Key Problem: How to sample from the input space? What is the input space? Computing the Input Space Specification" sum_of_roots takes an arbitrarily long sequence of real numbers, and computes the sum of their square roots. The real number sequence must be ended with the number Implementation Program sum_of_roots; Real sum, x, r; sum := 0; Do forever input x; if x = then exit else r := sqrt(x); sum := sum + r; print sum; end; There are ~2**32 possible different values for each input If n values are read in, then there are (2**32)**n different points in the input space The number of different input values read in is unlimited There is no limit (theoretically) to the size of the input space Some observations about the example program input space There is no real need to test every possible combination of input values Most executions behave the same But some input combinations are different Negative values will produce a failure There is a virtually limitless number of inputs that don t cause the negative square root failure A sufficiently large sequence of input values will cause an overflow failure Summary of Some Difficulties in Doing Testing Effectively Hard to cover program execution space effectively! Hard to select test data effectively! Hard to tell if test results are right or wrong! --if program computes complex function(s)! --if the number of test cases gets large! Hard to tell when you are done with testing! --Is it good or bad to keep finding faults?! Effective selection of test cases requires thought and care"

8 Testing is too long and hard to do all at once at the end of development Divide the job into subtasks Do some activities during development Can do test planning during development And should do so Phase testing at the end Using test plans previously developed Testing Phases Unit/Module Comparing a unit or module with design specifications. Integration Systematic combination of software components and modules to insure consistency of component interfaces and adherence to design specification Software System Comparing an integrated software system with software system requirements System Acceptance evaluation of an integrated hardware and software system Requirements! Specification! System! Testing! Development Phases! Architecting! Designing! Coding! Software! Sys Testing! Integration! Testing! Testing Phases! Unit Testing! Create Test Plans As Early as Possible No need to wait until after development to create test plan Desired behaviors specified in requirements Tests are run after development Test plans built beforehand Good reasons to plan early Testing may require developing test harnesses May help identify untestable requirements Or vague, poorly thought out requirements We can do this at first Then later. Requirements! Specification! Requirements! Specification! Architecting! System Test! Plan! System Test! Plan! Software Sys.! Test Plan! System! Testing! System! Testing! Software! Sys Testing!

9 And later still. DEVELOPMENT PHASES! Requirements! Specification! Architecting! Implementation! Designing! Requirements! Specification! Architecting! Implementation! Designing! Coding! System Test! Plan! Software Sys.! Test Plan! Integration! Test Plan! System Test! Plan! Software Sys.! Test Plan! TEST PLANNING! Integration! Test Plan! Unit! Test! Plan! System! Testing! Software! Sys Testing! Integration! Testing! System! Testing! Software! Sys Testing! Integration! Testing! Unit Testing! TESTING PHASES! Summary of Dynamic Testing Testing is Buying Knowledge The testing costs resources It should result in knowledge that is worth the cost? The value of the knowledge is up to the buyer The cost of the knowledge is something that software engineers should be able to estimate Strengths: Microscopic examination of execution details Evaluation in actual runtime environment Oldest approach, most familiar Weaknesses: Cannot demonstrate absence of faults Hard to generate test data Hard to know when testsets are adequate Hard to decide when enough testing has been done Testing aids can alter execution Analysis Testing can only prove the presence of faults, not their absence Static analysis doesn t require execution Build mathematical model of software Do reasoning, prove theorems Can show the absence of errors Responds to testing weaknesses No need to select input data No need to examine output results No problem knowing when you are done Limitations Only some theorems can be proven Some of them are very hard Static Analysis Technique for demonstrating the absence of faults without the need for execution! Specification of Intent: derived from requirements! Specification of Behavior: derived from model(s)! Comparison: Done analytically and mathematically! Results: Theorems about the program (eg. proofs that certain behaviors are impossible--or mandatory)!

10 Many Static Analysis Approaches Inspection Syntax Checking Semantic Checking Flow Analysis Formal Verification Specification of! Intended Behavior! Informal" Specification" Inspection Human" Inspector" Specification of! Actual Behavior! Source" Text" Informal" Error" Findings" Comparison of Behavior to Intent! Syntax Checker Static Semantic Analyzer Specification of! Intended Behavior! Specification of! Actual Behavior! Specification of! Intended Behavior! Specification of! Actual Behavior! Syntax" Specification" Source" Text" Semantic" Specification" Source" Text" Parser" Syntax " Faults" Semantic" Analyzer" Static" Semantic" Faults" Comparison of Behavior to Intent! Comparison of Behavior to Intent! Early Static Analyzers Example Semantic Checks Syntax checker: Proves that all executions are syntactically correct! Static semantics checkers: Demonstrate adherence to certain semantic rules and conditions! Line at a time checks! Combinational checks! Type mismatches! Argument/Parameter list mismatches! Division by 0 Parameter/argument mismatches??

11 Dataflow Analysis Specification of Intent: Sequence of events! Specification of Behavior: Derived from flowgraph model! Nodes annotated with events of interest! All possible executions modeled as all sequences of events along all flowgraph paths! Comparison: Analytic process! Are all possible event sequences the desired one(s)?! Result: Theorems demonstrating absence of event sequence errors! Examples:! No variable referenced before definition! No file read before it is opened! Elevator doesnʼt move until doors are shut! Rocket wonʼt try to fire thrusters after fuel is exhausted! Static Dataflow Analysis Specification of! Intended Behavior! Event" Sequences" Dataflow " Propagation" Algorithms" Specification of! Actual Behavior! Possible" Execution" Sequences" Comparison of Behavior to Intent! Proofs of the" Presence or" Absence of" Faults" totalpay := 0.0;! for i = 1 to last_employee! if salary[i] < ! then salary[i] := salary[i] * 1.05;! else salary[i] := salary[i] * 1.10;! totalpay := totalpay + salary[i];! end loop;! print totalpay;! Example Flowgraph totalpay := 0.0;! for i = 1 to last_employee! salary[i] := salary[i] * 1.05! if salary[i] < ! salary[i] := salary[i] * 1.10! Skeleton Flowgraph Do If 7 totalpay := totalpay + salary[i]! 8 9 end loop! 10 print totalpay! 11 Annotated Flowgraph X := Do X := Something Wrong Here Do := X X := If 7 If

12 Better := X Do 2 3 X := Tougher Situation Do If 7 If X := 8 9 := X More Serious Issue Here Open Door Move Elevator Do 2 3 Close Door 4 If 7 A Problem 6 5 Do 2 If Open Door Move Elevator 8 9 Close Door Another Problem? Close Door Move Elevator 6 5 Do 2 If Open Door Static Analysis of Pre-code Artifacts Flowgraphs of designs are possible Apply analyses to the graphs There are other graphs too Data flow graphs Finite State Machines Petri Nets This is what should happen at the end of design, and other pre-code phases.

13 What about analysis of functionality Preceding suggests analysis for non-functional behaviors Especially robustness Major interest in analysis for functional correctness How to keep control of software? We need a way to prevent runaways and other forms of bad behavior Analogous to Heat sink (thermodynamics) Electrical ground (electrical engineering) Containment Vessel (nuclear energy) What do we have???? Why should we care? A Story from the Past: The Story of Bob G. The position of software in society The infrastructure in the infrastructure Banks Telephones Computer Networks Roads Air Travel Software is in all of them In the most critical positions

14 Old software never dies What are our obligations when we produce software? And doesn t fade away either Other Moral Dilemmas (for later) Whistle-blowing Trap doors Theft of Intellectual Property (IP) Worms, Viruses, etc. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx! xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx! Proof of Correctness! Formal Verification INTENT:! o Usually specification of functionality! --What function(s) does the software compute?! o Sometimes accuracy, timing,...! BEHAVIOR:! o Inferred from semantically rich program model! o Generally requires most of semantics of programming!! language! o Generally uses symbolic execution! COMPARISON:! o Use of formal mathematics (eg. predicate logic)! o Probably source of misleading name: PROOF of!!correctness! --Proof is probably OK! --Correctness is dangerously misleading! Specification of " Intended Behavior " Formal Verification Final and" Intermediate" 1st-Order Logic" Assertions" First-Order Logic" Theorems and" Proofs" Comparison of Behavior to Intent" Specification of " Actual Behavior" Symbolic" Execution " of Path" Segments" Proof of the" Absence of" All Functional" Faults" Floyd Method of Inductive Assertions Show that each program fragment behaves as intended Use induction to prove that all sequences of executable fragments behave as intended Show that the program must terminate

15 Loop Invariants Should be able to identify loop invariant for every loop that you write Should be able to state initial and final assertions too Maybe not in formal mathematics Should be able to reason about how they support overall program logic Use of Assertions Assertion: Specification of a condition that is intended to! be true at a specific given site in the program text! In Floyd's Method, assertions are written in Predicate Logic! In Floyd's Method there are three types of assertions:! o Initial, As: Sited at the program initial statement! o Final, AF: Sited at the program final statement! o Intermediate Ai: Often called a "loop invariants"! Sited at various internal program locations subject!! to the rule:! EVERY LOOP ITERATION (CFG CYCLE) SHALL PASS THRU! THE SITE OF AT LEAST ONE INTERMEDIATE ASSERTION! Net Effect: Every program execution sequence is divided! into a finite number of segments of non-looping code! bounded on each end by a predicate logic assertion! Mathematical Induction Goal: prove that a given property holds for all elements of a set Approach: (initial step) show property holds for "first" element (induction step) show that if property holds for element i, then it must also hold for element i + 1 Often used when direct analytic techniques are too hard or complex Example: How many edges in C n Theorem: let C n = (V n, E n ) be a complete, unordered graph on n nodes, then E n = n * (n-1)/2 Initial Step Induction Step show the property is true for C1: graph has one node, 0 edges E1 = n(n-1)/2 = 1(0)/2 = 0 assume true for Cn: En = n(n-1)/2 graph Cn+1 has one more node, but n more edges (one from the new node to each of the n old nodes) so, En+1 = n(n-1)/2 + n» = n(n-1)/2 +2n/2 = (n(n-1)+2n)/2» = (n(n-1+2))/2 = n(n+1)/2» = (n+1) ((n+1)-1)/2» = (n+1)(n)/2

16 Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER ASSERT OUTER ASSERT FINAL end sort; A Harder Example: Wensley's Algorithm Procedure Wensley (P: input, Q: input, E: input, Y: output); --assume 0 P<Q, 0< E! Declare P, Q, E, Y, A, B, D real;! A :=0.0; B :=Q / 2.0; D :=1.0; Y := 0.0;! Do_While (D>=E)! If (P - A - B >= 0.0) then {Y := Y+(D / 2.0); A := A+B};! B :=B / 2.0; D := D / 2.0;! End_do;! End Wensley;! Wensley s Algorithm Input P, Q, E A 0.0 B Q/2 D 1.0 Y 0.0 A0! AI! AF! Initial: Final: Assertions A 0 : {(0 P<Q) (0< E)}!A F : {((P/Q-E)<Y (P/Q))}! computed quotient! D E P-A-B 0.0 B B/2.0 D D/2.0 Y Y+(D/2.0) A A+B computed error! Intermediate:!!!A I : {(A=Q*Y) (B=Q*(D/2))!! (k 0, k integer D=2 -k )!!! ((P/Q)-D)<Y (P/Q)}! Y is within the computed error D of P/Q! Floyd s Method of inductive assertions (informal description) Pictorially intermediate assertions Place assertions at the start, final, and intermediate points in the code. Any path is composed of sequences of program fragments all of which: start with an assertion are followed by some assertion free code and end with an assertion eg. (As =) A0, C1, A1, C2, A2, An-1, Cn-1, An, Cn, An+1(=Af) Show that for any executable path, if Ai is assumed true for any Ai and code Ci is executed, then Ai+1 must always be true initial assertion A i! C i! STRAIGHT-LINE CODE! A i+1! final assertion

17 Must be sure: assuming A i, then executing Code C i, necessarily A i+1 A i! C i! STRAIGHT-LINE CODE! A i+1! Why does this work? suppose P = arbitrary path through the program can denote it by P = A s C 1 A 1 C 2 A 2...C n A F where A s - Initial assertion A F - Final assertion A i - Intermediate assertions C i - Loop free, uninterrupted, straight-line code If it has been shown that! i, 1 i < n: A i C i A i+1! Then, by induction! A s... A f! Loop Invariants (Loop Breakers) Problem: infinite number of paths Must find a way to deal with loops Solution: Assertion, Ai, that is True for any number of loop iterations connects up to adjacent assertions Such an assertion: Is invariant with respect to loop iterations Must be embedded in (break) every loop A loop invariant must capture the essence! Of the work that the loop is to perform! Schematic Example of a Loop Invariant Ai is a loop invariant because of its relation to other assertions: NOTE THAT: A I, false branch, => A I A I, true branch, => A I BUT ALSO: Initial assertion A s to A I => A I A I, false branch, => final assertion A F A I, true branch, => final assertion A F true! A s " A F! A I " false! Floyd s Method (carefully stated) Specify initial, final assertions to capture intent Intermediate assertions "cut" every program loop For each pair of assertions with an executable (assertion-free) path from the first to the second, Assume that the first assertion is true Show that for all (assertion-free, executable) paths from the first assertion to the second, that the second assertion is true This establishes partial correctness Show that the program terminates This establishes total correctness Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER ASSERT OUTER ASSERT FINAL end sort;

18 Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER ASSERT OUTER ASSERT FINAL end sort; ASSERT OUTER values[i] is > values[j] for all j, where i < j < size Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER ASSERT OUTER ASSERT FINAL end sort; ASSERT INNER values[i] > values[k] for all i < k < j Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER ASSERT OUTER ASSERT FINAL end sort; ASSERT FINAL For all i, j < size values[i] > values [j]

19 Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER ASSERT OUTER ASSERT FINAL end sort; size > 0 and size < 1000 ASSERT INITIAL Loop Invariants A Harder Example: Wensley's Algorithm Should be able to identify loop invariant for every loop that you write Should be able to state initial and final assertions too Maybe not in formal mathematics Should be able to reason about how they support overall program logic Procedure Wensley (P: input, Q: input, E: input, Y: output); --assume 0 P<Q, 0< E! Declare P, Q, E, Y, A, B, D real;! A :=0.0; B :=Q / 2.0; D :=1.0; Y := 0.0;! Do_While (D>=E)! If (P - A - B >= 0.0) then {Y := Y+(D / 2.0); A := A+B};! B :=B / 2.0; D := D / 2.0;! End_do;! End Wensley;! Wensley s Algorithm Input P, Q, E A 0.0 B Q/2 D 1.0 Y 0.0 D E P-A-B 0.0 A0! AI! AF! Y Y+(D/2.0) A A+B What does Wensley's algorithm do? Approximating P/Q (=Y) with error E On the kth iteration of the loop A k = a 1 Q2-1 + a 2 Q a k Q2 -k a i {0,1} B k = Q2 -k D k = 2 -k Y k = a a a k 2 -k a i {0,1} Input P, Q, E A 0.0 B Q/2 D 1.0 Y 0.0 D E A0! AI! AF! B B/2.0 D D/2.0 P-A-B 0.0 B B/2.0 D D/2.0 Y Y+(D/2.0) A A+B

20 What does Wensley's algorithm do? since 0 P/Q<1, then P/Q can be estimated as a sum of the series a a a k 2 -k a i {0,1} therefore Y k is the computed value of the quotient given Y k, A k is the computed dividend P D k is the computed error P-(A k + B k ) says when to add 2 -(k+1) to Y k+1 (a k+1 = 1) B k = 2 -(k+1) * Q! Initial: Final: Assertions A 0 : {(0 P<Q) (0< E)}!A F : {((P/Q-E)<Y (P/Q))}! computed quotient! computed error! Intermediate:!!!A I : {(A=Q*Y) (B=Q*(D/2))!! (k 0, k integer D=2 -k )!!! ((P/Q)-D)<Y (P/Q)}! Y is within the computed error D of P/Q! Summary of Four Lemmas Needed I. Initial assertion, A0, to AI II. AI, false branch, to AI III. IV. AI, true branch, to AI AI, to AF, final assertion Input P, Q, E A 0.0 B Q/2 D 1.0 Y 0.0 D E P-A-B 0.0 B B/2.0 D D/2.0 Lemmas called verification conditions A0! AI! AF! Y Y+(D/2.0) A A+B This is only partial correctness Must also prove termination In general, can not prove termination For most programs, can usually do it by showing that each loop must terminate For our example: given that (E>0) observe that D decreases on each iteration and E does not change Thus, eventually D<E and the loop terminates Input P, Q, E A 0.0 B Q/2 D 1.0 Y 0.0 D E P-A-B 0.0 B B/2.0 D D/2.0 A0! AI! AF! Y Y+(D/2.0) A A+B Observations Proofs are long, tedious & often hard Assertions are hard to get right Invariants are difficult to get right. need to be invariant, but also need to support overall proof strategy Proofs themselves often require deep program insight Often require axioms about the domain Software Tools Can Help Proof Checkers: Scrutinize the steps of a proof and determine if they are sound Identify the rule(s) of inference, axiom(s), etc. needed to justify each step How to know if the proof checker is right (verify it? with what?...)

21 Software Tools Can Help Verification Assistants Facilitate precise expression of assertions Accept rules of inference Accept axioms Construct statements of needed lemmas Check proofs Assist in construction of proofs (theorem provers) Human/computer collaboration Most successful -- human/computer collaboration Human architects the proof Computer attempts the proof (generally by exhaustive search of space of possible axioms and inferences at each step) Human intervention after computer has tried for a while Current Status: Have verified some non-trivial programs or important parts of programs e.g., protocol verification Improved theorem provers Improved specification languages Verification and testing/analysis research now viewed more as a continuum Summary Verification has had a very positive impact on software engineering major argument for structured programming Dijkstra's "goto's considered harmful" letter one-in one-out structures easier to reason about major impetus for abstract data types centralized all changes to a data structures input/output assertions for all operations testing--> finite state verification--> verifications Formal Development Start with assertions, develop software artifacts to fulfill them A top-down approach Very popular in Europe: A hard sell in the U.S. Need to prove lemmas in higher level software dictates the functional requirements (eg. input/output assertion) pairs of lower level software artifacts. Also suggests the use of libraries of reusable verified software artifacts for commonly needed utilities This is Component-based software development Integration of Testing Analysis and Formal Methods Testing! --Is dynamic in nature, entailing execution of the program! --Requires skillful selection of test data to assure good! exercising of the program! --Can show program executing in usage environment! --Can support arbitrarily detailed examination of virtually! any program characteristics and behavior! --Is generally not suitable for showing absence of faults! Analysis! --Is static, operating on abstract program representations! --Supports definitive demonstration of absence of faults! --Generally only for certain selected classes of faults! Formal Methods! --Most through, rigorous, mathematical! --Apply primarily to checking functional characteristics! --Most human and cost intensive! The types of capabilities are complementary; suggests need for skillful integration!

22 DEVOLOP! CODE! CODE! STATIC! ANALYSIS FOR! CODE FLAWS! PATHS WITH FLAWS! SYMBOLIC! EXECUTION! ERRORS! ERROR FEEDBACK (FIX ERRORS)! ERRORS! FOUND! NO! ERRORS! FOUND! CODE! ERRORS! ERRORS! FULLY! INSTRUMENTED! CODE! DYNAMIC! TESTING! DEVELOP! ASSERTIONS! INSERT! TEST! PROBES! STATIC! ANALYSIS FOR! PROBE REMOVAL! SYMBOLIC! EXECUTION! CONSTRAINT! SOLUTION! CODE WITH! SOME PROBES! REMOVED! PATH! SELECTION! PATHS! THROUGH! PROBES! VERIFICATION FEEDBACK (GET MORE COMPLETE ASSERTIONS)! Testing & Analysis Process Architecture