Coding and Testing. Class Schedule rest of semester

Transcription

1 Coding and Testing Computer Science 320 Fall 2012 Prof. Leon Osterweil Class Schedule rest of semester Wednesday, Dec. 5: Unit test demos Thursday, Dec. 6: Integrative Experience papers due: to Deadline is 11:59 PM on Dec. 6 There will be NO FINAL TEST ON DEC. 6 Demo Day: 10:30AM on Thursday, Dec. 13 Final Project submissions also due at Demo Day An hour test that day too (probably)

2 Unit Testing Demos Test one or (preferably) two modules For each module present one or (preferably) two TSSs Each TSS aimed at exercising/demonstrating a different behavior of the module For each TSS at least one example TES Present results of having run the TSS previously Demonstrate the module by running some data through it live Integrative Experience CS 320 is an Integrative Experience course "Students [in an IE course such as CS 320, will] reflect on and integrate their learning and experience from the broad exposure in their General Education courses and the focus in their major." In order to meet the IE mission, the CS 320 course incorporates two specific reflection exercises into the course curriculum. These reflection components focus on reflecting about how this course catalyzes the transition from student to practitioner, more specifically from thinking of software development as a classroom programming exercise to thinking of the development of real software systems as key components of the infrastructure of society.

3 Integrative Experience Exercise A pair of written assignments, one submitted in September, one now Previously you wrote a paper addressing What skills and knowledge you bring to the course that might help you succeed with the course project, whether these skills and knowledge come from courses in the major, General Education courses, or from non-academic experience What further skills and knowledge you thought would be likely to be needed in order to succeed with the course project How you intended to get those skills and knowledge, whether individually or through the CS 320 project teams. Now you will write a second paper that will Begin by reviewing your first paper in the light of your subsequent course experience, Assess what further skills and knowledge you believe are still needed in order to become a successful software practitioner in the real world Indicate how you plan to acquire those skills and knowledge, in the remainder of your UMass career or afterward. Second Integrative Experience Paper Due on Thursday, Dec. 6 at 11:59PM Submit to ljo@cs.umass.edu Be sure to include BOTH your first paper and your new paper Graded by me Expected length of second paper: 2-3 pages

4 There will be no final test on Thursday, December 6 Instead my plan is to give a 50-minute final test to start our course final exam period on Thursday, December 13 at 10:30 Then we will proceed to Demo Day demos NOTE: This is subject to my getting a legal opinion on this plan As time permits on Thursday, Dec. 6, There will be a discussion of ETHICS for software engineers

5 Requirements Spec. Design Test Results must match required behavior Characteristics of System to be built must match required characteristics Hi level design must show HOW requirements can be met Hi Level consistent views Low level Code must implement design Test plan exercises this code Test Plan Code Code-related artifacts parse trees Object relations to low level design relations to test cases executables Source instrumented source

6 The Coding Phase Goal: Produce executable code in a coding language Gets down to very specific details: --Procedures/algorithms --Data structures --The interactions between them --THE DEVIL IS IN THE DETAILS Coding is usually 10-15% of the effort on a software development project: We will spend little time on it in this course Coding should follow closely the specifications resulting from the final phases of design --Modular structure of the code --Object specifications (data modules) (Data abstractions) Coding Goal: Create code that can be executed on a computer Developer writes source code Object code emitted from a compiler So, is code really just another model? Executable results from loading object code with libraries, utilities, etc. Important to keep all of these straight Some designed to support specific design methodologies Some are special-purpose, well adapted to application domains

7 What makes a programming language good? What makes a programming language good? If it meets the needs of its stakeholders

8 A good language is one that meets the needs of its stakeholders Different kinds of projects Quality is super-important Rapid deployment is key Evolvability is paramount Emphasis on user interface Etc. Suggest languages with strengths like Readability Expressive power Low level (close to the machine) Dynamism and late binding Etc. On Languages Bad code can be written in any language But some languages encourage bad practices Good code can be written in any language But some languages encourage it/make it easier And discourage bad practices Most modern languages try to encourage good practices Like those we have been advocating (in discussing design) Modularity Information hiding Data abstraction Incorporation of design and requirements specification into code Support for testing and analysis

9 Information Hiding in Implementation Implementation units should hide internal details as specified by a Modular design Superior procedure semantics support this better Implementation units should communicate through welldefined interfaces (not global variables). Some languages make global data easier than others Some languages make it hard to inspect internals of Modules. Others make it easier Different decisions are harder or easier to hide Algorithm Data representation Lower-level modules Policy Data Abstractions User's (client's)-eye view of the data types to be used Essentially the same as Parnas notion of a "data module" --and the notion of an "object" Cluster of "accessing primitives" / "methods" whose purpose is to provide the only mechanisms for manipulating data of a given type Problem: How to specify the semantics of these types --without specifying their implementation Being rigorous helps separate (even slightly) different notions of an ADT from each other

10 Assertion Languages An Assertion is a Boolean predicate the specifies some state of affairs that must be true. Assert statements are used to define assertions Assertions defined by programmer Locations identified by programmer Reactions to violations defined by programmer Different assertion language semantics Usually Boolean logic Assertion failure may raise exception, cause IF test to fail, etc. Sometimes private data space Tool Suites Better tools make languages more useful Better editors Better diagnostics Better testing aids More powerful libraries Etc.

11 C Gets you down close to the machine Little restriction on use of pointers Little restriction (help with) dynamic storage allocation Little support for encapsulation C++ Adds support for objects to C Thus, support for objects (encapsulation) Type hierarchies Still little discipline over pointers, storage allocation

12 Java Early language with special attention paid to dynamism and the web Designed to facilitate distributed applications Host readily on various machines (across the web) Support by lots of tools Highly dynamic language Various sorts of late binding But more discipline than C (eg. over use of pointers) Lisp Very dynamic language Very little compilation done Mostly interpretation Create code on the fly and interpret it Excellent vehicle for rapid prototyping Virtually no concept of types Types with Lisp extensions Primitive flow of control structures Very hard to encapsulate

13 Ruby/Python/Perl Highly dynamic Interpreted, not compiled Sometimes used as a scripting language, sometimes as general-purpose programming language Object-orientedness varies Extensive libraries Supporting frameworks RoR for web applications Prolog Rule-based language No real procedural flow of control Emphasis on reaction Favorite language for trying to capture human knowledge Data is subordinated Structuring, modularization are difficult

14 Coding closely tied up with Testing The essence is dealing with Faults They manifest themselves as Failures Basic Definitions Failure: inconsistency between actual behavior of software and specification of intent Fault: software flaw whose execution caused the failure Error: human action that results in software containing a fault Testing: The systematic (?) search of a program's execution space for the occurrence of a failure Debugging: Searching for the fault that caused an observed failure

15 More Definitions Testing: The systematic (?) search of a program's execution space for the occurance of a failure Debugging: Searching for the fault that caused an observed failure Analysis: The static examination of a program's textual representation for the purpose of inferring characteristics Verification: Using analytic inferences to formally prove that all executions of a program must be consistent with intent Validation and Verification (V&V) Informal Requirements Validation Formal Requirements Software Implementation Verification

16 Other Quality/Reliability Improvement Approaches Fault avoidance: software development techniques that reduce the incidence of faults (e.g., design principles and methods, formal specification, prototyping) Fault elimination: software analysis techniques that detect and remove faults (e.g., reviews and inspections, static analysis, testing, formal verification) Fault prediction: software analysis techniques that predict the occurence of faults and direct further efforts (e.g., reliability assessment, metrics) Fault tolerance: software execution techniques that detect and correct errors before a failure occurs (e.g., self-checking assertions, recovery blocks, n- version programming) The Basic Fault Elimination Approach COMPARISON of BEHAVIOR to INTENT INTENT: - Originates with requirements - Different types of intent (requirements) - Each type relatively better captured with different set of formalisms BEHAVIOR - Can be observed as software executes - Can be inferred from execution model - Different models support different sorts of inferences COMPARISON - Can be informal--done by human eyeballs - Can be done by computers--eg. comparing text strings - Can be done by formal machines (eg. FSM's) - Can be done by rigorous mathematical reasoning Results obtained will vary as a function of the above

17 A Unifying Framework Specification of " Intended Behavior " Development" (Synthesis)" Process" Specification of " Actual Behavior" Evaluation" (Analysis) " Process" Comparison of Behavior to Intent (constraint evaluation)" Testing/Analysis Results" Testing Behavior determined by examining test execution results Intent derived (somehow) from (various) specifications Comparison is done by textual examination Testing must select test cases likely to reveal failures Equivalence partitioning is the typical approach a test of any value in a given class is equivalent to a test of any other value in that class if a test case in a class reveals a failure, then any other test case in that class should reveal the failure Two basic types of testing functional testing is based on the functional specification (black box testing) structural testing is based on the software structure (white box testing)

18 Black box vs. white box testing White box testing (vs. Black box testing) Black Box testing No knowledge of internals Testing relies on observing external behavior Driven by users, customers, etc. White Box testing Uses knowledge of program structure Uses monitoring of program internals Often driven by developers, based on implementation issues Complementary approaches Each produces results that stimulate the other Black Box vs. White Box Testing SELECTED INPUTS RESULTANT OUTPUTS DESIRED OUTPUT BLACK BOX TESTING SELECTED INPUTS RESULTANT OUTPUTS INTERNAL BEHAVIOR DESIRED OUTPUT SOFTWARE DESIGN WHITE BOX TESTING (CLEAR BOX TESTING)

19 Dynamic Testing Specification of Intended Behavior Required" Outputs" Test " Execution" Results" Specification of Actual Behavior Result " Comparator" (Human or" Machine)" Comparison of Behavior to Intent Failure" Reporting" Testing Aims to Answer Does the software do what it is supposed to do? When might it fail? How fast does it run? How accurate are the results? What are its failure modes and characteristics? What can I count on? What should I worry about? What are its strengths and weaknesses?

20 A Simplistic View Input Space" Program " Output Space" A More Realistic View Program " Input Space Divided into Domains"

21 Partitioning the Input Space Rationale: Test the program one function at a time The requirements specification is used to enumerate the functions Then test the different functions And the non-functional requirements Do this by sampling from the different partitions Testing is Sampling the Input Space Key problem: What is the input space? What is the software intended to do? Subproblem: The input space is large One dimension for each program input Each dimension can have as many elements as there are legal inputs (eg. 2**32 different integers) Each input really is different How different? Which differences matter? Key Problem: How to sample from the input space?

22 What is the input space? Specification" sum_of_roots takes an arbitrarily long sequence of real numbers, and computes the sum of their square roots. The real number sequence must be ended with the number Implementation Program sum_of_roots; Real sum, x, r; sum := 0; Do forever input x; if x = then exit else r := sqrt(x); sum := sum + r; end do; print sum; end; Computing the Input Space There are ~2**32 possible different values for each input If n values are read in, then there are (2**32)**n different points in the input space The number of different input values read in is unlimited There is no limit (theoretically) to the size of the input space

23 Some observations about the example program input space There is no real need to test every possible combination of input values Most executions behave the same But some input combinations are different Negative values will produce a failure There is a virtually limitless number of inputs that don t cause the negative square root failure A sufficiently large sequence of input values will cause an overflow failure Effective selection of test cases requires thought and care" Summary of Some Difficulties in Doing Testing Effectively Hard to cover program execution space effectively Hard to select test data effectively Hard to tell if test results are right or wrong --if program computes complex function(s) --if the number of test cases gets large Hard to tell when you are done with testing --Is it good or bad to keep finding faults?

24 Testing is too long and hard to do all at once at the end of development Divide the job into subtasks Do some activities during development Can do test planning during development And should do so Phase testing at the end Using test plans previously developed Testing Phases Unit/Module Comparing a unit or module with design specifications. Integration Systematic combination of software components and modules to insure consistency of component interfaces and adherence to design specification Software System Comparing an integrated software system with software system requirements System Acceptance evaluation of an integrated hardware and software system

25 Development Phases Requirements Specification Architecting Designing Coding System Testing Software Sys Testing Integration Testing Unit Testing Testing Phases Create Test Plans As Early as Possible No need to wait until after development to create test plan Desired behaviors specified in requirements Tests are run after development Test plans built beforehand Good reasons to plan early Testing may require developing test harnesses May help identify untestable requirements Or vague, poorly thought out requirements

26 We can do this at first Requirements Specification System Test Plan System Testing Then later. Requirements Specification System Test Plan Architecting Software Sys. Test Plan System Testing Software Sys Testing

27 And later still. Requirements Specification Architecting Implementation Designing System Test Plan Software Sys. Test Plan Integration Test Plan System Testing Software Sys Testing Integration Testing DEVELOPMENT PHASES Requirements Specification Architecting Implementation Designing Coding System Test Plan Software Sys. Test Plan TEST PLANNING Integration Test Plan Unit Test Plan System Testing Software Sys Testing Integration Testing Unit Testing TESTING PHASES

28 Testing is Buying Knowledge The testing costs resources It should result in knowledge that is worth the cost? The value of the knowledge is up to the buyer The cost of the knowledge is something that software engineers should be able to estimate Summary of Dynamic Testing Strengths: Microscopic examination of execution details Evaluation in actual runtime environment Oldest approach, most familiar Weaknesses: Cannot demonstrate absence of faults Hard to generate test data Hard to know when testsets are adequate Hard to decide when enough testing has been done Testing aids can alter execution

29 Analysis Testing can only prove the presence of faults, not their absence Static analysis doesn t require execution Build mathematical model of software Do reasoning, prove theorems Can show the absence of errors Responds to testing weaknesses No need to select input data No need to examine output results No problem knowing when you are done Limitations Only some theorems can be proven Some of them are very hard Static Analysis Technique for demonstrating the absence of faults without the need for execution Specification of Intent: derived from requirements Specification of Behavior: derived from model(s) Comparison: Done analytically and mathematically Results: Theorems about the program (eg. proofs that certain behaviors are impossible--or mandatory)

30 Many Static Analysis Approaches Inspection Syntax Checking Semantic Checking Flow Analysis Formal Verification Inspection Specification of Intended Behavior Specification of Actual Behavior Informal" Specification" Source" Text" Human" Inspector" Informal" Error" Findings" Comparison of Behavior to Intent

31 Syntax Checker Specification of Intended Behavior Specification of Actual Behavior Syntax" Specification" Source" Text" Parser" Syntax " Faults" Comparison of Behavior to Intent Static Semantic Analyzer Specification of Intended Behavior Specification of Actual Behavior Semantic" Specification" Source" Text" Semantic" Analyzer" Static" Semantic" Faults" Comparison of Behavior to Intent

32 Early Static Analyzers Syntax checker: Proves that all executions are syntactically correct Static semantics checkers: Demonstrate adherence to certain semantic rules and conditions Line at a time checks Combinational checks Type mismatches Argument/Parameter list mismatches Example Semantic Checks Division by 0 Parameter/argument mismatches??

33 Dataflow Analysis Specification of Intent: Sequence of events Specification of Behavior: Derived from flowgraph model Nodes annotated with events of interest All possible executions modeled as all sequences of events along all flowgraph paths Comparison: Analytic process Are all possible event sequences the desired one(s)? Result: Theorems demonstrating absence of event sequence errors Examples: No variable referenced before definition No file read before it is opened Elevator doesn t move until doors are shut Rocket won t try to fire thrusters after fuel is exhausted Static Dataflow Analysis Specification of Intended Behavior Event" Sequences" Specification of Actual Behavior Possible" Execution" Sequences" Dataflow " Propagation" Algorithms" Comparison of Behavior to Intent Proofs of the" Presence or" Absence of" Faults"

34 Example Flowgraph totalpay := 0.0; for i = 1 to last_employee if salary[i] < then salary[i] := salary[i] * 1.05; else salary[i] := salary[i] * 1.10; totalpay := totalpay + salary[i]; end loop; print totalpay; totalpay := 0.0; for i = 1 to last_employee salary[i] := salary[i] * 1.05 if salary[i] < salary[i] := salary[i] * 1.10 totalpay := totalpay + salary[i] end loop print totalpay Skeleton Flowgraph 6 5 Do If

35 Annotated Flowgraph X := Do X := If Something Wrong Here 6 5 Do := X 4 X := If

36 1 X := Better := X 6 5 Do If Tougher Situation 6 5 Do If 7 X := := X

37 More Serious Issue Here Open Door Move Elevator Do Close Door If A Problem Move Elevator 6 5 Do If Open Door Close Door

38 Another Problem? Close Door Do Open Door If 7 Move Elevator Static Analysis of Pre-code Artifacts Flowgraphs of designs are possible Apply analyses to the graphs There are other graphs too Data flow graphs Finite State Machines Petri Nets This is what should happen at the end of design, and other pre-code phases.

39 What about analysis of functionality Preceding suggests analysis for non-functional behaviors Especially robustness Major interest in analysis for functional correctness How to keep control of software? We need a way to prevent runaways and other forms of bad behavior Analogous to Heat sink (thermodynamics) Electrical ground (electrical engineering) Containment Vessel (nuclear energy) What do we have????

40 Why should we care? A Story from the Past: The Story of Bob G.

41 The position of software in society Banks Telephones The infrastructure in the infrastructure Computer Networks Roads Air Travel Software is in all of them In the most critical positions

42 Old software never dies And doesn t fade away either What are our obligations when we produce software?

43 Other Moral Dilemmas (for later) Whistle-blowing Trap doors Theft of Intellectual Property (IP) Worms, Viruses, etc. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Proof of Correctness Formal Verification INTENT: o Usually specification of functionality --What function(s) does the software compute? o Sometimes accuracy, timing,... BEHAVIOR: o Inferred from semantically rich program model o Generally requires most of semantics of programming language o Generally uses symbolic execution COMPARISON: o Use of formal mathematics (eg. predicate logic) o Probably source of misleading name: PROOF of correctness --Proof is probably OK --Correctness is dangerously misleading

44 Specification of " Intended Behavior " Formal Verification Specification of " Actual Behavior" Final and" Intermediate" 1st-Order Logic" Assertions" Symbolic" Execution " of Path" Segments" First-Order Logic" Theorems and" Proofs" Comparison of Behavior to Intent" Proof of the" Absence of" All Functional" Faults" Floyd Method of Inductive Assertions Show that each program fragment behaves as intended Use induction to prove that all sequences of executable fragments behave as intended Show that the program must terminate

45 Loop Invariants Should be able to identify loop invariant for every loop that you write Should be able to state initial and final assertions too Maybe not in formal mathematics Should be able to reason about how they support overall program logic Use of Assertions Assertion: Specification of a condition that is intended to be true at a specific given site in the program text In Floyd's Method, assertions are written in Predicate Logic In Floyd's Method there are three types of assertions: o Initial, As: Sited at the program initial statement o Final, AF: Sited at the program final statement o Intermediate Ai: Often called a "loop invariants" Sited at various internal program locations subject to the rule: EVERY LOOP ITERATION (CFG CYCLE) SHALL PASS THRU THE SITE OF AT LEAST ONE INTERMEDIATE ASSERTION Net Effect: Every program execution sequence is divided into a finite number of segments of non-looping code bounded on each end by a predicate logic assertion

46 Mathematical Induction Goal: prove that a given property holds for all elements of a set Approach: (initial step) show property holds for "first" element (induction step) show that if property holds for element i, then it must also hold for element i + 1 Often used when direct analytic techniques are too hard or complex Example: How many edges in C n Theorem: let C n = (V n, E n ) be a complete, unordered graph on n nodes, then E n = n * (n-1)/2

47 Initial Step show the property is true for C1: graph has one node, 0 edges E1 = n(n-1)/2 = 1(0)/2 = 0 Induction Step assume true for Cn: En = n(n-1)/2 graph Cn+1 has one more node, but n more edges (one from the new node to each of the n old nodes) so, En+1 = n(n-1)/2 + n» = n(n-1)/2 +2n/2 = (n(n-1)+2n)/2» = (n(n-1+2))/2 = n(n+1)/2» = (n+1) ((n+1)-1)/2» = (n+1)(n)/2

48 Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER end do; ASSERT OUTER end do; ASSERT FINAL end sort; Floyd s Method of inductive assertions (informal description) Place assertions at the start, final, and intermediate points in the code. Any path is composed of sequences of program fragments all of which: start with an assertion are followed by some assertion free code and end with an assertion eg. (As =) A0, C1, A1, C2, A2, An-1, Cn-1, An, Cn, An+1(=Af) Show that for any executable path, if Ai is assumed true for any Ai and code Ci is executed, then Ai+1 must always be true

49 Pictorially intermediate assertions initial assertion A i C i STRAIGHT-LINE CODE A i+1 final assertion Must be sure: assuming A i, then executing Code C i, necessarily A i+1 A i A i+1 C i STRAIGHT-LINE CODE

50 Why does this work? suppose P = arbitrary path through the program can denote it by P = A s C 1 A 1 C 2 A 2...C n A F where A s - Initial assertion A F - Final assertion A i - Intermediate assertions C i - Loop free, uninterrupted, straight-line code If it has been shown that i, 1 i < n: A i C i A i+1 Then, by induction A s... A f Loop Invariants (Loop Breakers) Problem: infinite number of paths Must find a way to deal with loops Solution: Assertion, Ai, that is True for any number of loop iterations connects up to adjacent assertions Such an assertion: Is invariant with respect to loop iterations Must be embedded in (break) every loop A loop invariant must capture the essence Of the work that the loop is to perform

51 Schematic Example of a Loop Invariant Ai is a loop invariant because of its relation to other assertions: NOTE THAT: A I, false branch, => A I A I, true branch, => A I BUT ALSO: Initial assertion A s to A I => A I A I, false branch, => final assertion A F A I, true branch, => final assertion A F true A s " A F A I " false Floyd s Method (carefully stated) Specify initial, final assertions to capture intent Intermediate assertions "cut" every program loop For each pair of assertions with an executable (assertion-free) path from the first to the second, Assume that the first assertion is true Show that for all (assertion-free, executable) paths from the first assertion to the second, that the second assertion is true This establishes partial correctness Show that the program terminates This establishes total correctness

52 Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER end do; ASSERT OUTER end do; ASSERT FINAL end sort; Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER end do; ASSERT OUTER end do; ASSERT FINAL end sort;

53 ASSERT OUTER values[i] is > values[j] for all j, where i < j < size Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER end do; ASSERT OUTER end do; ASSERT FINAL end sort;

54 ASSERT INNER values[i] > values[k] for all i < k < j Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER end do; ASSERT OUTER end do; ASSERT FINAL end sort;

55 ASSERT FINAL For all i, j < size values[i] > values [j] Sort Example procedure sort(values, size); declare values real array[1000], temp real, i, j, size integer; ASSERT INITIAL do for i = 1 to size-1 do for j = i+1 to size if values[j] > values[i] then temp := values[i]; values[i] := values[j]; values[j] := temp; ASSERT INNER end do; ASSERT OUTER end do; ASSERT FINAL end sort;

56 ASSERT INITIAL size > 0 and size < 1000 Loop Invariants Should be able to identify loop invariant for every loop that you write Should be able to state initial and final assertions too Maybe not in formal mathematics Should be able to reason about how they support overall program logic

57 Summary Verification has had a very positive impact on software engineering major argument for structured programming Dijkstra's "goto's considered harmful" letter one-in one-out structures easier to reason about major impetus for abstract data types centralized all changes to a data structures input/output assertions for all operations Formal Development Start with assertions, develop software artifacts to fulfill them A top-down approach Very popular in Europe: A hard sell in the U.S. Need to prove lemmas in higher level software dictates the functional requirements (eg. input/output assertion) pairs of lower level software artifacts. Also suggests the use of libraries of reusable verified software artifacts for commonly needed utilities This is Component-based software development