TDDD04 Software Testing Lecture Notes 7 March June 2010 Mohsen Torabzadeh-Tari (presenter), (originator) Department of Computer and Information Science Linköping University, Sweden
Outline of the Lecture 2 Levels of Testing, Alternative Life Cycle Models Waterfall Spin-Offs Specification-based Life Cycle Models Rapid prototyping life cycle Executable specification System Testing steps Function testing Threads Cause-and-Effect Performance testing Acceptance testing Installation testing
Levels of Testing, Alternative Life Cycle Models Waterfall Spin-Offs Specification-based Life Cycle Models Rapid prototyping life cycle Executable specification http://www.csee.usf.edu/~mikhail/ http://www.istqb.org/downloads/glossary-current.pdf Cause Effect Graph to Decision Table Generation, Praveen Ranjan Srivastava, Parshad Patel, Siddharth Chatrola, http://www.docstoc.com/docs/7838462/cause-effect-graph-to-decision-table/
Reflection 4 Mention the drawbacks of waterfall model What is the problem with V-Model? How can we do system testing with rapid protoyping? Compare the waterfall and prototype life-cycle model in terms of risk Give some example of performance testing
Traditional Waterfall Model Waterfall model: top-down development and design by functional decomposition. Preliminary design results into a structure of functional components. Software testing: bottom up testing order (unit, integration system). 5 Requirements Specification System Testing Preliminary Design Integration Testing Detailed Design Unit Testing Coding
Weaknesses of traditional waterfall model: 6 functional decomposition can only be well done when the system is completely understood. very long separation between requirements specification and a completed system. no opportunity for feedback from the customer Q: is it possible? A: early 1980s, practitioners have devised alternatives
Composition 7 Closer to the way people work. Start with something known and understood, then add to it gradually, and may be remove undesired portions analogy : Michelangelo s David; negative sculpture: start with a piece of marble, and chip away all non-david. consequence of mistake (negative sculpture): the whole work must be throw away and restarted. (a museum in Florence, Italy, contains half a dozen such false starts to David) positive sculpture: is often done with a medium like wax. The central shape is approximated, and then wax is either added or removed until the desired shape is attained. consequence of right approximation (positive sculpture): the erroneous part is simply removed and replaced
Levels of Testing, Alternative Life Cycle Models Waterfall Spin-Offs Specification-based Life Cycle Models Rapid prototyping life cycle Executable specification
Waterfall Spin-Offs Three mainline derivatives of waterfall model: 9 1. Incremental development, 2. Evolutionary development 3. Spiral model each of these involves a series of increments or builds*. the differences among the 3 spin-off models are to how the builds are identified. one advantage common to all these spin-off models is that all yield earlier synthesis which results in earlier customer feedback. within a build, the normal waterfall phases from detailed design through testing occur (system testing is split into two steps: regression and progression testing). * a build is a set of deliverable end user functionality.
10 Detailed Design Requirements Specification Coding Preliminary Design Unit Testing Series of Builds Regression testing: ensures that things worked correctly in the previous build still work with the newly added code Integration Testing Progression testing: assumes that regression testing was successful and that the new functionality can be tested Build: a set of deliverable end user functionality. System Testing Regression Testing Progression Testing
Levels of Testing, Alternative Life Cycle Models Waterfall Spin-Offs Specification-based Life Cycle Models Rapid prototyping life cycle Executable specification
Specification-Based Life Cycle Models: Rapid Prototyping 12 Rapid prototyping: reduces the specification-to-customer-feedback loop to produce very early synthesis. Rather than build a final system, a quick and dirty prototype is built and then used to elicit customer feedback. Depending on the feedback, more prototyping cycles may occur. once the developer and the customer agree, a correct specification will be built. At this point any of the waterfall spin-offs might also be used.
Series of Prototypes Preliminary Design Define Prototype Objectives 13 Build Prototype Detailed Design Coding Customer Feedback Unit Testing Rapid prototyping life cycle Integration Testing System Testing
Rapid Prototyping 14 Q: Where are the requirements? Is the last prototyp the specification? A: use the prototyping cycles as informationgathering activities, and then produce a requirements specification in a more traditional manner. Q: How are system test cases traced back to the prototype? A: Capture what the customer does with the prototypes, define these as scenarios that are important to the customer, and then use these as system test cases.
Rapid Prototyping 15 The main contributation of rapid prototyping is that it brings the operational (or behavioral) viewpoint to the requirements specification phase. Usually, requirements specification techniques emphasize the structure of a system, not its behavior. This is unfortunate, becuse most customers do not care about the structure, and they do care about the behavior.
Levels of Testing, Alternative Life Cycle Models Waterfall Spin-Offs Specification-based Life Cycle Models Rapid prototyping life cycle Executable specification
Specification-Based Life Cycle Models: Executable specification 17 Executable specification: an extension of rapid prototyping concept. The requirements are specified in an executable format (finite state machines, StateCharts, or Petri nets). The customer executes the specification to observe the intended system behavior and provides feedback.
Executable Specification Preliminary Design Define executable specification 18 execute spec Detailed Design Coding Customer Feedback Unit Testing Executable specification Integration Testing System Testing
System Testing: Function testing Threads Cause-and-Effect Performance testing Acceptance testing Installation testing
System Testing 20 Objective: to ensure that the system does what the customer wants it to do. Customer Developer Requirements definition Requirements specification Functional requirements Nonfunctional requirements
21 Component code Unit test Tested components Design Specification Integration test Component code Unit test Tested components Integrated modules
Integrated modules System functional requirements Function test Functioning systems Other software requirements Performance test 22 Verified validated software Acceptance test Customer requirements spec. Accepted system Installation test User environment System In Use!
System Testing: Function testing Threads Cause-and-Effect Performance testing Acceptance testing Installation testing
A repetition of earlier lecture: Path-Based Integration
Path-Based Integration 25 MM-Path: when a unit executes, some path of source statements is traversed. Suppose that a call goes to another unit along such a path: at that point, control is passed from the calling unit to the called unit, where some other path of source statements is traversed.
Example: An MM-Path module A calls module B, which in turn calls module C 26 Source nodes A B C 1 1 2 1 Sink nodes MM path 3 4 5 2 3 4 2 3 4 5 6
Function testing/thread testing (testing one function at a time) functional requirements 27 Threads: A scenario of normal usage A stimulus/response pair Behavior that results from a sequence of system-level input An interleaved sequence of port input and output events A sequence of MM-Path A sequence of atomic system functions (ASF) (ASF: an atomic system function is an action that is observable at the system level in terms of port input and output events)
Example: 3 candidate threads in SATM system 1. Entry of a personal identification number (PIN) (family of stimulus/response pairs): 28 1. A screen requesting PIN digits 2. An interleaved sequence of digit keystrokes and screen responses 3. The possibility of cancellation by the customer before the full PIN is entered 4. A system disposition: a customer has 3 chances to enter the correct PIN. Once a correct PIN has been entered, the user sees a screen requesting the transition type; otherwise, a screen advises the customer that the ATM card will not be returned, and no access to ATM functions is provided. 2. A simple transaction: ATM Card Entry, PIN entry, select transaction type (deposits, withdraw), present account details (checking or savings, amount), conduct the operation, and report the results (involves the interaction of several ASFs) 3. An ATM session (a sequence of threads) containing two or more simple transactions (interaction among threads)
Definitions 29 System Thread: a path from a source ASF to a Sink ASF in the ASF graph of a system. Threads: A sequence of atomic system functions (ASF) (ASF: an atomic system function is an action that is observable at the system level in terms of port input and output events) ASF graph: Given a system defined in terms of atomic system functions (ASFs), the ASF graph of the system is the directed graph in which nodes are ASFs and edges represent sequential flow. Source ASF: an ASF that appears as a source node in the ASF graph of a system (ex in SATM: Card Entry ASF) Sink ASF: an ASF that appears as a sink node in the ASF graph of a system (ex in SATM: session termination)
System Testing: Function testing Threads Cause-and-Effect Performance testing Acceptance testing Installation testing
Cause-and-Effect-Graph (test case generation from req.) 31 Causes: inputs Effects: outputs and transformations Causes-and-Effect Graph: boolean graph reflecting causes and effects relationships is a formal language into which a natural language specification is translated
Basic cause-effect graph symbols 32 a b Identity: if a then b a b c And: if (a and b) then c a b d a b c Or: if (a or b or c) then d Not: if (not a) then b
Sample cause-effect graph 35 Student Worksheet
Constraint symbols 37 E a b I cause-constraint: at least one of a, b and c must always be true I a b E cause-constraint: at most one of a or b can be true a a c O a R M b b b O cause-constraint: one, and only one, of a and b must be true R cause-constraint: for a to be true, b must be true M effect-constraint: If effect a is true, Effect b is forced to be false
Sample cause-effect graph with exclusive constraint 38 Student Worksheet
Procedure of generation decision table 40 Causes are Conditions Effects are Actions 1. Select an effect to be present (1) state. 2. Tracing back through the graph, find all combinations of causes (subject to constrains) that will set this effect to 1. 3. Create a column in the decision table for each combination of causes 4. For each combination, determine the states of all other effects and place these in each column.
Considerations used when tracing the graph 41 a b x if x is to be 1, do not bother with the situation where a = b = 1 if x is to be 0, enumerate all situations where a = b = 0 a if x is to be 1, enumerate all situations where a = b = c = 1 b c x if x is to be 0, include only one situation where a = b = c = 0. For the states 001, 010, 100, 011, 101, and 110 of a, b, c, include only one situation each
Decision table for cause-and effect graph 42 Student Worksheet Test 1 Test 2 Test 3 Test 4 Cause 1 Cause 2 Cause 3 Effect E1 Effect E2 Effect E3
45 Student Worksheet Causes: 1. The first five characters of the command Level 2. Effects: 1.
Cause-and Effect Graph 47 Student Worksheet
Decision table for cause-and effect graph 49 Student Worksheet
Sample cause-effect graph (page 455 SE, James F. Peters) 51 Intermediate node C1 or E1 C2 and E2 C3 and and E3 E4 C4 and E5
Decision table for cause-and effect graph 52 Test 1 Test 2 Test 3 Test 4 Test 5 Cause 1 0 1 X X 1 Cause 2 0 X 1 1 X Cause 3 X 0 1 1 1 Cause 4 X X 0 1 1 Effect 1 1 0 0 0 0 Effect 2 0 1 0 0 0 Effect 3 0 0 1 0 0 Effect 4 0 0 0 1 0 Effect 5 0 0 0 0 1
System Testing: Function testing Threads Cause-and-Effect Performance testing Acceptance testing Installation testing
Performance Testing nonfunctional requirements 54 Stress tests Volume tests Configuration tests Compatibility tests Regression tests Security tests Timing tests Environment tests Quality tests Recovery tests Maintenance tests Documentation tests Human factors tests / usability tests
Acceptance Testing customers, users need 55 Benchmark test: a set of special test cases Pilot test: everyday working Alpha test: at the developer s site, controlled environment Beta test: at one or more customer site. Parallel test: new system in parallel with previous one
Installation Testing users site 56 Acceptance test at developers site installation test at users site otherwise may not be needed!!
Summary: Levels of Testing 57 Waterfall Spin-Offs Specification-based Life Cycle Models Rapid prototyping life cycle Executable specification System Testing steps System Testing steps Function testing Threads Cause-and-Effect Performance testing Acceptance testing Installation testing
Reflection 58 Mention the drawbacks of waterfall model What is the problem with V-Model? How can we do system testing with rapid protoyping? Compare the waterfall and prototype life-cycle model in terms of risk Give some example of performance testing