door Sasa Berberovic

door Sasa Berberovic

Overview Reusable Components Subsystems

Reusable Components Reuse Mechanisms The Role of Testing in Reuse Reusing Test Suites Test Design Patterns Abstract Class Test Generic Class Test New Framework Test Popular Framework Test

Reuse Mechanisms Component: standalone sofware artifact that may be used in multiple contexts. Component toolkit: collection of programming utilities that support reuse of a certain component library. Reuse differs according to binding time: Source code augmentation Translation time binding Link time/runtime interfaces

Reuse Mechanisms Foundation class library supprts translation time reuse by inheritance. Provide a broad range of related capabilities. OpenStep Foundation Kit class library include in Smalltalk development env MS Foundation Classes Java Foundation Classes

Reuse Mechanisms Executable components support link time/runtime reuse. Sun s JavaBeans MS ActiveX (COM/DCOM)

Reuse Mechanisms Abstract Class exports method interfaces that do not have an implementation within the abstract class. Root class of polymorphic hierarchy Supports dynamic polymorphism

Reuse Mechanisms Generic Class requires a type parameter that binds generic instance variables and operations to a specific type (class), providing static polymorphism. Wel suited for implementing data types whose general behavior is nearly identical Ada 95, C++ and Eiffel Not in Java, Objective C and Smalltalk

Reuse Mechanisms Framework is an implementation of a reusable, domain specific design. White box framework: consumers typically develop implementations for abstract classes provided in the framework and additional application specific code. After refinement and stabilization white box framework may become black box frameword, in which composition plays a larger role.

The Role of Tes9ng in Reuse Solution to chronic software development problems: High development time and cost; Unacceptably frequent failures; Low maintainability; Low adaptability. Study of C++ development: Reuse reduced design errors by a ratio of 49:1 Reuse reduced coding errors by a ratio of 26:1 100:1 reduction in rework an debug time

The Role of Tes9ng in Reuse Simple in principle, development of reusable oocomponents is surprisingly difficult. Dependability is the barrier. We test reusable components to remove bugs and increase dependability. Testing can also show that component implementation is awkward, inelegant of difficult to use.

The Role of Tes9ng in Reuse Effective producer testing leads to: more reliable components; enhances component documentation; improves design level robustness and reusability; encourages consumer reuse. Focus here is on test strategies for components that support reuse by source code augmentation. (see table)

The Role of Tes9ng in Reuse Reusable Component Abstract class Test Instantiation Overriding subclass method implementation Testable Responsibilities Class contract Producer s Test Scope Class/flattened class Generic class Type parameter Class contract Class/flattened class Application framework Application implementation Framework use cases Association invariants System

The Role of Tes9ng in Reuse Instantiation means something different for each kind of component: abstract class instantiation is a subclass that provides an implementation of the abstract class interface; generic class instantiation is the class that results when a parameter is provided in a declaration or a definition. framework instantiation is a system of subclasses that realizes one possible implementation of the framework.

The Role of Tes9ng in Reuse Framework often evolve from insights about design solutions that can work in more than one context. As it evolves, the focus of testing shifts: New framework test: applies to initial and early version; Popular framework test: applies to a framework that has become widely used.

Reusing Test Suites Reuse not limited to code, it can benefit both the consumer as the producer to reuse the test suites. Consumer: can extend the test suite to parallel application extensions. Producer: consumer implementation can provide a rich source for controlled testing by producer.

Test Design Pa>erns Abstract Class Test Generic Class Test New Framework Test Popular Framework Test

Abstract Class Test Intent: Develop a test implementation of and a test suite for an abstract class interface. Context: Abstract class can not be instantiated and can not be tested as written. It must be tested by developing a subclass by implementing all the unimplemented superclass methods. Test suite is designed for entire hierarchy by using a pattern that corresponds to this test instantiation.

Abstract Class Test Context: Example abstract class (C++): class Account { public: Account () {} virtual ~Account () {} virtual Money balance () = 0; virtual void credit (Money& amount) = 0; virtual void debit (Money& amount) = 0; }

Abstract Class Test Fault Model: Design errors: incomplete or inconsistent identification of common responsibilities; incorrect or incomplete representation of responsibilities in the interface, definition of instance variables that should be deferred; missing capabilities. Simple coding errors.

Abstract Class Test Strategy: Develop a subclass that implements all the abstract methods; Develop a specification based test suite for the contract of the flattened subclass using the applicable method and class scope test design patterns. Polymorphic Server Test may be used for a test suite for the hierarchy; Test Case for each subclass method that implements an abstract superclass method en run on subclass object; Test Case for each implemented superclass method en run on the superclass method; Reuse above test case for subclass methods; Integration testing at flattened scope for each subclass ; The Subclass Driver is appropriate as a class scope driver.

Abstract Class Test Entry criteria: An abstract class may be tested when it is compiled. Exit criteria: The test instantiation should achieve the coverage recommended for test pattern applied. Method scope coverage is the recommended minimum.

Abstract Class Test Consequences: The development of the test subclass and its test suite may reveal design errors or reusability problems. Test suite may provide useful documentation for consumers.

Generic Class Test Intent: Develop a test implementation of and a test suite for a class that requires a type parameter. Context: This pattern applies to any generic class; Generic classes must be instantiated with a type parameter and tested with that type binding;

Generic Class Test Fault Model: Requires strong type checking; Type depended faults; Type specific faults; Multiple type interaction faults;

Generic Class Test Strategy: Single Type Parameter Type independent tests; Type specific tests; Type overloading tests. Two or More Type Parameters More complex testing problem Exercise combination of types

Generic Class Test Strategy: Single type parameter: the following steps will allow later test to reuse earlier tests. 1. Choose a representative type parameter; 2. Select applicable method and class test patterns and develop test suites accordingly. (Category Partition and Nonmodal Class test) 3. Run the test suite on an instantiation of the representative type, debug it, and achieve the appropriate coverage. 4. Determine how many additional type dependent instantiations should be tested. 5. Determine how many additional type specific instantiations should be tested. 6. If the class exports overloadable operations, then develop a test for each possible combination.

Strategy: Two or more type parameters: Test as many type combinations as time permits; Select the types that will be used most frequently; Replace the secondary parameter, revise a copy of the test suite and repeat; Repeat the cycle until all combination have been tested and pass with acceptable branch coverage.

Generic Class Test Automation: Implement a test driver for generic class as a generic class that accepts the same types parameters as the class under test. Develop the test driver progressively.

Generic Class Test Entry criteria: All accepted classes identified as type should pass their own test suites; Meet entry criteria for applicable test patterns. Exit criteria: At 100% branch coverage or better on every tested instantiation of the CUT. Achieve exit criteria for the applicable test patterns.

New Framework Test Intent: Develop a test implementation of and a test suite for a framework that has few if any instantiations. Context: This pattern is applicable to a system of classes intended to support development of specific apps by extension and to class hierarchies that offer a generic programming infrastructure. A framework may be developed in 2 ways: A modal based framework: supports a family of app systems that share a common set of classes Generalized framework: generalizing an antecedent app system (follow the patterns for evolving frameworks)

New Framework Test Fault Model: Frameworks make three general promises: They are a complete and consistent representation of the salient aspects of the problem domain as classes and associations; They provide basic capabilities to create, read, update and delete objects of these classes and the associations among them; They enforce sequential control necessary for required behaviour.

New Framework Test Fault Model: Likely kinds of bugs: Design omission; Representational integrity bugs; Control bugs; Infrastructure bugs.

New Framework Test Strategy: Test Model A testable instantiation must be designed and developed (demo app) The use case model should include all use cases that developers can identify for the entire framework Use Extended Use Case Test do develop tests for each use case Use Class Association Test to develop tests for each association Model a state machine and develop a test suite to achieve N+ coverage The demo app must implement features that are the same or nearly the same as those of the predecessor system It does not need to implement all features of the predecessor system It can implement features not found in the predecessor system.

Popular Framework Test Intent: Develop a test suite for a framework that has many application specific instantiations. Fault Model: A feature interaction; A compatibility bug; A latent bug;

Popular Framework Test Strategy: Test model: Feature interaction testing: Test suite for the version of the FUT; Additional feature interaction tests from anti requirements; Feature combinations; A regression test suite should be run; Test cases for the new features using Extended use case test and class association test; Automation: Test suite implemented with a driver suited to the external interface of the demo app.

Popular Framework Test Entry Criteria: A testable system scope should be available; New and changed classes/subsystems should have passed their individual test suites; System scope test harness should be installed, tested and stabilized; Test environment should be installed, tested and stabilized; Optional: you have build 1 actual instantiation of the current release; You have conducted an analysis of the new and changed features.

Popular Framework Test Exit Criteria: Same as for New Framework Test;

Subsystems What is a Subsystem? Why Test Subsystem Scope? Design patterns: Class Association Test; Controlled Exception Test; Round trip Scenario Test; Mode Machine Test.

What? Any testable collection of classes, objects, components and modules Characteristics: Executable and testable as whole Has parts that can be tested in isolation Implements cohesive set of responsibilities Doesn t provide all the functionality of the application under test.

Types Small clusters Pattern participants Developer subclasses Family of Ada 95 packages Large clusters Build group Task group

Why? It is a precondition for testing at system scope It is a practical necessity Support for internal release during incremental, iterative development Use cases can model requirements for any subsystem with public interface Some classes and other components have been bundled for historical reasons and must be tested as whole Some subsystems must work for additional funding Testing budget or schedule is limited or reduced Dependability of some subsystems is higher than of others The target environment is unique or different or must be simulated apart from the rest of the application

Test Design Pa>erns Class Association Test Controlled Exception Test Round trip Scenario Test Mode Machine Test

Class Associa9on Test Intent: Design a test suite to verify the implementation of required associations among classes Context: Correctly implement the relationships in a Class Diagram It will reveal typical bugs in the implementation of associations.

Class Associa9on Test Fault model: Kinds of faults Incorrect multiplicity Update anomaly Delete anomaly Missing link Wrong link Exercise the code that implements the associations constraints

Class Associa9on Test Strategy: Test Model: Multiplicity Test Sets Testing for Update /Delete Bugs Wrong and Missing Links Automation: Increase testing by interleaving anomaly tests with max value multiplicity tests Via an API driver interface Set up test cases with a db loader utility

Class Associa9on Test Entry Criteria: Each class in the scope of the model has passed an alpha omega cycle Keyed collections are typically used to implement associations. These classes should be individually tested with Quasi modal Class Test

Class Associa9on Test Exit Criteria: Each multiplicity on point test passes Each multiplicity off point test passes A deletion anomaly test passes An update anomaly test passes All tests for mutual exclusion pass

Round trip Scenario Test Intent: Extract a control flow model from a UML Sequence Diagram and develop a path set that provides minimal branch and loop coverage

Round trip Scenario Test Fault Model: A scenario is one path through a Sequence Diagram Following bugs can occur: Incorrect/missing output Acceptance of incorrect selection or object Action missing on external interface Missing function/feature in a participant Correct msg passed to wrong object Incorrect msg passed to right object Msg sent to destroyed object Correct exception raised, but by a wrong object Incorrect exception raised to right object

Round trip Scenario Test Strategy: Test Model: The information in the Sequence Diagram can be transformed into a flow graph, witch is highly testable model. Drawing the control flow graph will probably reveal ambiguities and omissions in the Sequence Diagram.

Round trip Scenario Test Strategy: Test procedure: 4 steps 1. Translate the Sequence Diagram into a Flow Graph; 2. Identify Paths to Test from the Flow Graph; 3. Identify Special Cases; 4. Identify Inputs and States Needed to Cause a Prticular Path to Be Taken.

Round trip Scenario Test Strategy: Automation: Test suite can be implemented with a GUI scripting language or in an API test driver. Entry Criteria: The objects, components or subsystems participating in the scenarios should have passed at least a minimal test suite or have been sufficiently reliable in actual use.

Round trip Scenario Test Exit Criteria: Every branch in a derived flowchart is taken at least once; Each loop is bypassed or iterated for minimum number of iterations; Each loop is iterated once and then exited; Each loop is iterated for the maximum number of iterations.

Controlled Excep9on Test Intent: Design a test suite that will exercise exceptions handlers and provide a systematic means to generate all exceptions Context: The Controlled Exceptions Test pattern applies to implementations expected to handle application specific exceptions or exceptions raised by target environment. Computational exceptions; I/O errors File errors Main storage management Task management

Controlled Excep9on Test Fault Model: General kinds of faults that can occur: An exception is incorrectly passed from client to server; An exception propagates out of scope; An application specific exception handler is missing/incorrect; An exception is ignored and results in an immediate process/task abend; An exception is ignored and results in a mysterious failure in unrelated code; An exception handler creates an adverse side effect; The recovery attempted by an exception is incorrect or incomplete; The exception structure loops;

Controlled Excep9on Test Fault Model: If exist, these can be reached only by activating the code that handles the exceptions; This pattern does not directly target faults in logic or computation leading to incorrect output, sequential behavior constraints or performance problems.

Controlled Excep9on Test Strategy: Test Model: A state machine or combinational logic can be used as test model (Category Partition); Exception testing can be accomplished in 3 ways: Activation; Inducement; Simulation;

Controlled Excep9on Test Strategy: Test Model: Exception activation: activated through input and state values supplied by test cases. Exception inducement: are induced by directly manipulating the implementation under test or target environment: Mistune; Cripple; Pollute; Mutate/Zap. Exception simulation: simulated by using controllable wrappers for application server or virtual machine services.

Controlled Excep9on Test Strategy: Automation: Stubs are commonly used to simulate application specific exceptions; Command line scripts can be developed to generate anomalous conditions to trigger exceptions; A wrapper class can be used to simulate exceptions.

Controlled Excep9on Test Entry Criteria: Do exception testing after responsibility testing of the server and client has been completed. Exit Criteria: Each server exception should be raised at least once, and each handler in the IUT should be activated at least once; If the handler is nontrivial, coverage appropriate for the responsibility of the exception handler is achieved; If the exception has a long percolation chain, each link should be traversed at least once.

Mode Machine Test Intent: Develop a test suite for subsystem scope behavior by analysing component behavior. Context: This test pattern is useful for testing subsystems whose behavior is sequentially constrained; A modal subsystem includes a dominant controle object, implements a state pattern or has similar control strategy; This pattern produces a test suite that helps solve several hard testing problems: Identification of end to end threads; Unambiguous determination of external sequences necessary to test a thread given a certain constellation of class states; Unambiguous determination of external sequences necessary to place an object in a given state.

Mode Machine Test Fault Model: The state of an application subsystem consists of the collective state of its object, or mode; When sequences of external events are constrained, a correctly functioning subsystem must 1. Accept legal events; 2. Reject events that are illegal in these states; 3. Produce a correct resultant state for accepted and rejected events; 4. Produce correct action for each test event msg.

Mode Machine Test Fault Model: A subsystem implementation can have 5 kinds of control faults: Missing transition; Incorrect action; Invalid resultant state; The production of a corrupt state; A sneak path that allows an event to be accepted when it should have been rejected.

Mode Machine Test Strategy: Test Model: Mode Machine Test produces a state based model of subsystem behavior that can generate N+ test suite. Test model is developed in the following steps: Indentify the dominant control object for the subsystem under test and develop its state model; Identify the external events and actions of the subsystem under test; Map the external events onto the dominant control object (the mode machine); Develop a modal test suite for this state model.

Mode Machine Test Strategy: Test Model: The test generation procedure is the same as used in the modal class pattern: 1. Identify the Dominant Control Object; 2. Model External Events and Actions; 3. Develop an N+ Test Suite (composed of 5 kinds of tests): 1. Conformance tests; 2. Guard expansion; 3. Iteration expansion; 4. Implicit transition expansion; 5. Sneak path analysis.

Mode Machine Test Strategy: Automation: A modal test suite may be implemented with any of the API test Harness patterns (Chapter 19). Entry Criteria: Each component class or cluster has met the exit criteria for the applicable test pattern. Leaf node test cannot be reached until all predecessor test have passed; Each cluster has met minimal operability requirements. Exit Criteria: There is a test for every root to leaf path in the expanded transition tree and each sneak path pair. These test sequences pass.

The End