Class Modality. Modality Types. Modality Types. Class Scope Test Design Patterns

Transcription

1 Class Scope Test Design Patterns Testing methods in isolation is not enough Instance variables act like global variables within a class Need to test intraclass interactions Message sequences Class Modality Different classes impose different constraints on message sequence and variable content Categorizing classes in terms of modality helps focus on the most likely faults Different test suites will be generated for classes of different modality 1 2 Modality Types Nonmodal: No constraints on the sequence of messages accepted Basic data types are often nonmodal Example: DateTime. Any interleaving of get/set methods is allowed Unimodal: Constraints on the acceptable sequence of messages despite content Example: TrafficLight. All objects of this class will impose the same constraints on message sequence Modality Types Quasi-modal: Method sequence constraints are indirectly related to the content of the object Example: Stack. Only the number of elements in the stack, not their content, influences the behaviour Container and collection classes are quasimodal 3 4

2 Modality Types Modal: Constraints on message sequence directly related to the variable content Example: Account. A withdrawal will not be accepted if balance is negative Problem domain entities are often modal Pattern List We will discuss test design patterns for various modality types Invariant Boundaries Nonmodal Class Test Modal Class Test Quasi-modal Class Test 5 6 Invariant Boundaries Assumes that unusual combinations of instance variable values will reveal bugs Exercises all invariant boundaries Creates test cases that another test design pattern can use Does not consider message sequence Strategy Define the class invariant Develop on points and off points for each condition in the invariant using the 1x1 domain model Complete the test suite by developing in points for variables not referenced in a condition Represent results in a domain matrix 7 8

3 Example Consider the following class class CustomerProfile { Account account1 = new Account(); Account account2 = new Account(); Money creditlimit = new Money(); short txcounter; } The abstract states of class Account are Open, overdrawn, frozen, inactive, closed Example The class invariant is assert( (txcounter >=0 && txcounter <= 5000) && (creditlimit > && creditlimit <= ) && (!account1.isclosed()!account2.isclosed()) Represents business policy that places limits on the number of transactions, the credit limit etc Develop on and off points creditlimit and txcounter are scalar, so on and off points are easy to identify For Account objects they have to be identified based on the various states Abstract state in points: Open balance = 345, Overdrawn balance = Frozen isfrozen() Inactive Last transaction six years ago Closed isclosed() Developing on points Abstract state on points Open balance = 0.0 Overdrawn balance = Frozen isfrozen() Inactive Last transaction five years and one day ago Closed isclosed() 11 12

4 Developing off points Domain matrix for CustomerProfile Abstract state off points (focus state is open) Overdrawn balance = Frozen isfrozen() Inactive Last transaction five years and one day ago Closed isclosed() Similarly for the other four states Invariant Boundaries Entry Criteria A validated invariant is developed The entry criteria of the follow-up pattern will also apply Exit Criteria A complete set of domain tests has been developed Coverage criteria of the follow-up pattern will apply Invariant Boundaries Consequences Except for unusual circumstances, all domain bugs will be revealed Developing the invariant can help reveal bugs as well The test suite can be developed in less than an hour for many systems 15 16

5 Nonmodal Class Test A nonmodal class does not constrain message sequence There is only one abstract state State-based testing is inappropriate Still, many sequence related bugs are possible Fault model A legal sequence is rejected A legal sequence produces an incorrect value Methods that report abstract state are inconsistent An illegal modifier argument is accepted Accessor methods have side effects The class invariant gets violated by an incorrect computation Behaviours to test Message sequence strategies Define-use: A definition method followed by all of the use methods Random: Sequence of accessor and modifier calls is assigned randomly Suspect: Any sequence that may raise suspicion, e.g. one involving February 29 as the date 19 20

6 Nonmodal test strategy Example nonmodal class 1. Develop a set of test cases using Invariant Boundaries 2. Select a message sequence strategy 3. Set the OUT to a test case from the domain matrix 4. Send all accessor messages to verify resulting values 5. Repeat 3 and 4 for all sequences Variable Domains On and Off points 23 24

7 Test strategy Invariant Boundaries test set Pick any of the 24 test vectors Create an object with these values using the constructor or set methods OFF points should throw exceptions ON points should bring the object to the desired state Call all get methods to verify they return the correct value Verify that the object state did not change Exit criteria All define-use method pairs have been exercised Each test vector in the Invariant Boundaries test suite has been used at least once Branch coverage or better has been achieved on each method (or better on the class flow graph) Quasi-modal Class Test A quasi-modal class has sequential constraints that reflect the organization of information used by the class Container and collection classes Quasi-modal testing focuses on the structure of the class 27 28

8 A real world example Quasi-Modal Test Model A state model of the generic collection behaviour An Invariant Boundaries set for the parameters and parameter relationships that determine behaviour E.g. the maximum and the current number of elements in a stack Constraints placed on pairs of operations by the specific container E.g. cannot add an element to a set twice Generic State Model Transition tree 31 32

9 Exit criteria At least branch coverage on each method N+ coverage is provided. One test for each path in the transition tree and a full set of sneak path pairs A class flow graph can be developed to identify uncovered alpha-omega paths Modal Class Test Message sequence and domain constraints Message sequence constraints mean that the class will reject some sequences of messages (e.g., an Account can only be frozen if the Account is not closed and not frozen) Domain constraints mean that the class will reject some messages depending on the current content of the object (e.g., a withdraw cannot be made on an Account with balance <= 0) Modal Class Failures A modal class can fail in five ways: 1. Missing transition a valid message is rejected in a valid state 2. Incorrect action the wrong response is returned for an accepting state 3. Invalid resultant state a method produces the wrong state for a transition 4. A corrupt state is produced, i.e. violation of class invariant 5. A sneak path allows a message to be accepted when it should have been rejected 35 36

10 Modal Class Test Pattern: Essentials 1. Develop a state model for the class 2. Generate an augmented transition tree from the state model, including: All explicit round-trip paths Combinational expansions of guarded transitions Sneak path expansion 3. Test each transition path 1. Generating a State Model The state model is generated from a specification of how the class is intended to behave Generating a Transition Tree Determine the initial state of an object of the class under test (CUT). Make this initial state the root node of the transition tree. For each transition out of the root state, draw a branch to a node representing the resultant state Repeat for each resultant state node unless the resultant state is already an ancestor in the tree or it is a final state

11 2a. From Tree to Conformance Test Cases Make a test case for each full and partial branch in the transition tree b. Generate Conditional Transition Tests Some transitions are conditional Conformance testing ensures we fire the transitions when the conditions are met What if the condition is not true? We must: Develop a truth table for each conditional transition Add additional test cases for truth table entries not covered in conformance tests Add tests to the transition tree > 43 44

12 2c. Test for Illegal Transitions Test suite developed from the transition tree shows CUT correctly implements its behaviour model Also want to show the class does not allow excluded behavior 2c. Testing for Illegal Transitions To ensure no sneak paths exist, set the CUT to a particular state and attempt each illegal message The message should be rejected The state of the object should be unchanged A matrix is a good way to keep track of the possible sneak paths when deriving test cases

13 3. Completing the Test Cases Each test case, of course, requires the selection of parameter values, expected state, and exceptions To run a test case, an object of the class must be put into the appropriate state, the sequence run, and the results checked 49