Błaej Pietrzak Goal. Two approaches. Process. If we limit testing to method scope

Transcription

1 Błaej Pietrzak Class scope integration Design by Contract Invariant Boundaries pattern Non-modal class pattern FREE state model Quasi-modal class pattern Modal class pattern If we limit testing to method scope Easily miss faults related to intraclass usages Method interactions are a common source of faults Side effects and fault masking are sources of error Eercising methods in various sequences is necessary to reveal intraclass bugs Goal Demonstrate that class under test is ready to test Two approaches Small pop Alpha-Omega cycle A Big Bang integration at class level Ecersise all untrusted components simultaneously Effective when CUT is small and simple Servers of the class are stable Inherited features are stable Few intraclass dependencies eist Process Develop entire class Write a test driver using any appropriate test pattern Run the test suite Debug as needed 1

2 Alpha-Omega States: Alpha State: object before it is constructed Omega State: object after it has been deleted Alpha-Omega Cycle: Take an object from alpha to omega Send a message to every method at least once Shows that class is ready for more etensive testing No attempt is made to achieve any coverage Order of calling methods: Constructor method Accessor method Boolean (predicate/is) method Modifier (set) method Iterator method Destructor method Order within steps: private, protected, public Precondition Conditions that must hold before a method can eecute Client class must fulfill precondition Postcondition Specify conditions that must hold after a method completes Server class must fulfill postcondition Class invariant Specifies a condition that must hold anytime a client class could invoke an object s method Must be true on entry and eit of public methods Constructor Before: check class precondition After: check class invariant, check constructor postcondition Destructor Before: check class invariant, check destructor precondition After: check destructor postcondition Method Before: check class invariant, check method precondition After: check method postcondition, check class invariant! " Contracts are inherited Subclass must fulfill its superclass s public type contract Require no more ensure no less! Inheritance rules Subclass may keep or strengthen the class invariant Subclass may keep or weaken the precondition Subclass may keep or strengthen the postcondition Intent Select test-efficient test value combinations for classes, interfaces, and components composed of comple and primitive data types Contet The valid and invalid combinations of instance variable values may be specified by the class invariant The class invariant typically refers to instance variables that are instances of primitive and comple data type The Invariant Boundaries pattern does not consider input/output relationship or message sequence

3 Fault Model Bugs in implementation of constraints needed to define and enforce a domain formed by several comple boundaries Subclass invariants are often implicit or accidental, leading to misinterpretation and misuse Strategy 1. Develop the class invariant. Develop on points and off points for each condition in the invariant using Develop in points for variables not referenced in a condition. Represent the results in a domain matri Entry Criteria The class invariant eists or can be created test suite s entry criteria designed with another pattern must be fulfilled Eit Criteria A complete set of domain tests has been developed Consequences The class invariant development is difficult and time consuming After the class invariant is developed test suite creation is fast The number of test cases grows linearly with the number of conditions in the invariant class ClientProfile { Account account = new Account(); Money creditlimit = new Money(); short trcounter; Account abstract states: Opened, Closed, Idle, Locked, Debit creditlimit is in range ± creditlimit >= trcounter * trcounter must be >= 0 and <= 5000 Define Class Invariant assert (trcounter >= 0) && (trcounter <= 5000) && (creditlimit>= trcounter * ) &&!account.isclosed()); Develop on points and off points Condition trcounter >= 0 trcounter <= 5000 creditlimit >= trcounter * !account.isClosed() On point Opened Off point Closed Develop on points and off points cont. creditlimit min = (0 * 100) = creditlimit ma = (5000 * 100) = trcounter central = ( ) / = 500 creditlimit central = (500 * 100) = creditlimit off = creditlimit central =

4 Variable trcounter creditlimit Account Represent the results in a domain matri Constraint Test case Condition >= 0 <= 5000 Typical >= trcounter * Typical!isClosed() Typical Point On Off On Off In On Off In On Off In open ,99 open debit Lockd idle 8..1 open open closed Represent the results in a domain matri cont. Each test case has only one on or off-point Select in-points for all other values in the test case Avoid to repeat in points (increase chance of finding bugs) Accept any message in any state No domain state constraint No message sequence constraint Classes that implement basic data types are often nonmodal i.e. Time class Time { private int hh, mm, ss; public Time(int hh, int mm, int ss) { public void sethour(int value) { public void getsecond() { public boolean equals(time t) { public Time subtract(time t) { Only one abstract state (any possible time) and many concrete states (e.g. 3:00:01, 1:15:9 etc.) Any possible message can accur after any possible message with eception for constructor and destructor No domain state constraint Message sequence constraint Classes that are application controllers are usually unimodal Eample: traffic lights turnonredlight can only be accepted after turnonyellowlight turnongreenlight only after turnonred, turnonyellowlight Imposes sequential constraints on message acceptance that change with the contents of the object Many container and collection classes are quasi-modal e.g. Stack Behaves like modal class, but message sequence constraints are implicitly connected with the contents of the object i.e. stack - size determines whether stack is full, empty, or loaded. It doesn t matter what are the contents of the stack or in what order. Test model is focused on parameters describing the contents i.e. stack size In modal class the contents control the behavior directly

5 Domain state constraint Message sequence constraint Eample Account class Account { private Money saldo; private int accountnumber; private Date lastoperation; public void open() { public Money getsaldo() { public void credit(money creditamount) { public void debit(money debitamount) { public void lock() { public void unlock() { public void regulate() { public void close() { If [saldo == 0] then close() If close() was invoked it cannot accept close() Intent Develop a class scope test suite for a class that does not constrain message sequences Contet Nonmodal class imposes few constraints on message sequence but usually has a comple state space and a comple interface Few or no message sequences are illegal; most message sequences are legal Fault Model Sequential constraints on messages are not imposed so the state control model is trivial State-based testing (see Modal Class Test) is inappropriate Allowed sequence is rejected Allowed sequence produces bad value Methods reporting about abstract state are inconsistent Allowed modifier sequence is rejected Not allowed modifier s argument is accepted resulting in corrupted state Accessor method has incorrect side effect, which changes or corrupts object state Not allowed calculation violates class invariant Some modifier or accessor methods cause inconsistent abstract state view Strategy 1. Develop a set of test cases using Invariant Boundaries pattern. Select a message sequence strategy: define-use (setters and getters), random (setters and getters in random order), or suspect (only suspected combinations of setters and getters) 3. Set the Object Under Test to a test case from the domain matri. Send all the accessor messages 5. Verify that the returned and resulting values are consistent. Repeat until all sequences have been eercised Basic test strategy 1. To set the Object Under Test to a test vector value with a modifier (set, constructor). Verify that modifier has produced a correct state 3. Try each accessor (get) to see that it reports the state correctly without buggy side effect 5

6 Entry Criteria Alpha omega cycle Eit Criteria Object Under Test has taken values of each test case at least once Achieve at least branch coverage on each method in the Class Under Test Consequences The number of tests is m n+1 where m is the number of methods in CUT, n is a number of modifier accessor pairs typically n = public void testtime1() { Time temp = new Time(0,, 9); assert(0, temp.gethour()); assert(, temp.getminute()); assert(9, temp.getsecond()); public void testtime() { time.setminute(); time.sethour(0); time.setsecond(9); assert(0, temp.gethour()); assert(9, temp.getsecond()); assert(, temp.getminute()); public void testtime3() { try { Time temp = new Time(-1,, 8); fail(); catch (SomeEception e) { A system model composed of events (inputs), states and actions (outputs): Output is determined by both current input and past inputs An information concerning past inputs is represented by states Robert V. Binder: Testing Object-Oriented Systems. Models, Patterns and Tools, Addison-Wesley 000 State The Information concerning past inputs Initial State first event is accepted, not need to be entered by a transition Transition An allowable two-state sequence Final State stop accepting events Accepting state Resulting state Machine may be in only one state at a time

7 Event An input or interval of time Action The result or output that follows an event A transition may have an output action Any output action may be used in more than one transition No output action is associated with state, state is passive Incomplete Specification Equivalent (A minimal state machine) Reachability analysis Nonreachable states Dead state no paths lead out of a dead state Dead loop no states outside of the loop may be reached Magic state has no inbound transition but provides transitions to other states A guarded transition cannot fire unless The system is in accepting state for this transition The guarded event occurs The guarded predicate evaluates to true A transition without an eplicit guard: [TRUE] An event not accepted in a certain state: [FALSE] Event-name arg-list [ guard-condition ] / action-epression Not specific for object oriented systems An OO interpretation of generic events, actions and states is needed Limited scalability Up to 0 states on diagram is readable Basic state models do not scale well No Partitioning, hierarchic abstraction If.. else, switch statements Many design patterns in GOF collection: State, Chain of Responsibility, Interpreter, Mediator Other patterns Concurrency cannot be modeled The basic model cannot accomodate two or more simultaneous transition concurrency 7

8 Complete and accurate reflection of implementations to be tested Preserve details essential to fault detection and conformance demonstration Represents all events to the IUT must respond Represents all actions Unambiguous and testable definition of state in a way that checking the resulting state can be automated Flattened Regular Epression (FREE) state model The class under test is flattened to represent the behavior of inherited features Behavior using definitions of state, event, and action that support the development of effective test suites Can be epressed using the UML Object state at method activation and deactivation is the focus of the FREE state model Object state is observable Hybrids are not welcome! Use only one representation (e.g. Mealy machine) Transition is a unique combination of State invariants An associated event Optional guard epressions Event is either A message sent to the class under test A response received from a server of the class under test An interrupt or similar eternal control action that must be accepted by the CUT Guard is A predicate epression associated with an event Action is either A message sent to a server object The response provided by an object of the class under test to its client The (alpha) state A null state representing the declaration for an object before its construction Differs from an initial state An alpha transition The (omega) state It is reached after an object has been destructed or deleted, or has gone out of scope It may differ from an eplicitly modeled final state It is reached by eplicit or implicit destructors 8

9 # %$ &'$ )(* + To obtain the model for subclasses, the class hierarchy is flattened The flattened transition diagram shows the complete behavior of the class under test. Class flattening is not always necessary: When only test within subclass boundaries without checking inherited properties Not enough information about superclass Lack of time May mean that implicit transition is illegal, ignored, or impossible May also be a specification omission An illegal event is an otherwise valid event(message) that should not be accepted given the current state of the UIT A sneak path is the bug that allows an illegal transition or eludes a guard Complete behavior testing should eercise both eplicit and implicit behavior Events and guards constructor Event 1 Event = 0 Event 3 i <= 1000 NI Accepting state epected output A B C Events and guards Accepting state epected output Events and guards Accepting state epected output A B C A B C Event i!= k < ma Event 5 j > 10 k = ma isreset() Ture 9

10 Events and guards destructor don t care ecluded - eplicit transition Accepting state epected output A B C Resp. code Response codes for illegal states Name Acceptance Queue Ignored Marker Rejection Silence Damage stopping Response Eecute eplicit transition Place illegal event in queue for calculation and omitting Do nothing Return non-zero error code Throw IllegalEventEception Turn off the source of event Go to damage stopping routine (e.g. Memory dump) and stop the process An appropriate error message or eception should be produced The abstract state of the object should remain unchanged after rejecting the illegal event Prefer defensive school easier to test 1. Create a testable state model (e.g. FREE state model). Validate state model Structure State names Guarded transitions Well-formed subclass behavior Robustness Strategies: Ehaustive preffered N+ strategy preffered Number of tests about k n /, where k is states no., n is events no. All transitions minimum at least once all states, all events, and all actions Although detects bad or omitted event-action pairs, it cannot be proved that bad state occured. If state machine is incomplete cannot detect sneak paths Number of tests = k n, where k is states no., n is events no. Piecewise (pol. sztukowanie) not recommended Advanced state-based testing UML state models Testing considerations unique to OO implementations It uses a flattened state model All implicit transitions are ecersised to reveal sneak paths The implementation must have a trusted ability to report the resultant state 10

11 1. Develop a testable (e.g. FREE) model of the implementation under test Validate the model using the checklists Epand the statechart Develop a response matri. Generate the round-trip path test cases 3. Generate the sneak path test cases. Sensitize the transitions in each test case FREE model of the implementation under test Flattened transition diagram Response matri FREE state model -> Transition tree Initial state or alpha state (if eists) is a root of the tree. Check the state of every not-final node leaf in the tree and every transition going from that state. For each transition create at least one edge. Each new edge and node represent an event and resulting state reached by outgoing transition:. a) If not guarded condition create one new branch. b) If guard is a simple predicate or composed of only AND operators create one new branch. c) If guard is composed predicate or with OR operators then create new branch for each combination of values that make guard = true.. d) If guard defines dependency that is reached after repetition of some event (e.g. [counter >= 10]) then the test sequence demands at least that number of repetitions. The transition is marked with * 3. For each edge and node from step : 1. Note event, guard and action. If the state represented by new node is already represented in another node (anywhere on diagram) or is ending state then mark this node as ending node don t make new transitions from it. Transition tree -> Conformance test suite Response matri -> Sneak path test suite 11

12 Path sensitization is the process of determining argument and instance variable values that will cause a particular path to be taken. Paths that cannot be eecuted. Short circuit evaluation and comple predicate epressions Contradictory or mutually eclusive conditions,e.g. >1&&<-1 Mutually eclusive, redundant predicates, if (==0) if (!=0). Unstructured code that include upward-branching gotos or branches that enter or eit a loop body without first activating the loop control. Dead code. This should never happen impossible conditions. Intent Develop a class scope test suite for a class whose constraints on message sequence change with the state of the class Contet A quasi-modal class has sequential constraints that reflect the organization of information used by the class Container and collection classes are often quasi-modal Effective testing must distinguish between content that determines behavior and content that does not affect behavior Fault Model Quasi modal class failures occur When invariants related to all members of a collection are not observed Strategy 1. Create N+ state model test suite Use N+ test strategy Model constraint parameters as states Develop test data using invariant boundaries. Run class specific operation-pairs Strategy detailed 1. Develop FREE state model. Characterize state with constraint params not content. Treat each constraint param as state variable.. Generate transition tree. 3. Tabulate events and actions along each path.. Develop test data for each path using Invariant Boundaries pattern for events, messages and actions. 5. Eecute conformance test suite until all tests pass.. Develop sneak path test suite. Add all forbidden transitions in all states and define eception to be thrown. 7. Eecute sneak path test suite until all tests pass. 8. Run class specific operation-pairs. Entry Criteria Alpha omega cycle Eit Criteria Achieve at least branch coverage on each method Provide N+ coverage Optionally develop a class flow graph Identify uncovered alpha-omega paths Ecersise those paths Consequences Testable behavior model is available or can be developed CUT is state observable we have trustworthy built-in tests or feasible access to the CUT so we can determine the resultant state of each test run 1

13 class MyList { protected int size; final public static int MA = 1000; protected List list = new Vector(); public void add(object obj) throws DuplicateEception { public void remove(object obj) { public Object get(int id) { Class Invariant (not content) size >= 0 && size <= MA Transition tree remove()[ size == 1 ] new Empty add() Loaded remove()[size>1]* get() add()[size<ma-1]* add()[size=ma-1] remove() Full get() Empty Loaded Loaded Loaded Loaded Full Conformance test suite public void assertempty() { assert(mylist.isempty()); assert(mylist.isloaded()); assert(mylist.isfull()); public void testalphaemptyloadedempty() { MyList mylist = new MyList(); assertempty(); mylist.add(new Object()); assert(mylist.isloaded()); assert(mylist.isempty()); assert(mylist.isfull()); assertequals(1, mylist.size()); mylist.remove(obj); assertempty(); Response matri for add (not all guards are shown) Events and guards for size add() < 0 = 0 < ma - 1 = ma - 1 > ma - 1 Accepting state epected output Empty 0 Loaded don t care ecluded 0 Acceptance - eecute transition - eplicit transition (specified on state chart) Rejection - throw IllegalEventEception Full Invariant Boundaries for add Condition size < 0 size = 0 size < ma 1 size = ma 1 size > ma 1 On point 0 0 ma 1 ma 1 ma 1 Off point -1-1, 1 ma ma, ma ma 13

14 Sneak path test suite public void testcorruptedempty() { assertempty(); mylist.size = MA ; try { mylist.add(new Object()); fail(); catch (Eception e) { assertempty(); public void testcorruptedfull() { // make a full list assertfull(); mylist.size = MA ; try { mylist.add(new Object()); fail(); catch (Eception e) { assertfull(); public void testaddduplicate() { assertequals(, y); mylist.add(); // check state try { mylist.add(y); fail(); catch (DuplicateEception e){ Run class specific operation-pairs // check state public void testaddremove() { Object elem = new Object(); mylist.add(elem); // check state mylist.remove(elem); assertequals(0, mylist.size()); assert(mylist.contains(elem)); // check state Intent Develop a class scope test suite for a class that has fied constraints on message sequence Contet A modal class has both message and domain constraints on the acceptable sequence of messages Interactions between message sequences and state are often subtle and comple, therefore error prone Fault Model Omited transition message is rejected in allowed state Incorrect action bad response is chosen, although the state and method used were correctly accepted Resulting state is not allowed method develops a bad state in a transition Corrupt state is developed Sneak path allows to accept the message although it should be rejected Strategy 1. Create FREE state model. Generate transition tree 3. Equip the transition tree with full eplication of conditional transition cases. Tabulate events and actions along each path 5. Develop test data for paths use Invariant Boundaries for message events and actions. Develop Conformance tests until all tests pass 7. Develop Sneak path test suite until all tests pass 8. Event Path Sensitization Entry Criteria Alpha omega cycle If critical method/scope functions eist, they should be tested before running the modal because lead-node cannot be reached until all the predecessor tests have passed Eit Criteria Achieve branch coverage on each method in the Class Under Test A full modal class suite provides N+ coverage A higher level of confidence can be obtained by developing a class flow graph to identify and any uncovered alpha-omega paths 1

15 Consequences A testable behavior model is available or can be developed The Class Under Test is state-observable Robert V. Binder: Testowanie systemów obiektowych. Modele wzorce i narzdzia. WNT 003 Jtest - Automatic test generator (commercial) =Jtest&itemId=1 Jcontract Design by Contract in Java (commercial) =Jcontract&itemId=8 Icontract Design by Contract in Java Assertion Definition Language JML Specs Thank You for your attention What is your general impression (1-) Was it too slow or too fast? What important did you learn during the lecture? What to improve and how? 15