ombinational Models Generating test cases when the test model is a decision table Textbook Reading: hapter 6 ecision tables Ideal representation for a test model for the following reasons: Straightforward representation of requirements Effective for revealing bugs They support test design at any scope, from method to system They support manual or automated generation of test cases Testing with combinational models Five basic steps Model the implementation with a decision table Validate the decision table model erive the logic function hoose a test suite generation strategy Generate test cases Necessary haracteristics of the Implementation One of several distinct responses is selected according to district cases of input variables that can be modeled by boolean expressions Response does not depend on the order input variables are set Response does not depend on prior input or output eriving decision tables ecision tables Four steps: Identify the decision variables Identify the resultant actions to be selected Identify which actions should be produced in response to particular combinations of conditions Verify the model s completeness and consistency ondition section Lists conditions and combinations of conditions ondition: relationship between decision variables that evaluates to true or false ecision variables: Inputs ction section Lists responses to be produced when particular conditions are true Many actions may be specified for a particular combination
n example ecision tables Variant 5 6 ondition section Number of claims - - Insured ge <=5 >=6 <=5 >=6 <=5 >=6 Premium increase 5 5 5 ction section Send warning ancel N conditions => at most N variants The decision table in the previous slide has 6 conditions, i.e. 6 possible variants Only seven explicit variants variant not given is an implicit variant 7 5+ ny Implicit variants Valid abbreviations on t care Type-safe exclusions Incorrect modeling an t happen on t know on t care esignates a condition that may be true or false (for a particular variant) without affecting the action, e.g. ny age in variant 7 Two distinct cases if ((w>x) (w>y)) Type-safe exclusions: ge cannot be both and an t happen ssumption that certain inputs are mutually exclusive or cannot be produced hronic source of bugs riane 5: an t happen that a certain parameter requires more than 6 bits $5 million of uninsured damage If age is modeled as two boolean variables younger_than_6 and older_than_5, we cannot assume that both of them being True can t happen on t know Incomplete model The applicability of a condition or the expected outcome may be unknown ocumentation unavailable Incomplete requirements For example, what should happen if age is or. Unlikely that the right response is to renew the policy
ecision tables and OO Method scope: Switch statement lass scope class public utopolicy { utopolicy (ate O); public void makelaim(); public void annualrenewal(); public Money getpremiumrate(); public bool isanceled(); public bool isctive(); } eriving the Logic Function Review boolean algebra = and + = or ~ = not logic function maps n boolean input variables to m boolean output variables truth table is an enumeration of all possible input and output values Example truth table Truth table definitions Input Vector Number 5 6 7 8 9 rmal Pressure all For Heat amper Shut Manual Mode Ignition Enable Each row of a truth table is an input vector The logic function for the example is = (~ + ) In sum of products form: = ~ + Each term in a sum of products is an implicant. If true, the whole function is true single-term formula with all variables that evaluates to true is a minterm Examples:, ~, ~~ 5 More definitions Logic minimization cube is a compressed partial truth table Variables that do not affect the result are compressed The example truth table has four cubes cube is a prime implicant if it evaluates to true and is not a subset of another cube Techniques to derive compact boolean expressions Useful to software testing Minimal expressions can be covered with fewer tests utomatic test generation runs faster The process may reveal inconsistencies or omissions in the spec We will look at two different techniques
Karnaugh Maps Graphical technique Good for up to 5 variables The truth table is transformed to a Karnaugh map. In our example Karnaugh maps Find the largest group of adjacent cells. Groups are formed only horizontally or vertically Groups have k members and may wrap around the edges of the map Karnaugh maps Transcribe the product term for the group. For each variable If the group covers only values, enter the variable If the group covers only values, enter the negated variable If the group covers both and values, enter nothing The group in the previous slide transcribes to. Karnaugh maps Repeat until all s in the map are covered The second group transcribes to ~, which gives the final result = ~ + ause-effect graphs Useful for analyzing the relationships to be modeled in a decision table Often beneficial at early stages of specification development node is drawn for each cause and effect line from a cause to an effect indicates that the cause is a necessary condition for the effect Multiple lines to an effect are joined by or Intermediate nodes may be added The insurance policy example Variant 5 6 7 ondition section Number of claims - - 5+ Insured ge <=5 >=6 <=5 >=6 <=5 >=6 ny Premium increase 5 5 5 ction section Send warning ancel
Insurance policy cause-effect graph <=5 $5 >=6 $5 $ O claims $ claim - claims $ Warning 5+ claims ancel Logic function derivation ause-effect graphs can be translated directly to boolean functions an be done one effect at a time The effect can be transcribed in terms of the intermediate nodes that can then be refined further The cause-effect graph in the next page has 8 causes. full truth table would have 56 variants Logic function derivation ecision Table Validation The derived logic function is = + ~E + F + G + H + ~ + ~~E + ~F + ~G + ~H + ~ + ~~E + ~F + ~G + ~H E F G ~ Q S ~ P R decision table must be validated before it is used to generate test cases spreadsheet can be used to verify that the logic functions have been derived correctly Manual inspection is important to catch requirements analysis problems ertain conditions that will ease testing and help catch errors can be checked H ecision Table hecklist ction selection is independent of the order of variant evaluation T ction selection is independent of the order of condition evaluation T ction selection is independent of prior actions T Each variant is mutually exclusive T ecision Table hecklist If no explicit variant is satisfied by an input vector, either a default action is selected or the implementation will do something acceptable T Each variant is unique Each action has a testable specification ll specified output actions are observable 5
ecision Table hecklist ll specified decision variables are controllable domain specification is given for each nonbinary decision variable, e.g. age: 6-85, Number of laims - The domains of a non-binary decision variable are adjacent, or if not there is a good reason n explicit boolean formula is given for nontrivial decision tables ecision Table hecklist The chosen action for the function has been independently verified in all cases The action specified for a variant with a don t care condition is acceptable for all possible values of the condition T If there are implicit variants, there is a designated default action T The designated default action is appropriate for all implicit variants ecision Table hecklist Implicit variants for type-safe exclusions are based on trusted type checking ttempt to identify cases where assumed exclusions do not hold T Scrutinize all can t happen conditions: reject, revise, or replace with an explicit default action Scrutinize all don t know conditions: reject, revise, or replace with an explicit default action Testing strategies ll-explicit variants ll-variants, ll-true, ll-false, ll- Primes Each-ondition/ll-onditions -eterminants Variable Negation nbinary Variable omain nalysis Fault models Each testing strategy is guided by a fault model fault model identifies relationships and components of the system that are likely to have faults ased on common sense, experience, suspicion, analysis, or experiment Fault model for decision tables Incorrect value assigned to a decision variable Incorrect or missing operator in a predicate Incorrect or missing variable in a predicate Incorrect structure in a predicate (e.g. dangling else, misplaced semicolon) 6
Fault model for decision tables Incorrect or missing default case Incorrect or missing actions Extra actions Structural errors in the implementation (e.g. falling through) esign errors indicated by the validation checklist (items with T) ll-explicit Variants Each explicit variant is produced at least once efaults actions (if any) are also tested ecision variable boundaries are systematically exercised cceptable if implicit variants are due to type safe exclusions an t happen conditions and undefined domain boundaries cause problems Truth table approaches ll-variants: Every variant is tested once. The number of tests is N, infeasible for large tables ll-true: ll variants that produce a true outcome are tested. Equivalent to testing all minterms of the logic function. Inappropriate if false actions are an important part of the behaviour Truth table approaches ll-false: ll variants that produce a false outcome are tested. Equivalent to testing the complements of all minterms of the logic function. Inappropriate if true actions are important ll-primes: Each prime implicant is tested once. Subset of the ll-true strategy due to don t care conditions Each-ondition/ll-conditions ssumption: For the logic functions =PQR and Y=P+Q+R, the only relevant cases are: Each-ondition/ll-conditions For a logic function such as = ~ + we apply the heuristic to every term P Q R P Q R Y Test cases for ~ Test cases for 7
Each-ondition/ll-conditions The number of tests increases linearly with the number of product terms Implicit conditions are a problem for this technique Values need to be assigned to the on t are variables. an be done randomly or by suspicion. Try different combinations for different test cases ecision tree for boiler example inary ecision iagrams Reduction process First Reduction ompact representation of a truth table an be used to create test suite can be generated from the truth table Start by drawing the full decision tree left branch represents False () right branch represents True () Working from left to right, replace leaf nodes with equivalent constants or variables and prune the branches This way we can prune the entire left branch of the tree since it only depends on the value of Second Reduction 8
Final inary ecision iagram determinant table Test suite generation Variant Variable Negation Strategy ll combinations of sensitive variables, e.g., are created determinant is a root-to-leafpath in the The determinant table has one row per path Variant esigned to reveal faults that hide in a don t care The test suite contains:.. Example truth table Unique true points: variant per term t, so that t is True and all other terms are False Near False Points: variant for each literal in a term. The variant is obtained by negating the literal and is selected only if it makes = Each variant creates a test candidate set Unique true point candidate sets in boiler example: {} {9,,5} Negation variants Input Vector Number rmal Pressure all For Heat amper Shut Manual Mode Ignition Enable 5 6 7 8 9 5 9
Selecting the test cases t least one variant from each candidate set an be done by inspection (next slide) Random selection is also used Near False Points exercise combinations of don t care values 6% of all possible tests are created 98% of simulated bugs can be found omain nalysis oiler Test ase Selection omain analysis for Variant Each variant with non-binary variables defines a subdomain (7 in the insurance policy example) Minimal domain test strategy:pick one ON point and one OFF point per boundary These points will often overlap reducing the number of test cases omain analysis for Variant 6 hoosing a test strategy Tradeoff: ost (roughly proportional to the number of tests) vs. onfidence in the test implementation Each testing strategy produces a test suite of different size (next slide) Hard to evaluate the fault-revealing power of each strategy Quantity effectiveness
Test suite sizes Strategy hierarchy Practical observations Practical observations For small tables with non-binary decision variables, the explicit variant strategy augmented domain tests is prudent and feasible ll-variants is prudent and feasible for small and medium-sized tables Each-ondition/ll-onditions can be used to create a compact test suite for all sizes of tables Variable negation strategy is the clear choice for large tables. It is highly effective and produces a small suite. lgorithms for automatic generation have been developed For a quick-and-dirty strategy: llprimes and ll-explicit can be considered