Introduc)on to So,ware Tes)ng (2nd edi(on) Overview Graph Coverage Criteria. Paul Ammann & Jeff OffuA. hap:// so,waretest/

Transcription

1 Tes)ng (2nd edi(on) Overview Graph Coverage Criteria Paul Ammann & Jeff OffuA hap:// so,waretest/ First version, 23 September 2013

2 Graph Coverage Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source FSMs Applied to Specs DNF Source Specs Source Models Design Tes)ng, Edi)on 2 (Ch 06) Use cases Integ Input Ammann & OffuA 2

3 Covering Graphs (6.1) Graphs are the most commonly used structure for tes)ng Graphs can come from many sources Control flow graphs Design structure FSMs and statecharts Use cases Tests usually are intended to cover the graph in some way Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 3

4 Defini)on of a Graph A set N of nodes, N is not empty A set N 0 of ini)al nodes, N 0 is not empty A set N f of final nodes, N f is not empty A set E of edges, each edge from one node to another ( n i, n j ), i is predecessor, j is successor Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 4

5 Three Example Graphs Not a valid graph N 0 = { 1} N 0 = { 1, 2, 3 } N 0 = { } N f = { 4 } N f = { 8, 9, 10 } N f = { 4 } Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 5

6 Paths in Graphs Path : A sequence of nodes [n 1, n 2,, n M ] Each pair of nodes is an edge Length : The number of edges A single node is a path of length 0 Subpath : A subsequence of nodes in p is a subpath of p Reach (n) : Subgraph that can be reached from n A Few Paths [ 1, 4, 8 ] [ 2, 5, 9, 6, 2 ] [ 3, 7, 10 ] Reach (1) = { 1, 4, 5, 8, 9, 6, 2, 10 } Reach ({1, 3}) = G Reach([3,7]) = {3, 7, 10} 8 9 Tes)ng, Edi)on 2 (Ch 06) 10 Ammann & OffuA 6

7 Test Paths and SESEs Test Path : A path that starts at an ini)al node and ends at a final node Test paths represent execu)on of test cases Some test paths can be executed by many tests Some test paths cannot be executed by any tests SESE graphs : All test paths start at a single node and end at another node Single-entry, single-exit N0 and Nf have exactly one node Double-diamond graph Four test paths [1, 2, 4, 5, 7] [1, 2, 4, 6, 7] [1, 3, 4, 5, 7] [1, 3, 4, 6, 7] Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 7

8 Visi)ng and Touring Visit : A test path p visits node n if n is in p A test path p visits edge e if e is in p Tour : A test path p tours subpath q if q is a subpath of p Path [ 1, 2, 4, 5, 7 ] Visits nodes 1, 2, 4, 5, 7 Visits edges (1, 2), (2, 4), (4, 5), (5, 7) Tours subpaths [1, 2, 4], [2, 4, 5], [4, 5, 7], [1, 2, 4, 5], [2, 4, 5, 7] Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 8

9 Tests and Test Paths path (t) : The test path executed by test t path (T) : The set of test paths executed by the set of tests T Each test executes one and only one test path A loca)on in a graph (node or edge) can be reached from another loca)on if there is a sequence of edges from the first loca)on to the second Syntac(c reach : A subpath exists in the graph Seman(c reach : A test exists that can execute that subpath Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 9

10 test 1 Tests and Test Paths many-to-one test 2 test 3 Test Path Deterministic software a test always executes the same test path test 1 test 2 test 3 many-to-many Test Path 1 Test Path 2 Test Path 3 Non-deterministic software a test can execute different test paths Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 10

11 Tes)ng and Covering Graphs (6.2) We use graphs in tes)ng as follows : Developing a model of the so,ware as a graph Requiring tests to visit or tour specific sets of nodes, edges or subpaths Test Requirements (TR) : Describe properties of test paths Test Criterion : Rules that define test requirements Satisfaction : Given a set TR of test requirements for a criterion C, a set of tests T satisfies C on a graph if and only if for every test requirement in TR, there is a test path in path(t) that meets the test requirement tr Structural Coverage Criteria : Defined on a graph just in terms of nodes and edges Data Flow Coverage Criteria : Requires a graph to be annotated with references to variables Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 11

12 Node and Edge Coverage The first (and simplest) two criteria require that each node and edge in a graph be executed Node Coverage (NC) : Test set T satisfies node coverage on graph G iff for every syntactically reachable node n in N, there is some path p in path(t) such that p visits n. This statement is a bit cumbersome, so we abbreviate it in terms of the set of test requirements Node Coverage (NC) : TR contains each reachable node in G. Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 12

13 2 1 Structural Coverage Example Node Coverage TR = { 1, 2, 3, 4, 5, 6, 7 } Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 6, 5, 7 ] Edge Coverage TR = { (1,2), (1, 3), (2, 3), (3, 4), (3, 5), (4, 7), (5, 6), (5, 7), (6, 5) } Test Paths: [ 1, 2, 3, 4, 7 ] [1, 3, 5, 6, 5, 7 ] Edge-Pair Coverage TR = {[1,2,3], [1,3,4], [1,3,5], [2,3,4], [2,3,5], [3,4,7], [3,5,6], [3,5,7], [5,6,5], [6,5,6], [6,5,7] } Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 3, 4, 7 ] [ 1, 3, 5, 6, 5, 6, 5, 7 ] Complete Path Coverage Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 2, 3, 5, 6, 5, 6 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 7 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 6, 5, 7 ] Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 13

14 Loops in Graphs If a graph contains a loop, it has an infinite number of paths Thus, CPC is not feasible SPC is not sa)sfactory because the results are subjec)ve and vary with the tester AAempts to deal with loops: 1970s : Execute cycles once ([4, 5, 4] in previous example, informal) 1980s : Execute each loop, exactly once (formalized) 1990s : Execute loops 0 )mes, once, more than once (informal descrip)on) 2000s : Prime paths Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 14

15 Simple Paths and Prime Paths Simple Path : A path from node ni to nj is simple if no node appears more than once, except possibly the first and last nodes are the same No internal loops A loop is a simple path Prime Path : A simple path that does not appear as a proper subpath of any other simple path Simple Paths : [1,2,4,1], [1,3,4,1], [2,4,1,2], [2,4,1,3], [3,4,1,2], [3,4,1,3], [4,1,2,4], [4,1,3,4], [1,2,4], [1,3,4], [2,4,1], [3,4,1], [4,1,2], [4,1,3], [1,2], [1,3], [2,4], [3,4], [4,1], [1], [2], [3], [4] Prime Paths : [2,4,1,2], [2,4,1,3], [1,3,4,1], [1,2,4,1], [3,4,1,2], [4,1,3,4], [4,1,2,4], [3,4,1,3] Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 15

16 Prime Path Coverage A simple, elegant and finite criterion that requires loops to be executed as well as skipped Prime Path Coverage (PPC) : TR contains each prime path in G. Will tour all paths of length 0, 1, That is, it subsumes node and edge coverage PPC does NOT subsume EPC If a node n has an edge to itself, EPC will require [n, n, m] [n, n, m] is not prime Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 16

17 Round Trips Round-Trip Path : A prime path that starts and ends at the same node Simple Round Trip Coverage (SRTC) : TR contains at least one round-trip path for each reachable node in G that begins and ends a round-trip path. Complete Round Trip Coverage (CRTC) : TR contains all roundtrip paths for each reachable node in G. These criteria omit nodes and edges that are not in round trips That is, they do not subsume edge-pair, edge, or node coverage Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 17

18 Prime Path Example The previous example has 38 simple paths Only nine prime paths Prime Paths [1, 2, 3, 4, 7] [1, 2, 3, 5, 7] [1, 2, 3, 5, 6] [1, 3, 4, 7] [1, 3, 5, 7] [1, 3, 5, 6] [6, 5, 7] [6, 5, 6] [5, 6, 5] Execute loop 0 )mes Execute loop once Execute loop more than once Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 18

19 Touring, Sidetrips and Detours Prime paths do not have internal loops test paths might Tour : A test path p tours subpath q if q is a subpath of p Tour With Sidetrips : A test path p tours subpath q with sidetrips iff every edge in q is also in p in the same order The tour can include a sidetrip, as long as it comes back to the same node Tour With Detours : A test path p tours subpath q with detours iff every node in q is also in p in the same order The tour can include a detour from node ni, as long as it comes back to the prime path at a successor of ni Tes)ng, Edi)on 2 (Ch 06) Ammann & OffuA 19

20 Overview A common applica)on of graph criteria is to program source Graph : Usually the control flow graph (CFG) Node coverage : Execute every statement Edge coverage : Execute every branch Loops : Looping structures such as for loops, while loops, etc. Data flow coverage : Augment the CFG defs are statements that assign values to variables uses are statements that use variables Tes)ng, Edi)on 2 (Ch 6) Ammann & OffuA 20

21 Control Flow Graphs A CFG models all execu)ons of a method by describing control structures Nodes : Statements or sequences of statements (basic blocks) Edges : Transfers of control Basic Block : A sequence of statements such that if the first statement is executed, all statements will be (no branches) CFGs are some)mes annotated with extra informa)on branch predicates defs uses Rules for transla)ng statements into graphs Tes)ng, Edi)on 2 (Ch 6) Ammann & OffuA 21

22 CFG : The if Statement if (x < y) { y = 0; 1 x = x + 1; x < y x >= y } y = 0 else x = x { x = y; } 4 x = y if (x < y) { y = 0; x = x + 1; } y = 0 x = x + 1 x < y 2 1 x >= y 3 Tes)ng, Edi)on 2 (Ch 6) Ammann & OffuA 22

23 CFG : The if-return Statement if (x < y) { return; } print (x); return; return x < y 2 1 x >= y 3 print (x) return No edge from node 2 to 3. The return nodes must be distinct. Tes)ng, Edi)on 2 (Ch 6) Ammann & OffuA 23

24 Loops Loops require extra nodes to be added Nodes that do not represent statements or basic blocks Tes)ng, Edi)on 2 (Ch 6) Ammann & OffuA 24

25 CFG : while and for Loops x = 0; while (x < y) { y = f (x, y); x = x + 1; } x = 0 x < y y =f(x,y) x = x + 1 x >= y 4 dummy node for (x = 0; x < y; x++) { y = f (x, y); } implicitly initializes loop y = f (x, y) x = 0 x < y x >= y x = x + 1 Tes)ng, Edi)on 2 (Ch 6) implicitly increments loop Ammann & OffuA 25

26 CFG : do Loop, break and con)nue x = 0; do { y = f (x, y); x = x + 1; } while (x < y); println (y) x = 0 x >= y Tes)ng, Edi)on 2 (Ch 6) y = f (x, y) x = x+1 x < y x = 0; while (x < y) { y = f (x, y); if (y == 0) { break; } else if y < 0) { y = y*2; continue; } x = x + 1; } print (y); 1 x = Ammann & OffuA y =f(x,y) y == 0 4 y < 0 6 x = x + 1 break y = y*2 con)nue

27 CFG : The case (switch) Structure read ( c) ; switch ( c ) { case N : y = 25; break; case Y : y = 50; break; default: y = 0; break; } print (y); y = 25; break; c == N 1 read ( c ); c == Y default y = 50; break; 5 print (y); y = 0; break; Tes)ng, Edi)on 2 (Ch 6) Ammann & OffuA 27

28 Example Control Flow Stats public static void computestats (int [ ] numbers) { int length = numbers.length; double med, var, sd, mean, sum, varsum; sum = 0; for (int i = 0; i < length; i++) { sum += numbers [ i ]; } med = numbers [ length / 2]; mean = sum / (double) length; varsum = 0; for (int i = 0; i < length; i++) { varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean)); } var = varsum / ( length ); sd = Math.sqrt ( var ); } Tes)ng, Edi)on 2 (Ch 6) System.out.println ("length: " + length); System.out.println ("mean: System.out.println ("median: " + mean); " + med); System.out.println ("variance: " + var); System.out.println ("standard deviation: " + sd); Ammann & OffuA Ammann & OffuA 28

29 Control Flow Graph for Stats public static void computestats (int [ ] numbers) { int length = numbers.length; double med, var, sd, mean, sum, varsum; sum = 0; for (int i = 0; i < length; i++) { sum += numbers [ i ]; } med = numbers [ length / 2]; mean = sum / (double) length; i = 0 i >= length } Tes)ng, Edi)on 2 (Ch 6) varsum = 0; i < length for (int i = 0; i < length; i++) { i++ 4 varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean)); } var = varsum / ( length ); sd = Math.sqrt ( var ); System.out.println ("length: " + length); System.out.println ("mean: System.out.println ("median: " + mean); " + med); System.out.println ("variance: " + var); System.out.println ("standard deviation: " + sd); Ammann & OffuA Ammann & OffuA i = 0 i < length i >= length 7 i

30 Control Flow TRs and Test Paths EC 1 Edge Coverage TR A. [ 1, 2 ] B. [ 2, 3 ] C. [ 3, 4 ] D. [ 3, 5 ] E. [ 4, 3 ] F. [ 5, 6 ] G. [ 6, 7 ] H. [ 6, 8 ] I. [ 7, 6 ] Test Path [ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ] 7 8 Tes)ng, Edi)on 2 (Ch 6) Ammann & OffuA 30

31 Control Flow TRs and Test Paths EPC 1 Edge-Pair Coverage Tes)ng, Edi)on 2 (Ch 6) TR A. [ 1, 2, 3 ] B. [ 2, 3, 4 ] C. [ 2, 3, 5 ] D. [ 3, 4, 3 ] E. [ 3, 5, 6 ] F. [ 4, 3, 5 ] G. [ 5, 6, 7 ] H. [ 5, 6, 8 ] I. [ 6, 7, 6 ] J. [ 7, 6, 8 ] K. [ 4, 3, 4 ] L. [ 7, 6, 7 ] Ammann & OffuA Test Paths i. [ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ] ii. [ 1, 2, 3, 5, 6, 8 ] iii. [ 1, 2, 3, 4, 3, 4, 3, 5, 6, 7, 6, 7, 6, 8 ] TP TRs toured sidetrips i A, B, D, E, F, G, I, J C, H ii A, C, E, H iii A, B, D, E, F, G, I, J, K, L C, H 31

32 4 7 Control Flow TRs and Test Paths PPC Tes)ng, Edi)on 2 (Ch 6) TR A. [ 3, 4, 3 ] B. [ 4, 3, 4 ] C. [ 7, 6, 7 ] D. [ 7, 6, 8 ] E. [ 6, 7, 6 ] F. [ 1, 2, 3, 4 ] G. [ 4, 3, 5, 6, 7 ] H. [ 4, 3, 5, 6, 8 ] I. [ 1, 2, 3, 5, 6, 7 ] J. [ 1, 2, 3, 5, 6, 8 ] 8 Prime Path Coverage Ammann & OffuA Test Paths i. [ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ] ii. [ 1, 2, 3, 4, 3, 4, 3, 5, 6, 7, 6, 7, 6, 8 ] iii. [ 1, 2, 3, 4, 3, 5, 6, 8 ] iv. [ 1, 2, 3, 5, 6, 7, 6, 8 ] v. [ 1, 2, 3, 5, 6, 8 ] TP TRs toured sidetrips i A, D, E, F, G H, I, J ii A, B, C, D, E, F, G, H, I, J iii A, F, H J iv D, E, F, I J v J 32

33 CITS5501 So,ware Quality and Tes)ng Logic and Path Coverage From Introduc(on to SoFware Tes(ng, 2 nd ed, Ammann & OffuL

34 Ch. 3 : Logic Coverage Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source FSMs Applied to Specs DNF Source Specs Source Models Design Introduction to Software Testing (Ch 3) Use cases Integ Input Ammann & Offutt 34

35 Covering Logic Expressions (3.1) Logic expressions show up in many situations Covering logic expressions is required by the US Federal Aviation Administration for safety critical software Logical expressions can come from many sources Decisions in programs FSMs and statecharts Requirements Tests are intended to choose some subset of the total number of truth assignments to the expressions Introduction to Software Testing (Ch 3) Ammann & Offutt 35

36 Logic Predicates and Clauses A predicate is an expression that evaluates to a boolean value Predicates can contain boolean variables non-boolean variables that contain >, <, ==, >=, <=,!= boolean function calls Internal structure is created by logical operators the negation operator the and operator the or operator the implication operator the exclusive or operator the equivalence operator A clause is a predicate with no logical operators Introduction to Software Testing (Ch 3) Ammann & Offutt 36

37 Examples (a < b) f (z) D (m >= n*o) Four clauses: (a < b) relational expression f (z) boolean-valued function D boolean variable (m >= n*o) relational expression Most predicates have few clauses It would be nice to quantify that claim! Sources of predicates Decisions in programs Guards in finite state machines Decisions in UML activity graphs Requirements, both formal and informal SQL queries Introduction to Software Testing (Ch 3) Ammann & Offutt 37

38 Testing and Covering Predicates (3.2) We use predicates in testing as follows : Developing a model of the software as one or more predicates Requiring tests to satisfy some combination of clauses Abbreviations: P is the set of predicates p is a single predicate in P C is the set of clauses in P C p is the set of clauses in predicate p c is a single clause in C Introduction to Software Testing (Ch 3) Ammann & Offutt 38

39 Predicate and Clause Coverage The first (and simplest) two criteria require that each predicate and each clause be evaluated to both true and false Predicate Coverage (PC) : For each p in P, TR contains two requirements: p evaluates to true, and p evaluates to false. When predicates come from condi)ons on edges, this is equivalent to edge coverage PC does not evaluate all the clauses, so Clause Coverage (CC) : For each c in C, TR contains two requirements: c evaluates to true, and c evaluates to false. Introduction to Software Testing (Ch 3) Ammann & Offutt 39

40 Predicate Coverage Example ((a < b) D) (m >= n*o) predicate coverage Predicate = true a = 5, b = 10, D = true, m = 1, n = 1, o = 1 = (5 < 10) true (1 >= 1*1) = true true TRUE = true Predicate = false a = 10, b = 5, D = false, m = 1, n = 1, o = 1 = (10 < 5) false (1 >= 1*1) = false false TRUE = false Introduction to Software Testing (Ch 3) Ammann & Offutt 40

41 Clause Coverage Example ((a < b) D) (m >= n*o) Clause coverage (a < b) = true (a < b) = false D = true D = false a = 5, b = 10 a = 10, b = 5 D = true D = false m >= n*o = true m = 1, n = 1, o = 1 m >= n*o = false m = 1, n = 2, o = 2 false cases true cases Two tests 1) a = 5, b = 10, D = true, m = 1, n = 1, o = 1 2) a = 10, b = 5, D = false, m = 1, n = 2, o = 2 Introduction to Software Testing (Ch 3) Ammann & Offutt 41

42 Problems with PC and CC PC does not fully exercise all the clauses, especially in the presence of short circuit evaluation CC does not always ensure PC That is, we can satisfy CC without causing the predicate to be both true and false This is definitely not what we want! The simplest solution is to test all combinations Introduction to Software Testing (Ch 3) Ammann & Offutt 42

43 Combinatorial Coverage CoC requires every possible combination Sometimes called Multiple Condition Coverage Combinatorial Coverage (CoC) : For each p in P, TR has test requirements for the clauses in Cp to evaluate to each possible combination of truth values. a < b D m >= n*o ((a < b) D) (m >= n*o) 1 T T T T 2 T T F F 3 T F T T 4 T F F F 5 F T T T 6 F T F F 7 F F T F 8 F F F F Introduction to Software Testing (Ch 3) Ammann & Offutt 43

44 Combinatorial Coverage This is simple, neat, clean, and comprehensive But quite expensive! 2 N tests, where N is the number of clauses Imprac)cal for predicates with more than 3 or 4 clauses The literature has lots of sugges)ons some confusing The general idea is simple: Test each clause independently from the other clauses Geng the details right is hard What exactly does independently mean? The book presents this idea as making clauses ac(ve Introduction to Software Testing (Ch 3) Ammann & Offutt 44

45 Active Clauses Clause coverage has a weakness : The values do not always make a difference Consider the first test for clause coverage, which caused each clause to be true: (5 < 10) true (1 >= 1*1) Only the first clause counts! To really test the results of a clause, the clause should be the determining factor in the value of the predicate Determination : A clause c i in predicate p, called the major clause, determines p if and only if the values of Introduction to Software Testing (Ch 3) the remaining minor clauses c j are such that changing c i changes the value of p This is considered to make the clause ac(ve Ammann & Offutt 45

46 Determining Predicates P = A B if B = true, p is always true. so if B = false, A determines p. if A = false, B determines p. P = A B if B = false, p is always false. so if B = true, A determines p. if A = true, B determines p. Goal : Find tests for each clause when the clause determines the value of the predicate This is formalized in several criteria that have subtle, but very important, differences Introduction to Software Testing (Ch 3) Ammann & Offutt 46

47 General and Restricted ICC Unlike ACC, the notion of correlation is not relevant ci does not determine p, so cannot correlate with p Predicate coverage is always guaranteed General Inactive Clause Coverage (GICC) : For each p in P and each major clause c i in Cp, choose minor clauses c j, j!= i, so that c i does not determine p. The values chosen for the minor clauses c j do not need to be the same when c i is true as when c i is false, that is, c j (c i = true) = c j (c i = false) for all c j OR c j (c i = true)!= c j (c i = false) for all c j. Restricted Inactive Clause Coverage (RICC) : For each p in P and each major clause c i in Cp, choose minor clauses c j, j!= i, so that c i does not determine p. The values chosen for the minor clauses cj must be the same when c i is true as when c i is false, that is, it is required that c j (c i = true) = c j (c i = false) for all c j. Introduction to Software Testing (Ch 3) Ammann & Offutt 47

48 Logic Coverage Criteria Subsumption Combinatorial Clause Coverage COC Restricted Active Clause Coverage RACC Correlated Active Clause Coverage CACC Restricted Inactive Clause Coverage RICC General Inactive Clause Coverage GICC General Active Clause Coverage GACC Introduction to Software Testing (Ch 3) Clause Coverage CC Predicate Coverage PC Ammann & Offutt 48

49 Making Clauses Determine a Predicate Finding values for minor clauses c j is easy for simple predicates But how to find values for more complicated predicates? Definitional approach: p c=true is predicate p with every occurrence of c replaced by true p c=false is predicate p with every occurrence of c replaced by false To find values for the minor clauses, connect p c=true and p c=false with exclusive OR p c = p c=true p c=false After solving, p c describes exactly the values needed for c to determine p Introduction to Software Testing (Ch 3) Ammann & Offutt 49

50 Examples p = a b p a = p a=true p a=false = (true b) XOR (false b) = true XOR b = b p = a b p a = p a=true p a=false = (true b) (false b) = b false = b Introduction to Software Testing (Ch 3) p = a (b c) p a = p a=true p a=false = (true (b c)) (false (b c)) = true (b c) = (b c) = b c NOT b NOT c means either b or c can be false RACC requires the same choice for both values of a, CACC does not Ammann & Offutt 50

51 Repeated Variables The definitions in this chapter yield the same tests no matter how the predicate is expressed (a b) (c b) == (a c) b (a b) (b c) (a c) Only has 8 possible tests, not 64 Use the simplest form of the predicate, and ignore contradictory truth table assignments Introduction to Software Testing (Ch 3) Ammann & Offutt 51

52 Introduction to Software Testing (Ch 3) A More Subtle Example p = ( a b ) ( a b) p a = p a=true p a=false = ((true b) (true b)) ((false b) (false b)) = (b b) false = true false = true p = ( a b ) ( a b) p b = p b=true p b=false = ((a true) (a true)) ((a false) (a false)) = (a false) (false a) = a a = false a always determines the value of this predicate b never determines the value b is irrelevant! Ammann & Offutt 52

53 Infeasible Test Requirements Consider the predicate: (a > b b > c) c > a (a > b) = true, (b > c) = true, (c > a) = true is infeasible As with graph-based criteria, infeasible test requirements have to be recognized and ignored Recognizing infeasible test requirements is hard, and in general, undecidable Software testing is inexact engineering, not science Introduction to Software Testing (Ch 3) Ammann & Offutt 53

54 Logic Expressions from Source Predicates are derived from decision statements in programs In programs, most predicates have less than four clauses Wise programmers actively strive to keep predicates simple When a predicate only has one clause, COC, ACC, ICC, and CC all collapse to predicate coverage (PC) Applying logic criteria to program source is hard because of reachability and controllability: Reachability : Before applying the criteria on a predicate at a particular statement, we have to get to that statement Controllability : We have to find input values that indirectly assign values to the variables in the predicates Variables in the predicates that are not inputs to the program are called internal variables Introduction to Software Testing (Ch 3) Ammann & Offutt 54

55 Specifications in Software Specifications can be formal or informal Formal specs are usually expressed mathematically Informal specs are usually expressed in natural language Lots of formal languages and informal styles are available Most specification languages include explicit logical expressions, so it is very easy to apply logic coverage criteria Implicit logical expressions in natural-language specifications should be re-written as explicit logical expressions as part of test design You will often find mistakes One of the most common is preconditions Introduction to Software Testing (Ch 3) Ammann & Offutt 55

56 Preconditions Programmers often include preconditions for their methods The preconditions are often expressed in comments in method headers Preconditions can be in javadoc, requires, pre, Example Saving addresses // name must not be empty // state must be valid // zip must be 5 numeric digits // street must not be empty // city must not be empty Conjunctive Normal Form Rewriting to logical expression name!= state in statelist zip >= zip <= street!= city!= Introduction to Software Testing (Ch 3) Ammann & Offutt 56

57 Covering Finite State Machines FSMs are graphs nodes represent state edges represent transitions among states Transitions often have logical expressions as guards or triggers Find a logical expression and cover it As we said: Introduction to Software Testing (Ch 3) Ammann & Offutt 57

58 Example Subway Train secondplatform = right All Doors Open secondplatform= left Le, Doors Open emergencystop overrideopen doorsclear (all three transitions) Right Doors Open trainspeed = 0 platform=left (instation (emergencystop overrideopen)) All Doors Closed trainspeed = 0 platform=right (instation (emergencystop overrideopen)) Introduction to Software Testing (Ch 3) Ammann & Offutt 58

59 Determination of the Predicate trainspeed = 0 platform=left (instation (emergencystop overrideopen)) trainspeed = 0 : platform = left (instation (emergencystop overrideopen)) platform = left : trainspeed = 0 (instation (emergencystop overrideopen)) instation : trainspeed = 0 platform = left ( emergencystop overrideopen) emergencystop : trainspeed = 0 platform = left ( instation overrideopen) overrideopen : trainspeed = 0 platform = left ( instation emergencystop) Introduction to Software Testing (Ch 3) Ammann & Offutt 59