MTAT : Software Testing

Transcription

1 MTAT : Software Testing Lecture 03: White-Box Testing (Textbook Ch. 5) Dietmar Pfahl Spring

2 Lecture Chapter 5 White-box testing techniques (Lab 3)

3 Structure of Lecture 3 Introduction to White-Box Testing Control-Flow Testing Loop Testing Symbolic Execution Data-Flow Testing Lab 3

4 Testing Strategies requirements output input events Black Box Testing White Box Testing

5 Types of Testing system integration module unit Level of detail portability maintainability efficiency usability reliability functionality white box black box Accessibility Characteristics

6 White-Box Testing If-then-else There are many possible paths! 5 20 (~10 14 ) different paths loop < 20x Selective Testing

7 Selective Testing a selected path 2 Major Strategies Control flow testing Data flow testing

8 Code Coverage (Test Coverage) Definition: Measures the extent to which certain code items related to a defined test adequacy criterion have been executed (covered) by running a set of test cases (= test suite) Goal: Define test suites such that they cover as many (disjoint) code items as possible

9 Code Coverage Measure Example Statement Coverage (CV s ) Portion of the statements tested by at least one test case. S CV t s 100% S p S : number of statements tested S t p : total number of statements

10 Code Coverage Analysis Code coverage analysis is the process of: Finding areas of a program not exercised by a set of test cases Creating additional test cases to increase coverage Determining a quantitative measure of code coverage, which is (believed to be) a predictor of code quality Code coverage analyzers automate this process For java: EclEmma (JaCoCo), Clover, etc. Additional aspect of code coverage analysis: Identifying redundant test cases that do not increase coverage

11 Main Classes of Test Adequacy Criteria Control Flow Criteria: Statement, decision (branch), condition, and path coverage are examples of control flow criteria They rely on syntactic characteristics of the program (ignoring the semantics of the program computation) Data Flow Criteria: Require the execution of path segments that connect parts of the code that are intimately connected by the flow of data (-> annotated control flow graph )

12 Structure of Lecture 3 Introduction to White-Box Testing Control-Flow Testing Loop Testing Symbolic Execution Data-Flow Testing Lab 3

13 Control Flow Graph (CFG) MTAT / Lecture 03 / Dietmar Pfahl 2015

14 Control Flow Graph (CFG) empty Blocks (=Nodes): 4 Edges: 4 MTAT / Lecture 03 / Dietmar Pfahl 2015

15 Control Flow Graph (CFG) d 1 is a dummy node e 1 e 2 s 1 s 2 e 3 e 4 s 3 d 1 s 5 e 5 e 6 s 4 entry and exit nodes are dummy nodes e 7 e 8 s 6 Nodes: 8 Edges: 8 empty Blocks: 4 Edges: 4 MTAT / Lecture 03 / Dietmar Pfahl 2015

16 Control Flow Example s 1 d 2 e 3 e 1 e 2 e s 6 3 e d 7 e 5 d 14 3 e 8 e 9 d 1 e 10 d4 e 4 s 4 e 5 e s 12 2 e 13 e 11 If (d1) then { if (d2) then {s1} s2 while (d3) do {s3} } else { if (d4) then { repeat {s4} until (d5) } }

17 Control Flow Example CFG(t) d 1 e 1 e 2 CFG(f) If (d1) then { if (d2) then {s1} s2 If (d1) then { CFG(d1=true) while (d3) do {s3} } } else { else { if (d4) then { CFG(d1=false) repeat {s4} until (c5) } } } }

18 Control Flow Example CFG(t) d 1 e 1 e 2 CFG(f) If (d1) then { if (d2) then {s1} s2 If (d1) then { CFG(d1=true) while (d3) do {s3} } } else { else { if (d4) then { CFG(d1=false) repeat {s4} until (d5) } } } }

19 Control Flow Example CFG(if) d 1 e 1 e 2 d4 If (d1) then { if (d2) then {s1} s2 If (d1) then { CFG(if) s2 e 4 e 10 while (d3) do {s3} } CFG(while) } CFG (while) s 2 e 6 e 9 CFG (repeat) e 14 e 11 else { if (d4) then { repeat {s4} until (c5) } } else { if (d4) then { CFG(repeat) } }

20 Control Flow Example CFG(if) d 1 e 1 e 2 d4 If (d1) then { if (d2) then {s1} s2 If (d1) then { CFG(if) s2 e 4 e 10 while (d3) do {s3} } CFG(while) } CFG (while) s 2 e 6 e 9 CFG (repeat) e 14 e 11 else { if (d4) then { repeat {s4} until (d5) } } else { if (d4) then { CFG(repeat) } }

21 Control Flow Example s 1 d 2 e 3 e 1 e 2 e s 6 3 e d 7 e 5 d 14 3 e 8 e 9 d 1 e 10 d4 e 4 s 4 e 5 e s 12 2 e 13 e 11 If (d1) then { if (d2) then {s1} s2 while (d3) do {s3} } else { if (d4) then { repeat {s4} until (d5) } }

22 Overview of Control Flow Criteria s 1 s 3 d 2 e 3 d 1 e 1 e 2 e 10 d4 e 4 s 4 e 5 e s 12 2 e 13 e 6 e d 7 e 5 d 14 3 e 11 Decision (or Branch) Coverage all edges Condition Coverage Condition/Decision Coverage Multiple Condition Coverage Modified Condition Decision Coverage (MC/DC) Linearly Independent Paths Loop Testing Statement (or Block) Coverage all nodes e 8 e 9

23 Statement Coverage Execute each statement at least once Use tools to monitor execution More practice in Lab 3 A possible concern may be: Dead code

24 Life Insurance Example boolean AccClient(int age; gtype gender) S 1: 2: 3: 4: 5: if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); d 1 = c 1 & c d 2 = c 3 & c 4 In the following assume that the following preconditions have been checked: - Parameter gender is in {female, male} - Parameter age is integer and >= 18 E 5

25 Statement Coverage /1 boolean AccClient(int age; gtype gender) S 1: 2: 3: 4: 5: if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); d 1 = c 1 & c d 2 = c 3 & c 4 Test: AccClient(83, female)->accept AccClient(83, male)->reject E 5 0 %

26 Statement Coverage /2 boolean AccClient(int age; gtype gender) S 1: 2: 3: 4: 5: if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); d 1 = c 1 & c d 2 = c 3 & c 4 Test: AccClient(83, female)->true AccClient(83, male)->reject E 5 40 %

27 Statement Coverage /3 boolean AccClient(int age; gtype gender) S 1: 2: 3: 4: 5: if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); d 1 = c 1 & c d 2 = c 3 & c 4 Test: AccClient(83, female)->true AccClient(83, male)->false E 5 80 %

28 Statement Coverage /4 boolean AccClient(int age; gtype gender) S 1: 2: 3: 4: 5: if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); d 1 = c 1 & c d 2 = c 3 & c 4 Test: AccClient(83, female)->true AccClient(83, male)->false E 5 AccClient(25, male)->true 100 %

29 Same Test Suite but Incorrect Code in Life Insurance Example (1) 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age < 80) return(true); if (gender == male & age < 80) return(true); return(false); 1 failure d 1 = c 1 & c S 3 d 2 = c 3 & c 4 Test: Where is the bug? AccClient(83, female)->false AccClient(83, male)->false AccClient(25, male)->true 80 % E 5

30 Same Test Suite but Incorrect Code in Life Insurance Example (1) 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age < 80) return(true); if (gender == male & age < 80) return(true); return(false); 1 fault triggers 1 failure Test: AccClient(83, female)->false AccClient(83, male)->false AccClient(25, male)->true 1 fault 1 failure 80 % d 1 = c 1 & c 2 2 E 4 1 S d 2 = c 3 & c 4 3 5

31 Same Test Suite but Incorrect Code in Life Insurance Example (2) 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age > 85) return(true); if (gender == male & age < 80) return(true); return(false); 1 fault triggers 1 failure Test: AccClient(83, female)->false AccClient(83, male)->false AccClient(25, male)->true 1 fault 1 failure 80 % d 1 = c 1 & c 2 2 E 4 1 S d 2 = c 3 & c 4 3 5

32 Same Test Suite but Incorrect Code in Life Insurance Example (2) 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age > 80) return(true); if (gender == male & age < 80) return(true); return(false); 2 faults trigger 0 failures Test: AccClient(83, female)->true AccClient(83, male)->false AccClient(25, male)->true 2 faults d 1 = c 1 & c % 2 E 4 1 S d 2 = c 3 & c 4 3 5

33 Statement Coverage : Dead Code? 1: 2: 3: 4: 5: 6: 7: 8: 9: boolean AccClient(int age; gtype gender) if (gender == female){ if (age < 85) return(true); return(false);} if (gender == male){ if (age < 80) return(true); return(false);} return(false); Test: T d 2 = c 2 AccClient(83, female)->true AccClient(83, male)->false AccClient(25, male)->true 3 2 T/F d 1 = c 1 4 E 1 T/F d 4 = c % 8 S T d 3 = c 3 5 9

34 Statement Coverage : Dead Code? 1: 2: 3: 4: 5: 6: 7: 8: 9: boolean AccClient(int age; gtype gender) if (gender == female){ if (age < 85) return(true); return(false);} if (gender == male){ if (age < 80) return(true); return(false);} return(false); Dead code? Test: T d 2 = c 2 AccClient(83, female)->true AccClient(83, male)->false AccClient(25, male)->true 3 2 T/F d 1 = c 1 4 E 1 T/F d 4 = c % 8 S T d 3 = c 3 5 9

35 Statement Coverage : Dead Code? boolean AccClient(int age; gtype gender) S 1: 2: 3: 4: 5: 6: 7: 8: 9: if (gender == female){ if (age < 85) return(true); return(false);} if (gender == male){ if (age < 80) return(true); return(false);} return(false); Test: d 2 = c 2 AccClient(83, female)->true 3 2 d 1 = c 1 1 d 4 = c E 5 d 3 = c 3 9 AccClient(83, male)->false AccClient(25, male)->true 100 %

36 Decision (Branch) Coverage /1 boolean AccClient(int age; gtype gender) S 1: 2: 3: 4: 5: if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); d 1 = c 1 & c d 2 = c 3 & c 4 Test: AccClient(83, female)->accept AccClient(83, male)->reject E 5 0 %

37 Decision (Branch) Coverage /2 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); Test: AccClient(83, female)->true AccClient(83, male)->reject AccClient(25, male)->accept 25 % T d 1 = c 1 & c 2 2 E 4 1 S d 2 = c 3 & c 4 3 5

38 Decision (Branch) Coverage /3 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); Test: AccClient(83, female)->true AccClient(83, male)->false AccClient(25, male)->accept 75 % T/F d 1 = c 1 & c 2 2 E 4 1 S F d 2 = c 3 & c 4 3 5

39 Decision (Branch) Coverage /4 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); Test: AccClient(83, female)->true AccClient(83, male)->false AccClient(25, male)->true 100 % T/F d 1 = c 1 & c 2 2 E 4 1 S T/F d 2 = c 3 & c 4 3 5

40 Condition Coverage Test all conditions (in all predicate nodes): Minimum: Each condition must evaluate at least once Simple: Each condition must evaluate at least once to true and once to false Example of a decision (predicate) with two conditions: A predicate may contain several conditions connected via Boolean operators If (A==female & B<85) then

41 Condition Coverage /1 boolean AccClient(int age; gtype gender) S 1: 2: 3: 4: 5: if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); d 1 = c 1 & c d 2 = c 3 & c 4 Test: AccClient(83, female)->accept AccClient(83, male)->reject E 5 0 %

42 Condition Coverage /2 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); Test: AccClient(83, female)->true AccClient(83, male)->reject T T T d 1 = c 1 & c 2 2 E 4 1 S d 2 = c 3 & c % or 25 %

43 Condition Coverage /3 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); Test: AccClient(83, female)->true AccClient(83, male)->false T/F T/F T d 1 = c 1 & c 2 2 E 4 1 S F T F d 2 = c 3 & c % or 62.5 %

44 Condition Coverage /4 1: 2: 3: 4: 5: boolean AccClient(int age; gtype gender) if (gender == female & age < 85) return(true); if (gender == male & age < 80) return(true); return(false); Test: AccClient(83, female)->true AccClient(83, male)->false AccClient(25, male)->true T/F T/F T d 1 = c 1 & c 2 2 E 100 % or 75 % 4 1 S F/T T F/T d 2 = c 3 & c 4 3 5

45 Advanced Condition Coverage Condition/Decision Coverage (C/DC) as DC plus: every condition in each decision is tested in each possible outcome Modified Condition/Decision coverage (MC/DC) as above plus, every condition shown to independently affect a decision outcome (by varying that condition only) Def: A condition independently affects a decision when, by flipping that condition s outcome and holding all the others fixed, the decision outcome changes this criterion was created at Boeing and is required for aviation software according to RCTA/DO-178B Multiple-Condition Coverage (M-CC) all possible combinations of condition outcomes within each decision is checked

46 CC, DC, C/DC, M-CC, MC/DC Examples If (A==fem & B<85) Minimum and Simple Condition (CC): (TF) A = fem; B = 200 (D: False) [(FT) A = male; B = 80 (D: False)] Decision (DC): (TT) A = fem; B = 80 (D: True) (FT) A = male; B = 80 (D: False) Condition/Decision (C/DC): (TT) A = fem; B = 80 (D: True) (FF) A = male; B = 200 (D: False) Multiple Condition (M-CC): (TT) A = fem; B = 80 (D: True) (FT) A = male; B = 80 (D: False) (TF) A = fem; B = 200 (D: False) (FF) A = male; B = 200 (D: False) Modified Condition/Decision (MC/DC): (TT) A = fem; B = 80 (D: True) (FT) A = male; B = 80 (D: False) (TF) A = fem; B = 200 (D: False)

47 Modified Condition/Decision (MC/DC) If (A=fem and B<85) then TC A B Dec 1 T (fem) T (80) T 2 F (male) T (80) F 3 T (fem) F (200) F 4 F (male) F (200) F TC1+TC2: change in A -> Dec changed TC1+TC3: change in B -> Dec changed All other TC combinations in which only one condition outcome changes don t have an effect on the decision outcome. Result: only TC1, TC2, and TC3 needed Multiple Condition: (TT) A = fem; B = 80 (D: True) (FT) A = male; B = 80 (D: False) (TF) A = fem; B = 200 (D: False) (FF) A = male; B = 200 (D: False) Modified Condition/Decision: (TT) A = fem; B = 80 (D: True) (FT) A = male; B = 80 (D: False) (TF) A = fem; B = 200 (D: False)

48 Independent Path Coverage McCabe cyclomatic complexity estimates number of test cases needed The number of independent paths needed to cover all simple paths at least once in a program Visualize by drawing a CFG CC = #(edges) #(nodes) + 2 CC = #(decisions) + 1 while-loop if-then-else case-of

49 Independent Paths Coverage Example Independent Paths Coverage Requires that a minimum set of linearly independent paths through the control flow-graph be executed This test strategy is the rationale for McCabe s cyclomatic number (McCabe 1976) which is equal to the number of test cases required to satisfy the strategy. Cyclomatic Complexity = = 6

50 Independent Paths Coverage Example Edges: Path1: Path2: Path3: Path4: Path5: Path6:

51 Independent Paths Coverage Example Edges: Why no need to cover Path7??? Path7: Because it equals Path1+Path2-Path3!!! Path1: Path2: P1+P2: Path3: P3:

52 Independent Paths Coverage Example Edges: Why no need to cover Path7??? Path7: Because it equals Path1+Path2-Path3!!! Path1: Path2: P1+P2: Path3: P3:

53 Control-Flow Coverage Relationships Subsumption: a criterion C1 subsumes another criterion C2, if any test set {T} that satisfies C1 also satisfies C2 Multiple Condition MC/DC C/DC Condition Decision All Path Linearly Indep. Path Statement

54 Structure of Lecture 3 Introduction to White-Box Testing Control-Flow Testing Loop Testing Symbolic Execution Data-Flow Testing Lab 3

55 Loop Testing simple loop nested loops concatenated loops unstructured loops MTAT / Lecture 03 / Dietmar Pfahl 2015

56 Loop Testing: Simple Loops Minimum conditions - simple loops 1. skip the loop entirely 2. only one pass through the loop 3. two passes through the loop 4. m passes through the loop m < n 5. set loop counter to (n-1), n and (n+1): passes twice through the loop and once not where n is the maximum number of allowable passes MTAT / Lecture 03 / Dietmar Pfahl 2015

57 Nested Loops Extend simple loop testing Reduce the number of tests: start at the innermost loop; set all other loops to minimum values conduct simple loop test; add out of range or excluded values work outwards while keeping inner nested loops to typical values continue until all loops have been tested MTAT / Lecture 03 / Dietmar Pfahl 2015

58 Structure of Lecture 3 Introduction to White-Box Testing Control-Flow Testing Loop Testing Symbolic Execution Data-Flow Testing Lab 3

59 How to find Test Cases? Analyse requirements yielding an outcome at each predicate node contained in a path Consider all requirements together Guess a value that will satisfy these requirements --- Can we improve on this?

60 Symbolic Execution How to find test inputs to exercise a path? Need to make a choice at each predicate node Give a symbolic value to each variable Walk the path collecting requirements on symbolic input Then have a set of (in)equalities to solve Example: Find test cases for each path by symbolic execution

61 Example from Black-Box Testing Magic Function M M (x, y) result with x, y: int (32 bit) 1st if = true: x <= 100 2nd if = true: y <= 100 3rd if = true: x + y = 200 if ( x <= 0 ) if ( y <= 0 ) if ( x + y == 0 ) crash();... (if_1) && (if_2) && (if_3) -> M (100, 100) -> crash() If_1 If_2 If_3 crash

62 Program CFG Program finds the smallest factor of a prime decomposition of a given number smallest(int p) (*p>2*) { int q = 2; while(p mod q > 0 AND q < sqrt p) do q := q+1 ; if (p mod q = 0) then print(q, is factor ) else print(p, is prime ) ; } p. Cyclomatic Complexity: v(g) = = = 3 Sets of linearly independent paths: Example Set 1: Example Set 2:

63 Symbolic Execution Tree p: x PC: x>2 PC: path condition Solve path conditions Test input!

64 Symbolic Execution Tree p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition Solve path conditions Test input!

65 Symbolic Execution Tree p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) while-predicate = false (don t enter while-loop) Solve path conditions Test input!

66 Symbolic Execution Tree p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) while-predicate = false (don t enter while-loop) Explanation of PC: Solve path conditions Test input! while-predicate = false can happen if: - p mod q = 0 -> x mod 2 = 0 -> x = 2 * n or - sqrt(p) <= q -> sqrt(x) <= 2 -> x <= 4

67 Symbolic Execution Tree p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) while-predicate = false (don t enter while-loop) Solve path conditions Test input! p: x is prime q: 2 PC: x > 2 and (2 sqrt(x) and x = 2*n + 1) if-predicate = false (take the left branch)

68 Symbolic Execution Tree p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition Solve path conditions Test input! p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) p: x is prime q: 2 PC: x > 2 and (2 sqrt(x) and x = 2*n + 1) Explanation of PC: if-predicate = false happens if: - p mod q > 0 -> - x mod 2 > 0 -> - x mod 2 = 1 -> - x = 2*n + 1

69 Symbolic Execution Tree p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition Solve path conditions Test input! p: x is prime q: 2 PC: x > 2 and (2 sqrt(x) and x = 2*n + 1) sqrt(x) <= 2 x <= 4 2*n + 1 <= 4 n = 1 x = 3

70 Symbolic Execution Tree p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition Solve path conditions Test input! p: x is prime q: 2 PC: x > 2 and (2 sqrt(x) and x = 2*n + 1) sqrt(x) <= 2 x <= 4 2*n + 1 <= 4 n = 1 x = 3 [also satisfies PC: x > 2]

71 Symbolic Execution Tree p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) while-predicate = false (don t enter while-loop) Solve path conditions Test input!

72 Symbolic Execution Tree p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) while-predicate = false (don t enter while-loop) Solve path conditions Test input! if-predicate = true (take the right branch) p: x q: 2 and factor PC: x > 2 and x = 2*n

73 Symbolic Execution Tree p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition Solve path conditions Test input! Explanation of PC: if-predicate = true happens if: - p mod q = 0 -> - x mod 2 = 0 -> - x = 2*n p: x q: 2 and factor PC: x > 2 and x = 2*n

74 Symbolic Execution Tree p: x q: 2 PC: x > 2 and (2 sqrt(x) or x = 2*n) p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition Solve path conditions Test input! x > 2 and x = 2*n x in {2, 4, 6,...} p: x q: 2 and factor PC: x > 2 and x = 2*n

75 Symbolic Execution Tree p: x PC: x>2 p: x q: 2 PC: x>2 1-2 PC: path condition while-predicate = true (enter while-loop) p: x q: 2 PC: x > 2 and 2 < sqrt(x) and x <> 2*n Continue as voluntary homework... Solve path conditions Test input!

76 Structure of Lecture 3 Introduction to White-Box Testing Control-Flow Testing Loop Testing Symbolic Execution Data-Flow Testing Lab 3

77 Data Flow Testing Identifies paths in the program that go from the assignment of a value to a variable to the use of such variable, to make sure that the variable is properly used. X:=14;.. Y:= X-3;

78 Data Flow Testing Motivation Middle ground in structural testing Node (=statement) and edge (=branch) coverage don t test interactions between statements All-path testing is infeasible Need a coverage criterion that is stronger than branch testing but feasible Intuition: Statements interact through data flow Value computed in one statement, used in another Bad value computation revealed only when it is used

79 Data Flow Testing Definitions Def assigned or changed Uses utilized (not changed) C-use (Computation) e.g. righthand side of an assignment, an index of an array, parameter of a function. P-use (Predicate) branching the execution flow, e.g. in an if statement, while statement, for statement. [0] bool AccClient(int age; gtype gender) [2] bool accept = false [1] if (gender==female & age<85) [2] accept = true; [3] if (gender==male & age<80) [4] accept = true; [5] return accept Def(2) = {accept} 2 Def(4) = {accept} Def(0) = {age, gender, accept} P-use(1) = {age, gender} P-use(3) = {age, gender} C-use(5) = {accept} 5

80 Data Flow Testing Criteria All def-use paths requires that each simple (i.e., traversing a loop at most once) definition-clear path from a definition of a variable to its use is executed All uses paths requires that for each definition-use pair of a variable at least one simple definition-clear path is executed All definitions paths... requires that at least one path from the definition of a variable to one of its uses is executed

81 Data Flow Testing Example [0] bool AccClient(int age; gtype gender) [2] bool accept = false [1] if (gender==female & age<85) [2] accept = true; [3] if (gender==male & age<80) [4] accept = true; [5] return accept Test cases needed: AccClient() is executed Considering age, there are two DU paths: (a)[0]-[1] (b)[0]-[3] Test cases: AccClient(*, *)-> *

82 Data Flow Testing Example [0] bool AccClient(int age; gtype gender) [2] bool accept = false [1] if (gender==female & age<85) [2] accept = true; [3] if (gender==male & age<80) [4] accept = true; [5] return accept Test cases needed: AccClient() is executed Considering gender, there are two DU paths: (a)[0]-[1] (b)[0]-[3] Test cases: AccClient(*, *)-> *

83 Data Flow Testing Example [0] bool AccClient(int age; gtype gender) [2] bool accept = false [1] if (gender==female & age<85) [2] accept = true; [3] if (gender==male & age<80) [4] accept = true; [5] return accept Test cases needed: (a) AccClient() is executed and if[1] and if[3] are false (b) AccClient() is executed and if[1] is true and if[3] is false (c) AccClient() is executed and if[3] is true Considering accept, there are three DU paths: (a)[0]-[5] (b)[2]-[5] (c)[4]-[5] Test cases: (a) AccClient(*, 85)-> false (b) AccClient(f, 80)-> true (c) AccClient(m, 80)-> true

84 DF-Testing for Web Applications

85 Data Flow Criteria All c-uses All defs All p-uses Weaker All c-uses, some p-uses All p-uses, some c-uses All uses All def-use paths Stronger # tests

86 Data Flow Criteria All branches All c-uses All defs All p-uses Weaker All c-uses, some p-uses All p-uses, some c-uses All uses All def-use paths Stronger # tests All paths

87 Why is white-box testing not enough? Missing features Missing code Different states of the software Infinite amount of different paths through the software Different paths can reveal different defects Variations of perspectives Control flow/data flow Input/Output behaviour Test data generation Quality attributes Performance Robustness Reliability Usability

88 Recommended Textbook Exercises Chapter 5 2, 5, 6, 9, 10, 11, 14

89 Structure of Lecture 3 Introduction to White-Box Testing Control-Flow Testing Loop Testing Symbolic Execution Data-Flow Testing Lab 3

90 Lab 3 White-Box Testing Lab 3 (week 29: Mar 14 - Mar 17) - White-Box Testing (10%) Lab 3 Instructions Lab 3 Sample Code Submission Deadlines: Monday Lab: Sunday, 20 Mar, 23:59 Tuesday Labs: Monday, 21 Mar, 23:59 Wednesday Labs: Tuesday, 22 Mar, 23:59 Thursday Labs: Wednesday, 23 Mar, 23:59 Instructions Code Penalties apply for late delivery: 50% penalty, if submitted up to 24 hours late; 100 penalty, if submitted more than 24 hours late MTAT / Lecture 02 / Dietmar Pfahl 2015

91 Lab 3 White-Box Testing (cont d) Instructions Code Coverage Criteria: - Statement (Instruction) - Branch (Decision) Tool: Eclipse Plugin Control-Flow Graph Set of 10+ Test Cases 1 Set of 15+ Test Cases 2 Code Code Test Report 1 & Test Coverage 1a + 1b Test Report 2 & Test Coverage 2a + 2b MTAT / Lecture 02 / Dietmar Pfahl 2015

92 Next 2 Weeks Lab 3: White-Box Testing Lecture 4: Mutation Testing Static Testing (Inspection/Review) Defect Estimation In addition to do: Read textbook chapter 5 (available via OIS) MTAT / Lecture 02 / Dietmar Pfahl 2015