White-Box Testing Techniques III

Size: px

Start display at page:

Download "White-Box Testing Techniques III"

Stephen Allison
5 years ago
Views:

1 White-Box Testing Techniques III Software Testing and Verification Lecture 9 Prepared by Stephen M. Thebaut, Ph.D. University of Florida

2 White-Box Testing Topics Logic coverage (lecture I) Dataflow coverage (lecture II) Path conditions and symbolic evaluation (lecture III) Other white-box testing strategies (e.g., fault-based testing ) (lecture IV)

3 Path Conditions With a little luck, at least some whitebox coverage goals will have been met by executing test cases designed using black-box strategies. (How would you know if this were the case or not?) Designing additional test cases for this purpose involves identifying inputs that will cause given program paths to be executed. This can be difficult...

4 Path Conditions (cont d) To cause a path to be executed requires that the test case satisfy the path condition. For a given path, the PATH CONDITION is the conjunction of branch predicates that are required to hold for all the branches along the path to be taken.

5 Consider an example (1) input(a,b) if (A>0) then (2) Z := A (3) Z := 0 if (B>0) then (4) Z := Z+B end_if (5) output(z) 3 F F 1 5 A>0 T 2 B>0 T 4 What is the path condition for path <1,2,5>?

6 Consider ANOTHER example (1) input(a,b) if (A>B) then (2) B := B*B end_if if (B<0) then (3) Z := A (4) Z := B (5) output(z) 4 F 1 A>B T F 2 5 B<0 T 3 What is the path condition for path <1,2,3,5>?

7 Conclusions To be useful, path conditions should be expressed in terms that reflect relevant state changes along the path. A path is INFEASIBLE if its path condition reduces to FALSE. Question: if a path is infeasible, does this imply the presence of dead code?

8 Symbolic Evaluation Symbolic evaluation provides a technique for systematically tracking state changes for the purpose of expressing path conditions in useful terms.

9 Notation A variable A will have a succession of symbolic values, A 0, A 1, A 2,..., as a path is traversed. Subscripts refer to the number of the previous program statement (or block of statements) executed, so some numbers may be skipped. With loops, statements may be executed more than once, so double subscripts will be used when necessary.

10 Notation (cont d) At each statement, the same notation is used to represent program variables regardless of path, but the symbolic values will usually be different. For example: The notation used for the symbolic value of variable X immediately after executing statement 5 is always denoted X 5 ; however, the symbolic value itself depends on which execution path is taken to statement 5.

11 Example 1 T if (X<=0) or (Y<=0) then (1) X := X**2 Y := Y**2 (2) X := X+1 Y := Y+1 T if (X<1) or (Y<1) then (3) X := X+1 Y := Y+1 (4) X := X-1 Y := Y-1 There are 4 paths. PATH T,T 2 (1) X 1 = X 0 2 Y 1 = Y 0 (3) X 3 = X = X Y 3 = Y = Y

12 Example 1 (cont d) T if (X<=0) or (Y<=0) then (1) X := X**2 Y := Y**2 (2) X := X+1 Y := Y+1 F if (X<1) or (Y<1) then (3) X := X+1 Y := Y+1 (4) X := X-1 Y := Y-1 There are 4 paths. PATH T,F 2 (1) X 1 = X 0 2 Y 1 = Y 0 (4) X 4 = X 1-1 = X Y 4 = Y 1-1 = Y 2 0-1

13 Example 1 (cont d) F if (X<=0) or (Y<=0) then (1) X := X**2 Y := Y**2 (2) X := X+1 Y := Y+1 T if (X<1) or (Y<1) then (3) X := X+1 Y := Y+1 (4) X := X-1 Y := Y-1 There are 4 paths. PATH F,T (2) X 2 = X Y 2 = Y (3) X 3 = X = X Y 3 = Y = Y 0 + 2

14 Example 1 (cont d) F if (X<=0) or (Y<=0) then (1) X := X**2 Y := Y**2 (2) X := X+1 Y := Y+1 F if (X<1) or (Y<1) then (3) X := X+1 Y := Y+1 (4) X := X-1 Y := Y-1 There are 4 paths. PATH F,F (2) X 2 = X Y 2 = Y (4) X 4 = X 2-1 = X 0 Y 4 = Y 2-1 = Y 0

15 Path Conditions Revisited Having symbolically evaluated the program variables along a path, we can now symbolically represent the branch predicates that are required to hold in order for the path to be traversed. The symbolic variable values used in each branch predicate are the values which the variables have when the branch predicate is encountered.

16 Path Conditions Revisited To simplify the task of identifying inputs that will cause the path to be executed, path conditions should be expressed in terms of the initial symbolic values of variables.

17 Example 1 (revisited) T if (X<=0) or (Y<=0) then (1) X := X**2 Y := Y**2 (2) X := X+1 Y := Y+1 T if (X<1) or (Y<1) then (3) X := X+1 Y := Y+1 (4) X := X-1 Y := Y-1 PATH T,T Variable values: X 1 = X 0 2 Y 1 = Y 0 2 X 3 = X Y 3 = Y Path Condition: = ((X 0 0) or (Y 0 0)) and ((X 1 < 1) or (Y 1 < 1)) = ((X 0 0) or (Y 0 0)) and ((X 0 2 < 1) or (Y 0 2 < 1)) = ((X 0 0) or (Y 0 0) ) and ((-1 < X 0 < 1) or (-1 < Y 0 < 1))

18 Example 1 (revisited) T if (X<=0) or (Y<=0) then (1) X := X**2 Y := Y**2 (2) X := X+1 Y := Y+1 F if (X<1) or (Y<1) then (3) X := X+1 Y := Y+1 (4) X := X-1 Y := Y-1 PATH T,F Variable values: X 1 = X 0 2 Y 1 = Y 0 2 X 4 = X Y 4 = Y Path Condition: = ((X 0 0) or (Y 0 0)) and ((X 1 1) and (Y 1 1)) = ((X 0 0) or (Y 0 0)) and ((X 0 2 1) and (Y 0 2 1)) = ((X 0 0) or (Y 0 0)) and ((X 0-1) or (X 0 1)) and ((Y 0-1) or (Y 0 1))

19 Example 1 (revisited) F if (X<=0) or (Y<=0) then (1) X := X**2 Y := Y**2 (2) X := X+1 Y := Y+1 T if (X<1) or (Y<1) then (3) X := X+1 Y := Y+1 (4) X := X-1 Y := Y-1 PATH F,T Variable values: X 2 = X Y 2 = Y X 3 = X Y 3 = Y Path Condition: = ((X 0 > 0) and (Y 0 > 0)) and ((X 2 < 1) or (Y 2 < 1)) = ((X 0 > 0) and (Y 0 > 0)) and ((X < 1) or (Y < 1)) = ((X 0 > 0) and (Y 0 > 0)) and ((X 0 < 0) or (Y 0 < 0)) = FALSE

20 Example 1 (revisited) F if (X<=0) or (Y<=0) then (1) X := X**2 Y := Y**2 (2) X := X+1 Y := Y+1 F if (X<1) or (Y<1) then (3) X := X+1 Y := Y+1 (4) X := X-1 Y := Y-1 PATH F,F Variable values: X 2 = X Y 2 = Y X 4 = X 0 Y 4 = Y 0 Path Condition: = ((X 0 > 0) and (Y 0 > 0)) and ((X 2 1) and (Y 2 1)) = ((X 0 > 0) and (Y 0 > 0)) and ((X ) and (Y )) = ((X 0 > 0) and (Y 0 > 0)) and ((X 0 0) and (Y 0 0)) = (X 0 > 0) and (Y 0 > 0)

21 Summary of Path Conditions TT: ((X 0 0) or (Y 0 0) ) and ((-1 < X 0 < 1) or (-1 < Y 0 < 1)) TF: ((X 0 0) or (Y 0 0)) and ((X 0-1) or (X 0 1)) and ((Y 0-1) or (Y 0 1)) FT: FALSE FF: (X 0 > 0) and (Y 0 > 0) The path domains in the (X 0,Y 0 ) plane may also be depicted graphically.

22 Graph of Path Domains Y 0 TF Y 0 = 1 FF TT X 0 Y 0 = -1 X 0 = -1 X 0 = 1

23 Incremental Generation of Path Conditions Path conditions can also be generated incrementally, by considering the branches taken in a piecemeal fashion.

24 Example 2 if (B > A+1) then (1) A := A+1 (2) B := B-1 if (B > -A+2) then (3) A := A+2 (4) B := B+1 if (B <= 0) then... T PARTIAL PATH T Variable values: A 0, B 0 Partial path condition: (B 0 > A 0 +1)

25 Example 2 if (B > A+1) then (1) A := A+1 (2) B := B-1 if (B > -A+2) then (3) A := A+2 (4) B := B+1 if (B <= 0) then... F PARTIAL PATH F Variable values: A 0, B 0 Partial path condition: (B 0 A 0 +1)

26 Partial Path Domains B 0 T A 0 B 0 = A F

27 Example 2 (cont d) if (B > A+1) then (1) A := A+1 (2) B := B-1 if (B > -A+2) then (3) A := A+2 (4) B := B+1 if (B <= 0) then... T T PARTIAL PATH TT Variable values: A 1 = A 0 +1 B 1 = B 0 Partial path condition: (B 0 > A 0 +1) and (B 1 > -A 1 +2) = (B 0 > A 0 +1) and (B 0 > -A 0 +1)

28 Example 2 (cont d) if (B > A+1) then (1) A := A+1 (2) B := B-1 if (B > -A+2) then (3) A := A+2 (4) B := B+1 if (B <= 0) then... T F PARTIAL PATH TF Variable values: A 1 = A 0 +1 B 1 = B 0 Partial path condition: (B 0 > A 0 +1) and (B 1 -A 1 +2) = (B 0 > A 0 +1) and (B 0 -A 0 +1)

29 Example 2 (cont d) if (B > A+1) then (1) A := A+1 (2) B := B-1 if (B > -A+2) then (3) A := A+2 (4) B := B+1 if (B <= 0) then... F T PARTIAL PATH FT Variable values: A 2 = A 0 B 2 = B 0-1 Partial path condition: (B 0 A 0 +1) and (B 2 > -A 2 +2) = (B 0 A 0 +1) and (B 0 > -A 0 +3)

30 Example 2 (cont d) if (B > A+1) then (1) A := A+1 (2) B := B-1 if (B > -A+2) then (3) A := A+2 (4) B := B+1 if (B <= 0) then... F F PARTIAL PATH FF Variable values: A 2 = A 0 B 2 = B 0-1 Partial path condition: (B 0 A 0 +1) and (B 2 -A 2 +2) = (B 0 A 0 +1) and (B 0 -A 0 +3)

Partial Path Domains (cont d) B 0 = -A 0 + 1

31 Partial Path Domains (cont d) B 0 = -A TF B 0 TT FT A 0 B 0 = A FF B 0 = -A 0 + 3

32 Example 2 (cont d) if (B > A+1) then (1) A := A+1 (2) B := B-1 if (B > -A+2) then (3) A := A+2 (4) B := B+1 if (B <= 0) then... T T T PARTIAL PATH TTT Variable values: A 1 = A 0 +1 A 3 = A = A B 3 = B 1 = B 0 Partial path condition: (B 0 > A 0 +1) and (B 0 > -A 0 +1) and (B 3 0) = (B 0 0) = FALSE

33 Loops In general, the simplification of path conditions involving loops is very difficult. The simplest form that is obtainable for N iterations of a loop may involve N terms. N iterations = While loop body executes N-1 times Repeat_Until loop body executes N times

34 Example 3 (1) C := 0 while (X>=Y) do (2) X := X-Y C := C+1 end_while Variable values: (Y does not change) (1) X 1 = X 0 C 1 = 0 (2,1) X 2,1 = X 1 Y 1 = X 0 Y 0 C 2,1 = C = 1 (2,2) X 2,2 = X 2,1 Y 2,1 = (X 0 Y 0 ) - Y 0 = X 0 2Y 0 C 2,2 = C 2,1 + 1 = 2

35 Example 3 (cont d) (1) C := 0 while (X>=Y) do (2) X := X-Y C := C+1 end_while Variable values: (cont d) (2,2) X 2,2 = X 0 2Y 0 C 2,2 = 2 (2,3) X 2,3 = X 2,2 Y 2,2 = (X 0 2Y 0 ) - Y 0 = X 0 3Y 0 C 2,3 = C 2,2 + 1 = 3 (2,N) X 2,N = X 0 NY 0 C 2,N = N

36 Example 3 (cont d) Path Condition for Path F: (loop body executed 0 times) (X 1 < Y 1 ) = X 0 < Y 0 Path Condition for Path T,F: (loop body executed 1 time) (X 1 Y 1 ) and (X 2,1 < Y 2,1 ) = (X 0 Y 0 ) and (X 0 - Y 0 < Y 0 ) = (X 0 Y 0 ) and (X 0 < 2Y 0 ) = Y 0 X 0 < 2Y 0 (1) C := 0 while (X>=Y) do (2) X := X-Y C := C+1 end_while Variable values: X 1 = X 0 C 1 = 0 X 2,N = X 0 NY 0 C 2,N = N

37 Example 3 (cont d) Variable values: X 1 = X 0 C 1 = 0 X 2,N = X 0 NY 0 C 2,N = N (1) C := 0 while (X>=Y) do (2) X := X-Y C := C+1 end_while Path Condition for Path T,T,F: (loop body executed 2 times) (X 1 Y 1 ) and (X 2,1 Y 2,1 ) and (X 2,2 < Y 2,2 ) = (X 0 Y 0 ) and (X 0 - Y 0 Y 0 ) and (X 0-2Y 0 < Y 0 ) = (X 0 Y 0 ) and (X 0 2Y 0 ) and (X 0 < 3Y 0 ) = (X 0 Y 0 ) and (2Y 0 X 0 < 3Y 0 ) = 2Y 0 X 0 < 3Y 0 since (2Y 0 < 3Y 0 ) => Y 0 > 0 => (X 0 2Y 0 => X 0 Y 0 )

38 Example 3 (cont d) Path Condition for N>1 iterations of the loop: (X 1 Y 1 ) and (X 2,1 Y 2,1 ) and and (X 2,N-1 Y 2,N-1 ) and (X 2,N < Y 2,N ) It can be proven by induction that this condition may be expressed in CLOSED FORM as: (N-1)Y 0 X 0 < NY 0 with the implied (eventual termination) condition that Y 0 > 0.

39 Path Conditions & Symbolic Evaluation Summary To cause a path to be executed requires that the test case satisfy its path condition. The path condition is the conjunction of branch predicates that are required to hold for all the branches along the path. In general, the simplification of path conditions for loops is very difficult. (The simplest form that is obtainable for N iterations of a loop may involve N terms.)

40 Path Conditions & Symbolic Evaluation Summary (cont d) A path is infeasible if its path condition reduces to FALSE. Symbolic evaluation provides a systematic method for expressing path conditions in terms of the initial symbolic values of program variables.

41 Path Conditions & Symbolic Evaluation Summary (cont d) This simplifies the task of identifying inputs that will cause the path to be executed. But this satisfiability problem can be very difficult to solve

42 Exactly HOW Difficult? Given a Boolean expression E, decide if there is some assignment to the variables in E such that E will be true. This was the first problem shown to be NP-complete!

43 White-Box Testing Techniques III Software Testing and Verification Lecture 9 Prepared by Stephen M. Thebaut, Ph.D. University of Florida

White-Box Testing Techniques II

White-Box Testing Techniques II Software Testing and Verification Lecture 8 Prepared by Stephen M. Thebaut, Ph.D. University of Florida White-Box Testing Topics Logic coverage (lecture I) Dataflow coverage