SE 3S03 - Tutorial 10 Department of Computing and Software McMaster University Week of Mar 23, 2015 : Acknowledgments: The material of these slides is based on [1] 1/34
Outline Edge : Path : 2/34
Recall that means using the information about the internal structure of the software to determine test cases. It may also be called unit structural testing This tutorial will demonstrate white box testing on the Assignment 2 code given on the course website. : 3/34
Recall the statement coverage criterion from class: a test set T such that, by executing P for each d in T, each elementary statement of P is executed at least once. P behaves as a function d is an element in the domain D of P elementary statements are usually assignment statements, I/O statements and procedure calls Generally means execute every line of source code at least once Can you think of any weakness in just fulfilling this criterion? : 4/34
Recall the statement coverage criterion from class: a test set T such that, by executing P for each d in T, each elementary statement of P is executed at least once. P behaves as a function d is an element in the domain D of P elementary statements are usually assignment statements, I/O statements and procedure calls Generally means execute every line of source code at least once Can you think of any weakness in just fulfilling this criterion? It fails to measure if statements with a false decision outcome. : 4/34
Simple example: if (x < 6) { x += 1; } How many test cases do we need to make all lines of code execute? : 5/34
Simple example: if (x < 6) { x += 1; } How many test cases do we need to make all lines of code execute? Only 1 case, for example: <x = 4> : 5/34
Simple example: if (x < 6) { x += 1; } How many test cases do we need to make all lines of code execute? Only 1 case, for example: <x = 4> Note that the case where x 6 does not get checked using this criterion. : 5/34
Consider the following change to the example: while (x < 6) { x += 1; } Using the same value of x = 4, does this code have the false condition left unchecked? : 6/34
Consider the following change to the example: while (x < 6) { x += 1; } Using the same value of x = 4, does this code have the false condition left unchecked? No! The false condition executing is the only way the while loop will terminate. : 6/34
Example using If we look at the main() function within the RecordProcessor class, we know that the highlighted conditional statements determine whether or not the body code is executed. : 7/34
Our goal is to ensure every line of code is executed, so we simply have to create test cases that satisfy the different conditions. However, remember we want to try and keep our test set minimal for this testing technique. So, wherever we see nested conditional statements, we just need to have one test that satisfies all to meet the criterion. : 8/34
So we begin by defining a test with inputs inputvalid = false && count = 0 For the exception to run, we need the inputvalid = false (to enter the while loop) and count 4 so let s pick inputvalid = false && count = 4 So to complete statement coverage testing on the above, we only need 2 test cases. : 9/34
Continuing on to the process() function, we follow the same procedure: A test to enter the conditional loop and test the body code needs linenum RecordReader.MAX LINE NUM and endreached = false so let s pick linenum = 8, RecordReader.MAX LINE NUM = 50 and endreached = false : 10/34
To satisfy these three for loops and the if statement, linenum > 0, fieldchars.length > 0, chars.length > 1, and have at least one occcurence of fieldchars[j] == chars[k]. Keeping in mind we want to keep test cases minimal, let s pick linenum = 8, fieldchars = { A, D, W }, chars = { A, D, W }, : 11/34
With our previous test using linenum = 8, we are sure to enter the while loop and the for loop to print the first half of the line. In addition, the same test will allow us to enter the second half of the line when i = 1 and execute the for block since chars.length would equal 3. : 12/34
The code defaults the condition (i + SUBSECTION LENGTH linenum) to this else-block, and we currently do not have a set of inputs to test this. To enter the while-block, linenum must be at least 2. Since (1 + 6 2) let s pick linenum = 2 : 13/34
An example to satisfy these conditions could include, i = 5 and chars.length = 3. In context of testing, we need a test case to have at least 5 lines, and we do - with linenum = 8 and chars as previously defined. : 14/34
Here we need an i such that ((i+1) % (2*6)!= 6), so we can choose i = 1 for that case. Our test condition with linenum = 8 will execute this code. : 15/34
As a quick overview of what we just defined, Looking at each test case and bearing in mind the initial conditions, we can minimize the tests: : 16/34
If we look at the conditions we need for each case, we can see that some tests must be executed together (i.e. in order to test process() we must execute the branch of main() that calls it). So we continue to minimize... Leaving us with a final count of 3 test cases to invoke all the statements within the code: : 17/34
Edge Recall the edge coverage criterion from class: a test set T such that, by executing P for each d in T, each edge of P s control flow graph is traversed at least once. P is a program fragment d is an element in the domain D of P Can you think of any weakness in just fulfilling this criterion? : 18/34
Edge Recall the edge coverage criterion from class: a test set T such that, by executing P for each d in T, each edge of P s control flow graph is traversed at least once. P is a program fragment d is an element in the domain D of P Can you think of any weakness in just fulfilling this criterion? Compound boolean expressions (c 1 c 2 ) used in branching clauses can end up having unchecked constituents that may cause errors later on. : 18/34
Edge Simple example: if (x > 3 y = true) { x += 1; } How many test cases do we need to make sure the if-block is executed, and also that it is skipped? : 19/34
Edge Simple example: if (x > 3 y = true) { x += 1; } How many test cases do we need to make sure the if-block is executed, and also that it is skipped? 2 cases: one where the if-condition is met, and another where it is not. for example: < x = 4, y = true>, <x = 4, y = false> : 19/34
Edge Simple example: if (x > 3 y = true) { x += 1; } How many test cases do we need to make sure the if-block is executed, and also that it is skipped? 2 cases: one where the if-condition is met, and another where it is not. for example: < x = 4, y = true>, <x = 4, y = false> : Note that 2 cases are not covered: one where x 3 and y = false, and another where x 3 and y = false. 19/34
Edge Back to our example Our statement coverage has executed all edges of the control flow graph that have a line of code explicitly written. To ensure edge coverage, we need to go back through the program, and analyze which conditional statements we do not satisfy so we can make sure our tests also take the path where they skip the conditional block of code. : 20/34
Edge Looking at the main function again, we have: Our while loop is always guaranteed to execute the path where the condition is not met- that s how it terminates. Our only worry is the if-statement block here, in which both outcomes have been covered already by our test cases. : 21/34
Edge Let s consider our process function again: Again, this is a while loop, so we re fine. : 22/34
Edge Process function consideration cont d: Our only change we must make is ensure we have a test case in which the character in the third field is not one of the characters we re looking for. Instead of making a new test case, let s just alter the ones we have. : 23/34
Edge Our modified test cases now include the case where the if-condition is not met: : 24/34
Edge Our only concern is making sure a test case doesn t meet the condition- (i + SUBSECTION LENGTH < linenum)... : 25/34
Edge... which we already covered to meet the else condition: : 26/34
Edge The same goes for the next if-else condition, since we needed to execute both blocks of code to meet the statement coverage criterion. : The if (k < chars.length - 1) block does not execute at the end of each printed line, which covers the case when printing **** is skipped. 27/34
Edge Which brings us to the end of our edge coverage consideration, and we finish with the following test cases: : 28/34
Recall the condition coverage criterion from class: a test set T such that, by executing P for each element in T, each edge of P s control flow graph is traversed, and all possible values of the constituents of compound conditions are exercised at least once. Notice that this criterion strengthens the edge coverage criterion. Can you think of any weakness in just fulfilling this criterion? : 29/34
Recall the condition coverage criterion from class: a test set T such that, by executing P for each element in T, each edge of P s control flow graph is traversed, and all possible values of the constituents of compound conditions are exercised at least once. Notice that this criterion strengthens the edge coverage criterion. Can you think of any weakness in just fulfilling this criterion? We may traverse through all edges of the flow graph and possible constituent values, but this does not protect us from errors like division by 0. : 29/34
Simple example: if (x % 2 = 0 y % 2 = 0) { z = x/y; } How many test cases do we need to cover the condition coverage criterion? : 30/34
Simple example: if (x % 2 = 0 y % 2 = 0) { z = x/y; } How many test cases do we need to cover the condition coverage criterion? 4 cases: test each constituent in the compound boolean expression <x is even, y is even>,<x is odd, y is even>,<x is even, y is odd>,<x is odd, y is odd>: for example: < x = 4, y = 6>, <x = 5, y = 6> < x = 4, y = 5>, <x = 5, y = 5> : 30/34
Simple example: if (x % 2 = 0 y % 2 = 0) { z = x/y; } How many test cases do we need to cover the condition coverage criterion? 4 cases: test each constituent in the compound boolean expression <x is even, y is even>,<x is odd, y is even>,<x is even, y is odd>,<x is odd, y is odd>: for example: < x = 4, y = 6>, <x = 5, y = 6> < x = 4, y = 5>, <x = 5, y = 5> : Note that even covering every case here does not protect us from cases like division by 0. 30/34
Path Recall the path coverage criterion from class: a test set T such that, by executing P for each d in T, all paths leading from the initial to the final node of P s control flow graph are traversed. Note that path coverage criterion edge coverage criterion. Can you think of any weakness in just fulfilling this criterion? : 31/34
Path Recall the path coverage criterion from class: a test set T such that, by executing P for each d in T, all paths leading from the initial to the final node of P s control flow graph are traversed. Note that path coverage criterion edge coverage criterion. Can you think of any weakness in just fulfilling this criterion? Generally, this criterion is impractical as in the general case the number of execution paths is very large. This makes traversing through all control paths infeasible. : 31/34
Path Simple example: if (x % 2 = 0 y % 2 = 0) { z = x/y; } How many test cases do we need to cover the condition coverage criterion? : 32/34
Path Simple example: if (x % 2 = 0 y % 2 = 0) { z = x/y; } How many test cases do we need to cover the condition coverage criterion? Many, many cases- we would need to test every possible value of x and y that could make it s way into the program. We could catch the division by 0 error from before. : 32/34
Path Simple example: if (x % 2 = 0 y % 2 = 0) { z = x/y; } How many test cases do we need to cover the condition coverage criterion? Many, many cases- we would need to test every possible value of x and y that could make it s way into the program. We could catch the division by 0 error from before. : Note that this testing could end up taking a very long time as programs grow and become more complex- it s simply infeasible in most cases. 32/34
We covered a brief overview of statement coverage criterion, edge coverage criterion, conditional coverage criterion, and path coverage criterion and noted issues with each. We can see the value in testing programs with multiple techniques. If code is missing some sort of implementation, white box testing won t find it, but black box testing will! : 33/34
References I Appendix References C. Ghezzi, M.Jazayeri, D.Mandrioli; Fundamentals of Software Engineering Prentice Hall PTR, 2002. 34/34