Software Testing. 1. Testing is the process of demonstrating that errors are not present.

Transcription

1

2 What is Testing? Software Testing Many people understand many definitions of testing :. Testing is the process of demonstrating that errors are not present.. The purpose of testing is to show that a program performs its intended functions correctly. 3. Testing is the process of establishing confidence that a program does what it is supposed to do. These definitions are incorrect.

3 A more appropriate definition is: Testing is the process of executing a program with the intent of finding errors. 3

4 Why should We Test? Although software testing is itself an expensive activity, yet launching of software without testing may lead to cost potentially much higher than that of testing, specially in systems where human safety is involved. In the software life cycle the earlier the errors are discovered and removed, the lower is the cost of their removal. 4

5 Who should Do the Testing? o Testing requires the developers to find errors from their software. o It is difficult for software developer to point out errors from own creations. o Many organisations have made a distinction between development and testing phase by making different people responsible for each phase. 5

6 What should We Test? We should test the program s responses to every possible input. It means, we should test for all valid and invalid inputs. Suppose a program requires two 8 bit integers as inputs. Total possible combinations are 8 x 8. If only one second it required to execute one set of inputs, it may take 8 hours to test all combinations. Practically, inputs are more than two and size is also more than 8 bits. We have also not considered invalid inputs where so many combinations are possible. Hence, complete testing is just not possible, although, we may wish to do so. 6

7 Fig. : Control flow graph 7

8 The number of paths in the example of Fig. are 0 4 or 00 trillions. It is computed from ; where 5 is the number of paths through the loop body. If only 5 minutes are required to test one test path, it may take approximately one billion years to execute every path. 8

9 Some Terminologies Software Testing Error, Mistake, Bug, Fault and Failure People make errors. A good synonym is mistake. This may be a syntax error or misunderstanding of specifications. Sometimes, there are logical errors. When developers make mistakes while coding, we call these mistakes bugs. A fault is the representation of an error, where representation is the mode of expression, such as narrative text, data flow diagrams, ER diagrams, source code etc. Defect is a good synonym for fault. A failure occurs when a fault executes. A particular fault may cause different failures, depending on how it has been exercised. 9

10 Test, Test Case and Test Suite Test and Test case terms are used interchangeably. In practice, both are same and are treated as synonyms. Test case describes an input description and an expected output description. Test Case ID Section-I (Before Execution) Purpose : Pre condition: (If any) Inputs: Section-II (After Execution) Execution History: Result: If fails, any possible reason (Optional); Expected Outputs: Post conditions: Written by: Date: Any other observation: Any suggestion: Run by: Date: Fig. : Test case template The set of test cases is called a test suite. Hence any combination of test cases may generate a test suite. 0

11 Verification and Validation Verification is the process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. Validation is the process of evaluating a system or component during or at the end of development process to determine whether it satisfies the specified requirements. Testing= Verification+Validation

12 Alpha, Beta and Acceptance Testing The term Acceptance Testing is used when the software is developed for a specific customer. A series of tests are conducted to enable the customer to validate all requirements. These tests are conducted by the end user / customer and may range from adhoc tests to well planned systematic series of tests. The terms alpha and beta testing are used when the software is developed as a product for anonymous customers. Alpha Tests are conducted at the developer s site by some potential customers. These tests are conducted in a controlled environment. Alpha testing may be started when formal testing process is near completion. Beta Tests are conducted by the customers / end users at their sites. Unlike alpha testing, developer is not present here. Beta testing is conducted in a real environment that cannot be controlled by the developer.

13 Functional Testing Software Testing Input domain Output domain Input test data System under test Output test data Fig. 3: Black box testing 3

14 Boundary Value Analysis Consider a program with two input variables x and y. These input variables have specified boundaries as: a x b c y d Input domain d y c a b x Fig.4: Input domain for program having two input variables 4

15 The boundary value analysis test cases for our program with two inputs variables (x and y) that may have any value from 00 to 300 are: (00,00), (00,0), (00,00), (00,), (00,300), (00,00), (0,00), (,00) and (300,00). This input domain is shown in Fig Each dot represent a test case and inner rectangle is the domain of legitimate inputs. Thus, for a program of n variables, boundary value analysis yield 4n + test cases. y Input domain x Fig. 5: Input domain of two variables x and y with boundaries [00,300] each 5

16 Example- 8.I Consider a program for the determination of the nature of roots of a quadratic equation. Its input is a triple of positive integers (say a,b,c) and values may be from interval [0,00]. The program output may have one of the following words. [Not a quadratic equation; Real roots; Imaginary roots; Equal roots] Design the boundary value test cases. 6

17 Solution Quadratic equation will be of type: ax +bx+c=0 Roots are real if (b -4ac)>0 Roots are imaginary if (b -4ac)<0 Roots are equal if (b -4ac)=0 Equation is not quadratic if a=0 7

18 The boundary value test cases are : Test Case a b c Expected output 0 Not Quadratic Real Roots 3 Imaginary Roots 4 Imaginary Roots 5 00 Imaginary Roots 6 0 Imaginary Roots 7 Imaginary Roots 8 Imaginary Roots 9 00 Equal Roots 0 0 Real Roots Real Roots Imaginary Roots 3 00 Imaginary Roots 8

19 Example 8. Consider a program for determining the Previous date. Its input is a triple of day, month and year with the values in the range month day year 05 The possible outputs would be Previous date or invalid input date. Design the boundary value test cases. 9

20 Solution The Previous date program takes a date as input and checks it for validity. If valid, it returns the previous date as its output. With single fault assumption theory, 4n+ test cases can be designed and which are equal to 3. 0

21 The boundary value test cases are: Software Testing Test Case Month Day Year Expected output June, June, June, June, June, May, June, June, Invalid date January, February, November, December, 96

22 Example 8.3 Consider a simple program to classify a triangle. Its inputs is a triple of positive integers (say x, y, z) and the date type for input parameters ensures that these will be integers greater than 0 and less than or equal to 00. The program output may be one of the following words: [Scalene; Isosceles; Equilateral; Not a triangle] Design the boundary value test cases.

23 Solution The boundary value test cases are shown below: Test case x y z Expected Output Isosceles Isosceles 3 Equilateral 4 Isosceles 5 00 Not a triangle 6 Isosceles 7 Isosceles 8 Isosceles 9 00 Not a triangle 0 Isosceles Isosceles Isosceles 3 00 Not a triangle 3

24 Robustness testing Software Testing It is nothing but the extension of boundary value analysis. Here, we would like to see, what happens when the extreme values are exceeded with a value slightly greater than the maximum, and a value slightly less than minimum. It means, we want to go outside the legitimate boundary of input domain. This extended form of boundary value analysis is called robustness testing and shown in Fig. 6 There are four additional test cases which are outside the legitimate input domain. Hence total test cases in robustness testing are 6n+, where n is the number of input variables. So, 3 test cases are: (00,), (00,00), (00,0), (00,00), (00,), (00,300) (00,30), (,00), (00,00), (0,00), (,00), (300,00), (30,00) 4

25 y x Fig. 8.6: Robustness test cases for two variables x and y with range [00,300] each 5

26 Worst-case testing Software Testing If we reject single fault assumption theory of reliability and may like to see what happens when more than one variable has an extreme value. In electronic circuits analysis, this is called worst case analysis. It is more thorough in the sense that boundary value test cases are a proper subset of worst case test cases. It requires more effort. Worst case testing for a function of n variables generate 5 n test cases as opposed to 4n+ test cases for boundary value analysis. Our two variables example will have 5 =5 test cases and are given in table. 6

27 Table : Worst cases test inputs for two variables example Test case number x Inputs y Test case number x Inputs y

28 Example Consider the program for the determination of nature of roots of a quadratic equation as explained in example 8.. Design the Robust test case and worst test cases for this program. 8

29 Solution Robust test cases are 6n+. Hence, in 3 variable input cases total number of test cases are 9 as given on next slide: 9

30 Test case a b c Expected Output - Invalid input` 0 Not quadratic equation 3 Real roots 4 Imaginary roots 5 Imaginary roots 6 00 Imaginary roots 7 0 Invalid input 8 - Invalid input 9 0 Imaginary roots 0 Imaginary roots Imaginary roots 00 Equal roots 3 0 Invalid input 4 - Invalid input 5 0 Real roots 6 Real roots 7 Imaginary roots 8 00 Imaginary roots 9 0 Invalid input 30

31 In case of worst test case total test cases are 5 n. Hence, 5 test cases will be generated in worst test cases. The worst test cases are given below: Test Case a b c Expected output Not Quadratic 0 0 Not Quadratic Not Quadratic Not Quadratic Not Quadratic Not Quadratic 7 0 Not Quadratic 8 0 Not Quadratic 9 0 Not Quadratic Not Quadratic 0 0 Not Quadratic 0 Not Quadratic 3 0 Not Quadratic 4 0 Not Quadratic 3

32 Test Case A b c Expected output Not Quadratic Not Quadratic 7 0 Not Quadratic 8 0 Not Quadratic 9 0 Not Quadratic Not Quadratic Not Quadratic 0 00 Not Quadratic Not Quadratic Not Quadratic Not Quadratic Equal Roots 7 0 Imaginary 8 0 Imaginary 9 0 Imaginary Imaginary 3 0 Real Roots 3

33 Test Case A b C Expected output 3 Imaginary 33 Imaginary 34 Imaginary Imaginary 36 0 Real Roots 37 Real Roots 38 Real Roots 39 Real Roots Real Roots 4 0 Real Roots 4 Real Roots 43 Real Roots 44` Real Roots Real Roots Real Roots Real Roots Real Roots 33

34 Test Case A b C Expected output Real Roots Real Roots Equal Roots 5 0 Imaginary 53 0 Imaginary 54 0 Imaginary Imaginary 56 0 Real Roots 57 Imaginary 58 Imaginary 59 Imaginary Imaginary 6 0 Real Roots 6 Real Roots 63 Imaginary 64 Imaginary Imaginary 34

35 Test Case A b C Expected output 66 0 Real Roots 67 Real Roots 68 Imaginary 69 Imaginary Imaginary Real Roots 7 00 Real Roots Equal Roots Imaginary Imaginary Equal Roots 77 0 Imaginary 78 0 Imaginary 79 0 Imaginary Imaginary 8 0 Real Roots 8 Imaginary 35

36 Test Case A b C Expected output 83 Imaginary 84 Imaginary Imaginary 86 0 Real Roots 87 Real Roots 88 Imaginary 89 Imaginary Imaginary 9 0 Real Roots 9 Real Roots 93 Imaginary Roots 94 Imaginary Imaginary Real Roots Real Roots Imaginary 00 Imaginary Imaginary 36

37 Test Case A b C Expected output Equal Roots Imaginary Imaginary Imaginary Imaginary Real Roots Imaginary Imaginary Imaginary Imaginary 00 0 Real Roots 00 Real Roots 3 00 Imaginary 4 00 Imaginary Imaginary Real Roots 7 00 Real Roots 8 00 Imaginary 37

38 Test Case A b C Expected output 9 00 Imaginary Imaginary Real Roots Real Roots Imaginary Imaginary Imaginary 38

39 Example 8.5 Consider the program for the determination of previous date in a calendar as explained in example 8.. Design the robust and worst test cases for this program. 39

40 Solution Robust test cases are 6n+. Hence total 9 robust test cases are designed and are given on next slide. 40

41 Test case Month Day Year Expected Output Invalid date (outside range) June, June, June, June, June, Invalid date (outside range) Invalid date May, June, June, Invalid date Invalid date Invalid date January, February, 96 4 November, December, Invalid date 4

42 In case of worst test case total test cases are 5 n. Hence, 5 test cases will be generated in worst test cases. The worst test cases are given below: Test Case Month Day Year Expected output December, December, December, December, December, January, January, January, January, January, January, January, January, January, 04 4

43 Test Case A b c Expected output January, January, January, January, January, January, January, January, January, January, January, January, January, January, January, January, February,

44 Test Case Month Day Year Expected output 3 90 February, February, February, February, February, February, February, February, February, Invalid date Invalid date Invalid date Invalid date Invalid date Invalid date Invalid date Invalid date 44

45 Test Case Month Day Year Expected output Invalid date 3 05 Invalid date May, May, May, May, May, June, June, June, June, June, June, June, June, June, June, 05 45

46 Test Case Month Day Year Expected output June, June, June, June, June, Invalid date Invalid date Invalid date Invalid date Invalid date 3 October, October, October, October, October, November, November, 90 46

47 Test Case Month Day Year Expected output November, November, November, November, November, November, November, November, November, November, November, November, November, Invalid date Invalid date Invalid date 3 04 Invalid date Invalid date 47

48 Test Case Month Day Year Expected output November, November, November, November, November, December, December, December, December, December, December, December, December, December, December, December, December, December, 96 48

49 Test Case Month Day Year Expected output December, December, December, December, December, December, December, 05 49

50 Example 8.6 Consider the triangle problem as given in example 8.3. Generate robust and worst test cases for this problem.

51 Solution Robust test cases are given on next slide. 5

52 ` x y z Expected Output 0 Invalid input` Isosceles 3 Isosceles 4 Equilateral 5 Isosceles 6 00 Not a triangle 7 0 Invalid input 8 0 Invalid input 9 Isosceles 0 Isosceles Isosceles 00 Not a triangle 3 0 Invalid input 4 0 Invalid input 5 Isosceles 6 Isosceles 7 Isosceles 8 00 Not a triangle 9 00 Invalid input 5

53 Worst test cases are 5 and are given below: Test Case x y z Expected output Equilateral Not a triangle 3 Not a triangle 4 Not a triangle 5 00 Not a triangle 6 Not a triangle 7 Isosceles 8 9 Not a triangle Not a triangle 0 00 Not a triangle Not a triangle Not a triangle 3 Isosceles 4 Not a triangle 53

54 Test Case A b c Expected output 5 00 Not a triangle 6 Not a triangle 7 Not a triangle 8 Not a triangle 9 Isosceles 0 00 Not a triangle 00 Not a triangle 00 Not a triangle 3 00 Not a triangle 4 00 Not a triangle Isosceles 6 Not a triangle 7 Isosceles 8 Not a triangle 9 Not a triangle Not a triangle 3 Isosceles 54

55 Test Case A b C Expected output 3 Equilateral 33 Not a triangle 34 Not a triangle Not a triangle 36 Not a triangle 37 Not a triangle 38 Isosceles 39 Not a triangle Not a triangle 4 Not a triangle 4 Not a triangle 43 Not a triangle 44 Isosceles Scalene Not a triangle Not a triangle Not a triangle 55

56 Test Case A b C Expected output Scalene 00 Isosceles 5 00 Not a triangle 5 Not a triangle 53 Isosceles 54 Not a triangle 55 Not a triangle Not a triangle 57 Not a triangle 58 Isosceles 59 Not a triangle 60 Not a triangle 6 00 Isosceles 6 Isosceles 63 Equilateral 64 Isosceles 65 Not a triangle 56

57 Test Case A B C Expected output 66 Not a triangle 67 Not a triangle 68 Isosceles 69 Isosceles Scalene 7 00 Not a triangle 7 00 Not a triangle Not a triangle Scalene Isosceles 76 Not a triangle 77 Not a triangle 78 Not a triangle 79 Isosceles Not a triangle 8 Not a triangle 8 Not a triangle 57

58 Test Case A b C Expected output 83 Not a triangle 84 Isosceles Scalene 86 Not a triangle 87 Not a triangle 88 Isosceles 89 Isosceles Scalene 9 Isosceles 9 Isosceles 93 Isosceles 94 Equilateral Isosceles Not a triangle Scalene Scalene 00 Isosceles Isosceles 58

59 Test Case A b C Expected output 0 00 Not a triangle 0 00 Not a triangle Not a triangle Not a triangle Isosceles Not a triangle Not a triangle Not a triangle Scalene Isosceles 00 Not a triangle 00 Not a triangle 3 00 Not a triangle 4 00 Scalene Isosceles 6 00 Not a triangle 7 00 Scalene 8 00 Scalene 59

60 Test Case A b C Expected output 9 00 Isosceles Isosceles Isosceles Isosceles Isosceles Isosceles Equilateral 60

61 Equivalence Class Testing Software Testing In this method, input domain of a program is partitioned into a finite number of equivalence classes such that one can reasonably assume, but not be absolutely sure, that the test of a representative value of each class is equivalent to a test of any other value. Two steps are required to implementing this method:. The equivalence classes are identified by taking each input condition and partitioning it into valid and invalid classes. For example, if an input condition specifies a range of values from to 9, we identify one valid equivalence class [<item<9]; and two invalid equivalence classes [item<] and [item>9].. Generate the test cases using the equivalence classes identified in the previous step. This is performed by writing test cases covering all the valid equivalence classes. Then a test case is written for each invalid equivalence class so that no test contains more than one invalid class. This is to ensure that no two invalid classes mask each other. 6

62 Invalid input Valid inputs System under test Outputs Input domain Output domain Fig. 7: Equivalence partitioning Most of the time, equivalence class testing defines classes of the input domain. However, equivalence classes should also be defined for output domain. Hence, we should design equivalence classes based on input and output domain. 6

63 Example 8.7 Consider the program for the determination of nature of roots of a quadratic equation as explained in example 8.. Identify the equivalence class test cases for output and input domains. 63

64 Solution Output domain equivalence class test cases can be identified as follows: O ={<a,b,c>:not a quadratic equation if a = 0} O ={<a,b,c>:real roots if (b -4ac)>0} O ={<a,b,c>:imaginary roots if (b -4ac)<0} O ={<a,b,c>:equal roots if (b -4ac)=0}` The number of test cases can be derived form above relations and shown below: Test case a b c Expected output 0 Not a quadratic equation Real roots 3 Imaginary roots 4 00 Equal roots 64

65 We may have another set of test cases based on input domain. I = {a: a = 0} I = {a: a < 0} I 3 = {a: a 00} I 4 = {a: a > 00} I 5 = {b: 0 b 00} I 6 = {b: b < 0} I 7 = {b: b > 00} I 8 = {c: 0 c 00} I 9 = {c: c < 0} I 0 ={c: c > 00} 65

66 Test Case a b c Expected output 0 Not a quadratic equation - Invalid input 3 Imaginary Roots 4 0 invalid input 5 Imaginary Roots 6 - invalid input 7 0 invalid input 8 Imaginary Roots 9 - invalid input 0 0 invalid input Here test cases 5 and 8 are redundant test cases. If we choose any value other than nominal, we may not have redundant test cases. Hence total test cases are 0+4=4 for this problem. 66

67 Example 8.8 Consider the program for determining the previous date in a calendar as explained in example 8.3. Identify the equivalence class test cases for output & input domains. 67

68 Solution Output domain equivalence class are: O ={<D,M,Y>: Previous date if all are valid inputs} O ={<D,M,Y>: Invalid date if any input makes the date invalid} Test case M D Y Expected output June, Invalid date 68

69 We may have another set of test cases which are based on input domain. I ={month: m } I ={month: m < } I 3 ={month: m > } I 4 ={day: D 3} I 5 ={day: D < } I 6 ={day: D > 3} I 7 ={year: 900 Y 05} I 8 ={year: Y < 900} I 9 ={year: Y > 05} 69

70 Inputs domain test cases are : Test Case M D Y Expected output June, Invalid input invalid input June, invalid input invalid input June, invalid input (Value out of range) invalid input (Value out of range) 70

71 Example 8.9 Consider the triangle problem specified in a example 8.3. Identify the equivalence class test cases for output and input domain. 7

72 Solution Output domain equivalence classes are: O ={<x,y,z>: Equilateral triangle with sides x,y,z} O ={<x,y,z>: Isosceles triangle with sides x,y,z} O ={<x,y,z>: Scalene triangle with sides x,y,z} O ={<x,y,z>: Not a triangle with sides x,y,z} The test cases are: Test case x y z Expected Output Equilateral Isosceles 3 00 Scalene 4 00 Not a triangle 7

73 Input domain based classes are: I ={x: x < } I ={x: x > 00} I 3 ={x: x 00} I 4 ={y: y < } I 5 ={y: y > 00} I 6 ={y: y 00} I 7 ={z: z < } I 8 ={z: z > 00} I 9 ={z: z 00} 73

74 Some inputs domain test cases can be obtained using the relationship amongst x,y and z. I 0 ={< x,y,z >: x = y = z} I ={< x,y,z >: x = y, x z} I ={< x,y,z >: x = z, x y} I 3 ={< x,y,z >: y = z, x y} I 4 ={< x,y,z >: x y, x z, y z} I 5 ={< x,y,z >: x = y + z} I 6 ={< x,y,z >: x > y +z} I 7 ={< x,y,z >: y = x +z} I 8 ={< x,y,z >: y > x + z} I 9 ={< x,y,z >: z = x + y} I 0 ={< x,y,z >: z > x +y} 74

75 Test cases derived from input domain are: Test case x y z Expected Output 0 Invalid input 0 Invalid input 3 Equilateral 4 0 Invalid input 5 0 Invalid input 6 Equilateral 7 0 Invalid input 8 0 Invalid input 9 Equilateral Equilateral 60 Isosceles 60 Isosceles 3 60 Isosceles 75

76 Test case x y z Expected Output 4 00 Scalene 5 00 Not a triangle Not a triangle 7 00 Not a triangle Not a triangle 9 00 Not a triangle Not a triangle 76

77 Decision Table Based Testing Condition Stub Entry True False C True False True False C C 3 True False True False True False --- Action a Stub a a 3 a 4 Table : Decision table terminology 77

78 Test case design C :x,y,z are sides of a triangle? N Y C :x = y? -- Y N C 3 :x = z? C 4 :y = z? a : Not a triangle a : Scalene Y N Y N Y N Y N Y N Y N a 3 : Isosceles a 4 : Equilateral a 5 : Impossible Table 3: Decision table for triangle problem 78

79 Conditions C : x < y + z? F T T T T T T T T T T C : y < x + z? -- F T T T T T T T T T C 3 : z < x + y? F T T T T T T T T C 4 : x = y? T T T T F F F F C 5 : x = z? T T F F T T F F C 6 : y = z? T F T F T F T F a : Not a triangle a : Scalene a 3 : Isosceles a 4 : Equilateral a 5 : Impossible Table 4: Modified decision table 79

80 Example 8.0 Consider the triangle program specified in example 8.3. Identify the test cases using the decision table of Table 4. 80

81 Solution There are eleven functional test cases, three to fail triangle property, three impossible cases, one each to get equilateral, scalene triangle cases, and three to get on isosceles triangle. The test cases are given in Table 5. Test case x y z Expected Output 4 Not a triangle 4 Not a triangle 3 4 Not a triangle Equilateral 5??? Impossible 6??? Impossible 7 3 Isosceles 8??? Impossible 9 3 Isosceles 0 3 Isosceles Scalene Test cases of triangle problem using decision table 8

82 Example 8. Consider a program for the determination of Previous date. Its input is a triple of day, month and year with the values in the range month day year 05 The possible outputs are Previous date and Invalid date. Design the test cases using decision table based testing. 8

83 Solution The input domain can be divided into following classes: I = {M : month has 30 days} I = {M : month has 3 days except March, August and January} I 3 = {M 3 : month is March} I 4 = {M 4 : month is August} I 5 = {M 5 : month is January} I 6 = {M 6 : month is February} I 7 = {D : day = } I 8 = {D : day 8} I 9 = {D 3 : day = 9} I 0 ={D 4 : day = 30} I ={D 5 : day = 3} I ={Y : year is a leap year} I 3 ={Y : year is a common year} 83

84 The decision table is given below: Software Testing Sr.No C : Months in M M M M M M M M M M M M M M M C : days in D D D D D 3 D 3 D 4 D 4 D 5 D 5 D D D D D 3 C 3 : year in Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y a : Impossible a : Decrement day a 3 : Reset day to 3 a 4 : Reset day to 30 a 5 : Reset day to 9 a 6 : Reset day to 8 a 7 : decrement month a 8 : Reset month to December a 9 : Decrement year 84

85 Sr.No C : Months in M M M M M M 3 M 3 M 3 M 3 M 3 M 3 M 3 M 3 M 3 M 3 C : days in D 3 D 4 D 4 D 5 D 5 D D D D D 3 D 3 D 4 D 4 D 5 D 5 C 3 : year in Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y a : Impossible a : Decrement day a 3 : Reset day to 3 a 4 : Reset day to 30 a 5 : Reset day to 9 a 6 : Reset day to 8 a 7 : decrement month a 8 : Reset month to December a 9 : Decrement year 85

86 Sr.No C : Months in M 4 M 4 M 4 M 4 M 4 M 4 M 4 M 4 M 4 M 4 M 5 M 5 M 5 M 5 M 5 C : days in D D D D D 3 D 3 D 4 D 4 D 5 D 5 D D D D D 3 C 3 : year in Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y a : Impossible a : Decrement day a 3 : Reset day to 3 a 4 : Reset day to 30 a 5 : Reset day to 9 a 6 : Reset day to 8 a 7 : decrement month a 8 : Reset month to December a 9 : Decrement year 86

87 Sr.No C : Months in M 5 M 5 M 5 M 5 M 5 M 6 M 6 M 6 M 6 M 6 M 6 M 6 M 6 M 6 M 6 C : days in D 3 D 4 D 4 D 5 D 5 D D D D D 3 D 3 D 4 D 4 D 5 D 5 C 3 : year in Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y a : Impossible a : Decrement day a 3 : Reset day to 3 a 4 : Reset day to 30 a 5 : Reset day to 9 a 6 : Reset day to 8 a 7 : decrement month a 8 : Reset month to December a 9 : Decrement year 87

88 Test case Month Day Year Expected output June May, 964 June 96 3 May, 96 3 June June, June June, 96 5 June June, June June, 96 7 June June, June June, 96 9 June Impossible 0 June 3 96 Impossible May April, 964 May April, 96 3 May May, May May, 96 5 May May,

89 Test case Month Day Year Expected output 6 May May, 96 7 May May, May May, 96 9 May May, May May, 96 March February, 964 March 96 8 February, 96 3 March March, March March, 96 5 March March, March March, 96 7 March March, March March, 96 9 March March, March March, 96 89

90 Test case Month Day Year Expected output 3 August July, 96 3 August 96 3 July, August August, August August, August August, August August, August August, August August, August August, August August, 96 4 January December, January 96 3 December, January January, January January, January January,

91 Test case Month Day Year Expected output 46 January January, January January, January January, January January, 964 January January, 96 5 February January, February 96 3 January, February February, February February, February February, February 9 96 Impossible 57 February Impossible 58 February Impossible 59 February Impossible 60 February 3 96 Impossible 9

92 Cause Effect Graphing Technique Consider single input conditions Software Testing do not explore combinations of input circumstances Steps. Causes & effects in the specifications are identified. A cause is a distinct input condition or an equivalence class of input conditions. An effect is an output condition or a system transformation.. The semantic content of the specification is analysed and transformed into a boolean graph linking the causes & effects. 3. Constraints are imposed 4. graph limited entry decision table Each column in the table represent a test case. 5. The columns in the decision table are converted into test cases. 9

93 The basic notation for the graph is shown in fig. 8 Fig.8. 8 : Basic cause effect graph symbols 93

94 Myers explained this effectively with following example. The characters in column must be an A or B. The character in column must be a digit. In this situation, the file update is made. If the character in column is incorrect, message x is issued. If the character in column is not a digit, message y is issued. The causes are c : character in column is A c : character in column is B c 3 : character in column is a digit and the effects are e : update made e : message x is issued e 3 : message y is issued 94

95 Fig. 9: Sample cause effect graph 95

96 The E constraint states that it must always be true that at most one of c or c can be (c or c cannot be simultaneously). The I constraint states that at least one of c, c and c 3 must always be (c, c and c 3 cannot be 0 simultaneously). The O constraint states that one, and only one, of c and c must be. The constraint R states that, for c to be, c must be (i.e. it is impossible for c to be and c to be 0), 96

97 Fig. 0: Constraint symbols 97

98 Fig. : Symbol for masks constraint 98

99 Fig. : Sample cause effect graph with exclusive constraint

100 Example 8. Consider the triangle problem specified in the example 8.3. Draw the Cause effect graph and identify the test cases. 00

101 Solution The causes are c : side x is less than sum of sides y and z c : side y is less than sum of sides x and y c 3 : side z is less than sum of sides x and y c 4 : side x is equal to side y c 5 : side x is equal to side z c 6 : side y is equal to side z and effects are e : Not a triangle e : Scalene triangle e 3 : Isosceles triangle e 4 : Equilateral triangle e 5 : Impossible stage 0

102 The cause effect graph is shown in fig. 3 and decision table is shown in table 6. The test cases for this problem are available in Table 5. Conditions C : x < y + z? 0 C : y < x + z? C 3 : z < x + y? C 4 : x = y? C 5 : x = z? C 6 : y = z? e : Not a triangle e : Scalene e 3 : Isosceles e 4 : Equilateral e 5 : Impossible Table 6: Decision table 0

103 Fig. 3: Cause effect graph of triangle problem 03

104 Structural Testing A complementary approach to functional testing is called structural / white box testing. It permits us to examine the internal structure of the program. Path Testing Path testing is the name given to a group of test techniques based on judiciously selecting a set of test paths through the program. If the set of paths is properly chosen, then it means that we have achieved some measure of test thoroughness. This type of testing involves:. generating a set of paths that will cover every branch in the program.. finding a set of test cases that will execute every path in the set of program paths. 04

105 Flow Graph The control flow of a program can be analysed using a graphical representation known as flow graph. The flow graph is a directed graph in which nodes are either entire statements or fragments of a statement, and edges represents flow of control. Fig. 4: The basic construct of the flow graph 05

106 06

107 07

108 08

109 09

110 Fig. 5: Program for previous date problem 0

111 Fig. 6: Flow graph of previous date problem

112 DD Path Graph Table 7: Mapping of flow graph nodes and DD path nodes Flow graph nodes to 9 0 3,4 5,6, DD Path graph corresponding node n n n 3 n 4 n 5 n 6 n 7 n 8 n 9 n 0 n n Remarks There is a sequential flow from node to 9 Decision node, if true go to 3 else go to 44 Decision node, if true go to else go to 9 Decision node, if true go to 3 else go to 5 Sequential nodes and are combined to form new node n 5 Sequential nodes Edges from node 4 to 7 are terminated here Decision node, if true go to 0 else go to 37 Intermediate node with one input edge and one output edge Decision node, if true go to else go to 7 Intermediate node Decision node, if true go to 4 else go to 6 Cont.

113 Flow graph nodes 4, ,9 30 3,3 33,34, ,39 40,4,4 43 DD Path graph corresponding node n 3 n 4 n 5 n 6 n 7 n 8 n 9 n 0 n n n 3 n 4 Remarks Sequential nodes Two edges from node 5 & 3 are terminated here Two edges from node 6 & are terminated here. Also a decision node Sequential nodes Decision node, if true go to 3 else go to 33 Sequential nodes Sequential nodes Three edge from node 9,3 and 35 are terminated here Decision node, if true go to 38 else go to 40 Sequential nodes Sequential nodes Three edge from node 36,39 and 4 are terminated here Cont. 3

114 Flow graph nodes ,48,49, , DD Path graph corresponding node n 5 n 6 n 7 n 8 n 9 n 30 n 3 n 3 n 33 n 34 n 35 n 36 Decision node, if true go to 45 else go to 8. Three edges from 8,43 & 0 are also terminated here. Decision node, if true go to 46 else go to 77 Decision node, if true go to 47 else go to 5 Sequential nodes Remarks Decision node, if true go to 5 else go to 68 Intermediate node with one input edge & one output ege Decision node, if true go to 54 else go to 59 Intermediate node Decision node, if true go to 56 else go to 58 Sequential nodes Two edge from node 57 and 55 are terminated here Decision node, if true go to 60 else go to 63. Two edge from nodes 58 and 53 are terminated. Cont. 4

115 Flow graph nodes 60,6,6 63,64,65, ,70,7 7,73,74, ,78, ,83, ,87 DD Path graph corresponding node n 37 n 38 n 39 n 40 n 4 n 4 n 43 n 44 n 45 n 46 n 47 n 48 n 49 Remarks Sequential nodes Sequential nodes Two edge from node 6 and 66 are terminated here Decision node, if true go to 69 else go to 7 Sequential nodes Sequential nodes Four edges from nodes, 67, 7 and 75 are terminated here. Sequential nodes Two edges from nodes 76 & 79 are terminated here Intermediate node Sequential nodes Two edges from nodes 8 and 84 are terminated here Sequential nodes with exit node 5

116 Fig. 7: DD path graph of previous date problem 6

117 Fig. 8: Independent paths of previous date problem 7

118 Example 8.3 Consider the problem for the determination of the nature of roots of a quadratic equation. Its input a triple of positive integers (say a,b,c) and value may be from interval [0,00]. The program is given in fig. 9. The output may have one of the following words: [Not a quadratic equation; real roots; Imaginary roots; Equal roots] Draw the flow graph and DD path graph. Also find independent paths from the DD Path graph. 8

119 Cont. 9

120 Fig. 9: Code of quadratic equation problem 0

121 Solution Software Testing Fig. 9 (a) : Program flow graph

122 Fig. 9 (b) : DD Path graph

123 The mapping table for DD path graph is: Flow graph nodes to 0 3 4, , 3,4,5 DD Path graph corresponding node A B C D E F G H I J K L Remarks Sequential nodes Decision node Intermediate node Decision node Sequential node Two edges are combined here Two edges are combined and decision node Intermediate node Decision node Sequential node Decision node Sequential node Cont. 3

124 Flow graph nodes 6,7,8, ,33 34,35, ,39 DD Path graph corresponding node M N O P Q R S Remarks Sequential nodes Three edges are combined Decision node Sequential node Sequential node Three edges are combined here Sequential nodes with exit node Independent paths are: (i) ABGOQRS (iii) ABCDFGOQRS (v) ABGHIJNRS (vi) ABGHIKMNRS (ii) ABGOPRS (iv) ABCDEFGOPRS (vi) ABGHIKLNRS 4

125 Example 8.4 Consider a program given in Fig.8.0 for the classification of a triangle. Its input is a triple of positive integers (say a,b,c) from the interval [,00]. The output may be [Scalene, Isosceles, Equilateral, Not a triangle]. Draw the flow graph & DD Path graph. Also find the independent paths from the DD Path graph. 5

126 6

127 Fig. 0 : Code of triangle classification problem 7

128 Solution : Flow graph of triangle problem is: Fig.8. 0 (a): Program flow graph 8

129 The mapping table for DD path graph is: Flow graph nodes TO 9 0, 3 4 5, 6, , 3, 4 5, 6, 7 DD Path graph corresponding node A B C D E F G H I J K L Remarks Sequential nodes Decision node Decision node Sequential nodes Two edges are joined here Sequential nodes Decision nodes plus joining of two edges Decision node Sequential nodes Decision node Sequential nodes Sequential nodes Cont. 9

130 Flow graph nodes , 3 DD Path graph corresponding node M N O Three edges are combined here Decision node Sequential nodes Remarks 3, 33, , 37 P Q R Sequential nodes Three edges are combined here Sequential nodes with exit node Fig. 0 (b): DD Path graph 30

131 DD Path graph is given in Fig. 0 (b) Independent paths are: (i) ABFGNPQR (ii) ABFGNOQR (iii) ABCEGNPQR (iv) ABCDEGNOQR (v) ABFGHIMQR (vi) ABFGHJKMQR (vii)abfghjmqr Fig. 0 (b): DD Path graph 3

132 Cyclomatic Complexity Software Testing McCabe s cyclomatic metric V(G) = e n + P. For example, a flow graph shown in in Fig. with entry node a and exit node f. Fig. : Flow graph 3

133 The value of cyclomatic complexity can be calculated as : V(G) = = 5 Here e = 9, n = 6 and P = There will be five independent paths for the flow graph illustrated in Fig.. Path : a c f Path : a b e f Path 3 : a d c f Path 4 : a b e a c f or a b e a b e f Path 5 : a b e b e f 33

134 Several properties of cyclomatic complexity are stated below:. V(G). V (G) is the maximum number of independent paths in graph G. 3. Inserting & deleting functional statements to G does not affect V(G). 4. G has only one path if and only if V(G)=. 5. Inserting a new row in G increases V(G) by unity. 6. V(G) depends only on the decision structure of G. 34

135 The role of P in the complexity calculation V(G)=e-n+P is required to be understood correctly. We define a flow graph with unique entry and exit nodes, all nodes reachable from the entry, and exit reachable from all nodes. This definition would result in all flow graphs having only one connected component. One could, however, imagine a main program M and two called subroutines A and B having a flow graph shown in Fig.. Fig. 35

136 Let us denote the total graph above with 3 connected components as V ( M A B) = e n + P = 3-3+*3 = 6 This method with P can be used to calculate the complexity of a collection of programs, particularly a hierarchical nest of subroutines. 36

137 V ( M A B) = V ( M ) + V ( A) + V ( B) = Notice that. In general, the complexity of a collection C of flow graphs with K connected components is equal to the summation of their complexities. To see this let C i, I K denote the k distinct connected component, and let e i and n i be the number of edges and nodes in the ith-connected component. Then 6 V ( C) k = e n + p = ei ni + K k i= i= = k i= ( e i n i + ) = k i= V ( C i ) 37

138 Two alternate methods are available for the complexity calculations.. Cyclomatic complexity V(G) of a flow graph G is equal to the number of predicate (decision) nodes plus one. V(G)= + Where is the number of predicate nodes contained in the flow graph G.. Cyclomatic complexity is equal to the number of regions of the flow graph. 38

139 Example 8.5 Software Testing Consider a flow graph given in Fig. 3 and calculate the cyclomatic complexity by all three methods. Fig. 3 39

140 Solution Cyclomatic complexity can be calculated by any of the three methods.. V(G) = e n + P = = 5. V(G) = π + = 4 + = 5 3. V(G) = number of regions = 5 Therefore, complexity value of a flow graph in Fig. 3 is 5. 40

141 Example 8.6 Consider the previous date program with DD path graph given in Fig. 7. Find cyclomatic complexity. 4

142 Solution Number of edges (e) = 65 Number of nodes (n) =49 (i) V(G) = e n + P = = 8 (ii) V(G) = π + = 7 + = 8 (iii) V(G) = Number of regions = 8 The cyclomatic complexity is 8. 4

143 Example 8.7 Consider the quadratic equation problem given in example 8.3 with its DD Path graph. Find the cyclomatic complexity: 43

144 Solution Number of nodes (n) = 9 Number of edges (e) = 4 (i) V(G) = e n + P = = 7 (ii) V(G) = π + = 6 + = 7 (iii)v(g) = Number of regions = 7 Hence cyclomatic complexity is 7 meaning thereby, seven independent paths in the DD Path graph. 44

145 Example 8.8 Consider the classification of triangle problem given in example 8.4. Find the cyclomatic complexity. 45

146 Solution Number of edges (e) = 3 Number of nodes (n) =8 (i) V(G) = e n + P = = 7 (ii) V(G) = π + = 6 + = 7 (iii)v(g) = Number of regions = 7 The cyclomatic complexity is 7. Hence, there are seven independent paths as given in example

147 Graph Matrices A graph matrix is a square matrix with one row and one column for every node in the graph. The size of the matrix (i.e., the number of rows and columns) is equal to the number of nodes in the flow graph. Some examples of graphs and associated matrices are shown in fig. 4. Fig. 4 (a): Flow graph and graph matrices 47

148 Fig. 4 (b): Flow graph and graph matrices 48

149 Fig. 4 (c): Flow graph and graph matrices 49

150 Fig. 5 : Connection matrix of flow graph shown in Fig. 4 (c)

151 The square matrix represent that there are two path ab and cd from node to node. 5

152 Example 8.9 Consider the flow graph shown in the Fig. 6 and draw the graph & connection matrices. Find out cyclomatic complexity and two / three link paths from a node to any other node. Fig. 6 : Flow graph 5

153 Solution The graph & connection matrices are given below : To find two link paths, we have to generate a square of graph matrix [A] and for three link paths, a cube of matrix [A] is required. 53

154 54

155 Data Flow Testing Data flow testing is another from of structural testing. It has nothing to do with data flow diagrams. i. Statements where variables receive values. ii. Statements where these values are used or referenced. As we know, variables are defined and referenced throughout the program. We may have few define/ reference anomalies: i. A variable is defined but not used/ referenced. ii. iii. A variable is used but never defined. A variable is defined twice before it is used. 55

156 Definitions The definitions refer to a program P that has a program graph G(P) and a set of program variables V. The G(P) has a single entry node and a single exit node. The set of all paths in P is PATHS(P) (i) Defining Node: Node n ϵ G(P) is a defining node of the variable v ϵ V, written as DEF (v, n), if the value of the variable v is defined at the statement fragment corresponding to node n. (ii) Usage Node: Node n ϵ G(P) is a usage node of the variable v ϵ V, written as USE (v, n), if the value of the variable v is used at statement fragment corresponding to node n. A usage node USE (v, n) is a predicate use (denote as p) if statement n is a predicate statement otherwise USE (v, n) is a computation use (denoted as c). 56

157 (iii) Definition use: A definition use path with respect to a variable v (denoted du-path) is a path in PATHS(P) such that, for some v ϵ V, there are define and usage nodes DEF(v, m) and USE(v, n) such that m and n are initial and final nodes of the path. (iv) Definition clear : A definition clear path with respect to a variable v (denoted dc-path) is a definition use path in PATHS(P) with initial and final nodes DEF (v, m) and USE (v, n), such that no other node in the path is a defining node of v. The du-paths and dc-paths describe the flow of data across source statements from points at which the values are defined to points at which the values are used. The du-paths that are not definition clear are potential trouble spots. 57

158 Hence, our objective is to find all du-paths and then identity those du-paths which are not dc-paths. The steps are given in Fig. 7. We may like to generate specific test cases for du-paths that are not dc-paths. Fig. 7 : Steps for data flow testing 58

159 Example 8.0 Consider the program of the determination of the nature of roots of a quadratic equation. Its input is a triple of positive integers (say a,b,c) and values for each of these may be from interval [0,00]. The program is given in Fig. 9. The output may have one of the option given below: (i) Not a quadratic program (ii) real roots (iii) imaginary roots (iv) equal roots (v) invalid inputs Find all du-paths and identify those du-paths that are definition clear. 59

160 Solution Step I: The program flow graph is given in Fig. 9 (a). The variables used in the program are a,b,c,d, validinput, D. Step II: DD Path graph is given in Fig. 9(b). The cyclomatic complexity of this graph is 7 indicating there are seven independent paths. Step III: Define/use nodes for all variables are given below: Variable a b c d D Validinput Defined at node , 7 3,, 4 Used at node,3,8,0,4,7,8,8,0,4,8,8 9,,3,7 4,8 7,3 60

161 Step IV: The du-paths are identified and are named by their beginning and ending nodes using Fig. 9 (a). Variable a b Path (beginning, end) nodes 6, 6, 3 6, 8 6, 0 6, 4 6, 7 6, 8 8, 8, 8 8, 0 8, 4 8, 8 Definition clear? Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 6

162 Variable c d D validinput Path (beginning, end) nodes 0, 0, 8 8, 9 8, 8, 3 8, 7 3, 4 3, 8 7, 4 7, 8 3, 7 3, 3, 7, 3 4, 7 4, 3 Definition clear? Yes Yes Yes Yes Yes Yes Yes Path not possible Path not possible Yes no no no no yes yes 6

163 Example 8. Consider the program given in Fig. 0 for the classification of a triangle. Its input is a triple of positive integers (say a,b,c) from the interval [,00]. The output may be: [Scalene, Isosceles, Equilateral, Not a triangle, Invalid inputs]. Find all du-paths and identify those du-paths that are definition clear. 63

164 Solution Step I: The program flow graph is given in Fig. 0 (a). The variables used in the program are a,b,c, valid input. Step II: DD Path graph is given in Fig. 0(b). The cyclomatic complexity of this graph is 7 and thus, there are 7 independent paths. Step III: Define/use nodes for all variables are given below: Variable a b c valid input Defined at node , 3, 6 Used at node 0,, 9, 0,, 9, 0,, 9, 8, 9 64

165 Step IV: The du-paths are identified and are named by their beginning and ending nodes using Fig. 0 (a). Variable a Path (beginning, end) nodes 5, 0 5, 5, 9 5, Definition clear? Yes Yes Yes Yes b 7, 0 7, 7, 9 7, Yes Yes Yes Yes 65

166 Variable c valid input Path (beginning, end) nodes 9, 0 9, 9, 9 9, 3, 8 3, 9, 8, 9 6, 8 6, 9 Definition clear? Yes Yes Yes Yes no no no no Yes Yes Hence total du-paths are 8 out of which four paths are not definition clear 66

167 Mutation Testing Mutation testing is a fault based technique that is similar to fault seeding, except that mutations to program statements are made in order to determine properties about test cases. it is basically a fault simulation technique. Multiple copies of a program are made, and each copy is altered; this altered copy is called a mutant. Mutants are executed with test data to determine whether the test data are capable of detecting the change between the original program and the mutated program. A mutant that is detected by a test case is termed killed and the goal of mutation procedure is to find a set of test cases that are able to kill groups of mutant programs. 67

168 When we mutate code there needs to be a way of measuring the degree to which the code has been modified. For example, if the original expression is x+ and the mutant for that expression is x+, that is a lesser change to the original code than a mutant such as (c*), where both the operand and the operator are changed. We may have a ranking scheme, where a first order mutant is a single change to an expression, a second order mutant is a mutation to a first order mutant, and so on. High order mutants becomes intractable and thus in practice only low order mutants are used. One difficulty associated with whether mutants will be killed is the problem of reaching the location; if a mutant is not executed, it cannot be killed. Special test cases are to be designed to reach a mutant. For example, suppose, we have the code. Read (a,b,c); If(a>b) and (b=c) then x:=a*b*c; (make mutants; m, m, m 3.) 68