Integrating White- and Black-Box Techniques for Class-Level Regression Testing

Integrating White- and Black-Box Techniques for Class-Level Regression Testing Sami Beydeda, Volker Gruhn University of Dortmund Computer Science Department Software Technology 44221 Dortmund, Germany fsami.beydeda, volker.gruhng@uni-dortmund.de Abstract In recent years, several techniques have been proposed for class-level regression testing. Most of these techniques focus either on white- or black-box testing, although an integrated approach can have several benefits. As similar tasks have to be carried out for both white- and black-box testing, an integrated approach can improve efficiency and cost effectiveness. This article explains a new approach for class-level regression testing, integrating existing techniques. Particularly, those of Rothermel et al. and Hong et al. for whitebox regression testing and black-box testing, respectively, have been integrated into a single technique. The benefits of the resulting technique are shown by example. 1 Introduction Testing software to determine whether changes are implemented as intended and that they do not affect other unchanged parts of the software adversely is called regression testing. This article proposes a new technique for regression testing of classes combining two existing techniques. We have integrated the testing techniques of Rothermel et al. [7] and Hong et al. [5]. As the former is a white-box technique and the latter a black-box technique, the resulting technique can be used for simultaneous white- and blackbox testing. Generally, sufficient testing of software requires both white- and black-box techniques. Although both tasks are similar in that they have the same objectives, namely detecting faults within a program, often white- and black-box techniques are applied separately using different tools. The reason is a lack of techniques integrating both strategies. Few approaches have investigated the integration of whiteand black-box testing techniques for object-oriented software [4, 1, 2, 3]. Often, the software is tested independently against the specification and source code using different specialized tools. Using different techniques and tools often leads to inefficient test processes. The tester has to work with two different tools which could require different skills and thus separate training. The tester has to be familiar not only with the two different tools, but also with their different underlying concepts. Additionally, using several tools requires higher maintenance effort. Therefore, we think that integrated white- and black-box testing techniques can substantially improve the testing process with respect to both costs and time. 2 Integration of the techniques 2.1 Building blocks One of the building blocks of the proposed technique is the class regression testing technique proposed by Rothermel et al. [7]. The main idea for their technique is to compare two versions of a class to detect and analyze changes. Based on this analysis, test cases covering changed statements are selected from a given test set originating from previous tests. The comparison of the two class versions is carried out using representations called class control flow graphs (CCFGs). A CCFG is a graphical representation of a class visualizing its methods embedded in an abstract test driver. The comparison is carried out by simultaneously traversing both CCFGs and comparing corresponding nodes of the graphs. The other building is the black-box testing technique of Hong et al. [5]. The main idea of this technique is to identify definitions and uses [6] of each attribute and to associate them to each other according to some data flow criteria. After associating definitions and uses with each other, test cases covering these def-use pairs are generated. To ease

test case generation, Hong et al. suggest using a graphical representation of the class, class flow graph (CFG), generated on the basis of a special finite state machine called class state machine (CSM). 2.2 Demonstrative example The example consists of a class, called account, which simulates a bank account. It provides the appropriate methods for making bank account deposits (deposit()) and withdrawals (withdraw()). Furthermore, it provides methods for paying interest (payinterest()) and for printing bank statements (printbankstatement()). An account, as modeled by an account class, can have three different states, namely incredit, overdrawn and blocked determined by its balance. Figure 1 shows the specification of class account in form of a CSM. In this figure each state of class account is represented by a circle, while each transition is depicted through an arrow leading from its source state to its target state. These transitions are formally specified through 5-tuples (source; target; event; guard; action) below this figure. A transition consists besides a source and target state of an event causing the transition, a predicate guard which has to be fulfilled before the transition can occur, and the action defining the operations on the attributes during the transition. There are also two special circles labeled initial and f inal. These two circles represent the state of the object before its creation and after its destruction, respectively. Thus, they represent states in which the attributes and their values are not defined, meaning that these two states are no concrete states of the object. The source code of class account, built according to the specification in figure 1, is given in figure 2. 2.3 Class specification implementation graph Our method proposed for integrated white- and blackbox testing operates on a graphical representation of the class called class specification implementation graph (CSIG) [3]. The construction of a CSIG consists of three steps: 1. A prototype is constructed for each event type occurring within transitions of the CSM. 2. Then, control flow graphs are generated for each prototype and each method. A CCFG frame is built and the two control flow graphs of each method are inserted into this frame. 3. Def-use pairs for black-box testing are determined on the basis of the CSM and visualized by data flow edges. Each of these steps is described in detail below. initial t 1 overdrawn t 6 t 7 t t t t 2 3 4 5 incredit t 21 t 14 t 19 t 12 t 13 t 18 t 20 blocked t 8 t 9 t 10 t 11 t 15 t 16 t 17 t 22 final t 1 =(initial; incredit; account();true;balance=0.0;); t 2 =(incredit; incredit; deposit(amount); true; balance+=amount;); t 3 =(incredit; incredit; withdraw(amount); balance amount 0; balance-=amount;); t 4 =(incredit; incredit; payinterest();true; balance=ic*balance;); t 5 =(incredit; incredit; printbankstatement(); true; print); t 6 =(incredit; overdrawn; withdraw(amount); limit» balance amount < 0; balance-=amount;); t 7 =(overdrawn; incredit; deposit(amount); balance + amount 0; balance+=amount;); t 8 =(overdrawn; overdrawn; deposit(amount); balance + amount < 0; balance+=amount;); t 9 =(overdrawn; overdrawn; withdraw(amount); limit» balance amount < 0; balance-=amount;); t 10 =(overdrawn; overdrawn; payinterest(); balance + i d1 balance limit; balance+=id1*balance;); t 11 =(overdrawn; overdrawn; printbankstatement(); true; print); t 12 =(overdrawn; blocked; withdraw(amount); balance amount < limit; balance-=amount;); t 13 =(overdrawn; blocked; payinterest(); balance + i d1 balance < limit; balance+=id1*balance;); t 14 =(blocked; overdrawn; deposit(amount); limit» balance + amount < 0; balance+=amount;); t 15 =(blocked; blocked; deposit(amount); balance + amount < limit; balance+=amount;); t 16 =(blocked; blocked; payinterest(); true; balance+=id2*balance;); t 17 =(blocked; blocked; printbankstatement(); true; print); t 18 =(incredit; blocked; withdraw(amount); balance amount < limit; balance-=amount;); t 19 =(blocked; incredit; deposit(amount); balance + amount» 0; balance+=amount;); t 20 =(incredit; final; finalize(); true; ); t 21 =(overdrawn; final; finalize(); true; ); t 22 =(blocked; final; finalize();true;): Figure 1. Specification of class account by a class state machine 2.3.1 Constructing a prototype for each event type The construction of a prototype firstly consists of generating a prototype for each transition t = (source; target; event; guard; action) in form of a nested if-then- construct as shown in figure 3. predicate(source) refers to the predicate of the source state applied to attributes defining the occurrence of state source. For instance, predicate(incredit) is balance >= 0.0. After generating these prototypes, those having the same

1 package bank; 2 3 class transaction { 4 String desc; 5 double amount; 6 double balanceafter; 7 8 transaction(string desc_, double amount_, 9 double balanceafter_) { 10 desc = desc_; 11 amount = amount_; 12 balanceafter = balanceafter_; 13 } 14 } 15 16 class account { 17 private double balance; 18 private double limit; 19 private transaction[] t; 20 private int idx; 21 private double ic, id1, id2; 22 23 public account() { 24 balance = 0.0; 25 limit = -500.0; 26 t = new transaction[20]; 27 idx = 0; 28 ic = 0.05; 29 id1 = 0.1; 30 id2 = 0.2; 31 } 32 33 public void deposit(double amount) { 34 balance += amount; 35 t[idx++] = new transaction("deposit ", 36 amount, balance); 37 } 38 39 public void withdraw(double amount) { 40 if (balance >= limit) 41 balance -= amount; 42 t[idx++] = new transaction("withdraw", 43 amount, balance); 44 } 45 46 public void payinterest() { 47 double amount; 48 if (balance >= 0.0) { 49 amount = ic*balance; 50 balance = amount; 51 } 52 53 if (balance >= limit) { 54 amount = id1*balance; 55 balance += amount; 56 } 57 { 58 amount = id2*balance; 59 balance += amount; 60 } 61 t[idx++] = new transaction("interest", 62 balance, amount); 63 } 64 65 public void printbankstatement() { 66 System.out.println("Type\t\tAmount\tBalance"); 67 for (int j = 0; j < idx; j++) 68 System.out.println(t[j].desc + "\t" 69 + t[j].amount + "\t" + t[j].balanceafter); 70 } 71 } Figure 2. Java source code of class account if (predicate(source)) // state if (guard) // guard action; // action Figure 3. Prototype of a transition depositspec(amount) { if (predicate(incredit)) if (true) if (predicate(overdrawn)) if (balance + amount >= 0.0) if (balance + amount < 0) if (predicate(blocked)) if (limit <= balance + amount && balance + amount < 0) if (balance + amount < limit) if (balance + amount <= 0) } Figure 4. Prototype of method deposit() event type are combined. For instance, transitions t 2, t 7, t 8, t 14, t 15 and t 19 share event type deposit(). Their prototypes can be merged to the prototype depicted in figure 4. Having represented each method in this way, the identification of definitions and uses of attributes is obvious due to the simple and pre-defined structure of a prototype. After the identification, test cases covering the identified def-use pairs can be generated. Although the test cases are identified on the basis of prototypes, they are intended for (blackbox) testing of the corresponding class method. Although test cases are generated by structural techniques, usually used for white-box testing, they are black-box test cases as the prototypes have been generated solely on the basis of the specification. 2.3.2 Generating and inserting control flow graphs into a CCFG frame This step consists of generating control flow graphs of the methods and prototypes, a CCFG frame, and finally linking them together to a CSIG. The CSIG of the account class is shown in figure 5. A CSIG is a directed graph visualizing both control and data flow within a class. Each method is represented by two types of control flow graphs. One of them is a control flow graph of the prototype generated on the basis of the specification, whereas the other is a control flow graph determined using the source code of the method. The control flow graphs are interlinked with each other by control and data flow edges. Three types of control and data flow edges can be distinguished: Intra-method control and data flow edges This type of edges visualizes control and data dependencies within a single control flow graph. For instance, an intra-method data flow edge connects a node representing

a 2 g 2 g 7 a 7 a8 34 35 g 8 24 25 26 27 a 1 account() s 1 g 1 frame entry frame call constructors 28 frame return constructors CCFG frame 29 control flow graph of a prototype frame loop 30 control flow graph of a method frame call deposit() withdraw() payinterest() printbankstatement() s i g i a i intra-method control flow edge inter-method control flow edge inter-method data flow edge specification-implementation edge state i guard i action i change to the state s 2 s 7 g 14 s 14 a 14 a15 g 15 a 3 s 3 g 3 s 9 g 6 g 9 a 6 g 18 a 9 g 12 a18 a12 a 4 s 4 g 4 s 10 g 10 a 10 a13 g 13 a 16 g 16 s 16 s 5 g 5 s 11 a 5 g 11 17 a 11 g 17 s a 17 g 19 a 19 40 41 42 48 49 53 50 54 55 66 67 58 68 59 61 frame return frame exit Figure 5. Class specification implementation graph of class account

an attribute definition with another node representing a use in the same method. Intra-method control flow edges are drawn as thin arrows, whereas intra-method data flow edges have been omitted in figure 5. Inter-method control and data flow edges Edges of this type model control and data flow between control flow graphs of the same type. For instance, an invocation of a method within another is modeled by an intermethod control flow edge leading from the node representing the invoking statement in the first method to the node representing the entry statement of the second method. Similarly, an inter-method control flow edge also models triggering an event within the action part of a transition to cause a second transition. Inter-method control flow edges connect both control flow graphs of methods with each other and control flow graphs of prototypes with each other. Contrary to inter-method control flow edges, inter-method data flow edges only connect nodes within control flow graphs of prototypes. As stated above, the objective of prototypes is to ease identification and association of definitions and uses of attributes. Thus, inter-method data flow edges are not required for method control flow graphs. In figure 5, inter-method edges are drawn by bold arrows. Inter-method control flow edges also connect the frame call node of the CCFG frame with the entry nodes of the various methods. Inter-method data flow edges are shown by gray arrows in figure 5. Note that inter-method data flow edges which do not represent def-use pairs are not required. Specification-implementation edges This type of edges visualize the connection between specification and source code by connecting the two control flow graphs of a method. Thus, a specification-implementation edge leads from the node representing the action within a prototype to the node representing the entry node of the corresponding method. Specification-implementation edges are drawn as dashed arrows in figure 5. 2.3.3 Adding data flow edges for black-box testing Definitions and uses of attributes are determined using the CSM of the class according to the ideas of Hong et al. [5]. In our example, the following action parts of transitions contain an assignment to the attribute balance: a 1, a 2, a 3, a 4, a 6, a 7, a 8, a 9, a 10, a 12, a 13, a 14, a 15, a 16, a 18 and a 19. Uses of this attribute are located in the action parts: a 2, a 3, a 4, a 6, a 7, a 8, a 9, a 10, a 12, a 13, a 14, a 15, a 16, a 18 and a 19. Furthermore, uses are also contained in the following guards: g 3, g 6, g 7, g 8, g 9, g 10, g 12, g 13, g 14, g 15, g 18 and g 19. A use of the attribute balance can also be found in predicates defining the states: incredit, overdrawn and blocked. 1 algorithm SelectTests(G; G 0 ;T) 2 input G : Node in the CSIG of initial prototype 3 G 0 : Node in the CSIG of the modified prototype 4 T : Test cases used for testing the initial prototype 5 output T 0 : Test cases selected for testing the modified prototype 6 begin 7 further traversal = true 8 mark G as G 0 -visited 9 compare G and G 0 10 if G 0 is modified 11 // test cases for testing single methods 12 if G 0 respresents a simple assignment 13 for all intra-method data dependent successor N 0 of G 0 do 14 add all test cases covering G and N 0 to T 0 15 16 add all test cases covering G to T 0 17 further traversal = false 18 // test cases for testing method interaction 19 for all inter-method data dependent predecessor N 0 of G 0 do 20 add all test cases (t 1;t 2) to T 0, with t 1 covering N 0 and t 2 G 0 21 for all inter-method data dependent successor N 0 of G 0 do 22 add all test cases (t 1;t 2) to T 0, with t 1 covering G 0 and t 2 N 0 23 if further traversal 24 associate control dependent successors of G to successors of G 0 25 for all unassociateable successor N of G 26 add all test cases covering N to T 0 27 for all associateable successor N of G 28 T 0 = T 0 [ SelectTests(N;N 0 ;T) 29 return T 0 30 end Figure 6. Algorithm for test case selection After identification, these definitions and uses are associated with each other according to data flow criteria. Often, the definition and use of a def-use are within two different methods. Therefore, a test cases consists of a sequence of method calls, including the methods which define and use the attribute. Usually, methods cannot be invoked in an arbitrary order, this means associating definitions and uses has to take into account the possible method sequences. After identifying a definition in method A, the corresponding use has to be within a method B which can be invoked immediately after the invocation of method A. Immediately means that the second method has to be invoked before the attribute is redefined by a definition in another method. The possible method sequences are modeled by the CSM of the class. For instance, the all-definitions criterion, which requires testing of each definition of an attribute at least once by a use [6], could be fulfilled by test cases covering the following def-use pairs: (a 1 ;g 3 ), (a 2 ;g 3 ), (a 3 ;a 6 ), (a 4 ;s 5 ), (a 6 ;g 12 ), (a 7 ;a 3 ), (a 8 ;a 8 ), (a 9 ;a 9 ), (a 10 ;s 9 ), (a 12 ;a 15 ), (a 13 ;a 16 ), (a 14 ;s 4 ), (a 15 ;a 15 ), (a 16 ;a 16 ), (a 18 ;a 14 ) and (a 19 ;s 3 ). For instance, note that the definition in action a 6 cannot be associated with use in action a 18, since transition t 6 changes the state of an account object to state overdrawn, whereas transition t 18 requires state incredit. These def-use pairs are inserted in figure 5 as gray drawn arrows leading from the definition of the attribute to the corresponding use.

2.4 Using CSIGs for regression testing The CSIG of a class can be used in various ways for testing. Generally, a CSIG can be used to generate test cases which fulfill some coverage criteria including the above identified def-use pairs. Following the ideas of Rothermel et al., CSIGs can also be used to verify changes between two versions of a class. An algorithm implementing this strategy for testing classes is given in figure 6. The algorithm in figure 6 takes as input the root nodes of the CSIGs of the two class versions and a set of test cases which has been used for testing the former version of the class, after having removed obsolete test cases. Based on this input, it selects test cases for safe regression testing of the latter version. The selected test cases can be distinguished in two groups. The first group contains test cases required for white-box testing. Each of these test cases consists of an input for the constructor of the class, if one is required, and an input for the changed method. The second group contains test cases required for black-box testing. These test cases consist of an input for the constructor, an input for the method in which the value of an attribute is set, and an input for the method in which the value of this attribute is used. To ease the notation, test cases for whitebox testing are represented by a single input, whereas test cases for black-box testing are represented by a pair of test inputs, assuming the constructor requires no input. In the following description of the algorithm, nodes in a CSIG are referred by upper-case letters, nodes in the latter version are marked by a prime. Starting with two nodes G and G 0, the algorithm marks as a first step G as G 0 -visited to avoid multiple traversal of the same nodes (line 7). Next, it compares both nodes. A modification of the statement represented by G 0 is handled according to the type of statement represented by this node (line 12). A modification in a simple assignment is handled by adding all test cases covering this simple assignment and data-dependent successors to T 0 (lines 13 14). A modification of a statement which is not a simple assignment is handled by adding all test cases covering this statement to T 0 (line 16). In the latter case, no further traversal of the successor nodes is required, because all test cases which can reach these nodes are already selected (line 17). So far, only test cases for white-box testing were selected. Test cases for black-box testing are selected in the next five lines of the algorithm, which is carried out by testing inter-method data flow. The corresponding edges connect a definition in a method A with a use in another method B. In some cases, A and B are the same method, but invoked twice first to define the value of the attribute and then to use this value. In lines 18 22, test cases are determined by considering first inter-method data dependent predecessors and then inter-method data dependent successors. After having determined test cases for node G 0, its successors are considered if further traversal is necessary (line 23). Its successors are associated to those of node G (line 24) and are distinguished between those which cannot be matched to successors of G (lines 25 26) and those which have corresponding counterparts in the former version of the class (line 27 28). 3 Conclusions In this article, we have presented a new approach for class-level regression testing. The main constituent of this approach is a graphical respresentation called class specification implementation graph (CSIG) which combines information gathered from both the specification and implementation for analysis and testing purposes. Furthermore, testing techniques based on control flow graphs can also be used, with some adaptions, in conjunctions with CSIGs, as CSIGs include control flow graph information. We have shown this by adapting the test case selection algorithm of Rothermel et al. to CSIGs. References [1] S. Beydeda and V. Gruhn. Integrating white- and black-box techniques for class-level testing object-oriented prototypes. In SEA Software Engineering and Applications Conference (Las Vegas, Nevada, November 6 9), pages 23 28, 2000. [2] S. Beydeda and V. Gruhn. An integrated testing technique for component-based software (accepted). In AICCSA ACS/IEEE International Conference on Computer Systems and Applications (Beirut, Lebanon, June 26 29), 2001. [3] S. Beydeda, V. Gruhn, and M. Stachorski. A graphical representation of classes for integrated black- and white-box testing (accepted). In ICSM International Conference on Software Maintenance (Florence, Italy, November), 2001. [4] H. Y. Chen, T. H. Tse, F. T. Chan, and T. Y. Chen. In black and white: an integrated approach to class-level testing of objectoriented programs. ACM Transactions on Software Engineering and Methodology, 7(3):250 295, July 1998. [5] H. S. Hong, Y. R. Kwon, and S. D. Cha. Testing of objectoriented programs based on finite state machines. In Proceedings of the Second Asia-Pacific Software Engineering Conference (Brisbane, Australia, December 6 9), pages 234 241, 1995. [6] S. Rapps and E. J. Weyuker. Selecting software test data using data flow information. IEEE Transactions on Software Engineering, 11(4):367 375, Apr. 1985. [7] G. Rothermel, M. J. Harrold, and J. Dedhia. Regression test selection for C++ software. Software Testing, Verification & Reliability, 10(2):77 109, June 2000.