A Class-Level Unit Testing Tool for Java

Transcription

1 A Class-Level Unit Testing Tool for Java Tz-Fan Hu and Nai-Wei Lin Department of Computer Science and Information Engineering National Chung Cheng University Chiayi 621, Taiwan, R.O.C. Abstract Software testing is the main activity to ensure the quality of software. This article applies a black-box testing technique to automatically generate test cases for Java programs. This article generates test cases for Java classes based on the software specification using the UML state diagrams and the Object Constraint Language (OCL). This article is based on the UML class diagrams and the Object Constraint Language to specify the behavior of Java methods by defining the preconditions and postconditions of Java methods and class invariants proposed in the Design by Contract software development approach. This article then uses the UML state diagrams and the Object Constraint Language to specify the dynamic behavior of Java objects. An automatic testing tool is developed to generate Java test classes for the Java classes under test. The testing framework is based on the JUnit framework. The automatic generation of the test input and expected output for each test case is based on the powerful constraint solving capability of constraint logic programming. A novel characteristic of this approach is that a test input and its corresponding expected output are simultaneously generated in a unified way. 1. Introduction According to 2006 CHAOS report of the Standish Group, high quality software is still very hard to achieve at present [15]. The software testing activity still remains one of the main activities to assure software quality. The cost of the software testing activity usually takes about half of the cost of the entire software process. This high cost of the software testing activity is the main reason to hinder the achievement of high quality software. Many software testing techniques have been developed during the last several decades. However, most of these software testing techniques are still performed manually. This is why the cost of the software testing activity is so high. The automation of the software testing activity should be able to significantly reduce the cost of the software testing activity. The software testing activity consists of the design of test cases and the execution of test cases. There are two types of techniques for designing test cases: black-box testing (or specification-based testing) and white-box testing (or program-based testing) [3]. The black-box techniques are based on the functional specification of a software system and focus on the coverage of the specified external behaviors of the software system. The white-box techniques are based on the source code and focus on the coverage of the internal structure of the source code. These two types of techniques are complementary. The white-box techniques alone may fail to reveal that some portions of the specification are missed in the source code. On the other hand, the black-box testing techniques alone may fail to reveal that some portions of the source code are not contained in the specification. Therefore, employing both types of techniques is necessary to assure high quality software. Design by Contract is a systematic approach for developing bug free software [8]. In Design by Contract approach, the functional behaviors of a class are specified as a contract. The contract specifies the correct interactions between a class and its clients as a set of obligations that need to be fulfilled by both parties. If both parties fulfill their

2 obligations, the interactions between the two parties are guaranteed to be correct. Developing the contract of a software system is the first and perhaps the most difficult task of developing the software system. However, if we cannot develop the contract of a software system, it is little likelihood that the developed software system will behave as it is supposed to behave. In Design by Contract approach, three kinds of constraints are used to specify the functional behaviors of a class. The preconditions of a method in the class specify the set of constraints that need to be satisfied before the method can correctly execute. The postconditions of a method in the class specify the set of constraints that is guaranteed to be satisfied after the method is correctly executed. The invariants of a class specify the set of constraints that is always satisfied by an object of the class. Unified Modeling Language (UML) is a standardized general-purpose modeling language. A UML class diagram (CD) describes the attributes and the methods in a Java class. Object Constraint Language (OCL) is a specification language that can be used to specify the three kinds of constraints in Design by Contract approach [11]. Given the contract for a Java class specified in OCL, the test cases for the methods in the class can be automatically designed using the black-box technique. The contract in the Design by Contract approach specifies the functional behavior of a Java object for a single method invocation. The UML state diagrams (SD) and the OCL specification on the state diagrams can be used to specify the functional behavior of a Java object for a sequence of method invocations. Thus, a specification including class diagrams, state diagrams, and OCL is a more complete specification for a class. The structure of a class-level unit test tool for Java is shown in Figure 1. This tool contains three components: a test path generator, a test data generator, and a test class generator. The test path generator first reads in a UML class diagram, a state diagram, and the corresponding OCL specifications for the methods in the class. It then enumerates a possible test path from the specification. The test data generator then checks the feasibility of the test path by solving the set of logical constraints appearing on the test path. If the set of logical constraints is satisfiable, this test path is feasible. In this case, a solution of the set of logical constraints is used as a representative test input and its corresponding expected output. Otherwise, the test path is infeasible and is abandoned. The test path generator and the test data generator perform repeatedly until the set of generated feasible test paths satisfies a specific test coverage criterion. The test class generator then generates a Java test class for the tested Java class based on the set of generated test inputs and their corresponding expected outputs. Most modern black-box unit testing tools focus on the automation of test path generators and the test class generator. The tasks of test data generators are still usually performed manually by programmers or testers. This article proposes an approach that can automatically generate the test input and expected output simultaneously based on the powerful constraint solving capability of the Constraint Logic Programming system ECLiPSe [1]. The automatic execution of test cases is based on the JUnit framework [2]. This article uses the Java development environment Eclipse to develop the testing tool [12]. The rest of the article is organized as follows. Section 2 describes a running example for this article and gives a specification for the running example. Section 3 describes the test path generator. Section 4 describes the test data generator. Section 5 describes the test class generator. Section 6 reviews related work. Finally, Section 7 concludes this article. 2. A Running Example We now describe a running example that is used to illustrate the automatic test case generation for Java classes. Consider the following Java class Stack.

3 UML CD, SD, and OCL Test Path Generator Test Path Infeasible Test Path Test Data Generator Feasible Test Path + Test Input + Expected output Coverage Criterion Satisfied Y N Test Class Generator Java Test Class Figure 1. The structure of a class-level unit testing tool for Java. class Stack { private Sequence stack; public Stack(); public void push(int element); public void pop(); public int peek( ); public int size(); } Each object of the class Stack represents a stack of integer elements. The variable stack represents the sequence of integer elements in the stack. We assume that the type Sequence is an abstract ordered-bag collection type that can be implemented using a suitable Java collection type. The constructor Stack() creates a Stack object with zero element. The method push(int element) pushes an element element into the stack. The method pop() pops an element out of the stack. The method peek( ) returns the top element of the stack. The method size( ) returns the number of elements in the stack. The methods can be divided into two categories: update and query methods. The update methods like push and pop will change the state of the object. The query methods like peek and size only return the status of the state of the object.

4 The OCL specification for the methods in the class Stack is given as follows: context Stack:: Stack() post: stack = Sequence{} context Stack::push(element: Integer) post: stack = stack@pre->prepend(element) context Stack::pop() pre: stack@pre->size() > 0 post: stack = stack@pre-> subsequence(2, stack@pre->size()) context Stack::peek(): Integer pre: stack@pre->size() > 0 post: result = stack@pre->first() context Stack::size(): Integer post: result = stack@pre->size() The type Sequence is a predefined ordered-bag collection type in the OCL. The postcondition for the constructor Stack() specifies that an empty sequence is created and is set to attribute stack. The postcondition for the method push(element: Integer) specifies that an integer element element is inserted into the sequence at the beginning. In the postcondition, the expression stack denotes the value of attribute stack right after the method invocation, while the expression stack@pre denotes the value of attribute stack right before the method invocation. The OCL predefined operation prepend(x) inserts its argument x into the sequence at the beginning. In the preconditions for the methods pop() and peek(), the OCL predefined operation size() returns the number of elements in the sequence. Thus, the preconditions specify that the number of elements in the sequence must be greater than zero. The OCL predefined operation subsequence(n, m) returns the subsequence from the n th element to the m th element. The postcondition for the method pop() thus specifies that the attribute stack is set to the push() push() empty nonempty pop()[size() = 1] pop()[size() > 1] Figure 2. The state diagram of class Stack. subsequence from the second element to the last element. The OCL predefined operation first() returns the first element of the sequence. The postcondition for the method peek() thus specifies that the method returns the first element of the sequence. The postcondition for the method size() thus specifies that the method returns the number of elements in the sequence. The state diagram for the class is given in Figure 2. This diagram specifies the dynamic behavior of a stack object. Note that only the update methods push and pop appear in the state diagram because the query methods peek and size do not change the state of the object. 3. The Test Path Generator The test path generator consists of two steps. First, it reads in a state diagram and its corresponding OCL specification. Second, it then enumerates all the possible test paths in the state diagram. The enumeration schemes of test paths depend on the test coverage criterion chosen. The test coverage criterion can be control-flow based: all-decision, all-condition, or all-decision-condition, or data-flow based: all-define, all-use, or all-define-use. In this article, we will assume that the all-define-use test coverage criterion is used. Therefore, the test path generator needs to find all the define-use pairs in the state diagram before enumerating test paths. For the running example, stack is the only

5 def 0 def 1 use 1 push() push() def 2 use 2 following example. %Include constraint solving library empty nonempty :- lib(ic). attribute in the class. All the definitions and uses of stack are shown in Figure 3. There are a total of 9 define-use pairs in the state diagram: def 0 -use 1, def 1 -use 2, def 1 -use 4, def 2 -use 2, def 2 -use 3, def 3 -use 2, def 3 -use 3, def 3 -use 4, def 4 -use 1. Using the all-define-use test coverage criterion, the following two paths will be enumerated by the test path generator: 1: push-pop-push-pop. 2: push-push-push-pop-push-pop-pop-pop. The first test path covers the following define-use pairs: def 0 -use 1, def 1 -use 4, def 4 -use 1. The second test path covers the following define-use pairs: def 1 -use 2, def 2 -use 2, def 2 -use 3, def 3 -use 2, def 3 -use 3, def 3 -use 4. These two paths will be passed to the test data generator. 4. The Test Data Generator The test data generator generates a sequence of arguments for the sequence of method invocations in each test path passed from the test path generator. The sequence of arguments are generated by solving the set of constraints on the test path. This set of constraints is first transformed into ECLiPSe predicates and these predicate are then solved by the powerful constraint solving capability of the ECLiPSe system. def pop()[size() = 1] 3 use 3 def 4 pop()[size() > 1] use 4 Figure 3. The definitions and uses of stack in the state diagram of class Stack. The powerful constraint solving capability of the ECLiPSe system can be illustrated by the foo(a, B, C) :- % Domains of variables [A, B, C] :: 1.. 3, % Constraints on variables A #= B + C, % Solving constraints indomain(a), indomain(b), indomain(c), locate([a, B, C], 0.1). where ic is the library for solving constraints on finite domains. The ECLiPSe system supports several other libraries. Based on the unification mechanism of logic programming, the predicate foo/3 can be queried with any number of variable arguments. For example, the predicate foo/3 can be queried using any of foo(3, 1, 2), foo(3, 1, C), foo(3, B, C), foo(a, B, C), and so on. This unification mechanism allows us to solve the test input and expected output simultaneously. By specifying the domains of variables, the constraint solving library can find all the solutions satisfying the constraints in the predicate. For the example, the ECLiPSe system will returns A = 3, B = 1, C = 2, and A = 3, B = 2, C = 1, as solutions for the query foo(a, B, C). For each method method in the class class, the test data generator will convert its postcondition into an ECLiPSe predicate classmethod(ostate, Arguments, Result, NState). The argument OState is the state of the object right before the

6 method invocation. The state consists of the list of attribute values of the object. The argument Arguments is the list of arguments to the method invocation. The argument Result is the returned value of the method invocation. The argument NState is the state of the object right after the method invocation. For the running example, the test data generator will generate the following predicates: stackstack(_, [], [], stack(s)) :- sequencesequence([], S). stackpush(stack(s),[element],[], stack(ns)): [Element] :: , sequenceprepend(s, Element, NS). stackpop(stack(s),[],[], stack(ns)) :- sequencesize(s, Size), sequencesubsequence(s, 2, Size, NS). stackpeek(stack(s), [], Element, stack(s)) :- [Element] :: , sequencefirst(s, Element). stacksize(stack(s), [], Size, stack(s)) :- sequencesize(s, Size). where the predicates sequencesequence/2, sequenceprepend/3, sequencesize/2, sequencesubsequence/4, and sequencefirst/2 are the CLP implementation of the builtin operations of the OCL sequence collection type. The test data generator will also convert the sequence of method invocations and guards on the test paths into an ECLiPSe predicate testclass(pathno, Arguments, Input, Output). The argument PathNo is the path identification number. The argument Arguments is the sequence of arguments in the method invocations. The argument Inputs is the list of arguments passed to the object state queries. The argument Outputs is the list of values returned from the object state queries. We should use both query methods peek and size to query the state of the object. However, due to the comprehension of presentation, we only use the query method size in this example. Each test path corresponds to a clause of the predicate testclass/4. For the running example, the test data generator will generate the following clause for the test path 1: teststack(1, Arguments, Inputs, Outputs) :- % clause for test path 1 stackstack(_, Arg0, _, S0), Arguments = [Arg0 R1], stacksize(s0, In1, Out1, _), % check Inputs = [In1 RIn1], Outputs = [Out1 Rout1], stackpush(s0, Arg1, _, S1), R1 = [Arg1 R2], stacksize(s1, In2, Out2, _), % check RIn1 = [In2 RIn2], Rout1 = [Out2 Rout2], stacksize(s1, Arg2, Res1, S2), % guard R2 = [Arg2 R3], Res1 #= 1, stackpop(s2, Arg3, _, S3), R3 = [Arg3 R4], stacksize(s3, In3, Out3, _), % check RIn2 = [In3 RIn3], Rout2 = [Out3 Rout3], stackpush(s3, Arg4, _, S4), R4 = [Arg4 R5], stacksize(s4, In4, Out4, _), % check RIn3 = [In4 RIn4], Rout3 = [Out4 Rout4], stacksize(s4, Arg5, Res2, S5), % guard R5 = [Arg5 R6], Res2 #= 1, stackpop(s5, Arg6, _, S6), R6 = [Arg6], stacksize(s6, In5, Out5, _), % check RIn4 = [In5 RIn5], Rout4 = [Out5 Rout5].

7 Query the ECLiPSe system using the following query: teststack(1, Args, Ins, Outs). The ECLiPSe system will return Args = [[], [0], [], [], [0], [], []], Ins = [[], [], [], [], []], Outs = [0, 1, 0, 1, 0]; 5. The Test Class Generator The test class generator generates a Java test class for the class under test. The test class is based on the JUnit framework. For each test path, there is a corresponding test method. For each selected feasible test path, the test class generator generates code to invoke the sequence of method invocations and to check if the actual output of the invoked method is the same as the expected output using the JUnit macro assertequals. For the running example, the test class generator generates the following test method for the first test path. public class TriangleTest extends TestCase { public void teststack1() { Stack s = new Stack(); assertequals(0, s.size()); s.push(0); assertequals(1, s.size()); s.pop(); assertequals(0, s.size()); s.push(0); assertequals(1, s.size()); s.pop(); assertequals(0, s.size()); } } 6. Related Work The Design by Contract is first proposed and designed in the programming language Eiffel [9]. The Eiffel contracts are part of the Eiffel programming language and can be executed directly. The AutoTest unit testing framework automates test case generation based on the Eiffel contracts [10]. The AutoTest framework uses the adaptive random testing to generate test input and executes the postcondition to generate expected output. The Java Modeling Language (JML) is a Design by Contract specification language for Java [6]. The JML contracts are specified in Java syntax. The JML unit testing framework automates test case generation based on the JML contracts [4]. The JML framework uses the random testing and the constraint-solving on the precondition to generate test input and executes the postcondition to generate expected output. The CASE tool AutoFocus is a model-based development tool for reactive systems [5]. The AutoFocus tool uses the System Structure Diagrams to specify the interaction structure of components and the State Transition Diagrams to specifies the behavior of a component. The AutoFocus tool uses the symbolic execution of constraint logic programming to generate test input and expected output [7, 13, 14]. Our approach is most similar to the approach of the AutoFocus tool. 7. Conclusion Automatic generation of input data and expected output is the most difficult tasks in software testing activity. The automation of any task depends on the formal specification of the task. This article uses UML state diagrams and the Object Constraint Language as a formal language to specify the dynamic behavior of Java objects. This article uses the powerful constraint solving capability of constraint logic programming to automatically generate test input and expected output for test cases. This approach can generate the test input and expected output simultaneously. This article shows that the integration of UML state diagrams, OCL, and CLP makes the design and implementation of an

8 automatic testing tool for Java classes possible. Acknowledgements This work was supported in part by the National Science Council of R.O.C. under grant number NSC E References [1] K. R. Apt and M. G. Wallace, Constraint Logic Programming Using ECLiPSe, Cambridge University Press, [2] K. Beck and E. Gamma, JUnit Cookbook, [3] B. Bezier, Software Testing Techniques, 2nd Edition, Van Nostrand, [4] Y. Cheon, A. Cortes, M. Ceberio, and G. T. Leavens, Integrating Random Testing with Constraints for Improved Efficiency and Diversity, Proceedings of the 20th International Conference on Software Engineering and Knowledge Engineering, [5] F. Huber, B. Schatz, G. Einert, Consistent Graphical Specification of Distributed Systems, Proceedings of the Conference on Industrial Applications and Strengthened Foundations of Formal Methods, 1997, pp [6] G. T. Leavens, A. L. Baker, and C. Ruby, JML: A Notation for Detailed Design, In H. Kilov, B. Rumpe, and I. Simmonds (editors), Behavioral Specifications of Businesses and Systems, Kluwer, 1999, Chapter 12, pp [7] H. Lötzbeyer, A. Pretschner, and Er Pretschner, AutoFocus on Constraint Logic Programming, Proceedings of (Constraint) Logic Programming and Software Engineering, [8] B. Meyer, Design by Contract, In Advances in Object-Oriented Software Engineering, eds. D. Mandrioli and B. Meyer, Prentice Hall, 1991, pp [9] B. Meyer, Eiffel: the Language, Prentice Hall, [10] B. Meyer, H. Ciupa, A. Leitner, and L. Liu, Automatic Testing of Object-Oriented Software, SOFSEM 2007: Theory and Practice of Computer Science, Springer, 2007, pp [11] Object Management Group, Object Constraint Language Specification, Version 2.0, [12] Object Technology International Incorporation, Eclipse Platform Technical Overview, [13] A. Pretschner and H. Lötzbeyer, Model Based Testing with Constraint Logic Programming: First Results and Challenges, Proceedings of Second International Workshop on Automated Program Analysis, Testing and Verification, 2001, pp [14] A. Pretschner, O. Slotosch, E. Aiglstorfer, and S. Kriebel, Model Based Testing for Real: The Inhouse Card case Study, International Journal on Software Tools for Technology Transfer, 5(2-3): , March [15] Standish Group, 2006 CHAOS Report, 2007.