TITAN 5300 Software. Unit Test Guidelines. S. Darling, S. Harpster, R. Hite, K. Konecki, W. Martersteck, R. Stewart. Revision 2.0

Transcription

1 TITAN 5300 Software Unit Test Guidelines S. Darling, S. Harpster, R. Hite, K. Konecki, W. Martersteck, R. Stewart Revision , Tellabs Operations, Inc., 4951 Indiana Ave., Lisle, IL (312)

2 Table of Contents 1 Introduction Purpose, Scope and Field of Application Purpose Scope Field Of Application Revision History References Definitions Approach to Unit Testing Testing Guidelines Generating Test Cases Find Equivalence Classes Boundary-value analysis Error Guessing Various White-Box Techniques Unit Test Scripts Debugger Command Files...5

3 1 Introduction It is extremely important that the unit testing phase of a software project be thorough. Good unit testing often takes more time and effort than many developers realize. Thorough unit testing will, however, eliminate many more bugs before the code passes on to a later development stage, where finding and fixing errors becomes much more difficult and expensive. 1.1 Purpose, Scope and Field of Application Purpose This paper is meant as an introduction to unit testing techniques for software developers Scope This document covers the unit testing phase of a software project Field Of Application This document was created for the TITAN 5300 software developers Revision History Rev Date Author Revision Description 1.0 5/12/89 S. Darling Released for the 532L FP1 development staff 2.0 6/7/94 R. Hite Modified format, updated information, and changed the name from Unit Test Procedures to Unit Test Guidlines 1.2 References [1] CodeCenter Reference Manual, Center Line, 1992 [2] The Art of Software Testing, Glenford J. Myers, John Wiley & Sons, 1979; ISBN [3] Test Script Standards, Stephen Harpster, 5300 DocLib, Definitions stub unit test unit test script A dummy subroutine that simulates the functiionality of another subroutine for testing purposes. Individually verifying each subroutine A written description of how to run the test and how to know if it succeeds or fails. Introduction Page 1

4 2 Approach to Unit Testing The 5300 unit testing is typically done by the same developer who wrote the code. This approach has some advantages: It is easier to schedule, since it doesn t depend on other developers being available for unit testing The coder typically understands the structure and operation of the code better, and can generate better test cases. When a test fails, the developer is usually able to track down the error easier and is more likely to fix the error without introducing new problems. The coder often feels better about turning over the code since it gives them a chance to clean up the code before other developers have to deal with it. On the other hand, there are some disadvantages: Bugs in the code are often due to incorrect assumptions on the part of the coder. If the same person does the testing, these assumptions are carried over into testing. The coder can sometimes be too closely attached to the code, and be reluctant to find bugs in it. They may feel as if a testing session that finds few errors in the code indicates that they did an excellent job in writing the code in the first place. Unit testing should be approached with the assumption that there are bugs in the code, and that a successful debugging session finds as many of these bugs as possible. To temper these disadvantages, all unit test scripts are reviewed at the time of code review. Reviewers are responsible for double-checking the assumptions and thoroughness of the tests. 3 Testing Guidelines The following guidelines have been adopted for 5300 projects. Developers may find that these guidelines may be inappropriate for individual cases; the final form of the unit tests is left up to the developer and reviewers. Unit testing will generally be done interactively with a debugger. The debugger for 5300 is CodeCenter [1], which runs on the SUN workstations. For example a simulated 532L environment will be provided for use under the debuggers. This environment will provide pre-defined calls for MTOS system calls, DBMS routines, inter-object communications, and other kernel functions. Unit testing will focus on verifying each function call individually. The primary method will be to use the debugger to 1) set up any global data, 2) call the function with specified parameters, then 3) check the return values and global data for correctness. Unit testing should be done incrementally, bottom up. Stubs should be used for all routines that are not part of the current unit testing (except, of course, calls to the operating system or other kernel routines). This should be done even if the real code exists, to minimize concerns about interfaces between different areas of the system. Approach to Unit Testing Page 2

5 The only time the real code should be used instead of a stub is when the operation of the called module is so complex that it would be infeasible to write a stub to simulate it. 4 Generating Test Cases The set of test cases should be chosen such that the probability of finding the bugs in a program is as high as possible without making the number of test cases prohibitively high. There is no hard-and-fast procedure for generating such a set of test cases, but the method outlined in this section provides a beginning. 4.1 Find Equivalence Classes The first way to generate test cases is to partition the input and output parameters for the function into equivalence classes. Briefly, this involves breaking the parameter values into ranges that should have essentially the same effect when the function is called. An example is given in below. int count... if (count < 3) { } printf( count is less than three\n ); else if(count <= 9) { } else { } printf( count is less than nine\n ); printf( count is greater than 9\n ); This section of code has three equivalence classes. The first class is the set of integers less than three. The second class is the set {3, 4, 5, 6, 7, 8, 9}. The third class is the set of integers greater than nine. Equivalence partitioning is more fully explained in the Myers [2] book. Generating Test Cases Page 3

6 When doing this analysis, global variables should be considered in addition to parameters given as part of the function call. Ideally, the test cases should be thorough enough to cover all combinations of equivalence classes; however this sometimes becomes unwieldy due to the sheer number of possible combinations. If exhaustive coverage is infeasible, the number of test cases can be pared down through several techniques: Re-examine the function call, considering whether too much information has been put into a single call. Cause-effect graphing, as described in Myers, may provide a way to trim down the number of cases. Leave out test cases, using the structure of the code as a guide for which ones to leave out. For instance, consider the function call: foo(a, b, c); Where, within the function foo, the parameter a is checked for validity before b and c are used; if a is invalid, foo executes a return. After writing test cases for invalid values of a, the rest of the test cases can probably focus on different values for b and c while using only valid values for a. 4.2 Boundary-value analysis Errors are often due to incorrect handling of values around their boundary values (that is, the values just inside and just outside of the border between two equivalence classes.) Tests that concentrate on boundary values can uncover a disproportionately high number of errors. Test cases generated through equivalence partitioning can be supplemented with tests concentrating on boundary values. Actually, if the tester kept boundary values in mind while writing the equivalence partition tests, many of these cases will already have been covered. The tester will simply need to augment the original tests to fill out the boundary value coverage. 4.3 Error Guessing The tests generated in the previous two stages can often be supplemented with tests based on error guessing. In this type of test, the tester simply relies on intuition about cases that seem unusual, unexpected, or otherwise highly likely to be mishandled by the function being tested. 4.4 Various White-Box Techniques Finally, the tester can add to the above test cases based on an examination of the code itself. The Myers book discusses several techniques for making sure that each path of control flow through the function is exercised. The test writer should examine the code looking for any major areas of code that may not have been sufficiently exercised through the tests generated in the previous steps. Generating Test Cases Page 4

7 5 Unit Test Scripts All tests should be explicitly described in a unit test script; a thorough, written description of how to run the test and how to know if it succeeds or fails. Unit Test scripts serve several purposes: They describe the tests to the reviewers at the code review. They give the tester a well-defined way of keeping track of which tests have been run. This is especially important when an error is found and the function is fixed; all the unit tests for that function should be run again to make sure they all still pass. They describe the tests to other developers who may run the tests. They document a basic set of tests that can be modified and rerun when the code is changed in the future. They serve as quality records An in depth description of a unit test script is given in Test Script Standards [3] which may be found in DocLib. 6 Debugger Command Files Because unit tests often need to be rerun, and because the set up of input conditions and checking of output conditions can be time-consuming, it is helpful to set up files that can be used in the debugger to set initial conditions, run the test, and check final conditions. The debugger being used for this project has user commands that tell the debugger to run a command file. In most cases, it will be most efficient in the long run if developers create a command file for each test case and use the command file to execute the test. This reduces mistakes caused by faulty test procedures and allows tests to be rerun at any time. Unit Test Scripts Page 5