Question 1: What is a code walk-through, and how is it performed?

Transcription

1 Question 1: What is a code walk-through, and how is it performed? Response: Code walk-throughs have traditionally been viewed as informal evaluations of code, but more attention is being given to this valuable form of quality improvement. A better label for this exercise is a code inspection. A typical code inspection consists of a group of people assigned to review a section of code in an application. This inspection is scheduled for a time after the developers feel the code is complete but prior to integration with other parts of the application. The inspection team usually consists of at least two people with one serving as the recorder to take notes and make comments in the code. The focus of the inspection is to ensure the design requirements are met. The purpose is to provide feedback on the code, not to fix any defects. The most senior engineers are typically the best equipped to perform the inspection. The deliverable for a code inspection is a set of code that has sufficient notations to allow the developer to correct any derivation from the design and modify areas where the implementation of the code may result is unnecessary complexity or risk to the project. Question 2: How can automated testing tools be used in unit testing? Response: When the project is small, the time required to learn and use automated testing tools may be too high to justify their use. Automated testing can be valuable for larger projects. The most common automated tools feature record and playback capabilities. To prepare this type of tool, the tester navigates through the application and the clicks and keystrokes are recorded and saved. The recorded script can be modified if necessary, and the result is a script that can be played back later to automatically operate the application. This level of automated testing serves well for completed applications, but is not of great use in the unit testing phase. Data-driven automated testing tools can be more valuable for unit testing. Data-driven testing tools use test drivers and data files to drive the datacentric parts of an application. These types of tools can be effectively used to test units of code prior to integration with the user interface layer. Some unit testing tools can be applied directly to the unit testing needs. These tools include code analyzers that examine code complexity and adherence to standards, tools that examine the coverage provided by unit tests, memory analysis tools, and load testing tools. Because of the complex 1

2 nature of many applications, typical tools may be difficult to use for unit testing. In these cases, custom tools are often written to ensure unit testing coverage is adequate. Question 3: How can I select a good test automation tool? Response: No automated testing tool will serve all of the testing needs for an application. The tools that can be used effectively will require significant work to implement with any application that is more than just a simple program. A good start in the selection of an appropriate automated testing tool is to examine the manual testing process and determine which parts can be best addressed by automation. In manual testing, the tester must exercise various parts of the application in different ways to determine if the specifications are met and the software is behaving properly. The tester must examine results in the form of data produced by the application, user interface changes, program flow, and so forth. When automated testing tools are used, the human factor is removed, so it is important to remember that all examinations of the program results must be possible in a clear way. Judgment cannot be part of the process unless a tester will examine results files manually. Without the human judgment component, the testing process must be viewed differently. Planning and analysis are key factors in successful selection of an automated testing tool. The first step is to understand what parts of testing will be the focus of automation. More complete coverage, load testing, faster testing, or replication of testing for regression purposes are good candidates for automated testing. It is a good idea to look for places in the manual testing processes that require lots of time. Another hot spot is areas of testing that appear to miss a lot of defects. Repeated testing can benefit from automation, and tests that require significant data manipulation will be more efficient with automation. After the targets for test automation are identified, candidate tools can be reviewed to determine which ones meet the identified needs. The candidates can be processed using various criteria such as cost, quality, support, ease of use, and so forth. When the list has been narrowed, trial versions can be used to establish which one best serves the needs of the testing process. The entire selection process may be time-consuming, but test automation can provide significant benefits to offset the investment. 2

3 Question 4: How does object-oriented design affect the unit testing process? Response: There are many factors that make testing of object-oriented software different from traditional function-oriented software. Object-oriented software is comprised of objects that mimic their real-world counterparts and are built as black boxes. Information is hidden within the objects, and it is generally not possible to see this information, only the results of changes made to the object that are designed to be visible. Information hiding makes it difficult to test changes in the state of an object. Objects also encapsulate their functionality, so the view of the object is a very surface-level view. Another aspect of objects is that they can inherit many of their attributes (data) and methods (functionality) from other objects. To make matters more complicated, the child objects can also implement the same named functionality in different ways. This inheritance capability requires that the same functionality is tested in the parent and child objects. In some ways, object-oriented software provides easier mechanisms than traditional software for unit testing. The requirement for clear interfaces to the objects allows test engineers to more easily create wrappers to utilize objects and exercise their exposed methods. The black box nature of objects also allows testers to compartmentalize objects into more discrete units and determine if the exposed functionality meets the design and functionality requirements. Because objects are more independent than traditional code, relationships with other parts of an application can be more easily determined and excluded from unit testing processes. Question 5: How does extreme programming (XP) change how unit testing is performed? Response: XP actually relies heavily on unit testing. Unit tests are typically created and used for a test-driven development process that is vital to the overall success of the XP approach. Unlike traditional development in which a tester would prepare a set of unit tests, XP relies on the developer to write unit tests along with the unit of code as development progresses. The unit test will initially fail because the code has not yet been developed, but the test itself is written to cover one or more of the requirements of the software. After the test is written, then the developer creates the code that will allow the test to pass. 3

4 Every class in the application is unit tested in this fashion. The unit test always accompanies the code and can then be used at any point in the future to retest the unit if any changes are made to the underlying code. When pairs of programmers work together in an XP development environment, one programmer often writes the unit tests while the other implements the code. The close association between coding and unit testing results in a high level of code coverage and a substantial reduction in defects later in the process. Question 6: Can programmers perform their own unit testing sufficiently? Response: Unit testing examines the implementation of the software more than the functionality. The focus on the code requires an understanding of the implementation and the design used as the basis for the code. Because of this knowledge requirement, someone with coding knowledge and an intimate understanding of the implementation must be involved in the unit testing process; however, there are drawbacks if the same person who implements the code also performs the unit testing. Familiarity with the code can cause the tester to miss issues that may appear more obvious to someone with less exposure to the code on a regular basis. Independent testing can provide a fresh look at the code and can also establish a better overview of the relationship between the code and the intended design. Independent testing can also be less efficient because of the learning curve required. A combination of unit testing performed by the developers and independent testing through code reviews can achieve the optimum overall results. Question 7: How much code coverage is enough in unit testing? Response: The goal of unit testing is to make sure the code will behave properly and to also ensure that the code meets the requirements as specified in the technical design. If these objectives are met, then functional testing in later stages can focus on the larger application issues. The key to success during the unit testing phase is to provide sufficient code coverage to maximize the confidence levels. This goal does not mean every line of code must be unit tested, which would require excess effort in which incremental gains in quality are not realized. The question is how much coverage is enough? The test designer must determine whether the unit tests cover the range of operations for a unit of the code. In addition, unit tests should be designed to 4

5 include any special cases. For example, unit tests should cover minimum, maximum, and out-of-range values for parameters to a function, but testing all possible values for input is not required. The tests should be designed to determine if the code performs as intended in the design and that the implementation was done according to accepted standards and goals for efficiency in execution. It is important to understand the meaning of coverage. Coverage can include individual code statements, decision structures, or function results. The level of risk associated with potential problems in an area of code can also be important in decisions regarding adequate code coverage. Ultimately, unit tests should ensure all lines of code are executed at least once and all conditional branches operate as intended. Beyond this level of code use, additional tests will be redundant and unnecessary. In practice, if the level of risk is low for an area of code, the effort required to perform complete coverage may exceed the value achieved from the tests. Complexity can also factor into the requirements for code coverage. If a unit is more complex, then it is more likely to have errors and increased coverage should be applied. For most projects, the goal is to make sure the code performs all decisions correctly and that each statement is performed once. Complete coverage is normally difficult to achieve because some code is only in place to handle situations that should not occur in normal operation of the code. For example, error handling may cover a scenario in which data storage is exceeded but this situation would not occur during normal operation of the program. The more difficult to test units can be covered through code reviews. Question 8: How can I determine if my unit testing strategy is sufficient? Response: There are several criteria that can be used to determine if a unit testing strategy is sufficient for an application. Recall that unit testing aims to ensure that the code meets the requirements specified in the design. These requirements may include standards for code implementation and specific use of data and unit functionality. The unit testing strategy relies on the availability of proper design and standards documentation. The unit testing strategy should also integrate well with other phases of the testing process. Complex and high risk-units should be identified so they receive appropriate attention during the testing process. The unit tests should also be reviewable, repeatable, and archivable for documentation and later use. 5

6 There are some more specific issues to address in the evaluation of a unit testing strategy. The plan should ensure all statements are exercised by at least one test (and preferably only one test). All branches of conditional statements and the extents of boundary conditions should also be exercised. If the design included any assumptions, the tests should make sure these assumptions are valid. Finally, the tests should make sure the unit has been tested over the complete range of operation for which it was intended. Question 9: What are some of the limitations of unit testing? Response: Testing of any sort will not find all defects in a software application. Unit testing is a subset of the overall testing process and will catch a subset of the defects. Because unit testing addresses only the individual units, there are some defects that will be excluded from coverage by definition. Integration errors or functional issues in larger combinations of individual units will not be caught. Because of these limitations, unit testing should only be considered to be part of the overall testing strategy. Another limitation of unit testing is shared with all forms of testing. Unit tests can identify defects, but they cannot be used to prove that no defects remain. Another issue with unit tests is the level of effort required to prepare the tests versus the effort expended in writing the code. As a simple illustration, consider a conditional statement where only a Boolean value of true or false can result from exercise of the code. This conditional statement may only be one line of code but it would require two tests to examine the correct operation of the statement. This combinatorial problem makes it unrealistic to test all inputs for every unit in a program unless some level of automation is used, but the automation tools bring their own problems to the test process. Question 10: How do unit testing frameworks work, and why are they important? Response: A unit testing framework is a system that helps prepare and exercise unit tests. With the use of code frameworks such as Java Development Kit (JDK) or.net, programs have continued to rely more heavily on libraries and language-specific implementations to achieve better programming efficiency and functionality. Unit testing frameworks are designed to help wrap the units in modern software to provide more efficient development of appropriate unit tests. 6

7 Unit testing frameworks must be created for specific languages, and there are many choices available for the more popular coding languages and environments. While it is not required to use these frameworks, the development of all the code required for adequate unit testing would be a difficult and time-consuming task. Incorporation of unit testing frameworks directly in the code provides the ability to use language-specific assertion and exception mechanisms to test the code. These frameworks are often used in partnership with automated testing tools to provide efficient testing processes with good code coverage. 7