Unit Testing with JUnit and CppUnit

Size: px

Start display at page:

Download "Unit Testing with JUnit and CppUnit"

Marsha Underwood
6 years ago
Views:

1 Unit Testing with JUnit and CppUnit Software Testing Fundamentals (1) What is software testing? The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of system or component. (IEEE Definition) Why do we test? To ensure the quality and satisfaction of the product To gain the confidence in the correctness of the product Testing vs. debugging Testing is to show that a program has bugs Debugging is to locate and correct the error or misconception that cause the program failures 2

2 Software Testing Fundamentals (2) Software Testing Techniques White-box testing The tester has access to the details of the program under test and performs the testing according to such details. Examples: path testing, branch testing Black-box testing Black-box testing treats the program under test as a black box. No knowledge about the implementation is assumed. Examples: equivalent partitioning, boundary value analysis 3 Software Testing Fundamentals (3) Testing should begin in the small and progress toward testing in the large Unit testing Concentrates on each unit (i.e., component) of the software Integrated testing Building a system from its components and testing the resultant system for detecting component interaction problems System testing Concern with testing an increment to be delivered or entire system Acceptance testing Performed by the clients/users to confirm that the product meets the business requirements 4

3 Unit Testing Unit Testing Is normally considered as an adjunct to the coding step Focuses verification effort on the smallest unit of software design the software component or module Using the component-level design description as a guide Provide a release criterion for a programming task Unit Testing in the OO Context Smallest testable unit is the encapsulated class Conducting class testing (i.e., unit testing) Methods within the class are tested The state behavior of the class is examined 5 Unit Testing Example: Statement coverage (1) class X { void f() { ; Statement coverage: are all statements executed? Function under test Test case(s) class XTest { void testf() { X x; x.f(); CPPUNIT_ASSERT( ); ; 6

4 Unit Testing Example: Statement coverage (2) class X { void f(bool x) { if (x) { ; There are unexecuted statements class XTest { void testf() { X x; x.f(false); CPPUNIT_ASSERT( ); ; 7 Unit Testing Example: Statement coverage (3) class X { void f(bool x) { if (x) { ; All statements are executed class XTest { void testf() { X x; x.f(true); CPPUNIT_ASSERT( ); ; 8

5 Unit Testing Example: Branch coverage (1) Unexercised class X { void f(bool x) { ; if (x) { Brach coverage: are all control transfer exercised? class XTest { All statements void testf() { are executed X x; x.f(true); CPPUNIT_ASSERT( ); ; 9 Unit Testing Example: Branch coverage (2) class X { void f(bool x) { if (x) { ; All control transfers are exercised Two test cases class XTest { void testf() { X x; x.f(true); CPPUNIT_ASSERT( ); x.f(false); CPPUNIT_ASSERT( ); ; Is reusing x a good idea? 10

6 Unit Testing Example: Branch coverage (3) class X { void f(int x) { do { x--; while (x>0); Statement ; coverage? Yes Branch coverage? No class XTest { void testf() { X x; x.f(0); CPPUNIT_ASSERT( ); ; 11 Advantages of Unit Testing Write test = understand better what has to be done Silly bugs appear immediately = less time spent on debugging Provide a working specification of your functional code Speed-up the development write test once, use test to find errors many times Gain confidence in your code Repeatable and deterministic Regression tests incorrect changes discovered immediately; unautomated refactoring enabled Encouraged by extreme Programming community 12

7 Old-fashioned Low-level Testing Stepping through a debugger drawbacks: 1. not automatic 2. time-consuming Littering code with stream output calls drawbacks: 1. makes code ugly (ok, nowadays you could use aspects to avoid it;-)) 2. generates to much information 13 What is xunit? An automated unit test framework Provides the Driver for unit(s) Provides automatic test runs Provides automatic result checks Available for multiple languages: JUnit (from Kent Beck (XP) and Erich Gamma(Gang of Four)) cppunit httpunit NUnit 14

8 The Goals of JUnit To write a framework within which we have some glimmer of hope that developers will actually write tests. The framework has to use familiar tools, so there is little new to learn. The second goal of testing is creating tests that retain their value over time. Someone other than the original author has to be able to execute the tests and interpret the results. Creating a setup or fixture is expensive A framework has to enable reusing fixtures to run different tests. 15 The Design of JUnit Patterns Generate Architectures Used to present the design of JUnit. The idea is to explain the design of a system by starting with nothing and applying patterns, one after another, until you have the architecture of the system. Presentation flow (1) Present the architectural problem to be solved (2) Summarize the pattern that solves it (3) Show how the pattern was applied to JUnit. The details can refer to JUnit A Cook's Tour 16

9 The Patterns Used in JUnit 17 How to Use JUnit (1) Write a test case (Fixture) Create your own test case as a subclass of JUnit TestCase Add an instance variable for each known object in the fixture Override the setup() method to initialize object(s) under test Override the teardown() method to release object(s) under test Run the test Define a public test???() method for exercising the object(s) under test Verify the result Assert expected result of the test case using assertequals(), asserttrue, Clean up the fixture through the teardown() method 18

10 How to Use JUnit (2) Suite management A test suite is a collection of test cases that are intended to be used to show that a program under test has some specified set of behaviors Write a static suite() containing all the test???() in the fixture class Define a main() method that runs the TestCase in batch mode Error vs. Failures Error: unanticipated problem like an ArrayIndexOutOfBoundsException Failure: is anticipated and can be checked with assertions 19 JUnit FAQ: Best Practices Test-first programming Tests should be written before the code. Test-first programming is practiced by only writing new code when an automated test is failing. When all the tests pass, you know you're done! When a bug is reported, first write unit test(s) to expose the bug(s), then fix them. This makes it almost impossible for that particular bug to resurface later. Good tests tell you how to best design the system for its intended use. Test-driven development is a lot more fun than writing tests after the code seems to be working. 20

11 JUnit FAQ: Best Practices Do I have to write a test for everything? No, just test everything that could reasonably break. Investments in testing are equal investments in design. If defects aren't being reported, and your design responds well to change, then you're probably testing enough. If you're spending a lot of time fixing defects and your design is difficult to grow, you should write more tests. If something is difficult to test, it's usually an opportunity for a design improvement. 21 JUnit FAQ: Best Practices How simple is too simple to break? If it can't break on its own, it's too simple to break. Example getx() method cannot break unless the compiler is also broken. Therefore, don't test getx(). setx() method is also too simple to break. However, if it does any parameter validation, you likely need to test it. 22

12 JUnit FAQ: Best Practices How often should I run my tests? Run all your unit tests as often as possible Ideally every time the code is changed. Make sure all your unit tests always run at 100%. Frequent testing gives you confidence that your changes didn't break anything. For larger systems, you may just run specific test suites that are relevant to the code you're working on. Run all the tests of the a system at least once per day (or night). 23 JUnit FAQ: Best Practices What do I do when a defect is reported? Write a failing test that exposes the defect When the test passes, you know the defect is fixed! This is a learning opportunity Perhaps the defect could have been prevented by being more aggressive about testing everything that could reasonably break. Why not just use print? It requires that output be scanned manually every time the program is run to ensure that the code is doing what's expected. Tests should retain its value over time. Why not just use a debugger? The same as that of using print. 24

13 JUnit FAQ: Best Practices Testing Idioms Code a little, test a little, code a little, test a little... Begin by writing tests for the areas of code that you're most worried about breaking. Write tests that have the highest possible return on your testing investment. When you need to add new functionality to the system, write the tests first. If you find yourself debugging using System.out.println(), write a test case instead. The next time someone asks you for help debugging, help them write a test. Don't deliver software that doesn't pass all of its tests. 25 CppUnit Cookbook (1) Simple Test Case (Ordinarily, you will not use such simple test) Subclass the TestCase class. Override the method runtest(). When you want to check a value, call CPPUNIT_ASSERT(bool) and pass in an expression that is true if the test succeeds class ComplexNumberTest : public CppUnit::TestCase { public: ComplexNumberTest( std::string name ) : CppUnit::TestCase( name ) { void runtest() { CPPUNIT_ASSERT( Complex (10, 1) == Complex (10, 1) ); CPPUNIT_ASSERT(!(Complex (1, 1) == Complex (2, 2)) ); ; 26

14 CppUnit Cookbook (2) Fixture - a known set of objects served as a base for a set of test cases To add new tests Add member variables for each part of the fixture Override setup() to initialize the variables Override teardown() to release any resources allocated in setup() class ComplexNumberTest : public CppUnit::TestFixture { private: Complex *m_10_1, *m_1_1, *m_11_2; protected: void setup() { m_10_1 = new Complex( 10, 1 ); m_1_1 = new Complex( 1, 1 ); m_11_2 = new Complex( 11, 2 ); void teardown() { delete m_10_1; delete m_1_1; delete m_11_2; ; 27 CppUnit Cookbook (3) How do you write and invoke individual tests using a fixture? Write the test case as a method in the fixture class Create a TestCaller which runs that particular method class ComplexNumberTest : public CppUnit::TestFixture { private: Complex *m_10_1, *m_1_1, *m_11_2; protected: void setup() {... void teardown() {... void testequality() { CPPUNIT_ASSERT( *m_10_1 == *m_10_1 ); CPPUNIT_ASSERT(!(*m_10_1 == *m_11_2) ); void testaddition() { CPPUNIT_ASSERT( *m_10_1 + *m_1_1 == *m_11_2 ); ; CppUnit::TestCaller<ComplexNumberTest> test("testequality", &ComplexNumberTest::testEquality ); CppUnit::TestResult result; test.run( &result ); 28

15 CppUnit Cookbook (4) Use TestSuite class that runs any number of TestCases together CppUnit::TestSuite suite; CppUnit::TestResult result; suite.addtest( new CppUnit::TestCaller<ComplexNumberTest>( "testequality", &ComplexNumberTest::testEquality ) ); suite.addtest( new CppUnit::TestCaller<ComplexNumberTest>( "testaddition", &ComplexNumberTest::testAddition ) ); suite.run( &result ); TestSuite can contain any object implementing the Test interface CppUnit::TestSuite suite; CppUnit::TestResult result; suite.addtest( ComplexNumberTest::suite() ); suite.addtest( SurrealNumberTest::suite() ); suite.run( &result ); 29 CppUnit Cookbook (5) Run TestSuite using TestRunner Using Helper Macros Defined in cppunit/extensions/helpermacros.h which must be integrated for initiating and finishing a test suite (CPPUNIT_TEST_SUITE and CPPUNIT_TEST_SUITE_END) Rewrite the fixture to include the HelperMacros.h #include <cppunit/extensions/helpermacros.h> class ComplexNumberTest : public CppUnit::TestFixture {... TestSuite can be created with the HelperMacros CPPUNIT_TEST_SUITE( ComplexNumberTest ); CPPUNIT_TEST( testequality ); CPPUNIT_TEST( testaddition ); CPPUNIT_TEST_SUITE_END(); 30

16 CppUnit Cookbook (6) Run TestSuite using TestRunner #include <cppunit/ui/text/testrunner.h> #include "ComplexNumberTest.h" int main( int argc, char **argv) { CppUnit::TextUi::TestRunner runner; CPPUNIT_TEST_SUITE( ComplexNumberTest ); CPPUNIT_TEST( testequality );... CPPUNIT_TEST_SUITE_END(); bool wassuccessful = runner.run(); return wassuccessful? 0 : 1; If all the tests pass, you'll get an informative message. If any fail, you'll get the following information: The name of the test case that failed The name of the source file that contains the test The line number where the failure occurred All of the text inside the call to CPPUNIT_ASSERT() which detected the failure 31 CppUnit Cookbook (7) The classes of the CppUnit-framework are placed in a namespace which is defined by the macro CPPUNIT_NS CppUnit macros CPPUNIT_ASSERT Check whether the passed expression returns the value True CPPUNIT_ASSERT_EQUAL Check whether the first parameter is like the second one CPPUNIT_ASSERT_THROW Check whether the passed expression throws an exception of the passed type CppUnit Examples See LinkedListTest 32

17 Unit testing and Test-driven Development (TDD) How to write JUnit/CPPUnit tests? Do it TDD! Workflow: 1. Think about functionality to be implemented & scenarios in which the unit to be tested will play a role. 2. Create a stub of the unit you want to implement. 3. Write a test for each scenario and/or use of the unit. 4. Make the unit fail the tests (nothing is implemented yet!). 5. Develop the unit until it passes every test. Encountered new scenario/use? make a test before implementing it! 33 Test-driven Development (TDD) Test-driven development (TDD) is an evolutionary approach to development which combines Test-first development Write a test before you write just enough production code to fulfill that test Refactoring The goal of TDD Think through your design before your write your functional code To write clean code that works TDD is primarily a design technique with a side effect of ensuring that your source code is thoroughly unit tested 34

18 References W.-K Chen, Object-Oriented Programming - Software testing JUnit A Cook's Tour. htm CppUnit Cookbook. kbook.html Krzysztof Pietroszek, Unit testing with JUnit and CPPUnit. 35

JUnit: The Goals of JUnit

JUnit Cook s s Tour CSIE Department, NTUT Woei-Kae Chen 1 JUnit: The Goals of JUnit To write a framework within which we have some glimmer of hope that developers will actually write tests. The framework