Testing. My favourite testing quote: Program testing can be used to show the presence of bugs, but never to show their absence!

Transcription

1 Testing

2 Some resources The most time-consuming of a development project. See for example papers/softwaretesting.pdf - Peter Sestoft testing notes

3 Testing My favourite testing quote: Program testing can be used to show the presence of bugs, but never to show their absence! (Dijkstra) (See the Wikiquote page on Dijkstra for lots more entertaining comments. He is probably most famous for ridding the world of the goto statement.)

4 Testing What is testing? Compiling is a form of testing (you are testing the code to see if it compiles) Testing usually means: running code on some input data, and checking what happens (e.g. the output, or how the system state evolves) See Wikipedia: Software testing, Portal: Software testing, Test plan

5 Common terms functional/non-functional; defect/failure; testing/verification/validation; white box/black box; unit testing; corner case; automated testing; regression testing; test case

6 Some resources A good introductory article is

7 Testing all cases Look at the following code: void f(int x) { if(x < 10) {... // do something else {... // do something else If you want to test this code, you should test with a value of x less than 10, and also with a value of x greater than 10.

8 Testing all cases Look at the following code: void f(int x, int y, int z) { if(x < 10) { if(y < 20) {... else {... else { if (z > 30) {... else {... What are the cases here? Real world code can get very much more complicated than this.

9 Coverage testing It becomes very hard to figure out whether you have covered all cases Especially if the code keeps changing So design your tests, and then check (somehow) which lines are being tested This is called coverage checking A coverage checker will run the tests on your code, and record which lines actually run (and which don t!) You want to cover as many lines as possible!

10 Automated testing If you have run some tests on your code, and they have passed, and the system will never change, then you don t need to run the same tests again (they should still pass). But usually things do change. So you need to run the tests again, and again, and again. Doing this manually gets boring, so you write a program which runs the tests. This is automated testing.

11 Regression testing After automated testing, you really want to be able to compare the results of multiple testing runs, find out the failures, and somehow correlate this with changes to the code that happened between the testing runs. If a test passed before, but now fails, this is a regression (things have gone backwards). Regression testing is basically managing automated testing over time, identifying and tracking regressions, etc.

12 TDD (Test Driven Development) - software development process Focus on software requirements in lieu of writing the code and then designing the tests afterwards. TDD consists of the following simple steps: First, write an (initially failing) test Second, write the code that passes the test, doing the simplest thing that could possibly work. Third, see the test succeed and refactor the code. See for example https: //en.wikipedia.org/wiki/test-driven_development for more details

13 JUnit You can write tests manually. In a big project, you want everybody to write tests in the same way, so that you can run all the tests together. JUnit is a framework for writing tests, so that everybody writes tests in the same way. This has other advantages, e.g. JUnit has lots of gizmos such as a graphical test viewer, showing which tests succeeded and which failed See also for more details.

14 A simple example - A class book public class Book { private String title = ""; private String author = ""; Book(String title, String author) { this.title = title; this.author = author; public String gettitle() { return title; public String getauthor() { return author;

15 A simple example - Create a test unit import org.junit.test; import static org.junit.assert.assertequals; public class BookTest { Book book = new Book("Dune", "Frank public void testtitle() { assertequals( book.gettitle(), public void testauthor() { assertequals( book.getauthor(), "Frank Herbert" );

16 A simple example - Test runner import org.junit.runner.junitcore; import org.junit.runner.result; import org.junit.runner.notification.failure; public class TestRunner { public static void main(string[] args) { Result result = JUnitCore.runClasses(BookTest.class); for (Failure failure : result.getfailures()) { System.out.println(failure.toString()); System.out.println(result.wasSuccessful());

17 Demo Use of Junit4, see Compile using jar-files junit-4.12.jar and possibly hamcrest-core-1.3.jar Run the test in the command-line. See the code provided.

18 Some useful methods from JUnit API assertarrayequals(a, b) tests if array a is the same as b. assertequals (o1, o2) tests tests if o1 and o2 are equals using their corresponding equals method. The assertnull(o) and assertnotnull(o) methods test if the object o is null or not null. The asserttrue(boolean condition) asserts that condition is true. More on junit/assert.html

19 Important warning JUnit v.3 is quite well documented, but is the old version. Run the command junit -help to find out the version on your linux machine. JUnit 5 is described as the next generation of JUnit. See for more details. Junit 4 is the version most people are currently using. For example, it is the default in current releases of eclipse. So you should use v.4 Some differences between v.4 and v.5 are described here: http: //howtoprogram.xyz/2016/08/10/junit-5-vs-junit-4/