OHJ-306x: Software Testing Introduction to the Course Project Part 1: General Information and Project phases 1 & 2: Unit testing

Transcription

1 1 OHJ-306x: Software Testing Introduction to the Course Project Part 1: General Information and Project phases 1 & 2: Unit testing Antti Jääskeläinen, leading course assistant Matti Vuori, course assistant (Slide set authors: Mika Maunumaa, Antti Jääskeläinen, Matti Vuori)

2 2 Agenda Basic bureaucracy About the staff and their roles and responsibilities, how to participate to the course project, your responsibilities etc. Course project Phases, the schedule, items to return, guidance meetings etc. Project Phases 1 & 2: Unit testing How to plan and design tests, how to conduct the testing, what to observe, and what to report Java and JUnit How does Java differ from C++, and how does JUnit work.

3 3 Part 1 BASIC BUREAUCRACY

4 4 Course staff Roles and responsibilities Mika Katara Lecturer Examinations Overall grading Answers all questions related to issues above Antti Jääskeläinen Leading assistant, in charge of the Course Project Reception: Room TF216, Tuesdays 11:30 12:30 Matti Vuori Assistant on the Course Project Room TF216, Assistants answer all immediate questions about particular project tasks and hold review and feedback sessions

5 5 How to get information General information channels IDLE system Use Finnish or English Questions about the task at hand Use IDLE if the question is general enough More private questions to your assigned assistant Questions about Master Test Plan or the project in general To the leading assistant

6 6 Participating the course project 1/2 Register to the project Find a team mate as soon as possible Just one (1) Register yourself in to the IDLE system (idle.cs.tut.fi) Make sure that your team mate has done the same In IDLE, create a test team (a group) with your team mate The team is always a pair Test team will have an assistant assigned (almost) automatically Read the Course Project page General project information, deadlines, links Read the Master Test Plan (MTP) the practical, detailed instructions

7 7 Participating the course project 2/2 NOTE! While working with the assignment, do not copy ideas from others (in Finnish, use Google translate to get the idea) Copying will have severe consequences

8 8 Student s role and responsibilities Student is a member of a test team Students must Provide required test plans and test designs Execute tests according to their plan Report test execution log, defects and deviations Learn! Plans are nothing; planning is everything Dwight D. Eisenhower Plans are of little importance, but planning is essential Winston Churchill

9 9 Passing the course 1/2 See details on Course Project web page There are 4 phases in the project Each can have 0-4 points 0 = good job, but too many mistakes 4 = excellent job 0 (zero) means that you must work harder in the subsequent tasks There are 2 voluntary review sessions Possible to gain 1 point from each assistants decide Acceptable draft of a plan at review meeting required for a point Compensates any missing points from actual tasks At most 16 points (half the course total) may be obtained from the course project

10 10 Passing the course 2/2 You will get the right to take the exam, if You get 5 points or more from project All tasks are accepted with grade 0 or greater At any situation you must return something in order to continue in the course You will pass the course if you get 5 points or more from exam Roughly half of the available points on the course (~ 16/32)

11 11 Submitting your work general rules Submit electronic version in IDLE All electronic plans and reports must be in PDF format OpenOffice can export PDF MS Word can print PDF through PDFCreator ( See details in Master Test Plan

12 12 Preparation checklist You have Lintula (Birdland) account You have registered yourself in IDLE system idle.cs.tut.fi You have found your team mate You have created a team in IDLE You have read the Course Project web page You have read the Master Test Plan At least the part concerning the unit testing phases

13 13 Part 2 THE COURSE PROJECT

14 14 Introduction to course project Course project has four phases Phase 1: Planning of unit testing Phase 2: Execution and reporting of unit tests Phase 3: Planning of system testing Phase 4: Execution and reporting of system tests The application under test is an advanced text editor jedit Written in Java

15 15 Phases 1 & 2: Unit testing In unit testing phases the test team will do the following: Phase 1: Planning of unit testing Create functional unit tests Plan test structure (test set, test suites...) Design appropriate test cases Review the plan with an assistant Phase 2: Execution and reporting of unit tests Implement and execute the tests Create a test report from findings Create a defect report Submit the reports & updated plans

16 16 Phases 3 & 4: System testing In system testing phases the test team will test the application at user interface level Phase 3: Planning of system testing Select an approach to the testing Feel free to select an approach that you feel most comfortable with Plan test structure and how to conduct the testing Review the plans with an assistant Phase 4: Execution and reporting system tests Implement and execute the tests Create a test log to record testing activity Create a test report from findings Create a defect report Submit the reports to the assistant

17 17 Schedule for project phases Unit test design Week 38: Review meeting Week 39: Submission deadline Week 40: Feedback Meeting Unit test report See the exact dates and times on Course Project web page Week 43: Submission deadline System test design Week 45: Review meeting Week 46: Submission deadline Week 47: Feedback meeting System test report Week 49: Submission deadline

18 Review & feedback meetings with assistants 1/2 18 Review meetings Before submissions at phases 1 and 3 Questions and Answers about how the particular work should be done Voluntary Aim: Discussion about a preliminary version of the plan (DRAFT) To check that the task has been started in a right way A bonus point awarded for a good draft at assistant's discretion 20 minutes per team

19 Review & feedback meetings with assistants 2/2 19 Feedback meetings After submissions at phases 1 and 3 What is the verdict of your submitted work Obligatory At least one team member must be present Both members must attend at least one feedback session 15 minutes per team

20 Submissions for unit testing (phases 1 & 2) Phase 1: Phase 2: After planning you must return unit test plan a pdf Tar package containing Unit test report a pdf Fixed unit test plan f pdf Unit test files exported from Eclipse See details in Master Test Plan 20

21 Submissions for system testing (phases 3 & 4) Phase 3: System test plan See details in Master Test Plan a pdf Phase 4: Tar package containing System test report a pdf Fixed system test plan f pdf Other files according to your chosen approach

22 22 Part 3 PROJECT PHASES 1 AND 2: UNIT TESTING

23 23 Test objectives Unit testing of jedit The purpose is to create a set of tests that verifies the current functionality and works as a regression test suite later Testing of one class: PatternSearchMatcher Used to search regular expression patterns in document text Acts as a wrapper for Java API class Pattern Testing performed using JUnit framework

24 24 Making the plan How to select your test cases? How to determine the importance? Assess priorities and dependencies How do you manage the logistics of your project? How will you divide your tests into physical files? What is your approach? When you have tested enough? Code coverage may give guidance Pass/failure of individual test case Pass/failure of unit testing How much time will the testing process take?

25 25 Prioritization Prioritize according to a methodology of your choice Example: probability / effect analysis Probability The likelihood that the tested feature contains an error Depends on the complexity of the feature Effect The severity of the consequences of an error in the feature Depends on how important the feature is to the users Combine to get priority

26 Planning test cases some issues to document 26 Test case is a JUnit method Test identifier is the name of the test method Must be descriptive Creation and modification dates Modification date tells the date for last modification Priority and dependencies if any The four parts: setup, execution, comparison, cleanup

27 27 Structure of a test case Setup Create relevant objects and set them to a right state Execution A single method call with suitable inputs Comparison Check the return value and the states of the objects involved The most important step! Cleanup Release allocated resources (objects, file handles etc.)

28 28 Estimation example 1/2 For the class PatternSearchMatcher Constructing an object Test with at least two patterns (simple & complex)? Case sensitivity Altogether 2*2 = 4 choices? Separate class for each? Calling the methods Around 27 YES/NO questions in the class methods One test case for each? 4*27 = 108 test cases?

29 29 Estimation example 2/2 Setting up a test class: 5 minutes / class? Implementing a test case Figuring out inputs and outputs: 5 minutes / test case? Coding a test case: 2 minutes / test case? Compilation and execution: 4 minutes / class? Checking results: on average 1 minutes / test case? Altogether 4*(5+4) + 108*(5+2+1) = 900 minutes = 15 hours? Lots of other factors involved Not everything can be tested with just a few patterns Most conditions don't actually care about the pattern One test case will pass multiple conditions Times required will vary individually

30 30 Test report Summary of what has been tested. Did you have to change your plan while testing? If you don t change your plan when needed, your project will end up to the rocks. What were the test cases you had to change and why? In your opinion, was your testing complete enough. Did you find any faults or other peculiar things? Was your testing successful? Did you find faults, did you reach your goals etc. How well your planned usage of time corresponded with the real usage?

31 31 Error reports Tell what is wrong with the software Expressed concisely in the title Show where and how the problem occurs Test environment Inputs Expected and received results etc. Motivate people to fix the issue Impact on end user Impact on development and testing

32 32 Part 4 Java and JUnit

33 33 Java as compared to C++ 1/4 Objects are always allocated on the heap MyClass x = new MyClass(); Objects are always handled as references StringBuilder s = new StringBuilder( abc ); StringBuilder t = s; s.append( def ); t.tostring(); // returns string abcdef No explicit object deletion, garbage collection handles objects once they're no longer referred to x = null;

34 34 Java as compared to C++ 2/4 Object equality a == b // checks whether a and b are the same object a.equals(b) // checks whether a and b are equal based on definition of equality by a Class definition and inheritance class SubClass extends BaseClass implements Interface1, Interface2 { } No multiple inheritance

35 35 Java as compared to C++ 3/4 Primitive types work much as in C++ byte, short, int, long, float, double, boolean, char Each also has a corresponding class Strings are (immutable) objects but have special support String str = abc + anotherstr; They still need to be compared using equals() Arrays are mutable objects Integer[] intarray1 = new Integer[2]; Integer[] intarray2 = new Integer[] {1, 2}; Arrays.equals(intArray1, intarray2);

36 36 Java as compared to C++ 4/4 For more information see

37 37 JUnit 1/3 Unit testing framework for Java software Includes ready-made methods for setting up, executing and reporting tests Test cases presented in a structured, maintainable format JUnit method is a test case JUnit actually uses the term test case for classes, but you need not create a separate class for each test case Classes are better used to group test cases with similar setups For more information see Corresponding xunit frameworks are available for other languages

38 38 JUnit 2/3 public class TestClass extends TestCase { protected void setup() { } // for common setup actions protected void teardown() { } // for cleanup actions // actual test cases, marked with public void test1() { public void test2() {...} }

39 39 JUnit public void test1() { See Master Test Plan for a more detailed example and instructions PatternSearchMatcher matcher = new PatternSearchMatcher( b, false); Match m = matcher.nextmatch( abc, true, true, true, false); assertequals( start, 1, m.start); assertequals( end, 2, m.end); asserttrue( substitutions, Arrays.equals(new String[] { b }, m.substitutions)); }