Lecture 20: SW Testing Presented by: Mohammad El-Ramly, PhD

Transcription

1 Cairo University Faculty of Computers and Information CS251 Software Engineering Lecture 20: SW Testing Presented by: Mohammad El-Ramly, PhD

2 Outline Definition of Software Testing Testing as a Quality Assurance Technique Testing Terminology Levels and Types of Testing 2

3 3

4 What is a computer program? 4

5 What is a house? 5

6 Any better definition? The focus is on Stakeholders (for people) Purpose (to live in) Stakeholder: Any person affected by: success or failure of a project, or actions or inactions of a product, or effects of a service. 6

7 Apply this definition to prog. 7

8 Apply this definition to prog. A computer program is a communication among several humans and computers who are distributed over space and time, that contains instructions that can be executed by a computer. 8

9 Software Quality 9

10 Software Error (Bug) 10

11 Software Error (Bug) An attribute of a software product that reduces its value to a favored stakeholder or increases its value to a disfavored stakeholder without a sufficiently large countervailing benefit. An error may or may not be a coding error, or a functional error. Design errors are bugs too. 11

12 Software Testing is an empirical technical investigation conducted to provide stakeholders with information about the quality of the product or service under test 12

13 Software Testing 13

14 Software Testing is Search for Information 1. Find important bugs 2. Assess the quality of the product 3. Help managers assess the progress of the project 4. Help managers make release decisions 5. Block premature product releases 6. Help predict and control product support costs 7. Check interoperability with other products 14

15 Software Testing is Search for Information 8. Assess conformance to specifications 9. Certify the product meets a particular standard 10.Ensure the testing process meets accountability standards 11.Minimize the risk of safety-related lawsuits 12.Help clients improve their processes 13.Evaluate the product for a third party 15

16 Fact of testing Testing does not guarantee the absence of defects it gurantees their presence 16

17 17

18 Assume we want to test Font Increase functionality in Word 18 Calibri Font size from 7 to 13.5

19 Assume we want to test Font Increase functionality in Word 19 Arial Font size from 7 to 13.5

20 Assume we want to test Font Increase functionality in Word Times New Roman Font size from 7 to

21 Assume we want to test Font Increase functionality in Word 21 Courier New Font size from 7 to 13.5

22 22

23 So, what should we test? 23

24 So, what should we test? What to cover? Every font size (to a tenth of a point). Every character in every font. Every method of changing font size. Every user interface element associated with font size functions. Interactions between font sizes and other contents of a document. Interactions between font sizes and every other feature in Word. Interactions between font sizes and graphics cards & modes. Print vs. screen display. 24

25 A program can fail in many ways Program state System state Program state, including uninspected outputs System state Intended inputs Configuration and system resources System under test Monitored outputs Impacts on connected devices / system resources From other cooperating processes, clients or servers To other cooperating processes, clients or servers

26 Unit testing with JUnit 26

27 27

28 Why? Why testing? Improve software quality Increase customer satisfaction and reduce cost Finish faster and on budget Catch errors and bugs In short, to Produce Better Code 28

29 What should be tested? Test for boundary conditions Test for both success and failure Test for general functionality Test for configuration and resources Test for interoperability Etc.. 29

30 When to start testing Software quality and is a life-cycle process Testing can start as early as the project starts 30

31 Change and Configuration Management Reviews Bug Tracking Software Quality Landscape M a n a g e m e n t

32 Change and Configuration Management Reviews Bug Tracking Software Quality Landscape M a n a g e m e n t

33 Change and Configuration Management Reviews Bug Tracking Software Quality Landscape M a n a g e m e n t

34 Testing Organization (V-Model) Requirements specification Produce User Specification Validate Acceptance Testing System Specification Produce System Specification Validate System Testing System Design Produce Module Specification Validate Integration Testing Module Design Produce Software Specification Validate Unit Testing 12/15/2015 Coding CPSC , CPSC , 34 Lecture 11

35 Testing Terminology

36 What is test case 1 A test case is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly 36

37 Good test case design An good test case satisfies the following criteria: Reasonable probability of catching an error Does interesting things Doesn t do unnecessary things Neither too simple nor too complex Not redundant with other tests Makes failures obvious Mutually Exclusive, Collectively Exhaustive 37

38 Test case design technique Test case design techniques can be broadly split into two main categories 2 Black box (functional) 3 White box (structural) Unit testing with JUnit 38

39 Black Box tests 2 Input Output Targeted at the apparent simplicity of the software Makes no assumptions about implementation Good for testing component interactions Tests the interfaces and behavior 39

40 White Box tests 3 Input Output Targeted at the underlying complexity of the software Intimate knowledge of implementation Good for testing individual functions Tests the implementation and design 40

41 Black box vs. White box Black box testing: Test against the specifications of the component Tests the interface of the component. White box testing: Test design and implementation of the component. Test cases exercise specific sets of condition, loops, etc.

42 Another way to classify test cases Test to specifications: Also known as black-box, data-driven, functional, or input/output driven testing. Ignore the code use the specifications to select test cases. Test to code: Also known as glass-box, logic-driven, structured, or pathoriented testing. Ignore the specifications use the code to select test cases. Neither exhaustive testing to specifications nor exhaustive testing to code is feasible. CPSC , CPSC , Lecture 11

43 43

44 Test case writing example Suppose we have two parameters we want to cover in a set of tests. Parameters are as follows.. Operating system Win98 Win2k Winxp Printers HP 4100 HP 4200 How We should write test case for this?? Unit testing with JUnit 44

45 Types (Levels) of Tests Unit Testing Individual classes or types Component Testing Group of related classes or types Integration Testing Interaction between classes Unit testing with JUnit 45

46 Types (Levels) of Tests System Testing Testing to confirm that all code modules work as specified, and that the system as a whole performs adequately on the platform on which it will be deployed. Acceptance Testing Final testing usually involving users to ensure their requirements are met

47 Testing that Involves Users Alpha testing: In-house testing. By a test team or end users. Beta testing: 9 10 By users or selected subset of actual customers in a normal work environment. Product is very close to completion. Open beta release: Let public carry out the beta testing. Acceptance testing: By users to check that system satisfies requirements to decide whether to accept the system based on the test result.

48 48

49 What is a testing framework? A test framework provides reusable test functionality which: Is easier to use (e.g. don t have to write the same code for each class) Is standardized and reusable Provides a base for regression tests 11 49

50 Regression testing 12 New code and changes to old code can affect the rest of the code base Affect sometimes means break We need to run tests on the old code, to verify it works these are regression tests Regression testing is required for a stable, maintainable code base

51 Why use a testing framework? Each class must be tested when it is developed Each class needs a regression test Regression tests need to have standard interfaces Thus, we can build the regression test when building the class and have a better, more stable product for less work 51

52 Example: Old way vs. new way int max(int a, int b) { if (a > b) { return a; } else { return b; } } void testmax() { int x = max(3, 7); if (x!= 7) { } System.out.println("max(3, 7) gives " + x); } x = max(3, -7); if (x!= 3) { System.out.println("max(3, -7) gives " + x); } public static void main(string[] args) { new MyClass().testMax(); void testmax() { assertequals(7, max(3, 7)); assertequals(3, max(3, -7)); } 52

53 JUnit 13 JUnit is a framework for writing unit tests A unit test is a test of a single class A test case is a single test of a single method A test suite is a collection of test cases 14 Unit testing is particularly important when software requirements change frequently Code often has to be refactored to incorporate the changes Unit testing helps ensure that the refactored code continues to work 53

54 JUnit.. JUnit helps the programmer: Define and execute tests and test suites Formalize requirements and clarify architecture Write and debug code Integrate code and always be ready to release a working version 54

55 Test suites Obviously you have to test your code to get it working in the first place You can do ad hoc testing (testing when something happens), or You can build a test suite (set of tests that can be run at any time) Disadvantages of writing a test suite It s a lot of extra programming True but use of a good test framework can help quite a bit You don t have time to do all that extra work False Experiments repeatedly show that test suites reduce debugging time more than the amount spent building the test suite Advantages of having a test suite Your program will have many fewer bugs It will be a lot easier to maintain and modify your program 55