Through a Glass, Darkly: Object-Oriented Testing. Through A Glass, Darkly:

Size: px

Start display at page:

Download "Through a Glass, Darkly: Object-Oriented Testing. Through A Glass, Darkly:"

Silvester Atkinson
5 years ago
Views:

1 Through A Glass, Darkly: Object-Oriented Testing 1 EECS 814 Presentation Slide 1 of 110

2 Personal Info B.S. Computer Science MWSU B.S. Chemistry MWSU 2 15 years professional programming experience Reuters Financial market analysis software US Navy Weapons/Propellant simulations Slide 2 of 110

3 Source A Practical Guide to Testing Object-Oriented Software by John D. McGregor and David A. Sykes 3 Slide 3 of 110

4 Quote Never ascribe to malice that which can be explained by incompetence. 4 Napoleon (attributed) Slide 4 of 110

5 Road Map Introduction Object-Orientation Overview Specifications/Testing Classes Interactions Hierarchies Conclusion 5 Slide 5 of 110

6 Why Do I Care? Quality is important No one likes fixing bugs Prevent them from occurring 6 final exam Questions on the Practicality Slide 6 of 110

7 Question #1 How many of you write code for a living? 7 Slide 7 of 110

8 Question #2 If you write code for a living, how many of you write Object-Oriented code? C++? Java? Smalltalk? 8 Slide 8 of 110

9 Question #3 (Rhetorical) Have you noticed that most of the in-class examples have been imperative programming? Non Object-Oriented 9 Does this mean that the techniques we've learned aren't applicable to OO? No, they're still very applicable Slide 9 of 110

10 Practical ('prak-ti-kul) adjective 4a: disposed to action as opposed to speculation or abstraction Something you can accomplish Realistic goals10 Everyday uses Slide 10 of 110

11 Question #4 Do you work in a Practical Development Environment? 11 Slide 11 of 110

12 So What's an Impractical Environment? Space Shuttle on-board software team Must be 100% good They test everything Realistically, 12 fast and cheap are ignored No such thing as good enough Real world Don't have that luxury Must set priorities Slide 12 of 110

13 * 3-Way Practicality Process Matrix There are three things that can define any process Good Fast Cheap 13 *Pick any two Slide 13 of 110

14 Quote Everybody in the real world will agree - the moment a project is behind deadline, quality assurance tends to go out the window. 14 Alan Cox Slide 14 of 110

15 Road Map Introduction Object-Orientation Overview Specifications/Testing Classes Interactions Hierarchies Conclusion 15 Slide 15 of 110

16 Object-Oriented Systems Object Message Interface Class Inheritance Polymorphism 16 Slide 16 of 110

17 Object Operational entity As opposed to definitional entity Encapsulates Specific data17 values Code that manipulates data values Basic building block of OO systems Slide 17 of 110

18 Object: Examples Course Student Bill Grade 18 Slide 18 of 110

19 Object: Testing Implications Encapsulation Build up object and pass it around Easy to manipulate 19 Information hiding Difficult to change object Changes in object difficult to observe Has state Can become inconsistent Has a lifetime Construction/Destruction Slide 19 of 110

20 Message Request that an operation be performed by an object Allows objects to collaborate to solve problems 20 Different languages C++: member function call Java/Smalltalk: method invocation Slide 20 of 110

21 Message: Examples Student.AddCourse(Course) Student.SendBill(Bill) Student.MakePayment(CreditCard) 21 Grade.SetGrade( A ) Student.SetGradeForCourse(Grade,Course) Student.SendGrades() Slide 21 of 110

22 Message: Testing Implications Has a sender Sender determines when to send message May send incorrectly Has a receiver22 Receiver may not be ready Unexpected message may be ignored May include parameters/return value Used and/or updated by receiver Object state is important Receiver may throw exception Slide 22 of 110

23 Interface Aggregation of behavior declarations Different languages C++: abstract base class with public pure virtual methods 23 Java: interface (defined just like a class) Building block for specifications Slide 23 of 110

24 Interface: Examples Enrollment Residence Behavior associated with enrolling in courses 24 Behavior associated with living in on-campus housing Billable Behavior associated with being able to make payments Student Parent Company Slide 24 of 110

25 Interface: Testing Implications Possible poor design Some interfaces may have 'extra' behavior Interface as parameter 25be passed merely as its An object may interface Unwanted baggage may be carried along Slide 25 of 110

26 Class Set of objects (members) that share a common conceptual basis Class definition What the members have in common 26 What they will do together Objects Classes Elements for executing a program Elements for defining a program An Object is an instance of a Class Slide 26 of 110

27 Class: Specification Operation: action applied to an object to obtain a certain effect Two categories of operations Accessor 27 Returns information about the object Changes nothing in the object Modifier Changes the state of an object Slide 27 of 110

28 Preconditions/Postconditions Precondition Postcondition 28 Condition that must be true before an operation is attempted Condition that must be true after an operation is completed Invariant Condition that must be true before and after an operation Sort of a pre/post combination Slide 28 of 110

29 Preconditions State and attributes of the object containing the operation Attributes of parameters in the message 29 Slide 29 of 110

30 Postconditions State and attributes of the object containing the operation Attributes of parameters in the message Return value (if30any) Any exception that might arise Slide 30 of 110

31 Invariants Attributes that must be the same before and after Operational Invariants during 31 a single operation Class Invariants for the lifetime of an object Implies: during ALL operations Slide 31 of 110

32 Road Map Introduction Object-Orientation Overview Specifications/Testing Classes Interactions Hierarchies Conclusion 32 Slide 32 of 110

33 Question #5 Is OO Testing: Blackbox testing? Whitebox tesing? Why? 33 Slide 33 of 110

34 Trick Question: Both Whitebox Looking at how the code is written Analyzing code structure Class 34 Message Inheritance Blackbox How objects behave from the outside Interface Polymorphism Slide 34 of 110

35 OO Is More Than the Code It can also include the entire development process* Requirements Analysis and35 Design Implementation *Please see EECS 816: OO Analysis and Design Slide 35 of 110

36 The Big Zoom Out Many levels of testing Each has 'challenges' Class 36 Class Interactions Class Hierarchies Slide 36 of 110

37 So, What's a Class Again? Fundamental unit of OO programming First level of information-hiding Key to encapsulation Building block 37 of inheritance Slide 37 of 110

38 How Can You Test a Class? Two ways: Code reviews (static) Execution of test cases (dynamic) 38 Slide 38 of 110

39 Code Reviews (Pro/Con) Pros Ability to shift with changing specifications Exposes more of the team to the implementation 39 Cons Humans involved (mistakes) Time-consuming (especially for regression testing) Slide 39 of 110

40 Test Case Execution (Pro/Con) Pros Ability to shift with changing specifications Exposes more of the team to the implementation 40 Easily repeatable (Regression testing) Cons Humans involved Difficult to identify Time-consuming to write If coverage is good, maybe more time than writing the code itself Slide 40 of 110

41 So, Which One's the Best? That depends.... How often do you need to test? What development model do you use? Aren't they complementary? 41 Yes, they're complimentary Technically, a lot of overlap Better to do both, but may reduce the practicality Slide 41 of 110

42 Our Focus Execution-based testing Easily repeatable More useful in non-waterfall models 42 Slide 42 of 110

43 Main Trouble Identifying test cases 43 Slide 43 of 110

44 44 Slide 44 of 110

45 Assumption Quote We assume that a class to be tested has a complete and correct specification..... We assume the specification is expressed in a specification language such as the Object 45 Constraint Language (OCL) or a natural language and/or as a state transition diagram. Does that sound practical to you? Slide 45 of 110

46 Complete Specifications? Not me! 46 Slide 46 of 110

47 But, Seriously... In order to write tests, you need a specification of some kind 47 Slide 47 of 110

48 So, What's a Specification Exactly? Link between real world and model Part of the problem Part of the solution 48 Description of operation Class a whole Operations Preconditions/postconditions Invariants Slide 48 of 110

49 What If I Don't Have a Spec? Get one 49 Slide 49 of 110

50 No, Really, What If I Don't Have a Spec? Reverse-engineer from the code And how think the code is supposed to work Test-driven development 50 Write the tests Write the code Repeat When do you write comments? Slide 50 of 110

51 Better To Write As You Go... Tests are code Consider test code linked with implementation Unit testing 51 Slide 51 of 110

52 What is Unit Testing? Unit Test procedure used to verify that a particular module of source code is working properly Goal is to catch52regression errors Unit test frameworks Sunit (Smalltalk) Junit (Java) CppUnit (C++) NUnit (C#) Practical Slide 52 of 110

53 extreme Programming (XP) Agile development model Requires test-first model Must write the test before the code that will be tested 53 We're getting off track... Slide 53 of 110

54 Road Map Introduction Object-Orientation Overview Specifications/Testing Classes Interactions Hierarchies Conclusion 54 Slide 54 of 110

55 Finding Defects 55 Slide 55 of 110

56 Dimensions of Class Testing Who What When How How much? 56 Slide 56 of 110

57 Who Usually developers (of the code itself) Minimizes public knowledge (good) Minimizes peer review (bad) 57 Slide 57 of 110

58 What Code meets requirements No more No less 58 Slide 58 of 110

59 When Never Milestones When you have a specification When changes59occur Always Slide 59 of 110

60 How Drivers and stubs Unit test frameworks Instance vs. static 60 Slide 60 of 110

61 How Much? Spec vs. implementation Simple answer: Enough! Impractical answer: Everything! 61 Risk-based analysis Slide 61 of 110

62 Back To Not Having a Spec... Writing testable code Contract Approach Preconditions over postconditions 62 Defensive Approach Postconditions over preconditions Problems? Slide 62 of 110

63 Contract Approach Preconditions are an obligation of the sender If met, receiver is obligated to meet the requirements and postconditions 63 Design implications Need sufficient preconditions so receiver can fulfill postconditions a sender can determine whether all preconditions are met before sending a message May need extra accessor methods to check for postcondition success Slide 63 of 110

64 Defensive Approach Interface defined in terms of the receiver Operation typically returns a value Return code Status object64 Primary goal Sanity checking Returning status of processing Slide 64 of 110

65 Test Writing Implications Contract approach simplifies class testing complicates interaction testing must ensure any sender meets the 65 preconditions Defensive approach complicates class testing test cases must address all possible outcomes complicates interaction testing must ensure all possible outcomes are produced and that they are properly handled by the sender Slide 65 of 110

66 Road Map Introduction Object-Orientation Overview Specifications/Testing Classes Interactions Hierarchies Conclusion 66 Slide 66 of 110

67 Programming Tip Testing is for cowards A brave coder will bypass that step If we have full confidence in our coding ability, then testing will be unnecessary 67 - from How to Write Unmaintainable Code Slide 67 of 110

68 Class Interactions OO Program Collection of objects Collaborate together to solve a problem Ensure that messaging happens correctly 68 Assumptions Objects have been individually tested Unidirectional messages Interactions are sequential (not concurrent) Slide 68 of 110

69 What's an Interaction? Object 'A' calls a method 'X' of Object 'B' Object B's method X executes Reply from method X is returned to A 69 Slide 69 of 110

70 Impact of a Single Interaction Possible cascade of other object interactions State of the receiver Associated objects Possible outcomes 70 No change Changes in attribute values State changes Creation or deletion of objects Slide 70 of 110

71 Identifying Interactions Two types of classes Primitive classes Simple 71 needed No other classes Usually limited in number Non-primitive Support/require other objects Most common in OO programming Slide 71 of 110

72 Two Categories of Classes Collection Collaboration 72 Slide 72 of 110

73 Collection Classes Collection classes Have classes associated with them Never actually use them Store/Create/Delete 73 Examples Stack Queue Map Vector Slide 73 of 110

74 Testing Collection Classes Fairly straightforward Capacity limitations Store/retrieve order correctness Negative tests 74 Slide 74 of 110

75 Collaboration Classes More complex than collection classes Identification Uses other objects in operations 75 operation When, after an State is possibly affected An attribute is used or modified Handout # 1 Identify the classes Slide 75 of 110

76 What To Test? Best Every collaboration interaction Not reasonable Better Interactions most likely to have faults Good 76 Probability distribution Good enough (Not much choice here) Less than ideal probability distribution Slide 76 of 110

77 Test Case Sampling Random Use Profiling Stratified Sample OATS (Orthogonal Array Test System) 77 Slide 77 of 110

78 Use Profile Associate each use of the system with a frequency Ranked Probability of use 78 Higher frequency of use ---> More testing Slide 78 of 110

79 Stratified Sample Tests selected from categories Example: Test subsets Select a set of test cases that exercise every component in79the system Divide those into subsets based on component Example: Use Case Actors Rank uses by actor/frequency Handout #2 Slide 79 of 110

80 Orthogonal Array Test System (OATS) Method to limit explosion of test cases Uses pair-wise combination of factors Two-way interactions cause most faults 80 pairwise testing Sometimes called Used in Taguchi Methods Slide 80 of 110

81 OATS: Basic Technique 1) Determine number of independent variables 2) Determine number of values it will take (states) 81 3) Find an minimum-pair array (book) 4) Map factors into array 5) Build test cases These steps may vary slightly from user to user Slide 81 of 110

82 OATS: Example Consider a web page with three sections (Top, Middle, and Bottom). Sections that can be individually shown or hidden by the user. 82 You wish to test the interactions of the different sections. Slide 82 of 110

83 Question #6 How many test cases would there be if you wanted 100% coverage? 83 Slide 83 of 110

84 Answer #6 8 tests, 2^3 = 8 84 Slide 84 of 110

85 OATS 1-Independent Variables Three independent variables (page sections) Top Middle Bottom 85 Slide 85 of 110

86 OATS 2-Variable States Each variable can take two states Hidden Visible 86 Slide 86 of 110

87 OATS 3-Minimum Pair Array 87 Slide 87 of 110

88 OATS 4-Map Factors Into Array 88 Slide 88 of 110

89 OATS 5-Build Test Cases 1) Hide all sections 2) Display all but the top section 3) Display all but the middle section 89 the bottom section 4) Display all but Slide 89 of 110

90 OATS: OO Programming Independent variables are class families Values are members of the class family See Handout #3 90 Slide 90 of 110

91 OATS: Adequacy Criteria Exhaustive Minimal tester haphazardly selects cases from several of the classes Representative only interactions 91 between the base classes Random all combinations of all factors uniform sample -- every class is tested Weighted representative adds cases of relative importance or risk Slide 91 of 110

92 Road Map Introduction Object-Orientation Overview Specifications/Testing Classes Interactions Hierarchies Conclusion 92 Slide 92 of 110

93 Programming Tip Never, Ever Do Any Performance Testing Hey, if it isn't fast enough, just tell the customer to buy a faster machine 93 - from How to Write Unmaintainable Code Slide 93 of 110

94 Class Hierarchies (Inheritance) Relationship between classes Allows new class to be based on existing class Reuse of specification and implementation 94 Parent (source) class Superclass Base class New Class Subclass Derived class Slide 94 of 110

95 Inheritance (cont.) Inheritance hierarchy Each class has ancestors (except roots) Good OO Design 95 is-a relationship is-a-kind-of relationship Best when used to propagate specification Not implementation (code reuse) Slide 95 of 110

96 Inheritance: Testing Implications Provides a bug propagation mechanism Inherit defects from parent Good reason to test class hierarchy as it develops 96 Allows for reuse of test cases Reuse of spec and implementation Slide 96 of 110

97 A Tale of Two Classes C D Parent or superclass 97 Child or subclass Handout #4 Slide 97 of 110

98 Is-Kind-Of Relationship D is-a-kind-of C D can be used in place of C in a program Behavior of D must match (or supersede) C Class Behavior98 Observable states Operational semantics Enforced by the Substitution Principle Slide 98 of 110

99 Substitution Principle Only these changes are allowed in a subclass Preconditions Postconditions Must be the same or stronger Must expect as much as superclass Class Invariant Must be the same or weaker Less constraining to a client 99 Must be the same or stronger Can add more constraints Assume Substitution Principle Slide 99 of 110

100 Refinement Possibilities (Subclass) 1) Add new operations/methods to D 2) Change spec or implementation of and operation declared by C Change in D100 an operation declared in C Override in D a method inherited in C 3) Add new instance members in D 4) Change the class invariant Slide 100 of 110

101 Hierarchical, Incremental Testing (HIT) Incremental changes from C to D can guide test case selection All of the spec-based tests of C should apply to D 101 If not, substitution rule has been broken Simple process to determine Which C tests are still applicable What new tests need to be written Slide 101 of 110

102 1) HIT: Add Operations New operation implements new functionality New functionality needs to be tested Also need to add Class/Object102 implementation test Class Interaction tests Slide 102 of 110

103 2) HIT: Change Spec a) Change in D the spec for a method declared in C Must add new spec-based tests Must test new 103pre/post conditions Must re-run C test cases b) Override a C method in D that uses a C operation Can reuse C's spec-based tests Must review all implementation based tests Possibly new coverage Slide 103 of 110

104 HIT: Change Operations New instance variables added Most likely tested by new code Already covered by new tests? If a new variable 104 is not used, it doesn't need to be tested Slide 104 of 110

105 HIT: Change Invariant New postconditions Simply test the new invariant conditions Must re-run all inherited test cases to assure that the invariant 105 holds Slide 105 of 110

106 Conclusion OO brings new tool to the programmer New way of programming New ways of testing 106 Old testing methods still apply Slide 106 of 110

107 Thoughts? How do you test now? How can you change the testing of code in your environment? Will these techniques help you? 107 Slide 107 of 110

108 Questions? 108 Slide 108 of 110

109 References A Practical Guide to Testing Object-Oriented Software by John D. McGregor and David A. Sykes; 2001, Addison-Wesley. The Object Primer by Scott W. Ambler; , Cambridge University Press. JUnit Primer : er.html Unit Testing : Slide 109 of 110

110 References Orthogonal Array Testing System (OATS) : Temas/TemarioEnCurso/Prova orientada a objecte/orthogonal arrays.pdf 110 Slide 110 of 110

Lecture 15 Software Testing

Lecture 15 Software Testing Includes slides from the companion website for Sommerville, Software Engineering, 10/e. Pearson Higher Education, 2016. All rights reserved. Used with permission. Topics covered