Specification-based Testing of Embedded Systems H. Schlingloff, SEFM 2008

Size: px

Start display at page:

Download "Specification-based Testing of Embedded Systems H. Schlingloff, SEFM 2008"

Juniper Baker
5 years ago
Views:

1 SEFM School 2008 Specification-based Testing of Embedded Systems Prof. Dr. Holger Schlingloff Humboldt-Universität zu Berlin and Fraunhofer FIRST, Berlin Lecture 5: OCL, ParTeG Course Outline L1: Introduction testing vs. verification, embedded systems, L2: Specification formalisms labeled transition systems, UML / OCL, {CSL JML}, L3: Test generation Chinese postman algorithm, transition tours, coverage, L4: Test execution testing algorithms in detail, e.g. ParTeG, SpecExplorer L5: Test evaluation and assessment oracle problem, mutation analysis 2/30

2 Object-Oriented Modelling Class Diagrams assumed to be known class contains data fields and methods private and public components, associations each object belongs to a class 3/30 UML 2 13 diagram types Common meta-model Instances (objects) can occur in several diagrams, different views onto the same thing A structure diagram, e.g. a class, defines a collection of objects with similar properties, attributes and methods signature and structure A behavioural diagram, e.g. a statechart, defines a collection of behaviours of objects change of model in time 4/30

OCL Object constraint language (important) Part of UML Specifies constraints on model elements A constraint is a restriction on one or more values of (part of) an object-oriented model or system

3 OCL Object constraint language (important) Part of UML Specifies constraints on model elements A constraint is a restriction on one or more values of (part of) an object-oriented model or system Different kinds of constraints invariant - a constraint that must always be met by all instances of a class precondition of an operation - a constraint that must always be true before the execution of the operation postcondition of an operation - a constraint that must always be true after the execution of the operation guard of a transition a constraint that must be met before a state transition fires 5/30 Connection UML-OCL Each OCL formula can mention objects defined in UML diagrams OCL semantics relative to a certain UML model Expressions in OCL add information to UML diagrams not a stand-alone specification language OCL for constraints which cannot be expressed by diagrams - e.g. number of passengers in a flight is less or equal to the number of seats in the plane - fixed interpretation (includes arithmetics) 6/30

4 Standardisation early 1990s: Steve Cook and John Daniels, Syntropydesign method adaptation of Z to OOA 1996: OMG request for proposal; IBM and ObjectTime Ltd. submit joint proposal 1997: OCL : OCL 2.0 7/30 A Simple Example (from Wikipedia) Person +age:integer +Parents:Person[2] +Children:Person[0..*] GetsChild():Person +hasbirthday() +cars 0..* +owner 1 Car +License:String +Brand:String +Registration:Integer +Build:Integer Informal Description 1. The age of a person is not negative. 2. Each person is younger than the parents. 3. After the birthday a person is one year older. 4. Each person has exactly two parents. 5. if somebody becomes parent, the set of children is nonempty and the number is increased by one. 6. Only adults may own a car 7. The first registration date of a car is after its built OCL-Constraint Context Person inv: self.age >=0 Context Person inv: self.parents->forall(e e.age>self.age) Context Person::hasBirthday() post: self.age=self.age@pre+1 Context Person inv: self.parents->size()=2 Context Person::getsChild() post: self.children->notempty() and self.children- >size() > self.children@pre->size() Context Person inv: age<18 implies cars->size()=0 Context Car inv: Registration>=Build 8/30

5 OCL Types Basic type void : void::oclisundefined Boolean, Integer, Real, String, Enumerations: enum{val1, val2, val3} Set, Bag, Sequence union(bag{2,2,3}, Bag{3,3}) Class types: each class name can be used as a type most general class/type: OclAny Strong typing rules, subtyping according to OO Integer is subtype of Real Each type conforms to each of its supertypes 9/30 OCL Contexts The context attaches a constraint to a particular modelling element context <class name> :: <operation> (<parameters>) pre: <Boolean OCL expression> Dot-notation allows access to other (visible) modelling elements or objects Meeting.start, Customer.name self always refers to the object identifier from which the constraint is evaluated context Meeting inv: self.end > self.start Access to collections via -> Customer.booking->size() 10/30

6 OCL Operators Boolean operators: =, and, or, xor, not, implies, ifthen-else, forall sequential evaluation, i.e. (true or undefined) = true x implies y = (not x) or (x and refers to the previous value of an object in a postcondition select-operator collection->select(condition) is any element of the collection satisfying the condition e.g. Passagier.buchung->select(datum=TODAY) 11/30 Pre- and Postconditions Used to constrain methods context Meeting :: confirm() pre: Calendar.freeTimeSlot (self.start, self.duration()) post: self.isconfirmed = true context Meeting :: duration(): Integer post: result = self.end self.start context Meeting :: shift(d: Integer) post: start = start@pre + d and end = end@pre + d 12/30

7 Iterations c->iterate(x:t1; a:t2 = exp0 exp ) c is of type Collection(T) x is a name for a variable (sometimes called the cursor ) x is of a type T1 which is conformant to T a is the name for a variable (sometimes called the accumulator ) a is of type T2 exp0 is an OCL expression giving a value of type T2 exp is an OCL expression using the variables x and a and giving a value of type T2 The type of the whole iterate expression is of type T2 13/30 Predefined iterations c:collection(t)->size: Integer post: result = c->iterate(e:oclany; a:integer = 0 a+1) c:collection(t)->isempty: Boolean post: result = c->size = 0 c:collection(t)->forall(expr: OclExpression): Boolean post: result = c->iterate(e:oclany; a: Boolean = true a and expr) c:collection(t)->exists(expr: OclExpression): Boolean post: result = c->iterate(e:oclany; a: Boolean = false a or expr) 14/30

parameters according to preconditions Coverage criteria-oriented approach: Define

8 15/30 ParTeG Ideas: include the boundary values of linear ordered type variables as defined by the OCL expressions Deduce the value of system attributes from input parameters according to preconditions Coverage criteria-oriented approach: Define test goals Search backwards Create and adapt abstract domains for input parameters 16/30

9 ParTeG Start from Test Goals Test goal: Coverage criterion applied to a concrete model Example: one state for All-States Generate abstract test case Find a path Generate concrete test case Find concrete input values 17/30 ParTeG Find Paths Generation of test cases: Path from initial node to test goalcontains conditions (e.g. OCL) Due to conditions not each found path is feasible Consequence: include conditions into search algorithm Deal with the relations between OCL conditions along the path 18/30

10 Interpret OCL Expressions Generation of Test Cases: Classify all variables used in the OCL expressions - Which variables can change? Algorithm - for eachguard: - try to find postconditions that influence the result of the guard - Combine guard and postcondition to a new condition - If there are changeable variables in the condition: continue search Basic Idea: - Transform conditions on system attributes into conditions on input parameters - Use them as input partitions 19/30 Adapt Algorithm to Complex Coverage Criteria Satisfying coverage criteria: Algorithm generates tests thatsatisfyall guard conditions Several coverage criteria also need negative cases: - Condition Coverage - Decision Coverage - MC/DC Consequence: add transformed guard conditions and find test cases that satisfythem [actualweight > minweight] [actualweight > minweight] [actualweight <= minweight] 20/30

11 Quality Measurement with Coverage Criteria Coverage criteria are widely-accepted Concentration on two kinds of coverage criteria: Control-flow-based coverage: - Condition coverage - Decision coverage - MC/DC Boundary-based coverage: - One-boundary coverage - Multidimensional coverage - All-boundaries coverage 21/30 Combination of Coverage Criteria Control-flow coverage criteria Boundary coverage criteria GAP Abstract test cases + Value selection = Concrete test cases Test generation algorithm is based on transformation of guard conditions into conditions on input parameters (= input value partitions ) Apply boundary coverage criteria to abstract test cases for controlflow coverage criteria 22/30

12 ParTeG Example Test goal: Reach state move slow Start at this state and searchbackwards 23/30 ParTeG Example Step backwards Encountered guard: [actualweight > minweight] Conditions to be satisfied: [actualweight > minweight] 24/30

13 ParTeG Example Step backwards Encountered guard: [actualweight <= maxweight] Conditions to be satisfied: [actualweight > minweight] [actualweight <= maxweight] 25/30 ParTeG Example Step backwards Encountered guard: [b <> currentfloor and (b > basement or r > minrank)] Conditions to besatisfied: [actualweight > minweight] [actualweight <= maxweight] [b <> currentfloor and (b > basement or r > minrank)] 26/30

14 ParTeG Example Step backwards Encountered guard: [b <> currentfloor and (b > basement or r > minrank)] Conditions to be satisfied: [actualweight > minweight] [actualweight <= maxweight] Abstract domains for parameter: r: (minrank; 8) b: (basement; currentfloor) or (currentfloor; 8) 27/30 ParTeG Example Step backwards Found path feasible? No -> there are still unsatisfied conditions Conditions to be satisfied: [actualweight > minweight] [actualweight <= maxweight] 28/30

15 ParTeG Example Step backwards Encountered postcondition: [actualweight = w] Abstract domains for parameter: w: (minweight; maxweight] r: (minrank; 8) b: (basement; currentfloor) or (currentfloor; 8) 29/30 ParTeG Example Step backwards Found test case is feasible! Input sequence: insertweight(w) pressbutton(b, r) Abstract domains for parameter: w: (minweight; maxweight] r: (minrank; 8) b: (basement; currentfloor) or (currentfloor; 8) Value selection corresponding to abstract input domains 30/30

ParTeG Combination of Different Models Combination of state machine and class diagram Combined behavioural and structural information Integrating static conditions in control flow Pre-/postconditions

16 ParTeG Combination of Different Models Combination of state machine and class diagram Combined behavioural and structural information Integrating static conditions in control flow Pre-/postconditions of operations Operations can be transition effects Integration of operations conditions in the state machine Inheritance of state machines Redefinition of transitions, etc. results in a different behaviour Impact of structural changes on behaviour? Reuse of state machines along class inheritance 31/30 32/30

17 What Has Been Achieved Testing as an important Software Engineering task Testing as a formal method Test case generation algorithms Embedded Systems as the 21st century computers Specification methodology for Embedded Systems Concrete test generation examples 33/30 What Hasn t Been Discussed Test execution, i.e., how to connect SUT and test suite Improvement of test suites, e.g. by recombinations and genetic algorithms Testing of continuous signal processing Testing from other specification formalisms Test management, roles, documents 34/30

18 What Remains To Be Said You can (and should) do the exercises later (if you haven t done them already) Contact me if there are problems Send me solutions if you want feedback Have fun! 35/30

Specification-based Testing of Embedded Systems H. Schlingloff, SEFM 2008

SEFM School 2008 Specification-based Testing of Embedded Systems Prof. Dr. Holger Schlingloff Humboldt-Universität zu Berlin and Fraunhofer FIRST, Berlin Lecture 4: Mutations, OCL etc. Course Outline L1: