Testing Object Oriented Software 2010

Transcription

1 Testing Object Oriented Software 2010 Graph Coverage Barry van Veen Rick van der Zwet

2 2.1 Graph coverage Introduction into graph coverage Theoretical definition of graphs Theoretical definitions of graph coverage Practical applications of how graphs can be obtained from source code

3 Graph definition A graph G is a set N of nodes a set N 0 of initial nodes, where N 0 N a set N f of final nodes, where N f N a set E of edges, where E is a subset of N N Subgraphs a set of initial nodes is N sub N 0 a set of final nodes is N sub N f a set of edges is (N sub N sub ) E Paths [n 1,n 2,...,n M ]

4 Graph definition (continued)

5 Reachability idea Follow arrows and never go back Syntacticle reachability A node n (or an edge e) is reachable from node n i, if their exists a path p from n i to n (or edge e) Semantical reachability There is some input which makes it possible to execute one of these paths

6 Reachability idea (continued) Testpath ((non) deterministic) A path p, possibly of lenght zero, that starts at some node in N 0 and ends at some node in N f

7 Section 2.1 excises

8 2.2 Graph Coverage criteria Coverage criteria define a set of test requirements (TR) in terms of properties of test paths in a graph G. A test T satisfies the criteria on a graph if and only if for every test requirement (tr) in TR, there is at least one test path p in T such that p meets tr. A requirement is met by touring a particular path (or visiting a particular node).

9 Graph Coverage criteria (2) (NC) Node Coverage (EC) Edge... (EPC) Edge-Pair... (PPC) Prime Path... (SRTC) Simple Round Trip... (CRTC) Complete Round Trip.. (CPC) Complete Path... (SPC) Specific Path... sample only. limitation apply.

10 Graph Coverage criteria (3) (NC) Node Coverage TR contains each reachable node in G (EC) Edge Coverage TR contains each reachable path of length up to 1, inclusive, in G (EPC) Edge-Pair Coverage TR contains each reachable path of length up to 2, inclusive, in G

11 Graph Coverage criteria (4)

12 Graph Coverage criteria (5) Simple path path from n i to n j is simple if no node appears more than once except if the first and last node are the same Prime path path from n i to n j that is a simple path and does not appear as a subpath in any other simple path (PPC) Prime Path Coverage TR contains each prime path in G

13 Graph Coverage criteria (6) Round trip a prime path (of nonzero length) that start and ends at the same node (SRTC) Simple Round Trip Coverage TR contains at least one round-trip path for each reachable node in G that begins and ends a round-trip path (CRTC) Complete Round Trip Coverage TR contains all round-trip paths for each reachable node in G (CPC) Complete Path Coverage - TR contains all paths in G (SPC) Specified Path Coverage - TR contains specified paths

14 Touring, Sidetrips and Detours Touring A path p tours a subpath q if and only if q is a subpath of p. So the nodes have to be in exactly the same order. Sidetrips A path p tours a subpath q with sidetrips if and only if every edge in q is also in p in the same order. Detours A path p tours a subpath q with detours if and only if every node in q is also in p in the same order.

15 Touring, Sidetrips and Detours (2)

16 Infeasible Test Requirements Without sidetrips lots of infeasible requirements can exist. One could drop the strict notion of touring and allow sidetrips. Best Effort Touring Test all tr without sidetrips Test all remaining (unmet) tr with sidetrips

17 Finding Prime Test Paths Find paths of length 0 Find paths of length 1... Find paths of length n Mark paths that cannot be extended with "!" Mark paths that are cycles with "*" When you have the full set, eliminate subpaths. Extend longest prime path to beginning and end nodes. Extend second longest prime path... etc At the end one can optimize the set of test paths.

18 Section 2.2 excises

19 Data Flow Criteria When testing a program we should focus on data values. Definition (Def) A location where a value for a variable is assigned Use A Location where a variable's value is accessed Du-pairs The flow of values from defs to uses Du-Path Simple path that is def-clear with respect to v from node n i for which v is in def(n i ) to node n j for which v is in use(n j )

20 Data Flow Criteria (2)

21 Data Flow Criteria (3) Def-Path du(n i, v) is a set of du-paths wrt v starting at node n i Def-Pair du(n i, n j, v) is a set of du-paths wrt v starting at node n i and ending at node n j (ADC) All-Defs Coverage For each def-path set S, TR contains at least on path in S (AUC) All-Uses Coverage For each def-pair set S, TR contains at least one path in S (ADUPC) All-du-Paths Coverage For each def-pair set S, TR contains every path in S

22 Subsumption Relationships among Graph Coverage Criteria Rules of the ``game'': 1) Every use is preceded by a def. 2) Every def reaches at least one use. 3) for every node with multiple outgoing edges a) at least one variable is used on each out edge and b) the same variables are used on each out edge.

23 Subsumption Relationships among Graph Coverage Criteria assignment: draw graph

24 Structural Graph Coverage for Source Code Control Flow Graph (CFG) A graph in which each edge is associated with a branch in the program. Nodes are associated with sequences of statements.

25 Data Flow Graph Coverage for Source Code def: is a location in the program where a value for variable x is stored into memory: x is left side of an assignment. x is actual parameter in call site and value is changed within the method. x is a formal parameter of a method. x is input to the program.

26 Data Flow Graph Coverage for Source Code (continued) use: is a location in the program where variable's value x is accessed (e.g. read from memory): x appears on the right side of an assignment statement. x appears in a conditional test. x is an actual parameter to a method. x is an output of the program. x is an output of a method in a return statement or returned as parameter.

27 Excersises 2.3. w = x; // node 1 if (m > 0){ w++; // node 2 } else { w=2*w; // node 3 } // node 4 (no executable statement) if (y <= 10) { x = 5*y; // node 5 } else { x = 3*y+5; // node 6 } z = w + x; // node 7

28 Graph Coverage for design elements Call graph Visualize the coupling between software components Nodes represent methods Edges represent method calls Coverage criteria can be applied. Might need more than one call graph when testing a class.

29 Graph Coverage for design elements (2) Inheritance and Polymorphism: OO language features makes it difficult to test We can make a inheritance hierarchy Objects need to be instantiated

30 Graph Coverage for design elements (3) Some more coverage criteria OO Call Coverage (edge coverage): each edge must be applied on at least one object for each class OO All Object Call Coverage: OO Call Coverage must be applied for each instantiated object in each class

31 Data Flow Graph Coverage for Design Elements Caller a unit that invokes another unit, the Callee Call site statement that makes the call Actual parameter is in the caller. It is assigned to a formal parameter in the callee.

32 Data Flow Graph Coverage for Design Elements (2) Interface mapping of actual to formal parameters our point of interest, we want to check the appropiate use of formal parameters in the callee Last-def Set of nodes that define a variable x from which there is a def-clear path from the node through the call site to a use in the other unit. First-use The set of nodes that use a variable y, from which there is a path that is def-clear and use-clear from the entry point or the call site to the nodes

33 Inheritance and Polymorphism (Advanced topic) Direct OO du-pairs Indirect OO du-pairs

34 Inheritance and Polymorphism (Advanced topic) (2)

35 Available tools CLANG+LLVM had some nice tools for Flow Graphing especially c all graphs, but this is limited to LLVM byte code and not to actual C code.

36 References Introduction to Software Testing, Paul Ammann & Jeff Offutt, 2008