Introduction to Software Testing Chapter 2, Sections: 2.1 & 2.2 Overview Graph Coverage Criteria
|
|
- Loreen Wilkerson
- 6 years ago
- Views:
Transcription
1 Introduction to Software Testing Chapter 2, Sections: 2.1 & 2.2 Overview Graph Coverage Criteria Paul Ammann & Jeff Offutt retest/
2 Ch. 2 : Graph Coverage Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source FSMs Applied to Specs DNF Source Specs Source Models Design Use cases Integ Input Introduction to Software Testing (Ch 2) Ammann & Offutt 2
3 Covering Graphs (2.1) Graphs are the most commonly used structure for testing Graphs can come from many sources Control flow graphs Design structure FSMs and statecharts Use cases Tests usually are intended to cover the graph in some way Introduction to Software Testing (Ch 2) Ammann & Offutt 3
4 Definition of a Graph Introduction to Software Testing (Ch 2) Ammann & Offutt 4
5 Three Example Graphs Not a valid graph N 0 = { 0 } N 0 = { 0, 1, 2 } N 0 = { } N f = { 3 } N f = { 7, 8, 9 } N f = { 3 } E = {(0,1),(0,2),(1,3), (2,3)} Introduction to Software Testing (Ch 2) Ammann & Offutt 5
6 Paths in Graphs Path : A sequence of nodes [n 1, n 2,, n M ] Each pair of nodes is an edge Length : The number of edges A single node is a path of length 0 Subpath : A subsequence of nodes in p is a subpath of p Reach (n) : Subgraph that can be reached from n Paths [ 0, 3, 7 ] [ 1, 4, 8, 5, 1 ] [ 2, 6, 9 ] Reach ( 0 ) = { 0, 3, 4, 7, 8, 5, 1, 9 } Reach ( {0, 2} ) = G Reach ( [2,6] ) = { 2, 6, 9 } Introduction to Software Testing (Ch 2) Ammann & Offutt 6
7 Test Paths and SESEs Test Path : A path that starts at an initial node and ends at a final node Test paths represent execution of test cases Some test paths can be executed by many tests Some test paths cannot be executed by any tests SESE graphs : All test paths start at a single node and end at another node Single-entry, single-exit N 0 and N f have exactly one node Double-diamond graph Four test paths [ 0, 1, 3, 4, 6 ] [ 0, 1, 3, 5, 6 ] [ 0, 2, 3, 4, 6 ] [ 0, 2, 3, 5, 6 ] 2 5 Introduction to Software Testing (Ch 2) Ammann & Offutt 7
8 Visiting and Touring Visit : A test path p visits node n if n is in p A test path p visits edge e if e is in p Tour : A test path p tours subpath q if q is a subpath of p Path: [ 0, 1, 3, 4, 6 ] Visits nodes: { 0, 1, 3, 4, 6 } Visits edges: { (0, 1), (1, 3), (3, 4), (4, 6) } Tours subpaths { [0, 1, 3], [1, 3, 4], [3, 4, 6], [0, 1, 3, 4], [1, 3, 4, 6] } Introduction to Software Testing (Ch 2) Ammann & Offutt 8
9 Tests and Test Paths path(t): The test path executed by test t, note: t is a test case path(t): The set of test paths executed by the set of tests T note: T is a Test Set; set of test cases Each test executes one and only one test path (Deterministic) A location in a graph (node or edge) can be reached from another location if there is a sequence of edges from the first location to the second Syntactic reach : A subpath exists in the graph Semantic reach : A test exists that can execute that subpath can you Remember our talk about Reachability? Introduction to Software Testing (Ch 2) Ammann & Offutt 9
10 Tests and Test Paths test 1 many-to-one test 2 test 3 Test Path Deterministic software a test always executes the same test path test 1 test 2 test 3 many-to-many Test Path 1 Test Path 2 Test Path 3 Non-deterministic software a test can execute different test paths Introduction to Software Testing (Ch 2) Ammann & Offutt 10
11 Testing and Covering Graphs: Sec# 2.2 We use graphs in testing as follows : Developing a model of the software as a graph Requiring tests to visit or tour specific sets of nodes, edges, or subpaths Test Requirements (TR) : Properties that describe test paths Test Criterion (C) : Rules that define test requirements (TR) Satisfaction : Given a set TR of test requirements for a criterion C, a set of tests T satisfies C on a graph iff for every test requirement in TR, there is a test path in path(t) that meets the test requirement tr Graph Coverage Structural Coverage Criteria : Defined on a graph just in terms of nodes and edges (Control Flow Coverage Criteria) Data Flow Coverage Criteria : Requires a graph to be annotated with references to variables Introduction to Software Testing (Ch 2) Ammann & Offutt 11
12 Node Coverage The first (and simplest) two criteria require that each node and edge in a graph to be executed Node Coverage (NC) : Test set T satisfies node coverage on graph G iff for every syntactically reachable node n in N, there is some path p in path(t) such that p visits n. This statement is a bit cumbersome, so we abbreviate it in terms of the set of test requirements Node Coverage (NC) : TR contains each reachable node in G. Introduction to Software Testing (Ch 2) Ammann & Offutt 12
13 Edge Coverage Edge coverage is slightly stronger than node coverage Edge Coverage (EC) : TR contains each reachable path of length up to 1, inclusive, in G. The length up to 1 allows for graphs with one node and no edges NC and EC are only different when there is an edge and another subpath between a pair of nodes (as in an if-else statement) 0 Node Coverage : TR = { 0, 1, 2 } Test Path = { [ 0, 1, 2 ] } 1 2 Edge Coverage : TR = { (0,1), (0, 2), (1, 2) } Test Paths = { [ 0, 1, 2 ], [ 0, 2 ] } Introduction to Software Testing (Ch 2) Ammann & Offutt 13
14 Paths of Length 1 and 0 A graph with only one node will not have any edges 0 Formally, Edge Coverage needs to require Node Coverage on this graph Otherwise, Edge Coverage will not subsume (include) Node Coverage So we define length up to 1 instead of simply length 1 We have the same issue with graphs that only have one edge for Edge Pair Coverage 0 1 Introduction to Software Testing (Ch 2) Ammann & Offutt 14
15 Covering Multiple Edges Edge-pair coverage requires pairs of edges, or subpaths of length 2 Edge-Pair Coverage (EPC) : TR contains each reachable path of length up to 2, inclusive, in G. The length up to 2 is used to include graphs that have less than 2 edges EPC is valuable because it is useful to start the software in some state (that is, a node) and then follow transitions (that are, edges) so that the last state of one test is the same as the start state of another test. This type of testing is used to verify that the system is not changed by certain inputs. Introduction to Software Testing (Ch 2) Ammann & Offutt 15
16 Covering Multiple Edges Continue The logical extension is to require all paths Complete Path Coverage (CPC) : TR contains all paths in G. Unfortunately, this is impossible if the graph has a loop, So, a weak compromise is to make the tester decide which paths: Specified Path Coverage (SPC) : TR contains a set S of test paths, where S is supplied as a parameter. Introduction to Software Testing (Ch 2) Ammann & Offutt 16
17 1 0 Structural Coverage Example Node Coverage TR = { 0, 1, 2, 3, 4, 5, 6 } Test Paths: { [ 0, 1, 2, 3, 6 ], [ 0, 1, 2, 4, 5, 4, 6 ] } Edge Coverage TR = { (0,1), (0,2), (1,2), (2,3), (2,4), (3,6), (4,5), (4,6), (5,4) } Test Paths: { [ 0, 1, 2, 3, 6 ], [ 0, 2, 4, 5, 4, 6 ] } Edge-Pair Coverage TR = { [0,1,2], [0,2,3], [0,2,4], [1,2,3], [1,2,4], [2,3,6], [2,4,5], [2,4,6], [4,5,4], [5,4,5], [5,4,6] } Test Paths: { [ 0, 1, 2, 3, 6 ], [ 0, 1, 2, 4, 6 ], [ 0, 2, 3, 6 ], [ 0, 2, 4, 5, 4, 5, 4, 6 ] } Complete Path Coverage Test Paths:{ [ 0, 1, 2, 3, 6 ], [ 0, 1, 2, 4, 6 ], [ 0, 1, 2, 4, 5, 4, 6 ], [ 0, 1, 2, 4, 5, 4, 5, 4, 6 ], [ 0, 1, 2, 4, 5, 4, 5, 4, 5, 4, 6 ], } Introduction to Software Testing (Ch 2) Ammann & Offutt 17
18 Loops in Graphs If a graph contains a loop, it has an infinite number of paths Thus, CPC is not feasible (infeasible) SPC is not satisfactory because the results are subjective and vary with the tester Attempts to deal with loops: 1970s : Execute cycles once (i.e. [4, 5, 4] in previous example, informal) 1980s : Execute each loop, exactly once (formalized) 1990s : Execute loops 0 times, once, more than once (informal description) 2000s : Prime paths Introduction to Software Testing (Ch 2) Ammann & Offutt 18
19 Simple Paths and Prime Paths Simple Path : A path from node n i to n j is simple if no node appears more than once, except possibly the first and last nodes are the same No internal loops A loop is a simple path Includes all other subpaths Prime Path : A 1) simple path that 2) does not appear as a proper subpath of any other simple path. Introduction to Software Testing (Ch 2) Ammann & Offutt 19
20 Simple Paths and Prime Paths So, Prime Paths are: maximal length simple paths But not all Prime Paths Simple Paths: Length 0 [0], [1], [2], [3] Simple Paths: Length 1 [ 0, 1 ], [ 0, 2 ], [ 1, 3 ], [ 2, 3 ], [ 3, 0 ], Simple Paths: Length 2 [ 0, 1, 3 ], [ 0, 2, 3 ], [ 1, 3, 0 ], [ 2, 3, 0 ], [ 3, 0, 1 ], [ 3, 0, 2 ], Simple Paths: Length 3 [ 0, 1, 3, 0 ], [ 0, 2, 3, 0 ], [ 1, 3, 0, 1 ], [ 2, 3, 0, 2 ], [ 3, 0, 1, 3 ], [ 3, 0, 2, 3 ], [ 1, 3, 0, 2 ], [ 2, 3, 0, 1 ] Prime Paths: [ 0, 1, 3, 0 ], [ 0, 2, 3, 0 ], [ 1, 3, 0, 1 ], [ 2, 3, 0, 2 ], [ 3, 0, 1, 3 ], [ 3, 0, 2, 3 ], [ 1, 3, 0, 2 ], [ 2, 3, 0, 1 ] Introduction to Software Testing (Ch 2) Ammann & Offutt 20
21 Prime Path Coverage A simple, elegant, and finite criterion that requires loops to be executed as well as skipped Prime Path Coverage (PPC) : TR contains each prime path in G. Will tour all paths of length 0, 1, It subsumes node and edge coverage Note: The book has a mistake, PPC does not subsume EPC If a node n has an edge to itself, EPC will require: [n, n, m], But [n, n, m] is not prime Introduction to Software Testing (Ch 2) Ammann & Offutt 21
22 1 Simple & Prime Path Example Simple Paths Len 0 [0] [1] [2] [3] [4] [5] [6]! Len 1 [0, 1] [0, 2] [1, 2] [2, 3] [2, 4] [3, 6]! [4, 6]! [4, 5] [5, 4] Len 2 [0, 1, 2] [0, 2, 3] [0, 2, 4] [1, 2, 3] [1, 2, 4] [2, 3, 6]! [2, 4, 6]! [2, 4, 5]! [4, 5, 4] * [5, 4, 6]! [5, 4, 5] * Len 3 [0, 1, 2, 3] [0, 1, 2, 4] [0, 2, 3, 6]! [0, 2, 4, 6]! [0, 2, 4, 5]! [1, 2, 3, 6]! [1, 2, 4, 5]! [1, 2, 4, 6]!! : path terminates cannot be extended * : path cycles 6 5 Len 4 [0, 1, 2, 3, 6]! [0, 1, 2, 4, 6]! [0, 1, 2, 4, 5]! Prime Paths??? Introduction to Software Testing (Ch 2) Ammann & Offutt 22
23 Prime Path Example The previous example has 38 simple paths Only nine prime paths Prime Paths [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 5 ] [ 0, 1, 2, 4, 6 ] [ 0, 2, 3, 6 ] [ 0, 2, 4, 5] [ 0, 2, 4, 6 ] [ 5, 4, 6 ] [ 4, 5, 4 ] [ 5, 4, 5 ] Execute loop 0 times Execute loop once Execute loop more than once Introduction to Software Testing (Ch 2) Ammann & Offutt 23
24 Touring, Sidetrips and Detours Prime paths do not have internal loops test paths might Tour : A test path p tours subpath q iff q is a subpath of p Tour With Sidetrips : A test path p tours subpath q with sidetrips iff every edge in q is also in p in the same order The tour can include a sidetrip, as long as it comes back to the same node Tour With Detours : A test path p tours subpath q with detours iff every node in q is also in p in the same order The tour can include a detour from node n i, as long as it comes back to the prime path at a successor of n i ( only one edge of p can be skipped ) Introduction to Software Testing (Ch 2) Ammann & Offutt 24
25 Sidetrips and Detours Example Touring without sidetrips or detours 3 Touring with a sidetrip Touring with a detour Introduction to Software Testing (Ch 2) Ammann & Offutt 25
26 Round Trips Round-Trip Path : A prime path that starts and ends at the same node Simple Round Trip Coverage (SRTC) : TR contains at least one round-trip path for each reachable node in G that begins and ends a round-trip path. Complete Round Trip Coverage (CRTC) : TR contains all round-trip paths for each reachable node in G. These criteria omit nodes and edges that are not in round trips That is, they do not subsume edge-pair, edge, or node coverage Introduction to Software Testing (Ch 2) Ammann & Offutt 26
27 Infeasible Test Requirements An infeasible test requirement cannot be satisfied Unreachable statement (dead code) A subpath that can only be executed if a contradiction occurs (i.e. X > 0 and X < 0) Most test criteria have some infeasible test requirements It is usually undecidable whether all test requirements are feasible When sidetrips are not allowed, many structural criteria have more infeasible test requirements Always, allowing sidetrips weakens the test criteria Practical recommendation Best Effort Touring Satisfy as many test requirements as possible without sidetrips Allow sidetrips to try to satisfy unsatisfied test requirements Introduction to Software Testing (Ch 2) Ammann & Offutt 27
28 Data Flow Criteria Goal: Try to ensure that values are computed and used correctly Definition (def) : A location where a value for a variable is stored into memory Use : A location where a variable s value is accessed def (n) or def (e) : The set of variables that are defined by node n or edge e use (n) or use (e) : The set of variables that are used by node n or edge e X = Z = X*2 4 5 Z = X-8 6 Defs: def (0) = { X } def (4) = { Z } def (5) = { Z } Uses: use (4) = { X } use (5) = { X } Introduction to Software Testing (Ch 2) Ammann & Offutt 28
29 DU Pairs and DU Paths du-pair : A pair of locations (l i, l j ) such that a variable v is defined at l i and used at l j def-clear : A path from l i to l j is def-clear with respect to variable v if v is not given another value on any of the nodes or edges in the path reach : If there is a def-clear path from l i to l j with respect to v, the def of v at l i reaches the use at l j du-path : A simple subpath that is def-clear with respect to v from a def of v to a use of v du (n i, n j, v) the set of du-paths from n i to n j du (n i, v) the set of du-paths that start at n i Introduction to Software Testing (Ch 2) Ammann & Offutt 29
30 Touring DU-Paths A test path p du-tours subpath d with respect to v if p tours d and the subpath taken is def-clear with respect to v Sidetrips can be used, just as with previous touring Three criteria Use every def Get to every use Follow all du-paths Introduction to Software Testing (Ch 2) Ammann & Offutt 30
31 Data Flow Test Criteria First, we make sure every def reaches a use All-Defs Coverage (ADC) : For each set of du-paths S = du (n, v), TR contains at least one path d in S. Then we make sure that every def reaches all possible uses All-Uses Coverage (AUC) : For each set of du-paths to uses S = du (n i, n j, v), TR contains at least one path d in S. Finally, we cover all the paths between defs and uses All-du-paths Coverage (ADUPC) : For each set S = du (n i, n j, v), TR contains every path d in S. Introduction to Software Testing (Ch 2) Ammann & Offutt 31
32 Data Flow Testing Example Z = X*2 X = Z = X-8 All-defs for X All-uses for X All-du-paths for X [ 0, 1, 3, 4 ] [ 0, 1, 3, 4 ] [ 0, 1, 3, 5 ] [ 0, 1, 3, 4 ] [ 0, 2, 3, 4 ] [ 0, 1, 3, 5 ] [ 0, 2, 3, 5 ] Introduction to Software Testing (Ch 2) Ammann & Offutt 32
33 Graph Coverage Criteria Subsumption Complete Path Coverage CPC All-DU-Paths Coverage ADUP Edge-Pair Coverage EPC Prime Path Coverage PPC All-uses Coverage AUC Edge Coverage EC Complete Round Trip Coverage CRTC All-defs Coverage ADC Node Coverage NC Simple Round Trip Coverage SRTC Introduction to Software Testing (Ch 2) Ammann & Offutt 33
Introduction to Software Testing Chapter 2.1, 2.2 Overview Graph Coverage Criteria Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 2.1, 2.2 Overview Graph Coverage Criteria Paul Ammann & Jeff Offutt www.introsoftwaretesting.com Ch. 2 : Graph Coverage Four Structures for Modeling Software Graphs
More informationOverview Graph Coverage Criteria
Overview Graph Coverage Criteria Graph Coverage Four Structures for Modeling Software Graphs Logic Input Space Syntax Applied to Applied to Source FSMs Applied to Specs DNF Source Specs Source Models Design
More informationTesting Object Oriented Software 2010
Testing Object Oriented Software 2010 Graph Coverage Barry van Veen Rick van der Zwet 2.1 Graph coverage Introduction into graph coverage Theoretical definition of
More informationIntroduc)on to So,ware Tes)ng (2nd edi(on) Overview Graph Coverage Criteria. Paul Ammann & Jeff OffuA. hap:// so,waretest/
Tes)ng (2nd edi(on) Overview Graph Coverage Criteria Paul Ammann & Jeff OffuA hap://www.cs.gmu.edu/~offua/ so,waretest/ First version, 23 September 2013 Graph Coverage Four Structures for Modeling Software
More informationIntroduction to Software Testing Chapter 2.3 Graph Coverage for Source Code. Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 2.3 Graph Coverage for Source Code Paul Ammann & Jeff Offutt Overview The most common application of graph criteria is to program source Graph : Usually the control
More informationTesting: Coverage and Structural Coverage
Testing: Coverage and Structural Coverage Testing, Quality Assurance, and Maintenance Winter 2017 Prof. Arie Gurfinkel based on slides by Prof. Marsha Chechik and Prof. Lin Tan How would you test this
More informationIntroduction to Software Testing Chapter 3, Sec# 1 & 2 Logic Coverage
Introduction to Software Testing Chapter 3, Sec# 1 & 2 Logic Coverage Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/soft waretest/ Ch. 3 : Logic Coverage Four Structures for Modeling Software
More informationTesting: Coverage and Structural Coverage
Testing: Coverage and Structural Coverage Testing, Quality Assurance, and Maintenance Winter 2018 Prof. Arie Gurfinkel based on slides by Prof. Marsha Chechik and Prof. Lin Tan How would you test this
More informationCMPT 473 Software Quality Assurance. Graph Coverage. Nick Sumner
CMPT 473 Software Quality Assurance Graph Coverage Nick Sumner Recall: Coverage/Adequacy Can't look at all possible inputs. Need to determine if a test suite covers / is adequate for our quality objectives.
More informationIntroduction to Software Testing Chapter 5.1 Syntax-based Testing
Introduction to Software Testing Chapter 5.1 Syntax-based Testing Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/ softwaretest/ Ch. 5 : Syntax Coverage Four Structures for Modeling Software Graphs
More informationIntroduction to Software Testing Chapter 3.1, 3.2 Logic Coverage Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 3.1, 3.2 Logic Coverage Paul Ammann & Jeff Offutt www.introsoftwaretesting.com Ch. 3 : Logic Coverage Four Structures for Modeling Software Graphs Logic Input Space
More informationIntroduction to Software Testing Chapter 3.1, 3.2 Logic Coverage Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 3.1, 3.2 Logic Coverage Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/softwaretest/ Ch. 3 : Logic Coverage Four Structures for Modeling Software Graphs
More informationTesting: Dataflow Coverage
Testing: Dataflow Coverage Testing, Quality Assurance, and Maintenance Winter 2018 Arie Gurfinkel based on slides by Thibaud Lutellier, Marsha Chechik and Prof. Lin Tan Non-looping Path Selection Problem
More informationIntroduction to Software Testing Chapter 2, Sec#: 2.5 Graph Coverage for Specifications
Introduction to Software Testing Chapter 2, Sec#: 2.5 Graph Coverage for Specifications Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/softwa retest/ Design Specifications A design specification
More informationIntroduction to Software Testing Chapter 4 Input Space Partition Testing
Introduction to Software Testing Chapter 4 Input Space Partition Testing Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/ softwaretest/ Ch. 4 : Input Space Coverage Four Structures for Modeling
More informationGraph Coverage for Source Code. Data Flow Graph Coverage for Source Code
Graph Coverage for Source Code Data Flow Graph Coverage for Source Code 1 Graph Coverage for Design Elements Use of data abstraction and object oriented software has increased importance on modularity
More informationCombinatorial Clause Coverage CoC
Introduction to Software Testing Chapter 3.2 Logic Coverage Paul Ammann & Jeff Offutt Covering Logic Expressions Logic expressions show up in many situations Covering logic expressions is required by the
More informationCOMP 3705 Advanced Software Engineering: Software Testing
COMP 3705 Advanced Software Engineering: Software Testing Prof. Matt Rutherford For Next Week Readings For Midterm: Chapter 1 and Sections 2.1-2.6, 3.1 3.5 New material: Chapter 4 Homework #5 http://mjrutherford.org/teaching/2009/winter/comp3705/homeworks
More informationTesting Role in Unified Approach Coverage: Structural/Coverage Model Based Test Generation from Model Checking (project) Interaction of
Testing Role in Unified Approach Coverage: Structural/Coverage Model Based Test Generation from Model Checking (project) Interaction of Coverage/Model Based Testing Will Not Cover Statistical Methods Partition
More informationThe Spin Model Checker : Part I. Moonzoo Kim KAIST
The Spin Model Checker : Part I Moonzoo Kim KAIST Hierarchy of SW Coverage Criteria Complete Value Coverage CVC (SW) Model checking Complete Path Coverage CPC Concolic testing All-DU-Paths Coverage ADUP
More informationIntroduction to Software Testing Chapter 3, Sec# 3.3 Logic Coverage for Source Code
Introduction to Software Testing Chapter 3, Sec# 3.3 Logic Coverage for Source Code Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/soft waretest/ Logic Expressions from Source Predicates are derived
More informationIntroduction to Software Testing Chapter 2.4 Graph Coverage for Design Elements Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 2.4 Graph Coverage for Design Elements Paul Ammann & Jeff Offutt www.introsoftwaretesting.com OO Software and Designs Emphasis on modularity and reuse puts complexity
More informationLecture 2. White- box Tes2ng and Structural Coverage (see Amman and Offut, Chapter 2)
Lecture 2 White- box Tes2ng and Structural Coverage (see Amman and Offut, Chapter 2) White- box Tes2ng (aka. Glass- box or structural tes2ng) An error may exist at one (or more) loca2on(s) Line numbers
More informationLogic Coverage. Moonzoo Kim School of Computing KAIST. The original slides are taken from Chap. 8 of Intro. to SW Testing 2 nd ed by Ammann and Offutt
Logic Coverage Moonzoo Kim School of Computing KAIST The original slides are taken from Chap. 8 of Intro. to SW Testing 2 nd ed by Ammann and Offutt Covering Logic Expressions Logic expressions show up
More informationLogic Coverage for Source Code
Logic Coverage for Source Code CS 4501 / 6501 Software Testing [Ammann and Offutt, Introduction to Software Testing, Ch. 8] 1 Structural Logic Coverage for Source Code Aim: to identify test requirements
More informationProgram-based Mutation Testing
Program-based Mutation Testing CS 4501 / 6501 Software Testing [Ammann and Offutt, Introduction to Software Testing, Ch. 9.2] 1 Applying Syntax-Based Testing to Programs Test requirements are derived from
More informationAutomated Software Analysis Techniques For High Reliability: A Concolic Testing Approach. Moonzoo Kim
Automated Software Analysis Techniques For High Reliability: A Concolic Testing Approach Moonzoo Kim Contents Automated Software Analysis Techniques Background Concolic testing process Example of concolic
More informationINTRODUCTION TO SOFTWARE ENGINEERING
INTRODUCTION TO SOFTWARE ENGINEERING Structural Testing d_sinnig@cs.concordia.ca Department for Computer Science and Software Engineering Introduction Testing is simple all a tester needs to do is find
More informationLecture 2. White- box Tes2ng and Structural Coverage (see Amman and Offut, Chapter 2)
Lecture 2 White- box Tes2ng and Structural Coverage (see Amman and Offut, Chapter 2) White- box Tes2ng (aka. Glass- box or structural tes2ng) An error may exist at one (or more) loca2on(s) Line numbers
More informationCMPSCI 521/621 Homework 2 Solutions
CMPSCI 521/621 Homework 2 Solutions Problem 1 Direct data dependencies: 3 is directly data dependent on 1 and 5 5 is directly data dependent on 1,3, and 5 7 is directly data dependent on 1,3, and 5 Note,
More informationStructural Testing. (c) 2007 Mauro Pezzè & Michal Young Ch 12, slide 1
Structural Testing (c) 2007 Mauro Pezzè & Michal Young Ch 12, slide 1 Learning objectives Understand rationale for structural testing How structural (code-based or glass-box) testing complements functional
More informationMTAT : Software Testing
MTAT.03.159: Software Testing Lecture 03: White-Box Testing (Textbook Ch. 5) Dietmar Pfahl Spring 2017 email: dietmar.pfahl@ut.ee Lecture Chapter 5 White-box testing techniques (Lab 3) Structure of Lecture
More informationMTAT : Software Testing
MTAT.03.159: Software Testing Lecture 03: White-Box Testing (Textbook Ch. 5) Dietmar Pfahl Spring 2016 email: dietmar.pfahl@ut.ee Lecture Chapter 5 White-box testing techniques (Lab 3) Structure of Lecture
More informationDRAFT: Prepublication draft, Fall 2014, George Mason University Redistribution is forbidden without the express permission of the authors
Chapter 9 Syntax-based Testing Date last generated: September 29, 2014. DRAFT: Prepublication draft, Fall 2014, George Mason University Redistribution is forbidden without the express permission of the
More informationIntroduction to Software Testing
Introduction to Software Testing Paul Ammann and Jeff Offutt Solutions to Exercises Student Version November 18, 2015 Copyright Ammann & Offutt, 2002-2009, all rights reserved. 2 Tell me and I may forget.
More informationEVALUATING THE EFFECTIVENESS OF TEST COVERAGE CRITERIA USING MUTATION ANALYSIS. An evaluation of test coverage criteria in C#
EVALUATING THE EFFECTIVENESS OF TEST COVERAGE CRITERIA USING MUTATION ANALYSIS An evaluation of test coverage criteria in C# Examensarbete inom huvudområdet Informationsteknologi Grundnivå 30 Högskolepoäng
More informationGENERATING COST-EFFECTIVE CRITERIA-BASED TESTS FROM BEHAVIORAL MODELS
GENERATING COST-EFFECTIVE CRITERIA-BASED TESTS FROM BEHAVIORAL MODELS by Nan Li A Dissertation Submitted to the Graduate Faculty of George Mason University In Partial Fulfillment of The Requirements for
More informationProgram Structure. Simple Statements. Program consists of blocks Block. Can be represented by a flow graph. Sequence of statements
An Applicable Family of Data Flow Testing Criteria Assumptions about the program No goto statements with variant records Functions having var parameters By reference Procedural or functional parameters
More informationProgram Structure. Simple Statements. Program consists of blocks Block. Can be represented by a flow graph. Sequence of statements
An Applicable Family of Data Flow Testing Criteria Assumptions about the program No goto statements with variant records Functions having var parameters By reference Procedural or functional parameters
More informationSubject Software Testing Structural Testing
Subject Software Testing Structural Testing Objective: 1. Understand Concept of structural testing 2. How structural (code-based or glass-box) testing complements functional (black-box) testing 3. Recognize
More informationGénie Logiciel Avancé
L3 Mention Informatique Parcours Informatique et MIAGE Génie Logiciel Avancé Part VII : White-Box Test Burkhart Wolff wolff@lri.fr Idea: % Lets exploit the structure of the program!!! (and not, as before
More informationT Reactive Systems: Kripke Structures and Automata
Tik-79.186 Reactive Systems 1 T-79.186 Reactive Systems: Kripke Structures and Automata Spring 2005, Lecture 3 January 31, 2005 Tik-79.186 Reactive Systems 2 Properties of systems invariants: the system
More informationCHAPTER 5 GENERATING TEST SCENARIOS AND TEST CASES FROM AN EVENT-FLOW MODEL
CHAPTER 5 GENERATING TEST SCENARIOS AND TEST CASES FROM AN EVENT-FLOW MODEL 5.1 INTRODUCTION The survey presented in Chapter 1 has shown that Model based testing approach for automatic generation of test
More informationIntroduction au Génie Logiciel Partie3: Test: Static White-Box Test
Licence Mention Informatique L3/S6 2009/10 Introduction au Génie Logiciel Partie3: Test: Static White-Box Test Burkhart Wolff Département Informatique Difficulties with Static Unit Tests so far The generation
More informationAgenda. Predicate Testing. CSE 5321/4321, Ali Sharifara, UTA
Agenda Predicate Testing CSE 5321/4321, Ali Sharifara, UTA 1 Predicate Testing Introduction Basic Concepts Predicate Coverage Summary CSE 5321/4321, Ali Sharifara, UTA 2 Motivation Predicates are expressions
More informationIntegration Testing. Conrad Hughes School of Informatics. Slides thanks to Stuart Anderson
Integration Testing Conrad Hughes School of Informatics Slides thanks to Stuart Anderson 19 February 2010 Software Testing: Lecture 10 1 Unit Test vs Integration Testing 1 The ideal in unit testing is
More informationFeasibility of Testing to Code. Feasibility of Testing to Code. Feasibility of Testing to Code. Feasibility of Testing to Code (contd)
Feasibility of Testing to Code (contd) Feasibility of Testing to Code (contd) An incorrect code fragment for determining if three integers are equal, together with two test cases Flowchart has over 10
More informationPart I: Preliminaries 24
Contents Preface......................................... 15 Acknowledgements................................... 22 Part I: Preliminaries 24 1. Basics of Software Testing 25 1.1. Humans, errors, and testing.............................
More informationTimo Latvala. January 28, 2004
Reactive Systems: Kripke Structures and Automata Timo Latvala January 28, 2004 Reactive Systems: Kripke Structures and Automata 3-1 Properties of systems invariants: the system never reaches a bad state
More informationAdvanced Test Coverage Criteria: Specify and Measure, Cover and Unmask
Advanced Test Coverage Criteria: Specify and Measure, Cover and Unmask Sébastien Bardin & Nikolai Kosmatov joint work with Omar Chebaro, Robin David, Mickaël Delahaye, Michaël Marcozzi, Mike Papadakis,
More informationWhite-Box Testing Techniques II
White-Box Testing Techniques II Software Testing and Verification Lecture 8 Prepared by Stephen M. Thebaut, Ph.D. University of Florida White-Box Testing Topics Logic coverage (lecture I) Dataflow coverage
More informationStructural Testing. Testing Tactics. Why Structural? Structural white box. Functional black box. Structural white box. Functional black box
From Pressman, Software Engineering a practitioner s approach, Chapter 14 and Pezze + Young, Software Testing and Analysis, Chapters 12-13 Structural Testing Software Engineering Andreas Zeller Saarland
More informationCourse Review. Ajitha Rajan. Ajitha Rajan Course Review c 2015
Course Review Ajitha Rajan Ajitha Rajan Course Review c 2015 Software Faults, Errors & Failures Software Fault : A static defect in the software Software Failure : External, incorrect behavior with respect
More informationStructural Testing. White Box Testing & Control Flow Analysis
Structural Testing White Box Testing & Control Flow Analysis Functional vs. Structural Functional Have I built the right product? Tests derived from the program specification Internal Structure ignored
More informationProperties of Criteria. 1. Applicability Property. Criteria
Properties of Criteria Program-based To recognize a good adequacy criteria And to discard poor choices Objective, well-defined properties 1. Applicability Property For every program, there exists an adequate
More informationUsing the code to measure test adequacy (and derive test cases) Structural Testing
Using the code to measure test adequacy (and derive test cases) Structural Testing Objectives To describe a second approach to testing which is geared to find program defects To explain the use of program
More informationStructural Testing. Testing Tactics. Why Structural? Structural white box. Functional black box. Structural white box. Functional black box
From Pressman, Software Engineering a practitioner s approach, Chapter 14 and Pezze + Young, Software Testing and Analysis, Chapters 12-13 Structural Testing Software Engineering Andreas Zeller Saarland
More informationAn Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-uses and Prime Path Coverage
An Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-uses and Prime Path Coverage Nan Li, Upsorn Praphamontripong and Jeff Offutt Software Engineering George Mason University
More informationLogic Coverage from Source Code
Logic Coverage from Source Code Moonzoo Kim School of Computing KAIST The original slides are taken from Chap. 8 of Intro. to SW Testing 2 nd ed by Ammann and Offutt Logic Expressions from Source Predicates
More informationLogic-based test coverage
Eduardo Marques, Vasco Thudichum Vasconcelos Logic-based test coverage Basic approach Clauses and predicates Basic coverage criteria: CC, PC, CoC Structural logic coverage of source code Logic coverage
More informationMTAT : Software Testing
MTAT.03.159: Software Testing Lecture 03: White-Box Testing (Textbook Ch. 5) Spring 2013 Dietmar Pfahl email: dietmar.pfahl@ut.ee Lecture Chapter 5 White-box testing techniques (Lab 3) Structure of Lecture
More information3.4 Deduction and Evaluation: Tools Conditional-Equational Logic
3.4 Deduction and Evaluation: Tools 3.4.1 Conditional-Equational Logic The general definition of a formal specification from above was based on the existence of a precisely defined semantics for the syntax
More informationIntegration Testing. Unit Test vs Integration Testing 1. Unit Testing vs Integration Testing 2. Unit testing: isolation, stub/mock objects
Unit Test vs Testing 1 Testing Conrad Hughes School of Informatics Slides thanks to Stuart Anderson The ideal in unit testing is to isolate a single code unit and test it against its behavioural specification.
More informationTesting Tactics. Structural Testing. Why Structural? Why Structural? Functional black box. Structural white box. Functional black box
ing Tactics Structural ing Functional black box Structural white box Software Engineering Andreas Zeller Saarland University s based on spec covers as much specified behavior as possible s based on code
More informationHaving a BLAST with SLAM
Having a BLAST with SLAM Meeting, CSCI 555, Fall 20 Announcements Homework 0 due Sat Questions? Move Tue office hours to -5pm 2 Software Model Checking via Counterexample Guided Abstraction Refinement
More informationCSE P 501 Compilers. SSA Hal Perkins Spring UW CSE P 501 Spring 2018 V-1
CSE P 0 Compilers SSA Hal Perkins Spring 0 UW CSE P 0 Spring 0 V- Agenda Overview of SSA IR Constructing SSA graphs Sample of SSA-based optimizations Converting back from SSA form Sources: Appel ch., also
More informationTest Case Specifications and Test adequacy. Research Methods - Barbara Russo SwSE - Software and Systems Engineering
Test Case Specifications and Test adequacy Research Methods - Barbara Russo SwSE - Software and Systems Engineering 1 Test Case Selection How do we create tests? Test are defined in terms of their adequacy
More informationModule 11. Directed Graphs. Contents
Module 11 Directed Graphs Contents 11.1 Basic concepts......................... 256 Underlying graph of a digraph................ 257 Out-degrees and in-degrees.................. 258 Isomorphism..........................
More informationChapter S:I. I. Introduction. Examples for Search Problems Search Problem Abstraction
Chapter S:I I. Introduction Examples for Search Problems Search Problem Abstraction S:I-1 Introduction STEIN/LETTMANN 1998-2016 Road Map Problem Search for a route from place A to : A S:I-2 Introduction
More informationReading assignment: Reviews and Inspections
Foundations for SE Analysis Reading assignment: Reviews and Inspections M. E. Fagan, "Design and code inspections to reduce error in program development, IBM Systems Journal, 38 (2&3), 1999, pp. 258-287.
More informationSTRUCTURAL TESTING. AKA White Box Testing. Thanks go to Andreas Zeller for allowing incorporation of his materials. F. Tip and M.
F. Tip and M. Weintraub STRUCTURAL TESTING AKA White Box Testing Thanks go to Andreas Zeller for allowing incorporation of his materials STRUCTURAL TESTING Testing based on the structure of the code Test
More informationQualification of Verification Environments Using Formal Techniques
Qualification of Verification Environments Using Formal Techniques Raik Brinkmann DVClub on Verification Qualification April 28 2014 www.onespin-solutions.com Copyright OneSpin Solutions 2014 Copyright
More information(See related materials in textbook.) CSE 435: Software Engineering (slides adapted from Ghezzi et al & Stirewalt
Verification (See related materials in textbook.) Outline What are the goals of verification? What are the main approaches to verification? What kind of assurance do we get through testing? How can testing
More informationUser Interface Overview. What is Usability Engineering?
User Interface Overview Jeff Offutt http://www.ise.gmu.edu/~offutt/ SWE 432 Design and Implementation of Software for the Web What is Usability Engineering? Requires knowledge of some psychology theory
More informationTesting, Debugging, Program Verification
Testing, Debugging, Program Verification Automated Test Case Generation, Part II Wolfgang Ahrendt & Vladimir Klebanov & Moa Johansson 12 December 2012 TDV: ATCG II /GU 2011-12-12 1 / 17 Recap Specification-/Model-Based
More informationEmbedded Systems 7. Models of computation for embedded systems
Embedded Systems 7 - - Models of computation for embedded systems Communication/ local computations Communicating finite state machines Data flow model Computational graphs Von Neumann model Discrete event
More informationPage 1. Reading assignment: Reviews and Inspections. Foundations for SE Analysis. Ideally want general models. Formal models
Reading assignment: Reviews and Inspections Foundations for SE Analysis M. E. Fagan, "Design and code inspections to reduce error in program development, IBM Systems Journal, 38 (2&3), 999, pp. 258-28.
More informationSemantics with Applications 3. More on Operational Semantics
Semantics with Applications 3. More on Operational Semantics Hanne Riis Nielson, Flemming Nielson (thanks to Henrik Pilegaard) [SwA] Hanne Riis Nielson, Flemming Nielson Semantics with Applications: An
More informationData Flow Coverage 1. Stuart Anderson. Stuart Anderson Data Flow Coverage 1 c 2011
Data Flow Coverage 1 Stuart Anderson Why Consider Data Flow? 1 Control Flow: Statement and branch coverage criteria are weak. Condition coverage and path coverage are more costly and can become infeasible.
More informationSemantics via Syntax. f (4) = if define f (x) =2 x + 55.
1 Semantics via Syntax The specification of a programming language starts with its syntax. As every programmer knows, the syntax of a language comes in the shape of a variant of a BNF (Backus-Naur Form)
More informationLecture 2 Finite Automata
Lecture 2 Finite Automata August 31, 2007 This lecture is intended as a kind of road map to Chapter 1 of the text just the informal examples that I ll present to motivate the ideas. 1 Expressions without
More information7. Testing and Debugging Concurrent Programs
7. Testing and Debugging Concurrent Programs The purpose of testing is to find program failures => A successful test is a test that causes a program to fail. Ideally, tests are designed before the program
More informationFormal Approach in Software Testing
Formal Approach in Software Testing #Abhishek Dixit, #Shivani Goel 1 csed, TIET biodatadixit@yahoo.co.in 2 csed, TIET shivani@tiet.ac.in Abstract Testing is an important activity for checking the correctness
More informationUsing Petri Nets To Test Concurrent Behavior Of Web Applications
Using Petri Nets To Test Concurrent Behavior Of Web Applications Sunitha Thummala & Jeff Offutt Presented by Omar Al-Debagy Outline Introduction Scope of the Research Paper Petri Nets Modelling Web Applications
More informationComputational problems. Lecture 2: Combinatorial search and optimisation problems. Computational problems. Examples. Example
Lecture 2: Combinatorial search and optimisation problems Different types of computational problems Examples of computational problems Relationships between problems Computational properties of different
More informationOWL DL / Full Compatability
Peter F. Patel-Schneider, Bell Labs Research Copyright 2007 Bell Labs Model-Theoretic Semantics OWL DL and OWL Full Model Theories Differences Betwen the Two Semantics Forward to OWL 1.1 Model-Theoretic
More informationSTRUCTURAL TESTING. AKA White Box Testing. Thanks go to Andreas Zeller for allowing incorporation of his materials. F. Tip and M.
F. Tip and M. Weintraub STRUCTURAL TESTING AKA White Box Testing Thanks go to Andreas Zeller for allowing incorporation of his materials STRUCTURAL TESTING Testing based on the structure of the code Test
More informationOther criteria focus on single elements.
Path Coverage Other criteria focus on single elements. However, all tests execute a sequence of elements - a path through the program. Combination of elements matters - interaction sequences are the root
More information6.170 Lecture 6 Procedure specifications MIT EECS
6.170 Lecture 6 Procedure specifications MIT EECS Outline Satisfying a specification; substitutability Stronger and weaker specifications Comparing by hand Comparing via logical formulas Comparing via
More informationLexical Analysis. Chapter 2
Lexical Analysis Chapter 2 1 Outline Informal sketch of lexical analysis Identifies tokens in input string Issues in lexical analysis Lookahead Ambiguities Specifying lexers Regular expressions Examples
More informationFoundations: Syntax, Semantics, and Graphs
Foundations: Syntax, Semantics, and Graphs Testing, Quality Assurance, and Maintenance Winter 2018 Prof. Arie Gurfinkel based on slides by Ruzica Pizkac, Claire Le Goues, Lin Tan, Marsha Chechik, and others
More informationCoding and Unit Testing! The Coding Phase! Coding vs. Code! Coding! Overall Coding Language Trends!
Requirements Spec. Design Coding and Unit Testing Characteristics of System to be built must match required characteristics (high level) Architecture consistent views Software Engineering Computer Science
More informationInput Validation Testing: A Requirements-Driven, System level, Early Lifecycle Technique
Input Validation Testing: A Requirements-Driven, System level, Early Lifecycle Technique Jane Hayes Jeff Offutt SAIC George Mason University jane.e.hayes@cpmx.saic.com ofut@gmu.edu Support from U.S.National
More informationOn the Expressiveness of Infinite Behavior and Name Scoping in Process Calculi
On the Expressiveness of Infinite Behavior and Name Scoping in Process Calculi Pablo Giambiagi (KTH, Sweden) Gerardo Schneider (IRISA/INRIA) Speaker: Frank D. Valencia (Uppsala Univ., Sweden) FOSSACS 04,
More informationProgramming Embedded Systems
Programming Embedded Systems Overview of testing techniques Monday March 3, 2014 Philipp Rümmer Uppsala University Philipp.Ruemmer@it.uu.se 1/70 Lecture outline Background, testing in general Unit testing
More informationConstraint Programming for Timetabling
Constraint Programming for Faculty of Informatics, Masaryk University Brno, Czech Republic http://www.fi.muni.cz/~hanka ASAP Seminar, 2004 University Course demands for individual students conflicts among
More informationHaving a BLAST with SLAM
Having a BLAST with SLAM # #2 Topic: Software Model Checking via Counter-Example Guided Abstraction Refinement There are easily two dozen SLAM/BLAST/MAGIC papers; I will skim. #3 SLAM Overview INPUT: Program
More informationImplementation of Lexical Analysis
Implementation of Lexical Analysis Outline Specifying lexical structure using regular expressions Finite automata Deterministic Finite Automata (DFAs) Non-deterministic Finite Automata (NFAs) Implementation
More informationSoftware Verification and Validation. Prof. Lionel Briand Ph.D., IEEE Fellow
Software Verification and Validation Prof. Lionel Briand Ph.D., IEEE Fellow 1 White-Box Testing 2 White-Box vs. Black-BoxTesting: Reminder Software Representation (Model) Associated Criteria Test cases
More informationSpecification-based Testing 2
Specification-based Testing 2 Conrad Hughes School of Informatics Slides thanks to Stuart Anderson 26 January 2010 Software Testing: Lecture 5 1 Overview We consider issues in the generation of test cases
More information