Software Testing for Developer Development Testing. Duvan Luong, Ph.D. Operational Excellence Networks

Size: px
Start display at page:

Download "Software Testing for Developer Development Testing. Duvan Luong, Ph.D. Operational Excellence Networks"

Transcription

1 Software Testing for Developer Development Testing Duvan Luong, Ph.D. Operational Excellence Networks

2 Contents R&D Testing Approaches Static Analysis White Box Testing Black Box Testing 4/2/2012 2

3 Development Testing The type of testing that is done inside of the project, by Engineer(s) or by a designated project team member using the project level code library, complete before the Code complete gate. 4/2/2012 3

4 Verification vs. Validation Verification: The results/outputs from the step/activity are compared against the requirements for the step (e.g. the resulting design is compared against the architecture/design specifications) Validation: The results/outputs from the step/activity are comparing against the original requirements (e.g. the resulting design results are compared against the product level requirements. 4/2/2012 4

5 Top Down vs. Bottom Up Testing Top Down Testing: The highest level software components/modules are test first. Stubs are used to simulate the lower level components/modules to make the testing possible. Bottom Up Testing: The lowest level software components/modules are test first. Drivers are used to drive the testing. 4/2/2012 5

6 Static vs. Dynamic Testing Static Testing: Testing without the need to run the software program. Dynamic Testing: Testing with the execution of the software program 4/2/2012 6

7 White Box vs. Black Box Testing (Structural vs. Functional Testing) White Box Testing: Test cases are developed with the knowledge and access to source code structural elements. Black Box Testing: Test cases are developed with the emphasis on the software functional behaviors and specifications. No visibility to the underlying source code is required. 4/2/2012 7

8 Static Testing 4/2/2012 8

9 Static Testing Techniques People techniques: Desk checking: Individual checks work systematically Use of check lists Simulates execution of code Peer review Give it to someone to check (the buddy system) Walkthrough (for understanding) Author guides the group through a document and his thought processes Team agreement on changes to make Review (for decision making) Group discusses document and makes a decision about the content Formal Technical Review (to find defects) Formal individual and group checking according to specific rules Use of source and standards Static analysis: (tool support available) Control Flow Analysis Unreachable nodes Infinite loops Multiple entry/exit to/from loops check code structure (reducible) Any jumps to undefined labels any labels or functions not referenced Cyclomatic complexity and other metrics Data Flow Analysis (study of program variables) Variable is defines where a value is stored into it Variable is used where the stored value is accessed Variable is re-defined without being used Variable is used before it has been defined 4/2/2012 9

10 Static Testing Summary Benefits Can find errors difficult to see Gives objective quality assessment of code Some code may be easier to inspect than to test Many problems traditionally found in unit testing can be found during design and code inspections Limitations Does not execute the code, so not working under operating conditions Effectiveness Reviews/Inspections can find 50% - 75% of all defects in the requirements and design phases of development Source: Bob Grady, Software Metrics, HP Press 4/2/

11 White Box Testing 4/2/

12 Directed Graph Concept A way to represent the structural characteristics of software code: A node represents a begin or an end of an edge or code segment in the program An edge (or code segment) represents the sequence of code statements between the two nodes. If one statement in the edge or the code sequence is executed, the rest of the statements in the sequence will also be executed A decision node represents the splitting of the execution flow A looping node represents the iteration of the execution flow A way to model the complexity of software code: The code complexity is the number potential independent paths that can cover the edges in the program When the code elements are in sequence, the combined complexity is the multiplication of individual element complexity When the code elements are in parallel, the combined complexity is the summation of individual complexity 4/2/

13 A Directed Graph Example for a Function Begin Statement; ; ; If ( ---) Then ; ; Else ; ; endif; ; Function call(x); ; While (----) do ; ; ; enddo; ; ; Case (-----) of 1: ; 2: ; 3: ; 4: ; 5: ; 6: ; endcase; ; If (----) then ; if (----) then ; ; else ; ; endif; else ; ; endif; ; End. 4/2/

14 A Directed Graph Example for Functions Input Function 2 Function 1 Function 3 Function 4 Function 5 Output 4/2/

15 How to use Directed Graph Complexity To understand the software structural complexity To estimate the required testing effort (minimum number of test cases required) 2 possible branches 1 + n possible branches (where n = 0,1,2,,n number of iteration) 6 possible branches 3 possible branches Total Complexity = 72 (n=1 iteration) Minimum number of test cases = 6 4/2/

16 Code Coverage and Minimum Test Case Requirements Minimum number of test cases 100% Statement Coverage 6 100% Branch Coverage 6 Path Coverage no iteration 36 Path Coverage 1 iteration 72 Path Coverage 2 iterations 108 Path Coverage n iterations 36 + (36 x 2**(n-1)) 4/2/

17 Example of Code Coverage Complexity Input 72 Function 2 Function 1 18 Function Function 4 3 Function 5 Complexity = 72 x 18 x 12 x x 4 x 3 = Output 4/2/

18 White Box Testing (Structural Testing) IEEE STD defines White-box Testing as: Testing that takes into account the internal mechanism of a system or component Objectives Test cases are derived from the software's internal structure (code) or internal design specification Testing used to confirm that the actual structure matches the intended structure Specification are of secondary importance in test case design Specifications must be used to determine proper results A measure of completeness (coverage): The percentage of code exercised in testing Code Coverage: Statement Coverage Branch/segment Coverage Condition Combination Coverage Program Function Coverage Path Coverage Expected Results Inputs System Under Test Outputs 3 Pass Fail 4/2/

19 Code Path Coverage Statement and Branch Coverage are accepted today as the minimum mandatory testing requirement: Statement coverage is established as a minimum testing requirement in the IEEE unit test standard (ANSI87B) Statement and branch coverage have also been used for more than two decades as minimum mandatory unit test requirements for new code at IBM and other major compute and software companies Statement Coverage: Each statement is executed at least once by a test suite; some braches (code segments) may not be traversed Weakest criterion in path testing Typical ad hoc testing achieves 60-75% coverage (Beizer) Branch Coverage: Each branch or decision takes on all possible outcomes by a test suite Includes Statement Coverage Typical ad hoc testing achieves 40-60% coverage (Beizer) 4/2/

20 Minimum Path Criteria and Limits Minimum Path Selection Criteria Exercise every statement at least once Exercise every condition at least once in each direction Limitations of Simple Path Testing: May not detect totally wrong or missing functions Can't detect all problems associated with loops Can't detect all initialization errors Data value errors may be missed Loop minimum criteria: Bypass loop altogether 1 pass through loop 2 passes through loop Typical number of passes through loop 1 less than the maximum number of passes Maximum number of passes Attempt one more than the maximum number of passes 4/2/

21 Branch and Branches Combinations Test Description Branch Trace A 1 right S E B 1 wrong S E 4 S 1 do Branch Pairs S 1 A B S A Covered by Tests 1 3 B What isn't covered and why? All branchs covered. What does Branch Pairs show? Branch 1 A B 2 A Covered by Tests 3 B 4 A B 3 6 if if 2 wrong fail right 5 pass A B 4 5 A 4 6 B Create new tests 5 A 6 B E 5 E A 6 E B 4/2/

22 Branch Pair Coverage Test Description Branch Trace A 1 right (r) S E Branch Pairs Covered by Tests B 1 wrong (w) S E S S 1 A B D C no input S 4 6 E D r, w, r answer S E 4 1 do S A D 1 3 B D C Note: Test A can be eliminated, since Test D covers all are pairs covered by Test A. 3 6 if if 2 wrong fail right 5 pass 2 1 D 2 4 A D 3 1 D 3 4 B 4 5 A D 4 6 B C E 5 E A D 6 E B C 4/2/

23 Branch Pair Coverage Test Description Branch Trace A right- right S E B Wrong - wrong S E C no input S 4 6 E D r, w, r answer S E 4 S 1 3 do if 2 wrong right Branch Triples S 1 2 A D S 1 3 C S S 4 6 C D A D D B Covered by Tests D Branch Triples What isn't covered and why? Covered by Tests D 6 if fail 5 pass A D B E 4 5 E A D 4 6 E B C 4/2/

24 Branch Triple Coverage Extended 4 S E do if if 2 wrong fail right 5 pass Test Description Branch Trace A right, right S E B Wrong, wrong S E C no input S 4 6 E D Branch Triples NOT covered r, w, r answer S E Description right answers in a row a right answer last but failed test wrong answers in a row wrong answer last but passed test Note: Every structural technique generates infeasible things the deeper you go 4/2/

25 Benefits White Box testing Summary Can be efficient at finding certain types of defects Requires finite number of tests to satisfy objectives (coverage criteria) Measurable against coverage criteria Can be mechanized (in theory) Considerations Prerequisites: Requires careful selection of a strategy Coverage criteria are essential Limitations: Subject to proximity blindness Can't detect missing features Harder to do than functional testing Less maintainable than function testing Effectiveness: Can catch 50% - 75% of defects found in R&D Testing Will catch 25% - 40% of total defects Tends to find more obvious defects than subtle defects At most, 50% of testing effort Source: Boris Beizer, An Overview of Software Testing Technology, PNSC, /2/

26 Black Box Testing 4/2/

27 Black Box Testing (Behavioral Testing) IEEE STD defines Black-box Testing as: A system or component whose inputs, outputs, and general function are known but whose contents or implementation are unknown or irrelevant Testing Techniques: è è è è è è è è è Equivalence Partitioning Boundary-value Analysis Error Guessing Cause-effect Graphing Control-flow Testing Transaction-flow Testing Domain Testing Syntax Testing State Transition (Finite-state) Testing Objectives: Use specifications, knowledge, and intuition to derive test Demonstrate that the product conforms to its specifications Find defects Expected Results Inputs System Under Test Outputs 3 Pass Fail 4/2/

28 Equivalence Partitioning (EP) Identifies a set of class input / output conditions (valid or invalid), where each class covers a set of other possible tests Assumption: The function under test behaves the same for each value within a given equivalence class Some Guidelines for developing EP tests: Range of numbers, define 1 valid and 2 invalid classes Specify value, define 1 valid and 2 invalid Set of n values, define 1 valid and 1 invalid class Boolean (must be) value, define 1 valid and 1 invalid class Integer 1 to valid invalid 4/2/

29 Boundary-value Analysis (BVA) A refinement of Equivalence Partitioning looking at conditions directly on, above and below the edges of equivalence classes Some Guidelines for developing BVA tests: Range of numbers, test cases just above and just below the end points of the range Set of n values, exercise the minimum and maximum values as well as values just above and below these limits Special Case: negative Integer 1 to valid invalid 4/2/

30 Example Days until Christmas A program calculates the number of days until Christmas from a specified date, starting from June 1st. The program reads two integer values representing the month and day of the chosen date Equivalence Classes for Month valid invalid Condition Valid Class Invalid Class Valid Boundary Invalid Boundary Input Month 6 thru 12 < 6 6 > non-int /2/

31 Example Days until Christmas Month = 7, 8, 10, Month = 6, 9, Equivalence Classes for Day depend on Month value valid invalid Condition Valid Class Invalid Class Valid Boundary Invalid Boundary Month 6 thru 12 < 6 > 12 non-int Input Day (Month= 7, 8, 10, 12) 1 thru 31 < 1 > 31 non-int Day (Month= 6, 9, 11) 1 thru 30 < 1 > 30 non-int /2/

32 Example Days until Christmas Equivalence Classes for No. Days valid invalid Condition Valid Class Invalid Class Valid Boundary Invalid Boundary Output No. Days -6 thru 207 < -6-6 > /2/

33 Example Days until Christmas Tag Conditions Condition Valid Class Invalid Class Valid Boundary Invalid Boundary Month 6 thru 12 V1 < 6 > 12 non-int X1 X2 X VB1 VB XB1 XB2 XB3 Input Day (Month= 7, 8, 10, 12) 1 thru 31 V2 < 1 > 31 non-int X4 X5 X VB3 VB XB4 XB5 Day (Month= 6, 9, 11) 1 thru 30 V3 < 1 > 30 non-int X7 X8 X VB5 VB XB6 XB7 Output No. Days -6 thru 207 V4 < -6 > 207 X10 X VB7 VB XB8 XB9 4/2/

34 Example Days until Christmas Test Cases Valid Class Invalid Class Valid Boundary Invalid Boundary Month 6 thru 12 V1 < 6 > 12 non-int X1 X2 X VB1 VB XB1 XB2 XB3 Input Day (Month= 7, 8, 10, 12) 1 thru 31 V2 < 1 > 31 non-int X4 X5 X VB3 VB XB4 XB5 Day (Month= 6, 9, 11) 1 thru 30 V3 < 1 > 30 non-int X7 X8 X VB5 VB XB6 XB7 Output No. Days -6 thru 207 V4 < -6 > 207 X10 X VB7 VB XB8 XB9 Sept 9 June 1 4/2/

35 Example Days until Christmas Test Cases Test Case Description Expected Outcome Conditions Covered 1 Month = 9 Day = V1, V3, V4 2 Month = 6 Day = VB1, VB5, VB8 3 Month = 12 Day = 31-6 VB2, VB4, VB7 4 4/2/

36 Example Days until Christmas Conditions Covered Condition Valid Class Invalid Class Valid Boundary Invalid Boundary Month 6 thru 12 V1 < 6 > 12 non-int X1 X2 X VB1 VB XB1 XB2 XB3 Input Day (Month= 7, 8, 10, 12) 1 thru 31 V2 < 1 > 31 non-int X4 X5 X VB3 VB XB4 XB5 Day (Month= 6, 9, 11) 1 thru 30 V3 < 1 > 30 non-int X7 X8 X VB5 VB XB6 XB7 Output No. Days -6 thru 207 V4 < -6 > 207 X10 X VB7 VB XB8 XB9 covered infeasible 4/2/

37 Why do both EP and BVA? Test Objectives? For a thorough approach: Valid Equivalence Partitioning Invalid Equivalence Partitioning Valid Boundary Value Invalid Boundary Value If under time pressure (depends on the test objectives): Minimal user-confidence: Maximum error finding: Valid Equivalence Partitioning Valid Boundary Value then Invalid 4/2/

38 Error Guessing An ad hoc process to identify tests that are considered to expose errors. These tests are based on intuition and experience. The basic idea is to make a list of possible errors or error-prone situations and then write tests cases based on the list. Some possible situations to explore: Empty or null lists / strings Zero instances / occurrences Blanks or null characters in strings Negative numbers Extra parameters 4/2/

39 Other Black Box Testing Testing Techniques: Domain Testing Domain Testing is used to test software or portions of software dominated by numerical processing. It is based on specifications (functional test technique) and / or equivalent implementation (structural test technique) information. Syntax Testing Syntax Testing is a systematic method of generating valid and invalid input data to command-driven software. This approach of testing is a shotgun method that relies on creating many test cases. State transition (Finite-state) Testing The State Transition Testing is an analytical method to design tests for programs that have many similar, but slightly different, control functions. It has payoff in functional design verification. Some areas that call for finite-state testing: menu-driven software, protocols, and device drivers. 4/2/

40 Black Box testing Summary Benefits Catches many of the defects that users would see Objective assessment based on documented requirements Provides confidence that the unit (and system) meets requirements Can be used during any phase of testing -- unit through system Considerations Prerequisites: Requires expert product knowledge Potential effectiveness limited by the accuracy and completeness of the specification Require creative use of intuition Limitations: Can become resource intensive (when to quit) Won't exercise all the code Difficult to mechanize Effectiveness: Can catch 10% - 30% of defects found in Unit Testing Will catch 5% - 15% of total defects Catches the most blatant defects May consume as much as 50% of testing effort Source: Boris Beizer, An Overview of Software Testing Technology, PNSC, /2/

41 Development Testing Summary 4/2/

42 Development Testing Strategy 1 Start with Black Box techniques Sensible functional tests (breadth tests) More exploratory tests, boundaries, positive and negative, combinations (depth tests) Recommended Functional techniques: Equivalence Partitioning and Boundary Value Analysis always worth doing, a good way to start at any level Other functional techniques for additional testing, where appropriate and useful Error Guessing always worth including at any level, after using other techniques even if only for a few minutes Play Time for high test levels when there is something interesting to play with Variety of techniques is best: use a few quickly, rather than one to great detail 4/2/

43 Development Testing Strategy 2 For more thorough testing, use White Box techniques Measure coverage achieved by functional tests Derive additional tests to increase coverage as required Recommended Structural techniques: Statement, Branch, etc. where tool support available for unit testing level Module, etc. where tool or software support available for integration and system testing levels Branch Combination where there is a need to explore particular areas to lower testing levels, or to ensure a basic coverage at higher testing levels Structural techniques should be used with care, they usually require more effort than functional techniques 4/2/

44 Development Testing Effectiveness Testing Type % Defects Found in Unit Testing % Total Defects Static Analysis 75% * Black Box Testing 10% - 30% 5% - 15% White Box Testing 50% - 75% 25% - 40% * Percent of all design and requirements defects (Source: Boris Beizer, An Overview of Software Testing Technology, PNSQC, 1990) 4/2/

Facts About Testing. Cost/benefit. Reveal faults. Bottom-up. Testing takes more than 50% of the total cost of software development

Facts About Testing. Cost/benefit. Reveal faults. Bottom-up. Testing takes more than 50% of the total cost of software development Reveal faults Goals of testing Correctness Reliability Usability Robustness Performance Top-down/Bottom-up Bottom-up Lowest level modules tested first Don t depend on any other modules Driver Auxiliary

More information

Aerospace Software Engineering

Aerospace Software Engineering 16.35 Aerospace Software Engineering Verification & Validation Prof. Kristina Lundqvist Dept. of Aero/Astro, MIT Would You...... trust a completely-automated nuclear power plant?... trust a completely-automated

More information

CS 4387/5387 SOFTWARE V&V LECTURE 4 BLACK-BOX TESTING

CS 4387/5387 SOFTWARE V&V LECTURE 4 BLACK-BOX TESTING 1 CS 4387/5387 SOFTWARE V&V LECTURE 4 BLACK-BOX TESTING Outline 2 Quiz Black-Box Testing Equivalence Class Testing (Equivalence Partitioning) Boundary value analysis Decision Table Testing 1 3 Quiz - 1

More information

Darshan Institute of Engineering & Technology for Diploma Studies

Darshan Institute of Engineering & Technology for Diploma Studies CODING Good software development organizations normally require their programmers to follow some welldefined and standard style of coding called coding standards. Most software development organizations

More information

7.0 Test Design Techniques & Dynamic Testing

7.0 Test Design Techniques & Dynamic Testing 7.0 Test Design Techniques & Dynamic Testing Test Design Techniques 7.1 The Test Development Process 7.2 Categories of Test Design Techniques 7.3 Specification based or Black Box Techniques 7.4 Structure

More information

Software Quality Assurance. David Janzen

Software Quality Assurance. David Janzen Software Quality Assurance David Janzen What is quality? Crosby: Conformance to requirements Issues: who establishes requirements? implicit requirements Juran: Fitness for intended use Issues: Who defines

More information

Introduction to Software Engineering

Introduction to Software Engineering Introduction to Software Engineering (CS350) Lecture 17 Jongmoon Baik Testing Conventional Applications 2 Testability Operability it operates cleanly Observability the results of each test case are readily

More information

People tell me that testing is

People tell me that testing is Software Testing Mark Micallef mark.micallef@um.edu.mt People tell me that testing is Boring Not for developers A second class activity Not necessary because they are very good coders 1 What is quality?

More information

[IT6004-SOFTWARE TESTING] UNIT 2

[IT6004-SOFTWARE TESTING] UNIT 2 1. List the two basic Testing strategies. UNIT 2 Black box testing. White box testing. 2. What are the knowledge sources for Black box testing? Requirements Document specification Domain knowledge Defect

More information

Software Engineering Software Testing Techniques

Software Engineering Software Testing Techniques Software Engineering Software Testing Techniques 1 Testability Operability it it operates cleanly Observability the the results of each test case are readily observed Controllability the the degree to

More information

Software Testing Fundamentals. Software Testing Techniques. Information Flow in Testing. Testing Objectives

Software Testing Fundamentals. Software Testing Techniques. Information Flow in Testing. Testing Objectives Software Testing Fundamentals Software Testing Techniques Peter Lo Software Testing is a critical element of software quality assurance and represents the ultimate review of specification, design and coding.

More information

1 Visible deviation from the specification or expected behavior for end-user is called: a) an error b) a fault c) a failure d) a defect e) a mistake

1 Visible deviation from the specification or expected behavior for end-user is called: a) an error b) a fault c) a failure d) a defect e) a mistake Sample ISTQB examination 1 Visible deviation from the specification or expected behavior for end-user is called: a) an error b) a fault c) a failure d) a defect e) a mistake 2 Regression testing should

More information

Topics in Software Testing

Topics in Software Testing Dependable Software Systems Topics in Software Testing Material drawn from [Beizer, Sommerville] Software Testing Software testing is a critical element of software quality assurance and represents the

More information

Input Space Partitioning

Input Space Partitioning CMPT 473 Software Quality Assurance Input Space Partitioning Nick Sumner Recall Testing involves running software and comparing observed behavior against expected behavior Select an input, look at the

More information

CS 520 Theory and Practice of Software Engineering Fall 2018

CS 520 Theory and Practice of Software Engineering Fall 2018 Today CS 52 Theory and Practice of Software Engineering Fall 218 Software testing October 11, 218 Introduction to software testing Blackbox vs. whitebox testing Unit testing (vs. integration vs. system

More information

Equivalence Class Partitioning and Boundary Value Analysis -Black Box Testing Techniques

Equivalence Class Partitioning and Boundary Value Analysis -Black Box Testing Techniques Equivalence Class Partitioning and Boundary Value Analysis -Black Box Testing Techniques In this tutorial, we will discuss the approach to design the test cases and also how to apply the boundary value

More information

Software Testing. Software Testing

Software Testing. Software Testing Software Testing Software Testing Error: mistake made by the programmer/ developer Fault: a incorrect piece of code/document (i.e., bug) Failure: result of a fault Goal of software testing: Cause failures

More information

Computer Science and Software Engineering University of Wisconsin - Platteville 9-Software Testing, Verification and Validation

Computer Science and Software Engineering University of Wisconsin - Platteville 9-Software Testing, Verification and Validation Computer Science and Software Engineering University of Wisconsin - Platteville 9-Software Testing, Verification and Validation Yan Shi SE 2730 Lecture Notes Verification and Validation Verification: Are

More information

Software Testing. Testing: Our Experiences

Software Testing. Testing: Our Experiences Software Testing Testing: Our Experiences Test Case Software to be tested Output 1 Test Case Generation When to Stop? Test Case Software to be tested Verification Output No Enough? Test Coverage Yes A

More information

Chapter 3: Dynamic Testing Techniques

Chapter 3: Dynamic Testing Techniques Chapter 3: Dynamic Testing Techniques " The system was not fully tested to a satisfactory level of quality and resilience before full implementation on 26 October 1992." Extract from the main conclusions

More information

Subject Software Testing Structural Testing

Subject Software Testing Structural Testing Subject Software Testing Structural Testing Objective: 1. Understand Concept of structural testing 2. How structural (code-based or glass-box) testing complements functional (black-box) testing 3. Recognize

More information

Chapter 10. Testing and Quality Assurance

Chapter 10. Testing and Quality Assurance Chapter 10 Testing and Quality Assurance Different styles of doing code review Human Reviewer Code Inspection with continuous integration infrastructure Pinger s testing set up Testing Related topics 1.

More information

Lecture 26: Testing. Software Engineering ITCS 3155 Fall Dr. Jamie Payton

Lecture 26: Testing. Software Engineering ITCS 3155 Fall Dr. Jamie Payton Lecture 26: Testing Software Engineering ITCS 3155 Fall 2008 Dr. Jamie Payton Department of Computer Science University of North Carolina at Charlotte Dec. 9, 2008 Verification vs validation Verification:

More information

Darshan Institute of Engineering & Technology Unit : 9

Darshan Institute of Engineering & Technology Unit : 9 1) Explain software testing strategy for conventional software architecture. Draw the spiral diagram showing testing strategies with phases of software development. Software Testing: Once source code has

More information

INTRODUCTION TO SOFTWARE ENGINEERING

INTRODUCTION TO SOFTWARE ENGINEERING INTRODUCTION TO SOFTWARE ENGINEERING Structural Testing d_sinnig@cs.concordia.ca Department for Computer Science and Software Engineering Introduction Testing is simple all a tester needs to do is find

More information

Input Space Partitioning

Input Space Partitioning CMPT 473 Software Quality Assurance Input Space Partitioning Nick Sumner - Fall 2014 With material from Patrick Lam, Jeff Offutt Recall Testing involves running software and comparing observed behavior

More information

ASTQB Advance Test Analyst Sample Exam Answer Key and Rationale

ASTQB Advance Test Analyst Sample Exam Answer Key and Rationale ASTQB Advance Test Analyst Sample Exam Answer Key and Rationale Total number points = 120 points Total number points to pass = 78 points Question Answer Explanation / Rationale Learning 1 A A is correct.

More information

Testing! Prof. Leon Osterweil! CS 520/620! Spring 2013!

Testing! Prof. Leon Osterweil! CS 520/620! Spring 2013! Testing Prof. Leon Osterweil CS 520/620 Spring 2013 Relations and Analysis A software product consists of A collection of (types of) artifacts Related to each other by myriad Relations The relations are

More information

An Introduction to Systematic Software Testing. Robert France CSU

An Introduction to Systematic Software Testing. Robert France CSU An Introduction to Systematic Software Testing Robert France CSU Why do we need to systematically test software? Poor quality products can Inconvenience direct and indirect users Result in severe financial

More information

Part I: Preliminaries 24

Part I: Preliminaries 24 Contents Preface......................................... 15 Acknowledgements................................... 22 Part I: Preliminaries 24 1. Basics of Software Testing 25 1.1. Humans, errors, and testing.............................

More information

Test design techniques

Test design techniques INF3121 : Software Testing 12. 02. 2015 Lecture 4 Test design techniques Lecturer: Raluca Florea INF3121/ 12.02.2015 / Raluca Florea 1 Overview 1. The test development process 2. Categories of test design

More information

Introduction to Software Testing Chapter 4 Input Space Partition Testing

Introduction to Software Testing Chapter 4 Input Space Partition Testing Introduction to Software Testing Chapter 4 Input Space Partition Testing Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/ softwaretest/ Ch. 4 : Input Space Coverage Four Structures for Modeling

More information

Introduction to Dynamic Analysis

Introduction to Dynamic Analysis Introduction to Dynamic Analysis Reading assignment Gary T. Leavens, Yoonsik Cheon, "Design by Contract with JML," draft paper, http://www.eecs.ucf.edu/~leavens/jml//jmldbc.pdf G. Kudrjavets, N. Nagappan,

More information

Verification and Validation. Assuring that a software system meets a user s needs. Verification vs Validation. The V & V Process

Verification and Validation. Assuring that a software system meets a user s needs. Verification vs Validation. The V & V Process Verification and Validation Assuring that a software system meets a user s needs Ian Sommerville 1995/2000 (Modified by Spiros Mancoridis 1999) Software Engineering, 6th edition. Chapters 19,20 Slide 1

More information

Software Testing. Lecturer: Sebastian Coope Ashton Building, Room G.18

Software Testing. Lecturer: Sebastian Coope Ashton Building, Room G.18 Lecturer: Sebastian Coope Ashton Building, Room G.18 E-mail: coopes@liverpool.ac.uk COMP 201 web-page: http://www.csc.liv.ac.uk/~coopes/comp201 Software Testing 1 Defect Testing Defect testing involves

More information

Examination Questions Time allowed: 1 hour 15 minutes

Examination Questions Time allowed: 1 hour 15 minutes Swedish Software Testing Board (SSTB) International Software Testing Qualifications Board (ISTQB) Foundation Certificate in Software Testing Practice Exam Examination Questions 2011-10-10 Time allowed:

More information

Software technology 7. Testing (2) BSc Course Dr. Katalin Balla

Software technology 7. Testing (2) BSc Course Dr. Katalin Balla Software technology 7. Testing (2) BSc Course Dr. Katalin Balla Contents Testing techniques Static testing techniques Dynamic testing Black box testing White-box testing Testing in the agile environment

More information

MONIKA HEINER.

MONIKA HEINER. LESSON 1 testing, intro 1 / 25 SOFTWARE TESTING - STATE OF THE ART, METHODS, AND LIMITATIONS MONIKA HEINER monika.heiner@b-tu.de http://www.informatik.tu-cottbus.de PRELIMINARIES testing, intro 2 / 25

More information

Test design: Part I. Software Testing: INF3121 / INF4121

Test design: Part I. Software Testing: INF3121 / INF4121 Test design: Part I Software Testing: INF3121 / INF4121 Summary: Week 4 Test development process Analysis / Design / Implementation Categories of test design techniques Static / Dynamic Specification-based

More information

Question 1: What is a code walk-through, and how is it performed?

Question 1: What is a code walk-through, and how is it performed? Question 1: What is a code walk-through, and how is it performed? Response: Code walk-throughs have traditionally been viewed as informal evaluations of code, but more attention is being given to this

More information

Chapter 14 Testing Tactics

Chapter 14 Testing Tactics Chapter 14 Testing Tactics Moonzoo Kim CS Division of EECS Dept. KAIST moonzoo@cs.kaist.ac.kr http://pswlab.kaist.ac.kr/courses/cs550-07 Spring 2007 1 Overview of Ch14. Testing Tactics 14.1 Software Testing

More information

Lecture 20: SW Testing Presented by: Mohammad El-Ramly, PhD

Lecture 20: SW Testing Presented by: Mohammad El-Ramly, PhD Cairo University Faculty of Computers and Information CS251 Software Engineering Lecture 20: SW Testing Presented by: Mohammad El-Ramly, PhD http://www.acadox.com/join/75udwt Outline Definition of Software

More information

Theme 2 Program Design and Testing

Theme 2 Program Design and Testing Theme 2 Program Design and Testing Systematic Testing 1 Learning Objectives to present a few systematic testing techniques that increase the chance of finding defects while keeping the number of test cases

More information

Software Testing. Testing 1

Software Testing. Testing 1 Software Testing Testing 1 Background Main objectives of a project: High Quality & High Productivity (Q&P) Quality has many dimensions reliability, maintainability, interoperability etc. Reliability is

More information

Verification Overview Testing Theory and Principles Testing in Practice. Verification. Miaoqing Huang University of Arkansas 1 / 80

Verification Overview Testing Theory and Principles Testing in Practice. Verification. Miaoqing Huang University of Arkansas 1 / 80 1 / 80 Verification Miaoqing Huang University of Arkansas Outline 1 Verification Overview 2 Testing Theory and Principles Theoretical Foundations of Testing Empirical Testing Principles 3 Testing in Practice

More information

SFWR ENG 3S03: Software Testing

SFWR ENG 3S03: Software Testing (Slide 1 of 52) Dr. Ridha Khedri Department of Computing and Software, McMaster University Canada L8S 4L7, Hamilton, Ontario Acknowledgments: Material based on [?] Techniques (Slide 2 of 52) 1 2 3 4 Empirical

More information

Structural Testing. (c) 2007 Mauro Pezzè & Michal Young Ch 12, slide 1

Structural Testing. (c) 2007 Mauro Pezzè & Michal Young Ch 12, slide 1 Structural Testing (c) 2007 Mauro Pezzè & Michal Young Ch 12, slide 1 Learning objectives Understand rationale for structural testing How structural (code-based or glass-box) testing complements functional

More information

Verification and Validation

Verification and Validation Chapter 5 Verification and Validation Chapter Revision History Revision 0 Revision 1 Revision 2 Revision 3 Revision 4 original 94/03/23 by Fred Popowich modified 94/11/09 by Fred Popowich reorganization

More information

Topic: Software Verification, Validation and Testing Software Engineering. Faculty of Computing Universiti Teknologi Malaysia

Topic: Software Verification, Validation and Testing Software Engineering. Faculty of Computing Universiti Teknologi Malaysia Topic: Software Verification, Validation and Testing Software Engineering Faculty of Computing Universiti Teknologi Malaysia 2016 Software Engineering 2 Recap on SDLC Phases & Artefacts Domain Analysis

More information

Testing Theory. Agenda - What will you learn today? A Software Life-cycle Model Which part will we talk about today? Theory Lecture Plan

Testing Theory. Agenda - What will you learn today? A Software Life-cycle Model Which part will we talk about today? Theory Lecture Plan heory Lecture Plan 2 esting heory Lecture 8 Software Engineering DDC88/DDC93 autumn 28 Department of Computer and Information Science Linköping University, Sweden L - Course Introduction and Overview L2

More information

(From Glenford Myers: The Art of Software Testing)

(From Glenford Myers: The Art of Software Testing) A Testing Exercise: (From Glenford Myers: The Art of Software Testing) A program reads three integer values from a card. The three values are interpreted as representing the lengths of the sides of a triangle.

More information

Part 5. Verification and Validation

Part 5. Verification and Validation Software Engineering Part 5. Verification and Validation - Verification and Validation - Software Testing Ver. 1.7 This lecture note is based on materials from Ian Sommerville 2006. Anyone can use this

More information

Software Testing part II (white box) Lecturer: Giuseppe Santucci

Software Testing part II (white box) Lecturer: Giuseppe Santucci Software Testing part II (white box) Lecturer: Giuseppe Santucci 4. White box testing White-box (or Glass-box) testing: general characteristics Statement coverage Decision coverage Condition coverage Decision

More information

Testing & Debugging TB-1

Testing & Debugging TB-1 Testing & Debugging TB-1 Need for Testing Software systems are inherently complex» Large systems 1 to 3 errors per 100 lines of code (LOC) Extensive verification and validiation is required to build quality

More information

Chapter 11, Testing. Using UML, Patterns, and Java. Object-Oriented Software Engineering

Chapter 11, Testing. Using UML, Patterns, and Java. Object-Oriented Software Engineering Chapter 11, Testing Using UML, Patterns, and Java Object-Oriented Software Engineering Outline Terminology Types of errors Dealing with errors Quality assurance vs Testing Component Testing! Unit testing!

More information

Black-box Testing Techniques

Black-box Testing Techniques T-76.5613 Software Testing and Quality Assurance Lecture 4, 20.9.2006 Black-box Testing Techniques SoberIT Black-box test case design techniques Basic techniques Equivalence partitioning Boundary value

More information

Why testing and analysis. Software Testing. A framework for software testing. Outline. Software Qualities. Dependability Properties

Why testing and analysis. Software Testing. A framework for software testing. Outline. Software Qualities. Dependability Properties Why testing and analysis Software Testing Adapted from FSE 98 Tutorial by Michal Young and Mauro Pezze Software is never correct no matter what developing testing technique is used All software must be

More information

Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn. 4. Testing

Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn. 4. Testing 4. vs. Model Checking (usually) means checking the correctness of source code Model Checking means verifying the properties of a model given in some formal (not program code) notation Attention: things

More information

Last time. Reasoning about programs. Coming up. Project Final Presentations. This Thursday, Nov 30: 4 th in-class exercise

Last time. Reasoning about programs. Coming up. Project Final Presentations. This Thursday, Nov 30: 4 th in-class exercise Last time Reasoning about programs Coming up This Thursday, Nov 30: 4 th in-class exercise sign up for group on moodle bring laptop to class Final projects: final project presentations: Tue Dec 12, in

More information

Reasoning about programs

Reasoning about programs Reasoning about programs Last time Coming up This Thursday, Nov 30: 4 th in-class exercise sign up for group on moodle bring laptop to class Final projects: final project presentations: Tue Dec 12, in

More information

AXIOMS OF AN IMPERATIVE LANGUAGE PARTIAL CORRECTNESS WEAK AND STRONG CONDITIONS. THE AXIOM FOR nop

AXIOMS OF AN IMPERATIVE LANGUAGE PARTIAL CORRECTNESS WEAK AND STRONG CONDITIONS. THE AXIOM FOR nop AXIOMS OF AN IMPERATIVE LANGUAGE We will use the same language, with the same abstract syntax that we used for operational semantics. However, we will only be concerned with the commands, since the language

More information

CSE 403: Software Engineering, Fall courses.cs.washington.edu/courses/cse403/16au/ Unit Testing. Emina Torlak

CSE 403: Software Engineering, Fall courses.cs.washington.edu/courses/cse403/16au/ Unit Testing. Emina Torlak CSE 403: Software Engineering, Fall 2016 courses.cs.washington.edu/courses/cse403/16au/ Unit Testing Emina Torlak emina@cs.washington.edu Outline Software quality control Effective unit testing Coverage

More information

Software Testing CS 408

Software Testing CS 408 Software Testing CS 408 1/09/18 Course Webpage: http://www.cs.purdue.edu/homes/suresh/408-spring2018 1 The Course Understand testing in the context of an Agile software development methodology - Detail

More information

Software Testing. 1. Testing is the process of demonstrating that errors are not present.

Software Testing. 1. Testing is the process of demonstrating that errors are not present. What is Testing? Software Testing Many people understand many definitions of testing :. Testing is the process of demonstrating that errors are not present.. The purpose of testing is to show that a program

More information

UNIT-4 Black Box & White Box Testing

UNIT-4 Black Box & White Box Testing Black Box & White Box Testing Black Box Testing (Functional testing) o Equivalence Partitioning o Boundary Value Analysis o Cause Effect Graphing White Box Testing (Structural testing) o Coverage Testing

More information

Software Testing. Software Testing. in the textbook. Chapter 8. Verification and Validation. Verification Techniques

Software Testing. Software Testing. in the textbook. Chapter 8. Verification and Validation. Verification Techniques Software Testing in the textbook Software Testing Chapter 8 Introduction (Verification and Validation) 8.1 Development testing 8.2 Test-driven development 8.3 Release testing 8.4 User testing 1 2 Verification

More information

Software Testing 2.1 INTRODUCTION TO TESTING DESIGN STRATEGIES

Software Testing 2.1 INTRODUCTION TO TESTING DESIGN STRATEGIES 2.1 INTRODUCTION TO TESTING DESIGN STRATEGIES Generally software testing refers a set of activities which are well planned in advance and also conducted systematically. The testing strategy should be flexible

More information

Chap 2. Introduction to Software Testing

Chap 2. Introduction to Software Testing Chap 2. Introduction to Software Testing 2.1 Software Testing Concepts and Processes 2.2 Test Management 1 2.1 Software Testing Concepts and Processes 1. Introduction 2. Testing Dimensions 3. Test Concepts

More information

Testing & Continuous Integration. Kenneth M. Anderson University of Colorado, Boulder CSCI 5828 Lecture 20 03/19/2010

Testing & Continuous Integration. Kenneth M. Anderson University of Colorado, Boulder CSCI 5828 Lecture 20 03/19/2010 esting & Continuous Integration Kenneth M. Anderson University of Colorado, Boulder CSCI 5828 Lecture 20 03/1/20 University of Colorado, 20 1 Goals 2 Review material from Chapter of Pilone & Miles esting

More information

Integrating the classification tree method with combinatorial testing: test selection strategy

Integrating the classification tree method with combinatorial testing: test selection strategy Integrating the classification tree method with combinatorial testing: test selection strategy Eduardo Miranda, PhD. CMU/MSE MSE/NIST Seminar June 7th, 2011 Pittsburgh, PA The Classification Tree Method

More information

CMSC 132: OBJECT-ORIENTED PROGRAMMING II

CMSC 132: OBJECT-ORIENTED PROGRAMMING II CMSC 132: OBJECT-ORIENTED PROGRAMMING II Program Testing Department of Computer Science University of Maryland, College Park Debugging Is Harder Than Coding! Debugging is twice as hard as writing the code

More information

MTAT : Software Testing

MTAT : Software Testing MTAT.03.159: Software Testing Lecture 02: Basic Black-Box and White-Box Testing Techniques (Textbook Ch. 4 & 5) Spring 2018 Dietmar Pfahl email: dietmar.pfahl@ut.ee Structure of Lecture 2 Black-Box vs.

More information

Lecture 15 Software Testing

Lecture 15 Software Testing Lecture 15 Software Testing Includes slides from the companion website for Sommerville, Software Engineering, 10/e. Pearson Higher Education, 2016. All rights reserved. Used with permission. Topics covered

More information

Coding and Unit Testing! The Coding Phase! Coding vs. Code! Coding! Overall Coding Language Trends!

Coding and Unit Testing! The Coding Phase! Coding vs. Code! Coding! Overall Coding Language Trends! Requirements Spec. Design Coding and Unit Testing Characteristics of System to be built must match required characteristics (high level) Architecture consistent views Software Engineering Computer Science

More information

Testing: Test design and testing process

Testing: Test design and testing process Testing: Test design and testing process Zoltán Micskei Based on István Majzik s slides Dept. of Measurement and Information Systems Budapest University of Technology and Economics Department of Measurement

More information

Chapter 9. Software Testing

Chapter 9. Software Testing Chapter 9. Software Testing Table of Contents Objectives... 1 Introduction to software testing... 1 The testers... 2 The developers... 2 An independent testing team... 2 The customer... 2 Principles of

More information

CS 424 Software Quality Assurance & Testing LECTURE 3 BASIC CONCEPTS OF SOFTWARE TESTING - I

CS 424 Software Quality Assurance & Testing LECTURE 3 BASIC CONCEPTS OF SOFTWARE TESTING - I LECTURE 3 BASIC CONCEPTS OF SOFTWARE TESTING - I WHAT IS SOFTWARE TESTING? Testing can find faults in the software but cannot prove that the software is error-free. OBJECTIVES OF SOFTWARE TESTING To test

More information

UNIT-4 Black Box & White Box Testing

UNIT-4 Black Box & White Box Testing Black Box & White Box Testing Black Box Testing (Functional testing) o Equivalence Partitioning o Boundary Value Analysis o Cause Effect Graphing White Box Testing (Structural testing) o Coverage Testing

More information

STRUCTURAL TESTING. AKA White Box Testing. Thanks go to Andreas Zeller for allowing incorporation of his materials. F. Tip and M.

STRUCTURAL TESTING. AKA White Box Testing. Thanks go to Andreas Zeller for allowing incorporation of his materials. F. Tip and M. F. Tip and M. Weintraub STRUCTURAL TESTING AKA White Box Testing Thanks go to Andreas Zeller for allowing incorporation of his materials STRUCTURAL TESTING Testing based on the structure of the code Test

More information

Sample Exam. Certified Tester Foundation Level

Sample Exam. Certified Tester Foundation Level Sample Exam Certified Tester Foundation Level Answer Table ASTQB Created - 2018 American Stware Testing Qualifications Board Copyright Notice This document may be copied in its entirety, or extracts made,

More information

Program Correctness and Efficiency. Chapter 2

Program Correctness and Efficiency. Chapter 2 Program Correctness and Efficiency Chapter 2 Chapter Objectives To understand the differences between the three categories of program errors To understand the effect of an uncaught exception and why you

More information

Write perfect C code to solve the three problems below.

Write perfect C code to solve the three problems below. Fall 2017 CSCI 4963/6963 Week 12 David Goldschmidt goldschmidt@gmail.com Office: Amos Eaton 115 Office hours: Mon/Thu 1:00-1:50PM; Wed 1:00-2:50PM Write perfect C code to solve the three problems below.

More information

(See related materials in textbook.) CSE 435: Software Engineering (slides adapted from Ghezzi et al & Stirewalt

(See related materials in textbook.) CSE 435: Software Engineering (slides adapted from Ghezzi et al & Stirewalt Verification (See related materials in textbook.) Outline What are the goals of verification? What are the main approaches to verification? What kind of assurance do we get through testing? How can testing

More information

Software Engineering Theory. Lena Buffoni (slides by Kristian Sandahl/Mariam Kamkar) Department of Computer and Information Science

Software Engineering Theory. Lena Buffoni (slides by Kristian Sandahl/Mariam Kamkar) Department of Computer and Information Science Software Engineering Theory Lena Buffoni (slides by Kristian Sandahl/Mariam Kamkar) Department of Computer and Information Science 2015-09-20 Title/Lecturer SEPTEMBER 20, 2016 2 Requirement formalization

More information

Overview. State-of-the-Art. Relative cost of error correction. CS 619 Introduction to OO Design and Development. Testing.

Overview. State-of-the-Art. Relative cost of error correction. CS 619 Introduction to OO Design and Development. Testing. Overview CS 619 Introduction to OO Design and Development ing! Preliminaries! All sorts of test techniques! Comparison of test techniques! Software reliability Fall 2012! Main issues: There are a great

More information

Introduction to Software Testing

Introduction to Software Testing Introduction to Software Testing Software Testing This paper provides an introduction to software testing. It serves as a tutorial for developers who are new to formal testing of software, and as a reminder

More information

Quality Assurance in Software Development

Quality Assurance in Software Development Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Graz University of Technology Austria Summer Term 2017 1 / 47 Agenda

More information

Sample Exam Syllabus

Sample Exam Syllabus ISTQB Foundation Level 2011 Syllabus Version 2.9 Release Date: December 16th, 2017. Version.2.9 Page 1 of 46 Dec 16th, 2017 Copyright 2017 (hereinafter called ISTQB ). All rights reserved. The authors

More information

Sample Question Paper. Software Testing (ETIT 414)

Sample Question Paper. Software Testing (ETIT 414) Sample Question Paper Software Testing (ETIT 414) Q 1 i) What is functional testing? This type of testing ignores the internal parts and focus on the output is as per requirement or not. Black-box type

More information

Department of Information Technology

Department of Information Technology FT/GN/68/00/21.04.15 COURSE DELIVERY PLAN - THEORY Page 1 of 6 Department of Information Technology B.Tech : Information Technology Regulation: 2008 Sub. Code / Sub. Name : IT2032 / Software Testing Unit

More information

Testing Tactics. Structural Testing. Why Structural? Why Structural? Functional black box. Structural white box. Functional black box

Testing Tactics. Structural Testing. Why Structural? Why Structural? Functional black box. Structural white box. Functional black box ing Tactics Structural ing Functional black box Structural white box Software Engineering Andreas Zeller Saarland University s based on spec covers as much specified behavior as possible s based on code

More information

Best Practices Process & Technology. Sachin Dhiman, Senior Technical Consultant, LDRA

Best Practices Process & Technology. Sachin Dhiman, Senior Technical Consultant, LDRA Best Practices Process & Technology Sachin Dhiman, Senior Technical Consultant, LDRA Best Quality Software Product Requirements Design Coding Testing 2 Product Requirement Feature Requirement Security

More information

Smart Test Case Quantifier Using MC/DC Coverage Criterion

Smart Test Case Quantifier Using MC/DC Coverage Criterion Smart Test Case Quantifier Using MC/DC Coverage Criterion S. Shanmuga Priya 1, Sheba Kezia Malarchelvi 2 Abstract Software testing, an important phase in Software Development Life Cycle (SDLC) is a time

More information

STRUCTURAL TESTING. AKA White Box Testing. Thanks go to Andreas Zeller for allowing incorporation of his materials. F. Tip and M.

STRUCTURAL TESTING. AKA White Box Testing. Thanks go to Andreas Zeller for allowing incorporation of his materials. F. Tip and M. F. Tip and M. Weintraub STRUCTURAL TESTING AKA White Box Testing Thanks go to Andreas Zeller for allowing incorporation of his materials STRUCTURAL TESTING Testing based on the structure of the code Test

More information

Programming Embedded Systems

Programming Embedded Systems Programming Embedded Systems Lecture 8 Overview of software testing Wednesday Feb 8, 2012 Philipp Rümmer Uppsala University Philipp.Ruemmer@it.uu.se 1/53 Lecture outline Testing in general Unit testing

More information

MTAT : Software Testing

MTAT : Software Testing MTAT.03.159: Software Testing Lecture 02: Basic Black-Box and White-Box Testing Techniques (Textbook Ch. 4 & 5) Spring 2018 Dietmar Pfahl email: dietmar.pfahl@ut.ee Structure of Lecture 2 Black-Box vs.

More information

Software Testing Interview Question and Answer

Software Testing Interview Question and Answer Software Testing Interview Question and Answer What is Software Testing? A process of analyzing a software item to detect the differences between existing and required conditions (i.e., defects) and to

More information

In this Lecture you will Learn: Testing in Software Development Process. What is Software Testing. Static Testing vs.

In this Lecture you will Learn: Testing in Software Development Process. What is Software Testing. Static Testing vs. In this Lecture you will Learn: Testing in Software Development Process Examine the verification and validation activities in software development process stage by stage Introduce some basic concepts of

More information

Hardware versus software

Hardware versus software Logic 1 Hardware versus software 2 In hardware such as chip design or architecture, designs are usually proven to be correct using proof tools In software, a program is very rarely proved correct Why?

More information

What s Next INF 117 Project in Software Engineering

What s Next INF 117 Project in Software Engineering What s Next IN 117 Project in Software Engineering Lecture Notes - Spring Quarter, 2008 Michele Rousseau Set 8 - esting Set 8 - esting 2 Announcements k Drop Boxes We will use drop boxes for the remainder

More information