Software processes, quality, and standards Black box testing

Transcription

1 Software processes, quality, and standards Black box testing Jaak Tepandi, Jekaterina Tšukrejeva, Stanislav Vassiljev, Pille Haug Tallinn University of Technology Department of Software Science Moodle: Software Quality (Tarkvara kvaliteet) Alternate download: tepandi.ee Version

2 Black-box testing Testing, either functional or non-functional, without reference to the internal structure of the component or system. Black-box test design technique: Procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional, of a component or system without reference to its internal structure (ISTQB). - Equivalence partitioning - Boundary value coverage - Decision tables - Use cases - State transition diagrams This part is about black box functional testing

3 Motivation and overview Advantages of previous methods Problems of previous methods: systematic? Coverage? Predictable volume? Example of error not found by white code testing? Black-box functional Idea How to present specification? Full coverage with tests? Less tests 1: equivalence classes, boundary values Less tests 2: combining multiple inputs into tests +, -, correctness, assumptions? Decision table, use cases, forms, state transition diagrams,...

4 Black-box testing Test Input Program Output Based upon the task / spec Based upon the task /spec

5 Equivalence partitioning Partitioning the input domain into a collection of subsets (or equivalence classes) based on a specified criterion or relation A representative set of tests (sometimes only one test) is usually taken from each equivalence class. For each test from the subset it is assumed that if it fails, then all other tests from this subset fail, and vice versa. So there is no need to test other inputs from the same class (optimization No 1). In addition, when combining values from several inputs, not all their combinations are tested (for example, testing all values, but not all their combinations, optimization No 2)

6 Equivalence partitioning and boundary value coverage One test is correct => so are the others One test finds an error => so do the others The same on the boundaries Is that a separate test?

7 Equivalence classes Ordered interval Test Ordered interval 0 Set Yes Condition No

8 Boundary values Real T -R T-e T T+e R Integer I -M I-1 I I+1 M Ordered set n k n 0 n k-1 n k n k+1 n max

9 Instead of full coverage with tests... Optimisation 1: all data => equivalence classes, boundary => selected items Optimisation 2: not all combinations of inputs => tests

10 Usage of equivalence classes / boundary values Distinguish the input and output data equivalence classes, if necessary also input/output combinations Select data in each equivalence class and in boundary values (optimisation 1) Combine those data into tests (optimisation 2) As many correct data equivalence classes as possible into one test Errors one by one (sometimes also the boundary values) Evaluate corresponding outputs Identify tests, compose a testing plan Test, estimate

11 Equivalence partitioning example Inputs: Volume 1-4: discount 0%. Volume 5-9: discount 8%. Vol 10-25: discount 15% Unit price: Output: batch total price

12 Vol 1-4: discount 0%. 5-9: discount 8%. Vol 10-25: discount 15%. Unit price: With errors? Boundary values? 1 4/5 9/10Jaak Tepandi 25 0

13 Vol 1-4: discount 0%. 5-9: discount 8%. Vol 10-25: discount 15%. Unit price: No errors 1 4/5 9/10Jaak Tepandi ?

14 Vol 1-4: discount 0%. 5-9: discount 8%. Vol 10-25: discount 15%. Unit price: No errors ? 1 4/5 9/10 25 Jaak Tepandi

15 Equivalence partitioning example 2. Vol 1-4: discount 0%. 5-9: discount 8%. Vol 10-25: discount 15%. Unit price: Inputs: Product name (at least 2 classes incl one for correct name) Unit price (at least 3 classes incl 1 for correct) Volume (at least 5 classes incl 3 for correct) Output: Batch total price Number of tests: at least... (correct)+... (errors) +... (boundary value analysis)

16 Examples One input: wait A sec Several inputs: batch total price, fine for delay Equivalence classes / boundary values? Tests?

17 Fine for delay - equivalence partitioning Given: deadline (from incl.) payment (from incl.) amount (S, up to 100M incl.) Fine % per day P, 0<=P<=100 Find: fine and boundary values? To calculate the number of days, a tested function Days from , D (D,M,Y) is used. D (1,1,1990) = 0

18 Decision table testing Decision tables associate conditions with actions (eg, inputs with outputs), resulting in rules. Possible testing criteria: Test every condition (value) at least once Test every action at least once Test every rule at least once How related to equivalence partitioning?

19 Use case testing Possible criteria: At least one test for each use case primary scenario At least one test for each primary and alternative scenario Equivalence partitioning for use case inputs/outputs...

20 Use case testing example Use case name (goal). Login to Webstore. ID (to be associated with tests). Webstore-T1 Actors. Buyer, any agent (or computer) acting for the customer Preconditions,. The Buyer has registered as the Webstore user Postconditions. Success End Condition: Buyer has access to Webstore services. Primary scenario. 1. The system prompts the user to log on 2. The user enters his name and password 3. The system verifies the logon information 4. The system logs user into the system Alternative scenarios.... Jaak Tepandi

21 State transition testing Based on state transition diagrams Possible criteria: Test typical state sequences Test all states Test all transitions Test specific sequences of transitions Test invalid transitions

22 concept preconditions advantages disadvantages Summary about the method results relationship to other methods evaluation tools

23 Takeaway /learning outcomes What is black box testing? Why this name? Idea of equivalence partitioning Idea of boundary value coverage Two optimisations Decision tables, Use cases, State transition diagrams etc When to use? When not?

24 Additional reading (examples) Ian Sommerville. Software Engineering. Ninth Edition. Addison-Wesley, Ch 8. Daniel Galin, Software Quality assurance from theory to implementation, Pearson - Addison-Wesley. Chapter 9.5. Guide to the Software Engineering Body of Knowledge (SWEBOK), IEEE. Chapter 4. Certified Tester Foundation Level Syllabus, ISTQB. Chapter 2. Moodle: Software Quality (Tarkvara kvaliteet). Alternate download: tepandi.ee