UNIT-4 Black Box & White Box Testing

Transcription

1 Black Box & White Box Testing Black Box Testing (Functional testing) o Equivalence Partitioning o Boundary Value Analysis o Cause Effect Graphing White Box Testing (Structural testing) o Coverage Testing Statement Coverage Branch And Decision Coverage Path Coverage Comparison of black box and white box testing 1 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

2 :Blackbox Testing: Black box testing is the Software testing method which is used to test the software without knowing the internal structure of code or program. It is also known as Functional Testing. Basically software under test is called as Black-Box, we are treating this as black box & without checking internal structure of software we test the software. In Black Box Testing we just focus on inputs and output of the software system without bothering about internal knowledge of the software program. [Figure of Blackbox Testing] The above Black Box executable program can be any software system you want to test. For example : an operating system like Windows, a website like Google,a database like Oracle or even your own custom application. Under Black Box Testing, you can test these applications by just focusing on the inputs and outputs without knowing their internal code implementation. By applying black-box techniques, you derive a set of test cases that satisfy the following criteria: (1) test cases that reduce, by a count that is greater than one the number of additional test cases that must be designed to achieve reasonable testing, and (2) test cases that tell you something about the presence or absence of classes of errors, rather than an error associated only with the specific test at hand. Advantages: more effective on larger units of code than glass box testing 2 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

3 tester needs no knowledge of implementation, including specific programming languages tester and programmer are independent of each other tests are done from a user s point of view will help to expose any ambiguities or inconsistencies in the specifications test cases can be designed as soon as the specifications are complete Disadvantages: only a small number of possible inputs can actually be tested, to test every possible input stream would take nearly forever without clear and concise specifications, test cases are hard to design there may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried may leave many program paths untested Blackbox Techniques/ Methods: 1. Equivalence partitioning: It is a software test design technique that involves dividing input values into valid and invalid partitions and selecting representative values from each partition as test data. 2. Boundary Value Analysis: It is a software test design technique that involves determination of boundaries for input values and selecting values that are at the boundaries and just inside/ outside of the boundaries as test data. 3. Cause effect graphing: It is a software test design technique that involves identifying the cases (input conditions) and effects (output conditions), producing a Cause-Effect Graph, and generating test cases accordingly. 1. Equivalence partitioning: Equivalence Partitioning also called as equivalence class partitioning. It can be applied at any level of testing and is often a good technique to use first. 3 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

4 The idea behind this technique is to divide (i.e. to partition) a set of test conditions into groups or sets that can be considered the same (i.e. the system should handle them equivalently), hence equivalence partitioning. In equivalence-partitioning technique we need to test only one condition from each partition. This is because we are assuming that all the conditions in one partition will be treated in the same way by the software. If one condition in a partition works, we assume all of the conditions in that partition will work, and so there is little point in testing any of these others. Similarly, if one of the conditions in a partition does not work, then we assume that none of the conditions in that partition will work so again there is little point in testing any more in that partition. Equivalence partitioning is a testing technique where input values set into classes for testing. Valid Input Class = Keeps all valid inputs. Invalid Input Class = Keeps all Invalid inputs. Equivalence classes may be defined according to the following guidelines: 1. If an input condition specifies a range, one valid and two invalid equivalence classes are defined. 2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined. 3. If an input condition specifies a member of a set, one valid and one invalid equivalence class are defined. 4. If an input condition is Boolean, one valid and one invalid class are defined. By applying the guidelines for the derivation of equivalence classes, test cases for each input domain data item can be developed and executed. Example-1: A text field permits only numeric characters Length must be 6-10 characters long Partition according to the requirement should be like this: 4 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

5 [Figure- Example of Equivalence partitioning] While evaluating Equivalence partitioning, values in all partitions are equivalent that s why 0-5 are equivalent, 6 10 are equivalent and are equivalent. At the time of testing, test 4 and 12 as invalid values and 7 as valid one. It is easy to test input ranges 6 10 but harder to test input ranges Testing will be easy in the case of lesser test cases but you should be very careful. Assuming, valid input is 7. That means, you belief that the developer coded the correct valid range (6-10). Example-2: A store in city offers different discounts depending on the purchases made by the individual. In order to test the software that calculates the discounts, we can identify the ranges of purchase values that earn the different discounts. For example, if a purchase is in the range of $1 up to $50 has no discounts, a purchase over $50 and up to $200 has a 5% discount, and purchases of $201 and up to $500 have a 10% discounts, and purchases of $501 and above have a 15% discounts. Now we can identify 4 valid equivalence partitions and 1 invalid partition as shown below. Invalid Valid Partition(No Valid Valid Valid Partition Discounts) Partition(5%) Partition(10%) Partition(15%) $0.01 $1-$50 $51-$200 $201-$500 $501-Above 2. Boundary Value Analysis: It s widely recognized that input values at the extreme ends of input domain cause more errors in system. More application errors occur at the boundaries of input domain. Boundary value analysis testing technique is used to identify errors at boundaries rather than finding those exist in center of input domain. 5 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

6 Boundary value analysis is a next part of Equivalence partitioning for designing test cases where test cases are selected at the edges of the equivalence classes. Boundary value analysis is the process of testing between extreme ends or boundaries between partitions' of the input values. So these extreme ends like Start- End, Lower- Upper, Maximum-Minimum, Just Inside-Just Outside values are called boundary values and the testing is called "boundary value analysis testing". Example-1: Suppose you have very important tool at office, accepts valid User Name and Password field to work on that tool, and accepts minimum 8 characters and maximum 12 characters. Valid range 8-12, Invalid range 7 or less than 7 and Invalid range 13 or more than 13. [Figure-Example of Boundary Value Analysis] Write Test Cases for Valid partition value, Invalid partition value and exact boundary value. Test Cases 1: Consider password length less than 8. Test Cases 2: Consider password of length exactly 8. Test Cases 3: Consider password of length between 9 and 11. Test Cases 4: Consider password of length exactly 12. Test Cases 5: Consider password of length more than 12. Example-2: A store in city offers different discounts depending on the purchases made by the individual. In order to test the software that calculates the discounts, we can identify the ranges of purchase values that earn the different discounts. For example, if a purchase is in the range of $1 up to 6 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

7 $50 has no discounts, a purchase over $50 and up to $200 has a 5% discount, and purchases of $201 and up to $500 have a 10% discounts, and purchases of $501 and above have a 15% discounts. We can identify 4 valid equivalence partitions and 1 invalid partition as shown below. Invalid Valid Partition(No Valid Valid Valid Partition Discounts) Partition(5%) Partition(10%) Partition(15%) $0.01 $1-$50 $51-$200 $201-$500 $501-Above From this table we can identify the boundary values of each partition. We assume that two decimal digits are allowed. Boundary values for Invalid partition: 0.00 Boundary values for valid partition(no Discounts): 1, 50 Boundary values for valid partition(5% Discount): 51, 200 Boundary values for valid partition(10% Discount): 201,500 Boundary values for valid partition(15% Discount): 501, Max allowed number in the software application 3. Cause effect graphing: Cause-Effect Graph graphically shows the connection between a given outcome and all issues that manipulate the outcome. Cause Effect Graph is a black box testing technique. It is also known as Ishikawa diagram because of the way it looks, invented by Kaoru Ishikawa or fish bone diagram. It is generally uses for hardware testing but now adapted to software testing, usually tests external behavior of a system. It is a testing technique that aids in choosing test cases that logically relate Causes (inputs) to Effects (outputs) to produce test cases. A Cause stands for a separate input condition that fetches about an internal change in the system. An Effect represents an output condition, a system transformation or a state resulting from a combination of causes. 7 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

8 According to Myer Cause & Effect Graphing is done through the following steps: Step 1: For a module, identify the input conditions (causes) and actions (effect). Step 2: Develop a cause-effect graph. Step 3: Transform cause-effect graph into a decision table. Step 4: Convert decision table rules to test cases. Each column of the decision table represents a test case. Symbols used in Cause-effect graphs: 8 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

9 The Cause-Effect graph technique restates the requirements specification in terms of logical relationship between the input and output conditions. Since it is logical, it is obvious to use Boolean operators like AND, OR and NOT. The Print message is software that read two characters and, depending of their values, messages must be printed. The first character must be an A or a B. The second character must be a digit. If the first character is an A or B and the second character is a digit, the file must be updated. If the first character is incorrect (not an A or B ), the message X must be printed. If the second character is incorrect (not a digit), the message Y must be printed. Solution: The causes for this situation are: C1 First character is A C2 First character is B C3 Second character is a digit The effects (results) for this situation are E1 Update the file E2 Print message X E3 Print message Y LET S START!! First draw the causes and effects as shown below: Key Always go from effect to cause (left to right). That means, to get effect E,what causes should be true. 9 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

10 In this example, let s start with Effect E1. Effect E1 is to update the file. The file is updated when First character is A and second character is a digit First character is B and second character is a digit First character can either be A or B and cannot be both. Now let s put these 3 points in symbolic form: For E1 to be true following are the causes: C1 and C3 should be true C2 and C3 should be true C1 and C2 cannot be true together. This means C1 and C2 ae mutually exclusive. Now let s draw this: o So as per the above diagram, for E1 to be true the condition is (C1 C2) C3 o The circle in the middle is just an interpretation of the middle point to make the graph less messy. o There is a third condition where C1 and C2 are mutually exclusive. So the final graph for effect E1 to be true is shown below: 10 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

11 Lets move to Effect E2: o E2 states to print message X. Message X will be printed when First character is neither A nor B. o Which means Effect E2 will hold true when either C1 OR C2 is invalid. So the graph for Effect E2 is shown as (In blue line) 11 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

12 For Effect E3. o E3 states to print message Y. Message Y will be printed when Second character is incorrect. Which means Effect E3 will hold true when C3 is invalid. So the graph for Effect E3 is shown as (In Green line) This completes the Cause and Effect graph for the above situation. Now let s move to draw the Decision table based on the above graph. Writing Decision table based on Cause and Effect graph: First write down the Causes and Effects in a single column shown below Key is the same. Go from bottom to top which means traverse from effect to cause. Start with Effect E1. For E1 to be true, the condition is: (C1 C2) C3. Here we are representing True as 1 and False as 0 First put Effect E1 as True in the next column as 12 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

13 Now for E1 to be 1 (true), we have the below two conditions C1 AND C3 will be true C2 AND C3 will be true For E2 to be True, either C1 or C2 has to be false shown as For E3 to be true, C3 should be false. So it s done. Let s complete the graph by adding 0 in the blank column and including the test case identifier. 13 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

14 Writing Test cases from the decision table I am writing a sample test case for test case 1 (TC1) and Test Case 2 (TC2). In a similar fashion, you can create other test cases. (A test case contains many other attributes like preconditions, test data, severity, priority, build, version, release, environment etc. I assume all these attributes to be included when you write the test cases in actual situation) 14 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

15 :Whitebox Testing: White Box Testing (also known as Clear Box Testing, Open Box Testing, Glass Box Testing, Transparent Box Testing, Code-Based Testing or Structural Testing) is a software testing method in which the internal structure/ design/ implementation of the item being tested is known to the tester. The tester chooses inputs to exercise paths through the code and determines the appropriate outputs. Programming know-how and the implementation knowledge is essential. White Box Testing is like the work of a mechanic who examines the engine to see why the car is not moving. Using white-box testing methods, you can derive test cases that (1) guarantee that all independent paths within a module have been exercised at least once (2) exercise all logical decisions on their true and false sides (3) execute all loops at their boundaries and within their operational bounds, and (4) exercise internal data structures to ensure their validity. 15 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

16 LEVELS APPLICABLE TO: White Box Testing method is applicable to the following levels of software testing: Unit Testing: For testing paths within a unit. Integration Testing: For testing paths between units. System Testing: For testing paths between subsystems. However, it is mainly applied to Unit Testing Why and When White-Box Testing: White box testing is mainly used for detecting logical errors in the program code. It is used for debugging a code, finding random typographical errors, and uncovering incorrect programming assumptions. White box testing is done at low level design and implementable code. It can be applied at all levels of system development especially Unit, system and integration testing. White box testing can be used for other development artifacts like requirements analysis, designing and test cases. Advantages: o Side effects of having the knowledge of the source code is beneficial to thorough testing. o Optimization of code by revealing hidden errors and being able to remove these possible defects. o Gives the programmer introspection because developers carefully describe any new implementation. o Provides traceability of tests from the source, allowing future changes to the software to be easily captured in changes to the tests. o White box tests are easy to automate. o White box testing give clear, engineering-based, rules for when to stop testing 16 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

17 Disadvantages: o Since tests can be very complex, highly skilled resources are required, with thorough knowledge of programming and implementation. o Test script maintenance can be a burden if the implementation changes too frequently. o White box testing very expensive type of testing. o If the application under test large is size then exhaustive testing is impossible. Whitebox Testing Techniques: A major White box testing technique is Code Coverage analysis. Code Coverage analysis, eliminates gaps in a test case suite. It identifies areas of a program that are not exercised by a set of test cases. Once gaps are identified, you create test cases to verify untested parts of code, thereby increase the quality of the software product There are automated tools available to perform Code coverage analysis Following are Whitebox testing techniques: 1. Statement coverage: This technique is aimed at exercising all programming statements with minimal tests. 2. Branch and decision coverage: This technique is running a series of tests to ensure that all branches are tested at least once. Tools: An example of a tool that handles branch coverage testing for C, C++ and Java applications is TCAT-PATH 3. Path coverage: This technique corresponds to testing all possible paths which means that each statement and branch is covered. 1. Statement coverage: Statement coverage is a white box testing technique, which involves the execution of all the statements at least once in the source code. Through statement coverage we can identify the statements executed and where the code is not executed because of blockage. 17 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

18 Using this technique we can check what the source code is expected to do and what it should not. It can also be used to check the quality of the code and the flow of different paths in the program. The statement coverage covers only the true conditions. The main drawback of this technique is that we cannot test the false condition in it. The statement coverage is count as per below formula: (Statement coverage = No of statements Executed/Total no of statements in the source code * 100) Example: Read A Read B if A>B Print A is greater than B else Print "B is greater than A" endif Set1 :If A =5, B =2 No of statements Executed: 5 Total no of statements in the source code: 7 Statement coverage =5/7*100 = % Set1 :If A =2, B =5 No of statements Executed: 6 Total no of statements in the source code: 7 Statement coverage =6/7*100 = % Example-2: The average cost of an insurance premium for drivers is 300, however, this premium can increase or decrease depending on three factors: Age, gender and marital status. Drivers that are below the age of 25, male and single face an additional premium increase of If a 18 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

19 driver outside of this bracket is married or female their premium reduces by 100, and if there are aged between 46 and 65 inclusive, their premium goes down by another 150. [Figure- Flow chart for drivers insurance premium calculation] [Figure- Flow Graph for Statement Coverage] Figure 19 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

20 Note: Statement coverage must satisfy each node or statements are traversed at least once TC1: {age=20, gender= male, married = false} Path P1: {A, C, G} TC2: {age=60, gender= male, married=true} Path P2: {A, B, D, E, F, G} [Table of Test cases for Statement Coverage] 2. Branch/Decision coverage: Branch coverage is also known as Decision coverage or all-edges coverage. It covers both the true and false conditions unlikely the statement coverage. A branch is the outcome of a decision, so branch coveragesimply measures which decision outcomes have been tested. This sounds great because it takes a more in-depth view of the source code than simple statement coverage A decision is an IF statement, a loop control statement (e.g. DO-WHILE or REPEAT-UNTIL), or a CASE statement, where there are two or more outcomes from the statement. With an IF statement, the exit can either be TRUE or FALSE, depending on the value of the logical condition that comes after IF. The formula to calculate decision coverage is: Decision Coverage=(Number of decision outcomes executed/total number of decision outcomes)*100% 20 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

21 Research in the industries have shown that even if through functional testing has been done it only achieves 40% to 60% decision coverage. Decision coverage is stronger that statement coverage and it requires more test cases to achieve 100% decision coverage. Example-1: READ X READ Y IF X > Y PRINT X is greater that Y ENDIF To get 100% statement coverage only one test case is sufficient for this pseudo-code. TEST CASE 1: X=10 Y=5 However this test case won t give you 100% decision coverage as the FALSE condition of the IF statement is not exercised. In order to achieve 100% decision coverage we need to exercise the FALSE condition of the IF statement which will be covered when X is less than Y. So the final TEST SET for 100% decision coverage will be: TEST CASE 1: X=10, Y=5 TEST CASE 2: X=2, Y=10 Note: 100% decision coverage guarantees 100% statement coverage but 100% statement coverage does not guarantee 100% decision coverage. Example-2: Read A Read B IF A+B > 10 THEN Print "A+B is Large" ENDIF If A > 5 THEN Print "A Large" 21 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

22 ENDIF The above logic can be represented by a flowchart as: Result: o To calculate Branch Coverage, one has to find out the minimum number of paths which will ensure that all the edges are covered. In this case there is no single path which will ensure coverage of all the edges at once. The aim is to cover all possible true/false decisions. o (1) 1A-2C-3D-E-4G-5H o (2) 1A-2B-E-4F o Hence Branch Coverage is Path coverage: The basis path method enables the test-case designer to derive a logical 22 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

23 complexity measure of a procedural design and use this measure as a guide for defin-ing a basis set of execution paths. Test cases derived to exercise the basis set are guar-anteed to execute every statement in the program at least one time during testing. Path coverage refers to designing test cases such that all linearly independent paths in the program are executed at least once. A linearly independent path can be defined in terms of what s called a control flow graph of an application 23 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

24 Common Example: Read P Read Q IF P+Q > 100 THEN Print Large ENDIF If P > 50 THEN Print P Large ENDIF The flow chart is Statement Coverage (SC): To calculate Statement Coverage, find out the shortest number of paths following which all the nodes will be covered. Here by traversing through path 1A-2C-3D-E-4G-5H all the nodes are 24 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

25 covered. So by traveling through only one path all the nodes are covered, so the Statement coverage in this case is 1. Branch Coverage (BC): To calculate Branch Coverage, find out the minimum number of paths which will ensure covering of all the edges. In this case there is no single path which will ensure coverage of all the edges at one go. By following paths 1A-2C-3D-E-4G-5H, maximum numbers of edges (A, C, D, E, G and H) are covered but edges B and F are left. To covers these edges we can follow 1A- 2B-E-4F. By the combining the above two paths we can ensure of traveling through all the paths. Hence Branch Coverage is 2. The aim is to cover all possible true/false decisions. Path Coverage (PC): Path Coverage ensures covering of all the paths from start to end. All possible paths are- 1A-2B-E-4F 1A-2B-E-4G-5H 1A-2C-3D-E-4G-5H 1A-2C-3D-E-4F So path coverage is 4. Thus for the above example SC=1, BC=2 and PC=4. Memorize these. 100% LCSAJ coverage will imply 100% Branch/Decision coverage 100% Path coverage will imply 100% Statement coverage 100% Branch/Decision coverage will imply 100% Statement coverage 100% Path coverage will imply 100% Branch/Decision coverage Branch coverage and Decision coverage are same. *LCSAJ = Linear Code Sequence and Jump 25 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA

26 :Comparison of Blackbox Testing and Whitebox Testing: # Black Box Testing White Box Testing Black box testing is the Software White box testing is the software testing 1 testing method which is used to test method in which internal structure is being the software without knowing the known to tester who is going to test the internal structure of code or program. software. 2 This type of testing is carried out by Generally, this type of testing is carried out testers. by software developers Implementation Knowledge is not required to carry out Black Box Testing. Programming Knowledge is not required to carry out Black Box Testing. Testing is applicable on higher levels of testing like System Testing, Acceptance testing. Implementation Knowledge is required to carry out White Box Testing. Programming Knowledge is required to carry out White Box Testing. Testing is applicable on lower level of testing like Unit Testing, Integration testing. 6 Black box testing means White box testing means structural test or functional test or external testing. interior testing. In White Box testing is primarily In Black Box testing is primarily 7 concentrate on the functionality of the concentrate on the testing of program code of the system under test like code structure, system under test. branches, conditions, loops etc. 8 The main aim of this testing to check on what functionality is performing by the system under test. The main aim of White Box testing to check on how System is performing. 9 Black Box testing can be started based White Box testing can be started based on on Requirement Specifications Detail Design documents. documents. 10 The Functional testing, Behavior testing, Close box testing is carried out under Black Box testing, so there is no required of the programming knowledge. The Structural testing, Logic testing, Path testing, Loop testing, Code coverage testing, Open box testing is carried out under White Box testing, so there is compulsory to know about programming knowledge. 26 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA