Politehnica University of Timisoara Mobile Computing, Sensors Network and Embedded Systems Laboratory. Testing Techniques

Transcription

1 Politehnica University of Timisoara Mobile Computing, Sensors Network an Embee Systems Laboratory ing Techniques

2 What is testing? ing is the process of emonstrating that errors are not present. The purpose of testing is to show that a program performs its intene functions correctly. ing is the process of establishing confience that a system oes what it is suppose to o. ing is the process of fining errors.

3 Why o we nee a technique? A test technique / test strategy is use to generate tests An effective technique fins bugs ing techniques are base on requirements

4 Example: Car interior lighting system Customer requirements List of requirements: By opening the oor of the car, the interior light shoul be turne on If the oor stays open, the light keeps lightening for half an hour If the oor is close, the interior light will stay on another 15 secons an then it shoul fae out in 3 secons 4

5 Example: Car interior lighting system Automotive requirements open oor 30 min close oor OFF (Default) open oor 15 sec open oor Fae out 3 sec 5

6 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

7 Transaction Flow Moeling Technique s Description Transaction flows are a representation of the system states from the user point of view Ientifies all branches, loops, queues an processes that communicate by messages an efines test cases for these How to use it? You nee a sufficient number of transaction flowcharts to cover all possible transactions Select a sufficient number of paths through those transaction flows to assure complete coverage (every link an every ecision is being exercise at least once) 7

8 Transaction Flow Moeling Technique s Implementation Example One transaction flowchart 1 path = 1 test specification Many paths or combinations of paths to be teste OFF (Default) 1 30 min sec Q: what if a path fails? A = an example of a single path B = an example of a combination of paths 7 Fae out 3 sec 3 A OFF (Default) min 15 sec Fae out 3 sec 30 min 15 sec 30 min Fae out 3 sec OFF (Default) B OFF (Default) OFF (Default) min min 15 sec 15 sec Fae out 3 sec 30 min OFF (Default) Fae out 3 sec 30 min Fae out 3 sec OFF (Default) 8

9 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

10 All Roun-Trip Paths Technique s Description Use to traverse the graph (state machine) an to generate a transition tree Attempts to exercise roun trip paths Covers 100% of the paths roun trip paths = paths that start an en in the same state How to use it? Follow a path an when you meet again a previous state STOP an recor that path Create a test case for every resulte path 10

11 All Roun-Trip Paths Technique s Implementation Example OFF (Default) 30 min 30 min OFF (Default) 15 sec 30 min 1 15 sec Fae out 3 sec OFF (Default) 2 Fae out 3 sec 30 min 3 Fae out 3 sec 30 min 4 OFF (Default) 5 5 paths resulte with this technique => 5 test specifications (DOORS) If one path fails the other paths can pass 11

12 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

13 Loop ing Technique s Description Repetitive processes are ifficult to start or stop correctly Is effective for most graph moels that have loops loops = paths that en where they have starte How to use it? Set preconitions to start a loop Repeat loop N times Check if the results are the same after every cycle N = a etermine number 13

14 Loop ing Technique s Implementation Example Preconition Open oor: set state 30 min Description OFF (Default) 30 min 15 sec 1) Close oor: set state 15 sec 2) Wait 15 sec: set state Fae out 3 sec Fae out 3 sec 3) Open oor: set state 30 min 4) Repeat steps 1, 2, 3 for N times (N = 3) Expecte Result System is in state 30 min the interior light is on for 30 minutes 14

15 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

16 Data Flow ing Technique s Description A ata prouce in one state is expecte to be use later Information receive by a receiving state Rx has to be the same as the one sent from the transmission state Tx Information that passes from state to state is creating a ata flow State 1 i State 2 How to use it? Transmit a ata package from one state to another state Check if the information that arrives at the receiving state is correct by the action that state takes 16

17 Data Flow ing Technique s Implementation Example New requirement ae: If the light is an ignition is an the oor is close the light shoul fae out in 3 secons oor open oor close light on fae out light light off 30 min time elapse ignition on OFF (Default) 15 sec Fae out 3 sec 17

18 Data Flow ing Technique s Implementation Example 30 min 15 sec Fae out 3 sec OFF (Default) serial bus i Ignition on ata shoul be sent from state 30 min to state Fae out 3 sec through state 15 sec which is bypassing the timer

19 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

20 Equivalence Partitioning Technique s Description The input of a omain is ivie in classes of ata How to use it? The preconition is to etermine what type of input is: a value, a range, etc. Divie the input in classes, name partitions They can be: for a value, one vali an two invali partitions for a range, one vali an two invali partitions for a member of a set, one vali an one invali partition for a Boolean value, one vali an one invali partition 20

21 Equivalence Partitioning Technique s Implementation Example Time value set in EEPROM [min] invali partition vali partition invali partition Range of values that are possible to be use: [10 60] minutes All values are ivie in 3 partitions Diviing the range of values gives us the number of tests that can be mae for the bounary value analysis technique 21

22 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

23 Bounary Value Analysis Technique s Description Base on the partitions that where etermine with equivalence partitioning technique, we can etermine the tests that we have to make Bounaries an values aroun them are teste Example for 3 partitions: min-1 min min+1 max-1 max max+1 How to use it? Determine the bounary values an the values aroun them Create a test case for every relevant value Use these values to set preconitions Measure results to be accoringly with the pre-set values 23

24 Bounary Value Analysis Technique s Implementation Example Time value set in EEPROM [ ( ) ] [min] z1 < <= x <= 60 z2 > 60 Preconition 1) EEPROM time value is set to 9 2) Set state 1) EEPROM time value is set to 10 2) Set state 24 Description 1) Check light an start stopwatch 2) Stop stopwatch when light is OFF 1) Check light an start stopwatch 2) Stop stopwatch when light is OFF Expecte Result Light is for X minutes (Resolution = 1) Light is for 10 minutes (Resolution = 1)

25 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

26 Regression ing Technique s Description In a system a new feature is implemente ing the newer version of the system by using the same test cases of the previous version of the system, for the features that are not change How to use it? the system by using the test cases that alreay exist Write new test cases for the omains that inclue the new features Other testing methos can be use to cover all failure possibilities of the system Use the new test cases an test the system further on 26

27 Regression ing Technique s Implementation Example If the requirements were change by aing the Ignition / OFF conition, new tests have to be ae to the specifications The functionality of the system is change with aitional conitions, but the test cases mae for the first version can be reuse For example, the tests mae with All roun-trip paths technique are vali an can be reuse Ignition OFF Ignition For the new feature ae to the system, new test cases have to be create incluing the latest conitions OFF (Default) 30 min ( && ) 15 sec Fae out 3 sec 27

28 Positive vs. Negative ing Techniques comparison Positive ing giving as input vali ata Negative ing giving as input invali ata vali ata SYSTEM invali ata SYSTEM Remark: All tests that have been mae until now are examples of positive testing 28

29 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

30 Negative ing Technique s Description Defining as input an invali ata How to use it? Check the preconitions of the system Define some input values that coul appear, but are unspecifie an seem to be invali for the system Write test cases that uses those values as inputs 30

31 Negative ing Technique s Implementation Example After a complete cycle with the oor open the system is back to it s efault state OFF 30 min Then the oor is close There is nothing specifie in the requirements about this Q: What oes the system shoul o? OFF (Default) Fae out 3 sec 15 sec Preconition Open the oor an let it open for 31 minutes (until light goes OFF) Description 1) Close the oor 2) Check interior light Expecte Result The light shoul NOT turn after the oor is close 31

32 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

33 Error Guessing Technique s Description Base on experience an jugment of the tester Is the art of fining hien errors How to use it? Explore systems functionality Think where a location for an error coul be The errors prouce are mostly place in exceptional places The errors are actions that aren t specifie or unusual accesse 33

34 Error Guessing Technique s Implementation Example After one cycle with the oor open the system is back to it s efault state OFF Then the system is reset Q: Interior light is set to or it shoul stay in OFF state? Cycle 1 Cycle after reset OFF (Default) 30 min 15 sec Fae out 3 sec Preconition Open the oor an let it open for 31 minutes (until light goes OFF) Description 1) Reset system 2) Check interior light Expecte Result The light shoul turn if: oor open & Reset 34

35 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

36 Error Hanling ing Technique s Description A system shoul recognize an locate the error Initial State Error State 36

37 Error Hanling ing Technique s Implementation Example New requirement ae: The system shoul etect an save a DTC (Diagnostic Trouble Coe) if the light bulb is burne I A DTC If current I is 0 Amps then the DTC shoul set 37

38 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

39 Recovery ing Technique s Description A system shoul recover from it s error state to it s initial state Initial State Error State 39

40 Recovery ing Technique s Implementation Example For example, an error coul appear if the bulb is burne or if a shortcircuit was prouce The system nees to locate this error an to announce that the interior lighting system has a problem OFF (Default) 30 min DTC is not active DTC is active Next State Afterwars the bulb is replace with a goo one The system is restarte This time, the error shouln t appear anymore - the system has to recover 40

41 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

42 Stress ing Technique s Description Requires to hol the system in continuous action for a long time System shoul perform uner normal conitions, but it shoul perform uner extreme conitions, too How to use it? Repeating actions repeate times, without giving the system any breaks Every tact the system shoul process something 42

43 Stress ing Technique s Implementation Example ing the timer for the 15 sec state for 100 cycles 30 min OFF (Default) 15 sec Fae out 3 sec Preconition System is in OFF (Default) moe Description 1) Open oor (1s) 2) Close oor (1s) 3) Wait 15s 4) Check light status to be 5) Wait 3s 6) Check light status to be OFF 7) Repeat steps times Expecte Result Time while light is before is completely OFF (fae out) is 20s for every cycle

44 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing

45 Loa ing Technique s Description Putting heavy eman on a system Ientifies the maximum operating capacity of a system Error conitions are the expecte result How to use it? Determine maximum operating capacity Entertain the system with continuous actions over capacity Expect errors when the system is high emane

46 Loa ing Technique s Implementation Example New requirement ae: The maximum power of the lamp is 15W 15W 20W Preconition 5W lamp is mounte Description Set state 30 min Expecte Result System functions correctly 10W (efault) lamp is mounte 15W lamp is mounte 20/25/30W lamp is mounte Set state 30 min System functions correctly Set state 30 min System functions correctly Set state 30 min System not functioning / relay is amage

47 Summary ing Techniques Transaction Flow Moeling All Roun-Trip Paths Loop ing Data Flow ing Equivalence Partitioning Bounary Value Analysis Regression ing Negative ing Error Guessing Error Hanling ing Recovery ing Stress ing Loa ing