Timing Analysis of Event-Driven Programs with Directed Testing

Transcription

1 iming Analysis of Event-Driven Programs with Directed esting Mahdi Eslamimehr Hesam Samimi {eslamimehr, Communications Design Group, SAP Labs

2 alk Outline Introduction oyota UA Case Problem Definition and Previous Work Review of Classic Directed esting Motivating Experiments Our Approach Example VICE: Algorithms and tools Experiment Results Conclusion Event-Based Directed esting improves the -ofart

3

4

5

6

7

8

9 WCE Controllers in safety time-critical embedded systems are expected to finish their tasks within reliable time bounds. Underestimation causes missing deadlines and leads to bugs Overestimation wastes process availability. Question: what is the exact WCE across all inputs? 1. Program P, K is the WCE of all executions of P, if P s WCE never grows beyond K. 2. here is a possible schedule of events and an execution of the program P such that the WCE becomes K.

10 WCE in Literature Dynamic Analysis Random Algorithms: [Bernat et. Al., RSS 02] Genetic Algorithm: [Atanassov et. Al., EWDC 01] Classic Directed esting: [N. Williams and M.Roger, AS 09] Static Analysis [Holsti et. Al., ESA 2000] [C.erdinand, BIS 04] [Gustafsson and Ermedahl, RSS 06]

11 Classic Directed esting Generate concrete inputs one by one each input leads program along a different path On each input execute program both concretely and symbolically concrete execution guides the symbolic execution concrete execution enables symbolic execution to overcome incompleteness of theorem prover End Yes Input = Random While all paths covered? No Concolically execute and collect path constraints symbolic execution helps to generate concrete input for next execution increases coverage Solving constraints and generate inputs

12 Example int double (int v) { return 2*v; void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { ERROR;

13 Example int double (int v) { return 2*v; void testme (int x, int y) { N 2*y == x Y z = double (y); if (z == x) { N x > y+10 Y if (x > y+10) { ERROR; ERROR

14 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); x = 22, y = 7 x = x 0, y = y 0 if (z == x) { if (x > y+10) { ERROR;

15 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); if (z == x) { x = 22, y = 7, z = 14 x = x 0, y = y 0, z = 2*y 0 if (x > y+10) { ERROR;

16 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); if (z == x) { 2*y 0!= x 0 if (x > y+10) { ERROR; x = 22, y = 7, z = 14 x = x 0, y = y 0, z = 2*y 0

17 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); Solve: 2*y 0 == x 0 Solution: x 0 = 2, y 0 = 1 if (z == x) { 2*y 0!= x 0 if (x > y+10) { ERROR; x = 22, y = 7, z = 14 x = x 0, y = y 0, z = 2*y 0

18 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); x = 2, y = 1 x = x 0, y = y 0 if (z == x) { if (x > y+10) { ERROR;

19 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); if (z == x) { x = 2, y = 1, z = 2 x = x 0, y = y 0, z = 2*y 0 if (x > y+10) { ERROR;

20 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { x = 2, y = 1, z = 2 x = x 0, y = y 0, z = 2*y 0 2*y 0 == x 0 ERROR;

21 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); if (z == x) { if (x > y+10) { 2*y 0 == x 0 x 0 < y ERROR; x = 2, y = 1, z = 2 x = x 0, y = y 0, z = 2*y 0

22 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); Solve: (2*y 0 == x 0 ) Æ (x 0 > y ) Solution: x 0 = 30, y 0 = 15 if (z == x) { if (x > y+10) { 2*y 0 == x 0 x 0 < y ERROR; x = 2, y = 1, z = 2 x = x 0, y = y 0, z = 2*y 0

23 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { z = double (y); x = 30, y = 15 x = x 0, y = y 0 if (z == x) { if (x > y+10) { ERROR;

24 Directed esting Approach int double (int v) { Concrete Symbolic return 2*v; concrete symbolic path condition void testme (int x, int y) { Program Error z = double (y); if (z == x) { if (x > y+10) { 2*y 0 == x 0 x 0 > y ERROR; x = 30, y = 15 x = x 0, y = y 0

25 Explicit Path Model Checking raverse all execution paths one by one to detect errors assertion violations program crash uncaught exceptions

26 Explicit Path Model Checking raverse all execution paths one by one to detect errors assertion violations program crash uncaught exceptions

27 Explicit Path Model Checking raverse all execution paths one by one to detect errors assertion violations program crash uncaught exceptions

28 Explicit Path Model Checking raverse all execution paths one by one to detect errors assertion violations program crash uncaught exceptions

29 Explicit Path Model Checking raverse all execution paths one by one to detect errors assertion violations program crash uncaught exceptions

30 Explicit Path Model Checking raverse all execution paths one by one to detect errors assertion violations program crash uncaught exceptions

31 Motivating Experiments branch coverage across testing techniques 100% 75% 50% 25% 0% Binaryree LinkedList BubbleSort Decoder Oscilloscpoe annkuch MsgKernel estradio estusar estspi estadc Random esting GA Classical Directed esting

32 Motivating Experiments esting VS Static Analysis of WCE 1000 WCE Logarithmic Scale Binaryree LinkedList BubbleSort Decoder Oscilloscpoe annkuch MsgKernel estradio estusar estspi estadc Random esting GA Classical Directed esting Static Analysis

33 esting Event Driven Software Classical software: tester only devices a suite of single inputs. Event-Driven software (with real-time behavior): tester must device a suite of event sequences. In each sequence: # of events, types of events, values associated with the events e.g. registers value, and timing of events. Challenge: Quickly generate a small number of challenging event sequences to improve branch coverage.

34 VICE Example Round 1 [<main,(723452)>,<alt1,( 10038)>,<main,_>,<alt1, _>] Constraints: data_1 = msg data2 = msg 2048 < msg msg < 1024 Branch Coverage: 50% (3/6)

35 VICE Example Round 2 [<main,(-338)>,<alt1,(1001)>,<alt2,(6)>,<main, _>] Constraints: msg = s tmp = t s = 512 Branch Coverage: 83% (5/6)

36 VICE Example Round 3 [<main,(-338)>,<alt1,(1001)>,<alt2,(6)>,<main, _>] Constraints: data1 = data2 = msg = s = 512 Branch Coverage: 83% (5/6)

37 VICE Example Round 4 [<main,(512)>,<alt1,(512)>,<main,_>,<alt1, _>] Constraints: - Branch Coverage: 100% (6/6)

38 Event Based Directed esting (EBD) compiler: VirgilProgram! machinecode avrora : machinecode eventsequence! wcet random: ()! eventsequence timeoutcombos: eventsequence! (eventsequence list) concolic: (Virgil program eventseequence! (wcet branchcoverage constraints) solver: constraints! solution generator: solution! eventsequence

39 Algorithm

40 Experiment Results 100% Branch Coverage 75% 50% 25% 0% Binaryree LinkedList BubbleSort Decoder Oscilloscpoe annkuch MsgKernel estradio estusar estspi estadc Random esting GA Classical Directed esting VICE

41 Experiment Results 1000 WCE Logarithmic Scale Binaryree LinkedList BubbleSort Decoder Oscilloscpoe annkuch MsgKernel estradio estusar estspi estadc Random esting GA Classical Directed esting VICE Static Analysis

42 uture Works ormulating timeouts symbolically Using some static information Locate places where wcet happens, and direct execution towards candidates Replace random event generation with a IMR-certified model checker.