From Symbolic Execution to Concolic Testing. Daniel Paqué

Transcription

1 From Symbolic Execution to Concolic Testing Daniel Paqué

2 Structure Symbolic Execution Concolic Testing Execution Generated Testing Concurrency in Concolic Testing 2

3 Motivation Software Testing usually accounts for 50% of software development cost [Source: The economic impacts of inadequate infrastructure for software testing, NIST] complex and large Software Systems complicate finding small test suites with high coverage Symbolic Execution automic test case generation high code coverage 3

4 Symbolic Execution - Idea execute the program in symbolic domain explore all possible execution paths for each path the constraints of the branching points are collected generate test input based on the constraints 4

5 Symbolic Execution - Example 5

6 Symbolic Execution - Example symbolic domain: symbolic state path condition 6

7 Symbolic Execution - Example symbolic domain: 7

8 Symbolic Execution - Example symbolic domain: 8

9 Symbolic Execution - Example 1 symbolic domain: 2 9

10 Symbolic Execution - Example 1 symbolic domain:

11 Symbolic Execution - Example 11

12 Limits of Symbolic Execution 1 symbolic domain: 2 12

13 Solution Mix Symbolic Execution with Concrete Execution Concolic Testing Execution Generated Testing 13

14 Symbolic Execution 1979 J.C. King mix concrete with symbolic execution Concolic Testing 2005, Godefroid, Sen + improvements in constraint solving Execution Generated Testing (EGT) 2006 Cadar et. al 14

15 Concolic Testing execute program with concrete values and collect symbolic constraints during execution explore paths sequentially instead of forking infer input for next execution use concrete values to solve problematic constraints 15

16 Concolic Testing - Example symbolic state: 16

17 Concolic Testing - Example false symbolic state: 17

18 Concolic Testing - Example symbolic domain: new input: 18

19 Concolic Testing - Example 19

20 Concolic Testing - Example symbolic domain: evaluate condition in concrete true 20

21 Concolic Testing - Example symbolic state: evaluate bar() in concrete 21

22 Symbolic Execution 1979 J.C. King mix concrete with symbolic execution Concolic Testing sequential path exploration guided by concrete input Execution Generated Testing (EGT) fork execution for each path guided by symbolic execution 22

23 Execution Generated Testing further differences to Concolic Testing: EGT dynamically checks if all operands are concrete if so the operation can be executed in concrete elsewise the operation is executed symbolical concrete symbolic 23

24 How to deal with concurrent programs? 24

25 Main Challenge Problem Thread 0 Thread 1 25

26 Main Challenge Idea summarize redundant interleavings Thread 0 Thread 1 26

27 jcute Koushik Sen & Gul Agha: (2006) race-detection and flipping algorithm minimize redundant executions in concurrent programs uses vector clocks to identify races 27

28 Redundant Executions Thread t 0 : Thread t 1 : Execution 1: Execution 2: Execution 3: Execution 4: result: {4, 0, 16} {4, 0, 16} {3, 0, 15} {3, 0, 16} 28

29 Redundant Executions Race Detection two events are in a race if they stem from different threads both access the same memory location (without locking) the order both events can be permuted by changing the schedule Execution 1: Execution 2: Execution 3: Execution 4: result: {4, 0, 16} {4, 0, 16} {3, 0, 15} {3, 0, 16} races: 29

30 The Race-Detection and Flipping Algorithm init: generate a random input and a schedule loop: execute code with the generated input and schedule compute the race conditions and symbolic constraints generate a new schedule or a new input continue until all possible distinct execution paths have been explored (depth-first search strategy) 30

31 Generating new inputs/schedules new input: concolic testing new schedule: pick two events which are in a race delay the first event as much as possbile schedule1: schedule 2: 31

32 How to identify races? 32

33 How to identify races? vector clocks - each thread t gets it s own vector clock V t - each memory location gets another two 33

34 Vector Clocks - Example 34

35 Vector Clocks - Algorithm 35

36 Vector Clocks Example 36

37 Vector Clocks Example 37

38 Vector Clocks Example 38

39 Vector Clocks Example 39

40 Vector Clocks Example 40

41 Vector Clock Theorem 41

42 Questions? 42

43 Precise Definitions (just in case) 43

44 Race Relation Simple Defintion: 44

45 sequentially related 45

46 access precedence related 46

47 causally related 47

48 race related 48

49 Race-Detection and Flipping Algorithm Detailled Example 49