Active Testing for Concurrent Programs

Transcription

1 Active Testing for Concurrent Programs Pallavi Joshi Mayur Naik Chang-Seo Park Koushik Sen 1/8/2009 ParLab Retreat ParLab, UC Berkeley Intel Research

2 Overview Checking correctness of concurrent programs Active testing directed by Data races Atomicity violations Deadlocks Programmer directed active testing Breakpoints for parallel programs Conclusion

3 Correctness Personal Health Image Retrieval Hearing, Music Dwarfs Speech Parallel Browser Composition & Coordination Language (C&CL) C&CL Compiler/Interpreter Static Verification Parallel Libraries Parallel Frameworks Type Systems Efficiency Sketching Languages Autotuners Legacy Communication & Schedulers Code Synch. Primitives Efficiency Language Compilers OS Libraries & Services Legacy OS Hypervisor Intel Multicore/GPGPU RAMP Manycore Directed Testing Dynamic Checking Debugging with Replay

4 Checking Correctness of Concurrent Programs Multiple threads of execution with shared memory Want correctness for All executions All inputs need to be checked Interference among threads introduces non deterministic bugs All schedules need to be checked

5 Checking All Schedules Model checking Check all schedules systematically Does not scale to large programs Focus on potentially buggy schedules Schedules that may expose bugs Boundary case schedules What are those potentially buggy schedules?

6 Checking All Schedules Model checking Check all schedules systematically Does not scale to large programs Focus on potentially buggy schedules Schedules that may expose bugs Boundary case schedules What are those potentially buggy schedules? Schedules exhibiting data races, atomicity violations, deadlocks

7 Active Testing Check the potentially buggy schedules Practical and efficient Find many bugs quickly Create an actual execution showing a bug How?

8 Active Testing Check the potentially buggy schedules Practical and efficient Find many bugs quickly Create an actual execution showing a bug Actively control scheduler Explore schedules systematically or randomly

9 Active Testing Check the potentially buggy schedules Practical and efficient Find many bugs quickly Create an actual execution showing a bug Actively control scheduler Explore schedules systematically or randomly But, preempt threads at appropriate states so that buggy schedules get created Observe execution for any abnormal behavior (e.g. uncaught exceptions, assertion failures)

10 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

11 Race Directed X = 1; In race if(x == 0) ERROR; [Sen; PLDI 08]

12 Race Directed X = 1; State to preempt State to preempt if(x == 0) ERROR; [Sen; PLDI 08]

13 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

14 Race Directed X = 1; Paused if(x == 0) ERROR; [Sen; PLDI 08]

15 Race Directed X = 1; Paused if(x == 0) ERROR; [Sen; PLDI 08]

16 Race Directed X = 1; Paused if(x == 0) ERROR; Paused [Sen; PLDI 08]

17 Race Directed X = 1; Paused if(x == 0) ERROR; Paused [Sen; PLDI 08]

18 Race Directed X = 1; Paused if(x == 0) ERROR; [Sen; PLDI 08]

19 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

20 lock(l1) Deadlock Directed lock(l2) lock(l2) lock(l1) unlock(l2) unlock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

21 lock(l1) Deadlock Directed lock(l2) lock(l2) lock(l1) unlock(l2) unlock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

22 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

23 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

24 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

25 Paused lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

26 Paused lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

27 Paused lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

28 Paused lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) Paused unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

29 lock(l1) Deadlock Directed Paused lock(l2) unlock(l2) unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) Paused [Joshi, Park, Sen, Naik; Under submission]

30 lock(l1) Deadlock Directed lock(l2) unlock(l2) unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) Paused [Joshi, Park, Sen, Naik; Under submission]

31 lock(l1) Deadlock Directed lock(l2) lock(l2) unlock(l2) unlock(l1) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

32 lock(l1) Deadlock Directed lock(l2) lock(l2) unlock(l2) unlock(l1) Deadlock! lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

33 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

34 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

35 Atomicity Violation Directed t = x; atomic Atomicity Violation! t = 0; x = t + 1; [Park, Sen; FSE 08]

36 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

37 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

38 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

39 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

40 atomic Atomicity Violation Directed t = x; x = t + 1; Paused t = 0; [Park, Sen; FSE 08]

41 atomic Atomicity Violation Directed t = x; x = t + 1; Paused t = 0; Paused [Park, Sen; FSE 08]

42 atomic Atomicity Violation Directed t = x; x = t + 1; Paused t = 0; Paused [Park, Sen; FSE 08]

43 atomic Atomicity Violation Directed t = x; x = t + 1; Paused t = 0; [Park, Sen; FSE 08]

44 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

45 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

46 Bugs Found Tool implemented for Java Races, deadlocks, atomicity violations in Java Collections Framework Deadlocks found and reproduced in Jigsaw web server Java Swing GUI framework Java Database Connectivity (JDBC) Atomicity violations in Apache Commons Collections

47 Lessons Learned Controlling scheduler makes testing effective Focus on potentially buggy schedules Reproduce bugs with high probability Need the right preemption points Directed preemption uncovered many bugs Why not let the programmer choose the preemption points? Breakpoints for concurrent programs Language for describing schedules

48 Schedule Exploration Language Facility to specify thread schedules User specifies breakpoints Preempt a thread at a user specified state User specifies ordering constraints on events Control schedule to meet constraints Testing for logical (semantic) bugs Schedules are given as input to program Either generated automatically (directed) or programmer specified Inputs to unit tests but on schedules

49 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1)

50 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1) break 5

51 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1) break lock(l2)

52 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1) break lock(l2),5

53 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1) break lock(l2), 5, T1

54 Abstraction: Naming Runtime Objects Threads and locks are runtime objects Address changes across runs Need to identify consistently with only static information An abstraction based on object creation site Line number (PC) of where object allocated

55 Abstraction Example main { 1: Lock L1 = new Lock(); 2: Lock L2 = new Lock(); 3: Thread T1 = new MyThread(L1); 4: Thread T2 = new MyThread(L2); 5:... } MyThread(L) { 6: foo(l); } main T1 T2 foo(l1) foo(l2)

56 Abstraction Example main { 1: Lock L1 = new Lock(); 2: Lock L2 = new Lock(); 3: Thread T1 = new MyThread(L1); 4: Thread T2 = new MyThread(L2); 5:... } MyThread(L) { 6: foo(l); } main [3] [4] foo([1]) foo([2])

57 Abstraction: Naming Runtime Objects Threads and locks are runtime objects Address changes across runs Need to name consistently and statically Abstraction based on object creation site Line number (PC) of where object allocated Use and combine richer abstractions for increased separation Objects created in loops Objects created in different contexts

58 Another Example Usage T1 T2 T3 T4 wait(o) notify(o) before (notify(abs(o)),abs(t3)-> wait(abs(o)), abs(t1))

59 Conclusion Practical and efficient testing of concurrent programs Find and reproduce bugs to lessen burden of manual inspection Input language for user specified schedules