Active Testing for Concurrent Programs

Transcription

1 Active Testing for Concurrent Programs Pallavi Joshi, Mayur Naik, Chang-Seo Park, Koushik Sen 12/30/2008 ROPAS Seminar ParLab, UC Berkeley Intel Research

2 Overview ParLab The Parallel Computing Laboratory Checking correctness of concurrent programs Active testing directed by Data races Atomicity violations Deadlocks Programmer directed active testing Breakpoints for parallel programs Conclusion

3 The Parallel Computing Laboratory Goal: Productively create efficient and correct software for multicore/manycore Application driven Revisit entire HW / SW stack Collaborative Research Center 9 Professors, 40 Grad students Funded by Intel and Microsoft

4 Correctness ParLab Research Overview Personal Health Image Retrieval Hearing, Music Dwarfs Speech Parallel Browser Composition & Coordination Language (C&CL) C&CL Compiler/Interpreter Static Verification Parallel Libraries Parallel Frameworks Type Systems Efficiency Sketching Languages Autotuners Legacy Communication & Schedulers Code Synch. Primitives Efficiency Language Compilers OS Libraries & Services Legacy OS Hypervisor Intel Multicore/GPGPU RAMP Manycore Directed Testing Dynamic Checking Debugging with Replay

5 Correctness ParLab Research Overview Personal Health Image Retrieval Hearing, Music Dwarfs Speech Parallel Browser Composition & Coordination Language (C&CL) C&CL Compiler/Interpreter Static Verification Parallel Libraries Parallel Frameworks Type Systems Efficiency Sketching Languages Autotuners Legacy Communication & Schedulers Code Synch. Primitives Efficiency Language Compilers OS Libraries & Services Legacy OS Hypervisor Intel Multicore/GPGPU RAMP Manycore Directed Testing Dynamic Checking Debugging with Replay

6 Checking Correctness of Concurrent Programs Multiple threads of execution with shared memory Want correctness for All executions All inputs need to be checked Interference among threads introduces non deterministic bugs All schedules need to be checked

7 Checking All Schedules Model checking Check all schedules systematically Does not scale to large programs Focus on potentially buggy schedules Schedules that may expose bugs Boundary case schedules What are those potentially buggy schedules?

8 Checking All Schedules Model checking Check all schedules systematically Does not scale to large programs Focus on potentially buggy schedules Schedules that may expose bugs Boundary case schedules What are those potentially buggy schedules? Schedules exhibiting data races, atomicity violations, deadlocks

9 Active Testing Check the potentially buggy schedules Practical and efficient Find many bugs quickly Create an actual execution showing a bug How?

10 Active Testing Check the potentially buggy schedules Practical and efficient Find many bugs quickly Create an actual execution showing a bug Actively control scheduler Explore schedules systematically or randomly

11 Active Testing Check the potentially buggy schedules Practical and efficient Find many bugs quickly Create an actual execution showing a bug Actively control scheduler Explore schedules systematically or randomly But, preempt threads at appropriate states so that buggy schedules get created Observe execution for any abnormal behavior (e.g. uncaught exceptions, assertion failures)

12 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

13 Race Directed X = 1; In race if(x == 0) ERROR; [Sen; PLDI 08]

14 Race Directed X = 1; State to preempt State to preempt if(x == 0) ERROR; [Sen; PLDI 08]

15 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

16 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

17 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

18 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

19 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

20 Race Directed X = 1; if(x == 0) ERROR; [Sen; PLDI 08]

21 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

22 lock(l1) Deadlock Directed lock(l2) lock(l2) lock(l1) unlock(l2) unlock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

23 lock(l1) Deadlock Directed lock(l2) lock(l2) lock(l1) unlock(l2) unlock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

24 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

25 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

26 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

27 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

28 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

29 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

30 lock(l1) lock(l2) unlock(l2) Deadlock Directed unlock(l1) lock(l2) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

31 lock(l1) Deadlock Directed lock(l2) lock(l2) unlock(l2) unlock(l1) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

32 lock(l1) Deadlock Directed lock(l2) lock(l2) unlock(l2) unlock(l1) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

33 lock(l1) Deadlock Directed lock(l2) lock(l2) unlock(l2) unlock(l1) lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

34 lock(l1) Deadlock Directed lock(l2) lock(l2) unlock(l2) unlock(l1) Deadlock! lock(l1) unlock(l1) unlock(l2) [Joshi, Park, Sen, Naik; Under submission]

35 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

36 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

37 Atomicity Violation Directed t = x; atomic Atomicity Violation! t = 0; x = t + 1; [Park, Sen; FSE 08]

38 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

39 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

40 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

41 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

42 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

43 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

44 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

45 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

46 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

47 atomic Atomicity Violation Directed t = x; x = t + 1; t = 0; [Park, Sen; FSE 08]

48 Bugs Found Too implemented for Java Races, deadlocks, atomicity violations in Java Collections Framework Deadlocks found and reproduced in Jigsaw web server Java Swing GUI framework Java Database Connectivity (JDBC) Atomicity violations in Apache Commons Collections

49 Lessons Learned Controlling scheduler makes testing effective Focus on potentially buggy schedules Reproduce bugs with high probability Need the right preemption points Directed preemption uncovered many bugs Why not let the programmer choose the preemption points? Breakpoints for concurrent programs Language for describing schedules

50 Schedule Exploration Language Facility to specify thread schedules User specifies breakpoints Preempt a thread at a user specified state User specifies ordering constraints on events Control schedule to meet constraints Testing for logical (semantic) bugs Schedules are given as input to program Either generated automatically (directed) or programmer specified Inputs to unit tests but on schedules

51 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1)

52 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1) break 5

53 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1) break lock(l2)

54 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1) break lock(l2),5

55 Example Usage foo(l) { 1: lock(l) 2: f() 3: unlock(l) 4:... 5: lock(l) 6: g() 7: unlock(l) } foo(l1) foo(l2) foo(l3) T1 T2 T3 T4 foo(l2) foo(l1) foo(l1) break lock(l2), 5, T1

56 Abstraction: Naming Runtime Objects Threads and locks are runtime objects Address changes across runs Need to identify consistently with only static information An abstraction based on object creation site Line number (PC) of where object allocated

57 Abstraction Example main { 1: Lock L1 = new Lock(); 2: Lock L2 = new Lock(); 3: Thread T1 = new MyThread(L1); 4: Thread T2 = new MyThread(L2); 5:... } MyThread(L) { 6: foo(l); } main T1 T2 foo(l1) foo(l2)

58 Abstraction Example main { 1: Lock L1 = new Lock(); 2: Lock L2 = new Lock(); 3: Thread T1 = new MyThread(L1); 4: Thread T2 = new MyThread(L2); 5:... } MyThread(L) { 6: foo(l); } main [3] [4] foo([1]) foo([2])

59 Abstraction: Naming Runtime Objects Threads and locks are runtime objects Address changes across runs Need to identify consistently with only static information An abstraction based on object creation site Line number (PC) of where object allocated Use and combine richer abstractions for increased separation Objects created in loops Objects created in different contexts

60 Another Example Usage T1 T2 T3 T4 wait(o) notify(o) before (notify(abs(o)),abs(t3)-> notify(abs(o)), abs(t1))

61 Conclusion Practical and efficient testing of concurrent programs Find and reproduce bugs to lessen burden of manual inspection Input language for user specified schedules