FastTrack: Efficient and Precise Dynamic Race Detection (+ identifying destructive races)

Transcription

1 FastTrack: Efficient and Precise Dynamic Race Detection (+ identifying destructive races) Cormac Flanagan UC Santa Cruz Stephen Freund Williams College

2 Multithreading and Multicore! Multithreaded programming is notoriously difficult, in part due to schedule-dependent behavior! race conditions, deadlocks, atomicity violations,...! difficult to detect, reproduce, or eliminate

3 Race Conditions! Two threads access a shared variable without synchronization, and at least one thread does a write! Very common 2003 Blackout ($6 Billion) Therac-25

4 Dynamic Race Detection Precision Happens Before [Lamport 78] Compute partial order of operations Ensure conflicting access are not concurrent Sound & Complete Eraser [SBN+ 97] Cost

5 Dynamic Race Detection Precision Track locks held on all accesses to var. - empty lock set implies possible race Unsound & Incomplete Happens Before [Lamport 78] Eraser [SBN+ 97] Cost

6 Dynamic Race Detection Precision Vector Clocks [M 88] Goldilocks [EQT 07] DJIT+ [ISZ 99,PS 03] TRaDe [CB 01]... Happens Before [Lamport 78] Barriers [PS 03] Initialization [vpg 01]... Eraser [SBN+ 97] Cost

7 Dynamic Race Detection Precision Vector Clocks [M 88] Goldilocks [EQT 07] DJIT+ [ISZ 99,PS 03] TRaDe [CB 01] RaceTrack [YRC... 05] MultiRace [PS 03] Hybrid Race Detector [OC 03]... Barriers [PS 03] Initialization [vpg 01]... Happens Before [Lamport 78] Eraser [SBN+ 97] Cost

8 Dynamic Race Detection Precision FastTrack Eraser [SBN+ 97] Vector Clocks [M 88] Goldilocks [EQT 07] DJIT+ [ISZ 99,PS 03] TRaDe [CB 01] Happens Before [Lamport 78] RaceTrack [YRC... 05] MultiRace Design [PS Criteria: 03] Hybrid Race Detector [OC 03] - sound... (find at least 1st race on each var) Barriers [PS 03] Initialization [vpg 01] - complete (no false alarms)... - efficient Insight: Accesses to a var are almost always totally ordered in the Happens-Before relation Cost

9 Happens-Before! Event Ordering: program order synchronization order! Types of Races: Write-Write Write-Read " (write before read) Read-Write " (read before write) Thread A x = 0 rel(m)... y = x Thread B acq(m) x = 1

10 VC A VC B L m W x R x A B A B A B A B A B

11 VC A VC B L m W x R x x = Write-Write 0 8 Check: 0 0 W x 4 VC0 A? ? Yes Read-Write Check: R x VC A?? Yes O(n) time

12 VC A VC B L m W x R x x =

13 VC A VC B L m W x R x x = rel(m)

14 VC A VC B L m W x R x x = rel(m) acq(m)

15 VC A VC B L m W x R x x = rel(m) acq(m) x =

16 VC A VC B L m W x R x 4 1 x = rel(m) Write-Read Check: W x VC A? ? No O(n) 4 0 time x = y = x

17 Write-Write and Write-Read Races Thread A Thread B Thread C Thread D x = 3 x = 1 x = 0??? read x O(n)

18 No Races Yet: Writes Totally Ordered! Thread A Thread B Thread C Thread D x = 3 x = 1 x = 0??? read x O(n)

19 No Races Yet: Writes Totally Ordered! Thread A Thread B Thread C Thread D x = 3 x = 1 x = 0? read x O(1)

20 VC A VC B L m W x Last Write "Epoch" x = @B @A Write-Write Check: W x VC A? 1@B 4 1? Yes (1! 1?) O(1) time

21 VC A VC B L m W x 4 1 x = rel(m) acq(m) 4 8 x = @A 2 1 4@A 4 1 4@A 4 1 4@A 4 1 8@B

22 VC A VC B L m W x 4 1 x = rel(m) y = x @A Write-Read Check: W x VC A? 0 8@B (8! 1?) 0 0 4@A 5 1? No 4 0 4@A 4 acq(m) @A x = @B O(1) time

23 Read-Write Races -- Ordered Reads Thread A Thread B Thread C Thread D read x read x read x? x = 2 Most common case: thread-local, lock-protected,...

24 Read-Write Races -- Unordered Reads Thread A Thread B Thread C x = 0 fork read x? read x?? read x x = 2

25 VC A VC B W x R x 7 0 x = fork read x 8 0 x = @A @A - read x 7 1 7@A 1@B 7@A @A 8 1 Read-Write Check: R x VC A? O(1) O(n) O(n) ? No

26 Thread A Thread B Thread C Thread D read x? read x? x = 2 O(n)

27 Thread A Thread B Thread C Thread D read x read x x = 2

28 Thread A Thread B Thread C Thread D read x read x?? x = 2? x = 3 O(n)

29 Thread A Thread B Thread C Thread D read x read x Forget VC for R x and switch back to "last read epoch" x = 2? x = 3 O(1)

30 RoadRunner Architecture Standard JVM Instrumented Bytecode RoadRunner Instrumenter Event Stream A: acq(m) A: read(x) B: write(y) A: rel(m) Back-End Checker Java Bytecode Error: race on x...

31 Validation! Six race condition checkers all use RoadRunner share common components (eg, VectorClock) profiled and optimized! Further optimization opportunities unsound extensions, dynamic escape analysis, static analysis, implement inside JVM, hardware support,...! 15 Benchmarks 250 KLOC locks, wait/notify, fork/join, barriers,...

32 Warnings false positives 3 false negatives Eraser [SBN+ 97] MultiRace [PS 03] GoldiLocks [EQT 07] Basic VC [M 88] DJIT+ [PS 03] FastTrack

33 Slowdown (x Base Time) Empty Eraser MultiRace Goldilocks Basic VC DJIT+ FastTrack

34 O(n) Vector Clock Operations 1.0E E E+9 Basic VC DJIT+ FastTrack 1.0E+8 1.0E+7 1.0E+6 1.0E+5 1.0E+4 1.0E+3 1.0E+2 1.0E+1 1.0E+0 colt crypt lufact moldyn montecarlo mtrt raja raytracer sparse series sor tsp elevator philo hedc jbb

35 O(n) Vector Clock Operations 1.0E E E+9 1.0E+8 1.0E+7 1.0E+6 1.0E+5 1.0E+4 1.0E+3 1.0E+2 1.0E % of all ops are Reads/Writes R/W ops requiring O(n) time: Basic VC 100% DJIT+ 26.0% FastTrack <0.1% Basic VC DJIT+ FastTrack 1.0E+0 colt crypt lufact moldyn montecarlo mtrt raja raytracer sparse series sor tsp elevator philo hedc jbb

36 Memory Usage! FastTrack allocated ~200x fewer VCs Basic VC, DJIT+ Checker Memory Overhead 7.9x FastTrack 2.8x (Note: VCs for dead objects can be garbage collected)! Improvements accordion clocks [CB 01] analysis granularity [PS 03, YRC 05] (see paper)

37 Eclipse 3.4! Scale > 6,000 classes 24 threads custom sync. idioms! Precision (tested 5 common tasks) Eraser: ~1000 warnings FastTrack: ~30 warnings! Performance on compute-bound tasks > 2x speed of other precise checkers same as Eraser

38 Beyond Detecting Race Conditions! FastTrack finds real race conditions races correlated with defects cause unintuitive behavior on relaxed memory! Which race conditions are real bugs? that cause erroneous behaviors (crashes, etc) and are not benign race conditions

39

40 Thread 0 Thread 1 Thread 2 p = null! px = 0! py = 0! fork 1,2! read p // non-null! read px //?! read p // null! acquire! read p // null! p = new Point! px = 1! py = 1! release! read px // get 1! read py // get 1!

41 Thread 0 Thread 1 Thread 2 p = null! px = 0! py = 0! fork 1,2! read p // non-null! read px //?! read p // null! acquire! read p // null! p = new Point! px = 1! py = 1! release! read px // get 1! read py // get 1!

42 Thread 0 Thread 1 Thread 2 p = null! px = 0! py = 0! fork 1,2! read p // non-null! read px //?! read p // null! acquire! read p // null! p = new Point! px = 1! py = 1! release! read px // get 1! read py // get 1!! Race: can return either write (mm non-determinism)! Typical JVM: mostly sequentially consistent! Adversarial memory use heuristics to return older stale values

43 Adversarial Memory! Record history of all writes (plus VCs) to racy variables! At read determine all visible writes legal under JMM heuristically pick one likely to crash target program! Six heuristics: Sequentially consistent: return last write Oldest: return most stale value Oldest-but-different: never return same val twice " if (p!= null) p.draw() Random, Random-but-different

44 Experimental Results