ThreadSanitizer APIs for External Libraries. Kuba Mracek, Apple

Size: px

Start display at page:

Download "ThreadSanitizer APIs for External Libraries. Kuba Mracek, Apple"

Esther Randall
5 years ago
Views:

1 ThreadSanitizer APIs for External Libraries Kuba Mracek, Apple

2 ThreadSanitizer

3 ThreadSanitizer Data race detector

4 ThreadSanitizer Data race detector LLVM IR instrumentation:

5 ThreadSanitizer Data race detector LLVM IR instrumentation: memory reads and writes

6 ThreadSanitizer Data race detector LLVM IR instrumentation: memory reads and writes atomic operations (load, store, RMW, CAS)

7 Thread 1 Thread 2 Thread 3

8 Thread 1 Thread 2 Thread 3 events events events

9 Thread 1 Thread 2 Thread 3 events events events happens-before data race detector

10 Thread 1 Thread 2 Thread 3 events events Memory events reads Memory writes Synchronization events happens-before data race detector

11 Thread 1 Thread 2 Thread 3 events events events happens-before data race detector

12 Thread 1 Thread 2 Thread 3 events events events happens-before data race detector

13 Thread 1 Thread 2 Thread 3 events events events happens-before data race detector $./racyapp WARNING: ThreadSanitizer: data race (pid=19219) Write of size 4 at 0x7fcf47b21bc0 by thread T1: #0 Thread1 race.c:4 (exe+0x a360) Previous write of size 4 at 0x7fcf47b21bc0 by main thread #0 main race.c:10 (exe+0x a3b4)

14 Libraries and Frameworks

15 Libraries and Frameworks Precompiled code is not instrumented

16 Libraries and Frameworks Precompiled code is not instrumented instrumented

17 Libraries and Frameworks Precompiled code is not instrumented call API instrumented

18 Libraries and Frameworks Precompiled code is not instrumented call API.so instrumented non-instrumented

19 Libraries and Frameworks Precompiled code is not instrumented call API.so instrumented non-instrumented

20 Libraries and Frameworks Precompiled code is not instrumented call API.so instrumented non-instrumented

21 Libraries and Frameworks Precompiled code is not instrumented call API.so instrumented instrumented

22 Libraries and Frameworks Precompiled code is not instrumented call API.so instrumented instrumented

23 Libraries and Frameworks Precompiled code is not instrumented APIs expect users to ensure thread safety call API.so instrumented instrumented

24 Libraries and Frameworks Precompiled code is not instrumented APIs expect users to ensure thread safety call API.so instrumented

25 Libraries and Frameworks Precompiled code is not instrumented APIs expect users to ensure thread safety call API.so instrumented

26 New: APIs for Libraries

27 New: APIs for Libraries ThreadSanitizer provides callbacks for libraries to inform about read/write-like events of high-level objects:

28 New: APIs for Libraries ThreadSanitizer provides callbacks for libraries to inform about read/write-like events of high-level objects: tsan_external_read(void *addr, void *caller_pc, void *tag); tsan_external_write(void *addr, void *caller_pc, void *tag);

29 New: APIs for Libraries ThreadSanitizer provides callbacks for libraries to inform about read/write-like events of high-level objects: tsan_external_read(void *addr, void *caller_pc, void *tag); tsan_external_write(void *addr, void *caller_pc, void *tag); High-level object = basically any object you work with array, map, graph node, data object, UI element,

30 Example: CoreFoundation

31 Example: CoreFoundation Provides APIs for basic collections: CFMutableArrayRef CFArrayCreateMutable(/* */); void CFArrayAppendValue(CFArrayRef array, /* */); CFIndex CFArrayGetCount(CFArrayRef array);

32 Example: CoreFoundation Provides APIs for basic collections: CFMutableArrayRef CFArrayCreateMutable(/* */); void CFArrayAppendValue(CFArrayRef array, /* */); CFIndex CFArrayGetCount(CFArrayRef array); User must ensure thread safety

33 // Modifies the array void CFArrayAppendValue(CFArrayRef array, /* */) { tsan_external_write(array, CALLER_PC, tag); /* */ } // Reads the array CFIndex CFArrayGetCount(CFArrayRef array) { tsan_external_read(array, CALLER_PC, tag); /* */ }

34 // Modifies the array void CFArrayAppendValue(CFArrayRef array, /* */) { if (is_tsan_present) tsan_external_write(array, CALLER_PC, tag); /* */ } // Reads the array CFIndex CFArrayGetCount(CFArrayRef array) { if (is_tsan_present) tsan_external_read(array, CALLER_PC, tag); /* */ }

35 ================== WARNING: ThreadSanitizer: race on a library object Read-only access of CFMutableArray at 0x7b0c00046b30 by thread T2: #0 CFArrayGetCount (CoreFoundation:x86_64) #1 Thread1 main.m:16 (demoapp:x86_64) Previous modifying access of CFMutableArray at 0x7b0c00046b30 by thread T3: #0 CFArrayAppendValue (CoreFoundation:x86_64) #1 Thread2 main.m:21 (demoapp:x86_64) Location is heap block of size 40 at 0x7b0c00046b30 allocated by main thread: SUMMARY: ThreadSanitizer: race on a library object main.m:16 in Thread1 ==================

36 report description ================== WARNING: ThreadSanitizer: race on a library object Read-only access of CFMutableArray at 0x7b0c00046b30 by thread T2: #0 CFArrayGetCount (CoreFoundation:x86_64) #1 Thread1 main.m:16 (demoapp:x86_64) Previous modifying access of CFMutableArray at 0x7b0c00046b30 by thread T3: #0 CFArrayAppendValue (CoreFoundation:x86_64) #1 Thread2 main.m:21 (demoapp:x86_64) Location is heap block of size 40 at 0x7b0c00046b30 allocated by main thread: SUMMARY: ThreadSanitizer: race on a library object main.m:16 in Thread1 ==================

37 report description ================== WARNING: ThreadSanitizer: race on a library object Read-only access of CFMutableArray at 0x7b0c00046b30 by thread T2: type of the object #0 CFArrayGetCount (CoreFoundation:x86_64) #1 Thread1 main.m:16 (demoapp:x86_64) Previous modifying access of CFMutableArray at 0x7b0c00046b30 by thread T3: #0 CFArrayAppendValue (CoreFoundation:x86_64) #1 Thread2 main.m:21 (demoapp:x86_64) Location is heap block of size 40 at 0x7b0c00046b30 allocated by main thread: SUMMARY: ThreadSanitizer: race on a library object main.m:16 in Thread1 ==================

38 report description ================== WARNING: ThreadSanitizer: race on a library object Read-only access of CFMutableArray at 0x7b0c00046b30 by thread T2: type of the object #0 CFArrayGetCount (CoreFoundation:x86_64) #1 Thread1 main.m:16 (demoapp:x86_64) Previous modifying access of CFMutableArray at 0x7b0c00046b30 by thread T3: #0 CFArrayAppendValue (CoreFoundation:x86_64) #1 Thread2 main.m:21 (demoapp:x86_64) API call Location is heap block of size 40 at 0x7b0c00046b30 allocated by main thread: SUMMARY: ThreadSanitizer: race on a library object main.m:16 in Thread1 ==================

39 More Details

40 More Details Tags to identify the type of the object

41 More Details Tags to identify the type of the object Provide caller PC

42 More Details Tags to identify the type of the object Provide caller PC Weak imports

43 More Details Tags to identify the type of the object Provide caller PC Weak imports Detect ThreadSanitizer at initialization time

44 More Details Tags to identify the type of the object Provide caller PC Weak imports Detect ThreadSanitizer at initialization time Contact me or mailing list

45 More Details Tags to identify the type of the object Provide caller PC Weak imports Detect ThreadSanitizer at initialization time Contact me or mailing list Already used by Foundation, CoreFoundation and Swift

46 If you re developing a popular library used in multithreaded programs, consider adopting these APIs!

A program execution is memory safe so long as memory access errors never occur:

A program execution is memory safe so long as memory access errors never occur: Buffer overflows, null pointer dereference, use after free, use of uninitialized memory, illegal free Memory safety categories