ROSE-CIRM Detecting C-Style Errors in UPC Code

Transcription

1 ROSE-CIRM Detecting C-Style Errors in UPC Code Peter Pirkelbauer 1 Chunhuah Liao 1 Thomas Panas 2 Daniel Quinlan Microsoft Parallel Data Warehouse This work was funded by the Department of Defense and used elements at the Extreme Scale Systems Center, located at Oak Ridge. This work performed under the auspices of the U.S. Department of Energy by under Contract DE-AC52-07NA27344, P. O. Box 808, Livermore, CA UCRL- LLNL-PRES

2 Motivation Cost of Software Bugs is significant in % of the GDP [NIST02] Error Detection Support RTED Benchmark for Compilers and Runtime- Systems [Lue09a] [Lue09b] [RTED] Bug Detection Tools Static and Dynamic Analysis Source Code and Binary Code 2

3 Outline Unified Parallel C and C-Style Errors Implementation Code Instrumentation and Dynamic Analysis Evaluation Conclusion 3

4 Unified Parallel C (UPC) Extends C99 with: Partitioned Global l Address Space Language constructs for Parallelism e.g., shared pointers, parallel for loop, memory consistency models 4

5 Error Categories C-Style Errors out of bounds accesses, uninitialized iti variables, dangling pointers C-Style Errors in UPC s shared memory space UPC Library Functions upc_memput with wrong length Parallelism Related Errors deadlock, livelock, race conditions 5

6 UPC Bug Example 1 UPC Code int upc_main() { shared [] int *ptr; if (MYTHREAD == 0) { ptr = upc_alloc( ); upc_barrier; if (MYTHREAD == 1) { upc_free(ptr); 6

7 UPC Bug Example 1 (cont d) UPC Code int upc_main() { shared [] int *ptr; if (MYTHREAD == 0) { ptr = upc_alloc( ); upc_barrier; Thread 0 allocates local shared memory. ptr in Thread 1 remains uninitialized. Bug uninitialized pointer access if (MYTHREAD == 1) { Thread 1 accesses upc_free(ptr); uninitialized ptr. 7

8 UPC Bug Example 2 UPC Code int upc_main() { shared [] int *ptr; ptr = upc_all_alloc( ); upc_barrier; ptr[mythread] = ; if (MYTHREAD == 0) { upc_free(ptr); 8

9 UPC Bug Example 2 (cont d) UPC Code int upc_main() { shared [] int *ptr; ptr = upc_all_alloc( ); upc_barrier; ptr[mythread] = ; if (MYTHREAD == 0) { upc_free(ptr); Collective memory allocation Missing barrier: Thread 0 might free the memory early. Bug potential early memory release 9

10 Dynamic Analysis Original Code Thread 0 int upc_main() { allocates shared [] int *ptr; local shared memory. if (MYTHREAD == 0) { ptr = upc_alloc( ); Leaves ptr in Thread 1 uninitialized. upc_barrier; if (MYTHREAD == 1) { upc_free(ptr); Thread 1 accesses uninitialized iti ptr. Instrumented Code int upc_main() { shared [] int *ptr; if (MYTHREAD == 0) { ptr = upc_alloc( ); cirm_createheapptr(ptr, ); cirm_initvariable(&ptr, ); cirm_exitworkzone(); upc_barrier; cirm_enterworkzone(); if (MYTHREAD == 1) { cirm_freemem(&ptr); upc_free(ptr); 10

11 Dynamic Analysis (Scheme) Original Code int upc_main() { shared [] int *ptr; if (MYTHREAD == 0) { ptr = upc_alloc( ); upc_barrier; if (MYTHREAD == 1) { upc_free(ptr); Updates shadow memory and notifies other UPC threads about the heap allocation. Marks the location of the ptr as initialized. Note: ptr in Thread 0!= ptr in Thread 1. Thread 1 accesses uninitialized ptr. Instrumented Code int upc_main() { shared [] int *ptr; if (MYTHREAD == 0) { ptr = upc_alloc( ); cirm_createheapptr(ptr, ); cirm_initvariable(&ptr, ); cirm_exitworkzone(); upc_barrier; cirm_enterworkzone(); if (MYTHREAD == 1) { cirm_freemem(&ptr); upc_free(ptr); 11

12 The ROSE Compiler Infrastructure 12

13 ROSE-CIRM Toolchain ROSE - Code Instrumentation and Runtime Monitor 13

14 Runtime Architecture (1) 14

15 Runtime Architecture (2) Instrumented Code shared[] int *values = upc_all_alloc( ); cirm_createheap(values, ); cirm_initvariable(&values); if (MYTHREAD == 1) { values[1] = 7; cirminitvar(&values[1], ); 15

16 Runtime Monitor Coordination (1) Concurrent Access // shared int val; Instrumented Code if (MYTHREAD==0) { val = comp( ); cirm_initvariable(&val, ); cirm_enterbarrier(); upc_barrier; cirm_exitbarrier(); cirm_accessvar(&val, ); printf( %d\n, val); Sends update on initialization to other runtime managers. Messages are processed after barrier. Test succeeds 16

17 Runtime Monitor Coordination (2) Concurrent Access // shared int val; Instrumented Code if (MYTHREAD==0) { val = comp( ); cirm_initvariable(&val, ); // upc_barrier; cirm_accessvar(&val, ); printf( %d\n, val); If the input program contains race conditions, ROSE-CIRM may spuriously report an error. Sends update on initialization to other runtime managers. Missing barrier. Test fails if messages are not processed in time. 17

18 Coordination Early Release Problem (1) Instrumented Code shared[] int *values = upc_all_alloc( ); cirm_arrayaccess(&values[0], &values[idx]); values[idx] = useful_computation(idx); cirm_initvariable(&values[ ], ); // upc_barrier; if (MYTHREAD == 0) { cirm_exitworkzone(); cirm_freemem(&ptr); upc_free(ptr); cirm_enterworkzone(); Heap-memory access Missing barrier Thread 0 might free the memory early. 18

19 Coordination Early Release Problem (2) Isolate Destructive Updates Instrumented Code shared[] int *values = upc_all_alloc( ); cirm_arrayaccess(&values[0], &values[idx]); values[idx] = useful_computation(idx); cirm_initvariable(&values[ ], ); // upc_barrier; if (MYTHREAD == 0) { cirm_exitworkzone(); cirm_freemem(&ptr); upc_free(ptr); cirm_enterworkzone(); 19

20 Coordination Early Release Problem (3) Isolate Destructive Updates Instrumented Code shared[] int *values = upc_all_alloc( ); cirm_arrayaccess(&values[0], &values[idx]); values[idx] = useful_computation(idx); cirm_initvariable(&values[ ], ); // upc_barrier; if (MYTHREAD == 0) { cirm_exitworkzone(); cirm_freemem(&ptr); upc_free(ptr); cirm_enterworkzone(); 20

21 Coordination Early Release Problem (4) Isolate Destructive Updates Instrumented Code shared[] int *values = upc_all_alloc( ); cirm_arrayaccess(&values[0], &values[idx]); values[idx] = useful_computation(idx); cirm_initvariable(&values[ ], ); // upc_barrier; if (MYTHREAD == 0) { cirm_exitworkzone(); cirm_freemem(&ptr); upc_free(ptr); cirm_enterworkzone(); 21

22 Coordination Early Release Problem (5) Isolate Destructive Updates Instrumented Code shared[] int *values = upc_all_alloc( ); cirm_arrayaccess(&values[0], &values[idx]); values[idx] = useful_computation(idx); cirm_initvariable(&values[ ], ); // upc_barrier; if (MYTHREAD == 0) { cirm_exitworkzone(); cirm_freemem(&ptr); upc_free(ptr); cirm_enterworkzone(); 22

23 Coordination Early Release Problem (6) Isolate Destructive Updates Instrumented Code shared[] int *values = upc_all_alloc( ); cirm_arrayaccess(&values[0], &values[idx]); values[idx] = useful_computation(idx); cirm_initvariable(&values[ ], ); // upc_barrier; if (MYTHREAD == 0) { cirm_exitworkzone(); cirm_freemem(&ptr); upc_free(ptr); cirm_enterworkzone(); 23

24 Address Abstraction Implementation for GCCUPC 24

25 Bounds Checking C/C++ char* ptr = charrarr[1]; Instrumented Code cirm_accessarray(ptr, ptr+2, sizeof(*ptr), cirmwrite,...); ptr[2] = 8; 25

26 Bounds Checking Distributed Array shared[3] char chararr[threads][8]; 26

27 Tests - RTED Benchmark Suite Luecke et al.: RTED Benchmark Suite for UPC [RTED] Category Number of Correctly Identified Tests (in percent) Out of bounds accesses (indices) (94%) Out of bounds accesses (pointers) (94%) Uninitialized memory reads (97%) Dynamic memory handling related (100%) 27

28 Tests - Heat-Conduction Code El-Ghazawi et al.: Distributed Shared Memory Programming [ElG05] 80 elements per dimension 8 Threads Intel X5680, 6x2 3.3Ghz 24GByte Memory, Red Hat Linux Client 5.6 gccupc , g

29 Related Tools UPC Compilers and Runtime Systems GCCUPC, Berkeley UPC, Cray UPC,... Tools for C/C++ Commercial Software Insure++, Purify Open Source Software Valgrind Memory Checkers DMalloc,... 29

30 Conclusion ROSE-CIRM a dynamic analysis tool for UPC code helps programmers find some bugs works in mixed language projects (C/C++, UPC) performs well on a subset of the RTED benchmark implemented for GCCUPC 30

31 Future Work Generality Casts of blocksize Complex array subscript expressions Scope UPC Library, Parallelism related errors Scalability Runtime Monitor Design Performance Elimination of unnecessary checks (ROSE analysis) 31

32 Thank You! This work was funded by the Department of Defense and used elements at the Extreme Scale Systems Center, located at Oak Ridge. This work performed under the auspices of the U.S. Department of Energy by under Contract DE-AC52-07NA

33 References [BUPC] [DMalloc] Berkeley UPC, [ElG05] El-Ghazawi et al: UPC: Distributed Shared-Memory Programming, [GCCUPC] GCCUPC, [Insure] [Lue09a] [Lue09b] [NIST02] [Purify] [Pin] [RTED] Insure++, Luecke et al: Evaluating error detection capabilities of UPC run-time systems. PGAS 09. Luecke et al: The importance of run-time error detection. 3 rd Parallel Tools Workshop 09. National Institute of Standards & Technology: The Economic Impacts of Inadequate Infrastructure for Software Testing, May Purify, Pin - A Dynamic Binary Instrumentation Tool RTED Benchmark Suite, [UPC] UPC Language Specification v1.2, June [Valgrind] Valgrind, g 33

34 Appendix 34

35 Runtime Error Detection (RTED): Introduction Shadow memory stores: Information on memory state Instruments source code: Updates shadow memory when memory is allocated, freed, or initialized. Checks memory operations for consistency. RTED is a tool that detects software flaws and helps pinpoint their origin. RTED consists of a runtime system and a source-to-source transformation system. The runtime system utilizes a shadow memory to keep track of memory state (allocations, initializations, ). The source-to-source transformation adds statements to the original source code that inform the RTED runtime system about memory operations. 35

36 RTED for Unified Parallel C (UPC) Shadow memory: 1x per UPC thread Stores state of UPC process Instrumented Code: Notifies other UPC threads of updates. In addition to local storage, such as Stack and Heap, UPC defines a shared memory region, which can be accessed from any UPC thread. In order to safeguard memory operations, each RTED runtime systems requires access to the memory state. To do so, each UPC thread keeps a local copy. Any update of the memory state is communicated to all other UPC threads. 36

37 RTED for UPC: Address Representation UPC Thread ID Relative position (GCCUPC) base: upc_vm_map _addr relative position for shared pointers: GUPCR_PTS_OFFSET To uniquely identify a memory position, RTED s runtime systems communicate addresses as a tuple containing the thread-id and the relative position to the shared memory base. The thread-id is determined by MYTHREAD for local pointers (they can also point into the shared memory region) and upc_threadof for shared pointers. Finding the relative position is implementation dependent; this slide uses the GCCUPC interface. 37

38 Runtime Monitor - Coordination Issues? Thread 1 Thread 2 // shared int[] values[threads]; // shared int[] values[threads]; W: values[idx] =...; B: cirm_initvariable(&values[ ], ); upc_barrier; upc_barrier; P: cirmaccessarray(&values[ ], ); R: sum += values[ ]; 38

39 RTED: Runtime Error Detection Due to performance and other concerns, programming g languages g / compilers / runtime systems do not (always) guarantee safe execution of code. Undetected software defects are the source for costly problems, such as unstable code, security vulnerabilities, etc. RTED instruments potentially unsafe operations with calls to a runtime checking system, thereby providing a safety envelop for executable code. Supported Languages: C, C++, UPC Comparison with other tools (Valgrind): + type information + Higher level abstractions - Requires whole program 39

40 Tests - Heat-Conduction Code El-Ghazawi et al.: Distributed Shared Memory Programming [ElG05] 80 elements per dimension 8 Threads Intel X5680, 6x2 3.3Ghz 24GByte Memory, Red Hat Linux Client 5.6 gccupc , g