Carnegie Mellon Performance of Mul.threaded Code

Transcription

1 Performance of Mul.threaded Code Karen L. Karavanic Portland State University CS 410/510 Introduc=on to Performance Spring

2 Using Selected Slides from: Concurrent Programming : Introduc=on to Computer Systems 22 nd Lecture, Nov. 11, 2010 Instructors: Randy Bryant and Dave O Hallaron Modified by: Karen L. Karavanic

3 Tradi.onal View of a Process Process = process context + code, data, and stack Process context Code, data, and stack Program context: Data registers Condition codes Stack pointer (SP) Program counter (PC) Kernel context: VM structures Descriptor table brk pointer SP brk PC 0 stack shared libraries run-time heap read/write data read-only code/data 3

4 Alternate View of a Process Process = thread + code, data, and kernel context Thread (main thread) Code and Data SP stack Thread context: Data registers Condition codes Stack pointer (SP) Program counter (PC) brk PC 0 shared libraries run-time heap read/write data read-only code/data Kernel context: VM structures Descriptor table brk pointer 4

5 A Process With Mul.ple Threads Mul.ple threads can be associated with a process Each thread has its own logical control flow Each thread shares the same code, data, and kernel context Share common virtual address space (inc. stacks) Each thread has its own thread id (TID) Thread 1 (main thread) Shared code and data Thread 2 (peer thread) stack 1 Thread 1 context: Data registers Condition codes SP1 PC1 0 shared libraries run-time heap read/write data read-only code/data Kernel context: VM structures Descriptor table brk pointer stack 2 Thread 2 context: Data registers Condition codes SP2 PC2 5

6 Logical View of Threads Threads associated with process form a pool of peers Unlike processes which form a tree hierarchy Threads associated with process foo Process hierarchy T1 T2 shared code, data and kernel context T4 P0 P1 sh sh sh T5 T3 foo bar 6

7 Thread Execu.on Single Core Processor Simulate concurrency by =me slicing Mul.- Core Processor Can have true concurrency Thread A Thread B Thread C Thread A Thread B Thread C Time Run 3 threads on 2 cores 7

8 Logical Concurrency Two threads are (logically) concurrent if their flows overlap in.me Otherwise, they are sequen.al Examples: Concurrent: A & B, A&C Sequen=al: B & C Thread A Thread B Thread C Time 8

9 Threads vs. Processes How threads and processes are similar Each has its own logical control flow Each can run concurrently with others (possibly on different cores) Each is context switched How threads and processes are different Threads share code and some data Processes (typically) do not Threads are somewhat less expensive than processes Process control (crea=ng and reaping) is twice as expensive as thread control Linux numbers: ~20K cycles to create and reap a process ~10K cycles (or less) to create and reap a thread 9

10 Posix Threads (Pthreads) Interface Pthreads: Standard interface for ~60 func.ons that manipulate threads from C programs Crea=ng and reaping threads pthread_create() pthread_join() Determining your thread ID pthread_self() Termina=ng threads pthread_cancel() pthread_exit() exit() [terminates all threads], RET [terminates current thread] Synchronizing access to shared variables pthread_mutex_init pthread_mutex_[un]lock pthread_cond_init pthread_cond_[timed]wait 10

11 The Pthreads "hello, world" Program /* * hello.c - Pthreads "hello, world" program */ #include "csapp.h" void *thread(void *vargp); Thread attributes (usually NULL) int main() { pthread_t tid; } Pthread_create(&tid, NULL, thread, NULL); Pthread_join(tid, NULL); exit(0); /* thread routine */ void *thread(void *vargp) { printf("hello, world!\n"); return NULL; } Thread arguments (void *p) return value (void **p) 11

12 Execu.on of Threaded hello, world main thread call Pthread_create() Pthread_create() returns call Pthread_join() main thread waits for peer thread to terminate Pthread_join() returns peer thread printf() return NULL; (peer thread terminates) exit() terminates main thread and any peer threads 12

13 Threaded Execu.on Model Connection Requests Listening Server Client 1 data Client 1 Server Client 2 Server Client 2 data Mul=ple threads within single process Some state between them File descriptors 13

14 Approaches to Concurrency Processes Hard to share resources: Easy to avoid unintended sharing High overhead in adding/removing clients Threads Easy to share resources: Perhaps too easy Medium overhead Not much control over scheduling policies Difficult to debug Event orderings not repeatable 14

15 Mul.core Processor Core 0 Core n-1 Regs Regs L1 d-cache L1 i-cache L1 d-cache L1 i-cache L2 unified cache L2 unified cache L3 unified cache (shared by all cores) Main memory Intel Nehalem Processor E.g., Shark machines Mul=ple processors opera=ng with coherent view of memory 15

16 Memory Consistency int a = 1; int b = 100; Thread1: Wa: a = 2; Rb: print(b); Thread2: Wb: b = 200; Ra: print(a); Thread consistency constraints Wa Rb Wb Ra What are the possible values printed? Depends on memory consistency model Abstract model of how hardware handles concurrent accesses Sequen.al consistency Overall effect consistent with each individual thread Otherwise, arbitrary interleaving 16

17 Sequen.al Consistency Example int a = 1; int b = 100; Thread consistency constraints Wa Rb Thread1: Wa: a = 2; Rb: print(b); Thread2: Wb: b = 200; Ra: print(a); Wa Wb Rb Wb Ra Rb Ra Wb Ra Rb Ra Wa Rb Impossible outputs 100, 1 and 1, 100 Would require reaching both Ra and Rb before Wa and Wb Wa Wb Ra Rb Ra Rb Ra 100, 2 200, 2 2, 200 1, 200 2, , 2 17

18 Out- of- Order Processor Structure Instruction Control Instruction Decoder Instruction Cache Registers Op. Queue PC Functional Units Integer Arith Integer Arith FP Arith Load / Store Data Cache Instruc.on control dynamically converts program into stream of opera.ons Opera.ons mapped onto func.onal units to execute in parallel 18

19 Hyperthreading Instruction Control Reg A Op. Queue A Instruction Decoder Instruction Cache Reg B Op. Queue B PC A PC B Functional Units Integer Arith Integer Arith FP Arith Load / Store Data Cache Replicate enough instruc.on control to process K instruc.on streams K copies of all registers Share func.onal units 19

20 Summary: Crea.ng Parallel Machines Mul.core Separate instruc=on logic and func=onal units Some shared, some private caches Must implement cache coherency Hyperthreading Also called simultaneous mul=threading Separate program state Shared func=onal units & caches No special control needed for coherency Combining Shark machines: 8 cores, each with 2- way hyperthreading Theore=cal speedup of 16X Never achieved in our benchmarks 20

21 Summa.on Example Sum numbers 0,, N- 1 Should add up to (N- 1)*N/2 Par..on into K ranges N/K values each Accumulate lelover values serially Method #1: All threads update single global variable 1A: No synchroniza=on 1B: Synchronize with pthread semaphore 1C: Synchronize with pthread mutex Binary semaphore. Only values 0 & 1 21

22 Accumula.ng in Single Global Variable: Declara.ons typedef unsigned long data_t; /* Single accumulator */ volatile data_t global_sum; /* Mutex & semaphore for global sum */ sem_t semaphore; pthread_mutex_t mutex; /* Number of elements summed by each thread */ size_t nelems_per_thread; /* Keep track of thread IDs */ pthread_t tid[maxthreads]; /* Identify each thread */ int myid[maxthreads]; 22

23 Accumula.ng in Single Global Variable: Opera.on nelems_per_thread = nelems / nthreads; /* Set global value */ global_sum = 0; /* Create threads and wait for them to finish */ for (i = 0; i < nthreads; i++) { myid[i] = i; Pthread_create(&tid[i], NULL, thread_fun, &myid[i]); } for (i = 0; i < nthreads; i++) Pthread_join(tid[i], NULL); result = global_sum; /* Add leftover elements */ for (e = nthreads * nelems_per_thread; e < nelems; e++) result += e; 23

24 Thread Func.on: No Synchroniza.on void *sum_race(void *vargp) { int myid = *((int *)vargp); size_t start = myid * nelems_per_thread; size_t end = start + nelems_per_thread; size_t i; } for (i = start; i < end; i++) { global_sum += i; } return NULL; 24

25 Unsynchronized Performance N = 2 30 Best speedup = 2.86X Gets wrong answer when > 1 thread! 25

26 Thread Func.on: Semaphore / Mutex Semaphore void *sum_sem(void *vargp) { int myid = *((int *)vargp); size_t start = myid * nelems_per_thread; size_t end = start + nelems_per_thread; size_t i; } for (i = start; i < end; i++) { sem_wait(&semaphore); global_sum += i; sem_post(&semaphore); } return NULL; Mutex pthread_mutex_lock(&mutex); global_sum += i; pthread_mutex_unlock(&mutex); 26

27 Semaphore / Mutex Performance Terrible Performance 2.5 seconds è ~10 minutes Mutex 3X faster than semaphore Clearly, neither is successful 27

28 Separate Accumula.on Method #2: Each thread accumulates into separate variable 2A: Accumulate in con=guous array elements 2B: Accumulate in spaced- apart array elements 2C: Accumulate in registers /* Partial sum computed by each thread */ data_t psum[maxthreads*maxspacing]; /* Spacing between accumulators */ size_t spacing = 1; 28

29 Separate Accumula.on: Opera.on nelems_per_thread = nelems / nthreads; /* Create threads and wait for them to finish */ for (i = 0; i < nthreads; i++) { myid[i] = i; psum[i*spacing] = 0; Pthread_create(&tid[i], NULL, thread_fun, &myid[i]); } for (i = 0; i < nthreads; i++) Pthread_join(tid[i], NULL); result = 0; /* Add up the partial sums computed by each thread */ for (i = 0; i < nthreads; i++) result += psum[i*spacing]; /* Add leftover elements */ for (e = nthreads * nelems_per_thread; e < nelems; e++) result += e; 29

30 Thread Func.on: Memory Accumula.on void *sum_global(void *vargp) { int myid = *((int *)vargp); size_t start = myid * nelems_per_thread; size_t end = start + nelems_per_thread; size_t i; } size_t index = myid*spacing; psum[index] = 0; for (i = start; i < end; i++) { psum[index] += i; } return NULL; 30

31 Memory Accumula.on Performance Clear threading advantage Adjacent speedup: 5 X Spaced- apart speedup: 13.3 X (Only observed speedup > 8) Why does spacing the accumulators apart mamer? 31

32 False Sharing psum Cache Block m Cache Block m+1 Coherency maintained on cache blocks To update psum[i], thread i must have exclusive access Threads sharing common cache block will keep figh=ng each other for access to block 32

33 False Sharing Performance Best spaced- apart performance 2.8 X beper than best adjacent Demonstrates cache block size = byte values No benefit increasing spacing beyond 8 33

34 Thread Func.on: Register Accumula.on void *sum_local(void *vargp) { int myid = *((int *)vargp); size_t start = myid * nelems_per_thread; size_t end = start + nelems_per_thread; size_t i; size_t index = myid*spacing; data_t sum = 0; for (i = start; i < end; i++) { sum += i; } psum[index] = sum; return NULL; } 34

35 Register Accumula.on Performance Clear threading advantage Speedup = 7.5 X 2X bemer than fastest memory accumula.on 35

36 Performance Implications of Multithreading 1. Scalability Questions What is the speedup over a single-threaded version? How much can we improve the performance? 2. Variance Multiple levels of scheduling Complex memory hierarchy Shared Resources 3. Metrics CPU Utilization? Synchronization Overhead Memory 4. Measurement Tools Tool Scalability overhead to measure individual threads? How to Visualize? Carnegie Mellon 36

37 Scalability Carnegie Mellon 1. Scalability Questions What is the speedup over a single-threaded version? How much can we improve the performance? Amdahl s Law 37

38 Performance Implications of Multithreading 1. Variance Indices of Central Tendency Arithmetic Mean Harmonic Mean Weighted Mean Quantifying Variability Sample Variance how are the values distributed, in relation to the mean? Standard Deviation positive square root of the variance Changes the units to match the mean, easier for comparison Coefficient of Variation Standard deviation divided by mean Now there are no units Carnegie Mellon 38

39 Performance Implications of Multithreading Metrics CPU Utilization Average across the cores Total CPU time sum across the cores What about balance? (max, min, variance) Synchronization Overhead Lock wait time Memory Number of cache misses Better cache miss rate Memory contention Memory access latency Carnegie Mellon 39

40 Performance Implications of Multithreading 1. Measurement Tools Tool Scalability overhead to measure individual threads? How to Visualize? Example: HPC Toolkit Carnegie Mellon 40