Shared Memory Parallel Programming with Pthreads An overview

Transcription

1 Shared Memory Parallel Programming with Pthreads An overview Part II Ing. Andrea Marongiu Includes slides from ECE459: Programming for Performance course at University of Waterloo by Patrick Lam Hardware Software Design of and from Multicore Processors: Architecture & Programming course at New York University, 1 by Mohamed Zahran 5/14/2014 Embedded Systems M

2 5/14/2014 Hardware Software Design of Embedded Systems M 2 Recall: Parallel Programming Models Programming model is made up of the languages, compilers and libraries that create an abstract view of the machine Control How is parallelism created? How are dependencies (orderings) enforced? Data Is data shared or private? How is shared data accessed or private data communicated? Synchronization What operations can be used to coordinate parallelism? What are the atomic (indivisible) operations? Control Synchro Data

3 5/14/2014 Hardware Software Design of Embedded Systems M 3 Recall: Fork/Join Parallelism Sequential program Parallel program Initially only master thread is active Master thread executes sequential code Fork: Master thread creates or awakens additional threads to execute parallel code Join: At the end of parallel code created threads are suspended upon barrier synchronization

4 5/14/2014 Hardware Software Design of Embedded Systems M 4 Pthreads: POSIX Threads Pthreads standard for multithreaded programming IEEE Portable Operating System Interface, POSIX, section standard, 1995 Low-level threading libraries Native threading interface for Linux Use kernel-level threads (1:1 model) Pthread Library APIs (more than 100!) Thread Management: create, exit, join, Mutexes: init, destroy, lock, unlock, Condition: init, destroy, wait, Synchronization: barrier, semaphores..

5 5/14/2014 Hardware Software Design of Embedded Systems M 5 Posix Threads

6 5/14/2014 Hardware Software Design of Embedded Systems M 6 Creating threads Control

7 5/14/2014 Hardware Software Design of Embedded Systems M 7 Creating threads - Example Control

8 5/14/2014 Hardware Software Design of Embedded Systems M 8 Waiting for threads Control

9 5/14/2014 Hardware Software Design of Embedded Systems M 9 Waiting for threads - Example Control

10 5/14/2014 Hardware Software Design of Embedded Systems M 10 Detached threads Control

11 5/14/2014 Hardware Software Design of Embedded Systems M 11 Detached threads - Warning Control

12 5/14/2014 Hardware Software Design of Embedded Systems M 12 Thread termination Control

13 5/14/2014 Hardware Software Design of Embedded Systems M 13 Detached threads - Warning Control

14 5/14/2014 Hardware Software Design of Embedded Systems M 14 Attributes Control

15 5/14/2014 Hardware Software Design of Embedded Systems M 15 Attributes - Example Control

16 5/14/2014 Hardware Software Design of Embedded Systems M 16 Passing data to threads.. wrongly Data

17 5/14/2014 Hardware Software Design of Embedded Systems M 17 Passing data to threads.. wrongly Data

18 5/14/2014 Hardware Software Design of Embedded Systems M 18 Passing data to threads Data

19 5/14/2014 Hardware Software Design of Embedded Systems M 19 Passing data to threads Data

20 5/14/2014 Hardware Software Design of Embedded Systems M 20 Threading challenges

21 5/14/2014 Hardware Software Design of Embedded Systems M 21 Race conditions: π-finding code example double area, pi, x; int i, n; Synchro /* PARALLEL LOOP */ int LB = f1(my_id, num_procs); int UB = f2(my_id, num_procs); { for (i=lb; i<ub; i++) { x = (i + 0.5)/n; area += 4.0/(1.0 + x*x); } } pi = area/n;

22 5/14/2014 Hardware Software Design of Embedded Systems M 22 Race condition Synchro Ensure atomic updates of the shared variable area to avoid a race condition in which one process may race ahead of another and ignore changes

23 5/14/2014 Hardware Software Design of Embedded Systems M 23 Race condition (Cont d) time Synchro Thread A reads into a local register Thread B reads into a local register Thread A updates area with Thread B ignores write from thread A and updates area with

24 5/14/2014 Hardware Software Design of Embedded Systems M 24 Race conditions and mutual exclusion Synchro

25 5/14/2014 Hardware Software Design of Embedded Systems M 25 Creating mutexes - Example Synchro

26 5/14/2014 Hardware Software Design of Embedded Systems M 26 Using mutexes - Example Synchro

27 5/14/2014 Hardware Software Design of Embedded Systems M 27 Race conditions: π-finding code example double area, pi, x; int i, n; Synchro static pthread_mutex_t mtx = PTHREAD_MUTEX_INITIALIZER; /* PARALLEL LOOP */ int LB = f1(my_id, num_procs); int UB = f2(my_id, num_procs); { for (i=lb; i<ub; i++) { x = (i + 0.5)/n; pthread_mutex_lock (&mtx); area += 4.0/(1.0 + x*x); pthread_mutex_unlock (&mtx); } } pi = area/n; pthread_mutex_destroy (&mtx);

28 5/14/2014 Hardware Software Design of Embedded Systems M 28 Barriers double area, pi, x; int i, n; /* PARALLEL LOOP */ for (i=lb; i<ub; i++) A[i] =...; BARRIER /* PARALLEL LOOP */ for (i=lb; i<ub; i++) B[i] = A[i+n]; BARRIER /* PARALLEL LOOP */ for (i=lb; i<ub; i++) C[i] = B[i+n]+a[i+m]; Example: more than one loop inside a parallel function Second loop can only execute after first loop has finished Third loop can only execute after second loop has finised Why? How? Synchro BARRIER: A synchronization operation that involves all threads in a parallel construct. Forces parallel computation to stop until all threads have arrived at the barrier.

29 5/14/2014 Hardware Software Design of Embedded Systems M 29 Synchronization: barriers Declaration: pthread_barrier_t barrier; Functions: pthread_barrier_init pthread_barrier_wait Synchro

30 5/14/2014 Hardware Software Design of Embedded Systems M 30 Synchronization: barriers init int pthread_barrier_init( pthread_barrier_t *barrier, const pthread_barrattr_t *attr, int NUM_THREAD); Synchro RETURN 0 == SUCCESS barrier: (output) pointer to allocated barrier NUM_THREADS: (input) number of threads Another way to init using MACROS: pthread_barrier_t barrier = PTHREAD_BARRIER_INITIALIZER(3);

31 5/14/2014 Hardware Software Design of Embedded Systems M 31 Synchronization: barrier synch int pthread_barrier_wait(pthread_barrier_t *barrier); All thread wait for all thread at this barrier warning: take care about program control flow, avoiding deadlocks! Synchro