Parallel algorithm templates. Threads, tasks and parallel patterns Programming with. From parallel algorithms templates to tasks

Transcription

1 COMP528 Task-based programming in OpenMP Alexei Lisitsa Dept of Computer Science University of Liverpool Parallel algorithm templates We have seen several useful templates for parallel algorithms provided by MPI and OpenMP: parallel for parallel reduce scatter/gather Is that all what we need in parallel programming? From parallel algorithms templates to tasks Is that all we need in parallel programming? No, many tasks cannot (easily, or at all) be fitted in these patterns OpenMP provides with alternative task-based programming it is not ``native concept for OpenMP Support for tasks appeared within OpenMP quite late (v. 3.0, 2008) It is central in some other approaches (e.g. TBB) Flexibility, Coding overhead Threads, tasks and parallel patterns Programming with Parallel patterns Tasks Threads automation, Less coding overhead 1

2 Tasks Tasks are logical units of computation with clearly defined inputs and outputs; Tasks can typically be specified at high level in the natural language Tasks typically involve the creation of other tasks (subtasks) Recursive patterns of the task creation are very useful here Example: Tasks and Recursion Fibonacci numbers F(0) = 1, F(1) =1, F(n) = F(n-1) + F(n-2) Task: given n compute F(n) Specification of the solution: If n = 0 or n=1 return 1 Else create tasks for computing F(n-1) and F(n-2) wait for answers, once received return F(n-1) + F(n-2) Syntax #pragma omp task [clause, clause,...] structured-block How it works When a thread encounters a task construct, a task is generated from the code for the structured block. The data environment of the task is created according to the data-sharing attribute clauses and any defaults that apply. The encountering thread may immediately execute the task, or defer its execution. In the latter case, any thread in the team may be assigned the task. 2

3 Further details Completion of the task can be guaranteed using task synchronization constructs. A task construct may be nested inside an outer task The task region of the inner task is not a part of the task region of the outer task. Fibonacci numbers by serial programming Fibonacci by serial procedure: long SerialFib( long n ) { if ( n <2 ) return n; else return SerialFib(n-1)+SerialFib(n-2;) // simple recursion int fib( int n ) // Fibonacci by tasks in OpenMP { int i, j; if( n < 10 ) { return fibserial(n); // for small n compute serially else { //(cont. next slide) //for larger n run in parallel // spawn a task #pragma omp task shared( i ), untied i = fib( n - 1 ); // spawn another task #pragma omp task shared( j ), untied j = fib( n - 2 ); // wait for both tasks to complete return i + j; 3

4 Clauses in task directives untied Clauses in task directives mergeable By default, a task is tied and its suspended task region can only be resumed by the thread that started its execution. If the untied clause is present on a task construct, any thread in the team can resume the task region after a suspension. The implementation might generate a merged task Allows the implementation to re-use the environment from another task void foo( ) { Example of mergeable int x = 2 ; #pragma omp task mergeable { x++; // x i s by d e f a u l t f i r s t p r i v a t e printf ("%d\n ", x ) ; // p r i n t s 2 o r 3 depending on // merging the tasks // Non-deterministic behaviour!! Clauses in task directives private(list) specifies that each thread should have its own instance of a variable firstprivate(list) specifies that each thread should have its own instance of a variable, and that the variable should be initialized with the value of the variable, because it exists before the parallel construct. shared(list) specifies that one or more variables should be shared among all threads. 4

5 Tasks synchronization Taskyield and suspension wait until all tasks defined in the current task have completed. #pragma omp taskyield specifies that the current task can be suspended for another task Using taskyield pragma omp task { something _ useful ( ) ; while (! omp_test_lock ( lock ) ) { # pragma omp taskyield //if lock is not free the current //task can be suspended something _ critical ( ) ; //if lock acquired do //critical section omp_unset _lock ( lock ) ; // free the lock Summary OpenMP (3.0 and higher) supports taskbased programming; We have looked into the main principles and constructions used in task-based programming with OpenMP 5