OpenMP 4.5: Threading, vectorization & offloading

Transcription

1 OpenMP 4.5: Threading, vectorization & offloading Michal Merta 2nd of March 2018

2 Agenda Introduction The Basics OpenMP Tasks Vectorization with OpenMP 4.x Offloading to Accelerators

3 Why OpenMP? CPU frequency no longer increasing significantly Number of transistors still increases (Moore s law holds) Number of cores increasing SIMD vector capabilities increasing parallelization inevitable (Image:

4 Why OpenMP? OpenMP 4.5 now enables you to use the same standard for threading/tasking your application in shared memory, vectorization for the SIMD units, accelerating your code using the GPUs/many core coprocessors. Less need for a combination of CUDA, OpenCL, OpenGL, OpenACC, Intrinsic, Cilk, TBB,...

5 The Basics

6 Fork-join model Program starts with one thread master Program branches off for parallel execution worker threads are spawned at parallel regions Parallel code executed by master and workers Between the parallel regions workers are put to sleep. Synchronization barrier after each parallel region. (Image: Wikipedia)

7 Parallel region The parallel region is created using #pragma omp parallel directive. Executed by a newly created team of threads. Number of threads can be specified by environmental variable: export OMP NUM THREADS=..., or via num threads(...) clause. if clause can be used to control whether region will be executed in parallel # pragma omp parallel printf (" hello "); printf (" world "); # pragma omp parallel \ num_threads (2) // Executed by 2 threads... # pragma omp parallel if(a >1) // Executed in parallel // only if a >1...

8 For loops If only a parallel construct is used, all threads execute the same code. To speedup the program we have to employ worksharing Usually loops account for much of programs runtime. The most common worksharing construct: for Distributes the loop s iterations among threads in a team. # pragma omp parallel # pragma omp for for ( int i = 0; i < 100; ++i) a[i] = b[i] + c[i]; # pragma omp parallel for for ( int i = 0; i < 100; ++i) a[i] = b[i] + c[i];

9 For loop Canonical loop form OpenMP is only able to parallelize loops in the canonical form: for ( initialize ; test ; increment )... initialize: expression in the form var = lb where var is integer or random access iterator, lb is loop invariant, test: expression in the form var operator b where b is loop invariant, operator is one of the <, <=, >, >=, increment: expression in the form ++var, var++, --var, var--, var+=incr, var-=incr, var=var+incr, var=incr+var, var=var-incr. The for loop must not contain statements that allows the loop to be exited prematurely (break, return, exit, goto).

10 Loop scheduling We can influence the way the iterations are distributed among threads schedule(static [, chunk]) iterations are divided into blocks of chunk size and assigned to threads in a round-robin fashion. schedule(dynamic [, chunk]) iterations are divided into blocks of chunk size, when a thread finishes it will be assigned the next iteration that hasn t been executed yet. schedule(guided [, chunk]) similar to dynamic, but starts with implementation-defined block size and exponentially decreases to chunk. Default on most implementations is schedule(static).

11 Single construct The single construct ensures that only one thread will execute given structured block. Can be used for I/O, memory allocation/deallocation, creating tasks # pragma omp parallel... # pragma omp single // Only one thread executes the block // while others wait for work....

12 Synchronization A data race occurs when two threads access the same memory without proper synchronization. If between two synchronization points at least one thread writes to a memory location from which at least one other thread reads, the result is not deterministic (race condition). double sum = 0.0; # pragma omp parallel for for ( int i = 0; i < 1000; ++i) sum = sum + a[ i];

13 Synchronization critical section Ensures only one thread at a time will execute a given block. double sum = 0.0; # pragma omp parallel for for ( int i = 0; i < 1000; ++i) # pragma omp critical sum = sum + a[ i];

14 Synchronization atomic operation The memory update in the next instruction will be performed atomically, not the whole statement. A compiler may use special hw. instructions for better performance than with critical. double sum = 0.0; # pragma omp parallel for for ( int i = 0; i < 1000; ++i) // The word update is optional # pragma omp atomic update sum = sum + a[ i]; The atomic supports following operations ++x; --x; x ++; x - -; x += expr ; x -= expr ; x *= expr ; x /= expr ; x &= expr ; x = x+ expr ; x = x- expr ; x = x* expr ; x = x/ expr ; x = x& expr ; x = expr+x; x = expr -x; x = expr *x; x = expr /x; x = expr &x; x = expr ; x ˆ= expr ; x <<= expr ; x >>= expr ; x = x expr ; x = xˆ expr ; x = x<< expr ; x = x>> expr ; x = expr x; x = exprˆx; x = expr <<x; x = expr >>x;

15 Synchronization Reduction Reduction operator is applied to all variables in the list. Syntax: reduction(operator : list). The result is provided in the associated reduction variable. double sum = 0.0; # pragma omp parallel for reduction (+ : sum ) for ( int i = 0; i < 1000; ++i) sum = sum + a[ i];

16 Synchronization Reduction It is possible to create a user-defined reduction: #pragma omp declare reduction (reduction-identifier : typename-list : combiner) [initializer(expression)] reduction-identifier gives name to the operator typename-list list of types to which it applies combiner expression specifying how to combine values initializer can specify initializing value of the operator e.g. #pragma omp declare reduction (merge : std::vector<int> : omp out.insert(omp out.end(), omp in.begin(), omp in.end()))

17 Synchronization The barrier construct Implicit or explicit. All tasks created by any thread of the current team are guaranteed to be completed at barrier exit. All worksharing constructs have an implicit barrier. In some cases the barrier can be left out by the nowait clause. # pragma omp barrier

18 Data Scope Two types of data in a parallel region - shared or private. shared data any data declared outside a parallel region are by default shared (any thread using a variable x will access the same memory location). private data any variable declared inside the block following an OpenMP directive will be local to the executing thread. firstprivate private variable initialized by the value of the variable lastprivate after the parallel region the variable is set equal to the value of the private version of whichever thread executed the final iteration int i, j; int k = 0; double x; # pragma omp parallel for shared ( i) private ( j) \ firstprivate ( k) lastprivate ( x)

19 Controlling thread affinity Thread affinity becomes important on multi-socket system. OMP PLACES defines a series of places to which threads are assigned. threads: each place corresponds to a single hw. thread. cores: each place corresponds to a single core (consisting of one or more threads). sockets: each place corresponds to a single socket (consisting of one or more cores). OMP PROC BIND false: thread affinity disabled, runtime may move threads between OpenMP places true: locks threads to cores spread: spread threads evenly among the places close: pack threads close the master in the places list master: collocates threads with master

20 OpenMP Tasks

21 Tasks in OpenMP OpenMP specification version 3.0 introduced a new feature called tasking. Tasking enables parallelization of applications where units of work are generated dynamically, as in recursive structures or while loops. In OpenMP, an explicit task is specified using the task directive. # pragma omp task [ clause ]...

22 Task Execution When a thread encounters a task construct, it may choose to execute the task immediately or defer its execution until a later time. If task execution is deferred, task is placed on a task pool. A thread executing a task may differ from the thread that originally encountered it.

23 Data Scoping with Tasks Data scoping clauses: shared(list), private(list), firstprivate(list) Static and global variables are shared. Automatic storage (local) variables are private. Orphaned task variables are firstprivate by default. Non-orphaned task variables inherit the shared attribute. Variables are firstprivate unless shared in the enclosing context.

24 Example: Fibonacci Sequential code int main ( int argc, char * argv [] ) fib (N); int fib ( int n ) if (n < 2) return n; int x = fib (n - 1); int y = fib (n - 2); return x + y;

25 Example: Fibonacci Sequential code int main ( int argc, char * argv [] ) fib (N); int fib ( int n ) if (n < 2) return n; int x = fib (n - 1); int y = fib (n - 2); return x + y; Parallel code int main ( int argc, char * argv [] ) # pragma omp parallel # pragma omp single fib (N); int fib ( int n ) if (n < 2) return n; int x, y; # pragma omp task shared (x) x = fib (n - 1); # pragma omp task shared (y) y = fib (n - 2); # pragma omp taskwait return x + y;

26 barrier vs. taskwait vs. taskgroup barrier directive All tasks created by any thread of the current Team are guaranteed to be completed at barrier exit. # pragma omp barrier taskwait directive Encountering task is suspended until child tasks are complete. Applies only to direct children, not descendants! # pragma omp taskwait taskgroup directive Specifies the wait on completion of child tasks and their descendants. Deeper synchronization than taskwait Can be restricted to a subset of tasks (as opposed to barrier) Can be used for cancellation # pragma omp taskgroup...

27 Example: taskwait vs. taskgroup Taskwait int main () # pragma omp parallel # pragma omp single # pragma omp task # pragma omp critical printf (" Task 1\n"); # pragma omp task sleep (1); # pragma omp critical printf (" Task 2\n"); # pragma omp taskwait # pragma omp task # pragma omp critical printf (" Task 3\n"); Taskgroup int main () # pragma omp parallel # pragma omp single # pragma omp taskgroup # pragma omp task # pragma omp critical printf (" Task 1\n"); # pragma omp task sleep (1); # pragma omp critical printf (" Task 2\n"); /* end of taskgroup */ # pragma omp task # pragma omp critical printf (" Task 3\n");

28 Task dependency # pragma omp task depend ( dependency - type : list ) The task dependence is fulfilled when the predecessor task has completed in dependency-type: the generated task will be a dependent task of all previously generated sibling tasks that reference at least one of the list items in an out or inout clause. out and inout dependency-type: the generated task will be a dependent task of all previously generated sibling tasks that reference at least one of the list items in an in, out, or inout clause. # pragma omp task x = f (); # pragma omp task y = g(x); # pragma omp task depend ( out : x) x = f (); # pragma omp task depend ( in: x) y = g(x);

29 taskloop construct Parallelizes loop by creating tasks for one or more iterations of the loop. Cut loops into chunks and create a task for each loop chunk. Inherits clauses from worksharing and task construct. Provides better load balancing in some cases. Attitional clauses: grainsize(grain-size): chunks have at least grain-size and max 2*grain-size loop iterations, num tasks(num-tasks): create num-tasks tasks for iterations of the loop. # pragma omp parallel shared (a, b, c) # pragma omp single # pragma omp task long_running_comp (); // can execute concurrently # pragma omp taskloop grainsize (1000) for ( int i = 0; i < ; ++i) // can execute concurrently c[i] = a[i] + b[i];

30 Cancellation User can request a cancellation of a construct. Threads/tasks will be cancelled and execution continues after the end of the construct. Applicable to: parallel, for, taskgroup, sections. Threads/tasks stop execution at a certain point and not immediately. # pragma omp parallel shared ( matrix ) # pragma omp for for ( int row = 0; row < rows ; row ++) for ( int col = 0; col < cols ; col ++) if ( matrix (row, col ) == 0) # pragma omp cancel for # pragma omp cancellation point for

31 Vectorization with OpenMP 4.x

32 Vectorization with OpenMP 4.x (Image: Intel)

33 Creating a Code Suitable for Vectorization I. Original code double *x = new double [2* M]; // s stored in Array of Structures // x = [ x1_1, x1_2, x2_1, x2_2,...]... // quadrature points double *wx = new double [M];... double f = 0.0; double a = get_a (); for ( int i = 0; i < M; ++i) f = eval (x + 2*i, a); entry += f * wx[i]; return entry ; double eval ( double * x, double a) return ( sin (x [0]) + cos (x [1]) ) * a;

34 Creating a Code Suitable for Vectorization I. Code with aligned memory allocations and AoS converted to SoA Original code double *x = new double [2* M]; // s stored in Array of Structures // x = [ x1_1, x1_2, x2_1, x2_2,...]... // quadrature points double *wx = new double [M];... double f = 0.0; double a = get_a (); for ( int i = 0; i < M; ++i) f = eval (x + 2*i, a); entry += f * wx[i]; return entry ; double eval ( double * x, double a) return ( sin (x [0]) + cos (x [1]) ) * a; // allocate aligned arrays for 1 st // and 2 nd coordinates of points x double *x_1 = ( double *) _mm_malloc ( M* sizeof ( double ), 64 ); double *x_2 = ( double *) _mm_malloc ( M* sizeof ( double ), 64 ); // x stored in Structure of Arrays // x_1 = [ x1_1, x2_1, x3_1,...] // x_2 = [ x1_2, x2_2, x3_2,...]... double *w = ( double *) _mm_malloc ( M* sizeof ( double ), 64 );... double f = 0.0; double a = get_a (); for ( int i = 0; i < M; ++i) f = eval ( x_1 [i], x_2 [i], a); entry += f * wx[i]; return entry ; double eval ( double & x_1, double & x_2, double & a) return ( sin ( x_1 ) + cos ( x_2 ) ) * a;

35 SIMD Construct SIMD Construct: The simd construct can be applied to a loop to indicate that the loop can be transformed into a SIMD loop (that is, multiple iterations of the loop can be executed concurrently using SIMD instructions). Syntax: # pragma omp simd [ clause [, clause ]...] for - loop For-loop has to be in canonical loop form (see OpenMP 4.5 API:2.6)

36 SIMD Construct Clauses safelen(n1[,n2]...) n1, n2,... must be power of 2: The compiler can assume a vectorization for a VL of n1, n2,... to be safe private(v1, v2,...): Variables private to each iteration lastprivate(...): Last value is copied out from the last iteration instance linear(v1:step1, v2:step2,...) For every iteration of original scalar loop v1 is incremented by step1,... Therefore it is incremented by step1 * VL for the vectorized loop. reduction(operator:v1, v2,...) Variables v1, v2,... are reduction variables for operation operator collapse(n): Combine nested loops - collapse them aligned(v1:base, v2:base,...) Tell variables v1, v2,... are aligned; default is architecture specific alignment

37 SIMD Construct Example Ignore data dependencies, indirectly mitigate control flow dependence and assert alignment: (Image: Intel) Get the info from the optimization report by compiling with -qopt-report=[0-5].

38 Declare SIMD Construct SIMD-enabled function (aka. declare simd construct): The declare simd construct can be applied to a function [...] to enable the creation of one or more versions that can process multiple arguments using SIMD instructions from a single invocation from a SIMD loop. Syntax: # pragma omp declare simd [ clause [, clause ]...] function definition or declaration Intent: Express work as scalar operations (kernel) and let compiler create a vector version of it. The size of vectors can be specified at compile time (SSE, AVX,...) which makes it portable!

39 Declare SIMD Construct Clauses simdlen(len) len must be power of 2: Allow as many elements per argument (default is implementation specific) linear(v1:step1, v2:step2,...) Defines v1, v2,... to be private to SIMD lane and to have linear (step1, step2,... ) relationship when used in context of a loop uniform(a1, a2,...) Arguments a1, a2,... etc. are not treated as vectors (constant values across SIMD lanes) inbranch, notinbranch: SIMD-enabled function called only inside branches or never aligned(a1:base, a2:base,...) Tell arguments a1, a2,... are aligned; default is architecture specific alignment

40 Declare SIMD Construct Example Ignore data dependencies, indirectly mitigate control flow dependence and assert alignment: (Image: Intel)

41 OpenMP 4.5 OpenMP 4.5 was ratified November 2015: OpenMP simd linear clause: linear(val(var):[step]) (default): Specifies that the value of each list item on each lane corresponds to the value of the list item upon entry to the function plus the logical number of the lane times linear-step. linear(uval(var):[step]) (C++, Fortran): Similar to VAL but each invocation uses the same storage location for each SIMD lane. For val, a vector of addresses (references) is passed to the vector variant of the routine; for uval, only one address (reference) is passed, which may improve performance. linear(ref(var):step) (C++, Fortran): Specifies that the storage location of each list item on each lane corresponds to an array at the storage location upon entry to the function indexed by the logical number of the lane times linear-step. OpenMP simd declare newly for C++ virtual functions.

42 Creating a Code Suitable for Vectorization II. Original code double *x = new double [2* M]; // x stored in Array of Struct. // x = [ x1_1, x1_2, // x2_1, x2_2,...]... // quadrature points double *wx = new double [M];... double f = 0.0; double a = get_a (); for ( int i = 0; i < M; ++i) f = eval (x + 2*i, a); entry += f * wx[i]; return entry ; Code with aligned memory allocations, SoA, and vectorization pragmas double *x_1 = ( double *) _mm_malloc ( M* sizeof ( double ), 64 ); double *x_2 = ( double *) _mm_malloc ( M* sizeof ( double ), 64 );... double *w = ( double *) _mm_malloc ( M* sizeof ( double ), 64 );... double f = 0.0; double a = get_a (); # pragma omp simd reduction ( + : entry ) \ aligned ( x_1, x_2, w ) private ( f ) \ simdlen ( 4 ) for ( int i = 0; i < M; ++i) f = eval ( x_1 [i], x_2 [i], a); entry += f * wx[i]; return entry ; double eval ( double * x, double a) return ( sin (x [0]) + cos (x [1]) ) * a; # pragma omp declare simd \ linear ( ref ( x_1, x_2 ) ) uniform ( a ) \ simdlen ( 4 ) notinbranch aligned (x_1, x_2 ) double eval ( double & x_1, double & x_2, double & a) return ( sin ( x_1 ) + cos ( x_2 ) ) * a;

43 Offloading to Accelerators

44 Execution model host-centric the execution of an OpenMP program starts on the host device and it may offload target regions to the target device. If a target device is not present, not supported, or not available, the target region is executed by the host device. The most important OpenMP constructs #pragma omp target #pragma omp target data #pragma omp target update #pragma omp declare target, #pragma omp end declare target #pragma omp teams, #pragma omp distribute

45 target data construct Creates a device data environment for the extent of the region # pragma omp target data [ clause ]... Here, clause may be device( integer-expression ), map( map-type: list), if( scalar-expression ). The map clause maps a variable from current task s data environment to the device data environment associated with the construct. alloc-type: each new corresponding list item has an undefined initial value, to-type: each new corresponding list item is initialized with the original list item value, from-type: on the exit from the region the corresponding list item s value is assigned to the original list item, tofrom-type: combination of the previous two, default.

46 target construct Creates a device data environment and execute the construct on the same device. In addition to the target data construct specifies that the associated region is executed by a device. The encountering task waits for the device to complete the target region. # pragma omp target [ clause ]... Here, clause may be device( integer-expression ), map( map-type: list), if( scalar-expression ).

47 target update construct Makes the corresponding list items in the device data environment consistent with their original list items, according to the specified motion clauses. # pragma omp target update motion - clause [ clause ] motion-clause: to( list ), from( list ). clause: device( integer-expression ), if( scalar-expression )

48 declare target directive Specifies that variables, functions (C, C++, Fortran) and subroutines (Fortran) are mapped to a device If a list item is a function or subroutine then a device-specific version of the routines are created that can be called from a target region. If a list item is a variable then the original variable is mapped to a corresponding variable in the initial device data environment for all devices (if the variable is initialized it is mapped with the same value). Both declaration and definition of a function must have a declare target directive. # pragma omp declare target... # pragma omp end declare target

49 teams construct Creates a league of thread teams and the master thread of each team executes the region. The number of team is determined by the num teams clause, the number of threads in each team is determined by the thread limit clause. omp get team num() to identify current team. The team region is executed by the master thread of each team. Threads other than master do not begin execution until the master thread encounters a parallel region. Threads in different teams cannot synchronize with each other. Must be perfectly nested in a target construct. Only special OpenMP constructs can be nested inside a team construct: distribute, parallel, parallel for, and parallel sections.

50 distribute construct Work sharing construct for target and teams region Distribute the iterations of a loop across the master threads of the teams executing the region. No implicit barrier at the end of the construct. clause dist schedule( kind, chunk-size ): kind must be static, distributes chunks of chunk-size across master threads of teams in a round-robin fashion. double sum = 0.0; int i, i0; # pragma omp target map (to: B [0:N], C [0:N]) map ( tofrom : sum ) # pragma omp teams num_teams ( num_teams ) thread_limit ( block_threads ) \ reduction (+: sum ) # pragma omp distribute for ( i0 = 0; i0 < N; i0 += block_size ) # pragma omp parallel for reduction (+: sum ) for ( i = i0; i < min (i0+block_size, N); ++i ) sum += B[i] * C[i];

51 Other examples double sum = 0.0; int i, i0; # pragma omp target teams map (to: B [0:N], C [0:N]) \ map ( tofrom : sum ) reduction (+: sum ) # pragma omp distribute parallel for reduction (+: sum ) for ( i = 0; i < N; ++i ) sum += B[i] * C[i]; double sum = 0.0; int i, i0; # pragma omp target map (to: B [0:N], C [0:N]) map ( tofrom : sum ) # pragma omp teams num_teams (8) thread_limit (16) reduction (+: sum ) # pragma omp distribute parallel for reduction (+: sum ) / dist_schedule ( static, 1024) schedule ( static, 64) for ( i = 0; i < N; ++i ) sum += B[i] * C[i]; init (v1, v2, N); int i, i0; # pragma omp target teams map (to: v1 [0:N], v2 [0:N]) map (from : p [0:N]) # pragma omp distribute simd for ( i = 0; i < N; ++i ) p[i] = v1[i] * v2[i];