Lab: Scientific Computing Tsunami-Simulation

Transcription

1 Lab: Scientific Computing Tsunami-Simulation Session 4: Optimization and OMP Sebastian Rettenberger, Michael Bader Session 4: Optimization and OMP,

2 Department of Informatics V Linux-Cluster Session 4: Optimization and OMP,

3 Hardware Segment #Nodes #Cores aggregate peak performance (TFlop/s) aggregate memory (TByte) CoolMUC MPP UV Session 4: Optimization and OMP,

4 Intel Haswell EP Session 4: Optimization and OMP,

5 Rules 1. Do not use login nodes to run your application. 2. Do not start more than one interactive job. 3. Do not put large input/output files in your home directory. Session 4: Optimization and OMP,

6 Some Basics X-forward: ssh -Y module system: module load, list, unload, info, avail Compute Interactive: salloc --ntasks=1 --cpus-per-task=28 --time=00:15:00 srun./executable ARGS... Compute Batch: squeue sbatch sinfo sview Session 4: Optimization and OMP,

7 VTune Session 4: Optimization and OMP,

8 VTune Start Analysis Session 4: Optimization and OMP,

9 VTune Start Analysis Session 4: Optimization and OMP,

10 VTune Results Session 4: Optimization and OMP,

11 What is OpenMP? OpenMP is a specification for a set of compiler directives, library routines, and environment variables that can be used to specify high-level parallelism in Fortran and C/C++ programs Session 4: Optimization and OMP,

12 Department of Informatics V Shared-Memory-Parallelism Transparent programming view: Cores access same memory In hardware: Be aware of layout, e.g. NUMA; first touch policy Sandy Bridge CPU, cpus/2011/01/03/intel-sandybridge-review/1 Knights Corner and Knights Landing, Session 4: Optimization and OMP,

13 Fork-Join Model Session 4: Optimization and OMP,

14 A Simple Loop # i n c l u d e <omp. h> / / some i n i t i a l i z a t i o n #pragma omp p a r a l l e l f o r f o r ( i = 0; i < n ; i ++) { f o r ( j = 0; j < n ; j ++) { / / do some work } } OpenMP - Parallel programming on shared memory systems parallel/openmp/ Session 4: Optimization and OMP,

15 OMP: An Overview Compiler-directives: #pragma omp...[clause[[,] clause]...] Work-sharing #pragma omp for..., and many more Synchronization #pragma omp barrier, and many more Data Scope Clauses shared, private, firstprivate, reduction,... Library routines omp get num threads(), omp get thread num(),... Session 4: Optimization and OMP,

16 Compilation and Execution compile with additional Compiler-Flag -openmp (-fopenmp in case of gcc): icc -openmp -o my algorithm alg.c export OMP NUM THREADS=number of threads./my algorithm Session 4: Optimization and OMP,

17 Parallel Region #pragma omp p a r a l l e l [ clause [ [, ] clause ]... ] s t r u c t u r e d block code within parallel region is executed by all threads # i n c l u d e <omp. h> / / more i n i t i a l i z a t i o n... #pragma omp p a r a l l e l p r i v a t e ( size, rank ) { size = omp get num threads ( ) ; rank = omp get thread num ( ) ; p r i n t f ( Hello World! ( Thread %d of %d ), rank, size ) ; } Session 4: Optimization and OMP,

18 Work Sharing: for #pragma omp for [clause[[,] clause]...] for-loop within a parallel region and directly in front of a for-loop iterations are scheduled across different threads schedule clause determines how to map iterations to threads schedule(static[, chunksize]) default-value chunksize is #iterations divided by #threads schedule(dynamic[,chunksize]) default-value chunksize is 1 schudule(runtime) Scheduling is given by environment variable OMP SCHEDULE implicit synchronization ad the end of for-loop (can be disabled with nowait clause) shortcut possible: #pragma omp parallel for llnl.gov/tutorials/ openmp/ Session 4: Optimization and OMP,

19 Work Sharing: for #pragma omp for [clause[[,] clause]...] for-loop within a parallel region and directly in front of a for-loop iterations are scheduled across different threads schedule clause determines how to map iterations to threads schedule(static[, chunksize]) default-value chunksize is #iterations divided by #threads schedule(dynamic[,chunksize]) default-value chunksize is 1 schudule(runtime) Scheduling is given by environment variable OMP SCHEDULE implicit synchronization ad the end of for-loop (can be disabled with nowait clause) shortcut possible: #pragma omp parallel for Intel, Load Balance and Parallel Performance, intel.com/enus/articles/loadbalance-and-parallelperformance Session 4: Optimization and OMP,

20 Work Sharing: task #pragma omp task [clause[[,] clause]...] structured block where clause can be... final(expression): if expression is true, task is executed sequentially; no recursive task generation untied: the execution of a task is no tied one single thread shared firstprivate private [...] #pragma omp taskwait: waits for children completion Intel, Load Balance and Parallel Performance, intel.com/enus/articles/loadbalance-and-parallelperformance Session 4: Optimization and OMP,

21 Work Sharing : single #pragma omp single [clause[[,] clause]...] structured block where clause can be... use only one (arbitrary) thread implicit synchronization #pragma omp master structured block use only master thread for structured block NO synchronization at the end llnl.gov/tutorials/ openmp/ Session 4: Optimization and OMP,

22 Synchronization #pragma omp barrier: block execution until all threads have reached the barrier #pragma omp critical: structured block only one thread at a time can execute the structured block encapsulated by critical Warning: Use carefully; can definitely kill performance! Session 4: Optimization and OMP,

23 Reduction reduction(operator: list) executes a reduction of variables in list using operator available operators: +,, -, &&, since OpenMP 3.1: min, max #pragma omp p a r a l l e l f o r p r i v a t e ( r ), r e d u c t i o n ( + : sum) f o r ( i = 0; i < n ; i ++) { r = compute r ( i ) ; sum = sum + r ; } Session 4: Optimization and OMP,

24 Scopes private(list): declares variables in list as private for each thread (no copy!) shared(list): variables in list are used by all threads (race conditions are possible!), write accesses have to be handled by the programmer firstprivate(list): private variables and init them with the latest valid value before parallel region lastprivate(list): after parallel execution, in serial part reused last modification... default data scope is shared, but exceptions exit be precise! local variables are always private loop-variables of for-loops are private Changes between OpenMP-2 and OpenMP-3! Session 4: Optimization and OMP,

25 Fill the Gaps k = compute k ( ) ; #pragma omp p a r a l l e l f o r p r i v a t e (? ), shared (? ), f i r s t p r i v f o r ( i = 0; i < n ; i ++) { k = compute my k ( i, k ) ; r = compute r ( i, k, h ) ; sum = sum + r ; } Session 4: Optimization and OMP,

26 NUMA Non-uniform memory access (NUMA): Not all cores access all memory in the same time Pinning: Use close-to-core memory as far as possible Linux first touch policy: not the malloc counts but the first (write) access numactl has some options Georg Hager s Blog, Session 4: Optimization and OMP,