COL 730: Parallel Programming. OpenMP

Size: px
Start display at page:

Download "COL 730: Parallel Programming. OpenMP"

Transcription

1 COL 730: Parallel Programming OpenMP

2 Parallel Programming Break computation into small pieces Schedule for each processors(i) for all jobs j DO Job(i, j) Issues: Granularity Communication Synchronization Load balancing 2

3 Parallel Program Design Decompose into steps, Partition into concurrent tasks Determine granularity Start with ~5-10x #processors Target similar size Minimize dependence/communication Form task dependency graph nodes correspond to tasks and edges indicating dependence Form task interaction graph edges indicate data communication Manage data communication (sharing) Determine synchronization points Group tasks Balance load Reduce communication Map each group to a processor Statically or dynamically Issues: Granularity Communication Synchronization Load balancing 3

4 Hello OpenMP #pragma omp parallel { // I am now thread i of n switch(omp_get_thread_num()) { case 0 : blah1.. case 1: blah2.. // Back to normal Parallel Construct Extremely simple to use and incredibly powerful Fork-Join model Every thread has its own executon context Variables can be declared shared or private

5 Shared Memory Programming High level language for i=0 to N a[i] = f(b[i], c[i], d[i]) Derive parallelism Generate threads and map to processors Addresses for a, b, c, d accessible to all also the code for f Map i to threadid Impact on cache coherence?

6 User directed Shared Memory Programming A way to generate new threads of control functon per thread? Work sharing construct? Synchronize specify shared variables Maybe, for an arbitrary group of threads Ways to map each thread to processor? May have more threads than processors Need high level constructs for all these

7 Hello OpenMP #pragma omp parallel { // I am now thread i of n switch(omp_get_thread_num()) { case 0 : blah1.. case 1: blah2.. // Back to normal Parallel Construct Extremely simple to use and incredibly powerful Fork-Join model Every thread has its own executon context Variables can be declared shared or private

8 ExecuTon Model Encountering thread creates a team: Itself (master) + zero or more additonal threads. Applies to structured block immediately following Each thread executes separately the code in { But, also see: Work-sharing constructs There s an implicit barrier at the end of block Only master contnues beyond the barrier May be nested SomeTmes disabled by default

9 Memory Model NoTon of temporary view of memory Allows local caching Need to relax consistency model Supports threadprivate memory global scope Variables declared before parallel construct: Shared by default May be designated as private n-1 copies of the original variable is created May not be initalized by the system

10 Variable Sharing among Threads Shared: Heap allocated storage StaTc data members const variable (no mutable member) Private: auto Variables declared in a scope inside the construct Loop variable in for construct private to the construct Others are shared unless declared private You can change default Arguments passed by reference inherit from original

11 Relaxed Consistency Unsynchronized access: T1 If two Write threads X (5) write to the Flush(X) same shared variable the Write result X is undefined Flush(X) Read X (5) If a thread reads and another writes, the read value Read is undefined X Memory atom size is implementation dependent Flush All x,y,z threads.. enforces observe consistency. this order Specs say: T2 If the intersection of the flush-sets of two flushes performed by two different threads is nonempty, then the two flushes must be completed as if in some sequential order, seen by all threads. If the intersection of the flush-sets of two flushes performed by one thread is nonempty, then the two flushes must appear to be completed in that thread s program order. T Write X 11

12 Beware of Compiler Re-ordering a = b = 0 thread 1 thread 2 b = 1 a = 1 if (a == 0) { if (b == 0) { critical section critical section

13 Beware of Compiler Re-ordering a = b = 0 thread 1 thread 2 b = 1 a = 1 flush(b); flush(a); flush(a); flush(b); if (a == 0) { if (b == 0) { critical section critical section

14 Beware of Compiler Re-ordering a = b = 0 thread 1 thread 2 b = 1 a = 1 flush(b,a); flush(a,b); if (a == 0) { if (b == 0) { critical section critical section

15 Single Producer Single Consumer int data, flag = 0; F A Thread 0 // Produce data data = 42; // Flush #pragma omp flush(flag, data) // Set flag to signal Thread 1 flag = 1; // Flush #pragma omp flush(flag) Thread 1 // Busy-wait until flag is signalled #pragma omp flush(flag) while (flag!= 1) { #pragma omp flush(flag, data) #pragma omp flush(flag, data) // Consume data printf(data=%d\n, data); A had started and hence F had happened

16 Thread Control Environment Variable Ways to modify value Way to retrieve value Initial value OMP_NUM_THREADS * omp_set_num_threads omp_get_max_threads Implementation defined OMP_DYNAMIC omp_set_dynamic omp_get_dynamic Implementation defined OMP_NESTED omp_set_nested omp_get_nested false OMP_SCHEDULE * Implementation defined * Also see construct clause: num_threads, schedule

17 Parallel Construct #pragma omp parallel \ { if(boolean) \ private(var1, var2, var3) \ firstprivate(var1, var2, var3) \ default(private shared none) \ shared(var1, var2) \ copyin(var1, var2) \ reducton(operator:list) \ num_threads(n) RestricTons: Cannot branch in or out No side effect from clause: must not depend on any ordering of the evaluations Upto one if clause Upto one num_threads clause num_threads must be a +ve integer

18 int sumint(int data[]) { int sum; #pragma omp parallel reduction(+:sum) sum = partial_sum(data, omp_get_thread_num()); return sum; Reduction

19 Reductions Reductions are common scalar f(v1.. vn) Specify reduction operation and variable OpenMP code combines results from the loop stores partial results in private variables Automatically private, no need to designate Initialized based on operator

20 reduc/on Clause reduction (<op> :<variable>) + Sum * Product & Bitwise and Bitwise or ^ Bitwise exclusive or && Logical and Logical or Add to parallel for OpenMP creates a loop to combine copies of the variable The resultng loop may not be parallel

21 What s wrong? int Td, size; int numprocs = omp_get_num_procs(); #pragma omp parallel num_threads(numprocs) { size = getproblemsize()/numprocs; // assume divisible Td = omp_get_thread_num(); dotask(td*size, size); dotask(int start, int count) { // Each thread s instance has its own actvaton record for(int i = 0, t=start; i< count; i++, t+=1) doit(t); 21

22 Declare locally (private) int size; int numprocs = omp_get_num_procs(); #pragma omp parallel num_threads(numprocs) { size = getproblemsize()/numprocs; int Td = omp_get_thread_num(); dotask(td*size, size); dotask(int start, int count) { // Each thread s instance has its own actvaton record for(int i = 0, t=start; i< count; i++; t+=1) doit(t); 22

23 Private clause int Td, size; int numprocs = omp_get_num_procs(); #pragma omp parallel num_threads(numprocs) private(td) { size = getproblemsize()/numprocs; Td = omp_get_thread_num(); dotask(td*size, size); dotask(int start, int count) { // Each thread s instance has its own actvaton record for(int i = 0, t=start; i< count; i++; t+=stride) doit(t); 23

24 Parallel Loop #pragma omp parallel for for (i= 0; i < N; ++i) { blah Num of iteratons must be known when the construct is encountered Must be the same for each encountering thread Compiler puts a barrier at the end of parallel for But see nowait

25 Parallel For Construct #pragma omp for \ private(var1, var2, var3) \ firstprivate(var1, var2, var3)\ lastprivate(var1, var2) \ reducton(operator: list) \ ordered \ schedule(kind[,chunk_size])\ nowait Canonical For Loop No break RestricTons: same loop control expression for all threads in the team. At most one schedule, nowait, ordered clause chunk_size must be a loop/construct invariant, +ve integer ordered clause required if any ordered region inside

26 Firstprivate and Lastprivate IniTal value of private variable is unspecified firstprivate initalizes copies with the original Once per thread (not once per iteraton) Original exists before the construct The original copy lives auer the construct lastprivate forces sequental-like behavior thread executng the sequentally last iteraton (or last listed secton) writes to the original copy

27 Firstprivate and Lastprivate simple = 0; #pragma omp parallel for firstprivate(simple) for (int i=0; i<n; i++) { simple += foo[i][omp_get_thread_num()] process(simple) #pragma omp parallel for lastprivate(i, last) for( i=0; i<n; i++ ) { last = foo(i) process(last) #pragma omp parallel for lastprivate( doneearly ) for( i=0; i<n; i++ ) { doneearly = f0(i);

28 Subdivide Work int numprocs = omp_get_num_procs(); #pragma omp parallel num_threads(numprocs) private(size) { int size = getproblemsize()/numprocs; int Td = omp_get_thread_num(); dotask(td*size, size); dotask(int start, int count) { // Each thread s instance has its own actvaton record for(int t=start; t< start+count; t++) doit(t); 28

29 Work Sharing for #pragma omp parallel for { for(int i=0; i<problemsize; i++) doit(i); Works even if number of tasks is not divisible by number of threads 29

30 Reduction int sum2d(int data[n][n]) { int sum; #pragma omp parallel for reduction(+:sum) for (int i=0; i<n; i++) { for (int j=0; j<n; j++) { sum += data[i][j]; return sum;

31 Schedule(kind[, chunk_size]) Divide iteratons into contguous sets, chunks chunks are assigned transparently to threads sta/c: chunks are assigned in a round-robin fashion default chunk_size is roughly Load/num_threads dynamic: chunks are assigned to threads as requested default chunk_size is 1 guided: dynamic, with chunk size proportonal to #unassigned iteratons divided by num_threads chunk size is at least chunk_size iteratons (except the last) default chunk_size is 1 run/me: taken directly from environment variable

32 Worksharing: Single #pragma omp parallel { #pragma omp for for( int i=0; i<n; i++ ) a[i] = f0(i); #pragma omp single x = f1(a); #pragma omp for for(int i=0; i<n; i++ ) b[i] = x * f2(i); Only one of the threads executes Other threads wait for it unless NOWAIT is specified Hidden complexity Threads may not hit single

33 Worksharing: SecTons #pragma omp sectons { #pragma omp secton { // do this #pragma omp secton { // do that // omp sec/on pragma must be closely nested in a sectons construct, where no other work-sharing construct may appear.

34 Other SynchronizaTon DirecTves #pragma omp master { binds to the innermost enclosing parallel region Only the master executes No implied barrier

35 Master DirecTve #pragma omp parallel { #pragma omp for for( int i=0; i<100; i++ ) a[i] = f0(i); #pragma omp master x = f1(a); Only master executes. No synchronizaton.

36 CriTcal SecTon #pragma omp critcal (accessbankbalance) { A single thread at a Tme through all regions of the same name Applies to all threads The name is optonal Anonymous = global critcal region

37 Barrier DirecTve #pragma omp barrier Stand-alone Binds to inner-most parallel region All threads in the team must execute they will all wait for each other at this instructon Dangerous: if (! ready) #pragma omp barrier Same sequence of work-sharing and barrier for the entre team

38 Ordered DirecTve #pragma omp ordered { Binds to inner-most enclosing loop The structured block executed in loop sequential order The loop must declare the ordered clause Each thread must encounter only one ordered region

39 Flush DirecTve #pragma omp flush (var1, var2) Stand-alone, like barrier Only directly affects the encountering thread List-of-vars ensures that any compiler re-ordering moves all flushes together implicit: barrier, atomic, critcal, locks

40 atomic DirecTve #pragma omp atomic i++; Light-weight critcal secton Only for some expressions x binop= expr (no mutual exclusion on expr evaluaton) x++ ++x x-- --x

41 Atomic int sum2d(int data[n][n]) { int sum = 0; #pragma omp parallel for for (int i=0; i<n; i++) { for (int j=0; j<n; j++) { #pragma omp atomic sum += data[i][j]; return sum;

42 Tasks Encountering thread creates a task - Code, data, environment.. Some thread of the team executes the task - Scheduling points - Start, End, taskwait Barrier

43 Task Construct #pragma omp task \ { if(boolean) \ unted \ default(shared none) \ private(list) \ firstprivate(list) \ shared(list) \ RestricTons: Cannot branch in or out No side effect from clause: must not depend on any ordering of the evaluations Upto one if clause

44 struct node { node *left, *right; ; extern void process(node* ); void traverse(node* p) { if (p->left) #pragma omp task // p is firstprivate by default traverse(p->left); if (p->right) #pragma omp task // p is firstprivate by default traverse(p->right); process(p); #pragma omp taskwait

45 float seq_sum(const float *a, int n) { return (n == 0)? 0. : (n == 1)? *a : seq_sum(a, n/2) + seq_sum(a + n/2, n - n/2); float par_sum(const float *a, int n) { if (n < LARGE) return seq_sum(a, n); float x, y; Sum #pragma omp task shared(x) x = parallel_sum(a, n/2); #pragma omp task shared(y) y = parallel_sum(a + n/2, n - n/2); #pragma omp taskwait x += y; return x; #pragma omp parallel #pragma omp single nowait sum = par_sum(a, n);

46 Simd DirecTve #pragma omp simd safelen(length) linear(list[:linear-step]) aligned(list[:alignment]) private(list) lastprivate(list) reduction(reduction-identifier: list) collapse(n) for-loops Vectorizes within a thread

47 Helper Functions: General void omp_set_dynamic (int); int omp_get_dynamic (); void omp_set_nested (int); int omp_get_nested (); int omp_get_num_procs(); int omp_get_num_threads(); int omp_get_thread_num(); int omp_get_ancestor_thread_num(); double omp_get_wtime(); 47

48 Helper Functions: Mutex void omp_init_lock (omp_lock_t *); void omp_destroy_lock (omp_lock_t *); void omp_set_lock (omp_lock_t *); void omp_unset_lock (omp_lock_t *); int omp_test_lock (omp_lock_t *); nested lock versions: e.g., omp_set_nest_lock(omp_test_lock_t *); 48

49 Lock Lock is an abstract datatype, supports three operations: new creates a new lock that is initially open acquire closes a lock if open - blocks if closed, until someone else opens it release opens a lock Also exist: Non-blocking locks

50 Re-entrant lock new creates a new lock with no current holder and a count of 0 acquire blocks if held by someone different from the caller. - If caller is the holder increment count. - If not already held, caller gets a hold release sets the current holder to none if the count is 0. - Otherwise, count--

51 Race Condition Non-deterministic error Data races: - RW: One thread reads an object at the same moment that another thread writes the same object. - WW: One thread writes an object at the same moment that another thread also writes the same object.

52 Helper Functions: Mutex void omp_init_lock (omp_lock_t *); void omp_destroy_lock (omp_lock_t *); void omp_set_lock (omp_lock_t *); void omp_unset_lock (omp_lock_t *); int omp_test_lock (omp_lock_t *); nested lock versions: e.g., omp_set_nest_lock(omp_test_lock_t *); 52

53 NesTng RestricTons A critcal region may not be nested ever inside a critcal region with the same name Not sufficient to prevent deadlock Not allowed without intervening parallel region: Inside work-sharing, critcal, ordered, or master Work-sharing barrier Inside a work-sharing region master region Inside a critcal region ordered region

54 EXAMPLES

55 Simd Examples #pragma omp simd reduc/on(+:sum) aligned(a : 64) for(i = 0; i < num; i++) { c[i] = a[i] + b[i]; sum = sum + a[i]; #pragma omp simd void invokesfoo(float *a, float *x, int n) { #pragma omp parallel for simd for(i = 0; i < num; i++) { c[i] = a[i] + b[i]; for (int i=0; i<n; i++) a[i] = foo(x[i]); #pragma omp declare simd float foo(float x) { // func/on body

56 Ordered Construct int i; #pragma omp for ordered for (i=0; i<n; i++) { if(isgroupa(i) { #pragma omp ordered doit(i); else { #pragma omp ordered doit(partner(i)); 56

57 Wrong Use of multiple Orders int i; #pragma omp for ordered for (i=0; i<n; i++) { #pragma omp ordered doit(i); #pragma omp ordered doit(partner(i)); 57

58 OpenMP Matrix MulTply #pragma omp parallel for for( int i=0; i<n; i++ ) for( int j=0; j<n; j++ ) { c[i][j] = 0.0; for(int k=0; k<n; k++ ) c[i][j] += a[i][k]*b[k][j]; a, b, c are shared i, j, k are private

59 OpenMP Matrix MulTply #pragma omp parallel for for( int i=0; i<n; i++ ) #pragma omp parallel for for( int j=0; j<n; j++ ) { c[i][j] = 0.0; for(int k=0; k<n; k++ ) c[i][j] += a[i][k]*b[k][j];

60 OpenMP Matrix MulTply #pragma omp parallel for for( int i=0; i<n; i++ ) #pragma omp parallel for for( int j=0; j<n; j++ ) { int sum = 0.0; #pragma omp parallel for reducton(+:sum) for(int k=0; k<n; k++ ) sum += a[i][k]*b[k][j]; c[i][j] = sum;

61 OpenMP Matrix Multiply: Triangular #pragma omp parallel for schedule (dynamic, 1) for( int i=0; i<n; i++ ) for( int j=i; j<n; j++ ) { c[i][j] = 0.0; for(int k=i; k<n; k++ ) c[i][j] += a[i][k]*b[k][j]; This multiplies upper-triangular matrix A with B Unbalanced workload Schedule improves this

62 Jacobi IteraTon? for some number of Tmesteps/iteraTons { #pragma omp parallel for for (int i=0; i<n; i++ ) for( int j=0, j<n, j++ ) grid[i][j] = 0.25 * ( grid[i-1][j] + grid[i+1][j] grid[i][j-1] + grid[i][j+1] );

63 OpenMP Jacobi for some number of Tmesteps/iteraTons { #pragma omp parallel for for (int i=0; i<n; i++ ) for( int j=0, j<n, j++ ) temp[i][j] = 0.25 * ( grid[i-1][j] + grid[i+1][j] grid[i][j-1] + grid[i][j+1] ); #pragma omp parallel for for( int i=0; i<n; i++ ) for( int j=0; j<n; j++ ) grid[i][j] = temp[i][j]; This could be improved by using just one parallel region Implicit barrier after loops eliminates race on grid

64 OpenMP Jacobi for some number of Tmesteps/iteraTons { #pragma omp parallel for for (int i=0; i<n; i++ ) for( int j=0, j<n, j++ ) { temp[i][j] = 0.25 * ( grid[i-1][j] + grid[i+1][j] grid[i][j-1] + grid[i][j+1] ); #pragma omp barrier grid[i][j] = temp[i][j]; Is barrier sufficient? What change to the code is needed? Recall barrier is per-team

65 #pragma omp parallel shared(a, b, nthreads, locka, lockb) private(tid) #pragma omp sections nowait { #pragma omp section { omp_set_lock(&locka); for (i=0; i<n; i++) a[i] = i * DELTA; omp_set_lock(&lockb); for (i=0; i<n; i++) b[i] += a[i]; omp_unset_lock(&lockb); omp_unset_lock(&locka); #pragma omp section { omp_set_lock(&lockb); for (i=0; i<n; i++) b[i] = i * PI; omp_set_lock(&locka); for (i=0; i<n; i++) a[i] += b[i]; omp_unset_lock(&locka); omp_unset_lock(&lockb); /* end of sections */ Find the Error Assume: variables declared locks initalized 65

66 Find the Error void worksum(float *x, float *y, int *index, int n) { #pragma omp parallel shared(x, y, index, n) { #pragma omp for nowait for (int i=0; i<n; i++) { #pragma omp atomic x[index[i]] += work1(i); y[i] += work2(i); int work0 = x[0]

67 Efficiency Issues Minimize synchronizaton Avoid BARRIER, CRITICAL, ORDERED, and locks Use NOWAIT Use named CRITICAL sectons for fine-grained locking Use MASTER (instead of SINGLE) Parallelize at the highest level possible such as outer FOR loops keep parallel regions large FLUSH is expensive LASTPRIVATE has synchronizaton overhead Thread safe malloc/free are expensive Reduce False sharing Design of data structures Use PRIVATE

68 Common SMP Errors SynchronizaTon Race conditon depends on Tming Deadlock waitng for a non-existent conditon Livelock contnuously adjustng, but task progress stalled Try to Avoid nested locks Release locks religiously Avoid while true (especially, during testng) Be careful with Non thread-safe libraries Concurrent access to shared data IO inside parallel regions Differing views of shared memory (FLUSH) NOWAIT

CSL 730: Parallel Programming. OpenMP

CSL 730: Parallel Programming. OpenMP CSL 730: Parallel Programming OpenMP int sum2d(int data[n][n]) { int i,j; #pragma omp parallel for for (i=0; i

More information

CSL 860: Modern Parallel

CSL 860: Modern Parallel CSL 860: Modern Parallel Computation Hello OpenMP #pragma omp parallel { // I am now thread iof n switch(omp_get_thread_num()) { case 0 : blah1.. case 1: blah2.. // Back to normal Parallel Construct Extremely

More information

OpenMP C and C++ Application Program Interface Version 1.0 October Document Number

OpenMP C and C++ Application Program Interface Version 1.0 October Document Number OpenMP C and C++ Application Program Interface Version 1.0 October 1998 Document Number 004 2229 001 Contents Page v Introduction [1] 1 Scope............................. 1 Definition of Terms.........................

More information

Data Handling in OpenMP

Data Handling in OpenMP Data Handling in OpenMP Manipulate data by threads By private: a thread initializes and uses a variable alone Keep local copies, such as loop indices By firstprivate: a thread repeatedly reads a variable

More information

Advanced C Programming Winter Term 2008/09. Guest Lecture by Markus Thiele

Advanced C Programming Winter Term 2008/09. Guest Lecture by Markus Thiele Advanced C Programming Winter Term 2008/09 Guest Lecture by Markus Thiele Lecture 14: Parallel Programming with OpenMP Motivation: Why parallelize? The free lunch is over. Herb

More information

Parallel and Distributed Programming. OpenMP

Parallel and Distributed Programming. OpenMP Parallel and Distributed Programming OpenMP OpenMP Portability of software SPMD model Detailed versions (bindings) for different programming languages Components: directives for compiler library functions

More information

Programming with Shared Memory PART II. HPC Fall 2012 Prof. Robert van Engelen

Programming with Shared Memory PART II. HPC Fall 2012 Prof. Robert van Engelen Programming with Shared Memory PART II HPC Fall 2012 Prof. Robert van Engelen Overview Sequential consistency Parallel programming constructs Dependence analysis OpenMP Autoparallelization Further reading

More information

Programming with Shared Memory PART II. HPC Fall 2007 Prof. Robert van Engelen

Programming with Shared Memory PART II. HPC Fall 2007 Prof. Robert van Engelen Programming with Shared Memory PART II HPC Fall 2007 Prof. Robert van Engelen Overview Parallel programming constructs Dependence analysis OpenMP Autoparallelization Further reading HPC Fall 2007 2 Parallel

More information

OpenMP. OpenMP. Portable programming of shared memory systems. It is a quasi-standard. OpenMP-Forum API for Fortran and C/C++

OpenMP. OpenMP. Portable programming of shared memory systems. It is a quasi-standard. OpenMP-Forum API for Fortran and C/C++ OpenMP OpenMP Portable programming of shared memory systems. It is a quasi-standard. OpenMP-Forum 1997-2002 API for Fortran and C/C++ directives runtime routines environment variables www.openmp.org 1

More information

by system default usually a thread per CPU or core using the environment variable OMP_NUM_THREADS from within the program by using function call

by system default usually a thread per CPU or core using the environment variable OMP_NUM_THREADS from within the program by using function call OpenMP Syntax The OpenMP Programming Model Number of threads are determined by system default usually a thread per CPU or core using the environment variable OMP_NUM_THREADS from within the program by

More information

PC to HPC. Xiaoge Wang ICER Jan 27, 2016

PC to HPC. Xiaoge Wang ICER Jan 27, 2016 PC to HPC Xiaoge Wang ICER Jan 27, 2016 About This Series Format: talk + discussion Focus: fundamentals of parallel compucng (i) parcconing: data parccon and task parccon; (ii) communicacon: data sharing

More information

Review. Tasking. 34a.cpp. Lecture 14. Work Tasking 5/31/2011. Structured block. Parallel construct. Working-Sharing contructs.

Review. Tasking. 34a.cpp. Lecture 14. Work Tasking 5/31/2011. Structured block. Parallel construct. Working-Sharing contructs. Review Lecture 14 Structured block Parallel construct clauses Working-Sharing contructs for, single, section for construct with different scheduling strategies 1 2 Tasking Work Tasking New feature in OpenMP

More information

Introduction [1] 1. Directives [2] 7

Introduction [1] 1. Directives [2] 7 OpenMP Fortran Application Program Interface Version 2.0, November 2000 Contents Introduction [1] 1 Scope............................. 1 Glossary............................ 1 Execution Model.........................

More information

OPENMP OPEN MULTI-PROCESSING

OPENMP OPEN MULTI-PROCESSING OPENMP OPEN MULTI-PROCESSING OpenMP OpenMP is a portable directive-based API that can be used with FORTRAN, C, and C++ for programming shared address space machines. OpenMP provides the programmer with

More information

Introduction to OpenMP. OpenMP basics OpenMP directives, clauses, and library routines

Introduction to OpenMP. OpenMP basics OpenMP directives, clauses, and library routines Introduction to OpenMP Introduction OpenMP basics OpenMP directives, clauses, and library routines What is OpenMP? What does OpenMP stands for? What does OpenMP stands for? Open specifications for Multi

More information

OpenMP Application Program Interface

OpenMP Application Program Interface OpenMP Application Program Interface DRAFT Version.1.0-00a THIS IS A DRAFT AND NOT FOR PUBLICATION Copyright 1-0 OpenMP Architecture Review Board. Permission to copy without fee all or part of this material

More information

CS 470 Spring Mike Lam, Professor. Advanced OpenMP

CS 470 Spring Mike Lam, Professor. Advanced OpenMP CS 470 Spring 2018 Mike Lam, Professor Advanced OpenMP Atomics OpenMP provides access to highly-efficient hardware synchronization mechanisms Use the atomic pragma to annotate a single statement Statement

More information

OpenMP Application Program Interface

OpenMP Application Program Interface OpenMP Application Program Interface Version.0 - RC - March 01 Public Review Release Candidate Copyright 1-01 OpenMP Architecture Review Board. Permission to copy without fee all or part of this material

More information

Shared Memory Parallelism - OpenMP

Shared Memory Parallelism - OpenMP Shared Memory Parallelism - OpenMP Sathish Vadhiyar Credits/Sources: OpenMP C/C++ standard (openmp.org) OpenMP tutorial (http://www.llnl.gov/computing/tutorials/openmp/#introduction) OpenMP sc99 tutorial

More information

Parallel Programming

Parallel Programming Parallel Programming OpenMP Nils Moschüring PhD Student (LMU) Nils Moschüring PhD Student (LMU), OpenMP 1 1 Overview What is parallel software development Why do we need parallel computation? Problems

More information

Parallel Programming using OpenMP

Parallel Programming using OpenMP 1 OpenMP Multithreaded Programming 2 Parallel Programming using OpenMP OpenMP stands for Open Multi-Processing OpenMP is a multi-vendor (see next page) standard to perform shared-memory multithreading

More information

Parallel Programming using OpenMP

Parallel Programming using OpenMP 1 Parallel Programming using OpenMP Mike Bailey mjb@cs.oregonstate.edu openmp.pptx OpenMP Multithreaded Programming 2 OpenMP stands for Open Multi-Processing OpenMP is a multi-vendor (see next page) standard

More information

OpenMP. Diego Fabregat-Traver and Prof. Paolo Bientinesi WS16/17. HPAC, RWTH Aachen

OpenMP. Diego Fabregat-Traver and Prof. Paolo Bientinesi WS16/17. HPAC, RWTH Aachen OpenMP Diego Fabregat-Traver and Prof. Paolo Bientinesi HPAC, RWTH Aachen fabregat@aices.rwth-aachen.de WS16/17 Worksharing constructs To date: #pragma omp parallel created a team of threads We distributed

More information

Computational Mathematics

Computational Mathematics Computational Mathematics Hamid Sarbazi-Azad Department of Computer Engineering Sharif University of Technology e-mail: azad@sharif.edu OpenMP Work-sharing Instructor PanteA Zardoshti Department of Computer

More information

OpenMP 2. CSCI 4850/5850 High-Performance Computing Spring 2018

OpenMP 2. CSCI 4850/5850 High-Performance Computing Spring 2018 OpenMP 2 CSCI 4850/5850 High-Performance Computing Spring 2018 Tae-Hyuk (Ted) Ahn Department of Computer Science Program of Bioinformatics and Computational Biology Saint Louis University Learning Objectives

More information

Topics. Introduction. Shared Memory Parallelization. Example. Lecture 11. OpenMP Execution Model Fork-Join model 5/15/2012. Introduction OpenMP

Topics. Introduction. Shared Memory Parallelization. Example. Lecture 11. OpenMP Execution Model Fork-Join model 5/15/2012. Introduction OpenMP Topics Lecture 11 Introduction OpenMP Some Examples Library functions Environment variables 1 2 Introduction Shared Memory Parallelization OpenMP is: a standard for parallel programming in C, C++, and

More information

Introduction to. Slides prepared by : Farzana Rahman 1

Introduction to. Slides prepared by : Farzana Rahman 1 Introduction to OpenMP Slides prepared by : Farzana Rahman 1 Definition of OpenMP Application Program Interface (API) for Shared Memory Parallel Programming Directive based approach with library support

More information

HPCSE - I. «OpenMP Programming Model - Part I» Panos Hadjidoukas

HPCSE - I. «OpenMP Programming Model - Part I» Panos Hadjidoukas HPCSE - I «OpenMP Programming Model - Part I» Panos Hadjidoukas 1 Schedule and Goals 13.10.2017: OpenMP - part 1 study the basic features of OpenMP able to understand and write OpenMP programs 20.10.2017:

More information

OpenMP Overview. in 30 Minutes. Christian Terboven / Aachen, Germany Stand: Version 2.

OpenMP Overview. in 30 Minutes. Christian Terboven / Aachen, Germany Stand: Version 2. OpenMP Overview in 30 Minutes Christian Terboven 06.12.2010 / Aachen, Germany Stand: 03.12.2010 Version 2.3 Rechen- und Kommunikationszentrum (RZ) Agenda OpenMP: Parallel Regions,

More information

Parallel Programming with OpenMP. CS240A, T. Yang

Parallel Programming with OpenMP. CS240A, T. Yang Parallel Programming with OpenMP CS240A, T. Yang 1 A Programmer s View of OpenMP What is OpenMP? Open specification for Multi-Processing Standard API for defining multi-threaded shared-memory programs

More information

Introduction to Standard OpenMP 3.1

Introduction to Standard OpenMP 3.1 Introduction to Standard OpenMP 3.1 Massimiliano Culpo - m.culpo@cineca.it Gian Franco Marras - g.marras@cineca.it CINECA - SuperComputing Applications and Innovation Department 1 / 59 Outline 1 Introduction

More information

Parallel Computing Parallel Programming Languages Hwansoo Han

Parallel Computing Parallel Programming Languages Hwansoo Han Parallel Computing Parallel Programming Languages Hwansoo Han Parallel Programming Practice Current Start with a parallel algorithm Implement, keeping in mind Data races Synchronization Threading syntax

More information

HPC Practical Course Part 3.1 Open Multi-Processing (OpenMP)

HPC Practical Course Part 3.1 Open Multi-Processing (OpenMP) HPC Practical Course Part 3.1 Open Multi-Processing (OpenMP) V. Akishina, I. Kisel, G. Kozlov, I. Kulakov, M. Pugach, M. Zyzak Goethe University of Frankfurt am Main 2015 Task Parallelism Parallelization

More information

Data Environment: Default storage attributes

Data Environment: Default storage attributes COSC 6374 Parallel Computation Introduction to OpenMP(II) Some slides based on material by Barbara Chapman (UH) and Tim Mattson (Intel) Edgar Gabriel Fall 2014 Data Environment: Default storage attributes

More information

OpenMP loops. Paolo Burgio.

OpenMP loops. Paolo Burgio. OpenMP loops Paolo Burgio paolo.burgio@unimore.it Outline Expressing parallelism Understanding parallel threads Memory Data management Data clauses Synchronization Barriers, locks, critical sections Work

More information

Introduction to OpenMP.

Introduction to OpenMP. Introduction to OpenMP www.openmp.org Motivation Parallelize the following code using threads: for (i=0; i

More information

Module 10: Open Multi-Processing Lecture 19: What is Parallelization? The Lecture Contains: What is Parallelization? Perfectly Load-Balanced Program

Module 10: Open Multi-Processing Lecture 19: What is Parallelization? The Lecture Contains: What is Parallelization? Perfectly Load-Balanced Program The Lecture Contains: What is Parallelization? Perfectly Load-Balanced Program Amdahl's Law About Data What is Data Race? Overview to OpenMP Components of OpenMP OpenMP Programming Model OpenMP Directives

More information

OpenMP. Dr. William McDoniel and Prof. Paolo Bientinesi WS17/18. HPAC, RWTH Aachen

OpenMP. Dr. William McDoniel and Prof. Paolo Bientinesi WS17/18. HPAC, RWTH Aachen OpenMP Dr. William McDoniel and Prof. Paolo Bientinesi HPAC, RWTH Aachen mcdoniel@aices.rwth-aachen.de WS17/18 Loop construct - Clauses #pragma omp for [clause [, clause]...] The following clauses apply:

More information

EPL372 Lab Exercise 5: Introduction to OpenMP

EPL372 Lab Exercise 5: Introduction to OpenMP EPL372 Lab Exercise 5: Introduction to OpenMP References: https://computing.llnl.gov/tutorials/openmp/ http://openmp.org/wp/openmp-specifications/ http://openmp.org/mp-documents/openmp-4.0-c.pdf http://openmp.org/mp-documents/openmp4.0.0.examples.pdf

More information

Programming Shared-memory Platforms with OpenMP. Xu Liu

Programming Shared-memory Platforms with OpenMP. Xu Liu Programming Shared-memory Platforms with OpenMP Xu Liu Introduction to OpenMP OpenMP directives concurrency directives parallel regions loops, sections, tasks Topics for Today synchronization directives

More information

ECE 574 Cluster Computing Lecture 10

ECE 574 Cluster Computing Lecture 10 ECE 574 Cluster Computing Lecture 10 Vince Weaver http://www.eece.maine.edu/~vweaver vincent.weaver@maine.edu 1 October 2015 Announcements Homework #4 will be posted eventually 1 HW#4 Notes How granular

More information

Multi-core Architecture and Programming

Multi-core Architecture and Programming Multi-core Architecture and Programming Yang Quansheng( 杨全胜 ) http://www.njyangqs.com School of Computer Science & Engineering 1 http://www.njyangqs.com Programming with OpenMP Content What is PpenMP Parallel

More information

Introduction to OpenMP. Rogelio Long CS 5334/4390 Spring 2014 February 25 Class

Introduction to OpenMP. Rogelio Long CS 5334/4390 Spring 2014 February 25 Class Introduction to OpenMP Rogelio Long CS 5334/4390 Spring 2014 February 25 Class Acknowledgment These slides are adapted from the Lawrence Livermore OpenMP Tutorial by Blaise Barney at https://computing.llnl.gov/tutorials/openmp/

More information

Shared Memory Parallelism using OpenMP

Shared Memory Parallelism using OpenMP Indian Institute of Science Bangalore, India भ रत य व ज ञ न स स थ न ब गल र, भ रत SE 292: High Performance Computing [3:0][Aug:2014] Shared Memory Parallelism using OpenMP Yogesh Simmhan Adapted from: o

More information

OpenMP Programming. Prof. Thomas Sterling. High Performance Computing: Concepts, Methods & Means

OpenMP Programming. Prof. Thomas Sterling. High Performance Computing: Concepts, Methods & Means High Performance Computing: Concepts, Methods & Means OpenMP Programming Prof. Thomas Sterling Department of Computer Science Louisiana State University February 8 th, 2007 Topics Introduction Overview

More information

OpenMP Language Features

OpenMP Language Features OpenMP Language Features 1 Agenda The parallel construct Work-sharing Data-sharing Synchronization Interaction with the execution environment More OpenMP clauses Advanced OpenMP constructs 2 The fork/join

More information

Concurrent Programming with OpenMP

Concurrent Programming with OpenMP Concurrent Programming with OpenMP Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico October 11, 2012 CPD (DEI / IST) Parallel and Distributed

More information

Alfio Lazzaro: Introduction to OpenMP

Alfio Lazzaro: Introduction to OpenMP First INFN International School on Architectures, tools and methodologies for developing efficient large scale scientific computing applications Ce.U.B. Bertinoro Italy, 12 17 October 2009 Alfio Lazzaro:

More information

Session 4: Parallel Programming with OpenMP

Session 4: Parallel Programming with OpenMP Session 4: Parallel Programming with OpenMP Xavier Martorell Barcelona Supercomputing Center Agenda Agenda 10:00-11:00 OpenMP fundamentals, parallel regions 11:00-11:30 Worksharing constructs 11:30-12:00

More information

OpenMP. Application Program Interface. CINECA, 14 May 2012 OpenMP Marco Comparato

OpenMP. Application Program Interface. CINECA, 14 May 2012 OpenMP Marco Comparato OpenMP Application Program Interface Introduction Shared-memory parallelism in C, C++ and Fortran compiler directives library routines environment variables Directives single program multiple data (SPMD)

More information

Introduction to OpenMP

Introduction to OpenMP Introduction to OpenMP Ekpe Okorafor School of Parallel Programming & Parallel Architecture for HPC ICTP October, 2014 A little about me! PhD Computer Engineering Texas A&M University Computer Science

More information

Overview: The OpenMP Programming Model

Overview: The OpenMP Programming Model Overview: The OpenMP Programming Model motivation and overview the parallel directive: clauses, equivalent pthread code, examples the for directive and scheduling of loop iterations Pi example in OpenMP

More information

Programming Shared Address Space Platforms using OpenMP

Programming Shared Address Space Platforms using OpenMP Programming Shared Address Space Platforms using OpenMP Jinkyu Jeong (jinkyu@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Topic Overview Introduction to OpenMP OpenMP

More information

COMP4300/8300: The OpenMP Programming Model. Alistair Rendell. Specifications maintained by OpenMP Architecture Review Board (ARB)

COMP4300/8300: The OpenMP Programming Model. Alistair Rendell. Specifications maintained by OpenMP Architecture Review Board (ARB) COMP4300/8300: The OpenMP Programming Model Alistair Rendell See: www.openmp.org Introduction to High Performance Computing for Scientists and Engineers, Hager and Wellein, Chapter 6 & 7 High Performance

More information

COMP4300/8300: The OpenMP Programming Model. Alistair Rendell

COMP4300/8300: The OpenMP Programming Model. Alistair Rendell COMP4300/8300: The OpenMP Programming Model Alistair Rendell See: www.openmp.org Introduction to High Performance Computing for Scientists and Engineers, Hager and Wellein, Chapter 6 & 7 High Performance

More information

OpenMP - III. Diego Fabregat-Traver and Prof. Paolo Bientinesi WS15/16. HPAC, RWTH Aachen

OpenMP - III. Diego Fabregat-Traver and Prof. Paolo Bientinesi WS15/16. HPAC, RWTH Aachen OpenMP - III Diego Fabregat-Traver and Prof. Paolo Bientinesi HPAC, RWTH Aachen fabregat@aices.rwth-aachen.de WS15/16 OpenMP References Using OpenMP: Portable Shared Memory Parallel Programming. The MIT

More information

[Potentially] Your first parallel application

[Potentially] Your first parallel application [Potentially] Your first parallel application Compute the smallest element in an array as fast as possible small = array[0]; for( i = 0; i < N; i++) if( array[i] < small ) ) small = array[i] 64-bit Intel

More information

A brief introduction to OpenMP

A brief introduction to OpenMP A brief introduction to OpenMP Alejandro Duran Barcelona Supercomputing Center Outline 1 Introduction 2 Writing OpenMP programs 3 Data-sharing attributes 4 Synchronization 5 Worksharings 6 Task parallelism

More information

OpenMP Algoritmi e Calcolo Parallelo. Daniele Loiacono

OpenMP Algoritmi e Calcolo Parallelo. Daniele Loiacono OpenMP Algoritmi e Calcolo Parallelo References Useful references Using OpenMP: Portable Shared Memory Parallel Programming, Barbara Chapman, Gabriele Jost and Ruud van der Pas OpenMP.org http://openmp.org/

More information

Introduction to OpenMP. Martin Čuma Center for High Performance Computing University of Utah

Introduction to OpenMP. Martin Čuma Center for High Performance Computing University of Utah Introduction to OpenMP Martin Čuma Center for High Performance Computing University of Utah mcuma@chpc.utah.edu Overview Quick introduction. Parallel loops. Parallel loop directives. Parallel sections.

More information

An Introduction to OpenMP

An Introduction to OpenMP An Introduction to OpenMP U N C L A S S I F I E D Slide 1 What Is OpenMP? OpenMP Is: An Application Program Interface (API) that may be used to explicitly direct multi-threaded, shared memory parallelism

More information

OpenMP Technical Report 3 on OpenMP 4.0 enhancements

OpenMP Technical Report 3 on OpenMP 4.0 enhancements OPENMP ARB OpenMP Technical Report on OpenMP.0 enhancements This Technical Report specifies OpenMP.0 enhancements that are candidates for a future OpenMP.1: (e.g. for asynchronous execution on and data

More information

OpenMP+F90 p OpenMP+F90

OpenMP+F90 p OpenMP+F90 OpenMP+F90 hmli@ustc.edu.cn - http://hpcjl.ustc.edu.cn OpenMP+F90 p. 1 OpenMP+F90 p. 2 OpenMP ccnuma Cache-Coherent Non-Uniform Memory Access SMP Symmetric MultiProcessing MPI MPP Massively Parallel Processing

More information

Introduction to OpenMP

Introduction to OpenMP Introduction to OpenMP Le Yan Scientific computing consultant User services group High Performance Computing @ LSU Goals Acquaint users with the concept of shared memory parallelism Acquaint users with

More information

Introduction to OpenMP

Introduction to OpenMP Introduction to OpenMP Ricardo Fonseca https://sites.google.com/view/rafonseca2017/ Outline Shared Memory Programming OpenMP Fork-Join Model Compiler Directives / Run time library routines Compiling and

More information

Introduction to OpenMP

Introduction to OpenMP Christian Terboven, Dirk Schmidl IT Center, RWTH Aachen University Member of the HPC Group terboven,schmidl@itc.rwth-aachen.de IT Center der RWTH Aachen University History De-facto standard for Shared-Memory

More information

Lecture 4: OpenMP Open Multi-Processing

Lecture 4: OpenMP Open Multi-Processing CS 4230: Parallel Programming Lecture 4: OpenMP Open Multi-Processing January 23, 2017 01/23/2017 CS4230 1 Outline OpenMP another approach for thread parallel programming Fork-Join execution model OpenMP

More information

OpenMP. António Abreu. Instituto Politécnico de Setúbal. 1 de Março de 2013

OpenMP. António Abreu. Instituto Politécnico de Setúbal. 1 de Março de 2013 OpenMP António Abreu Instituto Politécnico de Setúbal 1 de Março de 2013 António Abreu (Instituto Politécnico de Setúbal) OpenMP 1 de Março de 2013 1 / 37 openmp what? It s an Application Program Interface

More information

Review. Lecture 12 5/22/2012. Compiler Directives. Library Functions Environment Variables. Compiler directives for construct, collapse clause

Review. Lecture 12 5/22/2012. Compiler Directives. Library Functions Environment Variables. Compiler directives for construct, collapse clause Review Lecture 12 Compiler Directives Conditional compilation Parallel construct Work-sharing constructs for, section, single Synchronization Work-tasking Library Functions Environment Variables 1 2 13b.cpp

More information

Practical in Numerical Astronomy, SS 2012 LECTURE 12

Practical in Numerical Astronomy, SS 2012 LECTURE 12 Practical in Numerical Astronomy, SS 2012 LECTURE 12 Parallelization II. Open Multiprocessing (OpenMP) Lecturer Eduard Vorobyov. Email: eduard.vorobiev@univie.ac.at, raum 006.6 1 OpenMP is a shared memory

More information

CS 470 Spring Mike Lam, Professor. Advanced OpenMP

CS 470 Spring Mike Lam, Professor. Advanced OpenMP CS 470 Spring 2017 Mike Lam, Professor Advanced OpenMP Atomics OpenMP provides access to highly-efficient hardware synchronization mechanisms Use the atomic pragma to annotate a single statement Statement

More information

Programming with OpenMP*

Programming with OpenMP* Objectives At the completion of this module you will be able to Thread serial code with basic OpenMP pragmas Use OpenMP synchronization pragmas to coordinate thread execution and memory access 2 Agenda

More information

Shared Memory Programming Models I

Shared Memory Programming Models I Shared Memory Programming Models I Peter Bastian / Stefan Lang Interdisciplinary Center for Scientific Computing (IWR) University of Heidelberg INF 368, Room 532 D-69120 Heidelberg phone: 06221/54-8264

More information

COSC 6374 Parallel Computation. Introduction to OpenMP. Some slides based on material by Barbara Chapman (UH) and Tim Mattson (Intel)

COSC 6374 Parallel Computation. Introduction to OpenMP. Some slides based on material by Barbara Chapman (UH) and Tim Mattson (Intel) COSC 6374 Parallel Computation Introduction to OpenMP Some slides based on material by Barbara Chapman (UH) and Tim Mattson (Intel) Edgar Gabriel Fall 2015 OpenMP Provides thread programming model at a

More information

OpenMP examples. Sergeev Efim. Singularis Lab, Ltd. Senior software engineer

OpenMP examples. Sergeev Efim. Singularis Lab, Ltd. Senior software engineer OpenMP examples Sergeev Efim Senior software engineer Singularis Lab, Ltd. OpenMP Is: An Application Program Interface (API) that may be used to explicitly direct multi-threaded, shared memory parallelism.

More information

Parallel Programming in C with MPI and OpenMP

Parallel Programming in C with MPI and OpenMP Parallel Programming in C with MPI and OpenMP Michael J. Quinn Chapter 17 Shared-memory Programming 1 Outline n OpenMP n Shared-memory model n Parallel for loops n Declaring private variables n Critical

More information

OpenMP 4.5: Threading, vectorization & offloading

OpenMP 4.5: Threading, vectorization & offloading OpenMP 4.5: Threading, vectorization & offloading Michal Merta michal.merta@vsb.cz 2nd of March 2018 Agenda Introduction The Basics OpenMP Tasks Vectorization with OpenMP 4.x Offloading to Accelerators

More information

Masterpraktikum - High Performance Computing

Masterpraktikum - High Performance Computing Masterpraktikum - High Performance Computing OpenMP Michael Bader Alexander Heinecke Alexander Breuer Technische Universität München, Germany 2 #include ... #pragma omp parallel for for(i = 0; i

More information

Introduction to OpenMP. Martin Čuma Center for High Performance Computing University of Utah

Introduction to OpenMP. Martin Čuma Center for High Performance Computing University of Utah Introduction to OpenMP Martin Čuma Center for High Performance Computing University of Utah mcuma@chpc.utah.edu Overview Quick introduction. Parallel loops. Parallel loop directives. Parallel sections.

More information

Parallel Programming. OpenMP Parallel programming for multiprocessors for loops

Parallel Programming. OpenMP Parallel programming for multiprocessors for loops Parallel Programming OpenMP Parallel programming for multiprocessors for loops OpenMP OpenMP An application programming interface (API) for parallel programming on multiprocessors Assumes shared memory

More information

Allows program to be incrementally parallelized

Allows program to be incrementally parallelized Basic OpenMP What is OpenMP An open standard for shared memory programming in C/C+ + and Fortran supported by Intel, Gnu, Microsoft, Apple, IBM, HP and others Compiler directives and library support OpenMP

More information

Multithreading in C with OpenMP

Multithreading in C with OpenMP Multithreading in C with OpenMP ICS432 - Spring 2017 Concurrent and High-Performance Programming Henri Casanova (henric@hawaii.edu) Pthreads are good and bad! Multi-threaded programming in C with Pthreads

More information

Introduction to OpenMP

Introduction to OpenMP Introduction to OpenMP Christian Terboven 10.04.2013 / Darmstadt, Germany Stand: 06.03.2013 Version 2.3 Rechen- und Kommunikationszentrum (RZ) History De-facto standard for

More information

OpenMP 4. CSCI 4850/5850 High-Performance Computing Spring 2018

OpenMP 4. CSCI 4850/5850 High-Performance Computing Spring 2018 OpenMP 4 CSCI 4850/5850 High-Performance Computing Spring 2018 Tae-Hyuk (Ted) Ahn Department of Computer Science Program of Bioinformatics and Computational Biology Saint Louis University Learning Objectives

More information

Introduction to OpenMP. Martin Čuma Center for High Performance Computing University of Utah

Introduction to OpenMP. Martin Čuma Center for High Performance Computing University of Utah Introduction to OpenMP Martin Čuma Center for High Performance Computing University of Utah m.cuma@utah.edu Overview Quick introduction. Parallel loops. Parallel loop directives. Parallel sections. Some

More information

Parallel Programming in C with MPI and OpenMP

Parallel Programming in C with MPI and OpenMP Parallel Programming in C with MPI and OpenMP Michael J. Quinn Chapter 17 Shared-memory Programming 1 Outline n OpenMP n Shared-memory model n Parallel for loops n Declaring private variables n Critical

More information

ME964 High Performance Computing for Engineering Applications

ME964 High Performance Computing for Engineering Applications ME964 High Performance Computing for Engineering Applications Parallel Computing using OpenMP [Part 2 of 2] April 5, 2011 Dan Negrut, 2011 ME964 UW-Madison The inside of a computer is as dumb as hell but

More information

Introduction to OpenMP

Introduction to OpenMP Presentation Introduction to OpenMP Martin Cuma Center for High Performance Computing University of Utah mcuma@chpc.utah.edu September 9, 2004 http://www.chpc.utah.edu 4/13/2006 http://www.chpc.utah.edu

More information

Distributed Systems + Middleware Concurrent Programming with OpenMP

Distributed Systems + Middleware Concurrent Programming with OpenMP Distributed Systems + Middleware Concurrent Programming with OpenMP Gianpaolo Cugola Dipartimento di Elettronica e Informazione Politecnico, Italy cugola@elet.polimi.it http://home.dei.polimi.it/cugola

More information

Shared memory programming model OpenMP TMA4280 Introduction to Supercomputing

Shared memory programming model OpenMP TMA4280 Introduction to Supercomputing Shared memory programming model OpenMP TMA4280 Introduction to Supercomputing NTNU, IMF February 16. 2018 1 Recap: Distributed memory programming model Parallelism with MPI. An MPI execution is started

More information

https://www.youtube.com/playlist?list=pllx- Q6B8xqZ8n8bwjGdzBJ25X2utwnoEG

https://www.youtube.com/playlist?list=pllx- Q6B8xqZ8n8bwjGdzBJ25X2utwnoEG https://www.youtube.com/playlist?list=pllx- Q6B8xqZ8n8bwjGdzBJ25X2utwnoEG OpenMP Basic Defs: Solution Stack HW System layer Prog. User layer Layer Directives, Compiler End User Application OpenMP library

More information

Parallel programming using OpenMP

Parallel programming using OpenMP Parallel programming using OpenMP Computer Architecture J. Daniel García Sánchez (coordinator) David Expósito Singh Francisco Javier García Blas ARCOS Group Computer Science and Engineering Department

More information

Objectives At the completion of this module you will be able to Thread serial code with basic OpenMP pragmas Use OpenMP synchronization pragmas to coordinate thread execution and memory access Based on

More information

Shared Memory Parallelism

Shared Memory Parallelism Introduction Shared Memory Parallelism Why shared memory parallelism is important Shared memory architectures POXIS threads vs OpenMP OpenMP history First steps into OpenMP Data parallel programs How to

More information

OpenMP 4.0/4.5: New Features and Protocols. Jemmy Hu

OpenMP 4.0/4.5: New Features and Protocols. Jemmy Hu OpenMP 4.0/4.5: New Features and Protocols Jemmy Hu SHARCNET HPC Consultant University of Waterloo May 10, 2017 General Interest Seminar Outline OpenMP overview Task constructs in OpenMP SIMP constructs

More information

DPHPC: Introduction to OpenMP Recitation session

DPHPC: Introduction to OpenMP Recitation session SALVATORE DI GIROLAMO DPHPC: Introduction to OpenMP Recitation session Based on http://openmp.org/mp-documents/intro_to_openmp_mattson.pdf OpenMP An Introduction What is it? A set

More information

OpenMP Lab on Nested Parallelism and Tasks

OpenMP Lab on Nested Parallelism and Tasks OpenMP Lab on Nested Parallelism and Tasks Nested Parallelism 2 Nested Parallelism Some OpenMP implementations support nested parallelism A thread within a team of threads may fork spawning a child team

More information

Programming Shared-memory Platforms with OpenMP

Programming Shared-memory Platforms with OpenMP Programming Shared-memory Platforms with OpenMP John Mellor-Crummey Department of Computer Science Rice University johnmc@rice.edu COMP 422/534 Lecture 7 31 February 2017 Introduction to OpenMP OpenMP

More information

UvA-SARA High Performance Computing Course June Clemens Grelck, University of Amsterdam. Parallel Programming with Compiler Directives: OpenMP

UvA-SARA High Performance Computing Course June Clemens Grelck, University of Amsterdam. Parallel Programming with Compiler Directives: OpenMP Parallel Programming with Compiler Directives OpenMP Clemens Grelck University of Amsterdam UvA-SARA High Performance Computing Course June 2013 OpenMP at a Glance Loop Parallelization Scheduling Parallel

More information

Introduction to OpenMP

Introduction to OpenMP Introduction to OpenMP Le Yan HPC Consultant User Services Goals Acquaint users with the concept of shared memory parallelism Acquaint users with the basics of programming with OpenMP Discuss briefly the

More information