OpenMP Library Functions and Environmental Variables. Most of the library functions are used for querying or managing the threading environment

Transcription

1 OpenMP Library Functions and Environmental Variables Most of the library functions are used for querying or managing the threading environment The environment variables are used for setting runtime parameters Runtime library takes precedence in conflict 1 OpenMP Components Directives Runtime Library Routines Environmental Variables 2 1

2 OpenMP Library Functions void omp_set_num_threads(int num_threads) sets the default number of threads to use for parallel regions (an alternative to the OMP_NUM_THREADS environment variable and the num_threads clause). int omp_get_num_threads(void) returns the number of threads currently executing the parallel region in which it is called. int omp_get_max_threads(void) returns the largest number of threads that could be used in subsequent parallel regions. int omp_get_thread_num(void) returns the thread number of the thread executing the function. 3 OpenMP Library Functions (II) int omp_get_num_procs(void) returns the number of physical processors available to the program/process. int omp_in_parallel(void) returns a non-zero value if called within a parallel region; otherwise it returns 0. void omp_set_dynamic(int dynamic_threads) enables or disables dynamic adjustment of threads (an alternative to the OMP_DYNAMIC environment variable). int omp_get_dynamic(void) returns a non-zero value if dynamic adjustment is enabled; otherwise it returns

3 OpenMP Library Functions (III) void omp_set_nested(int nested) enables or disables nested parallelism (an alternative to the OMP_NESTED environment variable). int omp_get_nested(void) returns a non-zero value if nested parallelism is enabled; otherwise it returns 0. void omp_init_lock(omp_lock_t *lock) initializes a lock. void omp_init_nest_lock(omp_lock_t *lock) initializes a nestable lock. void omp_destroy_lock(omp_lock_t *lock) ensures that a lock is uninitialized. 5 OpenMP Library Functions (IV) void omp_destroy_nest_lock(omp_lock_t *lock) ensures that a nestable lock is uninitialized. void omp_set_lock(omp_lock_t *lock) blocks the thread until the specified lock is available and then sets the lock; the lock must have been previously initialized. void omp_set_nest_lock(omp_lock_t *lock) blocks the thread until the specified nestable lock is available and then sets the nestable lock; the lock must have been previously initialized. void omp_unset_lock(omp_lock_t *lock) releases ownership of a lock. void omp_unset_nest_lock(omp_lock_t *lock) releases ownership of a nestable lock. 6 3

4 OpenMP Library Functions (V) int omp_test_lock(omp_lock_t *lock) attempts to set a lock but does not block execution of the thread. int omp_test_nest_lock(omp_lock_t *lock) attempts to set a nestable lock but does not block execution of the thread. double omp_get_wtime(void) returns the number of wall clock seconds since some (arbitrary) time in the past. double omp_get_wtick(void) returns the number of seconds between successive clock ticks. 7 OpenMP Components Directives Runtime Library Routines Environmental Variables 8 4

5 OpenMP Environment Variables OMP_SCHEDULE is used to set the default scheduling type and optional chunk size for and parallel for directives. OMP_NUM_THREADS is used to set the default number of threads to use during execution, unless it's overridden by calls to omp_set_num_threads() or by num_threads clause on a parallel directive. OMP_DYNAMIC can be set to TRUE or FALSE to enable or disable dynamic adjustment of threads, respectively. OMP_NESTED can be set to TRUE or FALSE to enable or disable nested parallelism, respectively. 9 When to use what? OMP_NUM_THREADS: How many to use in parallel region OMP_GET_NUM_THREADS, OMP_SET_NUM_THREADS Related: OMP_GET_THREAD_NUM, OMP_GET_MAX_THREADS, OMP_GET_NUM_PROCS OMP_DYNAMIC: Should runtime system choose number of threads? OMP_GET_NESTED, OMP_SET_DYNAMIC OMP_NESTED: Should nested parallel regions be supported? OMP_GET_NESTED, OMP_SET_NESTED OMP_SCHEDULE: Choose DO scheduling option Used by RUNTIME clause OMP_IN_PARALLEL: Is the program in a parallel region? 10 5

6 Introduction : OpenMP Hello, World! File name: hello.f program hello print *, Hello, world from thread:!$omp parallel print *, omp_get_thread_num()!$omp end parallel print*, Back to sequential: end 11 Discussion Before the class: Read the three Linux Magazine articles: OpenMP Multi-Processing, Part 1, 2, 3 by Forrest Hoffman Run the code examples on a SMP machine Prepare three presentations, one for each article In class: Present the articles 12 6

7 Programming Shared Memory Systems with OpenMP Part II Instructor Dr. Taufer Synchronization Synchronization: mechanisms by which a parallel program can coordinate the execution of multiple threads Implicit synchronizations Explicit synchronizations Main use of explicit synchronization to control the access to shared objects: Mutual exclusion Event synchronization 14 7

8 Mutual Exclusion This construct is used to control access to a shared variable by providing a thread exclusive access to a shared variable for the duration of the construct. critical directive atomic directive Explicit synchronizations > mutual exclusion 15 Critical Directive CRITICAL Only one thread at a time executes the enclosed block May specify a name that is common to multiple CRITICALs!$OMP PARALLEL PRIVATE(MYDATA)!$OMP CRITICAL(MYSECT) READ *, MYDATA!$OMP END CRITICAL(MYSECT)...!$OMP END PARALLEL 16 8

9 Example: Mandelbrot Program real*8 x, y integer I, j, m, n, maxiter integer depth(*,*) integer mandel_val, tot_iterats. maxiter = 2000 tot_iterats = 0 do i= 1, m do j = 1, n x = I / real (m) y = j / real(n) // evaluate the Mandelbrot equation We want to know the number of iterations executed by the generator!!! potential data race // depth will given to a graphics routine for drawing the image // depth is the actual number of iterations executed depth(j, i) = mandel_val (x, y, maxiter) tot_iterats = tot_iterats + depth(j,i) enddo enddo 17 Data Races All form of concurrent access to the same shared variable are called data races Data races must be coordinated to ensure correct results when at least one of the accesses to the variable is a write synchronization is required In Mandelbrot program:!$omp critical tot_iterats = tot_iterats + depth(j,i)!$omp end critical 18 9

10 Example: Mandelbrot Program (II) maxiter = 2000 tot_iterats = 0!$omp parallel do private(j, x, y) do i= 1, m do j = 1, n x = I / real (m) y = j / real(n) // evaluate the Mandelbrot equation // depth will given to a graphics routine for drawing the image // depth is the actual number of iterations executed depth(j, i) = mandel_val (x, y, maxiter)!$omp critical tot_iterats = tot_iterats + depth(j,i)!$omp end critical enddo enddo!$omp end parallel do 19 Mandelbrot Program with Critical Session critical sessions waiting to enter critical sessions 20 10

11 Example: Mandelbrot Program (III) maxiter = 2000 tot_iterats = 0!$omp parallel do private(j, x, y)!$omp+ reduction (+:tot_iterats) do i= 1, m do j = 1, n x = I / real (m) y = j / real(n) // evaluate the Mandelbrot equation // depth will given to a graphics routine for drawing the image // depth is the actual number of iterations executed depth(j, i) = mandel_val (x, y, maxiter) tot_iterats = tot_iterats + depth(j,i) enddo enddo!$omp end parallel do reduction(+:tot_iterats) tells the compiler that tot_iterats is the target of a reductions 21 Example: Data Race without Conflicts!$omp critical (MINLOCK) do i = 1, n a(i) = a(i) + b endo 22 11

12 Example: Data Race with Conflicts! Finding the largest element in a list of numbers cur_max = MINUS_INFINITY!omp parallel do do i = 1, n if (a(i).gt. cur_max) then cur_max= a(i)!!!! CONFLICT endif endo 23 Possible Execution Fragment Thread 0 read a(i) (value = 12) read cur_max (value = 10) if ( a(i) > cur_max ) (12>10) cur_max = a(i) (i.e. 12) Thread 1 read a(i) (value 11) read cur_max (value = 10) if ( a(i) > cur_max ) (11 > 10 ) cur_max = a(j) (i.e. 11) Is this correct? 24 12

13 Example: Acceptable Data Race foundit =.FALSE.!$omp parallel do do i = 1, n if (a(i).eq. item) then foundit =.TRUE. endif endo If at least one a(i) is equal to item, foundit becomes true 25 Named Critical Sessions If a thread is inside section updating an object, it prevents another thread from entering another critical section to update another object This can negatively affect the parallelism exploited in the program OpenMP allows named critical sections: a named critical section must synchronized with other critical sections of the same name but can execute concurrently with critical sections of different name Unnamed critical sections synchronized only with other unnamed critical sessions

14 cur_max = MINUS_INFINITY cur_min + PLUS_INFINITY!$omp parallel do do i = 1,n if (a(i).gt. cur_max) then!$omp critical (MAXLOCK) if (a(i).gt. cur_max) then cur_max = a(i) endif!$omp end critical (MAXLOCK) endif if (a(i).lt. cur_min) then!$omp critical (MINLOCK) if (a(i).lt. cur_min) then cur_min= a(i) endif!$omp end critical (MINLOCK) endif endo The two critical sections can run concurrently with two different threads The nested if guarantees the thread exclusive access to cur_max and cur_min 27 Atomic Directive ATOMIC Specifies an atomic update of a variable Can be used, if available (hardware support needed)!$omp PARALLEL SHARED(S)...!$OMP ATOMIC S = S !$OMP END PARALLEL Explicit synchronizations > mutual exclusion > atomic directive 28 14

15 Restrictions of Atomic Directives Atomic directive is similar to critical directive Express mutual exclusion Atomic directive is implemented using the hardware synchronization primitives Critical directives enclose a block of code Atomic directives can be applied only if the critical section consists of a single assignment statement that updates a scalar variable Guidelines: Use the atomic directives when updating either a single location or a few locations Use a critical directive when updating several locations Explicit synchronizations > mutual exclusion > atomic directive 29 Syntax of Atomic Directives!$omp atomic x = x operator expr!$omp atomic x = predefined_operator (x, expr) #pragma omp atomic x < binop >= expr #pragma omp atomic /* One of */ x++, ++x, x--, or --x Explicit synchronizations > mutual exclusion > atomic directive 30 15