Exercise: OpenMP Programming

Size: px

Start display at page:

Download "Exercise: OpenMP Programming"

Rudolf Horton
5 years ago
Views:

Exercise: OpenMP Programming Multicore programming with OpenMP 19.04.2016 A.

1 Exercise: OpenMP Programming Multicore programming with OpenMP A. Marongiu - amarongiu@iis.ee.ethz.ch D. Palossi dpalossi@iis.ee.ethz.ch ETH zürich

ES 2.0, OpenGL ES 1.1 and OpenCL 1.1 EP) 2Gbyte LPDDR3 RAM PoP USB 3.

2 Odroid Board Board Specs Exynos5 Octa Cortex -A15 1.4Ghz quad core and Cortex -A7 quad core CPUs PowerVR SGX544MP3 GPU (OpenGL ES 2.0, OpenGL ES 1.1 and OpenCL 1.1 EP) 2Gbyte LPDDR3 RAM PoP USB 3.0 Host x 1, USB 3.0 OTG x 1, USB 2.0 Host x 4 HDMI 1.4a output Type-D connector emmc 4.5 Flash Storage 2

Login, compile and run Login to Odroid board: ssh X stud_usr@<board-ip> Exercise files are already on the board under*: /home/stud_usr/asocd16/src Compile and run with make clean all run Take a look

3 Login, compile and run Login to Odroid board: ssh X stud_usr@<board-ip> Exercise files are already on the board under*: /home/stud_usr/asocd16/src Compile and run with make clean all run Take a look at test.c. Different exercises are #ifdef-ed To compile and execute the desired exercise: make clean all run e MYOPTS="-DEX1 DEX2 " * if you need a fresh copy is available at: /home/soc_master/3_openmp/asocd16.zip 3

4 Ex 1 Parallelism creation: Hello World! #pragma omp parallel num_threads (?) printf "Hello world, I m thread??" Use parallel directive to create multiple threads Each thread executes the code enclosed within the scope of the directive Use runtime library functions to determine thread ID All SPMD parallelization is based on this approach 4

5 ETH zürich Ex 2 Loop partitioning: Static scheduling T0 0 T1 4 T2 8 T3 12 #pragma omp parallel for \ num_threads (NTHREADS) schedule (static) for (uint i=0; i<niters; i++) work (W); } /* (implicit) SYNCH POINT */ With schedule(static) M iterations are statically assigned to each thread, where M = NITERS / NTHREADS Small overhead: loop indexes are computed according to thread ID Optimal scheduling if workload is balanced 5

6 Ex 2 Loop partitioning: Static scheduling Exercise 2a #pragma omp parallel for \ num_threads (NTHREADS) schedule (static) for (uint i=0; i<niters; i++) work (W); } /* (implicit) SYNCH POINT */ Use the following parameters: NITERS = 1024 NTHREADS = 1,2,4,8,16} W = Collect execution time in excel sheet for the various configurations Comment on the results 6

7 ETH zürich Ex 2 Loop partitioning: Dynamic scheduling T0 T1 T2 T3 #pragma omp parallel for num_threads (NTHREADS) schedule (dynamic, M) for (uint i=0; i<niters; i++) work (W); Iterations are dynamically assigned to threads in groups of M = NITERS/NTHREADS Same parallelization of schedule(static) Coarse granularity Overhead only at beginning and end of loop OVERHEAD OVERHEAD \ \ } /* (implicit) SYNCH POINT */ CHUNK CHUNK 7

8 ETH zürich Ex 2 Loop partitioning: Dynamic scheduling T0 T1 T2 T OVERHEAD #pragma omp parallel for num_threads (NTHREADS) schedule (dynamic, M) \ \ for (uint i=0; i<niters; i++) work (W); } /* (implicit) SYNCH POINT */ Iterations are dynamically assigned to threads one iteration per chunk Finest granularity Overhead at every iteration CHUNK CHUNK (size (size == 11 iter) iter) 8

9 Ex 2 Loop partitioning: Dynamic scheduling Exercise 2b #pragma omp parallel for \ num_threads (NTHREADS) \ schedule (dynamic, M) for (uint i=0; i<niters; i++) work (W); } /* (implicit) SYNCH POINT */ Use the following parameters: NITERS = 1024 NTHREADS = 1,2,4,8,16} M = NITERS/NTHREADS, 1} W = Add execution time for the various configurations to the excel sheet Comment on the results 9

10 Ex 2 Loop partitioning: Dynamic scheduling Exercise 2c #pragma omp parallel for \ num_threads (NTHREADS) \ schedule (dynamic, M) for (uint i=0; i<niters; i++) work (W); } /* (implicit) SYNCH POINT */ Use the following parameters: NITERS = NTHREADS = 1,2,4,8,16} M = NITERS/NTHREADS, 1} W = 10 Draw the same plot as exercise 2a, 2b What happens for smaller workload and higher iteration count? 10

11 ETH zürich Ex 3 Unbalanced Loop Partitioning T0 T1 T SYNCH POINT T3 #pragma omp parallel for \ num_threads (NTHREADS) \ schedule (dynamic, NITERS/NTHREADS) 12 for (uint i=0; i<niters; i++) work((i>>2) * W); } /* (implicit) SYNCH POINT */ Iterations have different duration Using coarse-grained chunks (NITERS/NTHREADS) creates unbalanced work among the threads Due to the barrier at the end of parallel region, all threads have to wait for the slowest one 11

12 ETH zürich Ex 3 Unbalanced Loop Partitioning T0 T1 T T SPEEDUP #pragma omp parallel for num_threads (NTHREADS) schedule (dynamic, 1) \ \ for (uint i=0; i<niters; i++) work((i>>2) * W); } /* (implicit) SYNCH POINT */ Iterations have different duration Using fine-grained chunks (the finest is 1) creates balanced work among the threads The overhead for dynamic scheduling is amortized by the effect of work-balancing on the overall loop duration 12

13 Ex 3 Unbalanced Loop Partitioning Exercise 3a, 3b #pragma omp parallel for \ num_threads (NTHREADS) \ schedule (dynamic, M) for (uint i=0; i<niters; i++) /* BALANCED LOOP CODE */ } /* (implicit) SYNCH POINT */ Use the following parameters: NITERS = 128 NTHREADS = 4 M = 32, 16, 8, 4, 1} W = Create a new excel sheet plotting results (execution time) for static and dynamic schedules (with chunk size M) Comment on results 13

new excel sheet plotting results (execution time) for static and dynamic schedules (with chunk size M)

14 Ex 4 Chunking Overhead Study the impact of chunk size for varying sizes of the workload W Use the following parameters: NITERS = 1024*256 NTHREADS = 4 M = 32, 16, 8, 4, 1} W = 1, 10, 100, 1000} Create a new excel sheet plotting results (execution time) for static and dynamic schedules (with chunk size M) Comment on results HINT: Plot NORMALIZED execution time to the fastest scheduling option for a given value of W 14

15 Ex 5 Task parallelism with sections void sections() work( ); printf("%hu: Done with first elaboration!\n, ); work( ); printf("%hu: Done with second elaboration!\n, ); work( ); printf("%hu: Done with third elaboration!\n", ); work( ); printf("%hu: Done with fourth elaboration!\n", ); } a) Distribute workload among 4 threads using SPMD parallelization I. Get thread id II. Use if/else or switch/case to differentiate workload b) Implement the same workload partitioning with sections directive 15

16 Ex 6 Task parallelism with task void tasks() unsigned int i; } for(i=0; i<4; i++) work((i+1)* ); printf("%hu: Done with elaboration\n", ); } a) Distribute workload among 4 threads using task directive a) Same program as before b) But we had to manually unroll the loop to use sections Compare performance and ease of coding with sections 16

17 Ex 6 Task parallelism with task void tasks() unsigned int i; } Modify the EX6 exercise code as indicated on this slide for(i=0; i<1024; i++) work(( ); printf("%hu: Done with elaboration\n", ); } b) Parallelize the loop with task directive Use single directive to force a single processor to create tasks c) Parallelize the loop with single directive Use nowait clause to allow for parallel execution Compare performance and ease of coding of the two solutions 17

18 Projects: come to the software side If you are interested in course/semester/master project focused on OpenMP development for: Many-cores embedded devices Parallel-Ultra-Low-Power architectures High-Performance parallel machines Do not hesitate to contact us 18

MULTICORE PROGRAMMING WITH OPENMP

MULTICORE PROGRAMMING WITH OPENMP Dott. Alessandro Capotondi alessandro.capotondi@unibo.it Outline and Goals Understand OpenMP directives usage Manage data parallelism and load balancing using OpenMP Understand