Fractals exercise. Investigating task farms and load imbalance

Transcription

1 Fractals exercise Investigating task farms and load imbalance

2 Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. This means you are free to copy and redistribute the material and adapt and build on the material under the following terms: You must give appropriate credit, provide a link to the license and indicate if changes were made. If you adapt or build on the material you must distribute your work under the same license as the original. Note that this presentation contains images owned by others. Please seek their permission before reusing these images.

3 Aims Explore how the granularity of tasks impacts performance Trade-off between the amount of parallelism (number of parallel tasks) and amount of communication (size of tasks) Consider issues surrounding load balance Remember the runtime of the code is determined by the slowest running task so we want work to be as evenly distributed as possible The exercise introduces a Load Imbalance Factor (LIF) which illustrates how much faster your code could run if the load was evenly distributed

4 What are fractals? Ideas behind the Mandelbrot and Julia sets

5 The Mandelbrot Set The Mandelbrot Set is the set of numbers resulting from repeated iterations of the complex (i = -1) function: Z n Z 2 n C 1 with the initial condition Z 0 0 C = x 0 +iy 0 belongs to the Mandelbrot set if Z n remains bounded i.e. does not diverge Z n = x n + iy n, Z n 2 = (x n 2 y n 2 + 2ix n y n ), Z n 2 =(x n2 +y n2 )

6 The Mandelbrot Set cont. Separating out the real and imaginary parts gives: Z n = Z n r +iz n i Z r n = x 2 2 n-1 - y n-1 Z n i + x 0 = 2x n-1 y n-1 + y 0 Take the threshold value as: Z 2 ³ 4.0 Set the maximum number of iterations to N max - Assume that Z does not diverge at higher values of N max

7 The Julia Set Similar algorithm to Mandelbrot Set recall: Z n Z 2 n C Z 0 1, C x 0 iy 0, 0 There are an infinite number of Julia sets, parameterised by a complex number C Z n Z 2 C Z x iy 0 0 n 1, 0 for example, C = i 0.156

8 Visualisation To visualise a Mandelbrot/Julia set: Represent the complex plane as a 2D grid where complex numbers correspond to points on the grid (x, y) Calculate number of iterations N for the series to diverge (exceed the threshold) for each point on the grid If it does not diverge, N = N max Convert the value of N to a colour and plot this on the grid

9 Mandelbrot Set Very slow to compute Very quick to compute

10 Parallel implementation How do we parallelise computation of these fractals?

11 Parallelisation Values for each coordinate depend only on the previous values at that coordinate. decompose 2D grid into equally sized blocks no communications between blocks needed. Don t know in advance how much work is needed. number of iterations across the blocks varies. work dynamically assigned to workers as they become available. Implementation Split the grid into blocks: each block corresponds to a task. master process hands out tasks to worker processes. workers return completed task to master.

12 Example: Parallelisation on 4 CPUs master CPU workers CPU 2 CPU 3 CPU y x In diagram, colour represents which worker did the task number gives the task id tasks scan from left to right, moving upwards

13 Parallelisation cont in supplied code shading represents worker here we have added worker id as a number by hand e.g. taskfarm run on 5 CPUs 1 master 4 workers total number of tasks = 16

14 Notes about the exercise

15 Exercise You are supplied with source code etc. Compile and run on the machine Visualise results Quantify performance results For a fixed number of workers improve load balance by increasing number of tasks (decrease size) compute LIF to estimate minimum achievable runtime is this minimum ever reached?

16 Exercise outcomes What do the timings tell us about HPC machines?

17 Example results (fixed number of workers)

18 Results cont.

19 16 workers and 16 tasks -----Workload Summary (number of iterations)---- Total Number of Workers: 16 Total Number of Tasks: 16 Total Worker Load: Average Worker Load: Maximum Worker Load: Minimum Worker Load: Time taken by 16 workers was (secs) Load Imbalance Factor:

20 16 workers and 64 tasks -----Workload Summary (number of iterations) Total Number of Workers: 16 Total Number of Tasks: 64 Total Worker Load: Average Worker Load: Maximum Worker Load: Minimum Worker Load: Time taken by 16 workers was (secs) Load Imbalance Factor:

21 Key points to take away TASK FARMS Also known as the master/worker pattern Allows a master process to distribute work to a set of worker processors. Can be used for other types of tasks but it complicates the situation and other patterns may be more suitable for implementing. Master process is responsible for creating, distributing and gathering the individual jobs. Can improve load balance by using more tasks than workers with some overhead Load imbalance adversely affects performance especially as number of processors increases

22 Key points to take away TASKS Units of work Vary in size, do not have to be of consistent execution time. If execution times are known it can help with load balancing. QUEUES Master generates a pool of tasks and puts them in a queue Workers assigned task from queue when idle

23 Key points to take away LOAD BALANCING How a system determines how work or tasks are distributed across workers (processes or threads) Successful load balancing avoids idle processes and overloading single cores Poor load balancing leads to under-utilised cores, reducing performance.

24 Key points to take away COST Increasingly important Finite budgets require optimal use of resources requested. Load balancing is just one method of ensuring optimal usage and avoiding wasting resources. More power and resources do not necessarily mean improved performance. Always ask is it necessary to run this on 4000 cores or could it be run on 2000 more efficiently?