COMP528: Multi-core and Multi-Processor Computing

Transcription

1 COMP528: Multi-core and Multi-Processor Computing Dr Michael K Bane, G14, Computer Science, University of Liverpool m.k.bane@liverpool.ac.uk 17

2 Background Reading "Using OpenMP The Next Step: Affinity, Accelerators, Tasking and SIMD", van der Pas et al. MIT Press (2017) Homework: read Chapter 1 (a nice recap of v2.5 of OpenMP) "Using OpenMP: Portable Shared Memory Parallel Programming" Chapman et al. MIT Press (2007) D= &ppg=60 v2.5 so not cover: tasks, accelerators, some refinements

3 TODAY: Performance Issues synchronisation / nowait memory model / omp flush first touch / why initialisation parallelism matters thread affinity (important) & thread placement (not so important, generally) problem false sharing Other programming models tasks in OpenMP

4 False Sharing in reality (all) modern microprocessors use "cache lines" presumptions from history eg if we defined int x[100]; it's likely we will want to access x[0] and then soon after access x[1] and then x[2] SO, when fetching variable x why not get a "cache line" of memory (presumably including x,y,z ) so we have the data handy

5 False Sharing Fetching a cache line is (generally) good for serial with (limited if any) negative side effects we can always just fetch another cache line when we need it BUT for threaded applications one thread could fetch a cache line, update just one element, which then means the line is "dirty" so any other threads who have fetched that cache line will have to throw away whatever they have and re-fetch the whole line for whichever element they actually need we could have many threads "invalidating" elements on same cache line

6 False Sharing we could have many threads "invalidating" elements on same cache line ==> thrashing of the cache several threads having to keep getting the same cache line much more frequently, solely due to another thread updating a single item on that cache line TYPICAL WARNING SIGNS time of "par for" increases with #threads despite work being independent and load balanced (or very bad efficiency even on low #threads) profiling/valgrind show very high #cache fetches for given work sharing arrays, particularly when promoting scalar X to array X[myThread] for allows multi-thread access to X

7 Matrix-vector batch timings #threads 80K x 8 S(p) E(p) 800 x 800 S(p) E(p) 8 x 80K S(p) E(p) % % % % % % % % % % % % 3 input data sizes all have same amount of FLOPS (640K*2) 3 different rows:cols why does one of them not scale? FALSE SHARING data fetched as cache line lots of invalidations for 8x80k since in this case y[i] = A[i][j]*x[j] LHS: 8 floats = 32 bytes = single cache line ie all threads writing

8 DIY Reduction ==> False Sharing? demo: ~/OMP/falsesharing not good: timmattson.c potential fix: timmattson_padding.c qsub/qrsh tim.sh runs each (say) 500 times grep Milli nopad.txt sort --key=7 -n head to get fastest pre-fix grep Milli pad.txt sort --key=7 -n head to get fastest post-fix

9 Preventing False Sharing padding artificially increase length of allocated vector to prevent >1 thread accessing any cache line eg x[numthreads] to x[numthreads][pad] different strides through data / use of schedule(?,??) to force threads to use improved pattern (diff cache lines per thread) accessing allocated vector use a local x (ie private) for all the parallel work, then assign to y, shared, once finished

10 Tuesday Morning: general lessons from Assignment 1 Performance Issues a) thread placement/affinity b) memory placement c) dangers of false sharing DATA DEPENDENCIES TASKS SIMD and vectorisation Directives for Programming Accelerators

11 DEPENDENCIES

12 OpenMP Work-Sharing for loops but which can we parallelise?

13 Fibonnaci x[0] = 1; x[1] = 1; for (i=2; i<n; i++) { x[i] = x[i-1] + x[i-2]; } x[0] = 1 x[1] = 1 x[2] = 2 x[3] = 3 x[4] = 5 x[5] = 8 works fine sequentially iterating

14 Fibonnaci x[0] = 1; x[1] = 1; #pragma omp parallel for \ default(none) private(i) \ shared(n) shared(x) for (i=2; i<n; i++) { x[i] = x[i-1] + x[i-2]; } x[0] = 1 x[1] = 1 x[2] = x[1] + x[0] x[3] = x[2] + x[1] #0 #1# #2 i=2,3 i=4,5 i=5,6,7 X[2]=X[1]+X[0] X[3]=X[2]=X[1] i.e. there is a LOOP CARRIED DEPENDENCY (dependency but between different iterations of the loop) for i>2, x[i] depends on x[i-1] which may be being calculated on another thread for j>3, x[j] depends on x[j-2] which may be being calculated on another thread

15 Fibonnaci Presume #1 slow off mark, even though from an external view point we could see X[3], X[2] have been computed, it is likely that #0 has done so on a cache copy & that these entries have not yet been updated in the global/shared copy of X since there is no #pragma omp flush (X) x[0] = 1; x[1] = 1; #pragma omp parallel for \ default(none) private(i) \ shared(n) shared(x) for (i=2; i<n; i++) { x[i] = x[i-1] + x[i-2]; } x[0] = 1 x[1] = 1 x[2] = x[1] + x[0] x[3] = x[2] + x[1] x[4] = x[3] + x[2] #0 #1# #2 i=2,3 i=4,5 i=5,6,7 X[2]=X[1]+X[0] X[3]=X[2]=X[1] X[4]=X[3]+X[2] i.e. there is a LOOP CARRIED DEPENDENCY (dependency but between different iterations of the loop) for i>2, x[i] depends on x[i-1] which may be being calculated on another thread for j>3, x[j] depends on x[j-2] which may be being calculated on another thread

16 Fibonnaci BUT MORE REALISTICALLY #1 will be computing X[4] before #0 has calculated the required X[3], X[2] (and so on ) x[0] = 1; x[1] = 1; #pragma omp parallel for \ default(none) private(i) \ shared(n) shared(x) for (i=2; i<n; i++) { x[i] = x[i-1] + x[i-2]; } x[0] = 1 x[1] = 1 x[2] = x[1] + x[0] x[3] = x[2] + x[1] x[4] = x[3] + x[2] #0 #1# #2 i=2,3 i=4,5 i=5,6,7 X[2]=X[1]+X[0] X[4]=X[3]+X[2] X[3]=X[2]=X[1] i.e. there is a LOOP CARRIED DEPENDENCY (dependency but between different iterations of the loop) for i>2, x[i] depends on x[i-1] which may be being calculated on another thread for j>3, x[j] depends on x[j-2] which may be being calculated on another thread

17 One has to think whether a given loop can be parallelised The compiler will merely obey without question the OMP directives, even if the logic is wrong

18 5 minutes to determine which & how to parallelise

19 Data Dependency Analysis Theory ==> not in COMP528 BUT need to be able to spot dependencies need to be able to consider options for removing dependencies Quick test: if you ran the loop in the opposite direction, do you get same results? Y=> some loop invariance, possible parallelisable, but N=> definite ordering issues, no straight forward parallelisable

20 OpenMP 2.5 now covered SUMMARY Directives, Run-time Functions, Env. Vars Worksharing Synchronisation (& nowait clause) Performance OpenMP relaxed memory model first touch (& how to use it to gain performance) false sharing NOW OpenMP 4.5: tasks, vectors & accelerators "Using OpenMP The Next Step: Affinity, Accelerators, Tasking and SIMD", van der Pas et al. MIT Press (2017)

21 TASKS

22 Tasks Rather than considering workflow imperative stmt then stmt then stmt and work-sharing across a for loop Can we identify specific tasks, push them to a queue and let the system run next appropriate task when resources available closer to data flow appropriate => user defines some dependencies, RTS runs when they are resolved

23 Tasks / Tasking think task (as in a bundle of work) not threads can be used to exploit parallelism in workloads that are not a set of for loops also support while loops & recursion, & some dynamic load balancing task parallelism (using data flow)

24 create task Tasks: Concept

25 Tasks: Concept create task add to task pool

26 create task Tasks: Concept

27 Tasks: Concept create task add to pool

28 a little later we may have create task add to pool

29 BUT we can also run tasks on avail resources T I M E

30 BUT we can also run tasks on avail resources task runs on avail thread T I M E

31 BUT we can also run tasks on avail resources avail resource => assign task & repeat T I M E

32 BUT we can also run tasks on avail resources T I M E

33 BUT we can also run tasks on avail resources start new tasks when resources become available T I M E

34 BUT we can also run tasks on avail resources aha dependency purple depends on all yellow finished T I M E

35 BUT we can also run tasks on avail resources T I M E

36 Outline of syntax task creation done in a parallel region but given thread so usually within a singly threaded region master single (maybe critical as long as not same task created) #pragma omp task { block becomes task }

37 Running the task run-time decides! might be immediate might be deferred BUT programmer can use synchronisation to ensure when must be run by (or rather where to wait until task/s has run)

38 Definitions task construct task the actual instructions (& data created) when thread runs ( encounters ) the task construct different encounters of same task construct generate different tasks task region the code encountered during execution of a task

39 Two Examples Simple, non-recursive print out adjectives, any order ~/OMP/tasks/storyTelling.c with & without taskwait Fibonnaci ~/OMP/tasks/ex_fib_tasks.c ~/OMP/tasks/ex_fib_tasks_DETAIL.c (time it )

40 overheads of tasks can be high if omp par for fits, use it can arrange for task to be on a team of threads can nest tasks can use tasks for dynamic load balancing of irregular problems

41 Tasks useful to know worth consideration in depth further reading chpt 3: Using OpenMP: The Next Step forge.cineca.it/files/scuolacalcoloparallelo_webdav/public/anno- 2016/12_Advanced_School/OpenMP_4_tasks.pdf (Cineca)

42 Tuesday Morning: general lessons from Assignment 1 Still to come Performance Issues a) thread placement/affinity b) memory placement c) dangers of false sharing DATA DEPENDENCIES TASKS SIMD and vectorisation Directives for Programming Accelerators