Joe Hummel, PhD. Microsoft MVP Visual C++ Technical Staff: Pluralsight, LLC Professor: U. of Illinois, Chicago.

Transcription

1 Joe Hummel, PhD Microsoft MVP Visual C++ Technical Staff: Pluralsight, LLC Professor: U. of Illinois, Chicago stuff:

2 Async programming: Better responsiveness Parallel programming: Better performance GUIs (desktop, web, mobile) Engineering Cloud access (login, data, ) Oil and Gas Disk and network I/O Pharma Science Unpredictable operations Social media / big data Compute-intensive workloads 2

3 Common Solution: multithreading Involves running code on separate threads Main GUI <<start Work>> interact with user Main thread Work Stmt1; Stmt2; Stmt3; Worker thread Operating System Rapidly switches CPU from one thread to the other, so both execute & make forward progress 3

4 Asian options financial modeling 4

5 Issue Long-running event handlers pose a problem If the current event handler takes a long time then remaining events wait in queue app feels unresponsive event event event event App Current event being processed 5

6 Asyncprogramming with threads Attempt #1 void button1_click( ) var result = DoLongRunningOp(); lstbox.items.add(result); } using System.Threading; Thread t = new Thread( () => // lambda expression: var result = DoLongRunningOp(); listbox.items.add(result); }); Boom! t.start(); 6

7 Attempt #2 UI thread owns the UI, so worker threads must delegate access void button1_click( ) var result = DoLongRunningOp(); lstbox.items.add(result); } Thread t = new Thread( () => // lambda expression: var result = DoLongRunningOp(); }); this.dispatcher.invoke( () => listbox.items.add(result); } ); t.start(); 7

8 async/ await Language-based solution in C#... Method *may* perform async, longrunning op void button1_click( ) var result = DoLongRunningOp(); lstbox.items.add(result); } async void button1_click( ) } var result = await Task.Run(() => DoLongRunningOp()); lstbox.items.add(result); using System.Threading.Tasks; Tells compiler "don't wait for this task to finish" set aside the code that follows so it happens later and just return. No blocking! 8

9 Threads are expensive to create (involves OS) easy to over-subscribe (i.e. create too many) an obfuscation (intent of code is harder to recognize) tedious to program Lambda expressions in C#/C++/Java make threading slightly better Exception handling is particularly difficult 9

10 Asian options financial modeling 10

11 Parallel.For Parallel programming to speed up the simulation? // for( ) Parallel.For(0, sims, (index) =>... }); 11

12 using System.Threading.Tasks; Parallel.For(0, N, (i) => for (int i = 0; i < N; i++)... } ); fork Sequential Parallel Structured ( Fork-Join ) Parallelism join Sequential 12

13 Task-based execution model Windows Process (.NET) Parallel.For( ); tasktasktasktask App Domain App Domain App Domain worker thread worker thread worker thread.net Thread Pool worker thread Task Parallel Library Task Scheduler Resource Manager global work queue Windows 13

14 Where's the other data race? double sum = 0.0; string sumlock = "sumlock"; // for( ) Parallel.For(0, sims, (index) =>? lock (sumlock) sum += callpayoff; } }); 14

15 Various solutions use synchronization (e.g. locking) least preferred use thread-safe entities (e.g. parallel collections) redesign to eliminate (e.g. reduction) most preferred 15

16 Reduction is a common parallel design pattern: Each task computes its own, local result no shared resource Merge ( reduce ) results at the end minimal locking Parallel.For(0, sims, () => return new TLS(); }, // init: create thread-local storage (index, control, tls) =>... tls.sum += callpayoff; return tls; }, // loop body: one simulation (tls) => // thread has finished: integrate partial result lock(sumlock) sum += tls.sum; } } ); 16

17 State of mainstream parallel programming Language Support for Parallelism Technologies C No use Pthreads or other library C++ (before 2011) No use Pthreads or other library C++14 Minimal Built-in support for threads, async Java Better Threads, Tasks, Fork/Join, Parallel data structures C# Better++ Threads, Tasks, Async, Parallel loops and data structures 17

18 Other options for parallel performance? libraries: MPI, TBB, Boost, Actors, PLINQ, TPL, PPL, Pthreads, Thrust, language extensions: OpenMP, TBB (Thread Building Blocks), AMP, OpenACC, parallel languages: CPU-based: Chapel, X10, High Performance Fortran, GPU-based: CUDA, OpenCL, 18

19 Mandelbrot with OpenMP 19

20 OpenMP sum = 0.0; for (int i=0; i < N; ++i) sum = sum + A[i]; sum = 0.0; #pragma omp parallel for reduction(+:sum) for (int i=0; i < N; ++i) sum = sum + A[i]; OpenMP == Open Multiprocessing (Multithreading) an open standard for platform-neutral multithreading very popular, with widespread support in most compilers (e.g. gcc 4.2) programmer directs parallelization via code annotations compiler implements 20

21 OpenMP supports: parallel regions and loops reductions load balancing critical sections... OpenMP version generates same lock-free reduction we did by hand void dot_product(int64 *z, int32 x[], int32 y[], int32 N) int64 sum = 0; #pragma omp parallel for reduction(+:sum) for (int32 i = 0; i < N; ++i) sum += (x[i] * y[i]); } *z = sum;

22 By default you get static scheduling iteration space is divided evenly before execution more efficient, but assumes uniform workload Mandelbrothas non-uniform distribution of work void Mandelbrot() #pragma omp parallel for for (int row=0; row < N; ++row)).. } }.

23 OpenMP also supports dynamic scheduling iteration space is divided into small pieces, assigned dynamically slightly more overhead, but handles non-uniform workloads void Mandelbrot() divide iteration space dynamically to load-balance #pragma omp parallel for schedule(dynamic) for (int row=0; row < N; ++row)).. } }.

24 Matrix Multiplication with parallel_for 24

25 #include <ppl.h> // // Naïve parallel solution using parallel_for: result is structured parallelism, with // static division of workload by row. // //for (int i = 0; i < N; i++) Concurrency::parallel_for(0, N, [&](int i) for (int j = 0; j < N; j++) C[i][j] = 0.0; x y } ); } for (int k = 0; k < N; k++) C[i][j] += (A[i][k] * B[k][j]); z

26 Very good! matrix multiplication is "embarrassingly parallel" linear speedup 2x on 2 cores, 4x on 4 cores, Version Cores Time (secs secs) Speedup Sequential 1 30 Parallel

27 What's the other half of the chip? cache! Are we using it effectively? we are not Memory cache

28 No one solves MM using the naïve algorithm horrible cache behavior X HW prefetchesdataassuming program will go Left -> Right or Right -> Left. Do this whenever possible

29 for (int i = 0; i < N; i++) for (int j = 0; j < N; j++) C[i][j] = 0.0; #pragma omp parallel for for (int i = 0; i < N; i++) for (int k = 0; k < N; k++) for (int j = 0; j < N; j++) C[i][j] += (A[i][k] * B[k][j]); Another factor of 2-10x improvement!

30 Block size based on cache closest to core Level 1 largest integer BS such that < 1 #pragma omp parallel for for (int jj=0; jj<n; jj+=bs) // for each column block: int jjend = Min(jj+BS, N); // initialize: for (int i=0; i<n; i++) for (int j=jj; j < jjend; j++) C[i][j] = 0.0; // block multiply: for (int kk=0; kk<n; kk+=bs) // for each row block: int kkend = Min(kk+BS, N); } } for (int i=0; i<n; i++) for (int k=kk; k < kkend; k++) for (int j=jj; j < jjend; j++) C[i][j] += (A[i][k] * B[k][j]);

31 Caching impacts all programs, sequential & parallel Version Cores Time (secs secs) Speedup Sequential Naive 1 30 Blocked OpenMP Naïve Blocked

32 Parallelism alone is not enough HPC == Parallelism + Memory Hierarchy Contention Expose parallelism Maximize data locality: network disk RAM cache core Minimize interaction: false sharing locking synchronization 32

33 Thank for attending / listening Presenter: Joe Hummel joe@joehummel.net Materials: 33