9/5/15. n Class objectives, goals. n Why Fine grain parallelism? n Equational Programming (intro) Sanjay Rajopadhye Colorado State University

Transcription

1 Sanjay Rajopadhye Colorado State University n Class objectives, goals n Why Fine grain parallelism? n Equational Programming (intro) 2 1

2 n Every problem is underspecified n Questions are ill posed n Finding the right question is half the work n Communicate: write, write, write n Question everything 3 n Introductions: n Name n Year in school (e.g., MS in ECE, 2 nd year) n Parallelism experience (SW/HW) n Interesting fact 4 2

3 n Course website: n Very little on RamCT/Canvas (only for grades) n Course related cs560@cs.colostate.edu n How to succeed n Check the schedule tab/page every day frequent updates n Spend about 1-2 hrs every day outside class n General rule 4 credits = 8-12 hrs outside 5 n Short term Become macho paralle programmer: write heroically tuned codes. n Medium term Do it systematically: tuning for accelerator of the day e.g., Kepler k40 vs Xeon Phi: learn principles, not skills n Long term Do it automatically: Learn the foundations of automatic compilation. Focus on a regular subset of programs n Polyhedral Equational Model 6 3

4 n Big picture n Polyhedral Equations as programs: I m loath to write C n Equations vs (conventional) loop programs n Equations-to-code (compiling equations) n Schedule n (processor) allocation n (memory) allocation n But what about parallelism? 7 8 assignments (basic + advanced) + term project n Parallel program performance (2) n Mathematical foundations: polyhedra, affine functions, and operations (2) n Analysis: scheduling & allocation (2) n Equational programming: Alpha/AlphaZ (1) n Alpha analysis/transformation (1) n Code generation/tiling (2) 8 4

5 n Assignments (30%) n Midterms in class+take home (15+20=35%) n Final project (30% = ) n Proposal n Advancement report n Final report n Quality of work n Final poster n Participation/Discussion/Quizzes (5%) 9 n What are polyhedra? n Why are they useful/important n What is the polyhedral model? 10 5

6 n What is a model? n A mathematical/computational/mechanical/ abstraction of some other (physical) entity n Objects in the model must emulate the natural operations of the modeled entities semantics 11 From Feautrier s keynote at LCPC 2009 Introduction Prehistory State of the Art What Next? Dependences Karp, Miller, Winograd Irigoin, PF 1988, Pugh, Systolic Array Design Scheduling Quinton, Robert,, 1989 Quinton, Rajopadhye, Fortes,, PF Rajopadhye, 1987 Rau Placement PF, Pingali, 1994 H. T. Kung, 1978 Cousot, Halbwachs 1977 The Polytope Model Pugh, 1991 LC Lu, 1991 Code Generation Irigoin, Lengauer, Rajopadhye Bastoul, PF, Boulet, Tiling Irigoin, JL Xue,, 1988 Bernstein 1966 L. Lamport, 1974 Automatic Parallelization Dependence tests, Banerjee, 1976 Kuck Allen, Kennedy, 1987 Lam Irigoin Array Shrinking PF, Rajopadhye, Darte, 2005 Locality Wolfe + Lam, 1991 Bastoul, 2003 HLS Quinton, Risset, /

7 n Physical entity: programs/computations n The Polyhedral Model is a very high level intermediate representation (IR) of regular computations n Polyhedral equational model: real=abstract n Amenable to: n Mathematical static analysis n Transformation within model: closure n Transformation outside model: (tiled) code generation 13 n Class objectives, goals n Why Fine grain parallelism? n Equational Programming (intro) 14 7

8 n Parallel Programming is hard n End of the free lunch [Sut05] n Arrival of manycores signals the end of La-Z-Boy Programming [Pat06] Becoming a parallel programming expert will get you a good job But your skills may become obsolete new machines, new languages, Parallelism must return to La-Z-Boy programming [Sut05] Herb Sutter. The Free Lunch Is Over: A Fundamental Turn Toward Concurrency, in Software. Dr. Dobb's Journal, vol. 30, no. 3, [Pat06] David Patterson, in keynote talk at the International Workshop on Languages and Compilers For Parallel Computers LCPC 2006, New Orleans, LA. 15 n Moore s law of density still lives on n How to use the Si resources in the face of constraints n Dark silicon exponentially increasing fraction of the chip cannot be turned on n (One) proposed solution: accelerators n Specialized (domain specific) circuits n Energy efficiently do the computation n Powered off otherwise 16 8

9 n Class objectives, goals n Why Fine grain parallelism? n Equational Programming (intro) 17 n Many resources on the web (NVIDIA webinars) n Coalescing (HW1a) n Challenge question: Achieve maximum bandwidth, with fewest threads-per-block n For a strided-by-block access pattern. n Arithmetic peak: warps and virtualization n Bank conflicts in shared memory 18 9

10 n Oxford CUDA conf (CUDA webinar online) n Identifying Performance Limiters, Micikevicius NVIDIA/UCF (CUDA webinar) n Roofline for Fast Math Sam Williams, LBL 19 n Wiki page for Pascal s Triangle n and also a non-standard way to compute Fibonacci numbers 20 10