Recap. Practical Compiling for Modern Machines (Special Topics in Programming Languages)

Transcription

1 Recap Practical Compiling for Modern Machines (Special Topics in Programming Languages)

2 Why Compiling? Other reasons: Performance Performance Performance correctness checking language translation hardware synthesis instrumentation...

3 Parallelization

4 Stated Goals At the end of the course you will Know the basic architecture of modern practical compilers Know the fundamental algorithms and data structures used in implementing a compiler Know the basic correctness and performance issues for running programs on modern computers Have a first hand experience building a simple source-level parallelizing compiler through assignments over the course of the semester

5 30,000-foot View Fundamental architecture of practical compilers imperative model engineering trade-offs Algorithms and data structures SSA data flow analysis dependence analysis

6 Dependence Analysis Reordering transformations Data dependence iteration vectors control statements - - either if-conversion or control dependences Allen-Kennedy Section 7.3, Cytron et al. Section 7 Vectorization

7 Compiler Architecture: Front-end Scanning and parsing recursive descent bottom-up (LALR) Context-sensitive analysis attribute grammars ad-hoc syntax-directed translation Back-end handling procedure abstractions

8 Compiler Architecture: Back-end Handling procedure abstractions stack frame managing scoping - - linked stacked frames display table Handling object-oriented features field visibility virtual method invocation trampoline functions

9 Optimizations

10 Data flow Analysis Data flow analysis framework D = (L,, F) monotonicity existence of fixed point distributivity MOP solution Distributive Live variables, reaching definitions, available expressions Non-distributive constant propagation, type inference

11 Constant Propagation... {(A,2), (B,3)} A = 2 B = 3 A = 3 B = 2 {(A,3), (B,2)} C = A+B... {(A, ), (B, )}

12 SSA Equate names with values Eliminate output dependences Eliminate anti-dependences Efficient algorithm dominance frontier join points linear in the size of SSA graph

13 Size of Dominance Frontiers 140k oo loo 0 I I I I I 6~o ~~oo o Fig. 20. Size of dominance frontier mapping versus number of program statements.

14 Number of ϕ-functions ! 500 e 400 : 300 loo 0 I I I I I I --&-Too Fig, 21. Number of rj-functions versus number of program statements

15 Applications of SSA Constant propagation simple constant (SC) sparse simple constant (SSC) conditional constant (CC) sparse conditional constant (SCC) Global redundancy elimination partition-based scheme global value numbering with partitioning

16 Array SSA Modified semantics of ϕ-functions additional define kind element-wise merge for arrays Increased number of ϕ-functions offset by an optimization pass of forward substitution Merge implemented

17 Benefits of Array SSA Avoids expensive array copying Aids in breaking dependence cycles Applies standard compiler optimization techniques for optimizations defers until runtime, if must

18 Dependence-based Loop Transformations For vectorization loop distribution loop peeling loop interchange For coarse-grained parallelization loop interchange loop skewing loop fusion

19 HPF Dependence analysis Computation partitioning Communication analysis Communication placement Optimization communication aggregation communication-computation overlap

20 HPF with Multi-partitioning (NAS SP) Parallel Efficiency: Speedup/(Number of processors) hand coded multipartitioning dhpf multipartitioning dhpf 2D block PGI transpose Number of processors

21 HPF with Multi-partitioning (NAS BT) Parallel Efficiency: Speedup/(Number of processors) hand coded multipartitioning dhpf multipartitioning PGI transpose Number of processors

22 Co-Array Fortran Efficiency: Speedup/(Number of processors) MPI Itanium2+Quadrics CAF Itanium2+Quadrics CAF fix Itanium2+Quadrics CAF pa Itanium2+Quadrics CAF mb Itanium2+Quadrics CAF mb fix Itanium2+Quadrics Number of Processors

23 Chapel What is Chapel?! Chapel: Cascade High-Productivity Language! Goal!Simplify the creation of parallel programs!allow for experimental programming!support evolution from prototype to production!emphasize generality! Motivating Technologies!Multithreaded programming!locality-aware programming!object-oriented programming!generic programming

24 Chapel Multithreaded Programming! Global view of computation, data structures! Abstractions for data and task parallelism!data: domains, arrays, iterators, forall i in X!Task: cobegins, atomic transactions, syncs, cobegin { taska(); taskb(); }! Composition of parallelism Locality-aware Programming! Locale: machine unit of storage and processing! Programmer specifies number of locales at runtime prompt> mychapelprog nl=8! Built-in locale array const locales: [1..num_locales()] locale;! User-defined locale arrays var CompGrid: [1..GridRows, 1..GridCols] locale = ;! Domains (index sets) distribute across locales var D: domain(2) distributed(block(2), CompGrid) = ;! Computations on locales cobegin { on ALocs do taska(); on BLocs do taskb(); } forall i in D on B(i) do A(i) = B(i); Object-Oriented Programming Generic Programming! Objects help manage program complexity!encapsulate related data and code!facilitate reuse!separate interfaces from implementations! Chapel supports traditional and value classes!traditional: assign by reference, nominally typed!value: assign by value/name, structurally typed! OOP is not required (user s preference)! Advanced language features expressed using classes!user-defined distributions, reductions,! Type variables and parameters class Stack { type t; var buffsize: integer = 128; var data: [1..buffsize] t; function top(): t { }; }! Type query variables function copyn(data: [?D]?t; n: integer): [D] t { var newcopy: [D] t; forall i in 1..n do newcopy(i) = data(i); return newcopy; }! Elided types function inc(val): { var tmp = val; return tmp + 1; }! Chapel programs are statically-typed

25 Assignments Modern compiler tools building front-end program transformation Transformations for modern machines and languages parallelization for shared-memory vectorization - related to coarse-grained parallelism

26 What We Didn t See Inter-procedural optimization More loop transformations Parallelizing transformations software pipelining compiling for VLIW architectures Unified frameworks uniform techniques (Bill Pugh) unimodular transformations (Monica Lam)

27 Standard Compiler Applications Object code generation Automatic parallelization Pretty printing Source translation Fortran to C Automatic documentation generation Document typesetting Instrumentation

28 Other Applications VLSI design hardware synthesis Program verification Detecting security loopholes Automatic Differentiation

29 Automatic Differentiation... v2 = 2sin(v1) + 5 if (v2>0.0) then v4 = v2 + p1v3/v2 p1 = p endif call sub1(v2,v4,p1) dv2 = 2cos(v1)dv1 v2 = 2sin(v1) + 5 if (v2>0.0) then dv4 = dv2(1-p1v3/(v2v2)) + dv3p1/v2 v4 = v2 + p1v3/v2 p1 = p endif call sub1_d(v2,dv2,v4,dv4,p1)...

30 Future Analysis SSA data flow analysis dependence analysis type inference Transformations code specialization parallelization and vectorization library optimizations

31 Future High-level languages productivity complex execution environments more, and more demanding, applications Dynamic compilation time-bound compilation Evolving compiler self-evaluation learning from past experience seeking help from experts

32 Every worthwhile accomplishment, big or little, has its stages of drudgery and triumph; a beginning, a struggle and a victory. Mahatma Gandhi