Iterative Optimization in the Polyhedral Model: Part I, One-Dimensional Time

Transcription

1 Iterative Optimization in the Polyhedral Model: Part I, One-Dimensional Time Louis-Noël Pouchet, Cédric Bastoul, Albert Cohen and Nicolas Vasilache ALCHEMY, INRIA Futurs / University of Paris-Sud XI March 12, 2007 Fifth International Symposium on Code Generation and Optimization San Jose, California

2 Outline: Outline Context of this study: Focus on Loop Nest Optimization for regular loops Automatic method for parallelism extraction / loop transformation Combine iterative methods with the power of the polyhedral model Solution independent of the compiler and the target machine Our contribution: Search space construction 1 point in the space 1 distinct legal program version suitable for various exploration methods Performance 99% of the best speedup attained within 20 runs of a dedicated heuristic wall clock optimal transformation discoverable on small kernels 2

3 Scheduling in the Polyhedral Model: A Motivating Example One-Dimensional Scheduling Original Schedule for (i=0; i<n; ++i) {. S1(i);. for (j=0; j<n; ++j).. S2(i,j); } { θ S1 = i θ S2 = i for (i=0; i<n; ++i) {. S1(i);. for (j=0; j<n; ++j).. S2(i,j); } Specify the outer-most loop only Initial outer-most loop is i 3

4 Scheduling in the Polyhedral Model: A Motivating Example One-Dimensional Scheduling Distribute loops for (i=0; i<n; ++i) {. S1(i);. for (j=0; j<n; ++j).. S2(i,j); } { θ S1 = i θ S2 = i+n for (i=0; i<n; ++i). S1(i); for (i=n; i<2*n; ++i). for (j=0; j<n; ++j).. S2(i-n,j); Specify the outer-most loop only All instances of S1 are executed before the first S2 instance 3

5 Scheduling in the Polyhedral Model: A Motivating Example One-Dimensional Scheduling Distribute loops + Interchange loops for S2 for (i=0; i<n; ++i) {. S1(i);. for (j=0; j<n; ++j).. S2(i,j); } { θ S1 = i θ S2 = j + n for (i=0; i<n; ++i). S1(i); for (j=n; j<2*n; ++j). for (i=0; i<n; ++i).. S2(i,j-n); Specify the outer-most loop only The outer-most loop for S2 becomes j 3

6 Scheduling in the Polyhedral Model: A Motivating Example One-Dimensional Scheduling Distribute loops + Interchange loops for S2 for (i=0; i<n; ++i) {. S1(i);. for (j=0; j<n; ++j).. S2(i,j); } { θ S1 = i θ S2 = j + n for (i=0; i<n; ++i). S1(i); for (j=n; j<2*n; ++j). for (i=0; i<n; ++i).. S2(i,j-n); Transformation reversal skewing interchange peeling shifting fusion distribution Description Changes the direction in which a loop traverses its iteration range Makes the bounds of a given loop depend on an outer loop counter Exchanges two loops in a perfectly nested loop, a.k.a. permutation Extracts one iteration of a given loop Allows to reorder loops Fuses two loops, a.k.a. jamming Splits a single loop nest into many, a.k.a. fission or splitting 3

7 Scheduling in the Polyhedral Model: A Motivating Example One-Dimensional Scheduling for (i=0; i<n; ++i) {. S1(i);. for (j=0; j<n; ++j).. S2(i,j); } A schedule is an affine function of the iteration vector and the parameters θ S1 ( x S1 ) = t 1S1.i S1 + t 2S1.n + t 3S1.1 θ S2 ( x S2 ) = t 1S2.i S2 + t 2S2.j S2 + t 3S2.n + t 4S2.1 4

8 Scheduling in the Polyhedral Model: A Motivating Example One-Dimensional Scheduling for (i=0; i<n; ++i) {. s[i] = 0;. for (j=0; j<n; ++j).. s[i] = s[i]+a[i][j]*x[j]; } A schedule is an affine function of the iteration vector and the parameters θ S1 ( x S1 ) = t 1S1.i S1 + t 2S1.n + t 3S1.1 θ S2 ( x S2 ) = t 1S2.i S2 + t 2S2.j S2 + t 3S2.n + t 4S2.1 For 1 t 1, there are 3 7 = 2187 possible schedules 4

9 Scheduling in the Polyhedral Model: A Motivating Example One-Dimensional Scheduling for (i=0; i<n; ++i) {. s[i] = 0;. for (j=0; j<n; ++j).. s[i] = s[i]+a[i][j]*x[j]; } A schedule is an affine function of the iteration vector and the parameters θ S1 ( x S1 ) = t 1S1.i S1 + t 2S1.n + t 3S1.1 θ S2 ( x S2 ) = t 1S2.i S2 + t 2S2.j S2 + t 3S2.n + t 4S2.1 For 1 t 1, there are 3 7 = 2187 possible schedules But only 129 legal distinct schedules 4

10 Scheduling in the Polyhedral Model: Overview Our Objective 1 Search space construction Efficiently construct a space of all legal, distinct affine schedules 5

11 Scheduling in the Polyhedral Model: Overview Our Objective 1 Search space construction Efficiently construct a space of all legal, distinct affine schedules matmult locality fir h264 crout i-bounds 1,1 1,1 0,1 1,1 3,3 c-bounds 1,1 1,1 0,3 0,4 3,3 #Sched

12 Scheduling in the Polyhedral Model: Overview Our Objective 1 Search space construction Efficiently construct a space of all legal, distinct affine schedules matmult locality fir h264 crout i-bounds 1,1 1,1 0,1 1,1 3,3 c-bounds 1,1 1,1 0,3 0,4 3,3 #Sched #Legal

13 Scheduling in the Polyhedral Model: Overview Our Objective 1 Search space construction Efficiently construct a space of all legal, distinct affine schedules matmult locality fir h264 crout i-bounds 1,1 1,1 0,1 1,1 3,3 c-bounds 1,1 1,1 0,3 0,4 3,3 #Sched #Legal Rely on the polyhedral model and Integer Linear Programming to guarantee completeness and correctness of the space properties 5

14 Scheduling in the Polyhedral Model: Overview Our Objective 1 Search space construction Efficiently construct a space of all legal, distinct affine schedules matmult locality fir h264 crout i-bounds 1,1 1,1 0,1 1,1 3,3 c-bounds 1,1 1,1 0,3 0,4 3,3 #Sched #Legal Rely on the polyhedral model and Integer Linear Programming to guarantee completeness and correctness of the space properties Search space will emcoumpass unique, distinct compositions of reversal, skewing, interchange, fusion, peeling, shifting, distribution 5

15 Scheduling in the Polyhedral Model: Overview Our Objective 1 Search space construction Efficiently construct a space of all legal, distinct affine schedules matmult locality fir h264 crout i-bounds 1,1 1,1 0,1 1,1 3,3 c-bounds 1,1 1,1 0,3 0,4 3,3 #Sched #Legal Rely on the polyhedral model and Integer Linear Programming to guarantee completeness and correctness of the space properties Search space will emcoumpass unique, distinct compositions of reversal, skewing, interchange, fusion, peeling, shifting, distribution 2 Search space exploration Perform exhaustive scan to discover wall clock optimal schedule, and evidences of intricacy of the best transformation Build an efficient heuristic to accelerate the space traversal 5

16 Search Space Construction: Preliminaries Polyhedral Representation of Programs Static Control Parts Loops have affine control only 6

17 Search Space Construction: Preliminaries Polyhedral Representation of Programs Static Control Parts Loops have affine control only Iteration domain: represented as integer polyhedra for (i=1; i<=n; ++i). for (j=1; j<=n; ++j).. if (i<=n-j+2)... s[i] =... D S1 = i j n 1 0 6

18 Search Space Construction: Preliminaries Polyhedral Representation of Programs Static Control Parts Loops have affine control only Iteration domain: represented as integer polyhedra Memory accesses: static references, represented as affine functions of x S and p for (i=0; i<n; ++i) {. s[i] = 0;. for (j=0; j<n; ++j).. s[i] = s[i]+a[i][j]*x[j]; } f s ( x S2 ) = [ ]. [ f a ( x S2 ) = ]. f x ( x S2 ) = [ ]. x S2 n 1 x S2 n 1 x S2 n 1 6

19 Search Space Construction: Preliminaries Polyhedral Representation of Programs Static Control Parts Loops have affine control only Iteration domain: represented as integer polyhedra Memory accesses: static references, represented as affine functions of x S and p Data dependence between S1 and S2: a subset of the Cartesian product of D S1 and D S2 (exact analysis) for (i=1; i<=3; ++i) {. s[i] = 0;. for (j=1; j<=3; ++j).. s[i] = s[i] + 1; } D S1δS2 : i S1. i S2 j S2 1 0 = 0 S1 iterations S2 iterations i 6

20 Search Space Construction: Preliminaries Polyhedral Representation of Programs Static Control Parts Loops have affine control only Iteration domain: represented as integer polyhedra Memory accesses: static references, represented as affine functions of x S and p Data dependence between S1 and S2: a subset of the Cartesian product of D S1 and D S2 (exact analysis) Reduced dependence graph labeled by dependence polyhedra 6

21 Search Space Construction: Way to Go Space Construction Affine Schedules Legal Distinct Schedules 7

22 Search Space Construction: Way to Go Space Construction Affine Schedules Legal Distinct Schedules - Causality condition Property (Causality condition for schedules) Given RδS, θ R and θ S are legal iff for each pair of instances in dependence: θ R ( x R ) < θ S ( x S ) Equivalently: R,S = θ S ( x S ) θ R ( x R ) 1 0 7

23 Search Space Construction: Way to Go Space Construction Affine Schedules Legal Distinct Schedules - Causality condition - Farkas Lemma Lemma (Affine form of Farkas lemma) Let D be a nonempty polyhedron defined by A x + b 0. Then any affine function f ( x) is non-negative everywhere in D iff it is a positive affine combination: f ( x) = λ 0 + λ T (A x + b), with λ 0 0 and λ 0. λ 0 and λ T are called the Farkas multipliers. 7

24 Search Space Construction: Way to Go Space Construction Affine Schedules Valid Farkas Multipliers Legal Distinct Schedules - Causality condition - Farkas Lemma 7

25 Search Space Construction: Way to Go Space Construction Affine Schedules Valid Farkas Multipliers Many to one Legal Distinct Schedules - Causality condition - Farkas Lemma 7

26 Search Space Construction: Way to Go Space Construction Affine Schedules Valid Farkas Multipliers Legal Distinct Schedules - Causality condition - Farkas Lemma - Identification ( ( ) ) θ S ( x S ) θ R ( x R ) 1 = λ 0 + λ T xr D R,S + d R,S 0 x S D RδS i R : λ D1,1 λ D1,2 + λ D1,7 i S : λ D1,3 λ D1,4 λ D1,7 j S : λ D1,5 λ D1,6 n : λ D1,2 + λ D1,4 + λ D1,6 1 : λ D1,0 7

27 Search Space Construction: Way to Go Space Construction Affine Schedules Valid Farkas Multipliers Legal Distinct Schedules - Causality condition - Farkas Lemma - Identification ( ( ) ) θ S ( x S ) θ R ( x R ) 1 = λ 0 + λ T xr D R,S + d R,S 0 x S D RδS i R : t 1R = λ D1,1 λ D1,2 + λ D1,7 i S : t 1S = λ D1,3 λ D1,4 λ D1,7 j S : t 2S = λ D1,5 λ D1,6 n : t 3S t 2R = λ D1,2 + λ D1,4 + λ D1,6 1 : t 4S t 3R 1 = λ D1,0 7

28 Search Space Construction: Way to Go Space Construction Affine Schedules Valid Farkas Multipliers Legal Distinct Schedules - Causality condition - Farkas Lemma - Identification - Projection Solve the constraint system Use (optimized) Fourier-Motzkin projection algorithm Reduce redundancy Detect implicit equalities 7

29 Search Space Construction: Way to Go Space Construction Affine Schedules Valid Farkas Multipliers Valid Transformation Coefficients Legal Distinct Schedules - Causality condition - Farkas Lemma - Identification - Projection 7

30 Search Space Construction: Way to Go Space Construction Affine Schedules Valid Farkas Multipliers Valid Transformation Coefficients Bijection Legal Distinct Schedules - Causality condition - Farkas Lemma - Identification - Projection One point in the space one set of legal schedules w.r.t. the dependence 7

31 Search Space Construction: Way to Go Overview Algorithm Add constraints obtained for each dependence Bound the space Search space: set of linear constraints on the schedule coefficients (i.e. Z-polytope) To each integral point in the space corresponds a distinct program version where the semantics is preserved Benchmark i-bounds #Sched #Legal Time matmult 1, locality 1, fir 0, h264 1, crout 3,

32 Search Space Exploration: Framework for Iterative Optimization Workflow SCoP representation Polyhedral computing libraries Code generation PIPLib PolyLib CLooG Iterative compilation and run of base source code with transformed SCoP C compilable code Source Code Static Analysis Space Construction Space Exploration Kernel Generation Unit Generation Compilation Run Target Code Polyhedral representation of SCoP Bounded search space Feedback from hardware counter(s) CLooG: PiPLib: PolyLib: 9

33 Search Space Exploration: Exhaustive Scan Performance Distribution [1/2] 2e+09 matmult 4e+09 locality 1.8e e e+09 3e+09 Cycles 1.4e e+09 Cycles 2.5e+09 2e+09 1e+09 original 1.5e+09 8e+08 6e Transformation identifier 1e+09 original 5e Transformation identifier Figure: Performance distribution for matmult and locality 10

34 Search Space Exploration: Exhaustive Scan Performance Distribution [2/2] 1.42e+09 crout 1.34e+09 crout 1.4e e e e e e+09 Cycles 1.34e e+09 Cycles 1.3e e e e+09 original 1.28e e+09 original 1.26e Transformation identifier 1.26e Transformation identifier (a) GCC -O3 (b) ICC -fast Figure: The effect of the compiler 11

35 Search Space Exploration: Exhaustive Scan Performance Comparison Figure: Best Version vs Original 12

36 Search Space Exploration: Heuristic Scan Heuristic Scan Propose a decoupling heuristic: The general form of the schedule is embedded in the iterator coefficients Decouple the schedule: θ S ( x S ) = ( ı p c) x S n 1 13

37 Search Space Exploration: Heuristic Scan Heuristic Scan Propose a decoupling heuristic: The general form of the schedule is embedded in the iterator coefficients Decouple the schedule: θ S ( x S ) = ( ı p c) x S n 1 Parameters and constant coefficients can be seen as a refinement 13

38 Search Space Exploration: Heuristic Scan Heuristic Scan Propose a decoupling heuristic: The general form of the schedule is embedded in the iterator coefficients Decouple the schedule: θ S ( x S ) = ( ı p c) x S n 1 Parameters and constant coefficients can be seen as a refinement Adressing scalability to larger SCoPs: 1 impose a static or dynamic limit to the number of runs (limit to the ı part) 2 replace an exhaustive enumeration of the ı combinations by a limited set of random draws in the ı space. 13

39 Search Space Exploration: Heuristic Scan Results locality matmult mvt Maximum speedup achieved (in %) Decoupling Maximum speedup achieved (in %) Decoupling Maximum speedup achieved (in %) Decoupling 40 Random Random Random Runs Runs Runs Figure: Comparison between random and decoupling heuristics locality matmult matvecttransp 4e+09 2e e e e e+09 Cycles 3e e+09 2e e+09 1e+09 original Cycles 1.6e e e+09 1e+09 8e+08 original Cycles (M) 1.1e+09 1e+09 9e+08 8e+08 7e+08 6e+08 5e+08 Original 5e e e Transformation identifier Transformation identifier Transfo. ID 14

40 Conclusion: Conclusion Optimizing and / or Enabling transformation framework on top of the compiler Encouraging speedups, fast heuristic convergence On small kernels, optimal transformation can be discovered 15

41 Conclusion: Conclusion Optimizing and / or Enabling transformation framework on top of the compiler Encouraging speedups, fast heuristic convergence On small kernels, optimal transformation can be discovered 15

42 Conclusion: Conclusion Optimizing and / or Enabling transformation framework on top of the compiler Encouraging speedups, fast heuristic convergence On small kernels, optimal transformation can be discovered 15

43 Conclusion: Conclusion Optimizing and / or Enabling transformation framework on top of the compiler Encouraging speedups, fast heuristic convergence On small kernels, optimal transformation can be discovered Ongoing and future work: Couple with state-of-the-art feedback-directed iterative methods Part II: multidimensional schedules Integrate into GCC GRAPHITE branch 15

44 Conclusion: Conclusion Optimizing and / or Enabling transformation framework on top of the compiler Encouraging speedups, fast heuristic convergence On small kernels, optimal transformation can be discovered Ongoing and future work: Couple with state-of-the-art feedback-directed iterative methods Part II: multidimensional schedules Integrate into GCC GRAPHITE branch 15

45 Conclusion: Conclusion Optimizing and / or Enabling transformation framework on top of the compiler Encouraging speedups, fast heuristic convergence On small kernels, optimal transformation can be discovered Ongoing and future work: Couple with state-of-the-art feedback-directed iterative methods Part II: multidimensional schedules Integrate into GCC GRAPHITE branch 15

46 Questions: 16

47 Questions: A Transformation Example Intricacy of the Transformed Code Optimal Transformation for locality, GCC 4 -O3, P4 Xeon S1: B[j] = A[j] S2: C[j] = A[j + N] for (i=0;i<=m;i++) { for (j=0;j<=m;j++) { S1(i,j); S2(i,j); } } for (c1=-n;c1<=min(-2,m-n);c1++) for (j=0;j<=m;j++) S1(c1+N,j); for (c1=-1;c1<=m-n;c1++) { for (j=0;j<=m;j++) S2(c1+1,j); for (j=0;j<=m;j++) S1(c1+N,j); } for (c1=max(m-n+1,-1);c1<=m-1;c1++) for (j=0;j<=m;j++) S2(c1+1,j); 19.4% speedup, without vectorization 17