Estimation of Parallel Complexity with Rewriting Techniques

Transcription

1 Estmaton of Parallel Complexty wth Rewrtng Technques Chrstophe Alas, Carsten Fuhs, Laure Gonnord INRIA & LIP (UMR CNRS/ENS Lyon/UCB Lyon1/INRIA), Lyon, France, Brkbeck, Unversty of London, Unted Kngdom, Unversty of Lyon & LIP (UMR CNRS/ENS Lyon/UCB Lyon1/INRIA), Lyon, France, 15th Internatonal Workshop on Termnaton September 5, 2016, Obergurgl, Austra 1 / 15

2 Context: HPC's Automatc Parallelzaton Why Automatc Parallelzaton? Most computers are parallel (end of Dennard scalng...) Wrtng/debuggng a parallel program s (horrbly) dcult Automaton s requred... 2 / 15

3 Context: HPC's Automatc Parallelzaton Why Automatc Parallelzaton? Most computers are parallel (end of Dennard scalng...) Wrtng/debuggng a parallel program s (horrbly) dcult Automaton s requred... Challenges: How to represent the computaton? data dependences How much parallelsm? data dependences Whch parallelsm (schedulng)? data dependences Whch resource allocaton? data dependences 2 / 15

4 Context: HPC's Automatc Parallelzaton Why Automatc Parallelzaton? Most computers are parallel (end of Dennard scalng...) Wrtng/debuggng a parallel program s (horrbly) dcult Automaton s requred... Challenges: How to represent the computaton? data dependences How much parallelsm? data dependences Whch parallelsm (schedulng)? data dependences Whch resource allocaton? data dependences Bad news: checkng data dependences s undecdable. 2 / 15

5 Related Work and Contrbutons Focus on regular mperatve programs (polyhedral model) Unfyng framework for program parallelzaton Exact set of dependences, all the parallelsm s found Scalablty ssues 3 / 15

6 Related Work and Contrbutons Focus on regular mperatve programs (polyhedral model) Unfyng framework for program parallelzaton Exact set of dependences, all the parallelsm s found Scalablty ssues Over-approxmate the data dependences Scalable Can be (very) rough and mss most of the parallelsm 3 / 15

7 Related Work and Contrbutons Focus on regular mperatve programs (polyhedral model) Unfyng framework for program parallelzaton Exact set of dependences, all the parallelsm s found Scalablty ssues Over-approxmate the data dependences Scalable Can be (very) rough and mss most of the parallelsm Contrbutons: Assess the parallel complexty of (some) recursve programs... usng monotone nterpretatons Extends of the polyhedral model to recursve programs 3 / 15

8 Parallel Complexty //Compute y = Ax for := 0 to N-1 : y[] := 0; for j := 0 to N-1 : y[] := y[] + a[][j]*x[j]; λn = O(N) j Mnmum number of (parallel) computaton steps assumng unbounded parallel resources. Solved on regular programs (polyhedral model) Goal: recursve programs on trees! 4 / 15

9 Parallel Complexty Methodology for := 0 to N-1 : y[] := 0; for j := 0 to N-1 : y[] := y[] + a[][j]*x[j]; λn = O(N) j 1 Dvde an executon e nto a sequence of operatons O e How to represent/approxmate e? whch gran? 5 / 15

10 Parallel Complexty Methodology for := 0 to N-1 : y[] := 0; for j := 0 to N-1 : y[] := y[] + a[][j]*x[j]; λn = O(N) j 1 Dvde an executon e nto a sequence of operatons O e How to represent/approxmate e? whch gran? 2 Compute the dependences: e O e O e Impact of approxmaton? 5 / 15

11 Parallel Complexty Methodology for := 0 to N-1 : y[] := 0; for j := 0 to N-1 : y[] := y[] + a[][j]*x[j]; λn = O(N) j 1 Dvde an executon e nto a sequence of operatons O e How to represent/approxmate e? whch gran? 2 Compute the dependences: e O e O e Impact of approxmaton? 3 Compute the parallel complexty: λ e := heght( e ) How to express λ e? 5 / 15

12 Outlne 1 Parallel complexty of regular programs 2 Parallel complexty of recursve programs 6 / 15

13 Polyhedral Model at a Glance for := 0 to 2*N : c[] := 0; for := 0 to N for j := 0 to N : c[+j] := c[+j] + a[]*b[j]; j Automatc parallelzaton of regular loop nests wth arrays 7 / 15

14 Polyhedral Model at a Glance for := 0 to 2*N : c[] := 0; for := 0 to N for j := 0 to N : c[+j] := c[+j] + a[]*b[j]; j Automatc parallelzaton of regular loop nests wth arrays e s decdable and can be analyzed (e.g. wth ILP), : 0, 2N,, j : (, j) 0, N 2 Key analyss: array dependences, ane schedulng 7 / 15

15 (Ane) Array Dependences for := 0 to 2*N : c[] := 0; for := 0 to N for j := 0 to N : c[+j] := c[+j] + a[]*b[j]; j Idea: Gven a consumer, nd the last producer ILP. 8 / 15

16 (Ane) Array Dependences for := 0 to 2*N : c[] := 0; for := 0 to N for j := 0 to N : c[+j] := c[+j] + a[]*b[j]; j Idea: Gven a consumer, nd the last producer ILP. N s an ane relaton:, 1, j + 1 N,, j : > 0 j < N, N, 0, : 0 N, N, N, N : N < 2N 8 / 15

17 (Ane) Array Dependences N < 2N : () ( N, N) 0 N : () (0, ) j 0, j N, > 0, j < N : ( 1, j + 1) (, j) Idea: Gven a consumer, nd the last producer ILP. N s an ane relaton:, 1, j + 1 N,, j : > 0 j < N, N, 0, : 0 N, N, N, N : N < 2N 9 / 15

18 (Ane) Schedulng N < 2N : () ( N, N) θ l1 () = (0) 0 N : () (0, ) j θ l2 (, j) = (1, ) 0, j N, > 0, j < N : ( 1, j + 1) (, j) Assgn each operaton l, x wth a tmestamp θ l ( x) N d l. Correctness: l, x N l, y θ l ( x) θ l ( y) Ane schedule: θ l ( x) = A x + b ILP. 10 / 15

19 (Ane) Schedulng N < 2N : () ( N, N) θ l1 () = (0) 0 N : () (0, ) j θ l2 (, j) = (1, ) 0, j N, > 0, j < N : ( 1, j + 1) (, j) Assgn each operaton l, x wth a tmestamp θ l ( x) N d l. Correctness: l, x N l, y θ l ( x) θ l ( y) Ane schedule: θ l ( x) = A x + b ILP. Bonus: reverse the order: termnaton algorthm! [RanK, 2010] 10 / 15

20 Cross Fertlzaton HPC communty Data Dependence Graph Schedule Latency Recursve schedule TCS communty Integer transton system Rankng functon Computatonal complexty Monotonc nterpretatons 11 / 15

21 Target: recursve programs on trees publc nt treemax () { nt leftmax = Integer. MIN_VALUE ; nt rghtmax = Integer. MIN_VALUE ; f ( ths. left!= null ) leftmax = ths. left. treemax () ; f ( ths. rght!= null ) rghtmax = ths. rght. treemax () ; return Math. max ( ths. val, Math. max ( leftmax, rghtmax ) ) ; } Each node (subtree) of t s an operaton of e. e can be encoded as a term rewrte system (TRS): dep(tree(val, left, rght)) dep(tree(val, left, rght)) dep(left) dep(rght) How to schedule (check the termnaton of) a TRS? Wth monotone nterpretatons! [AProVE, KoAT] 12 / 15

22 Puttng t all together publc nt treemax () { nt leftmax = Integer. MIN_VALUE ; nt rghtmax = Integer. MIN_VALUE ; f ( ths. left!= null ) leftmax = ths. left. treemax () ; f ( ths. rght!= null ) rghtmax = ths. rght. treemax () ; return Math. max ( ths. val, Math. max ( leftmax, rghtmax ) ) ; } 2 4,3 3 1,1 4 2,2 7 1,1 Monotone nterpretaton [dep](x 1 ) = x 1 [Tree](x 1, x 2, x 3 ) = x 2 + x [dep](x 1 ) = x 1 [Tree](x 1, x 2, x 3 ) = max(x 2, x 3 ) + 1 Parallel complexty λ t = O(heght(t)) λ t = O(heght(t)) 13 / 15

23 What happens on regular programs? for ( =0; <= N ; ++) for ( j =0; j <= N ; j ++) // Block S { m1 [ ][j ] = Integer. MIN_VALUE ; for ( k =1; k <= ; k ++) m1 [ ][ j ] = max ( m1 [ ][ j ], H [ - k ][ j ] + W [ k ]) ; j } m2 [ ][j ] = Integer. MIN_VALUE ; for ( k =1; k <= j ; k ++) m2 [ ][ j ] = max ( m2 [ ][ j ], H [ ][ j - k ] + W [ k ]) ; H [ ][j ] = max (0,H( -1,j -1)+ s (a[],b[ ]), m1 [ ][ j ], m2 [ ][ j ]) ; dep(, j) dep(, j) dep(, j) dep( 1, j 1) : 0 n, 0 j n dep( k, j) : 0 n, 0 j n, 1 k dep(, j l) : 0 n, 0 j n, 1 l j Result: [dep](x 1, x 2 ) = x 1 + x 2 Same as n the polyhedral model! λ n 2n 14 / 15

24 Concluson Poston: Automatc parallelzaton can take prot of monotonc nterpretatons. Extenson of ane schedulng to recursve programs Locks: How to dene/nd the best schedule? How to count the steps? Steps towards a parallelzng compler: Computaton parttonng? Generaton of the parallel code gven a schedule? 15 / 15