Par4All. From Convex Array Regions to Heterogeneous Computing

Transcription

1 Par4All From Covex Array Regios to Heterogeeous Computig Mehdi Amii, Béatrice Creusillet, Stéphaie Eve, Roa Keryell, Oig Goubier, Serge Guelto, Jaice Oaia McMaho, Fraçois Xavier Pasquier, Grégoire Péa, Pierre Villalo IMPACT d Iteratioal Workshop o Polyhedral Compilatio Techiques 1/21

2 Par4All project: automatic source-to-source parallelizatio for heterogeeous targets HPC Project eeds tools for its hardware accelerators (Wild Nodes from Wild Systems) ad to parallelize, port & optimize customer applicatios Ureasoable to begi yet aother ew compiler project May academic Ope Source projects are available......but customers eed products Itegrate your ideas ad developmets i existig project...or buy oe if you ca afford (ST with PGI...) Not reivetig the wheel (o NIH sydrome) => Fudig a iitiative to idustrialize Ope Source tools Par4All is fully Ope-Source (mix of MIT/GPL licese) Accordig to Keshav Pigali, we're wrog at raisig automatic parallelizatio from low-level code. But we provide geerality across differet tools, each with its ow high-abstractio. (ad: versio released *today*, check it out!) 2/21

3 Par4All overview PIPS is the first project to eter the Par4All iitiative Preseted at Impact 2011: PIPS Is ot (just) Polyhedral Software Source code (with directives?) PIPS Trasformatios && Aalyses kerels Post-processor host code Par4all Rutime vcclike host compiler Fial Biary Par4All Pytho Driver 3/21

4 Demo Example: madelbrot writte i Scilab Coverted to C usig COLD, a i-house (commercial) scilab-to-c compiler The C code is processed by Par4All to target multi-core or GPU PIPS is iter-procedural ad thus eeds all the source code, we eed to provide stubs for the Scilab rutime

5 Focus o array regios aalyses Startig with Béatrice Creusillet thesis (1996) Fid out what part of a array is read or writte Approximatio: may/must/exact Set of liear relatios Applicatios: Parallelizatio Array privatizatio Scalarizatio Statemet isolatio Memory footprit reductio usig tilig 5/21

6 Focus o array regios aalyses // <a[phi1][phi2]-w-may-{0<=phi1, PHI1<=PHI2, PHI1+PHI2+1<=m, // 2PHI1+2<=> it triagular(it m, it, double a[][m]) { it h = /2; // <a[phi1][phi2]-w-exact-{0<=phi1, // PHI1<=PHI2, PHI1+PHI2+1<=m, // PHI1+1<=h, <=2h+1, 2h<=> for(it i = 0; i < h; i += 1) { M // <a[phi1][phi2]-w-exact-{phi1==i, i<=phi2, // PHI2+i+1<=m, 0<=i, // i+1<=h, <=2h+1, 2h<=> for(it j = i; j < m-i; j += 1) { // <a[phi1][phi2]-w-exact-{phi1==i, PHI2==j, // i<=j, j+i+1<=m, 0<=i, // i+1<=h, <=2h+1, 2h<=> a[i][j] = f(); N

7 IN/OUT Regios PIPS icludes iter-procedural IN ad OUT regios IN regios iclude part of the array read by a statemet, for which the value was produced earlier i the program No i regio o b Overwrite 1st b assigmet it i_regios(it, double a[], double b[], double c[]) { // <a[phi1]-out-exact-{0<=phi1, PHI1+1<=> for(it i=0; i<; i++) { a[i] = iit(); b[i] = iit(); // <a[phi1]-in-exact-{0<=phi1, PHI1+1<=> for(it i=0; i<; i++) { b[i] = a[i]+1; c[i] = f(a[i],b[i]); 7/21

8 IN/OUT Regios PIPS icludes iter-procedural IN ad OUT regios OUT regios iclude part of the array produced by a statemet ad that will be used later i the program No out regio o b No i regio o b Overwrite 1st b assigmet it i_regios(it, double a[], double b[], double c[]) { // <a[phi1]-out-exact-{0<=phi1, PHI1+1<=> for(it i=0; i<; i++) { a[i] = iit(); b[i] = iit(); // <a[phi1]-in-exact-{0<=phi1, PHI1+1<=> for(it i=0; i<; i++) { b[i] = a[i]+1; c[i] = f(a[i],b[i]); No out regio o b meas that a scalarizatio is possible Nobody would write such code... 8/21

9 IN/OUT Regios PIPS icludes iter-procedural IN ad OUT regios OUT regios iclude part of the array produced by a statemet ad that will be used later i the program No out regio o b No i regio o b Overwrite 1st b assigmet it i_regios(it, double a[], double b[], double c[]) { // <a[phi1]-out-exact-{0<=phi1, PHI1+1<=> for(it i=0; i<; i++) { a[i] = iit(); b[i] = iit(); // <a[phi1]-in-exact-{0<=phi1, PHI1+1<=> for(it i=0; i<; i++) { b[i] = a[i]+1; c[i] = f(a[i],b[i]); No out regio o b meas that a scalarizatio is possible Nobody would write such code... but what about automatically geerated code from higher level descriptio? 9/21

10 Applicatio to host-accelerator commuicatios void kerel(it, double X[][]) { it i1, i2; for (i1 = 0; i1 < /2; i1++) { // Sequetial for(i2 = i1; i2 < -i1; i2++) { // Parallel X[ i1][i2] = X[ i1][i2] - X[ - i1-3][i2]; it mai(it argc, char **argv) { if(argc!=2) { fpritf(stderr,"size expected as first argumet\"); exit(1); it size = atoi(argv[1]); // Usafe! double (*X)[size] = (double (*)[size])malloc(sizeof(double)*size*size); double (*A)[size] = (double (*)[size])malloc(sizeof(double)*size*size); double (*B)[size] = (double (*)[size])malloc(sizeof(double)*size*size); kerel(size,x,a,b); 10/21

11 Applicatio to host-accelerator commuicatios // <X[PHI1][PHI2]-R-MAY-{PHI2<=PHI1+2, <=PHI1+PHI2+3, <=2PHI1+4, PHI1+2<=, 0<=PHI2, PHI2+1<=, 2<=> // <X[PHI1][PHI2]-W-MAY-{PHI2<=PHI1+1, <=PHI1+PHI2+2, <=2PHI1+2, PHI1+2<=> for (i1 = 0; i1 < /2; i1++) { // Sequetial // <X[PHI1][PHI2]-R-EXACT-{<=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, i1<=phi2, PHI2+i1+1<=> for(i2 = i1; i2 < -i1; i2++) { // Parallel // <X[PHI1][PHI2]-R-EXACT-{PHI2==i2, <=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, PHI2==i2, 0<=i1, i1<=i2> X[ i1][i2] = X[ i1][i2] - X[ - i1-3][i2]; Read Read ad Writte 11/21

12 Applicatio to host-accelerator commuicatios // <X[PHI1][PHI2]-R-MAY-{PHI2<=PHI1+2, <=PHI1+PHI2+3, <=2PHI1+4, PHI1+2<=, 0<=PHI2, PHI2+1<=, 2<=> // <X[PHI1][PHI2]-W-MAY-{PHI2<=PHI1+1, <=PHI1+PHI2+2, <=2PHI1+2, PHI1+2<=> for (i1 = 0; i1 < /2; i1++) { // Sequetial // <X[PHI1][PHI2]-R-EXACT-{<=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, i1<=phi2, PHI2+i1+1<=> for(i2 = i1; i2 < -i1; i2++) { // Parallel // <X[PHI1][PHI2]-R-EXACT-{PHI2==i2, <=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, PHI2==i2, 0<=i1, i1<=i2> X[ i1][i2] = X[ i1][i2] - X[ - i1-3][i2]; Read Read ad Writte Writte o previous iteratios 12/21

13 Applicatio to host-accelerator commuicatios // <X[PHI1][PHI2]-R-MAY-{PHI2<=PHI1+2, <=PHI1+PHI2+3, <=2PHI1+4, PHI1+2<=, 0<=PHI2, PHI2+1<=, 2<=> // <X[PHI1][PHI2]-W-MAY-{PHI2<=PHI1+1, <=PHI1+PHI2+2, <=2PHI1+2, PHI1+2<=> for (i1 = 0; i1 < /2; i1++) { // Sequetial // <X[PHI1][PHI2]-R-EXACT-{<=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, i1<=phi2, PHI2+i1+1<=> for(i2 = i1; i2 < -i1; i2++) { // Parallel // <X[PHI1][PHI2]-R-EXACT-{PHI2==i2, <=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, PHI2==i2, 0<=i1, i1<=i2> X[ i1][i2] = X[ i1][i2] - X[ - i1-3][i2]; Read Read ad Writte Writte o previous iteratios 13/21

14 Applicatio to host-accelerator commuicatios // <X[PHI1][PHI2]-R-MAY-{PHI2<=PHI1+2, <=PHI1+PHI2+3, <=2PHI1+4, PHI1+2<=, 0<=PHI2, PHI2+1<=, 2<=> // <X[PHI1][PHI2]-W-MAY-{PHI2<=PHI1+1, <=PHI1+PHI2+2, <=2PHI1+2, PHI1+2<=> for (i1 = 0; i1 < /2; i1++) { // Sequetial // <X[PHI1][PHI2]-R-EXACT-{<=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, i1<=phi2, PHI2+i1+1<=> for(i2 = i1; i2 < -i1; i2++) { // Parallel // <X[PHI1][PHI2]-R-EXACT-{PHI2==i2, <=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, PHI2==i2, 0<=i1, i1<=i2> X[ i1][i2] = X[ i1][i2] - X[ - i1-3][i2]; Read Read ad Writte Writte o previous iteratios Optimize commuicatios (covex hull, pipelie, ) 14/21

15 Applicatio to host-accelerator commuicatios // <X[PHI1][PHI2]-R-MAY-{PHI2<=PHI1+2, <=PHI1+PHI2+3, <=2PHI1+4, PHI1+2<=, 0<=PHI2, PHI2+1<=, 2<=> // <X[PHI1][PHI2]-W-MAY-{PHI2<=PHI1+1, <=PHI1+PHI2+2, <=2PHI1+2, PHI1+2<=> for (i1 = 0; i1 < /2; i1++) { // Sequetial // <X[PHI1][PHI2]-R-EXACT-{<=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, i1<=phi2, PHI2+i1+1<=> for(i2 = i1; i2 < -i1; i2++) { // Parallel // <X[PHI1][PHI2]-R-EXACT-{PHI2==i2, <=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> // <X[PHI1][PHI2]-W-EXACT-{PHI1+i1==-2, PHI2==i2, 0<=i1, i1<=i2> X[ i1][i2] = X[ i1][i2] - X[ - i1-3][i2]; Read Read ad Writte Writte o previous iteratios 15/21

16 Applicatio to host-accelerator commuicatios for (i1 = 0; i1 < /2; i1++) { // Sequetial // Allocate all the array o the accelerator double (*accel_x)[2][-2*i1+]; P4A_accel_malloc((void **) &accel_x, sizeof(double)*i1*2)); Copy_to_accel_2d(sizeof(double),,, 2, -2*i1+, -i1+-3, i1, &X[0][0], *accel_x); for(i2 = 0; i2 < -i1-i1; i2++) { // Parallel (has bee skewed to start from 0) accel_x[1][i2] = accel_x[1][i2] - accel_x[0][i2]; Copy_from_accel_2d( Data writte by the kerel sizeof(double),,, // host size 1, -2*i1+, // trasfer -i1+-2, i1, // offset &X[0][0], &accel_x[1][0]); Accel_free(accel_X); Rectagular hull Exact data read by the kerel Read Read ad Writte Writte o previous iteratios Data trasferred o curret iteratio 16/21

17 Applicatio to host-accelerator commuicatios for (i1 = 0; i1 < /2; i1++) { // Sequetial // Allocate all the array o the accelerator double (*accel_x)[2][-2*i1+]; P4A_accel_malloc((void **) &accel_x, sizeof(double)*i1*2)); Copy_to_accel_2d(sizeof(double),,, 2, -2*i1+, -i1+-3, i1, &X[0][0], *accel_x); for(i2 = 0; i2 < -i1-i1; i2++) { // Parallel (has bee skewed to start from 0) accel_x[1][i2] = accel_x[1][i2] - accel_x[0][i2]; Copy_from_accel_2d( Data writte by the kerel sizeof(double),,, // host size 1, -2*i1+, // trasfer -i1+-2, i1, // offset &X[0][0], &accel_x[1][0]); Accel_free(accel_X); Rectagular hull Exact data read by the kerel Read Read ad Writte Writte o previous iteratios Data trasferred o curret iteratio Data trasferred o previous iteratio 17/21

18 Applicatio to host-accelerator commuicatios Ca we avoid redudat trasfers? Try a subtractio: <X[PHI1][PHI2]-{<=PHI1+i1+3, PHI1+i1+2<=, i1<=phi2, PHI2+i1+1<=> - <X[PHI1][PHI2]-{<=PHI1+(i1-1)+3, PHI1+(i1-1)+2<=, (i1-1)<=phi2, PHI2+(i1-1)+1<=> = <X[PHI1][PHI2]-{==PHI1+i1+3, i1<=phi2, PHI2+i1+1<=> Data trasferred o curret iteratio Data trasferred o previous iteratio Differece From Alias, Darte, ad Plesco, Impact 2012: I(I1 ) \ Out(i1 < I1 ) Load(i1 I1 ) Out(i1 < I1 ) Load(I1 ) = 18/21

19 Applicatio to host-accelerator commuicatios // <X[PHI1][PHI2]-R-MAY-{PHI2<=PHI1+2, <=PHI1+PHI2+3, <=2PHI1+4, PHI1+2<=, 0<=PHI2, PHI2+1<=, 2<=> // <X[PHI1][PHI2]-W-MAY-{PHI2<=PHI1+1, <=PHI1+PHI2+2, <=2PHI1+2, PHI1+2<=> double (*accel_x)[-2-(/2-1)+1][-1+1]; P4A_accel_malloc((void **) &accel_x, sizeof(double)*(-2-(/2-1)+1)*(-1+1)); // Data for first iteratio Copy_to_accel_2d(sizeof(double),,, 1,, -3, 0, &X[0][0], &accel_x[-2-(/2-1)+1][0]); for (i1 = 0; i1 < /2; i1++) { // Sequetial Copy_to_accel_2d(sizeof(double),,, 1,-2*i1+,-i (/2-1)+1, i1, &X[0][0],*accel_X); for(i2 = 0; i2 < -i1-i1; i2++) // Parallel X[ i1-2-(/2-1)+1][i2] = X[ i1-2-(/2-1)+1][i2] - X[ - i1 3-2-(/2-1)+1][i2]; Copy_from_accel_2d( sizeof(double),,, // host size 1, -2*i1+, // trasfer -i1+-2, i1, // offset &X[0][0], &accel_x[1][0]); Accel_free(accel_X); Further optimizatios (prefetch...) would easily allow overlap betwee commuicatios ad computatios. See for istace Alias, Darte, ad Plesco, Impact /21

20 Par4All future Source code (with directives?) PIPS Trasformatios && Aalyses Source code with directives feature extractor polyhedral optimizer kerels host code OpeSCoP Post-processor, optimizer,... Post-processor, optimizer,... Par4All Rutime Other Rutimes vcclike host compiler Fial Biary 20/21

21 Par4All future Source code (with directives?) PIPS Trasformatios && Aalyses Source code with directives Your? feature extractor Your? polyhedral optimizer kerels host code OpeSCoP Your? Post-processor, optimizer,... Your? Post-processor, optimizer,... Par4All Rutime Other Rutimes Icludig yours? vcclike host compiler Fial Biary