Grid Computing introduction & illustration. T. Gautier V. Danjean

Transcription

1 Grid Computing introduction & illustration T. Gautier V. Danjean SMAI 26 mai 2009

2 Facts No choice : parallelism is in any mputer MPSoC, Multi, Many, Cluster, Grid Exact Solution to the Quadratic Assignment Problem Combinatorial Problem: NUG30, 7 days, 650 processors Solving bigger CFD, CEM problems 100GBytes of memory Cluster & Grid mputing

3 Outline Context Parallel and distributed architecture Programming Challenges Parallel algorithm Scheduling & Communition Illustration with domain demposition method Conclusions

4 Rennes - 3 clusters Opteron, Xeon s Lille Paris AMD opteron, 1GB/s ethernet Nancy Bordeaux Lyon Grenoble -248 Xeon Intel, InfiniBand -16 s NUMA - 8 s NUMA Toulouse Sophia

5 Grid5000

6 Grid che ory che dual processor che ory che dual processor dual NUMA Non Uniform ory Access ory che ory che ory che ory che ory che ory che ory che ory che Network

7 Characterisics Cluster: set of homogeneous machines homogeneous resources (memory, cpu) Dual, Quad, Octo s cpu, GBytes / machine homogeneous & high performance network Ethernet, Myrinet, InfiniBand homogeneous administration domain 1 user = 1 acunt, home directory mounted using NFS

8 Characterisics Grid: set of clusters heterogeneous resources CPU, memory, network speed Each cluster has different number of machines multiple administration domains Network between clusters: high latency! 1 user = multiple acunt access to cluster through firewall dependability problem huge number of basic mponents

9 Programming Challenges Write once, run anyware heterogeneity! Keypoints parallel algorithm scheduling implementation [fault tolerance]

10 Parallel algorithm 30 years of theoretil studies & experiments of parallel architectures What s new? huge number of s/cpu fault tolerance ABFT: Algorithm Based Fault Tolerant

11 Let s back to a problem Domain demposition for matrix vector product Ωi,j Ωk,l (Ax) i, j = B i, j F i, j E i, j C i, j u i, j v i, j + 0 Ω k,l N i, j E k,l i, j v k,l ANR DISCOGRID, ordinateur S. Lanteri

12 Let s back to a problem Domain demposition for matrix vector product Ωi,j Ωk,l (Ax) i, j = B i, j F i, j E i, j C i, j u i, j v i, j + 0 Ω k,l N i, j E k,l i, j v k,l lol internal values lol interface values ANR DISCOGRID, ordinateur S. Lanteri

13 Let s back to a problem Domain demposition for matrix Ωi,j vector product Ωk,l external interface values (Ax) i, j = B i, j F i, j E i, j C i, j u i, j v i, j + 0 Ω k,l N i, j E k,l i, j v k,l lol internal values lol interface values ANR DISCOGRID, ordinateur S. Lanteri

14 Scheduling Once tasks and data are described by a parallel algorithm then: For each task, mpute where to execute it For each data, mpute where to store it Such that: Completion time is minimize,... NP-hard problem Use algorithms to approximate the problem Use heuristics, applition dependent

15 Strict multi-threaded Notations Ts : Sequential work, time of sequential execution D : Critil Path P: the P processors Properties mputations with high probability, number of steals is O(P x D) with high probability, execution time is Tp T1 / P + O(D)

16 Domain demposition Network dual processor dual dual processor dual NUMA Non Uniform ory Access

17 Domain demposition Graph partitioner - stch - metis - hierarchil : ANR DISCOGRID Network dual processor dual dual processor dual NUMA Non Uniform ory Access

18 That s all? Domain demposition Data = sub domain, mapped onto machines Task : mapped using owner mpute rule Program : one process per subdomain while (error < epsilon) { exchange interface do mputation inside subdomain mpute error }

19 That s all? Domain demposition Data = sub domain, mapped onto machines Task : mapped using owner mpute rule Program : one process per subdomain while (error < epsilon) { exchange interface Communition between neighbors do mputation inside subdomain mpute error }

20 That s all? Domain demposition Data = sub domain, mapped onto machines Task : mapped using owner mpute rule Program : one process per subdomain while (error < epsilon) { exchange interface Communition between neighbors do mputation inside subdomain mpute error Global reduction } mmunition

21 Improvement Assume that emission & reception of message are ncurrent with lol mputation while (error < epsilon) { begin send message to my neighbors do internal mputation wait until all messages have been received update internal mputation mpute error }

22 Improvement Assume that emission & reception of message are ncurrent with lol mputation may overlap while (error < epsilon) { some delay of mmunition begin send message to my neighbors do internal mputation wait until all messages have been received update internal mputation mpute error }

23 How to program MPI, standard but low level API scheduling and mapping should be ded bad overlapping of mmunition by mputation (at least in public domain implementation) bad support multi-threaded mputations bad support for inter-cluster mmunition Research languages UPC, Titanium, X10, Fortress Our language: Athapasn (API) with Kaapi AUTOMATIC SCHEDULING :

24 Experiments Code Kaapi / C++ de versus Fortran MPI de Platform Cluster : N processors on a cluster Grid : N/4 processors per cluster, 4 clusters D=256^3 # processors Cluster (s) Grid (s) Overhead KAAPI MPI , , , ,31

25 Optimized Poisson 3D Fortran de with non-blocking IO MPI_ISend, MPI_IRecv + MPI_Wait_all Overlapping of mmunition by mputation kaapi opt sendrecv ompi irecvisend ompi async ompi 256^3/proc between Rennes and Bordeaux 1.4 Mean time for an iteration (s) Nb proc

26 Conclusion SCHEDULING but also mpilation, grid-reservation, parallel launching, runtime environment, firewall management,... More references Herlihy, M. and Shavit, N. The Art of Multiprocessor Programming, ISBN Morgan Kaufmann Publishing, Foster I., Kesselman C. The GRID 2: Blueprint for a New Computing Infrastructure, ISBN Morgan Kauffman Publishing, 2 edition, Petasle Computing: Algorithms and Applitions, D. Bader (Editor), ISBN , Chapman & Hall/CRC, 2007 Parallel Algorithms and Cluster Computing: Implementations, Algorithms and Applitions. Karl Heinz Hoffmann (Editor), Arnd Meyer (Editor), ISBN , Springer, 2006.

27