Clusters of SMP s. Sean Peisert

Transcription

1 Clusters of SMP s Sean Peisert

2 What s Being Discussed Today SMP s Cluters of SMP s Programming Models/Languages Relevance to Commodity Computing Relevance to Supercomputing

3 SMP s Symmetric Multiprocessors Uniform Memory Architecture (UMA) Basic SMP architecture: Shared Memory $ $ $ $ PE PE PE PE

4 Intra-SMP Architecture Ideally we communicate using shared-memory so a fast memory with a low cache-miss penalty is important. The bus is also important if we re going to use flat messagepassing across the network.

5 Moore s Law Processor speed doubles every 18 months. (Actually processor speed is increasing faster than this but memory, disk and bus speed is much slower.)

6 Future of Commodity Computing Eventually gains in processor speed will start to slow. Multiprocessor PC s are the logical step when Moore s law starts to fail the model with single processors. Shared-memory model has its advantages over a simple dualprocessor, dual-memory machine.

7 Clusters of SMP s Hierarchical Machines Cluster of SMP s SMP SMP SMP SMP High-speed networking links Networking Link

8 Why Clusters of SMP s? Cost Many use commodity parts and links including the UltraSparc and PowerPC chips. Expandability Just add another SMP box. Best of both worlds Possible to take advantage of shared memory as well as message passing.

9 Networking in Clusters 100baseT (Fast Ethernet), Myrinet, FDDI (Fiber Optics Distributed Data Interface) 100 Mbps HiPPI (High Performance Parallel Interface) or Fibre Channel 800 Mbps one-way Gigabit Ethernet 1 Gbps

10 SMP s using Commodity Parts Today Sun Enterprise 6000 Wildfire and Starfire clusters DEC Alpha Rawhide Clusters

11 Why not Clusters of SMP s? Difficult to program. Hard enough to program just using threads or message-passing. Even harder to use both. Uses existing technology. What about new innovations? Scalability Uncertainty as to how well SMP s and clusters of SMP s scale?

12 Programming SMP s MPI MPI2 Pthreads OpenMP KeLP

13 Programming SMP s with MPI MPI can easily double the size of serial code. Doesn t take advantage of the SMP s shared-memory architecture. But, MPI is ubiquitous and widely known. MPI2 supports 1-sided communication.

14 Basic MPI Calls Point-to-point MPI_Send MPI_Recv Collective MPI_Gather MPI_Scatter MPI_Bcast MPI_Alltoall MPI_Barrier

15 MPI2 MPI2 adds 1-Sided Communication. get and put just like the T3E s shmem calls. One-sided communication can mimic shared memory in its structure and can be easier at times to program with. MPI2 also adds other important features including fast, parallel I/O (MPI-I/O).

16 Programming SMP s with Shared-Memory in Mind Utilizes the unique architecture of the SMP. Threads can be more logically programmed than message passing. Can t use threads across multiple SMP s. No explicit barrier synchronization.

17 Basic Pthreads Calls Thread Management pthread_create pthread_exit pthread_join (synchronization) Mutex Variables pthread_mutex_lock pthread_mutex_trylock pthread_mutex_unlock

18 Pseudo-code for sharedmemory image processing while (blocks are left) { } Lock number of blocks and the array that says which block to do next. Decrement number of blocks. Grab the next block to do. Unlock number of blocks and next-block array. Process the block from the source array and write (without boundaries) to destination array.

19 Programming SMP s with KeLP Code less complex. Irregular structures easier. Architecture can be more independent. KeLP is very different from threads or message-passing. Learning curve can be steep.

20 MPI vs. KeLP 7 6 Compute Communicate KeLP MPI ) c e s ( NAS-MG, 256x256x256 Compute (KeLP) Communicate (KeLP) Total (MPI) E M I T 10 5 KeLP MPI

21 Programming SMP s -- Summary Pros Cons Ubiquitous. Can be complex MPI to program. Doesn t utilize SMP s sharedmemory. MPI2 Pthreads OpenMP KeLP One-sided communication makes messagepassing easier. Uses sharedmemory instead of standard bus. Uses compiler directives in addition to library calls. Higher level than Pthreads Uses shared memory. Works with irregular grids. Latency hiding. Doesn t utilize shared-memory. Limited implementation so far. POSIX threads only on UNIX. Difficult to debug. Weak Fortran support. Very different from threads or message passing.

22 Programming Clusters of SMP s Hand-tuned Pthreads & MPI KeLP2

23 Hand-tuned Pthreads and MPI Utilizes both the SMP architecture and the networking layer between SMP s. Hand-tuned Pthreads and MPI is very complex.

24 KeLP2 Like KeLP, easy to program, handles irregular structures and can be easily architectureindependent. Also easy to do latency-hiding. Doesn t handle heterogeneous machines well -- yet.

25 Supercomputers: Clusters of SMP s ASCI Blue-Pacific (IBM/LLNL) ASCI Red (Intel/Sandia) IBM TFLOPS at SDSC.

26 Alternative Architecture Approaches SGI/Cray T3E (massively parallel machines) SGI/Cray T90 (vector machines) SGI Origin2000 (including ASCI Blue-Mountain at LANL) Simultaneous Multithreaded Architectures (Tera, PetaFLOPS initiative)

27 Future of Supercomputing Viable option to continue to build supercomputers as clusters of SMP s? Can we build clusters of SMP s using multithreaded chips? Are SMP s scalable?

28 Summary Clusters of SMP s are the immediate future of supercomputing and maybe personal and business computing as well. Issues: Scalability Difficulty to program Solving the major issues might make the lifespan of SMP clusters longer.