Holland Computing Center Kickstart MPI Intro

Transcription

1 Holland Computing Center Kickstart 2016 MPI Intro

2 Message Passing Interface (MPI) MPI is a specification for message passing library that is standardized by MPI Forum Multiple vendor-specific implementations: MPICH, OpenMPI, Intel MPI MPI implementations are used for programming systems with distributed memory Each process has a different address space Processes need to communicate with each other Synchronization Data Exchange Can also be used for shared memory and hybrid architectures MPI specifications have been defined for C, C++ and Fortran programs

3 MPI Intro MPI_Init() Starts MPI MPI_Finalize() Exits MPI MPI_Comm_size() Determine the total number of processes MPI_Comm_rank() Determine the rank of the calling process MPI_Send() Perform a basic send MPI_Recv() Perform a basic receive

4 MPI Hello World :: Preliminaries #include <stdio.h> #include <string.h> #include "mpi.h" int main(int argc, char* argv[]) { int my_rank; /*rank of process */ int p; /*number of processes */ int source; /*rank of sender */ int dest; /*rank of receiver */ int tag = 0; /*tag for messages */ char message[100]; /*storage for message */ MPI_Status status; /*return status for receive */

5 MPI Hello World :: Generic MPI calls /* Start up MPI */ MPI_Init(&argc, &argv); /* Find out process rank */ MPI_Comm_rank(MPI_COMM_WORLD, &my_rank); /* Find out number of processes */ MPI_Comm_size(MPI_COMM_WORLD, &p); All MPI code begins with MPI_Init and ends with MPI_Finalize seen later. Processes in a group in MPI code are defined by a Communicator the group of all processes is denoted by MPI_COMM_WORLD. For many scientific codes that are data parallel, this is the only Communicator needed. Each process in a group of p processes has a unique non-negative integer assigned to it, 0 (p-1).

6 MPI Hello World :: Worker Code if (my_rank!= 0) { /* Create message */ sprintf(message, "Greetings from process %d!", my_rank); dest = 0; /* Use strlen+1 so that '\0' gets transmitted */ MPI_Send(message,strlen(message)+1, MPI_CHAR, dest, tag, MPI_COMM_WORLD); } The same binary code is distributed to every process. Thus, case statements must be used to differentiate tasks for different processors. MPI_Send (buffer, count, type, destination process, tag, communicator) Each message is analogous to DATA + Envelope.

7 MPI Hello World :: Master Code else { /* my_rank == 0 */ for (source = 1; source < p; source++) { } MPI_Recv(message, 100, MPI_CHAR, source, tag, MPI_COMM_WORLD, &status); printf\("%s\n", message); } Note that the Master is rank == 0. This convention is almost always followed, although it is possible to circumvent. MPI_Recv (buffer, count, type, source process, tag, communicator, status) MPI_ANY_SOURCE can be a handy wildcard in many cases (not this one!) Status variable in principle can help to debug code. Good luck with that.

8 MPI Hello World :: Clean Up /* Shut down MPI */ } /* main */ MPI_Finalize(); Weird, unruly errors are spawned when you forget the MPI_Finalize call. Just do it. This code has left nothing to chance we ve explicitly sent a string back to the master, and even determined in which order the messages should be printed out (and only by the master).

9 MPI Hello World :: Summary Distributed memory Data parallel Message passing Identical binary running on all nodes

10 MPI Calls :: Collective vs. point-to-point Point-to-point Between two processes MPI_Send(), MPI_Recv(), etc. One process performs a send operation and the other task performs a matching receive operation There should be a matching receive routine for every send routine If a send is not paired with a matching receive then the code will have a deadlock

11 MPI Calls :: Collective vs. point-to-point Point-to-point: blocking vs. non-blocking Blocking A blocking receive only returns after the data has arrived and is ready for use by the program A blocking send routine returns after it is safe to modify the application buffer for reuse Non-blocking Non-blocking send and receive routines will return almost immediately It is unsafe to modify the application buffer until you know for a fact that the requested non-blocking operation was actually performed

12 MPI Calls :: Collective vs. point-to-point Collective Defined as communication between >2 processes One-to-many, many-to-one, many-to-many All processes within the communicator group call the same collective communication function with matching arguments Collective communication routines are blocking The size of data sent must exactly match the size of data received Examples: Broadcast, Scatter, Gather, Reduce

13 MPI :: Summary MPI is the de facto standard for programming parallel (tightly-coupled), distributed-memory code If you need to scale to 1000s of cores, use MPI API for C/C++/Fortran Identical binary runs everywhere Separate address space for each process Communication between processes is explicit via message passing MPI programming is challenging!

14 Questions? (Hands-on exercises)

15 Back Page