Introduction in Parallel Programming - MPI Part I

Transcription

1 Introduction in Parallel Programming - MPI Part I Instructor: Michela Taufer WS2004/2005 Source of these Slides Books: Parallel Programming with MPI by Peter Pacheco (Paperback) Parallel Programming in C with MPI and OpenMP by Michael J. Quinn (Hardcover) Slides: Parallel Programming with MPI, by Michael M. Resch : S_02/mpi_kurs.pdf 2 2 1

2 Message-passing Model 3 3 Program, Process, and Processor A process is a program in execution We can run identical copies of a program in parallel: distribute processes among processors Process 0 running on Processor A has access only to the local data in Memory A Process 1 running on Processor B has access only to the local data in Memory B. Each process has a different ID, starting from 0 Processes are distinguish by their ID Processes 0, 1, 2, 3, may follow a distinct flow of control because working with different data This policy is called: Single Program Multiple Data (SPMD) 4 4 2

3 Program mpi_simple.c 5 5 Program mpi_simple.c Each MPI program includes mpi.h 6 6 3

4 Compile and Execute To compile the program: mpicc -o mpi_simple mpi_simple.c The executable mpi_simple is generated in you directory To execute 2 processes: mpirun -np 2 mpi_simple If your program requires command arguments: mpirun -np 2 myprogram -mpiversion x y z is equivalent to: myprogram x y z where x y z are arguments passed to the program 7 7 Machines for Testing and Programming Car cluster: ssh porsche.cis.udle.edu Evans Hall Lab XP/Linux systems and Test Equip. for UG CPEG Labs Remote SSH Alias "linuxlab.acad.ece.udel.edu" McKinly Lab Lab XP/Solaris_x86/Linux systems for CIS labs Remote SSH Alias "hplab.acad.cis.udel.edu" 8 8 4

5 Options for mpirun mpirun [mpirun_options...] <progname> [options...] -arch <architecture> specify the architecture (must have matching machines.<arch> file in ${MPIR_HOME}/util/machines) if using the execer -h this help -machine <machine name> use startup procedure for <machine name> -machinefile <machine-file name> take the list of possible machines to run on from the file <machinefile name> -np <np> specify the number of processors to run on -nolocal do not run on the local machine (only works for p4 and ch_p4 jobs) 9 9 Program mpi_simple.c Each MPI program starts with MPI_init

6 MPI_Init() To establish the MPI environment we call MPI_Init() Required in any MPI program Must be the first MPI call The arguments of MPI_Init() are references to the arguments of main(), argc and argv Program mpi_simple.c Each MPI program terminates with MPI_Finalize()

7 MPI Program Framework #include "mpi.h" main(int argc, char** argv) { /* No MPI functions called before this */ MPI_Init(&argc, &argv); MPI_Finalize(); /* No MPI functions called after this */ } /* main */ Key Concepts of MPI An MPI communicator is a collection of processes that can send messages to each other A group of processes have a context not explicitly used in MPI calls and implicitly associated to a group of processes when the communicator is created Each process has a rank, I.e., integer value. mympi mympi mympi context process process process ranks MPI communicator

8 Program mpi_simple.c Each process belongs to the same communicator but has different rank Program mpi_simple.c Process to process communication using MPI_Send and MPI_Recv

9 PC1 PC2 user Message Passing

10 Communication in MPI Data values are transferred from one processor to another: One process sends the data Another receives the data MPI_Send( ) - to send message to another processor MPI_Recv( ) - to "get" the message Message = Envelope + Data A message is uniquely described by: An envelope indicating: The source or destination of the message The type of message The data message envelope data

11 Envelope An envelope consists of: Rank of the sender: this is an integer Rank of the receiver: this is an integer or MPI_ANY_SOURCE Unique tag or message type: this is an integer and can be MPI_ANY_TAG Communicator: a communicator created by the programmer or the standard communicator MPI_COMM_WORLD Example: (1, 99, MPI_COMM_WORLD) (MPI_ANY_SOURCE, MPI_ANY_TAG, COMM_WORLD) (0, 5, mympi) Data The data contents are described by: The buffer of data sent or received, e.g., variable, array The number of items in the buffer, e.g., 1 for a variable, n for an array where n is the number of elements in the array The type of data sent - note MPI has its own data types Examples: (array_name, 20, MPI_DOUBLE) (variable_name, 1, MPI_INT)

12 Predefined Types MPI has many different predefined data types All your favorite C data types are included MPI data types can be used in any communication operation The user can create user-defined types MPI uses a number of predefined constants, e.g., MPI_COMM_WORLD - a standard communicator MPI_SUM - predefined operation, sum MPI_ANY_TYPE and MPI_ANY_SOURCE Predefined types and constants are defined in the include file mpi.h Data Type: MPI and C MPI MPI_BYTE MPI_CHAR MPI_DOUBLE MPI_FLOAT MPI_INT MPI_LONG MPI_LONG_DOUBLE MPI_PACKED MPI_SHORT MPI_UNSIGNED_CHAR MPI_UNSIGNED MPI_UNSIGNED_LONG MPI_UNSIGNED_SHORT C signed char double float int long long double short unsigned char unsigned int unsigned long unsigned short

13 MPI_Send() ierr = MPI_Send( &buffer, count, datatype, destination, tag, communicator); buffer: the data count: length of source array (in elements, 1 for scalars) datatype: type of data, e.g.,mpi_double_precision, MPI_INT destination: processor number of destination processor in communicator tag: message type communicator: Your set of processors ierr: error return The send call blocks until the send buffer can be reclaimed - after send the processor can safely over-write the buffer MPI_Recv() ierr = MPI_Recv(&buffer, count, datatype, source, tag, communicator, &status); buffer: the data count: length of source array (in elements, 1 for scalars) datatype: type of data, for example : MPI_DOUBLE_PRECISION, MPI_INT, etc source: processor number of source processor in communicator or MPI_ANY_SOURCE tag: message type (arbitrary integer or MPI_ANY_TAG) communicator: your set of processors, e.g., MPI_COMM_WORLD status: information about message ierr: error return Status - contains information about incoming message MPI_Status status;

14 MPI_ANY_SOURCE and MPI_ANY_TAG ierr = MPI_Recv( messages, 20, MPI_INTEGER MPI_ANY_SOURCE, MPI_ANY_TAG, MPI_COMM_WORLD, status) data status information envelope How can the receiver find out about the sender and/or the tag that was sent? Is the size of the message the real size? Status (I) The status parameter returns additional information for some MPI routines additional error status information additional information with wildcard parameters In the status object we can find: Sender Tag Error code The actual size of the message (the size specified is an upper bond)

15 Status (II) status is a predefined structure of type MPI_Status which contains three fields MPI_SOURCE, MPI_TAG, and MPI_ERROR MPI_Status status; Accessing Status Information (I) The source of a received message status.mpi_source The tag of a received message status.mpi_tag The error code of the MPI call status.mpi_error

16 Accessing Status Information (II) To get the real size of the message: ierr = MPI_Get_Count (status, datatype, count); if (ierr!= MPI_SUCCESS) { printf( Error during MPI call MPI_Get_ Count \n ); exit (1); } In general MPI_xxxx calls return MPI_SUCCESS (0), if no error occurs MPI routine completed successfully Except for MPI_Wtime and MPI_Wtick Minimal MPI program: C Example All calls look like MPI_Xxxxx( )... #include <mpi.h> /* the mpi include file */ /* Initialize MPI */ int npes, iam; MPI_Status status; ierr=mpi_init(&argc, &argv); /* How many total processes are there */ ierr=mpi_comm_size(mpi_comm_world, &npes); /* What processor am I (what is my rank? */ ierr=mpi_comm_rank(mpi_comm_world, &iam);... code code... ierr=mpi_finalize(); MPI routines are functions and return an error value

17 Example in C (I) #include <stdio.h> #include "mpi.h" /**************************************************** This is an example to show how to access the status parameters *****************************************************/ int main(argc,argv) int argc; char *argv[]; { int myid, numprocs, count; int message =30; MPI_Status status; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); Example in C (II) if(myid == 0){ MPI_Send(&message, 1, MPI_INT, 1, 99, MPI_COMM_WORLD); } if(myid == 1){ MPI_Recv(&message, 1, MPI_INT, MPI_ANY_SOURCE, 99, MPI_COMM_WORLD, &status); printf( Tested source = %d \n, status.mpi_source); MPI_Get_count (&status, MPI_INT, &count); printf( Tested length = %d \n, count); } MPI_Finalize(); }

18 Timing your MPI Program MPI_Wtime() elapsed wall-clock time since some time in the past a floating-point number of seconds MPI_Wtick() Return the resolution of MPI_WTIME in second Example: if the clock increments every millisecond, the value returned should be 10-3 double starttime, endtime, totaltime, resolution; resolution = MPI_Wtick(); starttime = MPI_Wtime();... Do something... endtime = MPI_Wtime(); totaltime = endtime - starttime; Timing your Program: Example if (thispe == masterpe) { /*Code for master Pe */ starttime = MPI_Wtime(); for ( i =0 ; i< transferrepeats ; i++) { MPI_Send( senddata, msglen, MPI_BYTE, slavepe, 1, MPI_COMM_WORLD); MPI_Recv( recvdata, msglen, MPI_BYTE, slavepe, 1, MPI_COMM_WORLD, &status); } endtime = MPI_Wtime(); time=(endtime starttime)/transferrepeats/2; }

19 if (thispe == slavepe) { /*Code for slave Pe */ for ( i =0 ; i< transferrepeats ; i++) { MPI_Recv( recvdata, msglen, MPI_BYTE, masterpe, 1, MPI_COMM_WORLD, &status); MPI_Send( senddata, msglen, MPI_BYTE, masterpe, 1, MPI_COMM_WOLD); } } This algorithm is called sample round-trip (or pingpong) algorithm Why is the data sent fourth and back several times? program02.c Work in Group - Questions Consider the given MPI example, answer these questions: Q) If there are p processes executing a program what is their ranks? Q) What does the program do? Q) What is the output when the program runs with 2 processes? Q) What is the output when the program runs with 4 processes? program01.c

20 #include <stdio.h> #include "mpi.h" main(int argc, char** argv) { int my_rank; /* Rank of process */ int p; /* Number of processes */ int source; /* Rank of sender */ int dest; /* Rank of receiver */ int tag=50; /* Tag for messages */ char message[100] /* Storage for the message */ MPI_Status status; /* Return status for receive */ MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &my_rank); MPI_Comm_size(MPI_COMM_WORLD, &p); if (my_rank!= 0) { sprintf(message, "Greetings from process %d!", my_rank); dest = 0; /* Use strlen(message)+1 to include '\0' */ MPI_Send(message, strlen(message)+1, MPI_CHAR, dest, tag, MPI_COMM_WORLD); else { /* my_rank == 0 */ for (source = 1; source < p; source++) { MPI_Recv(message, 100, MPI_CHAR, source, tag, MPI_COMM_WORLD, &status); printf("%s\n", message); } } MPI_Finalize(); } /* main */ Deadlines Date Class activities Speaker Homework Deadline Project Deadlines 2/10 Lecture: Course Introduction / Parallel architectures 2/12 Lecture: MPI 2/17 Lecture: MPI 2/19 Lecture: MPI Deadline 0 2/20 SIG-SYS seminar NVIDIA + Penguin Computing 2/ No Class --- 2/26 Lecture: MPI Student: randomly selected Discussion: Master-slave paradigm 2/27 SIG-SYS seminar Karan Bhatia 3/03 Seminar presentations Student 1: Deadline 1 3/05 Lecture: OpenMP Homework