CS 351 Week The C Programming Language, Dennis M Ritchie, Kernighan, Brain.W

Transcription

1 CS 351 Week 6 Reading: 1. The C Programming Language, Dennis M Ritchie, Kernighan, Brain.W Objectives: 1. An Introduction to Message Passing Model 2. To learn about Message Passing Libraries Concepts: 1. Message Passing Model 2. Message Passing Libraries 3. Overview of Lab Outline: 1. Message Passing Model a. Introduction b. What is MPI? c. What is MPI Forum? 2. Message Passing Libraries a. Terminology b. Point-to-Point Communication c. Collective Communication d. Performance Guidelines 3. Overview of Lab Reference: Programming for Windows by Charles Petzold CS 351 Week 6 - Page 1

2 CS 351: Week 6 Lecture Notes 1. Message Passing Model A.The message passing model is defined as: set of processes having only local memory processes communicate by sending and receiving messages the transfer of data between processes requires cooperative operations to be performed by each process (a send operation must have a matching receive) Other models include: data parallelism data partitioning determines parallelism shared memory multiple processes sharing common memory space remote memory operation set of processes in which a process can access the memory of another process without its participation threads a single process having multiple (concurrent) execution paths combined models composed of two or more of the above B. What is MPI? MPI stands for Message Passing Interface. MPI is a library specification for message-passing, proposed as a standard by a broadly based committee of vendors, implementors, and users. The MPI standard is available. MPI was designed for high performance on both massively parallel machines and on workstation clusters. MPI is widely available, with both free available and vendor-supplied implementations. MPI was developed by a broadly based committee of vendors, implementors, and users. Information for implementors of MPI is available. Test Suites for MPI implementations are available. CS 351 Week 6 - Page 2

3 CS 351: Week 6 Lecture Notes C. What is MPI Forum? The Message Passing Interface Forum (MPIF) Participation from over 40 organizations, met from November 1992 to April 1994 to discuss and define a set of library interface standards for message passing. Discussions resulted in Version 1.1 (dated June, 1995) which contained only minor changes from Version 1.0. The MPIF is not sanctioned or supported by any official standards organization. 2.Message Passing Libraries The common components of message passing libraries include: Process management routines (initialize and finalize processes, determine number of processes and process identifiers) Point-to-point communication routines (basic sends and receives between any two processes) Process group/collective communication routines (broadcast/gather/scatter operations amongst a set of processes, synchronization of processes) A.Terminology Buffering Temporary copying of messages performed by system as a part of its transmission protocols. Blocking Communication A communication routine is blocking if the completion of the call is dependent on certain "events". Nonblocking Communication A communication routine is nonblocking if the call returns without waiting for any communications events to complete Synchronous Communication Communication in which the sender does not return until the matching receive has been posted on the destination process. Asynchronous Communication Communication in which the sender and receiver place no constraints on each other in terms of completion. B. Point-to-Point Communication The basic send and receive operations look like: send(address, length, destination, tag) CS 351 Week 6 - Page 3

4 CS 351: Week 6 Lecture Notes receive (address, max_length, source, tag, rec_length) where address = memory location of the beginning of the buffer containing the data to be sent or received length = length in bytes of message being sent max_length = length of the buffer into which received data is placed rec_length = number of bytes of data actually received destination = identifier of the process to which message is sent source is handled in one of the following ways by a given message passing library o as an output argument indicating where the message originated o as an input argument specifying where the message is to originate, messages not originating at the specified source must be queued tag = arbitrary (user defined) nonegative integer restricting receipt of messages C. Collective Communication Collective operations are coordinated among a "group" of processes Library routines are used to construct a group from a subset of processes Collective operations are blocking Three types of collective operations o Synchronization - processes wait until all members of the group have reached the synchronization point o Data movement broadcast scatter/gather all to all D.Performance Guidelines There are some basic steps that can be taken improve the performance of programs written using a message passing library. Start with tuned serial program,control process granularity Overlap communication and computation Avoid unnecessary synchronization Avoid buffering when possible Keep data local CS 351 Week 6 - Page 4

5 4. Overview of Lab C Language - Environment Management Routines Example CS 351: Week 6 Lecture Notes #include "mpi.h" #include <stdio.h> int main(argc,argv) int argc; char *argv[]; { int numtasks, rank, rc; rc = MPI_Init(&argc,&argv); if (rc!= MPI_SUCCESS) { printf ("Error starting MPI program. Terminating.\n"); MPI_Abort(MPI_COMM_WORLD, rc); MPI_Comm_size(MPI_COMM_WORLD,&numtasks); MPI_Comm_rank(MPI_COMM_WORLD,&rank); printf ("Number of tasks= %d My rank= %d\n", numtasks,rank); /******* do some work *******/ MPI_Finalize(); C Language - Blocking Message Passing Routines Example #include "mpi.h" #include <stdio.h> int main(argc,argv) int argc; char *argv[]; { int numtasks, rank, dest, source, rc, count, tag=1; char inmsg, outmsg='x'; MPI_Status Stat; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD, &numtasks); MPI_Comm_rank(MPI_COMM_WORLD, &rank); if (rank == 0) { dest = 1; source = 1; rc = MPI_Send(&outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD); rc = MPI_Recv(&inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat); else if (rank == 1) { dest = 0; CS 351 Week 6 - Page 5

6 source = 0; rc = MPI_Recv(&inmsg, 1, MPI_CHAR, source, tag, MPI_COMM_WORLD, &Stat); rc = MPI_Send(&outmsg, 1, MPI_CHAR, dest, tag, MPI_COMM_WORLD); CS 351: Week 6 Lecture Notes rc = MPI_Get_count(&Stat, MPI_CHAR, &count); printf("task %d: Received %d char(s) from task %d with tag %d \n", rank, count, Stat.MPI_SOURCE, Stat.MPI_TAG); MPI_Finalize(); C Language - Non-Blocking Message Passing Routines Example #include "mpi.h" #include <stdio.h> int main(argc,argv) int argc; char *argv[]; { int numtasks, rank, next, prev, buf[2], tag1=1, tag2=2; MPI_Request reqs[4]; MPI_Status stats[4]; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD, &numtasks); MPI_Comm_rank(MPI_COMM_WORLD, &rank); prev = rank-1; next = rank+1; if (rank == 0) prev = numtasks - 1; if (rank == (numtasks - 1)) next = 0; MPI_Irecv(&buf[0], 1, MPI_INT, prev, tag1, MPI_COMM_WORLD, &reqs[0]); MPI_Irecv(&buf[1], 1, MPI_INT, next, tag2, MPI_COMM_WORLD, &reqs[1]); MPI_Isend(&rank, 1, MPI_INT, prev, tag2, MPI_COMM_WORLD, &reqs[2]); MPI_Isend(&rank, 1, MPI_INT, next, tag1, MPI_COMM_WORLD, &reqs[3]); { do some work MPI_Waitall(4, reqs, stats); MPI_Finalize(); CS 351 Week 6 - Page 6