Practical Course Scientific Computing and Visualization

Transcription

1 July 5, 2006 Page 1 of 21

2 1. Parallelization Architecture our target architecture: MIMD distributed address space machines program1 data1 program2 data2 program program3 data data3.. program(data) program1(data1) program2(data2) program3(data3) communication program4(data4) program5(data5) program6(data6) Page 2 of 21

3 2. solution of worksheet 1 while(...){ compute_dt(...); boundary_val(...); compute_fg(...); compute_rhs(...); while(...) sor(...); compute_uv(...); output_uvp(...);} example to be computed: driven cavity Page 3 of 21

4 3. Message Passing Interface standard library connecting heterogeneous computers to a virtual parallel computer defined by the MPI-forum (first version: June, 1994) available for native compilers of several hardware manufacturers (IBM, SGI, HP, etc.), public domain ( one program for all processes identification of processes (and/or work to do) with the help of a parameter control of all processes by a master process (e.g. starting, ending) no need to send the executable to other processors in a local network Page 4 of 21

5 4. Example: communication.c P_ID P_ID P_ID π Page 5 of

6 5. communication.c compiler, program call compiler: mpicc (all compiler/linker options possible) mpicc -o communication communication.c program call: use shell-script mpirun start four processes on the local machine: mpirun -np 4 communication start six processes on several defined machines: * machinefile: oneprocessormachine.in.tum.de twoprocessormachine.in.tum.de:2 monstermachine.in.tum.de:100 mpirun -np 6 -machinefile machines.dat communication Page 6 of 21

7 6. communication.c part 1 program header: #include stdlib.h #include stdio.h #include mpi.h initializing: MPI_Init( &argc, &argv ); /* start n processes*/ MPI_Comm_size( MPI_COMM_WORLD, &nproc ); /* nr of processes? */ MPI_Comm_rank( MPI_COMM_WORLD, &myrank ); /* local process-id?*/ printf process-id to stdout: printf("process %2d von %2d\n", myrank, nproc); Page 7 of 21

8 7. communication.c part 2 broadcast π: if ( 0 == myrank ) pi = PI; else pi = 0.0; printf("proc %2d : before broadcast pi = %e\n", myrank, pi); MPI_Bcast( &pi, 1, MPI_DOUBLE, 0, MPI_COMM_WORLD ); printf("proz %2d : after broadcast pi = %e\n", myrank, pi); Page 8 of 21

9 8. communication.c part 3 determine neighbours: if ( 0!= myrank ) NeighbourID_le = myrank - 1; else NeighbourID_le = MPI_PROC_NULL; /* no neighbour */ if ( (nproz - 1)!= myrank ) NeighbourID_ri = myrank + 1; else NeighbourID_ri = MPI_PROC_NULL; /* no neighbour */ Page 9 of 21

10 9. communication.c part 4 send own id to neighbours: receive send own id to the left neighbour: id id+1 send MPI_Send( &myrank, 1, MPI_INT, NeighbourID_le, 1, MPI_COMM_WORLD ); receive id of the right neighbour from the right neighbour: MPI_Recv( &recv_neighbourid_ri, 1, MPI_INT, NeighbourID_ri, 1, MPI_COMM_WORLD, &status ); printf right neighbours to stdout: if ( MPI_PROC_NULL!= NeighbourID_ri ) printf("proc. %2d : ID right Neighbour = %2d\n", myrank, recv_neighbourid_ri); else printf("proc. %2d : ID right Neighbour = MPI_PROC_NULL\n", myrank); communicate left neighbours ids:... Page 10 of 21

11 10. communication.c part 5 alternative implementation based on MPI_Sendrevc: id id+1 receive send send own id to the left neighbour and receive id of the right neighbour from the right neighbour: MPI_SendRecv( &myrank, 1, MPI_INT, NeighbourID_le, 1, &recvneighbourid_ri, 1, MPI_INT, NeighbourID_ri, 1, MPI_COMM_WORLD, &status ); communicate left neighbours ids:... Page 11 of 21

12 11. communication.c part 6 finalizing: fflush(stdout); /* write all output buffers */ fflush(stderr); MPI_Barrier( MPI_COMM_WORLD ); /* synchronize all processes */ MPI_Finalize(); /* finalize MPI */ Page 12 of 21

13 12. Parallelization Strategy for Our Program decomposition of the overall domain into parts per process : Ω Ω Ω Ω 1,3 2,3 3,3 4,3 Ω Ω Ω Ω 1,2 2,2 3,2 4,2 Ω Ω Ω Ω 1,1 2,1 3,1 4,1 Page 13 of 21

14 degrees of freedom and boundary values for subdomains: j=jt j=jb i=il i=ir Page 14 of 21

15 13. initialization time loop compute_dt boundary_val compute_fg compute_rhs sor iteration loop sor update_uv output_uvp determine own rank, number of processes, subdomain size receive overall maximum values of u and v from the master process distinguish between interprocess and outer boundaries communicate pressure at interprocess boundaries, send parts of the residual to the master process, receive overall residual from the master process communicate velocities at interprocess boundaries, send maximum values of u and v to the master process output without baoundary values at interprocess boundaries Page 15 of 21

16 14. MPI_Init n processes after this call master process: computation of subdomains (il, ir, jb, jt, omg_i, omg_j) for all processes allocate fields for each process: P [il-1,ir+1] [jb-1,jt+1] U [il-2,ir+1] [jb-1,jt+1] V [il-1,ir+1] [jb-2,jt+1] F [il-2,ir+1] [jb-1,jt+1] G [il-1,ir+1] [jb-2,jt+1] RS [il,ir] [jb,jt] Page 16 of 21

17 15. Communication of Pressure Values Ω ip,jp Ω ip+1,jp Use MPI_SendRecv instead of MPI_Send and MPI_Recv as far as possible!!! Page 17 of 21

18 16. Communication of Velocities Ω ip,jp Ω ip+1,jp Use MPI_SendRecv instead of MPI_Send and MPI_Recv as far as possible!!! Page 18 of 21

19 17. Communication of Residuals and Maximum Values collecting values (partial residuals, local maximum values of u and v) to the master process and reducing them to a single value at the master process: MPI_Reduce(sendbuf, recvbuf, count, datatype, - op, root, comm) Sending values from the master process to all other processes: MPI_Bcast(buffer, count, datatype, root, comm) Page 19 of 21

20 18. List of Needed Functions, Types and Constants MPI_Init( &argc, &argv ) MPI_Finalize() MPI_Comm_size( comm, &nproz ) MPI_Comm_rank( comm, &myrank ) MPI_Barrier( comm ) MPI_Send( buf, count, datatype, dest, tag, comm ) MPI_Recv( buf, count, datatype, source, tag, comm, status) MPI_SendRecv( sendbuf, sendcount, sendtype, dest, sendtag, recvbuf, recvcount, recvtype, source, recvtag, comm, status) MPI_Reduce( sendbuf, recvbuf, count, datatype, op, root, comm ) MPI_SUM MPI_MAX Page 20 of 21

21 MPI_Bcast( buffer, count, datatype, root, comm ) MPI_COMM_WORLD MPI_DOUBLE MPI_PROC_NULL Page 21 of 21