Parallel Processing Experience on Low cost Pentium Machines

Transcription

1 Parallel Processing Experience on Low cost Pentium Machines By Syed Misbahuddin Computer Engineering Department Sir Syed University of Engineering and Technology, Karachi

2 Presentation Outline Introduction to Parallel Processing concepts Hardware and Software Architecture of PC cluster Demonstration of some simple applications on PC Cluster Remote Access to PC Cluster via Internet 2

3 Parallel Computing Parallel computing is the simultaneous use of multiple compute resources to solve a computational problem Parallel Programming is used to improve computational speed for a given problem In Parallel Computing, a problem is broken into discrete parts that can be solved concurrently 3

4 Some Areas of Parallel Computing Weather forecasting Numerical simulation of engineering and scientific problems Economic scenario analysis Data mining applications Modeling large DNA structures Speech recognition Image/Video Processing 4

5 Application Examples for Parallel Processing An oil exploration project can divide the seafloor into various areas. Nodes in a cluster can process the seismic data from these areas in parallel. An image can be broken into parts and given to cluster nodes for image recognition DNA sequence search algorithms can be mapped on cluster 5

6 Problem Decomposition for Parallel processing 6

7 Computing platforms for Parallel Computing A single computer with multiple Internal Processors Shared Memory multiprocessor model Multiple Interconnected computers Message passing Multi computer model 7

8 Shared Memory Multiprocessor Model Shared Memory Multiprocessor model is attractive due to data access convenience Hardware implementation for fast memory access is difficult 8

9 Shared Memory Model 9

10 Message passing Distributed Memory Multi- computer model or Cluster 10

11 Programming Models for Parallel Computers OpenMP for shared memory programming MPI (Message Passing Interface) for distributed memory programming 11

12 Advantage of Message Passing Cluster Model Off-the-shelf normal Intel PCs can be configured to form a Message Passing Multi computer model No special mechanism is needed for controlling simultaneous data access like shared memory multiprocessor model Clusters are very popular in universities and research labs for teaching and experimenting Parallel Processing concepts 12

13 PC cluster as Distributed Memory Sequential Applications Sequential Applications Sequential Applications Model Cluster Middleware Parallel Applications Parallel Applications Parallel Applications Parallel Programming Environment PC/Workstation PC/Workstation PC/Workstation PC/Workstation Communications Software Communications Software Communications Software Communications Software Network Interface Hardware Network Interface Hardware Network Interface Hardware Network Interface Hardware Cluster Interconnection Network/Switch 13

14 Hardware Architecture for PC Cluster Collection of Homogeneous or Heterogeneous Pentium machines with Network connectivity 14

15 Steps to Build PC Cluster LINUX Operating System installations on all nodes to be included in cluster NFS/NIS Server Configuration NFS/NIS Clients Configurations Installation of Parallel Programming System (PPS) Commonly used PPS are: PVM from University of Tennessee MPI from MPI forum BSP from Oxford University 15

16 Linux Cluster Practical Example Intranet Remote Web Clients

17 17

18 Message Passing Interface (MPI) MPI is a library of routines introduced by the Message- Passing Interface Forum in 1994 MPI is used for Parallel Processing on Cluster MPI allows transmitting data from one process running on one machine to another process running on a different machine The programmer is responsible for correctly identifying parallelism and implementing the resulting algorithm using MPI constructs MPI subprograms that can be invoked by C/C++ or Fortran 77 program 18

19 Six Main MPI routines 1. MPI_INIT: initializes the MPI environment 2. MPI_COMM_SIZE: returns the number of processes 3. MPI_COMM_RANK: returns this process's number (rank) 4. MPI_SEND: sends a message 5. MPI_RECV: receives a message 6. MPI_FINALIZE 19

20 MPI_COMM_WORLD The MPI communicator specifies a group of processes inside which a communication occurs In A MPI Communicator world, each process gets a RANK_ID MPI Communicator world is created over all machines in the system 20

21 21

22 Process control over the Cluster Rank_ID=0 Rank_ID=1 Rank_ID=2 22

23 A Parallel Computation Example Problem: F=(a-b)(a+b)(a*b) Suboperation1=(a-b) done by Machine1 and send partial result to Suboperation2=(a+b) done by Machine2 Suboperation3 =(a*b) done by Machine3 Each machine sends partial results to the head node or server Server computes F= (a-b)*(a+b)*(a*b) 23 23

24 Server computes (a-b)(a+b)(a*b) Server reads a,b anc d Server Node Tmp1=(a-b) Tmp2=(a+b) Tmp3=(a*b) Client Node1 Client Node2 Client Node

25 MPI PROGRAM 25

26 Popular MPI Implementation MPICH by Argonne National Lab and Mississippi State University Local Area Multicomputer MPI(LAM-MPI) by Ohio Supercomputer center CHIMP by Edinburgh Parallel Computing Center 26

27 LAM/MPI LAM/MPI is a high-quality open-source implementation of the Message Passing Interface specification LAM allows interconnected computers to act as one parallel computer for solving one compute intensive problem LAM features a full implementation of the MPI communication standard 27

28 Launching LAM on The Linux cluster lamboot command is used to launch LAM daemons on a Linux cluster LAM daemon running on each node provides process management, including signal handling and I/O management. A textfile contains the names of machines on LAM is to be launched $lamboot v machinefile LAM daemons are owned by the users There can be several parallel machines on same clusters 28

29 Running Programs on Cluster A special compiler called Handle-C compiler (hcc) is used to compile C codes for cluster $hcc o exec_file mpi_prog.c mpirun -np 3 exec_file 29

30 Example 1 Objective: Sample MPI Programs 1. Two client nodes send Greeting messages to the server node 2. Server node receives Greeting message and prints on display 30

31 Code for Progarm1 1. MPI_Init(&argc, &argv); 2. MPI_Comm_rank(MPI_COMM_WORLD, &my_rank); 3. MPI_Comm_size(MPI_COMM_WORLD, &p); 4. gherr = gethostname( hname, silen); 5. if (my_rank!= 0) { 6. /* Client code */ 7. sprintf(message, "Greetings from process %d on %s!", my_rank, hname); 8. dest = 0; 9. MPI_Send(message, strlen (message)+1, MPI_CHAR, dest, tag, MPI_COMM_WORLD); 10. } else { 11. /* Server code */ 12. printf ("Messages received by process %d on %s.\n\n", my_rank, hname); 13. for (source = 1; source < p; source++) { 14. MPI_Recv(message, 800, MPI_CHAR, source, tag, MPI_COMM_WORLD, &status); 15. printf("%s\n", message);} Presented in Open Source Series Workshop

32 Output of Program1 on Cluster 32

33 Example 2 Objective: To find sum of an Integer array Server node reads Integer array and sends it to clients Client 1 finds partial sum of first half array elements and sends partial sum to server Client 2 finds partial sum of second half array elements and sends partial sum to server Sever node adds partial sums to find total sum of input array 33

34 Output of Program2 on Cluster 34

35 Example 3: Matrix Multiplication on Cluster 35

36 Example 4 Numerical Integration on Cluster Each nodes computes the area of a slice of the curve and send the individual area to the server Serve computes the area of the curve by adding the individual areas. 36

37 Questions? 37