QLogic InfiniBand Specialist Lab Working with MPI Examples

Size: px

Start display at page:

Download "QLogic InfiniBand Specialist Lab Working with MPI Examples"

Winfred Morris
6 years ago
Views:

1 QLogic InfiniBand Specialist Lab Working with MPI Examples Allotted time for this lab: 45 minutes Important: For this lab, you will be using a number of MPI examples to demonstrate the use of a cluster. June 2010 Copyright QLogic Corporation All rights reserved.

2 Table of Contents Objective... 3 Locating the MPI Examples... 4 Running the OSU MPI Latency Test (version 2)... 4 Running the OSU MPI Bandwidth Test (version 2)... 5 Running the HPL (High Performance Linpack) Benchmark... 5 Running the Parallel Mandelbrot (pmandel) Demo... 7 June 2010 Copyright QLogic Corporation All rights reserved. 2

Objective Lab 202.07 At the end of this exercise you will have located, configured, and executed a number of MPI examples.

3 Objective Lab At the end of this exercise you will have located, configured, and executed a number of MPI examples. Lab Area Equipment Each vlab area will have similar equipment, differing only in IP Address of the devices. Refer to your Lab Schematic for details. June 2010 Copyright QLogic Corporation All rights reserved. 3

Locating the MPI Examples 1. By default, the MPI Examples are installed in the /opt/iba/src/mpi_apps directory. Change to the directory and list the directory contents.

4 Locating the MPI Examples 1. By default, the MPI Examples are installed in the /opt/iba/src/mpi_apps directory. Change to the directory and list the directory contents. # cd /opt/iba/src/mpi_apps # ls 2. Most of the examples require a list of hosts available. Create a file called mpi_hosts and list all the hosts in your cluster making sure to list the IBoIP hostnames. # vi mpi_hosts Sample mpi_hosts file where X is your lab area number 3. When finished, save the file. Running the OSU MPI Latency Test (version 2) The Latency Test will show the latency for numerous message sizes ranging from 1B to 4M. 4. Run the test with the following command: #./run_lat2 The latency test is carried out in a ping-pong fashion. The sender sends a message with a certain data size to the receiver and waits for a reply from the receiver. The receiver receives the message from the sender and sends back a reply with the same data size. This is done a number of times and the average one-way latency numbers are reported. June 2010 Copyright QLogic Corporation All rights reserved. 4

5 5. What message size delivered the lowest latency? Running the OSU MPI Bandwidth Test (version 2) The Bandwidth Test will show the bandwidth (MB/s) for various message sizes ranging from 1B to 4M. 6. Run the test with the following command: #./run_bw2 Take note of the bandwidth for the various message sizes. 7. At what message size does the bandwidth appear to level off? Refer to the message size from step 5. What is the bandwidth at that message size? Are the message sizes for best latency and best bandwidth the same? In most cases, the message size that delivers the best in latency is not the same as the message size that delivers the best in bandwidth. This is the classic trade off: latency vs bandwidth. The best message size will depend on the type of MPI application being used: bandwidth or latency dependant. Running the HPL (High Performance Linpack) Benchmark The High Performance Linpack Benchmark is the standard test used when ranking the Top500 (top500.org) supercomputers in the world. It is a software package that solves a random, dense linear system of equations. 8. To properly run the HPL, you need to configure it for your environment. Run the config_hpl script to get a list of the available configurations. #./config_hpl You should see a list of available configurations. The number in the configuration name is the number of nodes the test will be distributed to. The letter after the number represents the size of the test. (Example: t = tiny ) June 2010 Copyright QLogic Corporation All rights reserved. 5

9. Configure the benchmark for the smallest test on a single node with the following. This will configure HPL to run only on your head node. #./config_hpl 1t 10.

6 9. Configure the benchmark for the smallest test on a single node with the following. This will configure HPL to run only on your head node. #./config_hpl 1t 10. Run the benchmark with the following: #./run_hpl 1 After about 15-20s, the results should appear. Take note of the total time it took to solve the linear system and how many Gflops the 1 node cluster was capable of. Example: Solution was reached in seconds with rate of execution of Gflops. N = order of the coefficient matrix A P = number of process rows Time = seconds to solve linear system NB = partitioning block factor Q = number of process columns Gflops = rate of execution for solving 11. Fill in the chart below by configuring and running the following tests. Remember to run config_hpl with the proper parameter before running the test. Test Time Gflops 1T 2T 4T June 2010 Copyright QLogic Corporation All rights reserved. 6

7 Running the Parallel Mandelbrot (pmandel) Demo The Parallel Mandelbrot Demo is a graphical demo that renders a Mandelbrot Set (a set of points in the complex plane whose boundary forms a fractal) with the ability to zoom into the image. Each node of the cluster is given a particular region to generate. The more nodes, the faster the image will render. In this example, you will not use the Fast Fabric TUI. 12. From the /opt/iba/src/mpi_apps directory, perform a wget to download the pmandel.tgz, extract it and change directory to the newly created mandel directory. # wget # tar xvfz pmandel.tgz # cd mandel 13. Before performing a make, you will need to edit the pmandel.c file. Open it in vi: # vi pmandel.c 14. Locate the if statement that references the DISPLAY environment variable (on line 64) TIP: In vi, use 64G to jump down to line 64. Or search for DISPLAY Add the following line above the if (!getenv( DISPLAY )) line, where X is your lab area. Make sure you add it properly, including the ending semicolon: setenv( DISPLAY, qlogicxa-ib:0.0,1); June 2010 Copyright QLogic Corporation All rights reserved. 7

8 Where X is your lab number. You should have something that looks like this: 16. Save the file and exit from vi. 17. Perform the make. You can ignore any warnings that appear during the process. # make 18. Confirm the make was successful by locating the pmandel executable in the directory listing. # ll pmandel You should see something similar to the below. If you do not have a pmandel executable, contact your instructor. -rwxr-xr-x 1 root root Dec 10 05:25 pmandel 19. Copy the pmandel file to each node in your cluster. You will use Fast Fabric command line tool to do this. The first step is to create the target directory on all nodes by using the following cmdall command: # cmdall f /etc/sysconfig/iba/hosts mkdir /opt/iba/src/mpi_apps/mandel This will run the mkdir command on all the nodes listed in your hosts files. June 2010 Copyright QLogic Corporation All rights reserved. 8

20. Now, copy the pmandel executable to all nodes with the following scpall command: # scpall f /etc/sysconfig/iba/hosts pmandel /opt/iba/src/mpi_apps/mandel/pmandel 21.

9 20. Now, copy the pmandel executable to all nodes with the following scpall command: # scpall f /etc/sysconfig/iba/hosts pmandel /opt/iba/src/mpi_apps/mandel/pmandel 21. The default settings of the OS will prevent the other nodes from directing graphical output to the head node s display. Use gdmsetup to enable this. # gdmsetup From the Security Tab, uncheck the Deny TCP connections to Xserver and then click the Close button. 22. This change is not immediately applied. You will need to restart the GDM (Gnome Desktop Manager). Use the following command. As the GDM is restarted, your VNC session will be terminated. # gdm-restart 23. After a few seconds, reopen a VNC session to your Head Node. Once connected, open a terminal session and su over to the root s login shell. Then change directory back to the mpi_apps. June 2010 Copyright QLogic Corporation All rights reserved. 9

# cd /opt/iba/src/mpi_apps/mandel 24. Before running the demo, enable server access for X to the head node with: # xhost + 25. Now, try running the pmandel demo with only one node: #./run_mandel 1 26.

10 # cd /opt/iba/src/mpi_apps/mandel 24. Before running the demo, enable server access for X to the head node with: # xhost Now, try running the pmandel demo with only one node: #./run_mandel What happened? The app should have failed, as the test requires at least 2 nodes. 27. Now try running it with 2 nodes. A window with the title MPE should open showing a default Mandelbrot set. #./run_mandel 2 Demo Mandelbrot Set 28. Using the mouse, click and drag a zoom region. When you release the mouse the image will be redrawn. June 2010 Copyright QLogic Corporation All rights reserved. 10

11 Zoomed in. 29. Take note of the render time for the zoom. Close the app by closing the MPE window. (There is no gentle way to shut the app down) 30. Now, try running the pmandel demo with 3 and 4 nodes. You should note the renders appear to be quicker. #./run_mandel 3 #./run_mandel 4 Note: This is a classic example of parallel computing, where a problem can be simply broken into parts and handed off to multiple nodes. You have completed the lab exercise. Notify the instructor that you are done. June 2010 Copyright QLogic Corporation All rights reserved. 11

Fibre Channel Specialist FC Lab

Fibre Channel Specialist FC Lab Fibre Channel Specialist FC 201.01 Lab Getting to Know Your SAN Allotted time for this lab: 10 minutes Objective: This exercise is to introduce the lab equipment and methodology that will be used for the