High Permance omputing on Windows Debugging with VS2005 Debugging parallel programs hristian Terboven enter RWTH Aachen University 1 HP on Windows - 2007 enter
Agenda Enabling OpenMP and MPI Debugging serial programs Debugging parallel OpenMP programs Debugging parallel MPI programs Short demo: Jacobi--Hybrid 2 HP on Windows - 2007 enter
Agenda Enabling OpenMP and MPI Debugging serial programs Debugging parallel OpenMP programs Debugging parallel MPI programs Short demo: Jacobi--Hybrid 3 HP on Windows - 2007 enter
100000 10000 1000 100 OpenMP: Motivation How capable is a current computer? A 3 GHz Intel Xeon PU is approximately as fast as all human beings together, if they compute one floating point operation per second. Moore s Law: Number of transistors doubles every 18 months. Wrong interpretation: The permance doubles every 18 months. The n cores of current and future processors have to be exploited by a program. 10 1 0,1 1975 1983 1991 1999 2003 2007 2003 4 HP on Windows - 2007 enter Source: Herb Sutter www.gotw.ca/publications/concurrency-ddj.htm
OpenMP: Overview Master Thread Slave Threads Slave Threads Slave Threads Serial Part Parallel Region An OpenMP program starts just like a serial program with one thread: the so-called Master thread. At a so-called Parallel Region slave threads are spawned, together with the master thread they m a Team. Between parallel regions, in the serial part of a program, the slave threads are put to sleep. oncept: Fork-Join 5 HP on Windows - 2007 enter Allows an incremental parallelization of an existing program!
Enabling OpenMP OpenMP support has to be enabled in a configuration: OpenMP-enabled compilers: VS2005 /++ Intel /++ Intel FORTRAN 6 HP on Windows - 2007 enter
Enabling OpenMP Known problem with Visual Studio 2005 and OpenMP: The message above appears, if an OpenMP program has been compiled with Visual Studio 2005 with OpenMP support enabled, but omp.h had not been included. Solution: include omp.h in at least one file per project. 7 HP on Windows - 2007 enter
Enabling OpenMP Setting the number of threads debugging of OpenMP programs: set environment variable OMP_NUM_THREADS. 8 HP on Windows - 2007 enter
Enabling MPI As MPI is implemented by a library, an application includes a file containing the type and function declarations namedmpi.h and has to be linked with that library. Modify the project properties: Add Include Path: /++ General Additional Include Directories, mpi.h resides in :\Program Files\Microsoft ompute luster Pack\Include Add Library Path: Linker General Additional Library Directories, msmpi.lib resides in :\Program Files\Microsoft ompute luster Pack\Lib\[i386 amd64] 9 HP on Windows - 2007 enter
Agenda Enabling OpenMP and MPI Debugging serial programs Debugging parallel OpenMP programs Debugging parallel MPI programs Short demo: Jacobi--Hybrid 10 HP on Windows - 2007 enter
VS2005: Debugging serial programs Navigating in a /++ project: By choosing a function of the current scope, the cursor jumps to the function definition. You can use the class view to navigate through your code as well. 11 HP on Windows - 2007 enter
VS2005: Debugging serial programs A breakpoint can be set by clicking in the grey area left of the line number. licking again removes the breakpoint. Right-clicking on a breakpint shows the context menu with the following functions Disable a breakpoint (temporary) Set breakpoint trigger conditions Trigger filter selected threads or processes Define actions to be executed when the breakpoint is triggered Just hold the mouse over a variable a short moment to get the actual value displayed (hover). This is also possible marked expression (to some extent). 12 HP on Windows - 2007 enter
VS2005: Debugging serial programs During a debugging session, the actual program location is marked by a yellow arrow. You can drag this arrow up and down. 13 HP on Windows - 2007 enter
Agenda Enabling OpenMP and MPI Debugging serial programs Debugging parallel OpenMP programs Debugging parallel MPI programs Short demo: Jacobi--Hybrid 14 HP on Windows - 2007 enter
VS2005: Debugging OpenMP programs Debugging of OpenMP applications in Visual Studio 2005 works with all compilers: the Microsoft /++ compiler, the Intel /++ compiler and the Fortran compiler. Note: If you use the Intel compiler and let a program run with four threads, you will see five threads (one management thread). We advise you to compile without any optimization debugging, that means use the pre-configure Debug configuration and just enable OpenMP. ontrol debugging: Start / ontinue, Break, Stop, Restart Show next statement, Step Into, Step Over, Step Out 15 HP on Windows - 2007 enter
VS2005: Debugging OpenMP programs All threads stop at a breakpoint the moment one thread reaches it. Using the Threads register, you can select all threads. If you want a thread to stand still while further debugging, you have to freeze it. 16 HP on Windows - 2007 enter
Visual Studio 2005 Debuggen von OpenMP-Anwendungen For all threads, you can view the local and shared variables. The Locals register contains all variables of the current scope. The Autos register contains a set of interesting variables guessed by the compiler remarkably good. Some limitations when using the Intel Fortran compiler: The Autos register is empty, Locals is working fine. One can not (at least sometimes) identify the management thread by the name that is the one you get an error message that no source code is available if you select it ;-) Sometimes expressions have to be updated, because of (possible) compiler optimization. 17 HP on Windows - 2007 enter
Agenda Enabling OpenMP and MPI Debugging serial programs Debugging parallel OpenMP programs Debugging parallel MPI programs Short demo: Jacobi--Hybrid 18 HP on Windows - 2007 enter
VS2005: Debugging MPI programs Typically Microsoft MPI works best, but you should be able to use MPIH2 as well. At leasts the following instructions work both. Visual Studio 2005 supports debugging of MPI programs using the luster Debugger. As far as I know or was able to verify the cluster debugger only works with the Microsoft /++ compiler and not with projects using the Intel /++ compiler or the Intel FORTRAN compiler. In the project properties under Debugging, choose the MPI luster Debugger as Debugger to launch. MPIRun: c:\program files\microsoft compute cluster pack\bin\mpiexec MPIRun Arguments: example n 2 MPIShim Location: It is not possible to specify a path containing empty spaces here, so you have to copy MPIShim from c:\program files[ (x86)]\microsoft visual studio 8 \common7\ide\remotedebugger\x86[or x64]\mpishim to a suitable location. 19 HP on Windows - 2007 enter
VS2005: Debugging MPI programs In order to stop all processes at a breakpoint, please check the following option: In Tools Options Debugging General the checkbox Break all processes when one process breaks has to be activated. Select the current process using the Processes register. Problem: if one process make a step, all other processes make a step as well. Solution 1: Be clever setting breakpoints Solution 2: Detach the second (and third and fourth and ) process from the VS2005 debugger (think at the right-click and context menu), open up another VS2005 instance (maybe more) and attach to the process. Going the second way, VS2005 becomes the most powerful parallel debugger available today (and I tried nearly all debuggers)! 20 HP on Windows - 2007 enter
VS2005: Debugging MPI programs Master Slave 21 HP on Windows - 2007 enter
Agenda Enabling OpenMP and MPI Debugging serial programs Debugging parallel OpenMP programs Debugging parallel MPI programs Short demo: Jacobi--Hybrid 22 HP on Windows - 2007 enter
End of this part Questions? 23 HP on Windows - 2007 enter