HPC on Windows. Visual Studio 2010 and ISV Software

Transcription

1 HPC on Windows Visual Studio 2010 and ISV Software Christian Terboven / Aachen, Germany Stand: Version 2.3 Rechen- und Kommunikationszentrum (RZ)

2 Agenda Visual Studio at a Glance ISV Codes and the Batch System RZ: Christian Terboven Folie 2

3 Visual Studio at a Glance RZ: Christian Terboven Folie 3

4 VS2010 Overview Introduction into using Visual Studio 2010 Only text-mode programs are considered here, as HPC applications typically do not use GUIs VS2010 offers great support for GUI development on Windows VS2010 provides good support for Parallel Programming: Support for OpenMP for Shared-Memory parallel compilation Debugging of parallel programs: OpenMP and MPI Architecture-specific compiler optimizations The first start of VS2010 may take a few minutes as the help system has to be updated. You are prompted to choose from several predefined sets of VS2010 configuration options: I nowadays choose the C# settings (previously: C++). RZ: Christian Terboven Folie 4

5 Code Editor Solution Explorer + Other Views Tool Windows Here: Call Hierarchy RZ: Christian Terboven Folie 5

6 Visual Studio: Project Management (1/5) Everything that you do in Visual Studio will take place within the context of a Solution. A Solution is a higher-level container for other items, for example a Project. Any other kind of file type can also be added to a Solution, for example documentation items. A Solution can not contain another Solution. Solutions group and apply properties across projects. A Project maps one to one with a compiler target. A Project organizes the code. To start your work, a new Project has to be created with File New Project RZ: Christian Terboven Folie 6

7 Win32 Console Application: No GUI design support enabled. Visual C++ Win32 Win32 Console Application Name of the Project, A Solution of the same name is created as well Location where the folder structure is created RZ: Christian Terboven Folie 7

8 Visual Studio: Project Management (3/5) Choose Empty project if you already have source files. RZ: Christian Terboven Folie 8

9 Visual Studio: Project Management (4/5) An issue specific to our Cluster: The IntelliSense database may not be stored on a network drive. VS2010 resolves this automatically for you by selecting Ok. RZ: Christian Terboven Folie 9

10 Visual Studio: Project Management (5/5) Solution Project In many cases, the shortest way to a desired operation can be found by right-clicking on a GUI element and using the context menu. Adding existing source code items (files) to a project: right-click on the Project (not the Solution!) and Add Existing Item Adding new items: right-click on the Project and Add New Item The folders (e.g. Source Files) do not have any other meaning than aiding you in structuring the files in a project. They do not map to physical folders. Creating your own folders may help to organize large projects. RZ: Christian Terboven Folie 10

11 Source Navigation in Visual Studio 2010 (1/2) Selecting a scope + function to navigate right into it s implementation. Right-clicking a symbol opens up a corresponding context menu: RZ: Christian Terboven Folie 11

12 Source Navigation in Visual Studio 2010 (2/2) The Class View is available from the menu via View Class View as well as the Code Definition Window. RZ: Christian Terboven Folie 12

13 Directory Layout of Visual Studio Solutions The executable is created in the directory of the active configuration during the build process. Directory structure of a solution: <top level> Given user directory <project name> Created by VS2005 / VS2008 / VS2010 Debug Configuration: Debug Release Configuration: Release x64 Platform: x64 (64bit for Amd64/Intel64) Debug Configuration: Debug Release Configuration: Release RZ: Christian Terboven Folie 13

14 Visual Studio 2010: Performance Analyzer VS2010 comes with new tools to analyze your (parallel) application s performance: Analyze Profiler New Performance Session, then Analyze Launch Performance Wizard All this can be done on the local Workstation, or in the Cluster! CPU Sampling: Will run the application under the control of a sampling performance analyzer (snapshots of the program s call tree are taken at regular intervals nonintrusive, low overhead) RZ: Christian Terboven Folie 14

15 Performance Analyzer: CPU Sampling (1/2) Summary View highlights the program s Critical Path: RZ: Christian Terboven Folie 15

16 Performance Analyzer: CPU Sampling (2/2) Performance information can be display on source level: RZ: Christian Terboven Folie 16

17 Concurrency (1/3) RZ: Christian Terboven Folie 17

18 Concurrency (2/3) RZ: Christian Terboven Folie 18

19 Concurrency (3/3) RZ: Christian Terboven Folie 19

20 Performance Analyzer: Thread Contention (1/2) If threads compete for resources, they can get stalled: RZ: Christian Terboven Folie 20

21 Performance Analyzer: Thread Contention (2/2) Reason for this contention: OpenMP Barrier Support for OpenMP constructs is not yet optimal This analysis is crucial if you do your own synchronization! RZ: Christian Terboven Folie 21

22 Performance Analyzer: More future features... Performance tuning can be a never ending story, so you need metrics to decide where to work / when to stop: Hardware Counter Information. L2 information can be used to measure the memory bandwidth consumed by the application is your scalability limited by the system architecture? The FLOPS rate is good to estimate how efficient the code runs!... RZ: Christian Terboven Folie 22

23 Visual Studio Configurations (1/3) The set of compiler options is managed in a Configuration. There are two configurations pre-defined: Debug and Release. Debug: typical options for debugging, no optimization. Release: debugging still possible, some optimization options. The compile process can be triggered by right-clicking on the project and choosing Build. Or from the menu: Build Build <projectname>. Build Build Solution builds all projects in the solution. During and after the compile process compiler output (informational messages, warnings, errors) is displayed in the tool windows Output or Error List. By double-clicking on such a message, the cursor jumps to the corresponding place in the code. RZ: Christian Terboven Folie 23

24 Visual Studio Configurations (2/3) Right-clicking on a project and choosing Properties leads to the project configuration dialog. Cur. Configuration Cur. Platform RZ: Christian Terboven Folie 24

25 Visual Studio Configurations (3/3) Build Configuration Manager: Only Win32 and x64 are supported on our Cluster, not Itanium. You can create your own configurations. RZ: Christian Terboven Folie 25

26 Microsoft C/C++-specific Settings Important General Settings: C/C++ General Addition Include Directories: Include Path Linker General Additional Library Directories: Library Path Linker Input Additional Dependencies: Libraries to be used Important Optimization Settings: C/C++ Optimization Optimization: General Optimization Level Inline Function Expansion: Inlining C/C++ Code Generation Enable Enhanced Instruction Set: Vectorization RZ: Christian Terboven Folie 26

27 Using the Intel C/C++ Compiler 1. Create a (Win32) project using the Microsoft C/C++ compiler 2. Right-click on the Solution or on the Project 3. Intel Parallel Studio Installed: Intel Parallel Composer Use Intel C++ 4. Intel Parallel Studio Not Installed: Use Intel(R) C++ Solution or Project can be converted back to use Microsoft C/C++ as well. Currently only supported in Visual Studio 2008! RZ: Christian Terboven Folie 27

28 Intel C/C++-specific Settings Most settings are the same as with Microsoft C/C++, but: Additional Optimization Settings: C/C++ Optimization Generate Alternate Code Path and/or Use Intel(R) Processor Extensions: Optimization for specific CPU Parallelization: Ask the compiler for automatic parallelization C/C++ Language Recognize the Restrict Keyword: Enable C99 restrict (Tuning!) Enable C++0x Support: Enable first C++0x features Add to C/C++ Command Line and Linker Command Line: /Qtcheck: Enable source instrumentation for Thread Checker /Qtprofile: Enable source instrumentation for Thread Profiler RZ: Christian Terboven Folie 28

29 Using the Intel Fortran Compiler 1. Open the Project Creation dialog via File New Project 2. Select Intel(R) Fortran as project type 3. Typically select Empty Project as best-suited option Remaining project handling does not differ from C/C++! RZ: Christian Terboven Folie 29

30 Enabling OpenMP (1/3) OpenMP support has to be enabled in a configuration: OpenMP 2.0 / 2.5: VS2005 C/C++ VS2008 C/C++ VS2010 C/C++ OpenMP 3.0: Intel C/C++ Intel FORTRAN RZ: Christian Terboven Folie 30

31 Enabling OpenMP (2/3) Known problem with Visual Studio and OpenMP: The message appears if an OpenMP program has been compiled with OpenMP support enabled, but omp.h had not been included. Solution: include omp.h in at least one file per project. RZ: Christian Terboven Folie 31

32 Enabling OpenMP (3/3) Setting the number of threads for debugging of OpenMP programs: set environment variable OMP_NUM_THREADS. RZ: Christian Terboven Folie 32

33 Enabling MPI (1/2) As MPI is implemented by a library, an application includes a file containing the type and function declarations named mpi.h and has to be linked with that library. Modify the project properties (1/2): Include Path: C/C++ General Additional Include Directories MS-MPI 2008 on the cluster in Aachen: C:\Program Files\Microsoft HPC Pack 2008 SDK\Include I-MPI on the cluster in Aachen: C:\Program Files (x86)\intel\ict\3.1\mpi\3.1\[ia32 em64t]\include RZ: Christian Terboven Folie 33

34 Enabling MPI (2/2) Modify the project properties (2/2): Library Path: Linker General Additional Library Directories MS-MPI 2008 on the cluster in Aachen: C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\[i386 amd64] I-MPI on the cluster in Aachen: C:\Program Files (x86)\intel\ict\3.1\mpi\3.1\[ia32 em64t]\lib No significant performance difference, so our advise: Use MS-MPI with Visual Studio MPI Debugger Use I-MPI with Intel Trace Analyzer & Collector Sometimes a program does not like a specific MPI, so it is always a good thing to have a second one available RZ: Christian Terboven Folie 34

35 ISV Codes and the Batch System RZ: Christian Terboven Folie 35

36 ISV codes in the batch system (1/3) Several requirements No usage of graphical elements allowed No user input allowed (except redirected standard input) Execution of compiled set of commands, Termination Exemplary usage instructions for sequential Matlab job: Command line: \\cifs\cluster\software\matlab\ R2010b\bin\win64\matlab.exe /nodesktop /nosplash /logfile log.txt Save output /r "cd('\\cifs\cluster\home\your_userid\your_path'), YOUR_M_FILE Disable GUI elements The.M file should contain quit; as last statement Software share Change dir & Execute RZ: Christian Terboven Folie 36

37 ISV codes in the batch system (2/3) Exemplary usage instructions for sequential ANSYS job: 1. Create an Inputfile 2. Create a.cmd file containing the following lines setlocal set INPFILE=test.txt set OUTFILE=%INPFILE%.out set ANSCMD_NODIAG=TRUE net use x: %CCP_WORKDIR% x: call "\\cifs\cluster\\software\ansys_inc \v130\ansys\bin\winx64\ansys130.exe" -b nolist -j jobname -p aa_r -i %INPFILE% -o %OUTFILE% endlocal We have only two ANSYS parallel licenses. See examples: \\cifs\cluster\software\ansys_inc\v130\ansys\msccs RZ: Christian Terboven Folie 37

38 ISV codes in the batch system (3/3) Exemplary usage instructions for sequential ANSYS CFX job: Use the following command line with suited input file: \\cifs\cluster\software\"ansys Inc"\v130\CFX\bin\cfx5solve.exe -def input.def Parallel Jobs: Use either Solvermanager (GUI): 1. Specify Definition File 2. Run mode: Submit to CCS Queue 3. Use + to specify the number of cores (80 licenses) 4. Ignore Hostname 5. Take care: Result path \\cifs\cluster\documents\%username% is hard-coded or have full control (and knowledge) of what you are doing and adapt the job file on our homepage to your needs. RZ: Christian Terboven Folie 38

39 Please consult our Web Resources Slide decks of past events do not get updated! Please go to to check for upto-date instruction on how to use our batch system, especially with ISV codes. Currently we have detailed descriptions for: MATLAB, including parallel programming with MATLAB Abaqus Ansys / Ansys CFX Gaussian Starcd Possibly more in the future RZ: Christian Terboven Folie 39

40 The End Thank you for your attention. We are always looking for student workers: - You already have experiences in C/C++ or Fortran and got interested in Parallel Programming? - You wan to play with latest hardware from companies such as Intel, Nvidia, AMD, Sun,...? - Mail me: terboven@rz.rwth-aachen.de RZ: Christian Terboven Folie 40