Klaus-Dieter Oertel, May 28 th 2013 Software and Services Group Intel Corporation

Transcription

1 S c i c o m P T u t o r i a l Intel Xeon Phi Product Family Programming Tools Klaus-Dieter Oertel, May 28 th 2013 Software and Services Group Intel Corporation

2 Agenda Intel Parallel Studio XE 2013 Overview Intel Composer XE - Heterogenous Compiler - Offload - Native Intel Vtune Amplifier XE 2

3 Intel C/C++ and Fortran Compilers w/openmp* Intel MKL, Intel Cilk Plus, Intel TBB, and Intel IPP Intel Inspector XE, Intel VTune Amplifier XE, Intel Advisor XE + Intel MPI Library + Intel Trace Analyzer and Collector Intel Parallel Studio XE 3

4 Intel Parallel Studio XE 2013 Phase Productivity Tool Feature Benefit Advanced Parallel Design Intel Advisor XE Analyze existing code base and find opportunities for parallelization. Easier analysis and performance heuristics, find compute hotspots and check for parallelization strategies. Advanced Build and Debug Intel Composer XE C/C++ and Fortran compilers, performance libraries, and parallel models Application performance, scalability and quality for current multicore and future many-core systems. Advanced Verify Intel Inspector XE Memory & threading error checking tool for higher code reliability & quality Increases productivity and lowers cost, by catching memory and threading defects early Advanced Tune Intel VTune Amplifier XE Performance Profiler to optimize performance and scalability Removes guesswork, saves time, makes it easier to find performance and scalability bottlenecks Combines ease of use with deeper insights. 4 5/28/2013

5 Support for Latest Intel Processors and Coprocessors Intel Ivy Bridge microarchitecture Intel Haswell microarchitecture Intel Xeon Phi coprocessor Intel C++ and Fortran Compiler AVX AVX2, FMA3 IMCI Intel TBB library Intel MKL library AVX AVX2, FMA3 Intel MPI library Intel VTune Amplifier XE Hardware Events Hardware Events Hardware Events Intel Inspector XE Memory & Thread Checks Memory & Thread ( ) (Memory & Thread) 5

6 Intel Advisor XE Tool for what-if analysis Modeling: use code annotations to introduce parallelism Evaluation: estimate the effect e.g. the speedup GUI-driven assistant (5 steps) Productivity and Safety Parallel correctness is checked based on a correct program Non-intrusive API It s not auto-parallelization It s not modifying the code 6 5/28/2013

7 Intel Inspector XE 2013 Dynamic Analysis: Finds Memory and Threading Errors Find and eliminate errors Memory leaks, invalid access Races & deadlocks Analyze hybrid MPI cluster apps Heap growth analysis Faster & Easier to use Debugger breakpoints Break on selected errors Run faster to known error Pause/resume collection Narrow analysis focus Better performance Improved error suppression Find errors early (when they are less expensive). 7 5/28/2013

8 Agenda Intel Parallel Studio XE 2013 Overview Intel Composer XE - Heterogenous Compiler - Offload - Native Intel Vtune Amplifier XE 9

9 Heterogeneous Compiler Compiler compiles for host and Xeon Phi in one step by default Actually two compilations are executed But only if there is offload source code Two object files are created myfile.o and myfilemic.o Linker output creates one fat binary which includes host and Xeon Phi code 11 5/28/2013

10 Heterogeneous Compiler Commandline option things to know openmp is automatically set when you build Don t need no-offload if compiling only for Xeon Generates same Xeon only code as previous compilers But no-offload creates smaller binaries Most command line arguments set for the host are set for the coprocessor build Unless overridden by offload-option,mic,xx= Add watch=mic-cmd to display the compiler options automatically passed to the offload compilation 13

11 Heterogeneous Compiler Commandline options Offload-specific arguments to the Intel Compiler: Generate host only code (by default both host+coprocessor code is generated): -no-offload Produce a report of offload data transfers at compile time (not runtime) -opt-report-phase=offload Add Intel Xeon Phi compiler switches -offload-options,mic,compiler, switches Add Intel Xeon Phi assembler switches -offload-options,mic,as: switches Add Intel Xeon Phi linker switches -offload-options,mic,ld, switches Example: icc -I/my_dir/include -DMY_DEFINE=10 -offload-options,mic,compiler, -I/my_dir/mic/include -DMY_DEFINE=20 hello.c Passes -I/my_dir/mic/include -I/my_dir/include -DMY_DEFINE=10 - DMY_DEFINE=20 to the offload compiler 14

12 Heterogeneous Compiler Conceptual Transformation main() { f(); } Source Code f() { #pragma offload a = b + g(); } Linux* Host Program main() { copy_code_to_coproc(); f(); unload_coproc(); } f() { if (coproc_available()){ send_data_to_coproc(); start f_part_coproc(); receive_data_from_coproc(); } else f_part_host(); } Intel Xeon Phi Program This all happens automatically when you issue a single compile command attribute ((target(mic))) g() { } f_part_host() {a = b + g();} g() { } f_part_coproc() {a = b + g_coproc();} g_coproc() { } 15

13 Heterogeneous Compiler Conceptual Transformation main() { f(); } Source Code f() { #pragma offload a = b + g(); } Linux* Host Program main() { copy_code_to_coproc(); f(); unload_coproc(); } f() { if (coproc_available()){ send_data_to_coproc(); start f_part_coproc(); receive_data_from_coproc(); } else f_part_host(); } Intel Xeon Phi Program This all happens automatically when you issue a single compile command attribute ((target(mic))) g() { } f_part_host() {a = b + g();} g() { } f_part_coproc() {a = b + g_coproc();} g_coproc() { } 16

14 Heterogeneous Compiler Conceptual Transformation main() { f(); } Source Code f() { #pragma offload a = b + g(); } Linux* Host Program main() { copy_code_to_coproc(); f(); unload_coproc(); } f() { if (coproc_available()){ send_data_to_coproc(); start f_part_coproc(); receive_data_from_coproc(); } else f_part_host(); } Intel Xeon Phi Program This all happens automatically when you issue a single compile command attribute ((target(mic))) g() { } f_part_host() {a = b + g();} g() { } f_part_coproc() {a = b + g_coproc();} g_coproc() { } 17

15 Heterogeneous Compiler Conceptual Transformation main() { f(); } Source Code f() { #pragma offload a = b + g(); } Linux* Host Program main() { copy_code_to_coproc(); f(); unload_coproc(); } f() { if (coproc_available()){ send_data_to_coproc(); start f_part_coproc(); receive_data_from_coproc(); } else f_part_host(); } Intel Xeon Phi Program This all happens automatically when you issue a single compile command attribute ((target(mic))) g() { } f_part_host() {a = b + g();} g() { } f_part_coproc() {a = b + g_coproc();} g_coproc() { } 18

16 Fortran Support is Different Main method for parallelism is OpenMP* Use of library calls such as Intel MKL Intel MPI Offload with implicit copying is not supported no _Cilk_shared or _Cilk_offload keywords no simple way to offload non-sequence derived types Use explicit copy with offload directives!dir$ OFFLOAD Cannot use length parameter to offload part of an array; create a Fortran pointer and use that e.g.!dir$ OFFLOAD IN(FPTR:length(n):free_if(.false.)) Otherwise, much the same as C 19

17 Heterogeneous Compiler Reminder of What is Generated Note that for both techniques, the compiler generates two binaries: The host version includes all functions/variables in the source code, whether marked #pragma offload, attribute ((target(mic))), _Cilk_shared, _Cilk_offload, or not The coprocessor version includes only functions/variables marked #pragma offload, attribute ((target(mic))), _Cilk_offload, or _Cilk_shared in the source code Linkage creates one executable with both binaries included! 20

18 Agenda Intel Parallel Studio XE 2013 Overview Intel Composer XE - Heterogenous Compiler - Offload - Native Intel Vtune Amplifier XE 23

19 Intel Xeon Phi Coprocessor Native Compilation Purpose: Build standalone programs for execution directly on coprocessor Create coprocessor-specific libraries for use by offloaded code sections Use: Invoke the compiler with mmic to generate purely native code Caveats without NFS mounted file systems on the coprocessor: Standalone programs and their data need to be copied manually by the user using scp, ftp or copied using an offload proxy Shared libraries, such as libiomp5.so (which has no static counterpart) may need to be copied manually, even if you link your program statically. Performance analysis procedure is unchanged from offload code Debugging requires that you manually attach to the program while it runs on the coprocessor 24

20 Intel Xeon Phi Coprocessor Native Compilation - Example $ icc -mmic -openmp omp_app.cpp $ scp a.out myhost-mic0:/tmp/ a.out 100% 111KB 111.5KB/s 00:00 $ scp /opt/intel/composer_xe_2013/compiler/lib/mic/libiomp5.so myhostmic0:/tmp/ libiomp5.so 100% 939KB 939.2KB/s 00:01 Login to the card, then run the program # ssh myhost-mic0 # cd /tmp # ls a.out libiomp5.so # export LD_LIBRARY_PATH=/tmp #./a.out Testarg A big OpenMP hello to Testarg from 244 threads! 25

21 Agenda Intel Parallel Studio XE 2013 Overview Intel Composer XE - Heterogenous Compiler - Offload - Native Intel Vtune Amplifier XE 28

22 Start with host-based profiling to identify vectorization/ parallelism/ offload candidates Start with representative/reasonable workloads! Use Intel VTune Amplifier XE to gather hot spot data Tells what functions account for most of the run time Often, this is enough But it does not tell you much about program structure Alternately, profile functions & loops using Intel Composer XE Build with options -profile-functions -profile-loops=all -profileloops-report=2 Run the code (which may run slower) to collect profile data Look at the resulting dump files, or open the xml file with the data viewer loopprofileviewer.sh located in the compiler./bin directory Tells you which loops and functions account for the most run time how many times each loop executes (min, max and average) 29

23 Intel VTune Amplifier XE offers a rich GUI Menu and Tool bars Analysis Type Viewpoint currently being used Tabs within each result Grid area Current grouping Stack Pane Timeline area Filter area 30

24 Intel VTune Amplifier XE on Intel Xeon Phi coprocessors Adjust Data Grouping (Partial list shown) No Call Stacks Yet Double Click Function to View Source Filter by Timeline Selection (or by Grid Selection) Filter by Module & Other Controls 31

25 Collect events in Intel VTune Amplifier XE on offload and native program executions Application settings: Application: ssh Parameters: mic0 <app startup> Working directory: Usually does not matter Don t forget to set search directories under All files to map source 33

26 Coprocessor collections have their own analysis types 34

27 Performance Analysis Intel VTune Amplifier XE only collects Lightweight Hotspots for Intel Xeon Phi coprocessors Uses Event-Based Sampling - Uses the Performance Monitoring Unit No instrumentation ( Locks & Waits, Concurrency, etc.) More to come! Other analysis types need to be configured Use Lightweight Hotspots and create a copy of it Add the desired counters Add some useful name and documentation Multi-event collections (over 2) can multiplex or use multiple runs 35

28 Configuring a User-defined Analysis 36

29 Event collections on the coprocessor can generate volumes of data dgemm: on 60+ cores Tip: Use cpu-mask to reduce data set, while maintaining the same accuracy. 38

30 Backup 39

31 Agenda Intel Parallel Studio XE 2013 Overview Intel Composer XE - Heterogenous Compiler - Offload - Native - Math Kernel Library Intel Vtune Amplifier XE 40

32 41

33 Intel Math Kernel Library (Intel MKL) Support for Intel Xeon Phi Coprocessors Intel MKL 11.0 supports the Intel Xeon Phi coprocessors. Heterogeneous computing Takes advantage of both multicore host and many-core coprocessors. Optimized for wider (512-bit) SIMD instructions and threaded for many cores. All Intel MKL functions are supported: But optimized at different levels. Pairing highly parallel software with highly parallel hardware. 42

34 Usage Models on Intel Xeon Phi Coprocessors Automatic Offload No code changes required Automatically uses both host and target Transparent data transfer and execution management Compiler Assisted Offload Explicit controls of data transfer and remote execution using compiler offload pragmas/directives Can be used together with Automatic Offload Native Execution Uses the coprocessors as independent nodes Input data and binaries are copied to targets in advance 44

35 Automatic Offload (AO) Offloading is automatic and transparent. Can take advantage of multiple coprocessors. By default, Intel Math Kernel Library decides: When to offload Work division between host and targets Users enjoy host and target parallelism automatically. Users can still specify work division between host and target. (for BLAS only) 45

36 How to Use Automatic Offload Using Automatic Offload is easy Call a function: mkl_mic_enable() or Set an env variable: MKL_MIC_ENABLE=1 What if there doesn t exist a coprocessor in the system? Runs on the host as usual without penalty! 46

37 Automatic Offload Enabled Functions A selected set of Intel Math Kernel Library functions are AO enabled. Only functions with sufficient computation to offset data transfer overhead are subject to AO In , AO enabled functions include: Level-3 BLAS:?GEMM,?TRSM,?TRMM,?SYMM LAPACK 3 amigos: LU, QR, Cholesky Offloading happens only when matrix sizes are right?gemm: M, N > 2048, K > 256?SYMM: M, N > 2048?TRSM/?TRMM: M, N > 3072 LU: M, N >

38 Work Division Control in Automatic Offload Examples Notes mkl_mic_set_workdivision( MKL_TARGET_MIC, 0, 0.5) Offload 50% of computation only to the 1 st card. Examples Notes MKL_MIC_0_WORKDIVISION=0.5 Offload 50% of computation only to the 1 st card. Work division settings have no effects for LAPACK functions. 48

39 Compiler Assisted Offload (CAO) Offloading is explicitly controlled by compiler pragmas or directives. All Intel Math Kernel Library (Intel MKL) functions can be offloaded in CAO. In comparison, only a subset of Intel MKL is subject to AO. Can leverage the full potential of compiler s offloading facility. More flexibility in data transfer and remote execution management. A big advantage is data persistence: Reusing transferred data for multiple operations. 49

40 How to Use Compiler Assisted Offload An example in Fortran:!DEC$ ATTRIBUTES OFFLOAD : TARGET( MIC ) :: SGEMM!DEC$ OMP OFFLOAD TARGET( MIC ) &!DEC$ IN( TRANSA, TRANSB, M, N, K, ALPHA, BETA, LDA, LDB, LDC ), &!DEC$ IN( A: LENGTH( NCOLA * LDA )), &!DEC$ IN( B: LENGTH( NCOLB * LDB )), &!DEC$ INOUT( C: LENGTH( N * LDC ))!$OMP PARALLEL SECTIONS!$OMP SECTION CALL SGEMM( TRANSA, TRANSB, M, N, K, ALPHA, & A, LDA, B, LDB BETA, C, LDC )!$OMP END PARALLEL SECTIONS 51

41 Native Execution Use the coprocessor as an independent compute node. Programs can be built to run only on the coprocessor by using the mmic build option. Intel Math Kernel Library function calls inside an offloaded code region executes natively. Better performance if input data is already available on the coprocessor, and output is not immediately needed on the host side. 53

42 Considerations of Using Intel Math Kernel Library on Intel Xeon Phi Coprocessors High level parallelism is critical in maximizing performance. BLAS (Level 3) and LAPACK with large problem size get the most benefit. Scaling beyond 100 s threads, vectorized, good data locality Minimize data transfer overhead when offload. Offset data transfer overhead with enough computation. Exploit data persistence: CAO to help! You can always run on the host if offloading does not offer better performance. 54

43

44 Legal Disclaimer & Optimization Notice INFORMATION IN THIS DOCUMENT IS PROVIDED AS IS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. Copyright, Intel Corporation. All rights reserved. Intel, the Intel logo, Xeon, Xeon Phi, Core, VTune, and Cilk are trademarks of Intel Corporation in the U.S. and other countries. Optimization Notice Intel s compilers may or may not optimize to the same degree for non-intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision # Copyright 2013, 2012, Intel Corporation. All rights reserved.