Intel Many Integrated Core (MIC) Programming Intel Xeon Phi

Transcription

1 Intel Many Integrated Core (MIC) Programming Intel Xeon Phi Dmitry Petunin Intel Technical Consultant 1

2 Legal Disclaimer & INFORMATION IN THIS DOCUMENT IS PROVIDED AS IS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing. For more information on performance tests and on the performance of Intel products, reference Copyright 2012, Intel Corporation. All rights reserved. Intel, the Intel logo, Xeon, Core, Phi, VTune, and Cilk are trademarks of Intel Corporation in the U.S. and other countries. *Other names and brands may be claimed as the property of others. Intel s compilers may or may not optimize to the same degree for non-intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. revision #

3 Agenda Introduction Data Access Semantics Explicit Offloading Implicit Offloading Examples 3

4 Agenda Introduction Data Access Semantics Explicit Offloading Implicit Offloading Examples 4

5 Architecture Overview PCI Core L2 Core L2 Core L2 Core L2 GDDR MC GDDR MC GDDR MC GDDR MC L2 Core L2 Core L2 Core L2 Core >50 cores 8GB GDDR5 Memory 8 memory controllers, 16 GDDR5 channels, up to 5.5GT/s PCIe Gen2 (Client) x16 per direction ECC 5

6 Architecture Overview Core Instruction Decode Scalar Unit Scalar Registers 32k ICache 32k DCache Vector Unit 512K L2 Cache Vector Registers 4 thread Interleave Pentium scalar instruction set (x87!) Fully functional In order-operation Full 64bit addressing 4 HW threads/core 2 cycle decoder 2 issue Two pipelines: Scalar Vector/Scalar 6

7 Architecture Overview ISA/Registers Standard Intel64 Registers (EM64T) rax rbx rcx rdx rsi rdx rsp rbp r8 r9 r10 r11 r12 r13 r14 r bit SIMD Registers: zmm0 zmm mask registers (16bit wide) k0 (special, don t use) k7 No xmm(sse/128bit) and ymm(avx/256bit) registers! x87 present 7

8 Architecture Overview Vector Instructions Vector Instruction Format 3 operand form with explicit destination register instruction destination, source1, source2 Source registers are not destroyed Very compact code (Most) MIC instructions can be masked instruction destination {mask}, source1, source2 Result of masking is non-destructive, i.e. destination values are preserved Example: vaddps zmm1{k1},zmm2,zmm3 dest mask source1 source2 8

9 Knights Corner Architecture Overview Software Architecture IA Benefit: Wide Range of Development Options Parallelization Options Intel Math Kernel Library MPI* Vector Options Intel Math Kernel Library Auto vectorization Ease of use OpenMP* Semi-auto vectorization: #pragma (vector, ivdep, simd) Intel Threading Building Blocks Intel Cilk Plus Array Notation: Intel Cilk Plus C/C++ Vector Classes (F32vec16, F64vec8) OpenCL* Pthreads* Intrinsics Fine control

10 Agenda Introduction Data Access Semantics Explicit Offloading Implicit Offloading Examples 10

11 Spectrum of Programming Models and Mindsets Xeon Multi-Core Centric Multi-Core Hosted General purpose serial and parallel computing Symmetric Codes with balanced needs Many-Core Centric MIC Many Core Hosted Highly-parallel codes Offload Codes with highlyparallel phases ulti-core Main( ) Foo( ) MPI_*( ) Main( ) Foo( ) MPI_*( ) Main( ) Foo( ) MPI_*( ) (Xeon) any-core Foo( ) Main( ) Foo( ) MPI_*( ) Main( ) Foo( ) MPI_*( ) (MIC) Range of models to meet application needs 11

12 Options for Offloading Application Code Intel Parallel Studio XE 2013 for MIC supports three models: Automatic offload o Only trigger offload when a MIC device is present o Offload supported in MKL library Offload pragmas o Only trigger offload when a MIC device is present o Safely ignored by non-mic compilers Offload keywords o Only trigger offload when a MIC device is present o Language extensions, need conditional compilation to be ignored Offloading and parallelism is orthogonal Offloading only transfers control to the MIC devices Parallelism needs to be exploited by a second model (e.g. OpenMP*) 12

13 Automatic offload with Math Kernel Library Intel Math Kernel Library (Intel MKL) void foo() /* Intel Math Kernel Library */ { float *A, *B, *C; /* Matrices */ sgemm(&transa, &transb, &N, &N, &N, &alpha, A, &N, B, &N, &beta, C, &N); } Implicit automatic offloading requires no code changes, simply link with the offload MKL Library Intel Xeon processor Intel Xeon Phi coprocessor Intel High Performance Math Kernel Library is Applicable to Multicore and Many-core Programming 13 3/2/2013 Copyright 2012, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners.

14 Heterogeneous Compiler Data Transfer Overview The host CPU and the Intel Xeon Phi coprocessor do not share physical or virtual memory in hardware Two offload data transfer models are available: 1. Explicit Copy o o o o Programmer designates variables that need to be copied between host and card in the offload directive Syntax: Pragma/directive-based C/C++ Example: #pragma offload target(mic) in(data:length(size)) Fortran Example:!dir$ offload target(mic) in(a1:length(size)) 2. Implicit Copy o o o o o Programmer marks variables that need to be shared between host and card The same variable can then be used in both host and coprocessor code Runtime automatically maintains coherence at the beginning and end of offload statements Syntax: keyword extensions based Example: _Cilk_shared double foo; _Offload func(y); 14

15 Agenda Memory Basics Data Access Semantics Explicit Offloading Implicit Offloading Programming Examples 15

16 Heterogeneous Compiler Offload using Explicit Copies Data Movement pa Host 2 Copy over 4 Copy back MIC 1 Allocate 5 Free Default treatment of in/out variables in a #pragma offload statement At the start of an offload: {...} o Space is allocated on the coprocessor o in variables are transferred to the coprocessor At the end of an offload: o out variables are transferred from the coprocessor #pragma offload inout(pa:length(n)) o Space for both types (as well as inout) is deallocated on the coprocessor 3 16

17 Heterogeneous Compiler Offload using Explicit Copies Modifier Example float reduction(float *data, int numberof) { float ret = 0.f; #pragma offload target(mic) in(data:length(numberof)) { #pragma omp parallel for reduction(+:ret) for (int i=0; i < numberof; ++i) ret += data[i]; } return ret; } Note: copies numberof elements to the coprocessor, not numberof*sizeof(float) bytes the compiler knows data s type 17

18 Heterogeneous Compiler Offload using Explicit Copies Offload pragma Keyword for variable & function definitions Entire blocks of code Data transfer Synchronization C/C++ Syntax #pragma offload <clauses> <statement block> attribute ((target(mic))) #pragma offload_attribute(push, target(mic)) #pragma offload_attribute(pop) #pragma offload_transfer target(mic) #pragma offload_wait signal(signal_slot) Semantics Allow next statement block to execute on Intel MIC Architecture or host CPU Compile function for, or allocate variable on, both CPU and Intel MIC Architecture Mark entire files or large blocks of code for generation on both host CPU and Intel MIC Architecture Initiates asynchronous data transfer, or initiates and completes synchronous data transfer Wait asynchronous offload processes to complete 18

19 Heterogeneous Compiler Offload using Explicit Copies Offload directive Keyword for variable/function definitions Data transfer Fortran Syntax!dir$ omp offload <clause> <OpenMP construct>!dir$ offload <clauses> <statement>!dir$ attributes offload:<mic> :: <rtnname>!dir$ offload_transfer target(mic) Semantics Execute next OpenMP* parallel construct on Intel MIC Architecture Execute next statement (function call) on Intel MIC Architecture Compile function or variable for CPU and Intel MIC Architecture Initiates asynchronous data transfer, or initiates and completes synchronous data transfer 19

20 Heterogeneous Compiler Offload using Explicit Copies Clauses Variables and pointers restricted to scalars, structs of scalars, and arrays of scalars Clauses Syntax Semantics Target specification target( name[:card_number] ) Where to run construct Conditional offload if (condition) Boolean expression Inputs in(var-list modifiers opt ) Copy from host to coprocessor Outputs out(var-list modifiers opt ) Copy from coprocessor to host Inputs & outputs inout(var-list modifiers opt ) Copy host to coprocessor and back when offload completes Non-copied data nocopy(var-list modifiers opt ) Data is local to target Async. offload signal(signal-slot) Trigger async offload Async. offload wait(signal-slot) Wait for completion 20

21 Heterogeneous Compiler Offload using Explicit Copies Modifiers Variables and pointers restricted to scalars, structs of scalars, and arrays of scalars Modifiers Syntax Semantics Specify pointer length length(element-count-expr) Copy N elements of the pointer s type Control pointer memory allocation Control freeing of pointer memory Control target data alignment alloc_if ( condition ) free_if ( condition ) align ( expression ) Allocate memory to hold data referenced by pointer if condition is TRUE Free memory used by pointer if condition is TRUE Specify minimum memory alignment on target 21

22 Agenda Memory Basics Data Access Semantics Explicit Offloading Implicit Offloading Programming Examples 22

23 Heterogeneous Compiler Offload using Implicit Copies Section of memory maintained at the same virtual address on both the host and Intel MIC Architecture coprocessor Reserving same address range on both devices allows Seamless sharing of complex pointer-containing data structures Elimination of user marshaling and data management Use of simple language extensions to C/C++ Same address range C/C++ executable Offload code Host Memory Host Intel MIC KN* Memory 23

24 Heterogeneous Compiler Offload using Implicit Copies When shared memory is synchronized Automatically done around offloads (so memory is only synchronized on entry to, or exit from, an offload call) Only modified data is transferred between CPU and coprocessor Dynamic memory you wish to share must be allocated with special functions: _Offload_shared_malloc, _Offload_shared_aligned_malloc, _Offload_shared_free, _Offload_shared_aligned_free Allows transfer of C++ objects Pointers are no longer an issue when they point to shared data Well-known methods can be used to synchronize access to shared data and prevent data races within offloaded code E.g., locks, critical sections, etc. This model is integrated with the Intel Cilk Plus parallel extensions Note: Not supported on Fortran - available for C/C++ only 24

25 Heterogeneous Compiler Implicit: Offloading using _Offload Example void findpi() { int count = 10000; } // Initialize shared global // variables pi = 0.0f; // Compute pi on target _Offload compute_pi(count); pi /= count; // Shared variable declaration for pi _Cilk_shared float pi; // Shared function declaration for // compute _Shared _Cilk_shared void compute_pi(int void compute_pi(nt count) count) { int i; } #pragma omp parallel for \ reduction(+:pi) reduction(+:pi) for (i=0; i<count; i++) { float t = (float)((i+0.5f)/count); pi += 4.0f/(1.0f+t*t); } 25

26 Heterogeneous Compiler Keyword _Cilk_shared for Data/Functions What Syntax Semantics Function int _Cilk_shared f(int x) { return x+1; } 26 Versions generated for both CPU and card; may be called from either side Global _Cilk_shared int x = 0; Visible on both sides File/Function static static _Cilk_shared int x; Visible on both sides, only to code within the file/function Class class _Cilk_shared x { }; Class methods, members, and and operators are available on both sides Pointer to shared data int _Cilk_shared *p; p is local (not shared), can point to shared data A shared pointer int *_Cilk_shared p; p is shared; should only point at shared data Entire blocks of code #pragma offload_attribute( push, _Cilk_shared) #pragma offload_attribute(pop) Mark entire files or large blocks of code _Cilk_shared using this pragma

27 Heterogeneous Compiler Implicit: Offloading using _Offload Feature Example Description Offloading a function call x = _Offload func(y); x = _Offload_to (card_number) func(y); func executes on coprocessor if possible func must execute on specified coprocessor Offloading asynchronously Offload a parallel for-loop x = _Cilk_spawn _Offload func(y); Non-blocking offload _Offload _Cilk_for(i=0; i<n; i++) { a[i] = b[i] + c[i]; } Loop executes in parallel on target. The loop is implicitly outlined as a function call. 27

28 Heterogeneous Compiler Command-line Options Offload-specific arguments to the Intel Compiler: Generate host+coprocessor code (by default only host code is generated): -offload-build (Deprecated offload is default) Produce a report of offload data transfers at compile time (not runtime) -opt-report-phase:offload Add Intel MIC Architecture compiler switches -offload-copts: switches Add Intel MIC Architecture archiver switches -offload-aropts: switches Add Intel MIC Architecture linker switches -offload-ldopts: switches Example: icc g O2 mkl offload-build offload-copts= -g -03 offload-ldopts= -L/opt/intel/composerxe_mic/mkl/lib/mic foo.c 28

29 Module Outline Memory Basics Data Access Semantics Explicit Offloading Implicit Offloading Examples 29

30 Example 1: Using MKL for offloading Lapack and Blas routines int main{ // initialize variables #pragma offload target(mic) in(transa, transb, N, alpha, beta) \ in(a:length(matrix_elements)) in(b:length(matrix_elements)) \ inout(c:length(matrix_elements)) { } // continue code } sgemm(&transa, &transb, &N, &N, &N, &alpha, A, &N, B, &N, &beta, C, &N); Sgemm performs C=beta*C+alpha*A*B, transa and transb regulate the transposition of A and B and the Ns define the sizes of the matrices (see documentation). C is input and output, all others are input only. MKL will automatically make optimal use of MIC 30

31 Example 2: Simultaneous computation on host and accelerator Host Prework Compute workload parallel Postwork Target Compute workload parallel When using a straight #pragma offload the host blocks until completion of the of the offloaded region or function. In order to obtain max performance it is necessary to keep the host working at the time the offload computes. 31

32 Example 2: Simultaneous computation on host and accelerator - OpenMP double attribute ((target(mic))) myworkload(double input){ } // do something useful here return result; int main(void){ //. Initialize variables #pragma omp parallel sections { #pragma omp section { #pragma offload target(mic) } result1= myworkload(input1); #pragma omp section } } result2= myworkload(input2); Function is generated for both MIC and CPU Create two threads in an OpenMP sections env. One thread executes the offload code on MIC The other thread executes the same function on the host 32

33 Example 2: Simultaneous computation on host and accelerator - Cilk _Cilk_shared double myworkload(double input){ } // do something useful here return result; int main() { } result1 = _Cilk_spawn _Cilk_offload myworkload(input2); result2 = myworkload(input1); cilk_sync; Function is generated for both MIC and CPU One thread is spawned and executes the offload code on MIC The host executes the same function and waits 33

34 Example 3: Asynchronous Transfer & Double Buffering Overlap computation and communication Generalizes to data domain decomposition pre-work Host data block Target iteration 0 data block data block process iteration 1 data block data block process iteration n data block data block process iteration n+1 data block data block process last iteration data block process 34

35 Example 3 Using Signals This does nothing except allocating an array #pragma offload_transfer target(mic:0) \ nocopy(in1:length(cnt)) alloc_if(1) free_if(0)) Start an asynchronous transfer, tracking signal in1 #pragma offload_transfer target(mic:0) in(in1:length(cnt) alloc_if(0) free_if(0)) signal(in1) Start once the completion of the transfer of in1 in signaled #pragma offload target(mic:0) nocopy(in1) wait(in1) \ out(res1:length(cnt) alloc_if(0) free_if(0)) This does nothing except freeing an array #pragma offload_transfer target(mic:0) \ nocopy(in1:length(cnt) alloc_if(0) free_if(1)) 35

36 Example 3: Double Buffering I int main(int argc, char* argv[]) { // Allocate & initialize in1, res1, // in2, res2 on host #pragma offload_transfer target(mic:0) in(cnt)\ nocopy(in1, res1, in2, res2 : length(cnt) \ alloc_if(1) free_if(0)) Only allocate arrays on card with alloc_if(1), no transfer do_async_in(); #pragma offload_transfer target(mic:0) \ } return 0; nocopy(in1, res1, in2, res2 : length(cnt) \ alloc_if(0) free_if(1)) Only free arrays on card with free_if(1), no transfer 36

37 Example 3: Double Buffering II void do_async_in() { float lsum; int i; lsum = 0.0f; #pragma offload_transfer target(mic:0) in(in1 : length(cnt) \ alloc_if(0) free_if(0)) signal(in1) for (i = 0; i < iter; i++) { if (i % 2 == 0) { #pragma offload_transfer target(mic:0) if(i!=iter - 1) \ in(in2 : length(cnt) alloc_if(0) free_if(0)) signal(in2) #pragma offload target(mic:0) nocopy(in1) wait(in1) \ out(res1 : length(cnt) alloc_if(0) free_if(0)) { } compute(in1, res1); lsum = lsum + sum_array(res1); } else { Send buffer in1 Send buffer in2 Once in1 is ready (signal!) process in1 37

38 Example 3: Double Buffering III } else { #pragma offload_transfer target(mic:0) if(i!= iter - 1) \ in(in1 : length(cnt) alloc_if(0) free_if(0)) signal(in1) #pragma offload target(mic:0) nocopy(in2) wait(in2) \ } { } out(res2 : length(cnt) alloc_if(0) free_if(0)) compute(in2, res2); lsum = lsum + sum_array(res2); } async_in_sum = lsum / (float)iter; } // for } // do_async_in() Send buffer in1 Once in2 is ready (signal!) process in2 38

39 39

40 Module Outline Memory Basics Data Access Semantics Explicit Offloading Implicit Offloading Examples 40

41 Extension SCIF SCIF Host-KNC communications backbone Provides com. cap. within a single platform(node) Low latency, low overhead communication Provides uniform API for communication across the hosts PCI Express* system busses Directly exposes DMA capabilities for high bandwidth transfer Fully exposed (/usr/include/scif.h) 41

42 SCIF Symmetric Communications Interface The SCIF driver provides a reliable connectionbased messaging layer, as well as functionality which abstracts RMA operations. The SCIF API is documented in the Intel MIC SCIF API Reference Manual for User Mode Linux and the Intel MIC SCIF API Reference Manual for Kernel Mode Linux. A common API is exposed for use in both user mode (ring 3) and kernel mode (ring 0), with the exception of slight differences in signature, and several functions which are only available in user mode, and several only available in kernel mode. 42

43 SCIF - Nodes and Ports SCIF node: physical endpoint in the SCIF network. The host and MIC Architecture devices are SCIF nodes (all cores under a single OS). Each node has a node identifier assigned at boot time. Node IDs are generally based on PCIe discovery order. The host node is always assigned ID 0. SCIF port: logical destination on a SCIF node. Within a node, a SCIF port on that node may be referred to by its number, a 16-bit integer, similar to an IP port. SCIF port identifier: is unique across a SCIF network, comprising both a node identifier and a local port number (analogous to a complete TCP/IP address with port) 43

44 SCIF Connections Functionality scif_epd_t scif_open(void); Create a new endpoint int scif_bind(scif_epd_t epd, uint16_t pn); Bind Endpoint to port int scif_listen(scif_epd_t epd, int backlog); Set endpoint to listen int scif_connect(scif_epd_t epd, struct scif_portid* dst); Request connection to listening endpoint int scif_accept (scif_epd_t epd, struct scif_portid* peer, scif_epd_t* newepd, int flags); Accepts the connection request int scif_close (scif_epd_t epd); Closes the connection 44

45 SCIF Opening a connection epdi=scif_open() epdj=scif_open() scif_bind(epdi,pm) scif_bind(epdj,pn) scif_listen(epdj,qlen) scif_connect(epdi,(nj,pn)) scif_accept(*nepd,peer) 45

46 46

47 Summary The Intel Xeon Phi coprocessor supports native execution and host-centric computing w/ offloading The tool chain fully supports the traditional way of (cross-) compiling and optimization for the coprocessor Programmers can choose between explicit offloading and implicit offloading to best utilize the coprocessor 47