An Introduction to GPU Computing. Aaron Coutino MFCF

Transcription

1 An Introdction to GPU Compting Aaron Cotino MFCF

2 What is a GPU? A GPU (Graphical Processing Unit) is a special type of processor that was designed to render and maniplate textres. They were originally designed for rendering video games. The key differences between a CPU and a GPU is how many transistors are allocated to processing compared to memory.

3

4 Why to se a GPU While originally designed for graphical processing, the architectre of GPUs are sited for any application which perform many of the same simple task. In 2008, Nvidia developed the first general prpose API for interfacing with GPUs, CUDA. This has made it easy to interface with the GPU and take advantage of this type of comptation. The figre on the right shows a typical work flow for a code tilizing a GPU. When sed correctly GPU compting can provide a massive speed-p.

5 Speed p

6 When not to se a GPU The key to taking advantage of GPU compting is tilizing the mass parallelization of processes. If yor program or application cannot be made to perform many small tasks then it will not be improved by GPU compting (it wold probably make it worse). Another isse to look for is that memory transfer is not the time-limiting factor of yor code. To process data, the CPU needs to transfer the data to the GPU throgh the motherboard PCIe interface, which is mch slower than inter processor commnication. In many cases this commnication cost is the limiting factor of the code and may get worse by transferring to the GPU.

7 GPU Compting in Matlab Inclded in the Parallel Compting Toolbox. Extremely easy to se. To create a variable that can be processed sing the GPU, se the gparray fnction. This fnction transfers the storage location of the argment to the GPU. Any fnctions which se this argment will then be compted by the GPU. Many of the bilt in fnctions have been overloaded to accept gparray argments and will be evalated on the GPU. For a list of these fnctions see:

8 Example N=256; h_a = magic(n); d_a = gparray(h_a); d_b = abs(a.*a); h_b = gather(d_b);

9 GPU Compting in Matlab The final step in the previos example sed the gather fnction. This fnction retrns the variable created on the GPU (d_b) to the CPU so that it can be frther processed (plotted). That s it! A key se where GPUs can be particlarly sefl is the FFT (Fast-Forier Transform). To take advantage of this, the only change that needs to be made is sing the gparray to transfer the variable to the GPU and gather to bring it back.

10 Spectral Shallow Water Eqation Solver

11 Performance Difference

12 Monte Carlo Simlations Monte Carlo simlations are particlarly sited to GPU compting as yo can perform an ensemble of 1000s in roghly the same time as a single simlation. To perform them in Matlab yo se the arrayfn fnction. The sage of this fnction is: vector_answers =arrayfn(@somefnction,vector_inpt1,vector_inpt2, ). This will perform the fnction somefnction sing the inpts vector_inpt1(1), vector_inpt2(1), for each vector set. If these vector inpts are gparrays then each application of the fnction will be performed on a separate thread.

13

14 Parameter Sweep This is an NPZD model where we are investigating the dynamics of changing the parameter k. k goes from 0.05 to 10 by steps of 0.05 so there are 199 vales. The loop steps time forward and if it is a print step it will gather the variables back from the GPU.

15 GPU Compting in R Compting with GPUs is also extremely easy in R. All that is reqired is to download one of the packages that have prebilt fnctions. There are a nmber of different packages that can be downloaded that allow for GPU compting. Some examples are: gpr, rpd, gptools, cdabayesreg. Once the package is downloaded yo can call the provided fnctions which will operate on the GPU.

16 gptools gpmatmlt -- Perform Matrix Mltiplication with a GPU mata <- matrix(rnif(2*3), 2, 3) matb <- matrix(rnif(3*4), 3, 4) gpmatmlt(mata, matb) gpqr -- Estimate the QR decomposition for a matrix # get some random data of any shape at all x <- matrix(rnif(25), 5, 5) qr <- gpqr(x)

17 gptools gpdistclst -- Compte Distances and Hierarchical Clstering for Vectors on a GPU nmvectors <- 5 dimension <- 10 Vectors <- matrix(rnif(nmvectors*dimension), nmvectors, dimension) myclst <- gpdistclst(vectors, "maximm", "mcqitty ) gplm -- Fitting Linear Models sing a GPU enabled QR ctl <- c(4.17,5.58,5.18,6.11,4.50,4.61,5.17,4.53,5.33,5.14) trt <- c(4.81,4.17,4.41,3.59,5.87,3.83,6.03,4.89,4.32,4.69) grop <- gl(2,10,20, labels=c("ctl","trt")) weight <- c(ctl, trt) anova(lm.d9 <- gplm(weight ~ grop)) smmary(lm.d90 <- gplm(weight ~ grop - 1)) smmary(resid(lm.d9) - resid(lm.d90))

18 rpd Rpchol -- GPU Accelerated Cholesky Decomposition N <- 20 x <- matrix(rnif(n*5), ncol=n) A <- t(x) rpchol(a) rpcor.test -- Compte the p-vales of the correlation matrix nm <- 5 dim1 <- 6 dim2 <- 8 x <- matrix(rnif(nm*dim1), nm, dim1) y <- matrix(rnif(nm*dim2), nm, dim2) rpcor.test(x, y, method = "kendall") # introdce missing vales x[3,5] <- NA y[4,1] <- NA rpcor.test(x, y, method = "kendall", se="pairwise.complete.obs")

19 Behind the scenes In both Matlab and R the actal interface with the GPU is performed sing CUDA C++. As sch in order to se the fnctionality I have otlined before yo need to download CUDA. It can be fond at: For making yor own applications, beyond what yo can do within these langages, yo may wish to work in C++ so as to directly control the comptation. For compiling CUDA code yo mst se the compiler provided by Nvidia, nvcc. It is based on g++ and has most of its featres.

20 CUDA C++ Assming that yo have installed CUDA in yor path, yo need to provide the linker lcdart (CUDA rntime). In yor header yo will then need to add #inclde<cda>. The way coding in CUDA C++ works is that yo create kernels which are fnctions that will be evalated on the GPU. These fnctions are preceded by the declaration specifier global. To invoke a kernel yo call: SOME_KERNEL<<<n_blocks,size_blocks>>>(inpt1, inpt2, ) Yo may be wondering what n_blocks and size_blocks are

21 GPU Breakdown

22 Example

23 CUDA C++ In order to create a variable that is stored on the GPU yo mst first allocate is memory. This is done throgh the cdamalloc fnction. This fnction takes in the address of yor declared variable (&variable) and the size to allocate. For a vector of length N this wold be N*sizeof(float,int,doble,etc ). Together this is cdamalloc(&variable,n*sizeof(float)). Generally yo will have a corresponding copy of this variable on the CPU which is allocated sing the normal malloc() fnction.

24 CUDA C++ As with in Matlab, yo have to transfer yor data to and from the GPU if yo want to perform any other operations on it (for example write it ot). To do this yo se the cdamemcpy fnction. It takes in: target_vector, original_vector, size of vector, and direction. The target_vector and original_vector are both pointers to vectors where one of them is located on the GPU while the other is located on the CPU/RAM. The size is jst nmber of elements * sizeof(float,int,doble,etc ). The direction is either cdamemcpyhosttodevice or cdamemcpydevicetohost depending on which way the data is moving.

25 Example

26 CUDA Libraries In addition to the C++ API, CUDA comes with several libraries that are CUDA versions of standard libraries. Some examples inclde: cfftw, cblas, crand, and csparse. Some of these libraries even provide bilt in nderstanding of the original library, eg. cfftw nderstands most FFTW3 syntax. In addition to this there are several specific gides for optimizing yor code for a given GPU architectre. These can all be fond at:

27 MFCF Resorces We crrently have a GPU server with 8 Telsa K80s providing p to 128 TFLOPS of comptational power and 16GB of memory each. This server can be accessed throgh ssh at yorsername@gp01.stdent.math.waterloo.ca. Yo se yor Nexs login information. If yo are connecting from off-camps yo need to se the CISCO VPN Client before yo connect. This sever can be sed to test yor code.

28 Smmary GPU compting is an extremely powerfl tool for massively parallelizable problems. It can provide hge speed ps for very little investment compared to traditional CPU compting. Both Matlab and R provide very easy to learn and se interfaces for taking advantage of this. The general design idea s can be relatively easily implemented in CUDA C++. It shold also be mentioned that CUDA is not the only option. ATI have developed an open sorce API called OpenCL which will work with any GPU. MFCF crrently has a GPU clster which anyone can access to perform GPU comptations.

29 References /TalkSlides/Contribted/16Ag_1115_FocsI_3-HighPerfComp_1- Ligtenberg.pdf