Practical Experience on CUDA

Transcription

1 Practical Experience on CUDA Fang-an Kuo DATE:1/16/09

2 Outline Parallel loop via CUDA CUDA 簡介以 3D Array 之元素和為例傳統迴圈計算其元素和 (Sum) 利用 CUDA 平行計算元素和效能比較 FFT via CUDA FFTW 3.2alpha CUFFT 範例效能比較 Matrix multiplication on multigpu 結論參考資料 2

3 CUDA 簡介 CUDA 是 Nvidia 提出的 GPGPU 技術 透過 CUDA 開發 GPGPU 程式是利用其 C 語言函式庫與 CUDA 的延伸來編寫, 故不用 opengl Direct3D CUDA 架構 Host Device Grid Block thread SIMD Memory Management 3

4 以 3D Array 之元素和為例 3D Array: mat[256][256][256] 加總計算 mat 元素和 (Sum) 傳統方法 :triple loop GPGPU 方法 : Parallel loop via CUDA mat[i][j][k] j k i Thread 1 Thread 2 Thread 3 Block 3 Block 2 Block 1 4

5 傳統方式計算其元素和 (Sum) 利用 for loop 加總元素 在 CPU Q G 主機上計算約耗時 930(ms), 約為 0.144(Gflops) 5

6 利用 CUDA 平行計算元素和 Submit a job Sum_gpu1 with n blocks and n threads per block. Psum[i][k]= mat[][][ i j k] j Block 3 Block 2 Block 1 Thread 1 Thread 2 Thread 3 j CPU: Q6850 (quad-core 3.0G), OS : OpenSUSE 10.2 GPU: Tesla C870, MHz clock, MB memory Interface : PCI-e x ===========Sourse=============== sum_gpu1<<<n,n>>>(psum, dmat, n); global void sum_gpu1(float *psum, float *dmat, int n) { int shift = bid*n*n; psum += bid*n; dmat += shift; syncthreads(); for(int i=tid;i<n*n;i+=bdim) { psum[tid]+=dmat[i]; } syncthreads(); } ==========Result================= N =512, N2 =N*N, n =N bid =blockidx.x, tid =threadidx.x, bdim =blockdim.x GPU time : (ms), (Gflops) GPU time : (ms), (Gflops),Kernel only ================================= 6

7 利用 CUDA 方式平行計算元素和 Psum[i] = using shared memory. Psum[0]= using shared memory. Sum = Psum[0] i Psum[0] psum[][ i k] k psum[] i i Psum[i][k] k ===========Sourse=============== sum_gpu2<<<1,n,n*sizeof(float)>>>(psum, n); global void sum_gpu2(float *psum, int n) { extern shared float shared[]; shared[tid]=0; for(int i=tid;i<n*n;i+=bdim) { shared[tid]+=psum[i]; } syncthreads(); if(tid <256) { shared[tid] += shared[tid +256]; } syncthreads(); if(tid <128) { shared[tid] += shared[tid +128]; } syncthreads(); if(tid < 64) { shared[tid] += shared[tid + 64]; } syncthreads(); if(tid < 32) { shared[tid] += shared[tid + 32]; } syncthreads(); if(tid < 16) { shared[tid] += shared[tid + 16]; } syncthreads(); if(tid < 8) { shared[tid] += shared[tid + 8]; } syncthreads(); if(tid < 4) { shared[tid] += shared[tid + 4]; } syncthreads(); if(tid < 2) { shared[tid] += shared[tid + 2]; } syncthreads(); if(tid < 1) { shared[tid] += shared[tid + 1]; } syncthreads(); if(tid == 0) {psum[0] = shared[0];} } ==========Result================= N =512, N2 =N*N, n =N bid =blockidx.x, tid =threadidx.x, bdim =blockdim.x GPU time : (ms), (Gflops) GPU time : (ms), (Gflops),Kernel only ================================= 7

8 效能比較 Tesla C870 Kernel Only vs. Q6850 Tesla C870 vs. Q Gflops CPU GPU Glops GPU CPU Size Size Tesla C870 Kernel Only 表示僅計算 Kernel 函數所花費時間, 不包括記憶體交換時間 加速 2 到 50 倍 Tesla C870 表示總計算時間, 包括記憶體交換時間 約加速 2 到 5 倍 8

9 FFT via CUDA FFTW 3.2alpha vs CUDA CUFFT Comparing complex-complex single precision FFT 3D The test machine: intel Q6850(quad-core 3.0G),L2:4MB Tesla C870, 1.35G clock, 1.5G VRAM(PCI-e 1.0) CUDA 2.0(NVCC 1.1) opensuse

10 FFTW 3.2alpha 本次僅比較 CUFFT 3D 與 FFT 3D 單精度轉換部份比較 fftw 3.2 在單線程與多線程上的效能轉換 mat[n][n][n] 為 Fmat[N][N][N] =============Sourse=============== fftwf_complex *mat,*fmat; mat = (fftwf_complex*) fftwf_malloc(sizeof(fftwf_complex) *N3); Fmat= (fftwf_complex*) fftwf_malloc(sizeof(fftwf_complex) *N3); fftwf_plan forward; forward = fftwf_plan_dft_3d(n, N, N, mat, Fmat, FFTW_FORWARD, FFTW_ESTIMATE); fftwf_execute(forward); =============++++++============= N CPU Time(ms) CPU 4-thread Time(ms) N CPU (Gflops) CPU 4-thread (Gflops)

11 CUFFT 比較 cufft 核心函數與 cufft 包含資料交換上的的時間與效能 在 N=48 以上開始有較 fftw 好的 3 計算效率產生 ( 總元素為 48 個 ) =============Sourse=============== cufftcomplex *dmat,*fdmat; cudamalloc((void**) &dmat, sizeof(cufftcomplex)*n3); cudamalloc((void**) &Fdmat, sizeof(cufftcomplex)*n3); cudamemcpy(dmat, &mat[0], sizeof(cufftcomplex) *N3, cudamemcpyhosttodevice); cuffthandle plan; cufftplan3d(&plan, N, N, N, CUFFT_C2C); cufftexecc2c(plan, dmat, Fdmat,-1); cudamemcpy(&fmat[0], Fdmat, sizeof(cufftcomplex) *N3, cudamemcpydevicetohost); =============++++++============= N GPU Time(ms) GPU Kernel Time(ms) N GPU (Gflops) GPU Kernel (Gflops)

12 FFTW3 vs. CUDA CUFFT 600 FFTW serial FFTW 4 threads CUDA with memory transfer CUDA, Kernel Only FFT 3D Gflops Size 12

13 FFTW3 vs. CUDA CUFFT FFTW serial FFTW 4 threads CUDA with memory transfer CUDA, Kernel Only FFT 3D Elapsed time(ms) Size 13

14 Matrix multiplication on multigpu Comparing the Sgemm on 4-thread CPU,1xGPU and 2xGPU. Sgemm : single-precision matrix multiplication. The test machine: CPU:Q6850(quad-core),3.0G,L2:4MB. GPU:2x Tesla C870,1.35G clock,1.5g memory. PCI Interface : PCI-e 1.0(16x) OS:openSUSE 10.2 CUDA 2.0(NVCC 1.1) gcc 4.3.0(openMP 3.0) MKL

15 Matrix multiplication on multigpu openmp+cuda Sourse: A,B,C in Host A:MxK matrix B:KxN matrix C:MxN matrix da,db,dc in device da:m/2xk db:kxn dc:m/2xn In thread 1: dc=c+th_id*m/th*k #include "cublas.h" omp_set_num_threads(2); #pragma omp parallel shared(c) private(th,th_id,da,db,dc) { th_id = omp_get_thread_num(); th = omp_get_num_threads(); cudasetdevice( th_id ); cublasinit(); #pragma omp barrier cublasalloc( M/th * K, sizeof(float), (void**)&da ); cublasalloc( K * N, sizeof(float), (void**)&db ); cublasalloc( M/th * N, sizeof(float), (void**)&dc ); cudamemcpy( db, B, K * N*sizeof(float), cudamemcpyhosttodevice ); cudamemcpy( da, A+th_id*M/th*K, M/th*K*sizeof(float), cudamemcpyhosttodevice ); cudamemcpy( dc, C+th_id*M/th*N, M/th*N*sizeof(float), cudamemcpyhosttodevice ); cublassgemm( transa, transb, M/th, N, K, alpha, da, M/th, db, K, beta, dc, M/th ); cudamemcpy( C+th_id*M/th * N, dc, M/th * N*sizeof(float), cudamemcpydevicetohost ); } ==========Variable================= M, K, N, alpha, beta are constants. A:MxK, B:KxN, C:MxN Sgemm : C= alpha.* AB + beta.* C 15

16 Matrix multiplication on multigpu Model: C1 A1 C2 A2 * B openmp : shared memory mode MPI : distributed memory mode PC+GPU v.s. GPU Cluster 16

17 Matrix multiplication on multigpu Benchmark, CUDA with memory transfer. N 4-thread Time(ms) 1xGPU Time(ms) 2xGPU Time(ms) 4-thread Gflops 1xGPU Gflops 2xGPU Gflops

18 Matrix multiplication on multigpu 2-GPU 1-GPU 4-CPU 250 Sgemm on MKL vs. Sgemm on CUDA with memory transfer. 200 Gflops Size = Size

19 Matrix multiplication on multigpu 2-GPU 1-GPU 4-CPU 450 Sgemm on MKL vs. Sgemm on CUDA(Kernel Only) Gflops Size 19

20 Matrix multiplication on multigpu Sgemm on MKL vs. Sgemm on CUDA with memory transfer. 120 Elapsed time(ms) GPU 1-GPU 4-CPU Size 20

21 Matrix multiplication on multigpu Sgemm on MKL vs. Sgemm on CUDA(Kernel Only) Elapsed time(ms) GPU 1-GPU 4-CPU Size 21

22 結論 平行化 for loop 在 CUDA 上有非常高的運算速度 3 Cufft 對總元素個數達 48 以上的三維傅立葉轉換較 FFTW3 來的快速 CUDA 矩陣相乘, 當相乘矩陣大於 1560x1560 時, 使用兩張 GPU 相較使用一張 GPU 更為快速 利用 openmp+cuda 是單機上使用 CUDA multigpu 最方便且快速的方法 以上技巧使用於國網中心外部委託案 - 材料科學模擬之應用, 整體加速約 50 倍以上, 因為專案尚在研究中, 故僅提出部份 CUDA 技巧分享 22

23 參考資料 CUDA ZONE openmp Web Heresy s Home FFTW Vasily Volkov and James W.,2008, Benchmarking GPUs to tune dense linear algebra, SC

24 Thank you 24