How Powerful are GPUs? Pat Hanrahan Computer Science Department Stanford University Computer Forum 2007 Modern Graphics Pipeline Application Command Geometry Rasterization Texture Fragment Display Page 1
A Pitch from 5 Years Ago Cinematic games and media drive GPU market Current GPUs faster than CPUs (at graphics) Gap between the GPU and the CPU increasing Why? Efficiently use VLSI resources Programmable GPUs Stream processors Many applications map to stream processing Therefore, a $50 high-performance, massively parallel computer will soon ship with every PC Pat Hanrahan, circa 2002-2005 What Happened? Now AMD and Intel gave up on sequential CPUs with high clock rates and went multi-core (2-4) Gap between GPU and CPU stablelized GPUs are data parallel (64-128 cores) DX10 mandates unified graphics pipeline GPGPU many algorithms implemented Future Two main types of processors CPU fast sequential processor GPU fast data parallel processor Hybrid CPU/GPU Page 2
Overview Current programmable GPUs Performance Programming model: Stream abstraction Applications How General? Programmable GPUs Page 3
ATI R600 (X2X00) 80 nm process ~700 million transistors 64 4-wide unified shaders ~700 Mhz clock 512-bit GDDR memory GDDR3 @ 900Mhz = 115 GB/s GDDR4 @ 1100Mhz = 140 GB/s R300 not R600 230 Watt NVIDIA G80 (8800) 90 nm TSMC process 681million transistors 480 mm^2 128 scalar processors 1.3 Ghz clock rate 384-bit GDDR memory GDDR3 @ 900Mhz = 86.4 GB/s 130 Watts Page 4
GeForce 8800 Series GPU Host Input Assembler Vertex Thread Geometry Thread Rasterization ti Pixel Thread TF L1 TF L1 TF L1 TF L1 TF L1 TF L1 TF L1 TF L1 Thread Processor L2 L2 L2 L2 L2 L2 FB FB FB FB FB FB Shader Model 4.0 Architecture 32 4-32-bit Input Parameters 64K 32-bit Registers 32 4-32-bit 64K insts Program Textures 8 4-32-bit Output Page 5
Simple Graphics Pipeline # c[0-3] = modelview projection (composite) matrix # c[4-7] = modelview inverse transpose # c[32] = eye-space light direction # c[33] = constant eye-space half-angle vector # c[35].x = pre-multiplied diffuse light color & diffuse mat. # c[35].y = pre-multiplied ambient light color & diffuse mat. # c[36] = specular color; c[38].x = specular power DP4 o[hpos].x, c[0], v[opos]; # Transform position. DP4 o[hpos].y, c[1], v[opos]; DP4 o[hpos].z, c[2], v[opos]; DP4 o[hpos].w, c[3], v[opos]; DP3 R0.x, c[4], v[nrml]; # Transform normal. DP3 R0.y, c[5], v[nrml]; DP3 R0.z, c[6], v[nrml]; DP3 R1.x, c[32], R0; # R1.x = L DOT N' DP3 R1.y, c[33], R0; # R1.y = H DOT N' MOV R1.w, c[38].x; # R1.w = specular power LIT R2, R1; # Compute lighting MAD R3, c[35].x, R2.y, c[35].y; # diffuse + ambient MAD o[col0].xyz, c[36], R2.z, R3; # + specular END G80 = Data Parallel Computer Host Input Assembler Thread Execution Manager SIMD Core SIMD core SIMD core SIMD Core SIMD Core SIMD Core SIMD Core SIMD Core Parallel Parallel Parallel Parallel Parallel Parallel Parallel Parallel Data Cache Data Cache Data Cache Data Cache Data Cache Data Cache Data Cache Data Cache Load/store Global Memory Page 6
G80 core Parallel l Data Cache Each core 8 functional units SIMD 16/32 warp 8-10 stage pipeline Thread scheduler 128-512 threads/core 16 KB shared memory Total #threads/chip 16 * 512 = 8K GPU Multi-threading (version 1) Change threads each cycle (round robin) frag1 frag2 frag3 frag4 instr1 instr2 instr3 Page 7
GPU Multi-threading (version 2) Change thread after texture fetch/stall Run until stall at texture fetch frag1 frag2 frag3 frag4 (multiple instructions) 8800GTX Peak Performance 575 Mhz * 128 processors * 2 flop/inst * 2 inst/clock MAD instruction = 332.8 GFLOPS Page 8
Instructions Issue Rate http://graphics.stanford.edu/projects/gpubench/ ATI X1900XTX NVIDIA 7900GTX Instructions Issue Rate http://graphics.stanford.edu/projects/gpubench/ NVIDIA 7900GTX NVIDIA 8800GTX Page 9
Measured BLAS Performance SAXPY X1900 (DX9): 6 GFlops X1900 (CTM): 6GFlops 8800GTX (DX9): 12 GFlops SGEMV X1900 (DX9): 4 GFlops X1900 (CTM): 6 GFlops 8800GTX (DX9): 14 GFlops SGEMM X1900 (DX9): 30 GFlops X1900 (CTM): 120 GFlops 8800GTX (DX9): 105 Gflops 3 Ghz Core 2 40 Gflops Programming Abstractions Page 10
Approach I Run application using graphics library Graphics library-based programming models NVIDIA s Cg Microsoft s HLSL OpenGL Shading Language RapidMind Sh [McCool et al. 2004] Approach II Map application to parallel computer Communicating sequential processes (C) Threads: pthreads, Occam, UPC, Message passing: MPI Data parallel programming APL, SETL, S, Fortran90, C* (lisp*), NESL, Stream languages StreaMIT, StreamC/KernelC MS Accelerator, CUDA, DPVM, PeakStream Page 11
Stream Programming Environment Collections stored in memory Multidimensional arrays (stencils) Graphs and meshes (topology) Data parallel operators Application: map Reductions: scan, reduce (fold) Communication: send, sort, gather, scatter Filter ( O < I ) and generate ( O > I ) Brook Ian Buck PhD Thesis Stanford University Brook for GPUs: Stream computing on graphics hardware, I. Buck, T. Foley, D. Horn, J. Sugarman, K. Fatahalian, M. Houston, P. Hanrahan, SIGGRAPH 2004 Page 12
Brook Example kernel void foo ( float a<>, float b<>, out tfloat result<> ) { result = a + b; } float a<100>; float b<100>; float c<100>; foo(a,b,c); for (i=0; i<100; i++) c[i] = a[i]+b[i]; Classical N-Body Simulation Stellar dynamics Gravitational acceleration Gravitational accel. + jerk Molecular dynamics Implicit solvent models Lennard-Jones Coulomb Page 13
Folding@Home Performance Vijay Pande Group GROMACs on Brook GPU:CPUcore 40:1 CPU: 3.0 Ghz P4 GPU: ATI X1900X Current Statistics: March 19, 2007 Client type Current TFLOPS* Current Processors Windows 150 157457 Mac OS X/PPC 7 8710 Mac OS X/Intel 7 2520 Linux 34 24639 GPU 40 682 PS/3 26 877 Total 223 1824132 *TFLOPs is actual flops from software cores, not peak values Page 14
Folding@Home GPU Cluster 25 nodes Nforce4 SLI Dual core Opteron 2x ATI X1900XTX Linux 5 TFlops of folding power Not actual machine Future Page 15
Summary Cinematic games and media drive GPU market GPU evolving into a high throughput processor Data parallel multi-threaded machine Many applications map to GPUs Processor of the future likely to be a CPU/GPU Small number of traditional CPU cores Large number of GPU cores Opportunities Current hardware not optimal Incredible opportunity for architectural innovation Current software environment immature Incredible opportunity for reinventing parallel computing software, programming environments and languages Page 16
Acknowledgements Bill Dally Eric Darve Vijay Pande Bill Mark John Owens Kurt Akeley Mark Horowitz Ian Buck Mattan Erez Kayvon Fatahalian Tim Foley Daniel Horn Michael Houston Jeremy Sugarman Funding: DARPA, DOE, ATI, IBM, NVIDIA, SONY Questions? Page 17