A BRIEF HISTORY OF GPGPU. Mark Harris Chief Technologist, GPU Computing UNC Ph.D. 2003

Transcription

1 A BRIEF HISTORY OF GPGPU Mark Harris Chief Technologist, GPU Computing UNC Ph.D. 2003

2 2

3 A BRIEF HISTORY OF GPGPU fd General-Purpose computation on Graphics Processing Units 3

4 THE FIRST GPGPU: IKONAS RDS : Nick England & Mary Whitton founded Ikonas Graphics Systems Tim Van Hook wrote microcode for solid modeling, ray tracing (SIGGRAPH 86) From a 1985 Video: All computation is taking place in the Adage 3000 Display 4

5 UNC PIXEL PLANES AND PIXELFLOW Procedural textures on Pixel Planes 5 (Rhodes et al. 1992) dings nteractive 3D Graphics Massachusetts 1 April 1992 PixelFlow Co-Chairs 100, MHz 8-bit processors Early real-time programmable shading (Olano/Lastra 98) Kedem et al. ( 98) used for unix password cracking 5

6 GEFORCE 1-3: THE DAWN OF GPGPU ( 99-01) GeForce 256: First GPU GeForce 3: First programmable GPU Vertex Shaders programmable vertex transforms, 32-bit float Data-dependent, configurable texturing + register combiners Enabled early GPGPU results Hoff (1999) -- Voronoi diagrams on NVIDIA TNT2 Larsen &McAllister (2001): first GPU matrix multiplication (8-bit) Rumpf & Strzodka (2001): first GPU PDEs (diffusion, image segmentation) NVIDIA SDK Game of Life, Shallow Water (Greg James, 2001) 6

7 HYSICALLY BASED SIMULATION ON GEFORCE 3 Approximate simulation of natural phenomena Boiling liquid, fluid convection, chemical reaction-diffusion Inaccurate due to low GPU precision Physically-Based Visual Simulation on Graphics Hardware. Harris, Coombe, Scheuermann, and Lastra. Graphics Hardware

8 NAMING A TREND Application of graphics hardware to non-graphics applications General computations on graphics hardware Exploiting special-purpose hardware for alternative purposes Let s name this thing that people are doing! I coined GPGPU and created home page November 2002 home on the web to collect research / resources Interest grew quickly: launched GPGPU.org August

9 GEFORCE FX (2003) : FLOATING POINT PIXELS True Programmability enabled broader simulation research Ray Tracing (Purcell, 2002), Photon Maps (Purcell, 2003) Radiosity (Carr et al., 2003 & Coombe et al., 2004) PDE solvers Red-black Gauss-Seidel (Harris et al., 2003) Conjugate gradient (Bolz et al. 2003, Krueger et al. 2003) Multigrid (Goodnight et al. 2003) Physically-based simulation Fluid and cloud simulation: (Krueger et al. 2003, Harris et al. 2003) Cloth simulation (Green, 2003) Ice crystal formation (Kim and Lin, 2003) FFT (Moreland and Angel, 2003) High-level language: Brook for GPUs (Buck et al. 2004) 9

10 GPU CLOUD SIMULATION My Ph.D. Dissertation: visually realistic cloud simulation on GPUs 2D & 3D Incompressible Navier-Stokes fluid Thermodynamics (latent heat, diffusion) Water condensation / evaporation Light scattering simulation for rendering Programmed in OpenGL with pixel shaders Real-Time Cloud Simulation and Rendering. Mark Harris Ph.D. Dissertation U. of North Carolina

11 CUDA AND THE G80 GPU (2006) First GPU arch. and software platform designed for computing Dedicated computing mode threads rather than pixels/vertices General, byte-addressable memory architecture First C/C++ language and compiler for GPUs CUDA C++ defines minimally extended subset of C++ with parallelism 2007 began a massive surge in GPGPU development Not just graphics PhDs 11

12 ACCELERATING DISCOVERIES USING A SUPERCOMPUTER POWERED BY 3,000 TESLA PROCESSORS, UNIVERSITY OF ILLINOIS SCIENTISTS PERFORMED THE FIRST ALL-ATOM SIMULATION OF THE HI VIRUS AND DISCOVERED THE CHEMICAL STRUCTURE OF IT CAPSID THE PERFECT TARGET FOR FIGHTING THE INFECTION. WITHOUT GPUS, THE SUPERCOMPUTER WOULD NEED TO BE 5X LARGER FOR SIMILAR PERFORMANCE. 12

13 FROM HPC TO ENTERPRISE DATACENTERS Oil & Gas Higher Ed Government Supercomputing Finance Consumer Web Air Force Research Laboratory Tokyo Institute of Technology Naval Research Laboratory 13

14 MACHINE LEARNING USING DEEP NEURAL NETWORKS Input Result inton et al., 2006; Bengio et al., 2007; Bengio & LeCun, 2007; Lee et al., 2008; 2009 sual Object Recognition Using Deep Convolutional Neural Networks ob Fergus (New York University / Facebook) 14

15 GPU-ACCELERATED DEEP LEARNING START-UPS Image Detection Face Recognition Gesture Recognition Video Search & Analyti Speech Recognition & Translation Image and Video Understanding Recommendation Engin Indexing & Search 15

16 COMMON PROGRAMMING APPROACHES Across Heterogeneous Architectures Libraries AmgX cublas Compiler Directives Programming Languages x86 16

17 Unified Memory DRAMATICALLY LOWER DEVELOPER EFFORT Past Developer View Developer View With Unified Memory System Memory GPU Memory Unified Memory 17

18 PARALLELISM IN MAINSTREAM LANGUAGES Enable more programmers to write parallel software Give programmers the choice of language to use Parallel computing support in key languages C 18

19 FUTURE C++: PARALLEL STL d::vector<int> vec =... previous standard sequential loop d::for_each(vec.begin(), vec.end(), f); explicitly sequential loop d::for_each(std::seq, vec.begin(), vec.end(), f); permitting parallel execution d::for_each(std::par, vec.begin(), vec.end(), f); Complete set of parallel primitives: for_each, sort, reduce, scan, etc. ISO C++ committee voted unanimously to accept as official tech. specification working draft N3960 Technical Specification Working Draft: Prototype: 19

20 PARTING WORDS OF WISDOM Stand Up! Keep it narrow and doable Fred Brooks Write a little bit every day Fred Brooks If you measure it, you can improve it Jen-Hsun Huang But you have to measure the right thing! 20

21 NVIDIA GeForce : ADVENT OF THE GPU Coined the term Graphics Processing Unit A single-chip processor with integrated transform, lighting, triangle setup/clipping, and rendering engines that is capable of processing a minimum of 10 million polygons per second. Register Combiners configurable multipass shading Beginning of GPU programmability Texture Fetch Fragment Color Specular Color Fog Color/Factor Texture 0 Texture 1 Register Set 4 RGB Inputs 4 Alpha Inputs 3 RGB Outputs 3 Alpha Outputs 4 RGB Inputs 4 Alpha Inputs 3 RGB Outputs 3 Alpha Outputs Spare 0 Specular Color 6 RGB Inputs 1 Alpha Input General Combiner 0 General Combiner 1 Final Combiner 21

22 EARLY PC & WORKSTATION GRAPHICS Rasterizer, texture unit, z-buffer, frame buffer Fixed-point math, fixed-function interpolation / texturing Lengyel et al. (1990) Robot motion planning Use rasterizer to fill minkowski sum polygons HP 9000 TurboSRX Workstation Hoff (1999) -- Voronoi diagrams on NVIDIA TNT2 Render cones rasterizer and z-buffer compute voronoi diagram 22