Programmable Graphics Hardware

Transcription

1 Programmable Graphics Hardware Johan S. Seland Center of Mathematics for Applications University of Oslo INF November 2006

2 Programmable Graphics Hardware Programmable Graphics Hardware = GPU (Graphical Processing Unit) Installed base is enormous (many millions) Mainstream because of computer games

3 Programmable Graphics Hardware Programmable Graphics Hardware = GPU (Graphical Processing Unit) Installed base is enormous (many millions) Mainstream because of computer games Gflops

4 Programmable Graphics Hardware Programmable Graphics Hardware = GPU (Graphical Processing Unit) Installed base is enormous (many millions) Mainstream because of computer games Useful for: Computer Games CAD/CAGD/3D Modelling Image/Film Processing Scientific Computing

5 Introduction Introduction Programmable Graphics Pipeline Shading Languages GLSL Language Overview Interfacing OpenGL Creating shaders Passing parameters to shaders Example: Per-pixel lighting Hardware Architecture and Stream Computing GPU Hardware Stream Computing Example: The Wave Equation Conclusion

6 Introduction Curriculum Real-Time Rendering Vertex Shaders (Chapter 6.5) Pixel Shaders (Chapter 6.6) Shading Languages (Chapter 6.7) Architecture (Chapter 15.3) OpenGL Programming Guide OpenGL 2.0 and the OpenGL Shading Language (Chapter 15) Built-In OpenGL Shading Language Variables and Functions (Appendix I)

7 Introduction Goals for the day Learn the difference between the fixed function pipeline and the programmable pipeline Understand the role of shaders and shading languages Understand how to write and use shaders using OpenGL and the OpenGL Shading Language Know a little bit about GPU hardware architecture

8 Introduction Terminology GLSL = OpenGL Shading Language GPU = Graphics Processing Unit { Source code of program written in GLSL Shader = GPU Processing Unit executing above shader DirectX = Microsofts alternative to OpenGL

9 Introduction Introduction Graphics Hardware Modern computers are equipped with dedicated hardware to accelerate 3D-rendering operations Fierce competition in the marketplace Transistor count of GPU > CPU Programming Interface GPUs are closed architectures Graphics driver operating system OpenGL and DirectX are our software interface to the GPU 2006: Flow of new ways to program GPUs: CTM, Cuda, PeakStream, RapidMind, Brook, Sh, Seqoia...

10 Introduction History Lesson 1989: SGI standardize OpenGL as an interface graphics HW. Early 1990s: 2D acceleration for GUIs. Mid-Late 1990s: 3D acceleration, OpenGL and DirectX emerge as the leading APIs. ATI and Nvidia emerge as the leading GPU vendors. 2002: Programmable hardware. 2006: AMD buys ATI. 2006: Nvidia G80 has 681 million transistors, latest Intel Quad-Core has 582 million transistors. 2007: Microsoft Windows Vista will require 3D acceleration. 3D acceleration on the desktop has been driven by games

11 Programmable Graphics Pipeline Introduction Programmable Graphics Pipeline Shading Languages GLSL Language Overview Interfacing OpenGL Creating shaders Passing parameters to shaders Example: Per-pixel lighting Hardware Architecture and Stream Computing GPU Hardware Stream Computing Example: The Wave Equation Conclusion

12 Programmable Graphics Pipeline Fixed Function Pipeline Vertex Processor Assembly Cull Project Rasterize Fragment Processor Fragement & Framebuffer operations CPU Application RAM Textures Texture memory Render to texture Rendertarget (Framebuffer) By now, you should be comfortable with the OpenGL pipeline We will do a short recap

13 Programmable Graphics Pipeline Vertex Processor Assembly Cull Project Rasterize Fragment Processor Fragement & Framebuffer operations CPU Application RAM Textures Texture memory Render to texture Rendertarget (Framebuffer) The CPU Set up GL-state Upload textures Send geometry Execute the pipeline glbindtexture( tex, GL TEXTURE 2D ); glbegin( GL TRIANGLES); glnormal3f( 0.0, -0.35, 0.67 ); glvertex3f( -1.0, 0.0, 0.0 ); glnormal3f( 0.001, -0.49, ); glvertex3f( 1.0, 0.0, 0.0 );. glend(); glflush();

14 Programmable Graphics Pipeline Vertex Processor Assembly Cull Project Rasterize Fragment Processor Fragement & Framebuffer operations CPU Application RAM Textures Texture memory Render to texture Rendertarget (Framebuffer) (0, 1) The vertex processor transforms Position Normal Color Texture coordinates ( 1, 0) (1, 0)

15 Programmable Graphics Pipeline Vertex Processor Assembly Cull Project Rasterize Fragment Processor Fragement & Framebuffer operations CPU Application RAM Textures Texture memory Render to texture Rendertarget (Framebuffer) Primitive Assemby Vertices are assembled into primitives Primitives are clipped Vertices are projected into window coordinates Primitives are rasterized into fragments Attributes are interpolated across primitives (a fragment is a meta-pixel is has depth as well as (x, y)-coordinate)

16 Programmable Graphics Pipeline Vertex Processor Assembly Cull Project Rasterize Fragment Processor Fragement & Framebuffer operations CPU Application RAM Textures Texture memory Render to texture Rendertarget (Framebuffer) The fragment processor calculates the final color and depth This usually involves texture lookup and viewport calculations based on attributes from the vertex processor

17 Programmable Graphics Pipeline Vertex Processor Assembly Cull Project Rasterize Fragment Processor Fragement & Framebuffer operations CPU Application RAM Textures Texture memory Render to texture Rendertarget (Framebuffer) Fragment operations Fragments are discarded or blended Writing to auxiliary buffers

18 Programmable Graphics Pipeline Vertex Processor Assembly Cull Project Rasterize Fragment Processor Fragement & Framebuffer operations CPU Application RAM Textures Texture memory Render to texture Rendertarget (Framebuffer) Output Finally all primitives are displayed!

19 Programmable Graphics Pipeline

20 Programmable Graphics Pipeline Why programmability? The fixed function pipelines forces us to do T&L in a set of predefined ways. By giving the user the freedom to program the pipeline a host of new algorithms are possible. Examples Other lighting models Realistic materials (metals, stone, wood) Procedural textures Image Processing Non-photorealistic rendering The possibilities are endless

21 Programmable Graphics Pipeline Vertex Buffer Index Buffer Input Assembler Texture Vertex Shader Texture Buffer GPU RAM Geometry Shader Stream Out Setup/Rasterization Logical pipeline Programmers View Not the way the hardware works anymore Texture Fragment Shader Depth Color Output Merger

22 Programmable Graphics Pipeline Vertex Buffer Index Buffer Input Assembler Texture Vertex Shader Texture Buffer GPU RAM Texture Geometry Shader Stream Out Setup/Rasterization Fragment Shader Input Assembler Fixed-function Canonicalize vertex data Generate IDs Primitive Vertex Instance Depth Color Output Merger

23 Programmable Graphics Pipeline Vertex Buffer Index Buffer Texture Texture Buffer GPU RAM Texture Depth Color Input Assembler Vertex Shader Geometry Shader Stream Out Setup/Rasterization Fragment Shader Output Merger Vertex Shader Programmable Per vertex processing (same shader for all) No knowledge of the type of primitive No knowledge of neighboring vertices Can change a vertex (x, y, z, w) position 1 primitive in, 1 primitive out Read from memory Usage Transform vertex into eye coordinates Calculate values to be linearly interpolated across primitive (like texture coordinates) Move vertex position

24 Programmable Graphics Pipeline Vertex Buffer Index Buffer Input Assembler Texture Texture Buffer GPU RAM Texture Vertex Shader Geometry Shader Stream Out Setup/Rasterization Fragment Shader Geometry Shader Only in DX10 not mentioned in Red Book Per primitive processing 1 primitive in, n primitives out Read from memory Pipeline may terminate here Usage: Adjacency computations Kill primitive? Quadratic interpolation? Depth Color Output Merger

25 Programmable Graphics Pipeline Vertex Buffer Index Buffer Input Assembler Texture Vertex Shader Texture Buffer GPU RAM Geometry Shader Stream Out Setup/Rasterization Setup/Rasterization Fixed-function Clipping, perspective transform Convert primitives into fragments Texture Fragment Shader Depth Color Output Merger

26 Programmable Graphics Pipeline Vertex Buffer Index Buffer Input Assembler Texture Texture Buffer GPU RAM Texture Vertex Shader Geometry Shader Stream Out Setup/Rasterization Fragment Shader Fragment Shader Programmable Execute same shader for all fragments Has no knowledge of neighboring fragments Can not change the (x, y) coordinate of a fragment Can change a fragments (r, g, b, α) value 1 fragment in, 0 or 1 fragment out Read from memory Depth Color Output Merger

27 Programmable Graphics Pipeline Vertex Buffer Index Buffer Input Assembler Texture Vertex Shader Texture Buffer GPU RAM Geometry Shader Stream Out Setup/Rasterization Output Merger Fixed-function Depth/Stencil test Color buffer blending Read/write/modify memory Texture Fragment Shader Depth Color Output Merger

28 Programmable Graphics Pipeline Vertex Buffer Index Buffer Texture Texture Buffer GPU RAM Texture Depth Color Input Assembler Vtx Attr. Vertex Shader Uniform Geometry Shader Stream Out Setup/Rasterization Varying Fragment Shader Output Merger Communication in the pipeline Uniform variables are set by the CPU and are accessible in all shaders Vertex attributes are encoded in the geometry stream and accessible in {vertex,geometry} shaders Varying variables are set by {vertex,geometry}-shaders, interpolated across primitive and read in fragment shader.

29 Programmable Graphics Pipeline Vertex Buffer Index Buffer Texture Texture Buffer GPU RAM Texture Depth Color Input Assembler Vtx Attr. Vertex Shader Uniform Geometry Shader Stream Out Setup/Rasterization Varying Fragment Shader Output Merger Communication in the pipeline Uniform variables are set by the CPU and are accessible in all shaders Vertex attributes are encoded in the geometry stream and accessible in {vertex,geometry} shaders Varying variables are set by {vertex,geometry}-shaders, interpolated across primitive and read in fragment shader.

30 Programmable Graphics Pipeline Vertex Buffer Index Buffer Texture Texture Buffer GPU RAM Texture Depth Color Input Assembler Vtx Attr. Vertex Shader Uniform Geometry Shader Stream Out Setup/Rasterization Varying Fragment Shader Output Merger Communication in the pipeline Uniform variables are set by the CPU and are accessible in all shaders Vertex attributes are encoded in the geometry stream and accessible in {vertex,geometry} shaders Varying variables are set by {vertex,geometry}-shaders, interpolated across primitive and read in fragment shader.

31 Programmable Graphics Pipeline Vertex Buffer Index Buffer Texture Texture Buffer GPU RAM Texture Depth Color Input Assembler Vtx Attr. Vertex Shader Uniform Geometry Shader Stream Out Setup/Rasterization Varying Fragment Shader Output Merger Communication in the pipeline Uniform variables are set by the CPU and are accessible in all shaders Vertex attributes are encoded in the geometry stream and accessible in {vertex,geometry} shaders Varying variables are set by {vertex,geometry}-shaders, interpolated across primitive and read in fragment shader.

32 Programmable Graphics Pipeline Vertex Buffer Index Buffer Texture Texture Buffer GPU RAM Texture Depth Color Input Assembler Vtx Attr. Vertex Shader Uniform Geometry Shader Stream Out Setup/Rasterization Varying Fragment Shader Output Merger Communication in the pipeline Uniform variables are set by the CPU and are accessible in all shaders Vertex attributes are encoded in the geometry stream and accessible in {vertex,geometry} shaders Varying variables are set by {vertex,geometry}-shaders, interpolated across primitive and read in fragment shader.

33 Shading Languages Introduction Programmable Graphics Pipeline Shading Languages GLSL Language Overview Interfacing OpenGL Creating shaders Passing parameters to shaders Example: Per-pixel lighting Hardware Architecture and Stream Computing GPU Hardware Stream Computing Example: The Wave Equation Conclusion

34 Shading Languages Shading Languages We can program the vertex, geometry and fragment processors by using Shading Languages Offline Rendering (Production Shaders) Shading Languages initially arose in Pixar s RenderMan Designed to allow for extremely high-quality images and movies No demand for production shaders to be executed in real time Current real-time shading languages are inspired by Renderman Shader are executed on CPU

35 Shading Languages Real Time Rendering Arose since fixed function pipeline was not flexible enough Strict real-time requirement (games run at FPS) Shaders are executed on GPU Real-Time Shading Languages Different shading languages exist for consumer hardware: 1. Nvidia Cg (OpenGL/DirectX/WGF) 2. Microsoft HLSL (DirectX/WGF) 3. OpenGL Shading Language (OpenGL) They all provide (almost) the same functionality.

36 Shading Languages GLSL Language Overview The OpenGL Shading Language Based on C/C++ Shader is stored as source code Compile - Link - Run cycle Compiler is embedded in driver Hardware independent

37 Shading Languages GLSL Language Overview GLSL at a glance C/C++-like syntax Built-in Vector (2d, 3d, 4d) Built-in Matrix (2 2, 3 3, 4 4) Built-in standard library of functions Variable qualifiers Built in variables Examples looping for, while, do-while selection if, if-else basic datatypes bool, int, float, struct, arrays functions (call by value) no pointers no bitwise operators no strings no type promotion float f = 0.0;

38 Shading Languages GLSL Language Overview GLSL at a glance C/C++-like syntax Built-in Vector (2d, 3d, 4d) Built-in Matrix (2 2, 3 3, 4 4) Built-in standard library of functions Variable qualifiers Built in variables Examples vec4 v = vec4(1.0, 0.0, -3.0, 2.0 ); Swizzling vec3 w = v.yzx; // (0.0, -3.0, 1.0) Indexing float f = v[2]; // -3.0 Component-wise operation w.x = v.x + f; Component-wise mathops vec4 v = v+w;

39 Shading Languages GLSL Language Overview GLSL at a glance C/C++-like syntax Built-in Vector (2d, 3d, 4d) Built-in Matrix (2 2, 3 3, 4 4) Built-in standard library of functions Variable qualifiers Built in variables Examples mat2 m = mat2(1.0, 2.0, 3.0, 4.0 ); //» mat4 n = mat4(1.0); // n = diag(1.0) vec2 v = m[1]; // v = (1.0, 2.0) T float f = m[2][0];

40 Shading Languages GLSL Language Overview GLSL at a glance C/C++-like syntax Built-in Vector (2d, 3d, 4d) Built-in Matrix (2 2, 3 3, 4 4) Built-in standard library of functions Variable qualifiers Built in variables Examples Trigonometric sin, tan, atan... Common abs, sign, min, max... Geometric length, dot, cross... Vector relational

41 Shading Languages GLSL Language Overview GLSL at a glance C/C++-like syntax Built-in Vector (2d, 3d, 4d) Built-in Matrix (2 2, 3 3, 4 4) Built-in standard library of functions Variable qualifiers Built in variables Examples const qualifier attribute qualifier uniform qualifier varying qualifier

42 Shading Languages GLSL Language Overview GLSL at a glance C/C++-like syntax Built-in Vector (2d, 3d, 4d) Built-in Matrix (2 2, 3 3, 4 4) Built-in standard library of functions Variable qualifiers Built in variables Examples Uniforms: gl ModelViewMatrix Attributes gl color, gl vertex, gl normal Varyings: gl FrontColor, gl BackColor Special: gl Position, gl FragDepth, gl FragColor

43 Shading Languages Interfacing OpenGL Interfacing OpenGL GLSL is only half the story OpenGL has been extended with several API calls to allow for compiling, installing and interfacing shaders Unfortunately, using a shader involves many steps There are libraries to ease the process

44 Shading Languages Interfacing OpenGL Create Shader Objects 1. Create OpenGL managed data structure for each shader 2. Provide strings of GLSL that provides the shaders source code Example GLhandle vs, f s ; vs = g l C r e a t e S h a d e r ( GL VERTEX SHADER ) ; f s = g l C r e a t e S h a d e r ( GL FRAGMENT SHADER ) ; g l S h a d e r S o u r c e ( vs, 10, v s s r c, NULL ) ; g l S h a d e r S o u r c e ( f s, 20, f s s r c, NULL ) ;

45 Shading Languages Interfacing OpenGL Compile shaders and create program object 3. Compile each shader separately 4. Vertex and Fragment Shaders must be combined to a Program Object Example g l C o m p i l e S h a d e r ( vs ) ; g l C o m p i l e S h a d e r ( f s ) ; u i n t prog = glcreateprogram ( ) ; g l A t t a c h S h a d e r ( prog, vs ) ; g l A t t a c h S h a d e r ( prog, f s ) ;

46 Shading Languages Interfacing OpenGL Link, install and use program 5. Our compiled program must be linked 6. Finally we can install the program as part of the current GL-state 7. We issue geometry the normal way, our shader is executed instead of the fixed function pipeline Example gllinkprogram ( prog ) : gluseprogram ( prog ) ; rendergeometry ( ) ;

47 Shading Languages Interfacing OpenGL To summarize Steps for using shaders 1. Create object using glcreateshader 2. Provide source code by calling glshadersource 3. Compile each shader by calling glcompileshader 4. Create a program by calling glcreateprogram 5. Attach shader object by calling glattachshader 6. Link program by calling gllinkprogram 7. Install program as part of current state gluseprogram

48 Shading Languages Interfacing OpenGL Passing parameters to shaders Three types of parameters 1. Uniform variables Global variables for shader program (common uniform namespace for vertex and fragment shaders) Can be updated once pr. primitive 2. Vertex attributes Allows for passing additional vertex data Can be updated for every vertex 3. Varying variables For communication from vertex to fragment shader Value is set pr. vertex and interpolated across primitive

49 Shading Languages Interfacing OpenGL Setting a uniform 1. Activate the program 2. Get the location of the uniform 3. Pass value of uniform Passing uniforms gluseprogram ( prog ) ; i n t l o c = g l G e t U n i f o r m L o c a t i o n ( prog, beta ) ; g l U n i f o r m 1 f ( loc, 0.25 ) ;

50 Shading Languages Interfacing OpenGL Passing vertex attributes 1. Activate the program 2. Get the location of attributes 3. For every vertex, pass vertex data Example gluseprogram ( prog ) ; i n t l o c = g l G e t A t t r i b L o c a t i o n ( prog, tangent ) ; g l B e g i n ( GL TRIANGLES ) ; g l N o r m a l 3 f v ( n o r m a l s [ 0 ] ) ; g l C o l o r 3 f v ( c o l o r s [ 0 ] ) ; g l V e r t e x A t t r i b 3 f v ( l o c, t a n g e t s [ 0 ] ) ; g l V e r t e x 3 f v ( v e r t i c e s [ 0 ] ) ;. g l E n d ( ) ;

51 Shading Languages Interfacing OpenGL Varying Variables 1. Specify with same name and type in both vertex and fragment shaders 2. Set value in vertex shader 3. Read in fragment shader Example / VERTEX SHADER / v a r y i n g vec3 Normal ; v o i d main ( ) { Normal = gl NormalMatrix gl Normal ;. } / FRAGMENT SHADER / v a r y i n g vec3 Normal ; v o i d main ( ) { vec3 d i f f u s e = dot ( Normal, l i g h t V e c ) ;. }

52 Shading Languages Interfacing OpenGL Specifying textures 1. Activate texture as regular OpenGL 2. Specify texture as uniform (glgetlocation etc.) 3. Read and use in either vertex or fragment shader Shader texture example uniform sampler2d t e x t u r e ; void main ( ) { vec4 t e x = t e x t u r e 2 D ( t e x t u r e, gl TexCoord [ 0 ]. xy ) ;.. }

53 Shading Languages Example: Per-pixel lighting Example: Per-pixel lighting Fixed Function Approach Standard OpenGL calculates lighting per vertex and interpolates across primitive (Gouraud shading) This results in: Missing highlights Failure to capture spotlight effects Mach-banding Remedy: More vertices or Phong shading

54 Shading Languages Example: Per-pixel lighting Gouraud vs Phong shading Gouraud shading Lighting at each vertex is calculated Lighting samples are interpolated across triangle Phong shading Shading normal is interpolated across triangle Lighting at each pixel is calculated using the interpolated normal

55 Shading Languages Example: Per-pixel lighting Fixed function pipeline does not provide Phong shading Using programmable hardware it is a breeze! Shader setup Vertex shader passes normal along Lighting calculation in fragment shader

56 Shading Languages Example: Per-pixel lighting Lighting recap Local lighting equation i total = i ambient + i specular + i diffuse Diffuse component Based on Lambert s law For surfaces that are ideally diffuse, the reflected light is determined by the cosine between the surface normal n and the light vector l. i diffuse = n l = cos φ i diffuse = 0 if the surfaces faces away from the light.

57 Shading Languages Example: Per-pixel lighting Specular component Specular component h n l light source Makes a surface shiny by creating highlights View-dependent term v eye φ Blinn half-vector equation p i specular = n h h is the normal of the plane through the point p that reflects light from l into the camera h = l + v l + v

58 Shading Languages Example: Per-pixel lighting Vertex Shader v a r y i n g vec3 Normal ; void main ( ) { Normal = g l N o r m a l M a t r i x g l N o r m a l ; g l P o s i t i o n = g l M o d e l V i e w P r o j e c t i o n M a t r i x g l V e r t e x ; g l F r o n t C o l o r = g l C o l o r ; } Space transformation Input to the vertex shader is in object space, so we must do the transform into eye coordinates ourselves.

59 Shading Languages Example: Per-pixel lighting Fragment shader v a r y i n g vec3 Normal ; v o i d main ( ) { vec3 l i g h t V e c = g l L i g h t S o u r c e [ 0 ]. p o s i t i o n ; vec3 difmat = g l F r o n t L i g h t P r o d u c t [ 0 ]. d i f f u s e. vec3 specmat = g l F r o n t L i g h t P r o d u c t [ 0 ]. s p e c u l a r. xyz ; } vec3 NormalVec = n o r m a l i z e ( Normal ) ; // c a l c u l a t e h a l f a n g l e v e c t o r vec3 eyevec = vec3 ( 0. 0, 0. 0, 1. 0 ) ; vec3 h a l f V e c = n o r m a l i z e ( l i g h t V e c + eyevec ) ; // c a l c u l a t e d i f f u s e component vec3 d i f f u s e = vec3 ( max ( 0. 5 dot ( NormalVec, l i g h t V e c ), 0. 5 ) ) d i f M a t ; // c a l c u l a t e s p e c u l a r component vec3 s p e c u l a r = vec3 ( max ( dot ( NormalVec, h a l f V e c ), 0. 0 ) ) ; s p e c u l a r = pow ( s p e c u l a r. x, ) specmat ; // combine d i f f u s e and s p e c u l a r c o n t r i b u t i o n s and o utput f i n a l v e r t e x c o l o r g l F r a g C o l o r. rgb = d i f f u s e + s p e c u l a r ;

60 Shading Languages Example: Per-pixel lighting CPU application u i n t phong ; v o i d i n i t G L ( ) { GLhandle vs, fp ; vs = glcreateshader ( GL VERTEX SHADER ) ; glshadersource ( vs, 10, vssrc, 0 ) ; f s = glcreateshader ( GL FRAGMENT SHADER ) ; glshadersource ( fs, 10, f s s r c, 0 ) ; glcompileshader ( vs ) ; glcompileshader ( f s ) ; phong = glcreateprogram ( ) ; g l A t t a c h S h a d e r ( phong, vs ) ; glattachshader ( phong, f s ) ; g l L i n k P r o g r a m ( phong ) ; } v o i d r e n d e r ( ) { gluseprogram ( phong ) ; g l u t S o l i d S p h e r e ( 1. 0, 10, 10 ) ; gluseprogram ( 0 ) ; }

61 Shading Languages Example: Per-pixel lighting Result Gouraud Gouraud subdivided Phong

62 Shading Languages Example: Per-pixel lighting Bump mapping The previous example hints at another way of doing lighting. Instead of interpolating the normal in the vertex shader, we could look it up in a texture, thereby reducing the amount of geometry we have to send. (Image courtesy of Paolo Cignoni)

63 Hardware Architecture and Stream Computing Introduction Programmable Graphics Pipeline Shading Languages GLSL Language Overview Interfacing OpenGL Creating shaders Passing parameters to shaders Example: Per-pixel lighting Hardware Architecture and Stream Computing GPU Hardware Stream Computing Example: The Wave Equation Conclusion

64 Hardware Architecture and Stream Computing GPU Hardware GPU Hardware Parallel design Rendering applications are embarrassingly parallel GPUs take advantage of this fact in hardware Hardware specification The last year has seen great changes in GPU hardware design. We will look at the latest and greatest: Nvidia G Stream Processors (SP) Core clock up to 1.35GhZ Up to 2GiB texture memory Pre 2007: Dedicated vertex and fragment units

65 Hardware Architecture and Stream Computing GPU Hardware G80: Block Diagram Image courtesy of Nvidia Corporation

66 Hardware Architecture and Stream Computing GPU Hardware G80: Overview The architecture is based on the notion of threads Each thread is one of: Vertex Primitive Fragment At full speed, thousands of threads are in flight at the same time The architecture is blocked, meaning: 16 vertices are processed simultaniously 32 pixel are processed simultaniously...and they take the same branch Output from on block is fed into SPs at input for next logical stage This is called Unified Shaders (also on XBox360)

67 Hardware Architecture and Stream Computing GPU Hardware G80: SP details Image courtesy of Nvidia Corporation Each SP is a scalar unit Pre 2007: 4-component vector units Each SP can do 1 MAD (multiply-and-add) and 1 MUL (multiply) each cycle Almost IEEE 754 Single Precision Floating Point SP can write to memory 16 SPs share 4 texture address units, 8 texture filter units, 1 L1 cache

68 Hardware Architecture and Stream Computing Stream Computing Stream computing GPUs is one example of Stream Processors Other examples include: Old Cray Supercomputers Current NEC supercomputers IBM Cell BE (powering Sonys Playstation 3) The GPU is the only type of parallel processor that has seen widespread success Because developers generally don t know they are parallel Renewed and immense interest in stream computing for non-graphics, non-entertainment computing GPGPU (General Purpose Computation using GPUs)

69 Hardware Architecture and Stream Computing Stream Computing GPGPU Due to high floating point power, many scientific applications has been ported to GPUs Usually programmed through graphics APIs Unnatural programming model Requires deep knowledge of the hardware New languages (usually C/C++ variants), with different back-ends are coming. No clear winner yet Line between graphics programming and GPGPU programming is blurred. Many graphic engines uses multiple passes already Bottleneck is CPU GPU communication

70 Hardware Architecture and Stream Computing Stream Computing Successful GPGPU applications Worldwide: Sintef project: Games (obviously) Various PDEs PDEs Fast Fourier Transform Linear Algebra Conjugate Gradients Signal and image processing Silhouette detection and Geometric Computing refinement Advanced rendering Surface intersections Databases! Algebraic surface visualization See survey by John Owens We typically see speedups of 10 compared to optimized CPU solutions

71 Hardware Architecture and Stream Computing Example: The Wave Equation Example: The Wave Equation δ 2 u δt 2 = 2 u Using a forward difference scheme this reduces to: u t+1 i,j = 2u t i,j u t 1 i,j + k h 2 (ut i+1,j + u t i 1,j + u t i,j+1 + u t i,j 1 4u t i,j) + = t 1 t t + 1

72 Hardware Architecture and Stream Computing Example: The Wave Equation CPU Application RAM Vertex Processor Textures Assembly Cull Project Rasterize Texture memory Fragment Processor Fragement & Framebuffer operations Render to texture Rendertarget (Framebuffer) For every timestep (1, 1) Binds the two last grids as textures Set up an output buffer Draw a single quad (0, 0)

73 Hardware Architecture and Stream Computing Example: The Wave Equation CPU Application RAM Vertex Processor Textures Assembly Cull Project Rasterize Texture memory Fragment Processor Fragement & Framebuffer operations Render to texture Rendertarget (Framebuffer) The Vertex Processor Calculates texture coordinates Passes everything trough Vertex Shader varying vec4 Xcoord; varying vec4 Ycoord; Xcoord=gl MultiTexCoord0.yxxx + vec4(0.0,0.0,-1.0,1.0) Ycoord=gl MultiTexCoord0.xyyy + vec4(0.0,0.0,-1.0,1.0); gl Position = ftransform();

74 Hardware Architecture and Stream Computing Example: The Wave Equation CPU Application RAM Vertex Processor Textures Assembly Cull Project Rasterize Texture memory Fragment Processor Fragement & Framebuffer operations Render to texture Rendertarget (Framebuffer) Fixed function part The geometry is rasterized into pixels Texture coordinates are interpolated

75 Hardware Architecture and Stream Computing Example: The Wave Equation CPU Application RAM Vertex Processor Textures Assembly Cull Project Rasterize Texture memory Fragment Processor Fragement & Framebuffer operations Render to texture Rendertarget (Framebuffer) Calculation The pixel shader calculates our expression u t+1 i,j = 2u t i,j ut 1 i,j + k h 2 (ut i+1,j + ut i 1,j + ut i,j+1 + ut i,j 1 4ut i,j ) Textures are used as arrays Pixel shader varying vec4 Xcoord, Ycoord; uniform sampler2d n, n minus; vec4 tex = texture2d(n, Xcoord.yx); vec4 tex0 = texture2d(n, Xcoord.wx); vec4 tex1 = texture2d(n, Xcoord.zx); vec4 tex2 = texture2d(n, Ycoord.xw); vec4 tex3 = texture2d(n, Ycoord.xz); vec4 texl = texture2d(n minus, Xcoord.yx); gl FragData[0] = 2.0 * tex - texl + (k/h*h)*(tex0 + tex1 + tex2 + tex3-4.0*tex);

76 Hardware Architecture and Stream Computing Example: The Wave Equation CPU Application RAM Vertex Processor Textures Assembly Cull Project Rasterize Texture memory Fragment Processor Fragement & Framebuffer operations Render to texture Rendertarget (Framebuffer) Output Our computation is written to a texture It can immediately be reused in the next step No data is transfered to the CPU Nothing is displayed on screen

77 Hardware Architecture and Stream Computing Example: The Wave Equation CPU Application RAM Vertex Processor Textures Assembly Cull Project Rasterize Texture memory Fragment Processor Fragement & Framebuffer operations Render to texture Rendertarget (Framebuffer) Output Our computation is written to a texture It can immediately be reused in the next step No data is transfered to the CPU Nothing is displayed on screen Since it is on the GPU, visualization is easy

78 Hardware Architecture and Stream Computing Example: The Wave Equation The future The best way to predict the future is to invent it (Alan Kay) Multi-core, multi-chip (today) Unified shaders (2006) Scattered write (2006) Higher lever languages and tools (2006) Double precision (2007) CPU/GPU convergence (2010?) Basic primitive will still be triangles (2015) 6 GhZ, 32 GiB FrameBuffer (2015)

79 Hardware Architecture and Stream Computing Example: The Wave Equation The future The best way to predict the future is to invent it (Alan Kay) Multi-core, multi-chip (today) Unified shaders (2006) Scattered write (2006) Higher lever languages and tools (2006) Double precision (2007) CPU/GPU convergence (2010?) Basic primitive will still be triangles (2015) 6 GhZ, 32 GiB FrameBuffer (2015)

80 Conclusion Introduction Programmable Graphics Pipeline Shading Languages GLSL Language Overview Interfacing OpenGL Creating shaders Passing parameters to shaders Example: Per-pixel lighting Hardware Architecture and Stream Computing GPU Hardware Stream Computing Example: The Wave Equation Conclusion

81 Conclusion Goals for the day Learn the difference between the fixed function pipeline and the programmable pipeline Understand the role of shaders and shading languages Understand how to write and use shaders using OpenGL and the OpenGL Shading Language Know a little bit about GPU hardware architecture

82 Conclusion Conclusions The Fixed Function Pipeline is dead! Long live programmable shaders! Real-time rendering applications require efficient use of GPUs Shaders allow anything to be done on GPUs Hardware and APIs are still evolving GPU are important today, will be even more important tomorrow Programming GPUs is a heck of a lot of fun!

83 Conclusion For Further Reading Randi Rost OpenGL Shading Language Addison-Wesley 2004 GPU Gems 2 Addison-Wesley 2005 Randima Fernando, Mark J. Kilgard The Cg Tutorial Addison-Wesley 2003

84 Conclusion Thank you for listening! Questions?

85 Conclusion Thank you for listening! Questions?

86 Conclusion Open Master Theses Thank you for listening! Questions? The Sintef GPU project seeks eager master students within the following areas of research: GPGPU Programming Cell BE Programming Visualization