INF3320 Computer Graphics and Discrete Geometry The OpenGL pipeline Christopher Dyken and Martin Reimers 07.10.2009 Page 1
The OpenGL pipeline Real Time Rendering: The Graphics Processing Unit (GPU) (Chapter 3) Buffers and Buffering (Chapter 18.1) Perspective Correct Interpolation (Chapter 18.2) The Red Book: Blending, Antialiasing, and Polygon Offset (Chapter 6) The Framebuffer (Chapter 10) Order of Operations (Appendix A) Page 2
Pipeline operations Page 3
Classic OpenGL has a fixed-function pipeline: Implements a fixed sequence of operations: transformation of vertices lighting calculations for vertices combining vertices to primitives and rasterization texturing of fragments,... Operations can be turned on or off, but is still not too flexible since the operations themselves are quite fixed. Page 4
OpenGL 2.0 introduced a programmable pipeline: Program objects replace part of the pipeline ops with shaders: vertex shaders can replace vertex transform geometry shaders a can replace some of primitive setup fragment shader can replace some of texturing and fragment operations a available through the extension EXT_geometry_shader4 Green = programmable, yellow = configurable, red = fixed Vertex Shader Geometry Shader Clipping Screen Mapping Triangle Setup and Traversal Pixel Shader Merger Page 5
glvertex triggers processing of a vertex. Current color, normal, texcoord etc. are attached to the vertex. Fixed function vertex operations vertex coordinates in current normal current material and color vertex/normal transform lighting glenable(gl_lighting) transformed coordinates associated data processed vertex out Vertex position and normal vectors are transformed. current texcoord current edge tag and fog coord gltexgen() texgen texture matrix glenable(gl_texture_gen_*) Example vertex shader varying vec3 normal; need normal in eye coords in fragment shader void main() { gl_position = gl_modelviewprojectionmatrix * gl_vertex; gl_texcoord = gl_texturematrix[0] * gl_multitexcoord0; gl_frontcolor = gl_color; normal = gl_normalmatrix * gl_normal; } Page 6
Vertex and normal transform object coords window coords modelview matrix eye coords projection matrix perspective division viewport transform clip coords normalized device coords OpenGL uses the following frames: Vertices are specified in object coords. In eye coords, camera is in origin looking down negative z. Lighting calculations are done here. vec4 ec = gl_modelviewmatrix * gl_vertex In clip coords the volume of the frustum is transformed to [ 1, 1] [ 1, 1] [ 1, 1]. vec4 cc = gl_projectionmatrix * ec Normalized device coords is the Euclidean representation of clip-coords. vec3 nd = cc.xyz / cc.w Window coords is the coordinate system of the screen. Page 7
Primitive assembly Page 8
Primitive assembly and culling Primitive assembly combines transformed vertices into points, lines, and triangles according to glbegin-glend state. Triangles are classified 1 as front- or backfacing. When culling is enabled 2, a triangle is discarded if it is backfacing and glcullface(gl_back) is set, or it is frontfacing and glcullface(gl_front) is set. = If an object is closed, we can never see the back-side of the triangles, so why process them further? 1 Check the sign of the 2D-area of the triangle. 2 glenable(gl_cull_face) Page 9
Clipping Page 10
Hyperplanes and half-spaces A hyperplane divides a space into two, a pos and neg side: In R a point is a hyperplane, in R 2 a line is a hyperplane, and in R 3 a plane is a hyperplane. In R d, a hyperplane is defined implicitly by a 1,..., a d+1 : d a i p i + a d+1 > 0 implies that p is in the positive halfspace, i=1 d a i p i + a d+1 < 0 implies that p is in the negative halfspace, i=1 d a i p i + a d+1 = 0 implies that p is on the hyperplane i=1 Page 11
Recall that the point p = (x, y, z, w) is on the homogeneous plane m = [ a b c d ] if x [ ] a b c d y z = ax + by + cz + dw = 0. w We can find the intersection of the plane and a line segment p = (1 t)p 1 + tp 2, t [0, 1], by inserting into the equation above, and solve for t... m p = m ((1 t)p 1 + tp 2 ) = 0, Page 12
t = mp 1 ) gives the parameter point of the intersection. m (p 2 p 1 If p 1 and p 2 have associated data like color c 1 and c 2, the color c at the intersection point can be linearly interpolated as c = (1 t)c 1 + tc 2. Page 13
Clipping line segments in the plane Assume we want to clip against a rectangle in the plane (i.e. R 2 ): Our rectangle is specified by [ ] x [ w, w ] [ h, h ]. y which is the intersection of the four half-planes, m 1 = [ 1 0 w ] x w, m 2 = [ 1 0 w ] x w, m 3 = [ 0 1 h ] y h, and m 4 = [ 0 1 h ] y h. = A point p = [ x y 1 ] T is inside the rectangle, if m i p 0, for i = 1, 2, 3, 4. Page 14
We then clip in the following way: For each hyperplane i = 1, 2, 3, 4, clip the line segment: 1. If both end-points is outside the half-space = reject it. = reject the line segment. 2. If both end-points is inside the half-space = accept it. = accept the line segment. 3. If one end-point is inside, and one end-point is outside = define a new line segment between the end-point inside and the intersection point. Page 15
Clipping convex polygon in the plane The intersection of two convex sets are convex. Half-spaces are convex, and therefore, the clipped result is convex. A polygon is defined as a loop of vertices p 1 p 2 p n p 1 Then, for each half-space 1. Classify each vertex as inside or outside the half-space. 2. for each pair of adjacent vertices where one vertex is inside and one is outside, insert the intersection point between. 3. Remove the vertices classified as outside The remaining loop represents the clipped polygon. Page 16
Rasterization Page 17
Rasterization and the fragment pipeline glenable(gl_fragment_program) point rasterization programmable fragment pipeline fragment program from primitive assembly line rasterization to buffer operations texturing color sum fog polygon rasterization fixed fragment pipeline Rasterization converts primitives into fragments 3. Fragments are then subjected to fragment pipeline and then sent to buffer operations. 3 A pixel before it is written to the framebuffer Page 18
Which fragments do a triangle produce? All fragments whose centre is inside the triangle is produced. A tie-breaking rule is used when the centre is on the boundary. = A fragment belongs only to one of two adjacent triangles. Page 19
Fragment invariance Fragments hold the following invariance rules: Given the primitive p make a new primitive p by offsetting the coordinates with the integers x, y. Assume that neither p or p is clipped. = p and p produces the exact same fragments, except that fragment f of p is shifted x, y compared to fragment f of p. If you render two primitives with identical vertex coordinates, they produce exactly the same fragments. Why? This is of major importance in multi-pass techniques where one renders the same geometry more than once. Page 20
Interpolating varying variables Barycentric coordinates The barycentric coordinates (α 1, α 2, α 3) of p w.r.t the triangle [p 1, p 2, p 3] are α 1 = area(p, p2, p3) area(p 1, p 2, p 3) α 2 = They are the unique convex weights, i.e. area(p, p1, p3) area(p 1, p 2, p 3) α 3 = area(p, p1, p2) area(p 1, p 2, p 3) p2 α 1, α 2, α 3 0, α 1+α 2+α 3 = 1, u 2 s.t p = α 1p 1 + α 2p 2 + α 3p 3 p1 Yields a parameterization, i.e. a natural map between triangles: u = α 1u 1 + α 2u 2 + α 3u 3. p u u1 p u3 3 p 1 (1, 0, 0) p 2 (0, 1, 0) p 3 (0, 0, 1) 1 2 (p1 + p2) ` 1, 1, 0 2 2 1 4 p1 + 3 4 p2 ` 1, 3, 0 4 4 1 3 (p1 + p2 + p3) ` 1, 1, 1 3 3 3 Page 21
Let (α 1, α 2, α 3) be the barycentric coords of the fragment midpoint wrt. the screen-space coordinates of T = [p 1, p 2, p 3]. Interpolating linearly in screen space c 1, c 2, c 3 are the colors at the corners of T. Then, c = α 1c 1 + α 2c 2 + α 3c 3 interpolates the colors linearly in screen space. Looks wrong since perspective distortion is lost! Interpolating linearly in object space Perspective corrected interpolation is given by c = α1 c 1 c w 1 + α 2 c 2 w 2 + α 3 3 w 3 q α 1 q 1 w 1 + α 2 q 2 w 2 + α 3, 3 w 3 q 1, q 2, q 3 are the q-coordinate of the texcoords, for other data q 1 = q 2 = q 3 = 1. Perspective correct interpolation! Turn on with glhint(gl_perspective_correction_hint) Page 22
Fixed fragment pipeline Fragments are then submitted to the fragment pipeline: If texturing is enabled = Use texcoords and do a texture lookup = modify the surface color. If secondary color 4 = add secondary colour to the fragment s color If fog is enabled, = use depth to control a blend between the fragment color and a fog colour. The fragment is then sent to framebuffer operations. 4 If glenable(gl_separate_specular_color) is enabled, the specular contribution from the lighting calculations are maintained in a secondary color which is not tampered with by texturing. Page 23
The framebuffer Page 24
A framebuffer is a set of logical buffers pixel framebuffer height stencil buffer depth buffer blue channel green channel color buffer red channel framebuffer width color buffers (RGBA, front/back, left/right, typically 16 32 bits) depth buffer (typically 24 32 bits) stencil buffer (typically 0 8 bits) accumulation buffer (precision like color buffer or higher),... A pixel is the contents of all logical buffers for a location. Page 25
Basic Buffer operations Clearing glclear[color Depth Stencil Accum] sets the clear value, e.g. glcleardepth(1.0f) glclear(bit-mask) clear the chosen buffers, e.g. glclear(gl_depth_buffer_bit GL_COLOR_BUFFER_BIT) Read and Write gldrawbuffer select col. buffers for writing/clearing glreadbuffer select src for reading (e.g. glreadpixels) Fixed-function pipeline: each output-buffer recieve the same output Programmable pipeline: output-buffers can get individual output Page 26
Write masks specifies which of current framebuffer s logical buffers are updated: glcolormask(bool,bool,bool,bool) specifies if the R,G,B, or A channels should be updated. gldepthmask(bool) specifies if the depth buffer should be updated 1. glstencilmask(bitmask) specifies which of the stencil-buffer bit-planes that should be updated. (Comes in a separate front/back mode too) Render only to the stencil buffer turn off the color and depth masks and enable the stencil mask. 1 Use glenable/gldisable with GL_DEPTH_TEST to control the test itself. Page 27
Per-fragment operations Page 28
All fragments are subjected to a series of tests: Fragment and associated data ownership test scissor test alpha test (RGBA) stencil test write framebuffer read / update read / update Logical operations Dithering blending (RGBA) depth test If all tests succeed = fragment is written to the framebuffer If any test fail = fragment is discarded. Page 29
Fragment tests Pixel ownership test The OS discard fragments at pixel locations not owned by the context. Scissor test (glscissor(x,y,w,h)) All fragments outside a specified rectangle are discarded. Alpha test (glalphafunc(func,ref)) Compare (func) the fragment s alpha value with a reference value (ref) Let us e.g. render opaque and translucent objects, billboarding... Page 30
The stencil test Stencil test Compare value in the stencil buffer with a reference value. glstencilfunc(func,ref,mask) comparison function (never,always,<,<=,!=,...), reference value, and a bitmask for both reference and stencil buffer value Stencil operation The stencil buffer is updated from the result of the stencil and depth-tests. glstencilop specifies how the stencil buffer is updated when: the stencil-test fails the stencil-test passes, but dept-test fail both the stencil-test and the depth-test passes glenable/disable( GL_STENCIL_TEST ) turns the test on/off. Page 31
Stencil test applications DOOM3 shadows: count back and front facing fragments passing depth test Code example: http://www.iua.upf.es/~ggeiger/redbookhtml/ch11.html#id19884 Page 32
The depth test The depth-buffer (z-buffer) algorithm clear by setting all depth-buffer elements to far. for each fragment: if fragment s depth < depth-buffer depth = store fragment and update depth-buffer if fragment s depth depth-buffer depth = discard fragment result is that only the nearest fragment is stored for any sequence of fragments. glenable/disable( GL_DEPTH_TEST ) turns the test on/off. gldepthfunc specifies the depth test. Page 33
What is stored in the depth-buffer? depth(z) = 2 depth bits` z far z far z near 1 z far z near z z far z near depth-buffer value 16 12 8 4 0 z near = 10, z far = 1000 z near = 100, z far = 1000 z near = 100, z far = 5000 z near = 100, z far = 0 500 1000 z value depth is an n-bit integer proportional to 1 (i.e. non-linear). z much precision around the near plane little precision around the far plane Increase depth precision by pushing near plane as far away as possible! The far plane also influences precision, but to a much lesser extent. Z-fighting is the result of too little z-buffer precision Two fragments of different z-values get the same depth value, and the algorithm can t tell which is in front of the other. This results in an ugly pattern where some fragments of one primitive appears in front of the other while others do not. Page 34
The merging stage Incoming (src) fragments are merged into the framebuffer (dst): Depth/z-test: replace dst fragment if src is closer Logical operations: AND, XOR,... Blending: combine src and dst Logical operations, gllogicop(op) dst = src OP dst where OP is a logical operation, e.g. AND, OR,... Example: Rubberband Render the rubberband in white using gllogicop( GL_XOR). Rendering once more erases the rubberband ( (a b) b = a) Page 35
Blending Blending The incoming fragment s is combined with the color already in the framebuffer. src = F ssrc OP F d dst where src and dst are RGBA, F s are blend-factors and OP is e.g. add-operator Set up with gl[enable Disable](GL_BLEND) - enables blending glblendfunc(f s,f d ) - set RGBA blend factors glblendfuncseparate(f s,f d,fs α,fd α ) - separate RGB and α blend factors glblendequation(mode) - can be e.g. add, subtract, min, max, etc. The factors F can be: 0, 1, α, src, dst,... Page 36
Blending applications Blending is most often based on the alpha channel Transparent objects, render back to front Fog Billboarding Anti-aliasing Code example: http://www.iua.upf.es/~ggeiger/redbookhtml/ch07.html#id19353 Page 37
Fog Popular, cheap effect for giving hints on depth c = f c f + (1 f )c b where c f and c s are fog and surface color, and f can be linear: f (z) = z end z z end z start exponential: f (z) = e d f z squared exponential: f (z) = e (d f z) 2 Fog in OpenGL: GLfloat fogcolor[4] = {0.5, 0.5, 0.5, 1.0}; glenable (GL_DEPTH_TEST); //enable depth testing glenable (GL_FOG); glfogi (GL_FOG_MODE, GL_EXP2); glfogfv (GL_FOG_COLOR, fogcolor); glfogf (GL_FOG_DENSITY, 0.3); glhint (GL_FOG_HINT, GL_NICEST); Page 38
Anti-aliasing Points and lines can be antialiased by Enable blending Calling glenable(gl_*_smooth) Setting gllinewidt/glpointsize Filled polygons are more tricky Multisampling - estimate pixel coverage, combine with alpha Accumulation buffer - average over a number of perturbed images Code example: http://www.iua.upf.es/~ggeiger/redbookhtml/ch07.html#id44389 Page 39
Accumulation buffer A color buffer that can accumulate several renderings I glaccum(op,val), e.g. glaccum(gl_accum,0.1) I read from readable buffer (glreadbuffer) I After accumulation, glaccum(gl_return,1) writes back to write to output-buffers Applications: accumulate perturbed images I Antialiasing I Depth of field I Motion blur Page 40