Chapter IV Fragment Processing and Output Merging
Fragment Processing The per-fragment attributes may include a normal vector, a set of texture coordinates, a set of color values, a depth, etc. Using these data, the fragment processing stage determines the final color of each fragment. Two most important things to do in the fragment processing stage Per-fragment lighting (presented in Chapter 5) Texturing (presented in Chapters 8, 9, and 10). Let s see what texturing is. 4-2
Texture Coordinates An image texture is a 2D array of texels (texture elements). Each texel has a unique address, i.e., 2D array index. Let s map the texture into a square in the parameter space and use normalized texture coordinates (u,v) instead of the texel address. Then, multiple images of different resolutions can be glued to a surface without changing the texture coordinates. 4-3
Surface Parameterization The texture coordinates are assigned to the vertices of the polygon mesh. This process is called surface parameterization or simply parameterization. In general, parameterization requires unfolding a 3D surface onto a 2D planar domain. 4-4
Chart and Atlas The surface to be textured is often subdivided into a (hopefully small) number of patches. Each patch is unfolded and parameterized. Then, the artist draws an image for each patch. An image for a patch is often called a chart. Multiple charts are usually packed and arranged in a texture, which is often called an atlas. 4-5
Texture Coordinates to Texel Address Scan conversion is done with the texture coordinates. D3D performs the following computation to convert the texture coordinates to the texel address. t t x y = ( u S = ( v S x y ) 0.5 ) 0.5 4-6
Texturing Examples Observe that multiple images of different resolutions can be glued to a surface. 4-7
Texture Coordinates (revisited) Direct3D and OpenGL adopt different parameter spaces for texture coordinates. When a textured model is exported between Direct3D and OpenGL, the v- coordinates should be converted into (1-v). 4-8
Texture Addressing Mode The texture coordinates (u,v) are not necessarily in the range of [0,1]. The texture addressing mode handles (u,v)s outside the range. Border: The out-of-range coordinates are rendered with a separately defined border color. Wrap, repeat, or tile: The texture is tiled at every integer junction. Mirror: The texture is mirrored or reflected at every integer junction. We have smooth transition at the boundaries. 8-9
Texture Filtering Magnification & Minification Consider a quad. For each fragment located at (x,y) in the screen, its texture coordinates (u,v) are mapped to the texel address (t x,t y ). We say that the fragment at (x,y) is projected onto (t x,t y ). Note that (t x,t y ) are floating-point values in almost all cases. Consequently, texels around (t x,t y ) are sampled. This sampling process is called texture filtering. The screen-space quad may appear larger than the image texture, and therefore the texture is magnified so as to fit to the quad. There are more pixels than texels. 8-10
Texture Filtering Magnification & Minification (cont d) In contrast, the screen-space quad may appear smaller than the image texture, and the texture is minified. The pixels are sparsely projected onto the texture space. Summary Magnification: more pixels than texels Minification : less pixels than texels 8-11
Filtering for Magnification Option 1: Nearest point sampling A block of pixels can be mapped to a single texel. Consequently, adjacent pixel blocks can change abruptly from one texel to the next texel, and a blocky image is often produced. 8-12
Filtering for Magnification (cont d) Option 2: Bilinear interpolation It is preferred to nearest point sampling not only because the final result suffers much less from the blocky image problem but also because the graphics hardware is usually optimized for bilinear interpolation. 8-13
Filtering for Minification Consider the following checker-board image texture. If all pixels are surrounded by dark-gray texels, the textured primitive appears dark gray. If all pixels are surrounded by light-gray texels, the textured primitive appears light gray. 8-14
Z-buffering The output of the fragment program is often called the RGBAZ fragment. A or alpha for representing the fragment's opacity Z or depth used for z-buffering Using alpha and depth values, the fragment competes or is merged with the pixel of the color buffer. The z-buffer has the same resolution as the color buffer, and records the z- values of the pixels currently stored in the color buffer. When a fragment at (x,y) is passed from the fragment program, its z-value is compared with the z-buffer value at (x,y). If the fragment has a smaller z-value, its color and z-value are used to update the color buffer and z-buffer at (x,y), respectively. Otherwise, the fragment is discarded. 4-15
Z-buffering (cont d) Assume MaxZ is 1.0, the red triangle s depth is 0.8, and the blue triangle s is 0.5. 4-16
Z-buffering (cont d) Rendering-order independence!! 4-17
Ed Catmull Founder of Pixar and now president of Walt Disney and Pixar Animation Studios Academy awards 1993 Academy Scientific and Technical Award "for the development of PhotoRealistic RenderMan software which produces images used in motion pictures from 3D computer descriptions of shape and appearance" 1996 Academy Scientific and Technical Award "for pioneering inventions in Digital Image Compositing" 2001 Oscar "for significant advancements to the field of motion picture rendering as exemplified in Pixar's RenderMan" 2008 Gordon E. Sawyer Award for an individual in the motion picture industry whose technological contributions have brought credit to the industry Ed Catmull s work texture mapping z-buffering bicubic patches and subdivision surfaces 4-18
Alpha Blending The alpha channel in the range of [0,1] 0 denotes fully transparent. 1 denotes fully opaque. A typical blending equation is described as follows: For alpha blending, the primitives cannot be rendered in an arbitrary order. They must be rendered after all opaque primitives, and in back-to-front order. Therefore, the partially transparent objects should be sorted. 4-19
Z-culling The z-buffering test is done after the fragment processing stage. When the z- value of the fragment is greater than the buffered value, the fragment is determined to be occluded and thus discarded. It s inefficient!!! Let s kill the fragments before the fragment processing stage if possible. A tile is composed of nxn pixels, and records the maximum among their z-values. Let s compare it with the minimum z-coordinate of the three vertices of a triangle. 4-20
Z-culling (cont d) Z-culling is powerful, and is enabled by default. Consider the scene, where the first object occludes almost of triangles of the other objects. In a test, the average frame rate with the front-to-back ordered objects is 12.75 fps whereas that with the back-to-front ordered objects is 2.71. This shows that the triangles may need to be sorted in front-to-back order in order to maximize the performance increase brought by z-culling. 4-21
Pre-Z Pass Two-pass algorithm 1 st pass: The scene is rendered with no shading, e.g., no lighting and no texturing. Then, the color buffer is not filled, but the z-buffer is filled with the depths of the visible surfaces of the scene. 2 nd pass: The scene is rendered with full shading. Then, z-culling may cull out all fragments that are occluded by the visible surfaces. No occluded fragment enters the fragment processing stage. The pre-z pass algorithm even with back-to-front ordered objects outperforms the single-pass front-to-back rendering. 4-22