GDC Tutorial: Advanced OpenGL Game Development Reflections, Shadows, Transparency, and Fog March 8, 2000
Mark J. Kilgard Graphics Software Engineer NVIDIA Corporation
Quick Tutorial on Stencil Testing An extra test for fine-grain pixel control Standard OpenGL and DirectX 6 feature Per-pixel test similar to depth buffering Tests fragment against pixel s stencil value, rejects fragments that fail Also, can modify pixel s stencil buffer value based on stencil/depth test results Hardware accelerates stencil testing Typically free when depth testing too
Stencil Testing in the Fragment Pipeline Fragment + Associated Data Pixel Ownership Test Scissor Test Alpha Test Depth Test Stencil Test Blending Dithering Logic Op Frame buffer
Stencil Testing Similar to Depth Testing but Compares current reference value to pixel s stencil buffer value Same comparison functions as depth test: NEVER, ALWAYS LESS, LEQUAL GREATER, GEQUAL EQUAL, NOTEQUAL Stencil bit mask controls comparison (( ref & mask ) op ( svalue & mask )
Stencil Operations Way stencil buffer values are controlled Stencil side effects of Stencil test fails Depth test fails Depth test passes Possible operations Increment, Decrement (saturates) Increment, Decrement (wraps, DX6 option) Keep, Replace, Zero, Invert
Stencil Write Mask Controlling stencil buffer updates Bit mask for controlling write back of stencil value to the stencil buffer Applies to the clear too! Stencil compare & write masks allow stencil values to be treated as sub-fields
OpenGL Stencil API Stencil-related Routines glenable/gldisable(gl_stencil_test); glstencilfunc(function, reference, mask); glstencilop(stencil_fail, depth_fail, depth_pass); glstencilmask(mask); glclear( GL_STENCIL_BUFFER_BIT);
Requesting a Stencil Buffer Have to request one to use one! If using stencil, request sufficient bits of stencil Implementations may support from zero to 32 bits of stencil 8, 4, or 1 bit are common possibilities Easy for GLUT programs: glutinitdisplaymode(glut_double GLUT_RGB GLUT_DEPTH GLUT_STENCIL); glutcreatewindow( stencil example );
Stencil Performance Cheaper than you think With today s 32-bit graphics accelerator modes, 24-bit depth and 8-bit 8 stencil packed in same memory word TNT2 and GeForce are examples Performance implication: if using depth testing, stenciling is at NO PENALTY
Planar Reflections Dinosaur reflected in the floor Easy hack. Draw dino twice, second time has glscalef(1, -1, 1) to reflect through the floor.
Confining the Reflection Reflection should be limited to reflective surface Bad Good, stencil used to limit reflection.
How to do it Draw floor with a unique stencil value Clear stencil to zero. Draw floor polygon and set stencil to one. Only draw reflection where stencil is one.
Recursive planar mirrors Reflection idea can be applied repeatedly Requires more bits of stencil. Note bogus illumination.
A Magician reveals the Trick A Bird s Eye View
Shadows Important visual cue All real world scenes have shadows Lack of shadows = big clue that an image is computer generated! Occlusion from light s point-of-view of-view instead of viewer s Cue for light-object-object relationships Artistic effect Soft vs. hard feel, dramatic or spooky feel
Interactive Lighting Models Typically lack shadow support So-called local lighting models ignore occlusion of light sources that create shadows Except trivial self-shadowing shadowing (i.e., clamping L N) L Global lighting models account for shadows Examples: radiosity, ray tracing But too expensive for interactive use Interactive shadow algorithms typically operate independent of local lighting models
Interactive Shadow Simplications True shadows are expensive, cut corners Simple geometry tricks Projection Volume intersection Pre-compute static shadow results where possible Make simplifying assumptions Treat area light sources as points Exploit hardware features such as stencil
Interactive Shadow Rendering Techniques Different approaches, different tradeoffs Planar projected shadows Stenciled shadow volumes Pre-computed shadow textures (lightmaps) Shadow textures Shadow maps
Planar Projected Shadows So-called fake shadows Uses projective geometry trick Flattens 3D model into ground plane Shadow effect only can be applied to planes Supports either positional or directional lights
Projected Planar Shadow Example
Projected Planar Shadows Classic computer graphics trick Given Ground plane equation, Ax + By + Cz + D = 0 Light position (x, y, z, w) Construct projective transform that squishes vertices into ground plane based on light position Concatenate this with view transform Rendering 3D object creates shadow-like pile of polygons in ground plane
Projected Planar Shadow Matrix 4x4 Projective Matrix P - Lx A -Ly A -Lz A -Lw A -Lx B P - Ly B -Lz B -Lw B -Lx C -Ly C P - Lz C -Lw C -Lx D -Ly D -Lz D P - Lw D where P = Lx A + Ly B + Lz C + Lw D
Projected Planar Shadow Issues Shadow must be cast on infinite planes Stencil can limit shadow to finite planar region Pile of polygons creates Z-fighting Z artifacts Polygon offset fixes this, but disturbs Z values Stencil testing is better solution Difficult to blend shadow with ground texture Just blending creates double blends But stencil testing can eliminate double blends Specular highlights can show in shadow
Planar Projected Shadow Artifacts Bad Good extends off ground region Z fighting double blending
The Shadow Volume Concept Volumetric shadows, not just planar A single point light source splits the world in two shadowed regions unshadowed regions A shadow volume is the boundary between these shadowed and unshadowed regions First described by [Crow 77]
Visualizing Shadow Volumes Occluder and light projects a shadow volume
Shadow Volume Result Objects within the volume are shadowed
Shadow Volume Algorithm High-level view of the algorithm Given the scene and a light source position, determine the shadow volume (harder than it sounds) Render the scene in two passes Draw scene with the light enabled, updating only fragments in unshadowed region Draw scene with the light disabled, updated only fragments in shadowed region But how to control update of regions?
2D Cutaway of a Shadow Volume light source shadowing object surface outside shadow volume (illuminated) shadow volume (infinite extent) eye position partially shadowed object surface inside shadow volume (shadowed)
Tagging Pixels as Shadowed or Unshadowed Using stencil testing High-level algorithm does not say how to update only either pixels in or out of the shadow volume! The stenciling approach Clear stencil buffer to zero and depth buffer to 1.0 Render scene to leave depth buffer with closest Zs Render shadow volume into frame buffer with depth testing but without updating color and depth, but inverting a stencil bit This leaves stencil bit set within shadow!
Stencil Inverting of Shadow Volume Why it works light source shadowing object two inverts, left zero one invert, left one eye position zero inverts, left zero
Visualizing Stenciled Shadow Volume Tagging Shadowed scene Stencil buffer red = stencil value of 1 green = stencil value of 0
Computing Shadow Volumes Harder than you might think Easy for a single triangle, just project out three infinite polygons from the triangle, opposite the light position But shadow volume polygons should not intersect each other for invert trick to work This makes things hard For complex objects, projecting object s 2D silhouette is a good approximation (flat objects are easy) Static shadow volumes can be pre-compiled
For Shadow Volumes With Intersecting Polygons Use a stencil enter/leave counting approach Draw shadow volume twice using face culling 1st pass: render front faces and increment when depth test passes 2nd pass: render back faces and decrement when depth test passes This two-pass way is more expensive than invert And burns more fill rate drawing shadow volumes Inverting is better if no polygon intersections
Why Increment/Decrement Works Example in 2D light source shadowing object zero +1 zero eye position +1 +2 +2 +3
Shadow Volume Issues Practical considerations [Bergeron 86] If eye is in shadow volume, need to determine this and flip enabled & disabled lighting passes Shadow volume only as good as its tessellation Shadows tend to magnify limited tessellation of curved surfaces Must cap the shadow volume s intersection with the near clipping plane Open models and non-planar polygons
Reconstructing Shadow Volume From a Depth Buffer Very clever idea [McCool 98] Render scene from light source with depth testing Read back the depth buffer Use computer vision techniques to reconstruct the shadow volume geometry from the depth buffer image Very reasonable results for complex scenes
Multiple Lights and Shadow Volumes Requires still more rendering passes, but can work! Shadows from different light sources overlap correctly
Shadow Volumes from Area Light Sources Make soft shadows Shadow volumes work for point light sources Area light sources are common and make soft shadows Model an area light source as a collection of point light sources [Brotman & Badler 84] Use accumulation buffer or additive blending to accumulate soft shadow Linear cost per shadow volume sample
Soft Shadow Example Eight samples (more would be better) Note the banding artifacts
Combined Shadow Approaches Two Approaches at Once just logo shadow volume logo lacks shadow on the teapot teapot lacks shadow on the floor just planar projected shadows combined approach
Pre-computed Shadow Textures Lightmaps are static shadow textures Compute lightmaps off-line with radiosity solver local lighting models evaluated on tight grid can work well too Lightmaps with soft shadows can be built faster with convolution approach, accelerated using the Fast Fourier Transform [Soler & Silion 98] Pre-computed shadow textures work well for building interior with lots of diffuse surfaces
Rendering Soft Shadow Textures Fast soft shadows [Heckbert & Herf 96] Render a shadow texture For each shadowed polygon, project all light-source-occluding occluding polygons into the shadowed polygon s plane * Similar to projected planar shadows * Model area light sources as multiple jittered points * Use additive blending to accumulate all contributions Copy resulting image to the shadow texture Draw polygon in final scene with shadow texture
Rendered Soft Shadow Texture Example Shadow Texture and Final Result Shadow texture, accumulation of nine jittered occluder projections Final resulting scene, shadow texture modulated with wooden floor
Hardware Shadow Mapping Shadowing with a depth map texture Software version described by [Reeves 89] Requires hardware support [Segal, et.al. 92] for A depth texture (see SGIX_depth_texture) A depth compare texture filter operator (see SGIX_shadow) Uses eye-linear texgen to generate texture coordinates (s/q,t/q,r/q) in light s coordinate space
Hardware Shadow Mapping Algorithm Two pass approach First, render scene from light s point-of-view of-view with depth testing Copy the depth buffer to a depth texture Second, render scene from eye s point-of-view of-view as follows Configure eye-linear texgen to generate (s,t,r,q) coordinates in light s coordinate space ( r is z-planar z distance to the light) Configure special shadow texture filter operator Texture filter tests if r is less than depth texture texel Texture color is one if tests passes, zero if test fails Modulate fragment color with one or zero texture
Hardware Shadow Mapping Issues Why it works and problems Shadow map texturing is effectively a second occlusion test done with respect to the light Modulate with zero if inside shadow Modulate with one if outside shadow Traditional mipmap texture filtering not suited for filtering a depth map Explicit hardware support currently limited to InfiniteReality and RealityEngine
Transparency See-through objects Multiple objects contribute to pixel s final color Order of objects relative to view affects color Unsorted depth buffering insufficient for hidden surface removal Internal structure of object geometry can become visible With caveats, alpha blending enables transparency
Some Blending is Commutative Order independent blends Filter blending glblendfunc(gl_dst_color, GL_ZERO) Additive blending glblendfunc(gl_one, GL_ONE) But transparency blending is non-commutative glblendfunc(gl_src_alpha, GL_ONE_MINUS_SRC_ALPHA)
Transparency Ordering Rendering order matters Orange sphere in front of blue cone. Overlap should be more orange. Blue cone in front of orange sphere. Overlap should be more blue. Depth testing can mess up transparency. WRONG: Overlap should be blended!
Sorting for Transparency Ensuring proper blending ordering Draw all opaque objects first When transparent object do not intersect, back-to-front sort of transparent objects works Inexact sort may be acceptable When transparent objects intersect, must subdivide and sort intersecting primitives
Sorting Optimizations Transparency shortcuts Take advantage of face culling Draw back faces, then draw front faces Works for non-overlapping overlapping closed convex objects Disable depth testing Prevents front faces from hiding back faces For closed, convex, constant-colored colored objects, order of front and back face drawing does not matter
Screen Door Transparency Example Stipple simulates semi-opacity
Screen Door Transparency Implementation Polygon Stipple OpenGL supports 32x32 binary stipple pattern Percentage of 1 s in screen door pattern should match the object s opacity percentage Different objects should use distinct patterns Limitations Polygon stipple only applies to polygons Stipple pattern tied to window origin
Screen Door Transparency With Alpha Textures Alternative to polygon stipple Place screen door pattern in alpha texture Generate window-space texture coordinates Enable GL_EYE_LINEAR texgen with texgen plane equations mimicking the projection & viewport transforms Projective texturing s division by q will mimic vertex coordinate s division by w Use alpha testing to discard fragments based on screen door alpha texture Applies to both lines and polygons
Multiple Levels of Screen Door Transparency Opacity control 18% 56% 93% 25% 25% 25% Right image shows no sense of transparency between objects! Screen door transparency is defeated if multiple objects share the same stipple pattern.
Sub-pixel Screen Door Transparency Multisample masking SGIS_multisample extension supports multiple color samples per pixel SGI high-end only feature for antialiasing glsamplemasksgis specifies a multisample sub-pixel screen door pattern enabled via glenable(gl_sample_mask_sgis) Because sub-pixel, stipple pattern is not visible Limited number of samples (4 or 8 typical)
Atmospheric Effects Fog, Haze, Smoke As light travels through an atmosphere Some light is extinguished (absorbed) Some light may also be added due to scattering An atmospheric model tells how the atmosphere affects the appearance of objects viewed through the atmosphere
Fog Example Common in Games and Visual Simulation
OpenGL s Built-in Fog Model Assumptions Uniform fog color ( color of the atmosphere) Uniform fog density throughout scene Fog is essentially a weighted blend C = f C r + (1 - f) C f where C r = pre-fog fragment color, C f = uniform fog color, and f = fog factor Fog factor computed per-fragment based on eye-distance of given fragment
Computing the Fog Factor Based on Beer s Law Fog factor has an exponential drop-off off with the distance from the viewer f = e -(d z) Intuition: Clarity is 100% at 0 meters,, but at x + 10 meters is 99% of the clarity at x meters at 30 meters,, the clarity is (0.99 0.99 0.99)% continuous version of this idea is an exponential drop-off off
Standard OpenGL Fog Models Fog factor equations f GL_EXP2 f = e-(d z) GL_EXP f = e-(d z) 2 Zeye Given the fog model (EXP2 or EXP), fog density (d), and Z eye-distance (z), compute fog factor (f)
OpenGL Fog Limitations Emissive objects are over-attenuated Headlights should break through the fog, but don t OpenGL permits implementations to use window Z (a planar distance) instead of the radial eye-distance Objects at sides of the view frustum under-fogged Real world fog is not so uniform Fog varies with altitude, patchy fog
Planar Eye-distance Fog Artifacts Why they occur B planar eye-distance eye position view frustum A radial eye-distance Points A and B have the same Z planar distance but different radial distances.
Texture Fog Methods Textures to encode fog 1D, 2D, or 3D texture as fog lookup table Map eye position to texture coordinates with GL_EYE_LINEAR texgen Use luminance texture with GL_BLEND texture environment and set fog color in texture environment color Or an RGBA texture with GL_DECAL mode can encode the fog factor in A and the fog color in RGB
OpenGL Fog Coordinate Extension User-supplied supplied per-vertex fog distance Normally, OpenGL computes the fog distance automatically EXT_fog_coord extension allows application to supply the fog distance at each vertex via glfogcoord1fext OpenGL interpolates fog distance and computes the fog factor per-fragment using OpenGL fog mode (exponentiates)
A More Complex Fog Model Height-based fog Described by [Perlin 85, Legakis 98] Height-based fog factor equation b α(t) dt a f = e - The extension rate α (density) is integrated over the optical path from point A to point B This integral is the optical depth Assume α varies solely with height (altitude)
Height-based Fog Scenario Consider the optical depth from A to B Y Y 2 Y 1 C A B A = eye position B = fogged position C = position above A at height of B D 1 D 1 D D = (Bx - Ax) 2 + (Bz - Az) 2
Height-based Optical Depth Nice solution, good for glfogcoord b α(t) dt = a D α(y1), Y = 0 Y 1 + ( D / Y ) 2, 2 α(y) dy, Y 0 Y 1 Y 2 α(y) dy = α(y) dy - α(y) dy Y 1 Y 2 - This can be implemented as two 1D table lookups. Y 1 -
Height-based Fog Examples Foggy city normal uniform fog layered fog layered fog