Render-To-Texture Caching D. Sim Dietrich Jr.
What is Render-To-Texture Caching? Pixel shaders are becoming more complex and expensive Per-pixel shadows Dynamic Normal Maps Bullet holes Water simulation Detail Normal Maps Procedural Unique Texturing Marble shader can be dozens of instructions
What is Render-To-Texture Caching? Lightmap updates are expensive, and usually require the CPU Blood, Water and Bullethole decals don t fit into the lighting equation if they are just pasted into the frame buffer Procedural textures are difficult to anti-alias They aren t mip-mapped Anti-aliased shaders are even more costly Multi-sample AA doesn t help
Solution : Store Coherent Terms in Textures Efficiency : Take terms that stay constant for many frames, and cache them in their own texture Quality : Refine and/or re-use terms over many frames. Mip-mapping provides Anti-aliasing. Load Balancing : Opportunistically create textures & mip-maps for upcoming areas or objects, and free old textures Flexibility : Switch between dynamic and cached textures as needed
Example : Player Faces, Clothes & Equipment In RPGs, players like to customize their characters with varying costumes & weapons, etc. Drawing a character with a dozen textures for each piece of equipment, etc. can be very costly Especially when there are many characters in view Instead, every time a player switches clothes, render the new clothes colors & gloss term into the character s unique clothes texture Then, generate a normal map from the result
Example : Player Faces, Clothes & Equipment This way, you can draw a character with a single pass! Rather than paying the overhead for many texture changes and API calls : Draw more characters Do more detailed lighting Draw higher-poly characters
Example : Default Pose Space Normal Maps After creating a unique texture for the clothes, you can then sample this with a pixel shader and create a normal map Requires single pixel-shader pass on GF3+ 2-pass technique on GF1/2 When performing this step, use a vertex shader to put normals into the model s default model space with no animation applied Then, during per-pixel lighting, you can skip tangent space altogether Just put the light vectors into pose space
Character Example Summary By performing a small bit of very fast extra work Rendering into a texture w/ no z buffer you can save tons of calling overhead and vertex shader work Thus allowing more complex scenes, faster framerates or some combination of the two
Example : Cached Shadow Mapping Shadow maps can be expensive if everything is dynamic But, many scenes have at least some static or semi-static lights, why not take advantage of this? Rather than store a lightmap for, say, a wall section, store an occlusion map instead Can refine occlusion term over several passes and/or frames for softer edges
Cached Shadow Mapping Every section of static geometry in the scene could have its occlusion with respect to a light cached Think of replacing lightmaps with occlusion maps Rather than burning in light contributions, have a dynamic map of which lights are visible from that geometry
Cached Shadow Mapping Where does one cache the shadowing term? If shadowing is done in view space Like stencil shadows Or rendering shadow terms to DestAlpha then it s not cacheable As soon as the camera moves, start over Why not in the object s own texture space? Each object or piece of geometry can have its own occlusion map(s) That way, it s view independent and can be saved
Each Triangle in Model Space Maps to a Triangle in Texture Space
Well, usually Triangles in UV space can degenerate into a line or point
Cached Shadow Mapping In other words, render the object s occlusion values into its own texture space! Different objects and areas of objects have more or less texture resolution, reflecting their importance Objects must have a unique texture mapping! Update the occlusion maps via render-to-texture Can t use stencil shadows for the occlusion term Must use render-to-texture style method ObjectID Z Shadow Maps Color Shadow Maps Cookies
Cached Shadow Mapping The main potential advantage to this technique is quality Since you don t have to do all the work every frame, the total amount of visual interest in the scene can be higher Using a surface s own texture space as a cache allows us to enhance the effect over time Imagine getting basic shadow / no shadow for distant surfaces, and smoothing it out over the next few frames
Cached Shadow Mapping Using the frame buffer or stencil to store light occlusion results is inherently uncacheable As soon as the camera moves, or an object moves, or a light moves, it must be recalculated Render to Texture operations are inherently resampling operations You choose the space and frequency in which to sample a function If you choose a neutral space, like world or texture space, it can be saved when the camera moves
How to Implement Cached Shadow Mapping Create the shadow map See Projective Shadows Talk Cache the shadow map results in an object s texture space Optionally refine shadow map as needed or over several frames Make several light-space jittered lookups into the shadow map, and average them And/or Just blur the occlusion terms Use the light s occlusion term during its lighting pass
How to update the Occlusion Map : Vertex Shader Select the object s occlusion map as the render target Set the viewport to match the texture size In a vertex shader, use the lightmap U,V coordinates as the vertex X,Y coordinates, or switch your streams around with Fixed T&L Make UVW your XYZ position stream Turn off back face culling, to ensure you don t throw out triangles UV-mapped backwards
Cached Shadow Mapping One can store 4 occlusion terms per object surface, one each in R,G,B & A Then, during the lighting pass(es) one can get up to 4 light occlusion values with a single texture fetch, and some unpacking math For instance, you could do 4 shadowed diffuse pervertex lights and one shadowed diffuse per-pixel light in a single pass on ps.1.1 HW. Compare to doing 4 passes for 4 diffuse & specular lights with traditional shadow mapping
How to update the Occlusion Map : Pixel Shader Depending on which lights have changed, you can mask off R,G,B and/or A to leave the static ones unchanged If any lights haven t moved for a while, use the other shadow map tests for jittered versions of the same light to smooth its shadows If all 4 lights move, you can update all in one pass If only one moves, you can get 4-way jittered shadow tests in one pass
How to update the Occlusion Map : Pixel Shader Rather than replacing what is in the old occlusion map, you could blend the new occlusion result with what is already there Do a time-based blend, where the newer results get more weight If you stick to a constant cost : Slowly moving lights have soft edges Many moving lights have harder edges Still or slow lights have softer edges You could always do more passes to get them all soft-edged
How to update the Occlusion Map : Pixel Shader When choosing shadow map jitter locations, use a rotated-grid or pseudo-random pattern, which varies along the light s projection plane See Projective Shadows Talk Ordered Grid-aligned sampling gives fewer nearhorizontal and vertical values compared to jittered or rotated approaches
How to update the Occlusion Map : Pixel Shader In the pixel shader, perform your shadow map test(s), and store the result in the color With Z-Based shadows, you can do 4 lights in a single pass def c0, 1.0, 0.0, 0.0, 0.0 def c1, 0.0, 1.0, 0.0, 0.0 def c2, 0.0, 0.0, 1.0, 0.0 tex t0 // do shadow map test for light 0 tex t1 // do shadow map test for light 1 tex t2 // do shadow map test for light 2 tex t3 // do shadow map test for light 3 mul_sat r0, t0, c0 // put light 0 s occlusion in red mad_sat r0, t1, c1, r0 // put light 1 s occlusion in green mad_sat r0.rgb, t2, c2, r0 // put light 2 s occlusion in blue + mov r0.a, t3.a // put light 3 s occlusion into alpha
When You Can Skip Updating the Occlusion Map When the shadow-receiving geometry is not likely to be in view for a while Probably best to do distance based instead of viewbased, b/c players can change facing quickly in some games When you decide to do regular shadow map testing for dynamic objects ( ie non-cached ) Occlusion terms can be modulated together, and shadowed always wins, so you could do : MyShadow = ( MyStaticShadow * DynamicShadowTest )
Shadow Summary Caching Shadow results allows an app to achieve higher quality at a better frame rate than either fully dynamic or fully static techniques
Render To Texture Caching Another way to take advantage of RTT Caching is to apply decals to an object It s definitely easy to draw little quads into the scene, but : They typically aren t properly lit The normals don t really line up with the underlying surface They can increase fill requirements Engines typically have a limited # of them, so they go away over time
RTT Caching : Decals What if we could render the little quads directly into an object s texture? That way, we would only pay the fill cost once, and it would be free from then on It would require more texture memory for each object to have its own diffuse and/or normal map But we only need things nearby to be in memory
RTT Caching : Decals For each nearby piece of geometry that can receive decals, have a diffuse & normal map in which to render decals Blood, bullet holes, etc. Each time a decal is created, remember what kind and where So we can flush out textures & recreate as needed Render decals into unique per-surface texture Generate mip-maps via RTT or auto-generated
RTT Caching : Decals Render decals into unique per-surface texture Turn off Back Face Culling If decal s quad touches > 1 triangle, draw decal into each one Be careful of double hits in case your quad overlaps itself on multiple triangles One geometric edge may correspond to many areas in the texture map If you are summing normals, perform a signed add, and re-normalize Or, could manipulate heights directly and generate normals from them
RTT Caching : Decals Once decals are burned into the texture, you can create mip-maps for them Either all at once or over several frames Render decals into each mip-level Watch for geometric aliasing Render into top-level only Render bilinear quads into mip-maps Let FOURCC code create mip-maps for you
RTT Caching : Decals Summary With this technique : + Decals can be properly normal mapped & lit + Any # of decals can be supported + Fill costs are lower than rendering many quads + Properly anti-aliased - More complicated than other techniques - More texture memory required - RTT overhead, although amortized
RTT Caching Summary You can get more complex effects by amortizing their cost over time Perform when things change, not every frame To use RTT Caching, you need A unique texture space for each cached surface Enough texture memory for uncompressed RTT targets Consistent texel resolution No degenerate triangle mappings Spatially coherent texture mapping Triangles in model space correspond to a triangle of the same basic shape in texture space