Interactive Radiosity Using Mipmapped Texture Hardware

Transcription

1 Eurographics Workshop on Rendering (2002), pp. 1 6 Paul Debevec and Simon Gibson (Editors) Interactive Radiosity Using Mipmapped Texture Hardware Eric B. Lum Kwan-Liu Ma Nelson Max Department of Computer Science, University of California Davis Abstract We present a method for interactive radiosity using mipmapped textures as a multiresolution representation of patch hierarchies. The texture representation allows for the use of highly refined patches without the geometric overhead of polygon-based techniques. Mipmapping acts as a hardware accelerated oracle mechanism for the dynamic linking of patch hierarchies. Using a single PC, we are able to achieve sub-second radiosity for a model with over ten-thousand patches. 1. Introduction The radiosity method generates realistic images by solving the rendering equation 13 for Lambertian surfaces. Once a radiosity solution is found, graphics hardware can be used to render the scene, changing the viewpoint at interactive rates. On many occasions, however, it is desirable to be able to change parameters other than viewpoint, such as material or light properties, or to add, remove, or move lights and objects. Much work has been devoted to incremental techniques to accelerate the process of calculating the radiosity solution in such dynamic environments Our work differs in that in our interactive technique provides recalculation of the entire radiosity solution from scratch, making no assumptions about types of changes, and therefore allows for arbitrary changes to be made. Of particular relevance to our work is the technique describe by Hanrahan et al. 11 that uses a subdivision hierarchy of patches. We present a method for interactive radiosity using mipmapped textures as a multiresolution representation of patch hierarchies. Mipmap textures are created that encode the patch IDs each texel represents. Every patch at each refinement level is assigned a unique ID. This ID is encoded as a red, green, blue, alpha set that is later rendered to the frame buffer during the calculation of form factors. Thus, any given RGBA color can be decoded to a specific polygon, patch hierarchy level, and patch. (In our implementation colors are stored using 32-bits which limits the total number of subdivision patches to just over 4 billion.) The use of textures allows for the use of highly refined patch hierarchies that can capture subtle shading features, without the geometric overhead in data structures and polygon transmission and setup of a polygon-based representation. Mipmapping on the other hand acts as a hardware accelerated oracle mechanism for the dynamic linking of patch hierarchies. This technique permits high quality radiosity calculations with the speed advantages that come from the use of graphics hardware. In particular, using this technique we can calculate radiosity solutions, with over ten of thousand patches, in under a second on a 1.6 Ghz PC with a GeForce3 card. 2. Related Work The radiosity of a surface is the radiant power or flux (often, and here also, simply called energy) per unit area leaving a diffuse surface. Radiosity methods divide the surfaces in an environment into discrete patches, each with an unknown radiosity B i, and solve the following energy balance equations expressing the flux leaving patch i as the sum of its emissive flux (for light sources) plus the reflected flux, computed as a linear combination of the radiosities of all the other patches. A i B i = A i E i + ρ i A j B j F ji j Here ρ i is the reflectivity of patch i, E i is its emissivity, and A i is its area. (The area factors are required to convert radiosity into total flux over the whole area of the patch.) The "form factor" F ji from patch j to patch i is the fraction of the flux leaving patch j that arrives at patch i. See 6 and 20 for further details. Cohen et al. 5 developed the hardware-based hemicube method of computing these form factors by placing a cube

2 2 Lum, Ma and Max / Interactive Radiosity Using Mipmapped Texture Hardware at the surface of one of the patches, with its top face parallel to the surface. The top face, and half of each of the side faces, become windows for hardware rendering. Each pixel in these windows has a precomputed "delta form factor" expressing the fraction of the flux leaving in the direction of the pixel s solid angle. All the other patches are rendered into these windows with colors representing unique patch IDs. The image is read back from the frame buffer, and the flux fraction represented by the delta form factor of each pixel is summed into the form factor represented by the pixel s ID value. Our innovation is to render the ID values using texture maps, with the GL_NEAREST_MIPMAP_NEAREST setting, so that the polygon ID, refinement level, and patch ID are written directly to the color buffer without texture interpolation. Hanrahan et al. 11 describe how multilevel patch hierarchies can be used, with source and receiver patches treated at varying levels-of-detail. Their methods allow for the efficient solving of a radiosity solution by linking source and receiver patches at appropriate hierarchy levels. They use an oracle function for determining the linking of nodes that is based on an estimate of unoccluded form factors. Similar to our use of texture-maps to store form factor IDs, Max and Allison 16 use Gouraud shading hardware to interpolate basis functions along triangular elements. Our work differs from theirs in that the textures used in our work are rendered perspectively correct by modern texture mapping hardware, while mipmapping provides antialiasing. Fernando et al. 8 use mipmaps for the estimation of projected texel area for the generation of adaptive shadowmaps. This shares similarities with the use of mipmaps for the linking of patch hierarchies. Gershbein et al. 9 treat reflectivity and emissivity textures with wavelets that are an enhancement of the ideas of hierarchical radiosity. They project both the incoming irradiance and the reflectivity texture onto the wavelet basis functions at the various levels of resolution and maintain only those levels that the interaction oracle determines are necessary. They use a gathering iteration and do not take advantage of hardware. In contrast, we do progressive "shooting" and use hardware textures for patch IDs. Keller 14 describes a technique he calls instant radiosity which does particle shooting from light sources determined using quasi-monte Carlo integration for the starting position and direction of the particle, and then path tracing for subsequent bounces. Then hardware rendering with shadows is performed from the desired viewpoint, as lit from each particle start or bounce point, and these images are summed in an accumulation buffer. The result is an image from a single viewpoint, rather than a radiosity solution good for an arbitrary viewpoint. Heckbert 12 and Chen et al. 2 have used textures to record caustic maps, both for final rendering and for further light propagation. We do not consider specular reflection or transmission so we do not use caustic textures. Chen et al. 2 also do a final gather of their progressive radiosity solution, and we could use texture maps to do a similar final gather in hardware (see the future work section below). Myszkowski and Kunii 17 and others have used textures to speed up the final rendering of a radiosity solution, and our texture-based radiosity solution also benefits from the same rendering speed-up. Soler and Sillion 23 have used textures from the convolution-based shadow method of Max 15 and Soler and Sillion 22, integrated into a hierarchical radiosity method with clustering This is a different approximation to the partial visibility determination, which requires software intervention for the convolution step. However, they get very good performance, and their integration with clustering goes beyond what we have attempted. Section 2.2 of Soler and Sillion 23 surveys other related uses of textures in radiosity. 3. The Technique We have applied our mipmap technique to progressive radiosity shooting, as in Cohen et al. 4, with modifications to accommodate efficiently the subdivision hierarchy. With shooting, the radiosity B and unshot radiosity B for all patches are initialized to the emissivity, which is zero except at the light sources. Then a succession of shooting patches P i are chosen, and the energy of all patches is updated by shooting unshot energy from P i to all other patches. Shooting is typically performed by the following steps: 1. Select the shooting patch P i based on an unshot power criterion. 2. Calculate "column" form factors from that patch 3. Update the radiosity and unshot radiosity of receiving patches as described by B j = B j + ρ j F ij A i A j B i B i = 0 B j = B j + ρ j F ij A i A j B i where B is the patch radiosity, B is the unshot radiosity, i is the shooting patch, j is a receiving patch, ρ j is the reflectivity of patch j, and F ij is the form factor from patch i to patch j. In order to accommodate the mipmap hierarchy several changes are made to the algorithm. First, the shooting patch is selected using a greedy heuristic, described in Section 4.2, that delays push-pull operations on the hierarchies. The hemicube 5 is used with shooting patch size and resolution adapted based on the unshot energy of the patch hierarchy. During the form factor calculation step, the mipmapped texture IDs are used not only to calculate form factors, but also

3 Lum, Ma and Max / Interactive Radiosity Using Mipmapped Texture Hardware 3 to provide linking between the shooting patch and receiving patches at adapted subdivision levels. Finally, for the "update radiosity" step, rather than updating the radiosity B and unshot radiosity B, only an unpushed radiosity I is updated, that is subsequently used to update the B and B terms during the delayed push-pull operation. This is discussed in Section 4.1. The role of mipmapping can be thought of as a multiresolution patch linking oracle, or it can be treated as a simple antialiasing mechanism. For a patch linking point of view, mipmapping has the effect of linking a shooting patch to receiving patches at subdivision levels such that each receiving patch projects to a uniform number of elements on the hemicube. Those patches with small form factor value, projecting to sub-pixel size in the hemicube, get merged to a coarser node in the hierarchy, while those patches that project to a large number of elements are refined further. In addition, we use a lower resolution hemicube when shooting less unshot power, resulting in polygons projecting to less area on the screen, and thus resulting in coarser mipmap levels. The overall effect is that source to receiver links have relatively uniform unshot energy, so that computation is evenly distributed based on energy contribution to the radiosity solution. From a sampling perspective, mipmapping provides antialiasing of the regularly sampled hemicube. The highly refined patches we use in our work make antialiasing particularly important. With traditional hemicube-based methods, if the hemicube is not calculated at sufficiently high resolutions, streaking artifacts from aliasing can occur. With our method, if a low resolution hemicube is used, the quality of the rendering degrades much more gracefully, yielding softer shadows, rather than streaky artifacts. Thus, patches can be refined to very fine levels, without the strict requirement of using a high resolution hemicube. 4. Implementation Details 4.1. Hierarchy Updating For each polygon a quadtree patch hierarchy is stored. Each patch contains the radiosity B, the unshot radiosity B, and the unpushed radiosity I. During shooting only the I part of each node is updated, which is used later during the delayed push-pull operations. In addition to the hierarchy, for each polygon the total unshot power is stored and kept upto-date during the shooting process, in order to guide the choice of the next shooting polygon, as described in the next section. When patch i is shot, a hemicube is placed at a jittered sample within its area, and the scene is rendered using the ID textures. The ID values in the retrieved frame buffer are read in scanline order. Whenever a run of identical ID values terminates, the polygon p and its hierarchical patch j are retrieved from the ID. The sum of the pixel delta form factors A for the run is multiplied by the factors ρ i p A j and added to the I j for the patch, and it is also multiplied by ρ pa i and added to the total unshot power for polygon p. The push-pull operation described in 11 consists of pushing the received radiosity I to the leaves of the hierarchy by adding radiosity received at higher levels and then pulling energy toward the root by averaging the reflected radiosity from lower levels. Before shooting, a push-pull must occur so that the push-pull I values can be added to the B and B parts of all nodes in the hierarchy. Applying a push-pull to all polygons can be expensive since the cost of the operation is proportional to total number of patches, so the number of push-pulls should be minimized Selection of Shooting Patch Typically, the shooting patch selected is the patch with the highest unshot energy. The use of patch hierarchies makes this much more difficult because of the sheer number of patches, as well as the fact the unshot power isn t necessarily "known" for a given patch, since the push-pull operation may not have occurred. To avoid a push-pull operation on all patches, the shooting patch is chosen using an approximate greedy selection criteria as follows: 1. Select the polygon with greatest unshot power and apply push-pull 2. Recursively traverse down the patch hierarchy from the root in a greedy manner, following the patch with the greatest unshot power 3. Stop just before the unshot power or size of child patches reaches below threshold, and shoot Although this method does not find the optimal shooting patch, in practice we have found it to work well, and it only requires one push-pull operation per shooting polygon. Further push-pulls can be avoided by selecting patches from the same polygon for consecutive shooting iterations. Since texture memory is limited, it can be impractical to create a high resolution mipmap ID texture for every polygon in a scene. Fortunately this can be avoided by storing the patch ID within a polygon in the lower bits of the texture color (zeroing the higher bits), and the polygon ID in the higher bits of the interpolated color (zeroing the lower bits). Then, when ID rendering is performed, color blending between polygon color and texture color can be used to render polygons with colors that contain both the polygon and patch ID Monte Carlo Modifications Using mipmaps reduces aliasing by ensuring that patches are not skipped when sampled with a hemicube of lower resolution. Rather than skipping high resolution patches, coarser resolution patches are used. By default, this results in approximately a one-to-one mapping of hemicube pixel and

4 4 Lum, Ma and Max / Interactive Radiosity Using Mipmapped Texture Hardware patches; in other words, most patch appear as a single element on the hemicube. It is desirable to have patches map to several pixels on the hemicube for more accurate form factor calculations. This can be accomplished by adjusting the level-of-detail(lod) bias 10 of a texture, which has the effect of adding an offset to which mipmap level is used. Some streaking artifacts can occur, however, because neighboring patches can be mapped to a different number of hemicube pixels. For example, if the level-of-detail is biased so that patches on average map to approximately four hemicube pixels, rows of dark patches that map to three pixels and receive less energy can occur. We apply Monte Carlo techniques to reduce such artifacts. Jittering the shooting position and using more finely subdivided shooting patches stratifies the hemicube position, permitting the hemicube resolutions to be kept low while keeping aliasing to a minimum. It also improves the rendering of shadow penumbras. If a lower resolution hemicube, with finer shooting patches is used, the size of the sampling kernel should be reduced as described in the next subsection; otherwise the large kernel from a low resolution hemicube can yield overly soft shadows. To reduce artifacts that can occur from the regularly spaced hemicube sampling of regularly spaced patches, we jitter the size and rotation of the hemicube as well Sample Kernel shape By varying the level-of-detail bias of the mipmap ID textures, and applying image processing operations to the patch hierarchies, the size and shape of the hemicube sampling kernel is varied. With the default LOD bias of zero, each patch is mapped to approximately one texel on the hemicube. By reducing the LOD bias, coarser mipmap levels are used, widening the hemicube sampling kernel. In this case, however, the value of the shooting kernel itself is constant. A second way to widen the kernel and give it a non-uniform shape, is to filter the unpushed radiosity prior to the pushpull operations. Convolution of unpushed radiosity hierarchy with a kernel is equivalent to using that shape kernel in the receiving stage. For the scenes rendered in the results section, we used a LOD bias of one, with filtering by a 3 by 3 Gaussian filter. Using a wider kernel reduces aliasing and accelerates computation, while a narrower kernel can yield sharper shadows, but requires higher resolution sampling to avoid aliasing. 5. Results We evaluated our technique using a low-cost PC with an AMD 1.6 Ghz Athlon XP processor, 512 megabyte of memory, and Nvidia GeForce 3 Ti 500 graphics card. Hemicube faces were rendered one at a time to different parts of the frame buffer and read back in a single operation. The radiosity solution seen in Figure 1 was generated in 0.75 seconds with an additional 0.2 seconds required for push-pull and color texture generation for the image to be displayed. The scene consists of 402 polygons and 13,742 patches. Notice the well defined shadows under the chairs, as well as the green and red reflected lighting under the table. The scene in Figure 2 contains 3138 polygons, and 27,382 patches. It was rendered in 4.4 seconds, with 4.0 seconds required for calculating the radiosity solution and 0.4 seconds used for the push-pull operation and display. Color bleeding is visible on the ceiling, while the closer chairs have a bluish hue from wall light reflections. Also notice the subtle shadows under the table. Since our technique uses shooting, the radiosity solution can be viewed progressively for better interactivty. This can be seen in the accompanying video, which shows lights, objects, and viewpoint being changed at multiple frame per second. 6. Conclusion We have presented a method that facilitates the interactive calculation of radiosity solutions, permitting arbitrary changes to a scene. We introduce the use of mipmapping as a hardware accelerated oracle for calculating form factors of multi-resolution patch hierarchies. Our technique can be used for calculating radiosity solutions with extremely refined patch hierarchies since it lacks the overhead of polygon-based representations, and has antialiasing in the form of mipmaps. 7. Future Work Currently, we save reflected radiosity in I, which is not very efficient since none of our polygons have reflectivity textures. We could easily handle reflectivity and emissivity textures within our framework, and save incident irradiance during the shooting process, as in 9. We would then need to look up the reflectivity texture only when the leaves were reached in the delayed push-pull process. The CPU spends some idle time while the graphics pipeline generates and reads back the ID images. If we added multithreading, we could be using this time to process the information from the previous ID image. We are using OpenGL immediate mode to repeatedly transmit our models to the graphics pipeline. Since the geometry does not change during multiple hemicube renderings, we might be able gain some speed using vertex arrays for the geometry, polygon ID colors, and ID texture coordinates, or display lists which stay on the graphics card. Radiosity accuracy and appearance can often be improved by a final gathering step at each vertex, pixel, or texel, but this can be very slow, as in 2. Our hardware texture mapping approach can speed up gathering as well as shooting,

5 Lum, Ma and Max / Interactive Radiosity Using Mipmapped Texture Hardware 5 by computing the form factors for each gather using the mipmapped texture ID s, and gathering the radiosity from the patches at the appropriate resolution levels. Alternatively, the patch radiosities could be recorded in a color texture map of the sort used in the final rendering of our current interactive system, and then gathered as an incoming irradiance by directly imaging these colors, as in 16. To handle the dynamic range needed to include the bright light sources, we could either do a separate pass for the light sources, as in 16, or use the high dynamic range textures proposed by Cohen et al. 3. Figure 1: The radiosity solution shown was calculated in 0.95 seconds using 13,742 patches. 2. CHEN, S. E., RUSHMEIER, H. E., MILLER, G., AND TURNER, D. A progressive multi-pass method for global illumination. In Computer Graphics (Proceedings of SIGGRAPH 91) (Las Vegas, Nevada, July 1991), vol. 25, pp ISBN X. 3. COHEN, J., TCHOU, C., HAWKINS, T., AND DE- BEVEC, P. Real-time high-dynamic range texture mapping. In Rendering Techniques 2001: 12th Eurographics Workshop on Rendering (June 2001), Eurographics, pp ISBN COHEN, M., CHEN, S.E.,WALLACE, J.R., AND GREENBERG, D. P. A progressive refinement approach to fast radiosity image generation. In SIGGRAPH 88 Conference Proceedings (August 1988), pp COHEN, M., AND GREENBERG, D. P. The hemi-cube. In SIGGRAPH 85 Conference Proceedings (August 1985), pp COHEN, M.F.,AND WALLACE, J.R. Radiosity and Realistic Image Synthesis. Morgan Kaufmann, DRETTAKIS, G., AND SILLION, F. X. Interactive update of global illumination using a line-space hierarchy. In Proceedings of SIGGRAPH 97 (Los Angeles, California, August 1997), Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH / Addison Wesley, pp ISBN FERNANDO, R., FERNANDEZ,S.AND BALA, K., AND GREENBERG, D. P. Adaptive shadow maps. In SIG- GRAPH 01 Conference Proceedings (August 2001), pp GERSHBEIN, R., SCHRÖDER, P., AND HANRAHAN, P. Textures and radiosity: Controlling emission and reflection with texture maps. In SIGGRAPH 94 Conference Proceedings (August 1994), pp GL_Ext_texture_lod_bias, OpenGL extension registry. /texture_lod_bias.txt. Figure 2: This radiosity solution shown was calculated in 4.4 seconds using 27,382 patches. References 1. CHEN, S. E. Incremental radiosity: An extension of progressive radiosity to an interactive image synthesis system. In Computer Graphics (Proceedings of SIGGRAPH 90) (Dallas, Texas, August 1990), vol. 24, pp ISBN HANRAHAN,P.,SALZMAN, D., AND AUPPERELE,L. A rapid hierachical radiosity algorithm. In SIGGRAPH 91 Conference Proceedings (August 1992), pp HECKBERT, P. S. Adaptive radiosity textures for bidirectional ray tracing. In Computer Graphics (Proceedings of SIGGRAPH 90) (Dallas, Texas, August 1990), vol. 24, pp ISBN KAJIYA, J. T. The rendering equation. In SIGGRAPH 86 Conference Proceedings (August 1986), pp KELLER, A. Instant radiosity. In SIGGRAPH 97 Conference Proceedings (August 1997), pp

6 6 Lum, Ma and Max / Interactive Radiosity Using Mipmapped Texture Hardware 15. MAX, N. Unified sun and sky illumination for shadows under trees. In CGVIP: Graphical Models and Image Processing (1991), vol. 53, pp MAX, N.L.,AND ALLISON, M.J. Linear radiosity approximation using vertex-to-vertex form factors. Addison Wesley, 1992, ch. VI-9, pp MYSZKOWSKI, K., AND KUNII, T. L. Texture mapping as an alternative for meshing during walkthrough animation. In Fifth Eurographics Workshop on Rendering (1994), pp SCÖFFEL, F. Online radiosity in interactive virtual reality applications. In ACM symposium on Virtual reality software and technology (1997), pp SILLION, F. X., AND DRETTAKIS, G. Feature-based control of visibility error: A multi-resolution clustering algorithm for global illumination. In Computer Graphics (Proceedings of SIGGRAPH 95) (1995), pp SILLION,F.X.,AND PUECH,C.Radiosity and Global Illumination. Morgan Kaufmann, SMITS, B., ARVO, J., AND GREENBERG, D. A clustering algorithm for radiosity in complex environments. In Computer Graphics (Proceedings of SIGGRAPH 94) (1994), pp SOLER, C., AND SILLION, F. X. Fast calculation of soft shadow textures using convolution. In Computer Graphics (Proceedings of SIGGRAPH 98) (1998), pp SOLER, C., AND SILLION, F. X. Texture-based visibility for efficient lighting simulation. In ACM Transactions on Graphics (2000), vol. 19, pp WALLACE, J. R., ELMQUIST, K.A., AND HAINES, E. A. A ray tracing algorithm for progressive radiosity. In Computer Graphics (Proceedings of SIGGRAPH 89) (Boston, Massachusetts, July 1989), vol. 23, pp