Clustered Importance Sampling for Fast Reflectance Rendering

Transcription

1 LiU-ITN-TEK-A--08/082--SE Clustered Importance Sampling for Fast Reflectance Rendering Oskar Åkerlund Department of Science and Technology Linköping University SE Norrköping, Sweden Institutionen för teknik och naturvetenskap Linköpings Universitet Norrköping

2 LiU-ITN-TEK-A--08/082--SE Clustered Importance Sampling for Fast Reflectance Rendering Examensarbete utfört i medieteknik vid Tekniska Högskolan vid Linköpings universitet Oskar Åkerlund Handledare Rui Wang Examinator Anders Ynnerman Norrköping

3 Upphovsrätt Detta dokument hålls tillgängligt på Internet eller dess framtida ersättare under en längre tid från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: Oskar Åkerlund

4 Abstract In realistic image synthesis, a major challenge is how to account for largescale, complex lighting environments in an efficient and robust way. This thesis presents a simple and efficient method for bidirectional importance sampling from large-scale illumination sources. The method is based on a clustered representation of the potential light source samples. Combined with an efficient BRDF sampling strategy, an importance function can rapidly be constructed and used to select the lighting samples that will contribute the most in the rendering equation. Most existing methods similar to the one presented in this thesis are restricted to environment lighting only; in contrast, the proposed algorithm can also be used in local lighting scenarios where the illumination exists on an unstructured point set, which may not have the same natural parameterization as an environment map.

5 Acknowledgements I would like to give big thanks my advisor Rui Wang at the UMass Computer Graphics Research Group for all of the helpful discussions and hints that helped build this thesis. Thanks to my academic supervisor Anders Ynnerman and my opponent Per Lönroth for their feedback on the report. Finally a special thanks to my fiancée, Amy, for all of her love and support.

6 Abbreviations BVH Bounding Volume Hierarchy BRDF Bidirectional Reflectance Distribution Function CPU Central Processing Unit GLSL OpenGL Shading Language GPU Graphics Processing Unit GUI Graphical User Interface HDR High Dynamic Range RGB Red-Green-Blue SIMD Single Instruction, Multiple Data SISD Single Instruction, Single Data UMass University of Massachusetts

7 Contents List of Figures List of Tables iii iv 1 Introduction Problem Description Thesis Objectives Outline of Report Reader Prerequisites Background Realistic Image Synthesis The Rendering Equation Global Illumination Environment Mapping The BRDF Offline Rendering Monte Carlo Methods Real-time Rendering Precomputed Radiance Transfer Deferred Shading Methods and Models Bidirectional Importance Sampling Importance of Luminance Importance of the BRDF Joint Importance BRDF Models Lambertian Phong Ward Anisotropic Lightcuts Light Sources Distant Light i

8 CONTENTS ii Local Light Implementation Application Environment The Importance Function Generating the Light Cut BRDF Importance Sampling Pseudo Random Numbers Rendering Pipeline User Interaction Data Flow Final Sampling Visualization of Sample Distribution Parallelism Multithreading Streaming SIMD Extensions Results Test Environment Ground Truth Test Scenes Performance Distribution of Samples Discussion Conclusion Noise Related Work Future Work Bibliography 32

9 List of Figures 2.1 Coordinate frame used in the rendering equation The diffuse and specular components of the BRDF Shadow rays Coordinate frames for BRDF importance sampling Example of a cut through a tree with aggregated values Importance sampled scenes with ground truth images Comparison Hebe Comparison Car Comparison Box Comparison Plate Visualization of sample distribution iii

10 List of Tables 5.1 Rendering times using 128 samples per pixel iv

11 Chapter 1 Introduction This chapter introduces the reader to the thesis, by presenting the problem description along with the main objectives. The outline of the report and the recommended reader prerequisites are given as well. 1.1 Problem Description Realistic image synthesis requires simulating illumination effects from complex, large-scale area light sources. Examples include direct lighting from detailed High Dynamic Range (HDR) environment maps [1], and indirect lighting from complex scene models where all surface points contribute to the final global illumination results. While the theory behind illumination has been well understood, improving the efficiency in simulation algorithms continues to present a major challenge. As described by the rendering equation in [2], the illumination computation involves estimating a hemispherical integral of the lighting, visibility, and the Bidirectional Reflectance Distribution Function (BRDF) of the surface. This integral has in general no analytic solution; even with approximate methods, the solution is usually too expensive to compute at interactive rates. Numerical simulation of the rendering equation typically uses Monte Carlo sampling methods [3]. The efficiency of these methods can be dramatically improved by sampling according to an importance function. Several approaches have studied importance sampling schemes that are computed purely based on the lighting [4, 5]. These methods perform poorly in the case of shiny BRDFs, as high-frequency details of the BRDF are ignored. Similarly, approaches that are purely based on the BRDF s importance present significant problems when the lighting contains high-frequency information. Recently, researchers have presented several efficient bidirectional importance sampling methods for sampling according to the product function of the lighting and the BRDF [6, 7]. These methods account for high-frequency details in both the lighting and the BRDF, and can thus significantly im- 1

12 Introduction 2 prove the rendering quality over single function sampling. Most existing bidirectional methods, however, assume that the illumination is coming from distant environment maps, which can be naturally parameterized as twodimensional image functions. This makes it possible to apply basis methods such as nonlinear wavelet approximation to rapidly construct the importance function. Unfortunately, these approaches cannot be easily extended to local lighting scenarios where the light sources exists on an unstructured set of points, which may not have a natural parametrization. The focus of this thesis is to construct a robust and efficient algorithm for bidirectional importance sampling that can be used for both direct lighting from an environment map as well as indirect lighting from the local scene. To make this possible, the lighting from both scenarios will be represented as a piecewise constant approximation, as in [12]. 1.2 Thesis Objectives Following the problem description in 1.1, the thesis objectives are: Present a method for bidirectional importance sampling that is based on a piecewise constant approximation of the lighting making it useful in both distant and local lighting settings. Implement the method using OpenGL 1 and C++ 2. Evaluate the results. Note that the main objective of the thesis is not the optimized implementation of the solution, but rather a proof of concept. 1.3 Outline of Report In chapter 2, background information will be given on common rendering algorithms and simulation models used for both real-time rendering and offline rendering of synthetic images. Chapter 3 describes in detail how some of these methods and models can be used to solve the thesis objectives. Chapter 4 presents how they are modified and actually used in the implementation of this thesis. In chapter 5, the results of the implementation are presented. In conclusion, an analysis and discussion of the results compared to the objectives, together with suggestions on future work is given in chapter 6. 1 Open Graphics Library a cross-language cross-platform API for computer graphics. 2 A general-purpose programming language.

13 Introduction Reader Prerequisites In order to fully understand the contents of this thesis, basic knowledge and experience in computer graphics, linear algebra, data structures and statistics is recommended.

14 Chapter 2 Background This chapter introduces the reader to the common rendering algorithms and strategies used in the computer graphics industry and research. The first section will present the two ends of the strategy spectrum, being real-time rendering and offline rendering, together with a brief explanation of how light transfer and material properties can be modeled in both strategies. Finally, a more thorough explanation of the different strategies is given. 2.1 Realistic Image Synthesis In the pursuit of realism in computer graphics, the simulation of light plays a big part. In reality, everything our eyes observe is an active light source - even this thesis is casting some light towards you right now. Most objects do not produce light themselves; instead they let light bounce on them, coming from the sun, a lamp or any other light emitting source. The amount of light rays that enter our eyes is vast and infeasible to completely reproduce with a computer. Instead, there is always a level of approximation and sampling required in image synthesis. Ideally we want to produce photo realistic images in real-time rates where all factors (light sources, geometry and materials) are dynamic. However, one or more of these factors needs to be static and even precomputed to allow for the highest frame rates. The other end of the spectrum is offline rendering, where the image quality is more important than the rendering speed. In this case, many samples are used and the level of approximation is low. The big challenge in computer graphics research of today is to land somewhere in between of these extremes to produce realistic images while maintaining acceptable, interactive frame rates. 4

15 Background The Rendering Equation The rendering equation, introduced in [2], describes the amount of light L o that is exited from a surface point x along a certain direction w o. The intensity of the outgoing light is based on incoming light L i, the BRDF f r of the point and a geometric form factor G: L o (x, w o ) = L e (x, w o ) + f r (x, w i, w o )L i (x, w i )G(x, w i )dw i. (2.1) Ω In the above equation, L e describes the self-emitted light of point x in direction w o. This term is rarely considered 1, and will be ignored in this thesis. The form factor G is usually defined as: G(x, w i ) = (N w i)(l N w i ) L x x 2. (2.2) Here, N is the normal at point x. L N and L x is the normal and position of the light source as illustrated in figure 2.1. One important feature of the equation is the conservation of energy meaning that the outgoing light can not have a higher intensity than the sum of the incoming light and the self-emitting light. This is controlled by normalizing the BRDF and the form factor. N w h wo w r L N Lx w i x TN S N Figure 2.1: Coordinate frame used in the rendering equation, in which N, S N and T N forms an orthogonal basis. The half-angle vector w h is halfway between the view vector w o and the light vector w i. The vector w r is w o reflected across N. 1 Since most real world materials do not emit any light. It is, however, useful when rendering light sources as objects, such as a light bulb.

16 Background Global Illumination There are usually two ways that light can hit an object in the scene we are rendering. In direct illumination, light is coming directly from a selfemitting light source such as the sun or a lamp. The same light source can reach us indirectly by first bouncing on other objects in the scene, resulting in indirect illumination. In this case, the objects of the scene are themselves treated as light sources. Combining direct illumination with indirect illumination is essential for full realism and is called global illumination Environment Mapping One common method of casting direct lighting onto the scene is called Environment Mapping. Using a technique described in [1], HDR light can be captured in a real world location and mapped onto a sphere or a cube that is considered to be infinitely far away and centered around the objects we want to render. When gathering the direct lighting, the sphere (or cube) can either be sampled directly for best accuracy, or arranged in an approximating structure or projection basis for increased efficiency The BRDF The BRDF is the four-dimensional function f r in the rendering equation, that describes how light is reflected at a point on an opaque surface. The function takes two vectors as input: the view vector w o, normalized from the point to the viewer, and the light vector w i, normalized from the point to wherever we want to sample in the hemisphere. Together with the surface properties of the point, the resulting (colored) light exited along w o is computed. The surface properties of the point typically includes the normal, the material color and specularity constants. Most BRDF models include two types of reflections: Diffuse reflection, smooth and independent of the view vector. Specular reflection, dependent on the view vector. Yields specular highlights. These components are illustrated in figure 2.2. In order to conserve energy (an important feature of the rendering equation), the hemispherical integral of the BRDF multiplied with the form factor must be less than or equal to 1. Since the hemispherical integral of the form factor is π, this is usually done by specifying surface colors in the range [0,1] per channel and using a division by π in the BRDF model.

17 Background 7 Diffuse Specular Diffuse + Specular Figure 2.2: The diffuse and specular components of the BRDF. 2.2 Offline Rendering When performance is of second priority, the rendering equation can be evaluated more carefully and precise to yield a resulting image of higher quality. One common method for this is called ray tracing, where rays are shot from the view point through each pixel of the image plane, into the scene. If an object in the scene is blocking the path of the ray, new rays are spawned at the closest intersection point to the eye. Reflection and refraction rays will recursively continue to test intersections of the scene while shadow rays will travel from the intersection point to each light source as shown in figure 2.3. If the shadow rays are blocked by opaque objects, their correspondent light sources are not visible to the point, causing it to be shadowed. As described in 2.1, light is usually coming from virtually every direction in the hemisphere, so a few light sources will not be enough to produce realistic looking images. To account for all frequencies in the lighting environment, the more samples that are taken of the domain the better. However, increasing the number of light sources will also increase the computation time dramatically. Hence, we need a way of selecting where and how to sample the domain, keeping the resulting image quality high and the computational cost low. Light source View point Light source Blocking object Image plane Surface point Figure 2.3: Shadow rays are spawned at the closest surface point along the view vector, checking visibility of each light source.

18 Background Monte Carlo Methods A Monte Carlo method [11] is a numerical method where an equation (e.g. the rendering equation) is solved by repeated sampling with a random variable. When increasing the number of samples, the result will converge to a final value close or equal to the real function value. For a real-valued random variable X, we can define its cumulative distribution function as P (x) = P r{x x} = x 0 p(t) dt, (2.3) where p(t) is called the probability density function. If we want to evaluate the integral I = f(x) dx (2.4) Ω numerically, we can independently sample N points X 1... X N and compute the estimate of I as: Î N = 1 N f(x i ) N p(x i ). (2.5) i=1 This estimate is unbiased; the expected value is equal to the original integral: E[ÎN] = 1 N N i=1 Ω f(x) p(x) Ω p(x) dx = f(x) dx = I. (2.6) What makes a difference is computer graphics is how fast the estimation converges to the correct result. Studying the variance of ÎN we see that it decreases linearly with N: V [ÎN] = 1 N 2 N V i=1 [ ] f(xi ) p(x i ) = N [ ] f(x) N 2 V p(x) = 1 [ ] f(x) N V. (2.7) p(x) Hence, using more samples will lower the variance and reduce noise in the final result. However, an increased number of samples will also increase the computational time. Instead, the variance can be lowered by carefully constructing the probability density function p. In 3.1, it is shown how to use a bidirectional importance function to construct p. 2.3 Real-time Rendering When performance is of highest priority, the accuracy of the physical light transport plays a small part. Nevertheless, we still want good looking images. Precomputation solves the problem if some factors of the rendering equation are allowed to be static. However, in most real-time rendering applications, an ability to interact with the scene is desired. Consequently, some dynamic factors need to be evaluated on the fly.

19 Background Precomputed Radiance Transfer Early computer games such as Quake by id Software used a technique called Lightmaps to simulate global illumination. This is done by utilizing an offline renderer to precompute the light transfer in a scene and storing the result as texture data. These textures can later be applied on the geometry in a real-time rendering system. However, this is mainly useful for diffuse reflections - parts of the rendering equation where the view point is not included. Instead, storing the transfer information rather than the final result can be done to allow for dynamic lighting environments in real-time. In [8], a novel method for precomputed radiance transfer is proposed that precomputes the visibility for each vertex in the geometry while allowing fully dynamic BRDFs, lighting environment and viewpoint Deferred Shading In [13], Deering et al. introduces the idea of Deferred Shading, which is a useful method when rendering opaque objects. Using multiple screen-sized rendering targets, a first rendering pass can store input data of the per pixel lighting equation (e.g. the per-pixel position, normal and material properties) in auxiliary buffers. In the final rendering pass, these buffers are used to produce the pixel colors. The main advantage with deferred shading is that the final, most expensive pass will only deal with pixels that are seen by the viewer and have passed the Z-buffer test. While getting increasingly popular in real-time applications, this method can also be useful as a preprocessing step in offline rendering. For example, in a ray tracer, this could replace the first set of rays that determines the closest hit point per pixel.

20 Chapter 3 Methods and Models This chapter will present methods that are used to fulfill the objectives of this thesis. First, an extension of the Monte Carlo method will be explained. Following sections will present ways of representing light sources and material properties. 3.1 Bidirectional Importance Sampling When using uniformly distributed samples in Monte Carlo ray tracing (see 2.2.1), convergence is slow; an acceptable, relatively noise free result requires many samples. The probability density function, p (introduced in eq. 2.3) is constant in this situation, meaning that each light source has equal probability of being sampled. This is basically equivalent to a blind search without heuristics. To help solve this problem, we can design p based on what we know about the lighting environment and the surface BRDF of the point we are evaluating. For each potential sample x, an importance metric, Γ(x) is computed. Then, p(x) is computed as the normalized Γ(x): p(x) = Γ(x) Ni=1 Γ(i) (3.1) Several approaches have constructed Γ(x) purely based on the luminance of the light samples [4, 5]. This unfortunately ignores high-frequency details of glossy BRDFs. Similarly, approaches that are purely based on the BRDF s importance present significant problems when the lighting contains high-frequency information, such as very bright spotlights. The following subsections will describe how to compute Γ based on the light source luminance and the BRDF, and finally how to combine them. 10

21 Methods and Models Importance of Luminance For a given light source x with a color defined in the Red-Green-Blue (RGB) color space, the luminance can be computed as R G B. This follows the luminosity function, giving the green component more importance since humans experience green light as brighter than blue or red light [14]. Using the luminance as an importance metric will give intensive lights more importance. This makes sense because very bright light sources are in reality more likely to contribute than darker light sources. Moreover, relatively intensive lights in an environment are usually isolated and small, making it easy to miss those areas when no light based importance sampling is used Importance of the BRDF For a shiny surface, it is wasteful to send samples in random directions or in directions purely determined by the luminance of the light sources, since most energy will be in rays concentrated in and close to the specular lobe of the BRDF. It is possible to construct an importance sampling algorithm for most BRDF models [15], that can warp uniform random variables to light vectors which follows the distribution of the BRDF (specific examples of this can be found in 3.2). These sample vectors can then be used to predict which potential samples that will or will not contribute to the final result, resulting in an importance metric for the BRDF Joint Importance Joining the two importance metrics described in and into a single, joint bidirectional importance function is done by multiplying them together. It is often useful to incorporate the form factor G as well, since it will clamp the sample domain to the hemisphere and better follow the intensity distribution of the rendering equation over the potential samples. Ideally, we would also like to use the visibility factor in the joint importance function. However, this would require too much work since visibility testing is usually the most expensive part of the rendering equation. Using approximated or precomputed visibility is not very helpful, since the effect of importance sampling would be hard to demonstrate, as visibility approximation errors are easily spotted. As the final importance metric is constructed, we can compute the probability density function p as in eq. 3.1 and the cumulative distribution function P as in eq Using a random number generator ξ with an uniform distribution over (0, 1), we can send final samples according to: X = P 1 (ξ). (3.2)

22 Methods and Models 12 These samples are then used in the rendering equation to produce the pixel colors. 3.2 BRDF Models There are many proposed BRDF models with various levels of physical correctness and efficiency. The notation for the BRDF is f r (w i, w o ), where w o is the view vector and w i is the light sample vector. The normal is denoted by N. In the importance sampling scheme, two random numbers ξ 1 and ξ 2 that are uniformly distributed over (0, 1) will be warped into spherical coordinates: elevation angle θ and azimuth angle φ. The light vector can then be derived from these coordinates and used for sampling. In the following subsections, some of the most commonly used models in modern computer graphics applications will be described, together with their respective warping functions for importance sampling. Note that even though the Phong and Ward Anisotropic BRDF model have a diffuse component, their importance is concentrated in the specular lobe. As explained in [16], we can account for the diffuse component by sending samples according to the Lambertian importance in the ratio k d / k s where k d and k s are the diffuse and specular surface colors, respectively. The coordinate frames used for importance sampling of the following models are shown in figure Lambertian The Lambertian BRDF model is used for rough surfaces with no specular highlights. Many real world materials are rough rather than shiny, so the model is often useful. The reflectance is constant fully independent on incoming and outgoing vectors: f r (w i, w o ) = k d π. (3.3) Here, k d is the surface color. The division by π is explained in The importance of this model is also constant over the hemisphere. We can get the sample vector w i through these spherical coordinates with respect to N: (θ i, φ i ) = (arccos( ) ξ 1 ), 2πξ 2 (3.4) Phong The Phong BRDF is an extension of the Lambertian BRDF. Here, specularity is included, making this model view-dependent. The model is not

23 Methods and Models 13 physically accurate but can still give convincing visual results and is relatively fast to evaluate. The BRDF is defined as: f r (w i, w o ) = k d π + (w r w i ) α k s π. (3.5) Here, w r is the view vector w o reflected across N as shown in figure 2.1. The diffuse and specular color is given by k d and k s, respectively. The shininess parameter α controls the size of the specular highlight. The Phong importance sampling scheme, introduced in [16], yields the sample vector w i through these spherical coordinates with respect to w r : (θ i, φ i ) = (arccos( α+1 ) ξ 1 ), 2πξ 2 (3.6) Ward Anisotropic Anisotropic surfaces have a visible direction of shininess. While specular highlights generated by the Phong model tend to be round in shape, the Ward Anisotropic model can generate stretched out highlights in dominant directions, controlled by the anisotropy parameters α S and α T. The BRDF is defined as: f r (w i, w o ) = k d π + k s 4πα S α T (N wo )(N w i ) exp 2 ( w h S α S ) 2 + ( w h T α T ) H N (3.7) Here, the diffuse and specular colors are given by k d and k s, respectively. The vectors S and T form a basis with N, while α S and α T are their respective anisotropy parameters. The half-angle vector w h is depicted in figure 2.1 and is defined as: w h = (w i + w o ) w i + w o. (3.8) The importance sampling scheme for this model is explained in [17], yielding the half-angle vector (from which the sample vector w i can be derived) through these spherical coordinates, with respect to N: ( ) αt φ h = arctan tan(2πξ 1 ) (3.9) α S θ h = arctan ( cos φh α S log ξ 2 ) 2 ( ) + sin φh 2 (3.10) α T

24 Methods and Models 14 N w r N θi w i θi w i θh w h φ i TN φ i Tr φ h TN S N S r S N Figure 3.1: Coordinate frames for BRDF importance sampling of the Lambertian, Phong and Ward Anisotropic models. 3.3 Lightcuts In [12], Walter et al. presents a method for constructing a piecewise constant approximation of the lighting, called lightcuts. The method starts by distributing an initial set of 2 N sample points uniformly over the domain (e.g. the sphere for spherical environment lighting or the scene geometry for local lighting with indirect illumination) in a pre-processing stage. These unstructured sample points can be used in the rendering equation as point lights, as suggested in [8]. A global light tree is then constructed, which hierarchically partitions the lighting samples. This tree is a complete binary tree where the leaf values are the values of the light samples and the internal nodes are aggregated clusters. Useful data to store in this structure is the light source color, position and normal. A cut through the light tree is defined as a set of nodes such that every path from the root to a leaf will contain exactly one node of the cut, as shown in figure 3.2. In [9], the lightcut is selected based on the aggregated luminance of the internal nodes. This will make the cut dig deeper into the tree where the light samples are very bright, similar to importance sampling based on luminance (3.1.1). Up to 2 15 light samples are used to construct the light tree and its cut dynamically in real-time, with a fixed cut size of 1024 nodes. 3.4 Light Sources In the following, the difference between distant and local sources is explained, together with a description on how a light tree can be built upon each of them.

25 Methods and Models Figure 3.2: Example of a cut through a tree with aggregated values Distant Light As explained in 2.1.3, it is possible to capture the environment lighting in a real-world location and map it onto a shape. Uniformly spread samples on the unit sphere can then be used to build a light tree. This will model the light as distant 1 point lights, making the position of a sample directly usable as the light vector of the BRDF. A dynamic environment map can efficiently be re-sampled onto the leaves, using the leaf positions as sample directions. The internal nodes consists of a sum of its children s luminance, and their normalized averaged position Local Light Local light sources are close to (or within) the scene and thus have a different direction to each point of the geometry. This difference is taken care of using the form factor, which is a part of the rendering equation described in This means that we need to aggregate the indirect light normals, along with the positions and luminance, into the tree. First, sample points are uniformly distributed over the geometry of the scene, similar to environment lighting where the samples are spread onto the unit sphere. Then, each sample point will receive direct illumination of a local light source, such as a spotlight. This can be compared to the environment map sampling in The internal nodes will consist of a sum of their children s luminance, the average position and their normalized average normal. The tree is then used for rendering with one bounce of indirect illumination. The local, direct illumination light source can be modeled in different ways. One common model is the point light, which is omnidirectional it shines of equal intensity in all directions around it. This results in a relatively low frequency indirect illumination function, with no significant peaks in the importance function for luminance. To better show the effect of importance sampling, a directed light model can be used, such as the spotlight which 1 A distant light is imagined to be shining from infinitely far away, having the same direction vector to all points of the geometry.

26 Methods and Models 16 shines in a specified direction. The spotlight defines a cone-shaped volume with the cone tip located at the light source position. The volume boundary has a quadratic falloff, leaving a realistic looking penumbra 2 where it hits the scene geometry. Another more dynamic, directed light source is the projective light source, which like a real-world projector projects an image onto the scene geometry. In this case, a HDR image is naturally preferred since it can make the importance function high frequent. 2 Latin for almost shadow.

27 Chapter 4 Implementation In this chapter, the methods and models described in chapter 3 will be further discussed and modified to work in a combined, practical way. The complete implementation is presented, from the user interface to the rendered image. Finally, some techniques of increasing the run-time efficiency are described. It is highly recommended for the reader to fully understand the concepts of the previous chapters before continuing. 4.1 Application Environment Recent work published by the Computer Graphics Research Group at the University of Massachusetts (UMass) [8, 9] uses an unoptimized ray tracer for precomputation and a separate application for the final real-time rendering, programmed for the Graphics Processing Unit (GPU). When implementing this thesis, most work was put into rewriting the ray tracer to suit the objectives of the thesis. Some work was also done to connect the real-time application to the ray tracer, making it useful as a Graphical User Interface (GUI). Both applications are written in C++. The real-time application uses OpenGL and OpenGL Shading Language (GLSL) to display and handle graphics. The development environment was Microsoft Visual Studio Using the Intel Compiler 9, some Intel-specific intrinsic functions were used for speedup [10]. 4.2 The Importance Function This section describes the steps on how the final importance function Γ (described in 3.1) is created and used. The first step is to create the light cut, which will act as a guide for the importance sampling. The importance function is then defined on the cut, as the multiplication of the node luminance with the BRDF importance metric, described in 4.2.2, and the form factor, introduced in

28 Implementation 18 Note that the node luminance is aggregated from the leaves belonging to each node. This turns out to give an unbiased result, since large clusters should be given more samples if the average lighting per cluster would be constant. The following subsections will further describe the stages involved Generating the Light Cut As described in 3.3, generating the light cut based on the luminance of the nodes is similar to luminance based importance sampling since high intensity light sources will be given more detail. However, when using a real importance function, this similarity becomes redundant. Instead, the light cut generation is rewritten to be error based, much like the visibility cut generation in [8]. The initial light cut consists only of the root node. Then, the cut node that holds the highest cluster variance will be replaced by its two children iteratively until the desired cut size is present. Consequently, low frequency areas will not be subdivided as much as high frequency areas, making the cut suitable for all frequency lighting environments BRDF Importance Sampling Since the light cut represents a clustering of the potential light samples, the BRDF could be directly evaluated per cut node to yield a BRDF importance metric. However, this is not safe since the average cluster direction might be below the horizon, even though some samples are above it. Secondly, it is not an efficient strategy, especially for large cut sizes. Instead, a small number of samples can be sent using importance sampling of the BRDF. The BRDF importance sampling schemes described in 3.2 gives a sample vector that follows the distribution of the current BRDF. In order to know which cut nodes this vector hits, a Bounding Volume Hierarchy (BVH) is created in which each cut node holds an axis aligned bounding box of its cluster samples. An efficient ray-box intersection algorithm [19] is then used to test the sample vector against the cut boxes, which increases individual counters for each hit. After a total number of BRDF samples N s is sent, the final number of hit points Ns k per node k, is used to build the BRDF importance metric. As described in 3.2, it is common to generate sample vectors to cover both the diffuse and specular components of the BRDF. However, covering the diffuse part requires many more samples since the importance is constant over the whole hemisphere. Instead, each cut node is checked for visibility over the horizon, using the dot product between N and the normalized direction from the surface point to each corner of the cut node bounding box. As soon as one corner point is verified to be above the horizon, we

29 Implementation 19 know that the cluster will contribute to the diffuse part of the BRDF. Using this strategy, the diffuse part of the BRDF importance sampling can be removed. Then, just a few samples 1 need to be generated to cover for the specular importance. This is because the specular sample vectors will have similar directions, depending on the shininess factor of the BRDF. For the Lambertian BRDF, which has no specular component, no samples are sent the box visibility checking is sufficient. In order to construct an importance metric, we need to estimate the BRDF over the cluster using the information available: the number of specular hit points Ns k and the luminance of the specular and diffuse surface colors k s and k d. In the following, f s represents the specular component of the BRDF (excluding k s ). First, the average f s in a cluster k is given by: f sk = 1 f s, (4.1) Ω k Ω k where Ω k is the solid angle subtended by the cluster, given by: Ω k = A k cos θ i r 2. (4.2) Here, A k is the cluster area, θ i is the angle between L N and w i (zero in distant environment lighting) and r is the distance from L x to the surface point x. The cluster integral of f s can be evaluated as N k s times the average area per sample. This is because each importance based sample represents the same area under the function graph samples are naturally spread more densely where the function value is high. Ω k f s = N k s 1 N s Ω f s = N k s N s (4.3) As in the above equation, the average area per sample is given by the full hemispherical integral 2 of f s divided by N s, the total number of BRDF samples sent in the hemisphere. Combining equations 4.1, 4.2 and 4.3, we get: r 2 Ns k f sk =. (4.4) A k cos θ i N s The estimated BRDF in the cluster, which can be used as the BRDF importance metric, is then defined as: Γ fr = k s f sk + k d. (4.5) 1 32 specular samples are used in this thesis. 2 Since the specular part of the BRDF is normalized, this integral is 1.

30 Implementation Pseudo Random Numbers A random number generator is needed, as described in and 3.2. However, in most simulation applications a pseudo random number generator is used for efficiency. These generators produce a sequence of numbers that appear to be random but return periodically. This thesis uses the Mersanne Twister algorithm [18], since it is faster 3 and has a longer period than the C++ standard pseudo random number generator. 4.3 Rendering Pipeline This section describes the steps of the rendering pipeline, from the user interaction to the final rendered image User Interaction In the real-time rendering application, also used in [9], the user can control the viewpoint and lighting environment of the scene. Each object has editable material properties, including normal and specular mapping 4. The global light cut is generated per frame. In the GUI, the user can also select the number of samples to take in the importance sampler, which is called by the push of a button. For a smooth camera movement across multiple frames, an automatic camera positioning scheme is implemented so that the user does not need to re-adjust the camera between each frame. The camera moves along a parametric ellipse drawn on a sphere, while having a constant focus point in the scene. This can be used to create a short movie instead of single frames Data Flow As the importance sampler is called, several data sets are passed to the rewritten ray tracer application. First, a deferred shading buffer is output, containing the per-pixel position, normal and parameters to be used in the BRDF. Then the complete light tree is exported, along with the light cut and each node s bounding box (for use in the BRDF importance sampling). Using this data, the importance function Γ is built, resulting in an importance value for each cut node, per pixel 5. This is used to build the probability density function p and the cumulative distribution function P as described in Based on experiments. 4 A per-pixel rendering effect in which parameters of the BRDF is modulated by a function or an image texture. 5 Only pixels with material properties in the deferred shading buffer are processed.

31 Implementation Final Sampling In contrast to traditional Monte Carlo methods, the importance function is not defined directly on the sample domain but on the cluster nodes. As P is defined per node, it is piecewise constant over the samples, meaning that each sample point has an equal probability of being picked in its cluster. This is not a problem for a cut size of 512 and above, since high frequency light areas will have small sized clusters due to the nature of the light cut construction. For as many times as the user chose in the GUI, final samples are sent through the cut nodes to the leafs. First, using eq. 3.2, a cut node is chosen. The random number generator ξ is then reused to sample a leaf within this cluster. The leaf sample is tested for visibility using an un-optimized ray tracer where the scene geometry is represented in a BVH. If the shadow ray, going from the current position to the leaf light source, is blocked by any geometry, the leaf will not contribute to the pixel color and is simply discarded. If the ray is not blocked, the rendering equation will be evaluated, resulting in an output color for the current pixel Visualization of Sample Distribution In order to better understand how the samples are distributed when using bidirectional importance sampling in contrast to using single-factor based importance sampling, a visualization application was created. For a chosen pixel, the set of selected samples are output together with the importance function value for each potential light sample (constant within light cut clusters). The visualization application can then show the light samples at their positions, colored in gray scale with an intensity proportional to their importance value. Selected samples are colored in red to distinct them from the other samples. Examples of visualizations can be found in figure Parallelism This thesis is implemented on the Central Processing Unit (CPU), leaving the GPU idle most of the time. The GPU is only used in the pre-processing stages, i.e. for the GUI and the deferred shading. Many of the methods in the rendering pipeline are highly suitable for the GPU because of its high capacity of parallel floating point operations. However, some methods (i.e the ray tracer) are much harder to implement in current GPU frameworks than on the CPU side, making implementation work cumbersome and relatively inefficient. Even though the objectives of this thesis do not include an optimized implementation of the proposed method, some techniques were used to speed things up a bit.

32 Implementation Multithreading With multiple CPU cores and a separable task, an application can run two or more subtasks simultaneously using threads. In this thesis, each pixel is an independent job, making the use of threads suitable. The output image is divided into several parts, containing 32x32 pixels each. Each core of the CPU is then assigned a thread which picks a free block and starts processing it, pixel by pixel. This is done iteratively, until all parts of the image are rendered Streaming SIMD Extensions Most parts of the rendering equation involves vector math, since all positions, directions and normals are defined in the three-dimensional Cartesian coordinate system, and the colors are defined in the RGB color space. Vector math is common in many simulation applications, which is why there is hardware support for it. Singel Instruction, Multiple Data (SIMD) is an extension to a microprocessor which allows for parallel data operations. This thesis uses a C++ library which is designed for Intel s Streaming SIMD Extensions, SSE2 [10]. One example of the advantage of SIMD compared to Single Instruction, Single Data (SISD) is the elementwise multiplication of two three-dimensional vectors. In SISD, this would require three operations one for each multiplication but in SIMD the same result is given using one single operation. Data fetching and transfer also runs faster because of this, since the vector elements do not need to be extracted separately.

33 Chapter 5 Results This chapter presents the results obtained by implementing the chosen methods, as described in chapter 4. A set of test scenes are used to show the purpose of bidirectional importance sampling. The resulting images are shown in this chapter along with brief comments, which are extended into the next chapter. 5.1 Test Environment The rendering times presented in 5.4 are reported from a computer with the following relevant specifications: CPU: Intel Xeon Quad-Core 2.0 Ghz GPU: NVIDIA GeForce 8800 GTS Operating system: Microsoft Windows XP SP1 5.2 Ground Truth For visual comparison, a ground truth image can be generated. In this thesis, 2 15 light sources act as potential samples in the Monte Carlo rendering. The ground truth image is rendered using all these light sources exactly once instead of using a Monte Carlo method they are simply used in the rendering equation one after the other. Even though there is no overhead to pick the samples, this rendering takes a significantly longer time 1 than an importance sampling pass. Note that the ground truth renderer uses the same un-optimized ray tracer as the importance sampler, making it slower than it could be but useful for comparison. 1 Up to two hours for the test scenes used. 23

34 Results Test Scenes To show the advantages of bidirectional importance sampling compared to importance sampling purely based on luminance or BRDFs, and how it can be used with local lighting using the method proposed in this thesis, several test scenes have been constructed: Hebe: A scene for distant environmental lighting with the statue of Hebe on a floor. Both the statue and the floor have a shiny surface using the Phong BRDF. The HDR probe has a relatively low frequency lighting and is captured in the Pisa Courtyard, Pisa. Car: A scene for distant environmental lighting with a car on a floor. The car parts have the Phong BRDF while the floor has a Lambertian BRDF. The HDR probe used is captured in the Uffizi Gallery, Florence with a low frequency lighting. Box: A scene for local lighting with indirect illumination. The geometry consists of a sphere and a teapot inside of a chamfer box. A spotlight is casting direct light on the right wall, which shines indirect illumination on the rest of the scene. The box walls have a Lambertian BRDF. The teapot and the sphere use shiny Phong and Ward Anisotropic BRDFs, respectively. Plate: A scene for local lighting with indirect illumination. A shiny plate with a Ward Anisotropic BRDF is placed on the floor of a shiny box that uses the Phong BRDF. A projective light source shines a HDR image (captured at the Pisa Courtyard, Pisa) onto the back wall, which shines indirect illumination on the rest of the scene. The HDR image is intentionally flipped upside down to get the high intensity light of the sky near the floor. The ground truth images for these test scenes, along with the result of the bidirectional importance sampling, can be found in figure 5.1. Scene Rendering Times (seconds) Geometry Type # Faces # Pixels Joint Luminance BRDF Hebe Env 130k 104.5k Car Env 30k 146.2k Box Local 20k 307.2k Plate Local k Table 5.1: Rendering times using 128 samples per pixel.

35 Results Performance This section presents the performance of the implemented bidirectional importance sampler. In table 5.1, the rendering times for each scene are shown, using 128 samples per pixel and 32 specular BRDF samples for the bidirectional and BRDF based importance sampler. Using the same number of samples, the bidirectional importance function naturally takes longer time to construct than an importance function purely based on either luminance or the BRDF, since it requires more processing. For a fair comparison, the number of samples for the single-factor importance samplers are increased until the rendering times are the same as for the bidirectional importance sampler. However, even though the singlefactor importance samplers are given the same time and more samples, the bidirectional importance sampler outperforms them in visual quality of the rendered image. This is visible in figures 5.2 to 5.5, where the number of samples per pixel is given per rendered image. These figures show the result of using bidirectional importance sampling (Joint), luminance based importance sampling (Lum) and BRDF based importance sampling (BRDF). As shown in figure 5.2, with a fairly low frequency lighting environment, the Hebe scene works well for bidirectional and BRDF-based importance sampling but not for luminance based importance sampling. From figure 5.3 the same conclusion can be drawn for the car scene, although some uniform noise can be seen with the BRDF-based importance sampling. For the box scene shown in figure 5.4, the BRDF-based importance sampler performs poorly since the lighting is high frequent. The plate scene is a difficult case with both high frequent lighting and BRDFs. We can see in figure 5.5 that the bidirectional importance sampler performs well, while the BRDF-based importance sampler breaks down and the luminance-based importance samplers is catching up slowly Distribution of Samples As described in 4.3.4, a visualization application can for a selected pixel show how the selected samples were distributed. This gives a better understanding on how the distribution changes between the different approaches of importance sampling: bidirectional or single-factor based. A simple plane with the Ward Anisotropic BRDF was rendered with distant, direct environmental lighting from the HDR Probe captured in Grace Cathedral, San Francisco. The resulting distribution of samples for the center pixel are shown in 5.6, together with a visualization of the light cut bounding boxes.

36 26 Results (a) Hebe 128 samples (b) Hebe Reference (c) Car 128 samples (d) Car Reference (e) Box 128 samples (f) Box Reference (g) Plate 128 samples (h) Plate Reference Figure 5.1: Importance sampled scenes with ground truth images.

37 Results 27 (a) Joint 128 (b) BRDF 128 (c) Lum 128 (d) Lum 300 Figure 5.2: Comparison Hebe (a) Joint 128 (b) BRDF 128 (c) Lum 128 (d) Lum 280 Figure 5.3: Comparison Car (a) Joint 128 (b) BRDF 128 (c) Lum 128 (d) Lum 210 Figure 5.4: Comparison Box (a) Joint 128 (b) BRDF 128 (c) Lum 128 (d) Lum 320 Figure 5.5: Comparison Plate

38 Results 28 (a) Luminance only (b) BRDF only (c) Bidirectional (d) Bounding boxes Figure 5.6: Visualization of sample distribution. Selected samples are highlighted in red. In (d), the cut node bounding boxes are shown.