GPU Memory Management for Efficient Ray Tracing Applications

Transcription

1 Eurographics Symposium on Rendering 014 Wojciech Jarosz and Pieter Peers (Guest Editors) Volume 33 (014), Number 4 GPU Memory Management for Efficient Ray Tracing Applications Weilun Sun, Ling-Qi Yan, Yi Wu, John Kubiatowicz University of California, Berkeley Abstract In computer graphics, efficient ray tracing has always been the most important demand. However, since scenes could be rather complicated, ray tracing applications are mostly memory intensive, which limits efficient implementation on GPU only to small, experimental scenes. We seek for an efficient GPU memory management method that swaps data back and forth between GPU and CPU memory, and use the fact that ray tracing queries are order independent to achieve efficient GPU our of core rendering. Using our method, huge scenes can be rendered out-of-core on GPU, given maximum available video memory size limit. We tested convergence rate using our method with different video memory size limits to prove that our method introduces little overhead on memory swapping even with extremely low GPU memory size. At last, we demonstrated that our system may well converge faster than popular CPU ray tracer MITSUBA on large scenes if we had a better ray traversal implementation. Categories and Subject Descriptors (according to ACM CCS): Hardware Acceleration I.3.3 [Computer Graphics]: Image Generation 1. Introduction With the development of GPUs, it is more and more common for people to use them for general purpose computing rather than original rasterization and shading tasks. GPU is especially suitable to handle massive computations with low interdependency among input data. As one of the most fundamental topics in computer graphics, ray tracing is also starting to benefit from the great computational power of GPU. Ray tracing is a very complicated problem for the following reasons. First of all, many ray tracing algorithms use Monte Carlo sampling method to estimate integrals. Sparse sampling introduces noise in rendered images. Therefore, thousands of samples per pixel are often needed to generate smooth images, which is a huge computational demand. Second, scenes in ray tracing applications can be rather complex so that a big size of memory is often needed for high performance. GPU provides a good solution to the first problem because of its strong computational power. However, GPU suffers from the second problem since its memory size is often much smaller than CPU s. There are some ways to divide input data into independent blocks so that each block fits GPU memory to perform GPU computation. However, these methods are not applicable to ray tracing problems where input data are highly interdependent. During ray tracing, each ray that hits a diffuse object in a scene scatters to a random direction and may intersect with any part of the scene for the next bounce. In this paper, we propose a method to better utilize the power of GPU in ray tracing applications through eliminating its memory bottleneck. In the following content of the paper, we first introduce related works and preliminaries in understanding ray tracing problems. Then, we discuss our design and implementation of our system. After that, we show some rendering results using our method and make comparisons to show scalability of our method under various video memory limits. Finally, we discuss some issues and limitations, and propose possible extensions.. Related Works.1. Out-of-core Ray Tracing Fake out-of-core rendering There is a class of related work that aim to solve the out-of-core problem, but they either assume that the data needed are completely in-core, or make Computer Graphics Forum c 014 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd.

2 approximations i.e. doesn t guarantee correct rendering results. Fradin et al. [FMH05] enables out-of-core photon mapping by limiting a small size of geometry primitives to receive photons at a time, but still needs all the geometries in final gathering step. Wald et al. [War94, WPS 03, WDS04] proposed a page caching technique. They pre-fetch possibly related pages to memory. When a page fault occurs, instead of loading the needed page, they simply give the missing ray an approximated value using nearest neighbor. Teller et al. [TFFH94] applied out-of-core techniques to store intermediate results in radiosity method, which cannot be regarded as out-of-core. None of the above methods are GPU oriented. Distributed out-of-coure rendering Another class of related work tries to solve the out-of-core problem by using distributed storage techniques. Lefer [Lef93] stores a full copy of the data for each node in a cluster. Reinhard et al. [RCJ99a,RCJ99b] and Demarle et al. [DGP04] store different parts of the data to different nodes, then query each node one by one until the ray finds the data it needs. Kato and Saito [KS0] distributed data across all nodes in the cluster stochastically, yet still look for the data node by node. These methods do not try to lower the storage needed at run time, but rather increase memory consumption for better performance. Simplified out-of-coure rendering Christensen et al. [CLF 03] cache simplified version of the polygon meshes initially, and then dynamically tessellate them on the fly. They later extended their method to handle out-of-core photon mapping [CB04]. Yoon et al. [YLM06] proposed similar ideas by tracing level-of-detail models. These methods trades storage with accuracy. In contrast, our goal is to generate accurate renderings using full geometric information... Scheduling Coherent out-of-core rendering Some of the rendering techniques try to explore coherency among rays, using the fact that ray locality often leads to scene locality. Pharr et al. [PKGH97, PH10] reorder rays so that rays querying existing are traced first. These works are CPU oriented and reordering rays can be very GPU unfriendly, so that their GPU implementations remain unclear. Navratil et al. [NFLM07] improved upon Pharr et al. [PKGH97,PH10] by introducing a ray scheduling program to judge how to shoot coherent rays on the fly. Gribble and Ramani [GR08] uses SIMD instructions to process multiple coherent rays at once. General scheduling algorithms Our memory management strategy is somewhat related to scheduling algorithms, since we need to decide which part of data to swap incore, which is analogous to deciding which process to get scheduled. There are many works on scheduling such as [Ant06, CDM97, Nos98, BH04, MSS 06]. These algorithms gave us some inspirations. However, what we need is a new strategy specialized for ray tracing applications instead of general scheduling methods..3. Preliminaries.4. Ray Tracing In computer graphics, ray tracing is considered as a golden law to synthesize realistic images from given camera settings, models and lighting conditions. It is completely physically based because it simulates how rays interact with surfaces or volumes, and use correct physical equations to calculate the lighting. Compared with rasterization and projection based methods such as scanline method by the wellknown OpenGL or DirectX, it guarantees high quality rendering results. Since it is physically based, ray tracing naturally supports reflection, refraction, scattering and other interesting phenomena. Ray tracing works by tracing a ray backwardly from the camera to the scene. Specifically, for each pixel in the image, it traces a number of rays starting from the camera. At each intersection the ray interacts with a surface, it calculates how much radiance is contributed from the light to the incoming direction, then bounces once more to test another intersection. Note that, if the light source is an area light that has a certain size rather than a point, in order to capture the full illumination effect, we have to randomly sample positions on the light. Also, where the ray goes after bouncing is also undetermined, usually an importance sampling strategy according to the surface glossiness is used. All in all, ray tracing is a stochastic process that requires tons of samples to make a pixel converge to the correct color..5. Bounding Volume Hierarchy In ray tracing, the most time consuming step would arguably be the intersection detecting for a ray and the scene, i.e., to tell the ray tracer that the ray hits somewhere in the scene. A naive algorithm is to loop over all geometric primitives to check intersections with the given ray, which takes O(n) time. For large scenes, this is prohibitively too expensive. To lower this computation, different hierarchical structuring of spatial regions or objects of a scene is introduced. These structures are called Bounding Volume Hierarchies (BVHs). Among BVHs, the most widely used is the kd-tree, which recursively partitions space with planes that are perpendicular to the axes of a Descartes coordinate system. Every inner node of kd-tree has a splitting plane parallel to one of the axis of the coordinate system. This plane is used to subdivide the space into two subspaces for the two child nodes. The geometric primitives are also split to its children according to this plane. Those primitives intersecting both subspaces are considered in both of them. With KD-Trees, the expected complexity of one ray intersection test can be reduced from O(n) to O(log(n)), where n is the number of primitives in the object.

3 3. Our Method Figure 1: scheme. Overview of our GPU memory management The basic idea of our method is straight forward. We built a video memory manager to decide which geometric objects are in-core. The manager guarantees that the size of all incore geometric data is below a given video memory size limit at any time. All geometric data are swapped back and forth between GPU and CPU at an appropriate frequency to ensure that the tracing process flows and that most of the time are spent on actual ray intersection tests rather than memory swapping as shown in figure 1. During ray tracing, each ray queries potential geometries it may hit to perform intersection tests. Rays that try to query out-of-core geometries are blocked outside of their bounding boxes until these geometries are loaded in-core as shown in figure. Figure : Illustration of the main steps in our method. (1) Partition the scene into blocks. () Load some blocks into video memory. (3) Block rays that hit unloaded blocks and count the number of queries for that missing data. (4) Replace least used blocks with most wanted ones Preprocessing Before performing ray tracing, we need to preprocess all input geometric data. Like paging mechanism used in cache, a unit for our video memory management needs to be specified. Unlike paging, what we need is not uniform sized chunks of memory units. The problem of using uniform sized chunks is that we would not be able to keep relevant information together. For example, three coordinates of a vertex may be stored in different units using uniform sized chunks. In this case, we cannot perform ray intersections without loading both memory units. Here we define geometric block to be our unit. Suppose our video memory limit is S V. Then, we can specify a block size limit S B < S V. For each of the input geometry G, if size(g) S B, then G itself is a block, otherwise we subdivide G using KD-Tree split operation recursively until geometric data size of each leaf node of the tree is less than or equal to S B and generate one block for each leaf node. After processing all input geometries, we got a sequence of blocks {B 1,B,...,B n} ready for video memory management. 3.. Tracing All blocks we get from 3.1 have two status during ray tracing, namely in-core and out-of-core. When a block is incore, it means that all geometric data of this block is loaded to video memory and all ray intersections can be performed immediately. Out-of-core block means that its geometric data is swapped out of video memory and all intersection tests must wait for its data. For out-of-core blocks, we still keep their bounding boxes in video memory to rule out unnecessary intersection tests. To trace a ray r, the first step is to test intersection with all in-core blocks and find out the minimum intersection distance d in. Then, r performs intersection test with all bounding boxes of out-of-core blocks and find out the minimum intersection distance d out. If d in d out, r hits an in-core geometry, so it scatters on the closest in-core geometry as usual, otherwise r gets stuck outside the bounding box of an out-of-core block and marks its status as out-of-core. To make memory swapping overhead insignificant and for more efficient shadow ray connection, which is introduced in, we build a ray buffer that can hold several rays (4 in our implementation) for each pixel. With ray buffers, if one ray is blocked due to out-of-core geometry, other rays can still continue tracing. A pixel is completely blocked only when all rays in its ray buffer are blocked, so that more time is spent on intersection tests rather than swapping memory back and forth between GPU and CPU Connection Shadow ray connection is an important sampling strategy in ray tracing. Since the probability for a ray to hit the light source directly can be pretty small, which results in high variance in rendered images, for each vertex of a path traced from the camera, we can connect it with a vertex sampled on light source to generate a full path. The full path generated using connection may not exist, since the connection may be blocked by some other geometries, resulting in shadows.

4 So for each vertex on the path, we need to shoot an extra shadow ray to the vertex on light source to test visibility between the two connected vertices. Shadow ray connection can greatly improve convergence rate in scenes where the size of light source is small. However, this strategy cannot be directly used in our out-of-core tracer. Imagine we have two blocks X and Y and a light source S. Among X and Y, at most one of them can be in-core because of memory limit. Now if X is in-core and a ray bounced at X tries to connect with a vertex on S, it must test whether this connection is blocked by Y in order to generate correct shadow. However, since Y is out-of-core now, this shadow ray is blocked and the size of ray buffer increases. Once the ray buffer is full, there s no more space to generate an extra shadow ray, we can only either continue tracing the current ray and ignore connection or stop current ray and trace the shadow ray instead. If we ignore connection in this case, the tracer would generate images with high variance due to lack of connections. If we stop and trace the shadow ray instead, our tracer would only capture short paths, which would miss some important effects such as color bleeding in rendered image. Here, we applied a simple trade off solution. Russian Roulette is used to make a stochastic decision between the two options. The probability of choosing the two options are both Overlapping Bounding Boxes Our algorithm described so far can already work for scenes where all bounding boxes of blocks are not overlapping. However, this is often not the case, even KD-Tree split operation mentioned in 3.1 often produces overlapping bounding boxes. The problem with overlapping bounding boxes is that all rays that hit the overlapping area would stuck there forever. Suppose X and Y are two blocks with overlapping bounding boxes and only one of them can be in-core at the same time. Now, if a ray hits the overlapping area, no matter whether X or Y is in-core, this ray cannot proceed because the other block is always out-of-core. Our solution is that we assign to each block in the scene a unique number to specify an order. Let s assume that a ray hits an area where blocks with numbers {n 1,n,...,n m}(n 1 < n <... < n m) overlap with one another. An index I is stored in this ray to record the minimum overlapping block number. I is set to n 1 in the beginning. Once the block with number n 1 gets in-core, I is set to n and all intersection information is updated in the ray. Then, all blocks with number n < I are ignored in the next round. I keeps increasing as blocks are swapped back and forth, so that the ray continues eventually when I = n m and the block with number n m gets in-core Video Memory Management Our video memory management strategy is pretty straight forward. We store in each out-of-core ray the out-of-core block number that it is trying to trace. Then we can make a histogram of the number of querying rays for all out-of-core geometry. Then, we pick the block with most wanted rays to swap with other in-core blocks. The intuition is that by swapping in the most wanted block, the number of blocked rays that would be able continue in the next round is maximised. To decide which of those in-core blocks to get replaced, we maintain a histogram of the number of bounces occurred in last round for all in-core blocks. We pick the block with the smallest number of bounces and check if there s enough space for the out-of-core block to get in. If there is, swapping is done and tracing continues, otherwise we pick both the blocks with smallest and second to smallest number of bounces to check if there is enough space. This is done recursively until there s enough space to perform swapping. 4. Results and Comparisons We implemented our algorithm using NVIDIA CUDA, running on a quad-core Intel i7 processor at 3.5 GHz and NVIDIA GeForce GT 750M Graphics card with GB of GDDR5 memory (practically availble GPU memory size in CUDA 800 MB). Below we show some scenes rendered either out-of-core or with limited maximum available memory size which is much smaller than scene s size. Mesh Scene This scene is modified from the classic Cornell Box scene. We replaced the original small geometries with complex models of size (model size + acceleration structure) 70 MB, 30 MB and 360 MB. The output images are rendered at 104x768 resolution with at most 5 bounces for each light path, starting from a pinhole camera. For better comparisons, we omit the post processing steps such as pixel reconstruction filters. The images shown in Figure 3 are snapshots of our running ray tracing program at different times to illustrate the convergence process. Point Cloud Scene These point cloud scenes are rendered using small spheres as primitives. Scenes are culled according to view frustum and point cloud data filtered to fit image resolution. We show in Figure 4 and Figure 5 two renderings of parts of a 0 GB point cloud data. Convergence Rate Comparisons Finally we show performance using our method under different video memory limits. Performance is measured by computing the RMS error between the current intermediate result and the converged reference image. As shown in Figure 7, we limit the available video memory to different sizes. It is gratifying to see that our method still keeps reasonably good convergence rate with very small GPU memory consumption. KD-Tree traversal can be very complicated to efficiently implement on GPU, for efficient implementation, please refer to [AL09]. In this paper, we implemented a simple traversal code in CUDA. Here, we demonstrate that once we have efficient ray traversal in commercial in-core GPU tracers such as OPTIX, our ray tracer may well converge faster than popular CPU ray tracers like MITSUBA. In this experiment, we ran our program with different memory size limit for the

5 Authors / CS6 Course Project Figure 3: Renderings of a mesh scene. Data is more than 1GB in total and video memory limit is 51 MB. From left to right: 1 minute, 10 minutes and 60 minutes running time. Figure 4: Rendering of a small point cloud scene. Data is 1 GB in total after culling and filtering and video memory limit is 700 MB. From left to right: 1 minute, 10 minutes and 60 minutes running time. Figure 6: The mesh scene set up for convergence rate comparisons. Figure 5: Rendering of a large point cloud scene. Data is 6 GB in total after culling and filtering and video memory limit is 700 MB. 5 hours running time. 30 MB scene shown in figure 8. Maximum block size is set to 30MB and ray buffer size is 4. We first compared our implementation directly with MITSUBA as shown in Figure 9. However, because of our simple yet inefficient GPU ray traversal, our implementation is slower than MITSUBA. Now, let s try to factor out our traversal time. In the settings we tested, our total time spent on memory swapping in an hour is s, 0s and 16s for memory size limit 360MB, c 014 The Author(s) Computer Graphics Forum c 014 The Eurographics Association and John Wiley & Sons Ltd. 180MB and 45MB respectively. We ran the same scene using OPTIX on the same machine. The frame rate was 5 fps using OPTIX and 0.3 f ps using our implementation. Therefore, let s assume that OPTIX is 16x faster than our traversal implementation. Then for the setting that spent total memory swapping time t seconds in an hour, we rescale our time axis by a factor of s = ( 3600 t 16 + t)/3600 to assume that we have the same ray traversal implementation as OPTIX. Our comparison is shown in figure 10. We can see that for scenes that have twice the data size as memory limit, our tracer still converges faster than CPU tracers. Even for scenes that have eight times the data size as memory limit, our convergence rate is still comparable with CPU tracers.

6 Figure 7: RMS error vs time for different video memory limitations for the mesh scene in Figure 6. Note that 51 MB is enough to contain all the data completely, i.e. all the data are in-core. Figure 9: Comparison with CPU in original implementation. Figure 10: Comparison with CPU after factoring out ray traversal implementation. 5. Discussion Figure 8: Scene setting for our experiments Adaptive sampling As 4 shows, our method converges seamlessly for every pixel in the image. Nevertheless, these pixels do not converge at the same speed. Some converge very fast since either geometric data always exist or their lighting condition is simple. Others might be blocked for too long so that they converge slowly. The issue is that we don t actually know whether a pixel is converged, so that we cannot adaptively shoot more rays towards those pixels that need more samples. Current studies in computer graphics on adaptive sampling would help. We d like to consider it as a future work. 6. Conclusion and Future Works To summarize, we have presented a GPU memory management method to enable efficient out-of-core GPU ray tracing applications. We take advantage of the fact that ray tracing queries are independent in orders. By blocking queries for missing geometries, filling in new queries, and loading the most needed block to replace least used ones, we keep the tracing process flow while making memory swapping overhead very low. We tested our method in rendering outof-core triangle mesh and point cloud scenes on GPU and showed that with more sophisticated ray traversal implementation, our method could potentially be better than popular CPU tracers. In the future, we would like to investigate ways to combine adaptive sampling techniques with our scheduling method to get faster overall convergence rate. Other ray tracc 014 The Author(s)

7 ing strategies such as bidirectional path tracing and photon mapping may also benefit from our method. Finally, we would like to further improve our method to handle clustered GPU ray tracing. This would be a big challenge since every node in the cluster may demand any data in the entire scene. References [AL09] AILA T., LAINE S.: Understanding the efficiency of ray traversal on gpus. In Proceedings of the Conference on High Performance Graphics 009 (009), ACM, pp [Ant06] ANTICONA M. T.: A grasp algorithm to solve the problem of dependent tasks scheduling in different machines. In Artificial Intelligence in Theory and Practice. Springer, 006, pp [BH04] BOYER W. F., HURA G. S.: Dynamic scheduling in distributed heterogeneous systems with dependent tasks and imprecise execution time estimates. In 16 IASTED th International Conference on Parallel and Distributed Computing and Systems (PDCS 004) (004). [CB04] CHRISTENSEN P. H., BATALI D.: An irradiance atlas for global illumination in complex production scenes. In Rendering Techniques (004), Citeseer, pp [CDM97] CHAN Y.-N., DANDAMUDI S. P., MAJUMDAR S.: Performance comparison of processor scheduling strategies in a distributed-memory multicomputer system. In Parallel Processing Symposium, Proceedings., 11th International (1997), IEEE, pp [CLF 03] CHRISTENSEN P. H., LAUR D. M., FONG J., WOOTEN W. L., BATALI D.: Ray differentials and multiresolution geometry caching for distribution ray tracing in complex scenes. In Computer Graphics Forum (003), vol., Wiley Online Library, pp [DGP04] DEMARLE D. E., GRIBBLE C. P., PARKER S. G.: Memory-savvy distributed interactive ray tracing. In EGPGV (004), Citeseer, pp [FMH05] FRADIN D., MENEVEAUX D., HORNA S.: Out of core photon-mapping for large buildings. In Proceedings of the Sixteenth Eurographics conference on Rendering Techniques (005), Eurographics Association, pp [GR08] GRIBBLE C. P., RAMANI K.: Coherent ray tracing via stream filtering. In Interactive Ray Tracing, 008. RT 008. IEEE Symposium on (008), IEEE, pp [KS0] KATO T., SAITO J.: " kilauea": parallel global illumination renderer. In ACM International Conference Proceeding Series (00), vol. 9, pp [Lef93] LEFER W.: An efficient parallel ray tracing scheme for distributed memory parallel computers. In Proceedings of the 1993 symposium on Parallel rendering (1993), ACM, pp [MSS 06] MEHTA A., SMITH J., SIEGEL H. J., MACIEJEWSKI A. A., JAYASEELAN A., YE B.: Dynamic resource management heuristics for minimizing makespan while maintaining an acceptable level of robustness in an uncertain environment. In Parallel and Distributed Systems, 006. ICPADS th International Conference on (006), vol. 1, IEEE, pp. 8 pp. [NFLM07] NAVRÁTIL P. A., FUSSELL D. S., LIN C., MARK W. R.: Dynamic ray scheduling to improve ray coherence and bandwidth utilization. In Interactive Ray Tracing, 007. RT 07. IEEE Symposium on (007), IEEE, pp [Nos98] NOSSAL R.: An evolutionary approach to multiprocessor scheduling of dependent tasks. In Parallel and Distributed Processing. Springer, 1998, pp [PH10] PHARR M., HUMPHREYS G.: Physically based rendering: From theory to implementation. Morgan Kaufmann, 010. [PKGH97] PHARR M., KOLB C., GERSHBEIN R., HANRA- HAN P.: Rendering complex scenes with memory-coherent ray tracing. In Proceedings of the 4th annual conference on Computer graphics and interactive techniques (1997), ACM Press/Addison-Wesley Publishing Co., pp [RCJ99a] REINHARD E., CHALMERS A., JANSEN F. W.: Hybrid scheduling for parallel rendering using coherent ray tasks. In Proceedings of the 1999 IEEE symposium on Parallel visualization and graphics (1999), IEEE Computer Society, pp [RCJ99b] REINHARD E., CHALMERS A., JANSEN F. W.: Hybrid scheduling for realistic image synthesis. In PARCO (1999), pp [TFFH94] TELLER S., FOWLER C., FUNKHOUSER T., HAN- RAHAN P.: Partitioning and ordering large radiosity computations. In Proceedings of the 1st annual conference on Computer graphics and interactive techniques (1994), ACM, pp [War94] WARD G. J.: The radiance lighting simulation and rendering system. In Proceedings of the 1st annual conference on Computer graphics and interactive techniques (1994), ACM, pp [WDS04] WALD I., DIETRICH A., SLUSALLEK P.: An interactive out-of-core rendering framework for visualizing massively complex models. In Proceedings of the Fifteenth Eurographics conference on Rendering Techniques (004), Eurographics Association, pp [WPS 03] WALD I., PURCELL T. J., SCHMITTLER J., BENTHIN C., SLUSALLEK P.: Realtime ray tracing and its use for interactive global illumination. Eurographics State of the Art Reports 1, 3 (003), 5. [YLM06] YOON S.-E., LAUTERBACH C., MANOCHA D.: R- lods: fast lod-based ray tracing of massive models. The Visual Computer, 9-11 (006),