A distributed rendering architecture for ray tracing large scenes on commodity hardware. FlexRender. Bob Somers Zoe J.

Transcription

1 FlexRender A distributed rendering architecture for ray tracing large scenes on commodity hardware. GRAPP 2013 Bob Somers Zoe J. Wood

2 Increasing Geometric Complexity Normal Maps artifacts on silhouette edges, stretching Displacement Maps deals with high-frequency surface detail Level of Detail create multi-resolution assets, transition popping

3 Increasing Attractiveness of Commodity Hardware Performance Cost very high performance-per-watt Low Initial Cost order of magnitude cheaper than high-end machines Moderate Memory Capacity 4-8 GB is typical for a commodity box

4 What We Want Deal directly with high resolution assets and large scenes on commodity hardware with no preprocessing, visual artifacts, or slow out-of-core steps and a simple architecture.

5 What Is FlexRender? Ray tracer. Distributes geometry to workers (domain, or spatial decomposition). Passes rays between workers as messages.

6 Our Contributions 1. Feed forward messaging design with no replies. 2. Shading system that leverages this to avoid storing suspended shader execution state. 3. Adaptation of a stackless BVH traversal method (Hapala 2011) to suspend traversal on one machine and resume on another. 4. Simple self-throttling approach for keeping memory usage under control. 5. Documentation of the challenges and an open source implementation.

7 Related Work Djeu 2011 Efficient framework for multi-resolution ray tracing, with high memory requirements. Áfra 2012 Voxel-based paging system, but requires expensive preprocessing and has artifacts. DeMarle 2004 Distributed shared memory, complicated task scheduler to avoid floods, multithreaded cache storms.

8 Related Work Moon 2010 Reorder ray evaluation in cache oblivious way. Would complement our method very well. Reinhard 1999 Hybrid work scheduler to improve utilization, but has high memory overhead. Navrátil 2012 Dynamic scheduler which transitions from image plane decomposition to domain decomp. as rays diverge.

9 Tabellion 2011 Related Work Efficient out-of-core method for PBGI with high cache hit rates. Pantaleoni 2010 Compute directional occlusion and compress to spherical harmonics (not primary visibility). Garanzha 2011 GPGPU ray tracing, impressive performance for out-ofcore, but high memory requirements for the ray queue.

10 Badouel 1994 Related Work Similar asynchronous cluster design, but doesn t address shader execution (suspended state). Kato 2002 Distributes geometry randomly, duplicates rays across workers, and resolves with a depth test. Latency sensitive and network intensive. Both were written before commoditization of gigabit networks.

11 Radiometry Background Light Behaves Linearly location of computation doesn t matter Energy Is Conserved transmittance models energy conservation

12 Z-Order Curve Background

13 Two Types of Machines Workers Renderer

14 Fat Rays are Messages Origin Direction Source pixel (x, y) Transmittance Hit Record Traversal state A few others 128 bytes total

15 Rendering Overview 1. Scene data loaded and distributed. 2. Workers build BVHs in parallel. 3. Renderer builds top-level BVH. 4. Workers create image buffers. 5. Workers cast/process rays. 6. Renderer composites final image.

16 Rendering Overview Rays may be forwarded to other workers. When nearest intersection is found, illumination messages are sent to workers with emissive meshes. Those workers shoot light rays back towards intersection point.

17

18

19

20

21

22

23

24

25

26

27

28 Distributed BVH traversal with 5 workers A, B, C, D, E

29 A B 7 8 E C D

30 A B 7 8 E C D

31 A B 7 8 E C D

32 A B 7 8 E C D

33 A B 7 8 E C D

34 A B 7 8 E C D

35 A B 7 8 E C D

36 A B 7 8 E C D

37 A B 7 8 E C D

38 A B 7 8 E C D

39 A B 7 8 E C D

40 A B 7 8 E C D

41 A B 7 8 E C D

42 A B 7 8 E C D

43 Illumination and Shading

44 Ray Queues 1. Light rays. 2. Illumination messages. 3. Intersection rays. 4. Generate primary rays from camera.

45 Runaway Workers Renderer tracks primary ray casting progress of workers. Temporarily suspends primary ray casting on any worker that gets too far ahead of the cluster. Resumes primary ray casting on that worker when it s the furthest behind.

46 Render Completion Primary ray casting progress. Rays produced. Rays killed. Rays queued.

47 Experimental Setup Dell T3500 workstations Dual core Intel Xeon 2.4 GHz 4 GB of system RAM 4 GB swap partition CentOS 6 Image Plane Decomposition 8 workers, 128 pixel slices FlexRender (Domain Decomp.) 8 workers, cooperative

48 Scene Stats 41,694,206 triangles 1.09 GB of mesh data 4.97 GB of BVH data

49

50

51

52 Render Speedup Speedup Loading 0.89x Building BVH 9.18x Rendering 1.74x (avg) 8.48x (best) Total 6.57x Total (no load) 8.13x

53 Geometry Distribution Sales 0% 9% 10% 15% % 16% % 20% 8

54 Workers Touched Per Ray Expected performance O(log 8) = % 25.6% 21.8% 18.8% 8.9% 7.4% 1.5% 0.2% 63.2% at or below 36.8% above 82% at or below O(log n) % at or below O(log n) + 2

55 Ray Queue Sizes Queue Size (number of rays) Time (stats report ticks)

56 Ray Queue Sizes Avg 437 1,946 2,524 7,931 7,537 4,172 4,298 1 Max 43,954 70,261 25,254 84, , , , Max (MB) % of RAM % 0.21% 0.08% 0.25% 1.77% 0.96% 0.33% 0.00%

57 Future Work Better parallelism and workload distribution. Spatial distribution with respect to screen space. More complex work scheduling. Cache-oblivious ray reordering. Removing unnecessary +1 and +2 network hops. Asynchronous messaging allows for batching to specialized hardware (GPGPU, Xeon Phi, etc.)

58 Paper, Slides, and Code Author Contact