Practical Implementation of Light Scattering Effects Using Epipolar Sampling and 1D Min/Max Binary Trees

Transcription

1 Practical Implementation of Light Scattering Effects Using Epipolar Sampling and 1D Min/Max Binary Trees Egor Yusov Graphics Software Engineer Intel Corporation

2 Demo 2

3 Agenda Introduction Light Scattering Theory Solution to Inscattering Integral Epipolar Sampling 1D Min/Max Binary Trees Implementation Performance 3

4 Introduction Light scattering greatly enhances the realism It is hard to compute Achieving high-performance is challenging Pt C, O, L = e T P C β P p θ L In S O S C Ie T L P L P 2 V P ds B T A B = β P ds A p R θ = 3 16π p M θ = 1 3(1 g 2 ) 1 + cos 2 θ 4π 2(2 + g 2 ) 1 + g 2 2gcosθ 3/2 1 + cos2 θ 4

5 Previous Work Hoffman & Preetham, 2002 Alanalytical model for outdoor light scattering Sun et al, 2005 Semi-analytical solution to airlight integral Tevs et al, 2008 Using maximum mipmaps for dynamic height field rendering Chen et al, 2011 Using 1d min-max mipmaps for volumetric shadows Engelhardt & Dachsbacher, 2010 Epipolar Sampling 5

6 Inscattering Integral Derivation Aerial Perspective Light source L L Cam = L Obj e T O C +L In For homogeneous medium B T A B = β P ds A = β A B Camera C Object O 6

7 Inscattering Integral Derivation Radiant Intensity of a Point Light I radiant intensity v = O C O C view direction C L I O v L In = I 7

8 Inscattering Integral Derivation Attenuation and optical thickness L Amount of energy reaching P is Inversely proportional to distance squared Attenuated by the extinction in the media C P O L In = I T L P e L P 2 8

9 Inscattering Integral Derivation Visibility factor L V P = 1 if point is lit and 0 if shadowed Filtered in practice C P O L In = V P I T L P e L P 2 9

10 Inscattering Integral Derivation Scattering coefficient L Total scattered light (in any direction) is proportional to Section length ds Scattering coefficient β P C P O ds L In = β P ds V P I T L P e L P 2 10

11 Inscattering Integral Derivation Phase function L Phase function gives amount of light scattered towards the camera C dl In P O dl In θ L In = p θ β P ds V P I T L P e L P 2 11

12 Inscattering Integral Derivation Attenuation of inscattered light L Differential inscattered light is attenuated in the media Total inscattering is given by integrating differential amounts C dl In P O θ L In = C O e T P C ds p θ β P V P I T L P e L P 2 T P C e 12

13 Inscattering Integral Derivation Directional light L Phase function does not vary across the ray Light intensity is constant C O L In = p θ O e T P C β P V P E Sun C ds 13

14 Scattering Properties of the Media Rayleigh scattering Caused by molecules Wavelength-dependent β R. rgb = (5.8, 13.5, 33.1) 10 6 Almost isotropic: p R θ = 3 16π (1 + cos2 θ) 14

15 Scattering Properties of the Media Mie Scattering Caused by aerosols Wavelength-independent β M. rgb = Anisotropic p M θ = 1 3(1 g 2 ) 4π 2(2 + g 2 ) 1 + cos 2 θ 1 + g 2 2gcosθ 3/2 15

16 Scattering Properties of the Media Scattering due to both types of particles T A B = β Σ A B β Σ = β R + β M βp θ β R p R θ + β M p M θ P Σ (θ) 16

17 Scattering Properties of the Media Isotropic phase function p M θ = 1 4π 17

18 Scattering Properties of the Media Anisotropic phase function p M θ = 1 3(1 g 2 ) 4π 2(2 + g 2 ) 1 + cos 2 θ 1 + g 2 2gcosθ 3/2 18

19 Solution to Inscattering Integral Directional light fully analytical solution L L Dir In θ, S = E Sun P Σ (θ) β Σ 1 e β Σ S S = O C C O θ S 19

20 Solution to Inscattering Integral Point light source integral reformulation P C + L P L L Pt In = L Pt In h, S C, S O = I S O P Σ (θ) S C e β Σ s S C + h 2 +s 2 h 2 + s 2 ds C h s P O cos θ = s h 2 + s 2 L P 2 S C L 0 θ S O 20

21 Solution to Inscattering Integral Point light source look-up table L Precompute 2D LUT: L h, S = L Pt In h, S, + C h L 0 S 21

22 Solution to Inscattering Integral Point light source semi-analytical solution L L Pt In h, S C, S O = L h, S C e β Σ S O S C L h, S O C h L 0 O T C O S C S O 22

23 Volumetric Shadows T S i Ent C L Pt In = n 1 e β Ent Σ S i SC i=0 e β Ext Σ S i SC L h, S i Ent L h, S i Ext Ent S Ext Ent Ext Ent Ext 0 S 0 S S 2 S 2 1 S 1 T S i Ext C 23

24 Volumetric Shadows L Dir In = F(θ) n 1 i=0 e β Σ S i Ent e β Σ S i Ext F θ = E Sun P Σ (θ) β Σ Ent S Ext Ent Ext Ent Ext 0 S 0 S S 2 S 2 1 S 1 24

25 Volumetric Shadows Starting point - tracing the view ray in shadow map space For each pixel: Project the view ray onto the shadow map Set up PrevL In. rgb = L h, S C, L In. rgb = 0 Step through each texel: CurrL In. rgb = e β Σ S S C L h, S L In. rgb += PrevL In. rgb CurrL In. rgb V(P) PrevL In. rgb = CurrL In. rgb CurrL In. rgb PrevL In. rgb 25

26 Volumetric Shadows Perspective-correct interpolation 1 z = 1 z C + α 1 z O 1 z C Shadow Map plane S C S z = S C z C + α S O z O S C z C z C α 1 α z S z O S O 26

27 Volumetric Shadows Starting point - tracing the view ray in shadow map space Need to test all samples along the ray Too slow (> 90 ms on GeForce 680 GTX) Optimizations are necessary! 27

28 Epipolar Sampling 28

29 Epipolar Sampling 29

30 Epipolar Sampling 30

31 Epipolar Sampling Sample generation Exit point Entry point Entry point Exit point 31

32 Epipolar Sampling Sample generation improved sample placement Too dense sampling Much better sampling 32

33 Epipolar lines Epipolar Sampling Coordinate texture Sample coordinate on each line 33

34 Epipolar Sampling Sample classification Initial ray marching samples are placed equidistantly Additional ray marching samples are placed at depth breaks The remaining samples are interpolation samples 34

35 Epipolar Sampling Interpolation source texture 0,0 0,3 0,3 3,3 4,4 4,8 4,8 4,8 8,

36 Epipolar Sampling Transformation from epipolar to rectangular coordinates Cast epipolar line through the pixel Perfrom bilateral filtering One Gather() and two Sample() instructions are used 36

37 Epipolar Sampling Fix-up pass Not all samples can be properly interpolated from epipolar coordinates These samples are marked in stencil Additional fix-up pass is performed without 1D min/max acceleration 37

38 Epipolar Sampling 91 ms -> 14 ms We can do better! 38

39 1D min/max binary trees 1D height map & first level Depth 39

40 1D min/max binary trees Second level Depth 40

41 1D min/max binary trees Third level Depth 41

42 1D min/max binary trees Traversal optimization max d A, d B < d min : lit min d C, d D > d max : shadowed EF: neither lit nor shadowed A B max d A, d B Depth E F d min d max min d C, d D C D 42

43 Slice Origin and Direction v Location of the i-th sample is O UV + i D UV Shadow map D UV O UV O UV D UV u 43

44 Slice Origin and Direction S 0 = S 1 = L C L C S Exit C S Exit C N Slice S 1 Exit point View ray N Slice = S 1 S 0 S 0 N Light D = N Slice N Light D Shadow map plane O 44

45 Slice Origin and Direction O Shadow map plane 45

46 Slice Origin and Direction N Light Shadow map plane P 0 = c 1 N Slice + c 2 N Light + td O D P 0 t min t max O = P 0 + t min D S 1 S 0 N Slice 46

47 Slice Origin and Direction N Slice N Slice S 1 S 1 L L O D D O L L 47

48 Colored Light Shafts 48

49 Implementation Details Auxiliary textures Name Format Dimension Back Buffer RGBA_UNORM8 W Screen H Screen Camera Space Z R_FLOAT32 W Screen H Screen Camera Depth Stencil D24_S8 W Screen H Screen Coordinate texture RG_FLOAT32 N Samples N Slices Epipolar depth-stencil D24_S8 N Samples N Slices Epipolar camera space Z R_FLOAT32 N Samples N Slices Interpolation source RG_UINT16 N Samples N Slices Inscattering 2 RGBA_FLOAT16 N Samples N Slices Slice End Points RGBA_FLOAT32 N Slices 1 Slice UV origin & direction RGBA_FLOAT32 N Slices 1 1D min/max shadow map 2 RG_FLOAT16 D ShadowMap N Slices Algorithm specific data 49

50 Implementation Camera space Z 3. Render coordinate texture Camera depth buffer 2. Render Slice End Points in Screen Space 1. Reconstruct camera space Z Slice End Points Epipolar coordinates Epipolar camera space Z Epipolar depth stencil Slice UV Orig & Dir 5. Render Slice Origin and Direction in Shadow Map Space 4. Refine Sample Locations Interpola tion source texture

51 Implementation Shadow map Epipolar depth stencil 7. Mark Ray Marching Samples Interpolation source texture Slice UV Orig & Dir 8. Do Ray Marching Camera space Z 6. Build 1D min/max mipmap Initial inscattering Epipolar coordinates 1D min-max mipmap

52 Implementation Initial inscattering Epipolar camera space Z Camera space Z Source Color Buffer Interpolated inscattering 9. Interpolate inscattering 10. Transform to rectangular coordinates Interpolation source texture Slice End Points 52

53 Implementation Camera space Z Source Color Buffer Shadow map Final image 12. Fix inscattering 53

54 Implementation Details Discontinuities search with CS Each ray section is processed by one thread group Each thread processes one sample Thread Group Thread Group 54

55 Implementation Details Discontinuities search with CS Step 1: each thread sets 1-bit flag in group shared array indicating if there is a depth break InterlockedOr() must be used Step 2: find closest depth break or section end for each sample using bit manipulation and firstbitlow() and firstbithigh() intrinsics Up to 6x speed-up 55

56 Implementation Details Min/max binary tree construction 56

57 Implementation Details Min/max binary tree construction v Step 1: construct 1 st tree level using shadow map Use Gather() instructions to read min/max z for each pair of two adjacent samples Compute conservative min/max z Step 2: Construct coarser tree levels Use temporary texture to render to O D One texel u 57

58 Implementation Details Optimized sample placement Gives more than 5% speedup /H Less efficient sample placement More efficient sample placement -1 2/W 58

59 Performance ms Reconstruct cam Z resolution, NVIDIA GeForce 680 GTX, spot light Profile # slices # samples SM Time, ms Mem, MB Brute Force High Quality Balanced High performance Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering fixup 59

60 Brute Force Ray Marching 60

61 High Quality 61

62 Balanced 62

63 High Performance 63

64 Performance resolution, Nvidia Quadro 1000M, spot light Profile # slices # samples SM Time, ms Mem, MB Brute Force High Quality Balanced High performance ms Reconstruct cam Z Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering fixup 64

65 Brute Force Ray Marching 65

66 High Quality 66

67 Balanced 67

68 High Performance 68

69 Performance , Intel CPU with 3rd Gen HD Graphics, spot light Profile # slices # samples SM Time, ms Mem, MB Brute Force High Quality Balanced High performance ms Reconstruct cam Z Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering fixup 69

70 Brute Force Ray Marching 70

71 High Quality 71

72 Balanced 72

73 High Performance 73

74 Performance Spot light, balanced quality resolution, NVIDIA GeForce 480 GTX 1024 slices, 512 samples in slice, Shadow Map Time, ms ms Reconstruct cam Z Render slice enter/exit points Render coord texture 0.31 Detect depth breaks 0.01 Render slice origin and dir 0.38 Build 1D min/max trees Ray march Interpolate Transform to rect coords 0.26 Inscattering fix-up 74

75 Performance Directional light, high quality resolution, NVIDIA GeForce 480 GTX 1024 slices, 512 samples in slice, Shadow Map Time, ms ms Reconstruct cam Z Render slice enter/exit points Render coord texture 0.31 Detect depth breaks 0.01 Render slice origin and dir 0.65 Build 1D min/max trees Ray march Interpolate Transform to rect coords 0.20 Inscattering fix-up 75

76 Performance Ray marching, 5.49 Time, ms > 2x faster Binary tree construction, 0.38 ms No binary trees Ray marching, 1.67 With 1D min/max trees 76

77 Performance Colored light shafts Binary tree optimization is less efficient Ray marching: 1.67 ms -> 5.49 ms Total time: 3.36 ms -> 6.86 ms Still less than 7 ms on NVIDIA GeForce 480 GTX 77

78 Advantages Over Previous Approaches Semi-analytical solution is used to calculate the integral on each ray segment No sophisticated mathematics like SVD etc. Epipolar scattering is combined with 1D min/max binary trees Significant performance improvements Efficient implementation methods 78

79 Integration Into Existing Apps The technique is fully post-processing The only inputs are: Depth buffer Back buffer Shadow map [optional] Stained glass color The source code is available at

80 Extensions Large outdoor environments Integration with cascaded SM, VSM/EVSM Heterogeneous (dynamic) media Clouds Smoke Sparse octree/prt 80

81 References HOFFMAN N., PREETHAM A. J.: Rendering outdoor light scattering in real time. In Proceedings Game Developer Conference ENGELHARDT T., DACHSBACHER C.: Epipolar sampling for shadows and crepuscular rays in participating media with single scattering. In Proc ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, ACM, CHEN J., BARAN I., DURAND F., JAROSZ W.: Real-time volumetric shadows using 1d min-max mipmaps. In Proceedings of the Symposium on Interactive 3D Graphics and Games, GAUTRON P., MARVIE J.-E., FRANCOIS G.: Volumetric shadow mapping. ACM SIGGRAPH 2009 Sketches. TEVS A., IHRKE I., SEIDEL H.-P.: Maximum mipmaps for fast, accurate, and scalable dynamic height field rendering. In Symposium on Interactive 3D Graphics and Games (i3d 08), SUN B., RAMAMOORTHI R., NARASIMHAN S. G., NAYAR S. K.: A practical analytic single scattering model for real time rendering. ACM Trans. Graph. 24, 3,

82 Acknowledgments The presenter would like to thank: Artem Brizitsky Jeffery A. Williams Doug Mcnabb Dave Bookout Charu Chandrasekaran Quentin Froemke Glen Lewis Kyle Weicht 82

83 Thank you! 83