Game Technology Lecture 7 4.12.2017 Physically Based Rendering Dipl-Inform. Robert Konrad Polona Caserman, M.Sc. Prof. Dr.-Ing. Ralf Steinmetz KOM - Multimedia Communications Lab PPT-for-all v.3.4_office2010 2012.09.10.pptx author(s) of these slides including research results from the KOM research network and TU Darmstadt; otherwise it is specified at the respective slide 5-Dec-17 Template all v.3.4
Intro https://www.youtube.com/watch?v=okmuxgfjkpy KOM Multimedia Communications Lab 2
Light Starts from light source Bounces around Looses intensity with each collision Eventually reaches the camera KOM Multimedia Communications Lab 3
Light Sources KOM Multimedia Communications Lab 4
Rendering Lots of Lights Forward Rendering Iterate over all lights in the pixel shader Optionally use a pre-depth pass Deferred Rendering Render buffers of depth, normals, materials, Render simple geometry, approximating light distributions Add light inside the geometry using a pixel shader Forward+ Create 3D grid, assign most important lights to each grid Pull light info from the grid in the pixel shader KOM Multimedia Communications Lab 5
Point Lights Defined by a position and light color/intensity KOM Multimedia Communications Lab 6
Spot Lights Point light plus an angle KOM Multimedia Communications Lab 7
Directional Light Just a direction and light intensity/color KOM Multimedia Communications Lab 8
Directional Light KOM Multimedia Communications Lab 9
Area Lights Simple solution: Approximate using multiple point/spot lights Analytical solution: https://labs.unity.com/article/real-time-polygonallight-shading-linearly-transformed-cosines KOM Multimedia Communications Lab 10
Bounces KOM Multimedia Communications Lab 11
BRDF Bidirectional reflectance distribution function incoming light direction outgoing direction (for example to the camera) returns the ratio of reflected radiance f r (ω i, ω r ) KOM Multimedia Communications Lab 12
Path Tracing Extended Raytracing foreach (pixel) bounce around a lot Use BRDF at each collision Very slow but useful to create reference images and for prerendered lighting information KOM Multimedia Communications Lab 13
Realtime Lighting Consider only light rays from direct light sources First bounce Use shadow maps Second bounce Ignore further light bouncing No reflections No ambient light KOM Multimedia Communications Lab 14
Image-Based Lighting Put surroundings in cube map Use for example path tracing to generate the cube map Ignore lights, instead sample cube map A cube map is only correct for one position Ignores dynamic objects KOM Multimedia Communications Lab 15
HDR High dynamic range Use more than 32 bits of data for one pixel KOM Multimedia Communications Lab 16
BRDF Bidirectional reflectance distribution function incoming light direction outgoing direction (for example to the camera) returns the ratio of reflected radiance f r (ω i, ω r ) KOM Multimedia Communications Lab 17
BRDF Shortcomings Subsurface-Scattering Wavelength dependence KOM Multimedia Communications Lab 18
BRDF Only positive light f r ω i, ω r 0 KOM Multimedia Communications Lab 19
BRDF Inverted f r ω i, ω r = f r (ω r, ω i ) KOM Multimedia Communications Lab 20
BRDF Energy conserving ω i, න Ω f r ω i, ω r cos θ r dω r 1 KOM Multimedia Communications Lab 21
Phong Lighting color = ambient + diffuse + specular KOM Multimedia Communications Lab 22
Gamma Correction / srgb See Exercise 1 Transform textures to linear (pow 2.2) Or use srgb texture reading (also allows proper filtering) Lighting calculations in linear space (gamma 1) Then transform for srgb (pow 1 / 2.2) KOM Multimedia Communications Lab 23
Diffuse & Specular KOM Multimedia Communications Lab 24
Diffuse Lambertian reflectance / Phong diffuse I = L*N Good enough for modern engines Used for example in Unreal Engine 4 KOM Multimedia Communications Lab 25
Metals and Dielectrics http://www.manufato.com/wp-content/uploads/2010/10/imagem-35-english.jpg KOM Multimedia Communications Lab 26
Metals and Dielectrics Metals No diffuse High Specular Dielectrics Diffuse Lower but still surprisingly high Specular Note: Specular value is specified at low angles KOM Multimedia Communications Lab 27
Polarization of Reflected Light Specular Reflection Polarization does not change Diffuse Reflection Polarization is randomized KOM Multimedia Communications Lab 28
Cardboard KOM Multimedia Communications Lab 29
Cardboard Diffuse KOM Multimedia Communications Lab 30
Cardboard Specular KOM Multimedia Communications Lab 31
Metal KOM Multimedia Communications Lab 32
Metal Specular KOM Multimedia Communications Lab 33
Metal Diffuse KOM Multimedia Communications Lab 34
Specularity angle(normal, light) = angle(normal, camera) KOM Multimedia Communications Lab 35
Brick angle(normal, light) = angle(normal, camera) KOM Multimedia Communications Lab 36
Fresnel KOM Multimedia Communications Lab 37
Fresnel KOM Multimedia Communications Lab 38
Schlick Approximation Schlick(spec, light, normal) = spec + (1 - spec) (1 - (light*normal))^5 light*normal = 1 Schlick = spec light*normal = 0 Schlick = 1 spec characteristic specular reflectance specular color KOM Multimedia Communications Lab 39
Microfacet Model KOM Multimedia Communications Lab 40
Microfacet BRDF f l, v = F l, h G l, v, h D(h) 4(n l)(n v) KOM Multimedia Communications Lab 41
Microfacet BRDF Fresnel Reflectance f l, v = F l, h G l, v, h D(h) 4(n l)(n v) KOM Multimedia Communications Lab 42
Microfacet BRDF Active microfacets f l, v = F l, h G l, v, h D(h) 4(n l)(n v) KOM Multimedia Communications Lab 43
Microfacet BRDF Normal Distribution Function f l, v = F l, h G l, v, h D(h) 4(n l)(n v) KOM Multimedia Communications Lab 44
Normal Distribution Function D Evaluated for h The concentration of microfacets that have an orientation so they could reflect light to the camera Might still be occluded, KOM Multimedia Communications Lab 45
Normal Distribution D(h) Portion of microfacets pointing to h D tr m = π α2 tr n m 2 α 2 2 tr 1 + 1 Trowbridge-Reitz (GGX) α: Roughness KOM Multimedia Communications Lab 46
Microfacet BRDF Shadow Masking Function f l, v = F l, h G l, v, h D(h) 4(n l)(n v) KOM Multimedia Communications Lab 47
Shadow Masking Function G Also referred to as the Geometry Function Probability that microfacets with normal h are visible from both the light and the view direction KOM Multimedia Communications Lab 48
Geometry Factor G(l, v, h) Cook-Torrance: G ct l, v, h = min 1, 2 n h n v v h, 2 n h n l v h KOM Multimedia Communications Lab 49
Microfacet BRDF f l, v = F l, h G l, v, h D(h) 4(n l)(n v) KOM Multimedia Communications Lab 50
Physically Based Rendering In practice: Gamma correction Microfacet BRDF Lots of Cube Maps KOM Multimedia Communications Lab 51
Textures Typical Setup Diffuse texture Specular texture Roughness texture Normal Map KOM Multimedia Communications Lab 52
Diffuse Texture KOM Multimedia Communications Lab 53
Specular Texture KOM Multimedia Communications Lab 54
Smoothness Texture KOM Multimedia Communications Lab 55
Incorporating Image Based Lighting Precalculate Cube Maps Lots of Cube Maps Manually placed in level editor KOM Multimedia Communications Lab 56
Cube Maps KOM Multimedia Communications Lab 57
Cube Maps Can be interpolated Which is a rough approximation Can not capture dynamic objects KOM Multimedia Communications Lab 58
Reflection Rendering As done in Unreal Engine 4, Killzone Shadow Fall, Deferred Rendering Pass Raytrace depth buffer If no hit Interpolate local cube maps If no hit Use global cube map KOM Multimedia Communications Lab 59
Ambient Occlusion Small notches are normally shadowed Unless lit directly Calculations need very exact light bounces KOM Multimedia Communications Lab 60
Screen Space Ambient Occlusion Filter after rendering Darken at sharp normal changes KOM Multimedia Communications Lab 61
Global Illumination Ambient light Spherical Harmonic Lighting Voxel Cone Tracing KOM Multimedia Communications Lab 62
Summary Stay closer to nature Energy conservation Metals vs. Dielectrics Textures, parameters can be capured from nature More reusable assets No specific lighting information baked/painted into textures Physically Based Shading in Theory and Practice Course @SIGGRAPH http://blog.selfshadow.com/publications/ KOM Multimedia Communications Lab 63
GPU Internals NVidia GeForce GT 6600, 2004 KOM Multimedia Communications Lab 64
Memory Memory bandwidth is extremely important Textures Framebuffer Depth Buffer Memory access times not very important Most data is streamed Access times can be hidden by switching tasks KOM Multimedia Communications Lab 65
Memory Gigantic discrepancy from low-end to high-end High End CPU: 60 GB/s GeForce 1030: 48 GB/s PS4: 176 GB/s GeForce 1080 Ti: 484 GB/s KOM Multimedia Communications Lab 66
Vertex and Fragments Shaders Run on the same hardware Dynamically scheduled KOM Multimedia Communications Lab 67
CPU - GPU https://developer.nvidia.com/content/life-triangle-nvidiaslogical-pipeline KOM Multimedia Communications Lab 68
GPU KOM Multimedia Communications Lab 69
Rasterization, Pixel Shader KOM Multimedia Communications Lab 71
Work distribution KOM Multimedia Communications Lab 72
SMT (Symmetric Multithreading) Also used in CPUs (Hyperthreading) Switch to different thread when stalled (for example waiting for memory) KOM Multimedia Communications Lab 73
SIMD Compiler can put calculations on multiple vertices/pixels in one instruction Problem: Flow control Wrong paths pseudo-executed Can be efficient when all vertices/pixel in one pack take the same paths Shader variants Typically when you create a simple surface shader, it internally expands into 50 or so internal shader variants (Shader Compilation in Unity 4.5) KOM Multimedia Communications Lab 74
Parallelization Small work packages can prevent parallelization Performance dip for tiny triangles KOM Multimedia Communications Lab 75
CPU GPU Biggest performance trap Minimize state changes Minimize draw calls Send little data to the GPU If possible never read data from the GPU KOM Multimedia Communications Lab 76
Summary Broad overview of GPU internals CPU GPU interaction Work distribution Massively parallel execution (vertex, pixel shaders) Multithreading and SIMD KOM Multimedia Communications Lab 77