GLOBAL ILLUMINATION IN UNITY 5

Transcription

1 GLOBAL ILLUMINATION IN UNITY 5 Jesper Mortensen Software Engineer, Unity CPH

2 GLOBAL ILLUMINATION IN UNITY 5

3 GLOBAL ILLUMINATION IN UNITY 5

4 VIDEO Unity Shrine by Alex Lovett

5

6 AGENDA What is Enlighten? How does Enlighten work? Realtime Reflections What s next? Questions

7 WHAT IS ENLIGHTEN?

8 WHAT IS ENLIGHTEN? Middleware for computing Global Illumination (GI) Developed by Geomerics/ARM based in Cambridge Supports all modern platforms (including mobile) Used for several successful AAA titles Enlighten 2.22 is integrated into Unity 5.0

9 ENLIGHTEN GI OUTPUTS Following a precompute phase Realtime lightmaps Realtime light probes Realtime cubemaps (coming in 5.x) Following an additional baking phase Baked lightmaps Baked light probes

10 REALTIME GI OUTPUTS Realtime lightmaps/probes contain only indirect light Indirect light is generally low frequency Hence they can be low res It is radiosity only, so no specular bounce Light probes are L1 spherical harmonics Cubemaps for reflections low res (coming in 5.x)

11 BAKED GI OUPUTS High resolution baked lightmaps Direct lighting AO Indirect lighting Optional directional information for specular Relatively cheap But all lighting is static

12 WHAT DOES REALTIME GI MEAN Freely change all light properties in the player Add/remove lights Change light properties Freely modify material reflectance properties Customized shader pass Freely change emission properties of static geometry Customized shader pass Freely change the environment lighting (skybox)

13 WHAT CAN YOU DO WITH THIS? Dynamic time of day Glowy things Attached lights Flashlights Car headlights Flaming swords Lights that can be shot out Explosions Mood lighting Ghost of a Tale Time of day tests in U5

14

15 Blacksmith with indirect lighting

16 Blacksmith without indirect lighting

17

18

19 WHAT CAN YOU *NOT* DO NOW Dynamic geometry cannot bounce light Dynamic geometry cannot emit light The majority of your scene must be static The precompute/bake phases are Editor only No fully procedurally generated worlds However, we intend to fix these in 5.x!

20 SMOKE AND MIRRORS THE TRICKS Add glow by attaching lights to dynamic objects Realtime transparency is upcoming in 5.x Support destruction with realtime transparency Support doors etc. with realtime transparency Use invisible emissive static geometry for glow

21 HOW DOES ENLIGHTEN WORK?

22 HOW DOES ENLIGHTEN WORK? Enlighten has one purpose Compute indirect lighting fast It does that extremely well Everything else is built on top of that

23 HOW DOES ENLIGHTEN WORK? Solves an approximation to the rendering equation

24 HOW DOES ENLIGHTEN WORK? Assume finite elements Assume diffuse transport

25 HOW DOES ENLIGHTEN WORK? Well the radiosity of an element depends on the radiosity of all the other elements! Expensive ~O(n 2 ) This won t quite work

26 HOW DOES ENLIGHTEN WORK? Observation 1: Make n small

27 HOW DOES ENLIGHTEN WORK? Lightmaps are low res and tightly packed

28 HOW DOES ENLIGHTEN WORK? Observation 2: Decouple emitters and receivers So n receivers and m emitters, where m < n

29 HOW DOES ENLIGHTEN WORK? Scene is clustered Receivers (lightmap texels) Emitters (clusters)

30 HOW DOES ENLIGHTEN WORK? Observation 3: mn matrix is sparse Distant clusters can be merged for a given texel

31 HOW DOES ENLIGHTEN WORK? Build cluster hierarchy Relaxation metric: Area Distance Normal Visibility Scene root Leaf clusters

32 HOW DOES ENLIGHTEN WORK? For a given texel Select a hierarchy cut With a fixed budget Minimize error Scene root Leaf clusters

33 HOW DOES ENLIGHTEN WORK? Budget is a constant k Complexity is O(nk) O(n)

34 HOW DOES ENLIGHTEN WORK? Phase 1: Prime clusters with direct lighting We have a set of points per cluster Evaluate direct lighting for those Average to get per leaf cluster direct lighting Push this up the cluster hierarchy

35 HOW DOES ENLIGHTEN WORK? Phase 2: Read previously bounced lighting The cluster points has UVs mapping into output Read previous lighting value from output Average to get per leaf cluster bounced light Push this up the cluster hierarchy

36 HOW DOES ENLIGHTEN WORK? Phase 3: Gather For each pixel gather new lighting from cluster cut Done!

37 HOW DOES ENLIGHTEN WORK? Platform optimized runtimes Async from render loop Scheduling (coming in 5.x)

38

39

40

41 REALTIME REFLECTIONS

42 REFLECTION PROBES

43 REFLECTION PROBES Provide approximated specular reflection Component on a GameObject A box defines: the influence region the box projection [BEHC 10]

44 REFL. PROBE CONVOLUTION Needed for rough surfaces [LAMRUG]

45

46 REALTIME REFL. PROBE CONVOLUTION Runs on GPU in a pixel shader Seamless mipmap generation needs to support pre-dx10 hardware Accounts for projection sphere to cube

47 REALTIME REFL. PROBE SCHEDULING Time-slicing individual faces + 8 frames for convolution all faces at once + 8 frames for convolution all faces + convolution in one frame impacts framerate the most consistent results

48 SCREEN SPACE REFLECTIONS

49 SCREEN SPACE REFLECTIONS Image effect that extends built-in deferred shading pipeline User-land scripts and shaders Adds mirror & glossy SSRR to emissive buffer Falls back to reflection probes on ray miss (coming in Unity 5.x)

50 SCREEN SPACE RAYTRACING Depth-buffer as proxy for scene geometry March the ray using the depth buffer At an intersection find corresponding value in the color buffer use as radiance

51 REFLECTION PROBE ONLY

52 REFLECTION PROBE AND SSRR

53 WHAT S NEXT?

54 WHAT S NEXT? Fix bugs Improve documentation Optimize the precompute/bake phase LAN Lighting Build Cloud Lighting Build Level up interation speed by integrating PowerVR technology Improve mixed realtime/baked shadows

55 QUESTIONS? Global Illumination: (Jesper Mortensen) (Kasper Engelstoft) Reflection probes: ReJ (Renaldas Zioma) o Tomas Dirvanauskas o Benoit Sévigny Screen space reflections: (Morgan McGuire) (Michael Mara) Blacksmith: o S Holm demo team Unity Shrine: Ghost of a Tale: Dioramas: Unity Labs: (Peet Lee)