Enhancing Traditional Rasterization Graphics with Ray Tracing. October 2015

Enhancing Traditional Rasterization Graphics with Ray Tracing October 2015

James Rumble Developer Technology Engineer, PowerVR Graphics

Overview Ray Tracing Fundamentals PowerVR Ray Tracing Pipeline Using Ray Tracing 3

Overview Ray Tracing Fundamentals PowerVR Ray Tracing Pipeline Using Ray Tracing 4

Introduction to Ray Tracing Ray Tracing is not a new concept! Synonymous with photorealistic (offline) renders Traditionally too resource intensive for broad industry applications Ray tracing is emerging as the next big thing in real-time graphics Used in a limited fashion i.e. reflections and shadows Often limited to very few rays and low resolution/course grained scene representations The benefits ray tracing can offer are obvious 5

Ray Tracing Fundamentals Triangles Rays Intersection Testing Process of determining if a given ray has intersected a given triangle RAY = {origin, direction, clipping distance} 6

Tracing Rays Forward Ray Tracing vs Backward Ray Tracing Commonly visualised and explained as a process of modelling light Rays used to model real-world behaviour of light Forward Ray Tracing: Rays traced from the light source Many rays never reach our eyes Backward Ray Tracing: Rays traced from the eye towards light 7

Tracing Rays Forward Ray Tracing vs Backward Ray Tracing Commonly visualised and explained as a process of modelling light Rays used to model real-world behaviour of light Forward Ray Tracing: Rays traced from the light source Many rays never reach our eyes Backward Ray Tracing: Rays traced from the eye towards light 8

What is Ray Tracing? Backwards Ray Tracing How is ray tracing commonly used? Primary rays fired from eye through pixels of the screen Finds closest triangle intersection Secondary Rays used for additional effects 9

Ray Tracing Basics Backwards Ray Tracing Now using a concrete example Cast primary rays out into the world through each pixel Find the closest triangle intersection Secondary Rays used for additional effects (direct lighting) 10

What is Ray Tracing? Spatial Queries Can be used in a more abstract way than simply modelling light Can be used to calculate spatial queries between rays and objects/triangles i.e. Does ray x intersect object 1 A ray can be fired as long it has been given an origin and direction Fire a ray from a box towards a neighbouring object. It intersects the plane in between Fire a ray from a box towards a neighbouring object. It reaches the other box and is marked as intersected 11

What is Ray Tracing? Ray Tracing provides a mechanism for determining inter-object visibility using spatial queries Objects can be aware of both their own properties and position but now also those of other objects in the scene through the use of rays Provides elegant solutions to many traditionally difficult to render or process effects 12

Overview Ray Tracing Fundamentals PowerVR Ray Tracing Pipeline Using Ray Tracing 13

Introducing PowerVR Ray Tracing PowerVR Ray Tracing GPUs are built on top of the latest generation of PowerVR GPUs Additional ray tracing specific hardware designed to integrate efficiently and communicate perfectly with the rest of the GPU architecture Real-time, interactive ray traced graphics that can comfortably scale from mobile and embedded platforms to gaming consoles and mainstream gaming PCs 14

PowerVR Ray Tracing Pipeline Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy 15

Scene Hierarchy Organises triangles based on their size and position Scene Hierarchy Building Phase Vertex Processing Hierarchy Generation Memory Geometry Data Scene Hierarchy Geometry Submission Scene Components Scenes 16

Scene Hierarchy Generation Geometry submission Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy Geometry Submission Scene Components Scenes 17

Scene Hierarchy Generation Scene Components Windshield Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Chrome Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Car Body Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Headlights Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Tyres Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures 18

Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Component: - Program: - Vertex Shader - Ray Shader - Uniforms - Geometry Data - Buffers - Textures Scene Hierarchy Generation World Scenes Contain groups of scene components Ray traversal occurs within scene Windshield Chrome Windshield Chrome Car Body Car Body Headlights Tyres Headlights Tyres Windshield Chrome Car Body Headlights Tyres 19

Scene Hierarchy Generation Vertex processing Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy 20

Scene Hierarchy Generation Vertex processing Vertex shader used to transform geometry to world space 21

Scene Hierarchy Generation Hierarchy generation Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy 22

Scene Hierarchy Generation Hierarchy generation Vertex shader output is passed to a fixed function scene generator which calculates the scene hierarchy 23

Scene Ray Traversal Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy 24

Scene Ray Traversal Primary ray generation Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy 25

Scene Ray Traversal Primary ray generation Frame Shaders are run once per pixel to emit primary rays into the scene 26

Scene Ray Traversal Intersection tests Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy 27

Scene Ray Traversal Intersection tests Simple approach each ray tested against all triangles: Brute force m * n intersection tests (m rays, n objects) Use accelerated data structure to reduce intersection tests Application developer does not need to worry about handling this Accelerated triangle and box testing 28

Scene Ray Traversal Intersection tests Simple approach each ray tested against all triangles: Brute force m * n intersection tests (m rays, n objects) Use accelerated data structure to reduce intersection tests Fixed function hardware accelerated triangle and box testing 29

Scene Ray Traversal Ray shaders and accumulation Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy 30

Scene Ray Traversal Ray shaders and accumulation Ray Shaders: Defined for each scene component and invoked when a ray intersects a triangle Defines material properties of component (fully programmable) Can emit further rays causing additional intersection tests accumulate(0.85, 0.85, 0.75); reflect(incidentdirection, normal) 31

Types of Rays are important Rays are a communication medium making inter-object visibility possible An engine will likely use multiple types of ray Primary Rays If it hits anything shade it / emit additional rays Shadow Rays if it doesn t hit a light kill it etc. 32

Coherence-Gathering What if these rays could be automatically coalesced? 33

Pipeline Summary Scene Hierarchy Building Phase Ray Traversal Phase Vertex Processing Hierarchy Generation Primary Ray Generation Intersection Tests Ray Shaders Accumulation Memory Geometry Data Scene Hierarchy Geometry Submission Scene Components Scenes 34

Overview Ray Tracing Fundamentals PowerVR Ray Tracing Pipeline Using Ray Tracing 35

How do we use it? Hybrid Shadows, Reflections, etc. Augmented Reality Production- Order-Independent Quality Renders Transparency Ambient Occlusion Asset creation / compression Global Illumination Physics & Collision Detection Virtual Reality Lens correction, Ultra-low latency rendering, Lenticular Displays A.I. & Line of Sight Calculations Rapid photoquality output 36

Modern Raster-Based Game Engine Deferred shading used in many modern game engines Render the scene into a deep frame buffer (G-Buffer) G-Buffer Material IDs Normals Lighting Buffer(s) Compute Lighting on each G-Buffer pixel (No Overdraw) World-Space Positions or depth buffer Lighting calculations executed in screen space Assemble the final frame and perform screen-space effects Final Result Frame 37

Ray Trace Using the G-Buffer Who needs primary rays when we have rasterization G-Buffer Material IDs Normals World Space Positions or Depth values (1.0,0.0,0.0) (0.5,0.6,-0.2) (14.3,81.6,-0.4) 38

Ray Trace Using the G-Buffer Who needs primary rays when we have rasterization G-Buffer Material IDs Normals World Space Positions or Depth values (1.0,0.0,0.0) (0.5,0.6,-0.2) (14.3,81.6,-0.4) 39

Hybrid Ray Tracing Use the G-Buffer to set up your rays Render the scene into a deep frame buffer (G-Buffer) G-Buffer Material IDs Normals Lighting Buffer(s) Trace Rays to determine lighting World-Space Positions or depth buffer Assemble the final frame and perform screen-space effects Final Result Frame 40

Ray Traced Techniques Analytical 1 Ray Per Pixel Soft Shadows Ray traced Cascaded shadow maps http://cdn.imgtec.com/sdk-presentations/idc15_sf_enhancingtraditionalrasterizationgraphicswithraytracing.pdf 41

Ray Traced Techniques Low Overhead Probe based Global Illumination Direct Lighting Only Ray Traced Global Illumination http://cdn.imgtec.com/sdk-presentations/idf15_sf_lowoverheadprobebasedglobalilluminationusingraytracing.pdf 42

Summary An introduction to ray tracing The PowerVR Ray Tracing Pipeline Hybrid Ray Tracing Ray Traced Technique Teaser 43