Metal for Ray Tracing Acceleration

Similar documents
Rendering. Converting a 3D scene to a 2D image. Camera. Light. Rendering. View Plane

Working with Metal Overview

COMP 4801 Final Year Project. Ray Tracing for Computer Graphics. Final Project Report FYP Runjing Liu. Advised by. Dr. L.Y.

Enhancing Traditional Rasterization Graphics with Ray Tracing. October 2015

Shadows. COMP 575/770 Spring 2013

Enhancing Traditional Rasterization Graphics with Ray Tracing. March 2015

Introducing Metal 2. Graphics and Games #WWDC17. Michal Valient, GPU Software Engineer Richard Schreyer, GPU Software Engineer

Photorealistic 3D Rendering for VW in Mobile Devices

Metal. GPU-accelerated advanced 3D graphics rendering and data-parallel computation. source rebelsmarket.com

Enabling immersive gaming experiences Intro to Ray Tracing

NVIDIA Case Studies:

Interpolation using scanline algorithm

Metal for OpenGL Developers

Computer Graphics 1. Chapter 7 (June 17th, 2010, 2-4pm): Shading and rendering. LMU München Medieninformatik Andreas Butz Computergraphik 1 SS2010

What s New in Metal, Part 2

03 RENDERING PART TWO

Input Nodes. Surface Input. Surface Input Nodal Motion Nodal Displacement Instance Generator Light Flocking

Mach band effect. The Mach band effect increases the visual unpleasant representation of curved surface using flat shading.

Practical Techniques for Ray Tracing in Games. Gareth Morgan (Imagination Technologies) Aras Pranckevičius (Unity Technologies) March, 2014

Working With Metal Advanced

From Art to Engine with Model I/O

Real-Time Universal Capture Facial Animation with GPU Skin Rendering

CS354R: Computer Game Technology

Sung-Eui Yoon ( 윤성의 )

Computer Graphics. Lecture 02 Graphics Pipeline. Edirlei Soares de Lima.

CS 498 VR. Lecture 18-4/4/18. go.illinois.edu/vrlect18

Next-Generation Graphics on Larrabee. Tim Foley Intel Corp

CS230 : Computer Graphics Lecture 4. Tamar Shinar Computer Science & Engineering UC Riverside

Getting fancy with texture mapping (Part 2) CS559 Spring Apr 2017

NVIDIA RTX: Enabling Ray Tracing in Vulkan. Nuno Subtil, Sr. Devtech Engineer / Eric Werness, Sr. System Software Engineer March 27, 2018

Radeon ProRender and Radeon Rays in a Gaming Rendering Workflow. Takahiro Harada, AMD 2017/3

Topic 12: Texture Mapping. Motivation Sources of texture Texture coordinates Bump mapping, mip-mapping & env mapping

TSBK03 Screen-Space Ambient Occlusion

About Phoenix FD PLUGIN FOR 3DS MAX AND MAYA. SIMULATING AND RENDERING BOTH LIQUIDS AND FIRE/SMOKE. USED IN MOVIES, GAMES AND COMMERCIALS.

Game Graphics & Real-time Rendering

Agenda. Ray(tracing(algorithm( Code(structure( Ray(intersec;ons Sphere Triangle Hints Transforma;ons Debugging How(to(start

SceneKit: What s New Session 604

Topic 11: Texture Mapping 11/13/2017. Texture sources: Solid textures. Texture sources: Synthesized

The Graphics Pipeline

CSE 167: Introduction to Computer Graphics Lecture #5: Rasterization. Jürgen P. Schulze, Ph.D. University of California, San Diego Fall Quarter 2015

Topic 11: Texture Mapping 10/21/2015. Photographs. Solid textures. Procedural

Applications of Explicit Early-Z Culling

The Application Stage. The Game Loop, Resource Management and Renderer Design

CS 4620 Midterm, March 21, 2017

Octree-Based Sparse Voxelization for Real-Time Global Illumination. Cyril Crassin NVIDIA Research

Ray Tracing. Computer Graphics CMU /15-662, Fall 2016

DEFERRED RENDERING STEFAN MÜLLER ARISONA, ETH ZURICH SMA/

Ray tracing. Computer Graphics COMP 770 (236) Spring Instructor: Brandon Lloyd 3/19/07 1

6.837 Introduction to Computer Graphics Quiz 2 Thursday November 20, :40-4pm One hand-written sheet of notes allowed

GPGPU Applications. for Hydrological and Atmospheric Simulations. and Visualizations on the Web. Ibrahim Demir

INFOGR Computer Graphics. J. Bikker - April-July Lecture 10: Ground Truth. Welcome!

Rendering Part I (Basics & Ray tracing) Lecture 25 December 1, 2015

CS4620/5620: Lecture 14 Pipeline

Computer Graphics. Lecture 13. Global Illumination 1: Ray Tracing and Radiosity. Taku Komura

Integrating Apps and Content with AR Quick Look

Real - Time Rendering. Pipeline optimization. Michal Červeňanský Juraj Starinský

Voxel Cone Tracing and Sparse Voxel Octree for Real-time Global Illumination. Cyril Crassin NVIDIA Research

CS GPU and GPGPU Programming Lecture 2: Introduction; GPU Architecture 1. Markus Hadwiger, KAUST

CS451Real-time Rendering Pipeline

Applications of Explicit Early-Z Z Culling. Jason Mitchell ATI Research

CMSC427 Advanced shading getting global illumination by local methods. Credit: slides Prof. Zwicker

Direct Rendering of Trimmed NURBS Surfaces

Recollection. Models Pixels. Model transformation Viewport transformation Clipping Rasterization Texturing + Lights & shadows

Graphics Hardware. Instructor Stephen J. Guy

Me Again! Peter Chapman. if it s important / time-sensitive

Ray Casting on Programmable Graphics Hardware. Martin Kraus PURPL group, Purdue University

Real-Time Non- Photorealistic Rendering

Ray Casting of Trimmed NURBS Surfaces on the GPU

TDA362/DIT223 Computer Graphics EXAM (Same exam for both CTH- and GU students)

Advanced Shading I: Shadow Rasterization Techniques

Photorealism: Ray Tracing

CIS 581 Interactive Computer Graphics

Building Visually Rich User Experiences

INTRODUCTION TO OPTIX. Martin Stich, Engineering Manager

CS354 Computer Graphics Ray Tracing. Qixing Huang Januray 24th 2017

Windowing System on a 3D Pipeline. February 2005

The Traditional Graphics Pipeline

Multi-View Soft Shadows. Louis Bavoil

Real-Time Shadows. Last Time? Textures can Alias. Schedule. Questions? Quiz 1: Tuesday October 26 th, in class (1 week from today!

CSE 167: Lecture #17: Volume Rendering. Jürgen P. Schulze, Ph.D. University of California, San Diego Fall Quarter 2012

Computer Graphics. Lecture 10. Global Illumination 1: Ray Tracing and Radiosity. Taku Komura 12/03/15

Final Project: Real-Time Global Illumination with Radiance Regression Functions

Particle systems, collision detection, and ray tracing. Computer Graphics CSE 167 Lecture 17

For Intuition about Scene Lighting. Today. Limitations of Planar Shadows. Cast Shadows on Planar Surfaces. Shadow/View Duality.

Interactive Methods in Scientific Visualization

INFOGR Computer Graphics

ARM Multimedia IP: working together to drive down system power and bandwidth

Chapter 7 - Light, Materials, Appearance

CS GAME PROGRAMMING Question bank

Computer Graphics. Lecture 14 Bump-mapping, Global Illumination (1)

Hardware Accelerated Volume Visualization. Leonid I. Dimitrov & Milos Sramek GMI Austrian Academy of Sciences

Rasterization Overview

V-Ray RT: A New Paradigm in Photorealistic Raytraced Rendering on NVIDIA GPUs. Vladimir Koylazov Chaos Software.

GPU Ray Tracing at the Desktop and in the Cloud. Phillip Miller, NVIDIA Ludwig von Reiche, mental images

Consider a partially transparent object that is illuminated with two lights, one visible from each side of the object. Start with a ray from the eye

Intro to Ray-Tracing & Ray-Surface Acceleration

Pipeline Operations. CS 4620 Lecture Steve Marschner. Cornell CS4620 Spring 2018 Lecture 11

Overview: Ray Tracing & The Perspective Projection Pipeline

SUMMARY. CS380: Introduction to Computer Graphics Ray tracing Chapter 20. Min H. Kim KAIST School of Computing 18/05/29. Modeling

A free open source modelling/rendering software

Transcription:

Session #WWDC18 Metal for Ray Tracing Acceleration 606 Sean James, GPU Software Engineer Wayne Lister, GPU Software Engineer 2018 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

Metal Performance Shaders GPU-accelerated primitives, optimized for ios and macos Image processing Linear algebra Machine learning inference Using Metal 2 for Compute WWDC 2017 What s New in Metal, Part 2 WWDC 2016

Metal Performance Shaders NEW GPU-accelerated primitives, optimized for ios and macos Image processing Linear algebra Machine learning inference and training Metal for Accelerating Machine Learning Hall 3 Thursday 4:00PM

Metal Performance Shaders NEW GPU-accelerated primitives, optimized for ios and macos Image processing Linear algebra Machine learning inference and training Ray tracing Metal for Accelerating Machine Learning Hall 3 Thursday 4:00PM

Ray Tracing Tracing a ray s path as it interacts with a scene

Ray Tracing Tracing a ray s path as it interacts with a scene

Ray Tracing Tracing a ray s path as it interacts with a scene Applications Rendering Audio and physics simulation Collision detection AI and pathfinding

Rasterization Projects triangles onto the screen one at a time

Rasterization Projects triangles onto the screen one at a time Fast method of choice for games and real-time applications

Rasterization Projects triangles onto the screen one at a time Fast method of choice for games and real-time applications Difficult to model behavior of light

Ray Tracing Reflections Can be computed accurately with ray tracing Amazon Lumberyard Bistro 2017 Amazon.com, https://developer.nvidia.com/orca/amazon-lumberyard-bistro Ray traced with Metal, https://creativecommons.org/licenses/by/4.0/

Ray Tracing Soft shadows Can be computed directly with ray tracing Realistic transition from hard to soft shadows

Ray Tracing Global illumination Naturally modeled with ray tracing Sponza Atrium 2009 Frank Meinl, Crytek, https://casual-effects.com/data/ Ray traced with Metal, https://creativecommons.org/licenses/by/3.0/

Ray Tracing Many other effects ambient occlusion, refraction, area lights, depth of field, motion blur

Ray Tracing Many other effects ambient occlusion, refraction, area lights, depth of field, motion blur Method of choice for photorealistic, offline rendering

Ray Tracing Many other effects ambient occlusion, refraction, area lights, depth of field, motion blur Method of choice for photorealistic, offline rendering Significantly more computationally expensive doing more work to simulate physical effects

Rendering with Ray Tracing Work backwards from camera to light source

Rendering with Ray Tracing Work backwards from camera to light source Cast primary rays from the camera into the scene

Rendering with Ray Tracing Work backwards from camera to light source Cast primary rays from the camera into the scene Compute shading at intersection points

Rendering with Ray Tracing Compute direct lighting contribution

Rendering with Ray Tracing Compute direct lighting contribution Cast shadow rays from intersection point to light source

Rendering with Ray Tracing Compute direct lighting contribution Cast shadow rays from intersection point to light source In shadow if shadow ray does not reach light

Rendering with Ray Tracing Cast secondary rays from intersection point in random directions

Rendering with Ray Tracing Cast secondary rays from intersection point in random directions

Rendering with Ray Tracing Cast secondary rays from intersection point in random directions

Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Add direct lighting reflected from secondary intersection point

Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Add direct lighting reflected from secondary intersection point

Rendering with Ray Tracing Cast secondary rays from intersection point in random directions Add direct lighting reflected from secondary intersection point Repeat to simulate light bouncing

Rendering with Ray Tracing Ray count grows exponentially with number of bounces

Rendering with Ray Tracing Ray count grows exponentially with number of bounces Avoid growth one shadow ray and secondary ray per bounce

Rendering with Ray Tracing Ray count grows exponentially with number of bounces Avoid growth one shadow ray and secondary ray per bounce

Rendering with Ray Tracing Ray count grows exponentially with number of bounces Avoid growth one shadow ray and secondary ray per bounce Average over multiple frames

Rendering with Ray Tracing

Rendering with Ray Tracing Generate Primary Rays

Rendering with Ray Tracing Primary Rays Generate Primary Rays Intersect with Scene

Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading

Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Shadow, Secondary Rays

Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Shadow, Secondary Rays

Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Shadow, Secondary Rays Image

Rendering with Ray Tracing Primary Rays Intersections Generate Primary Rays Intersect with Scene Shading Shadow, Secondary Rays Image

MPSRayIntersector NEW Ray intersector accelerates ray/triangle intersection tests on the GPU Intersector

MPSRayIntersector NEW Ray intersector accelerates ray/triangle intersection tests on the GPU Intersector Accepts batches of rays in a Metal buffer Ray Buffer

MPSRayIntersector NEW Ray intersector accelerates ray/triangle intersection tests on the GPU Intersector Accepts batches of rays in a Metal buffer Ray Buffer Intersection Buffer Returns one intersection per ray

MPSRayIntersector NEW Ray intersector accelerates ray/triangle intersection tests on the GPU Intersector Accepts batches of rays in a Metal buffer Ray Buffer Intersection Buffer Returns one intersection per ray Command Buffer Encodes into a Metal command buffer

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

Accelerating Ray/Scene Intersection

MPSRayIntersector Scene represented by triangle vertices in a vertex buffer Intersector Ray Buffer Intersection Buffer Vertex Buffer

MPSRayIntersector Scene represented by triangle vertices in a vertex buffer Intersector Build an acceleration structure over the vertex buffer Ray Buffer Intersection Buffer Acceleration Structure Vertex Buffer

MPSRayIntersector Scene represented by triangle vertices in a vertex buffer Intersector Build an acceleration structure over the vertex buffer Ray Buffer Intersection Buffer Acceleration Pass acceleration structure to intersector Structure Vertex Buffer

Ray Tracing with Metal Performance Shaders

Ray Tracing with Metal Performance Shaders Primary Rays and Shading

Ray Tracing with Metal Performance Shaders Primary Rays and Shading Shadow Rays

Ray Tracing with Metal Performance Shaders Primary Rays and Shading Shadow Rays Secondary Rays

Ray Tracing with Metal Performance Shaders Primary Rays and Shading Shadow Rays Secondary Rays

Primary Rays and Shading

Primary Rays and Shading Intersector

Primary Rays and Shading Intersector Acceleration Structure Vertex Buffer

Primary Rays and Shading Generate Primary Rays Intersector Ray Buffer Acceleration Structure Vertex Buffer

Primary Rays and Shading Generate Primary Rays Intersector Ray Buffer Intersection Buffer Acceleration Structure Vertex Buffer

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Creating an Intersector Create an MPSRayIntersector with a Metal device: let intersector = MPSRayIntersector(device: device)

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Creating an Acceleration Structure Create an MPSTriangleAccelerationStructure with a Metal device: let accelerationstructure = MPSTriangleAccelerationStructure(device: device)

Creating an Acceleration Structure Create an MPSTriangleAccelerationStructure with a Metal device: let accelerationstructure = MPSTriangleAccelerationStructure(device: device) Assign vertex buffer: accelerationstructure.vertexbuffer = vertexbuffer accelerationstructure.trianglecount = trianglecount

Creating an Acceleration Structure Create an MPSTriangleAccelerationStructure with a Metal device: let accelerationstructure = MPSTriangleAccelerationStructure(device: device) Assign vertex buffer: accelerationstructure.vertexbuffer = vertexbuffer accelerationstructure.trianglecount = trianglecount Build acceleration structure: accelerationstructure.rebuild()

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Generating Primary Rays Launch one thread per pixel

Generating Primary Rays Launch one thread per pixel Write Ray struct to ray buffer: struct Ray { float3 origin; float3 direction; };

Generating Primary Rays Launch one thread per pixel Write Ray struct to ray buffer: struct Ray { float3 origin; float3 direction; };

Generating Primary Rays Launch one thread per pixel Write Ray struct to ray buffer: struct Ray { float3 origin; float3 direction; }; Origin

Generating Primary Rays Launch one thread per pixel Write Ray struct to ray buffer: struct Ray { float3 origin; float3 direction; Direction }; Origin

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Finding Intersections with the Scene Encode intersection test into a command buffer: intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Intersections with the Scene Encode intersection test into a command buffer: intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Intersections with the Scene Encode intersection test into a command buffer: intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Intersections with the Scene Encode intersection test into a command buffer: intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Intersections with the Scene Encode intersection test into a command buffer: intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Intersections with the Scene Encode intersection test into a command buffer: intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Shading Apply lighting and textures similar to fragment shader Depends on intersection point and vertex attributes

Shading struct Intersection { float distance; int primitiveindex; float2 coordinates; }; Intersection point

Shading struct Intersection { float distance; int primitiveindex; float2 coordinates; }; Intersection point Direction Origin

Distance Shading struct Intersection { float distance; int primitiveindex; float2 coordinates; }; Intersection point Direction Origin

Distance Shading struct Intersection { float distance; int primitiveindex; float2 coordinates; }; Intersection point Direction Origin

Distance Shading struct Intersection { float distance; int primitiveindex; float2 coordinates; }; Intersection point Direction Origin

<latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafoazamzk04yzvbbzkwqlny <latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbbu0ik0horevhygv2 u<latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafo <latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbb Shading struct Intersection { float distance; int primitiveindex; v<latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbb float2 coordinates; }; Barycentric coordinates u<latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafoazamzk04yzvbbzkwqlny and v<latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbbu0ik0horevhygv2

<latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafoazamzk04yzvbbzkwqlny <latexit sha1_base64="tasnsc1efest/gvdsvyv3+hx72e=">aaab9hicbvbns8naej3ur1q/qh69lbbbiygrqutbkhrxwmf+qbvkzrtpl242cxdtkaw/w4shrbz6y7z5b9ymedt6yodx3gwz8/yym6ud59pkrayurw/knwtb2zu7e8x9g6akeklog0q8km0fk8qzoa3nnkftwfic+py2/nh13g+nqvqsend6elmvxapbakawnpl3gc6ri85qymrck5yc+6jarrov5npoimvsggz1xvgj249ielkhccdkdvwn1t4us80ip7ncn1e0xmseb7rjqmahvd40pxqgtozsr0ektqmnuvxnxbshsk1c33sgwa/vx28u/ud1eh1uvsktcakpiitfqckrjta8adrnkhlnj4zgipm5fzehlphok1mhdsf7ezl8h9a8t13hdm/lpdpvfkcejuaytsgfcttgburqaal38ajp8gknrsfr1xpbtoasboyqfsf6/wkgp5ak</latexit> <latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbbu0ik0horevhygv2 u<latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafo <latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbb w<latexit sha1_base64="0esugjwib4mzf14746qxlnjzwd4=">aaab6hicbvbns8naej3ur1q/qh69lbbbu0ik0horevhygv2anptndtou3wzc7kypob/aiwdfvpqtvp Shading struct Intersection { float distance; int primitiveindex; v<latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbb float2 coordinates; }; Barycentric coordinates u<latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafoazamzk04yzvbbzkwqlny and v<latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbbu0ik0horevhygv2 Third coordinate: w =1 u v

<latexit sha1_base64="mzz9fjlkacbl/xz0lhwkxqawcnw=">aaab/nicbvdlssnafl3xwesrkq7cdbzbeepscq27ohuxfewd2ham00k7dpjgzlipoecvuhghifu/w51/4zsnonydwz2ccy/3zvfizqsyre9jzxvtfwozsfxc3tnd2zcpdtsysgshlrlxshq9lclniw0ppjjtxoliwoo0442v535nqovkuxinpjf1ajwmmc8ivlpyzemetv0lxacjrrau97pwxlnkls/r1bpdq3bzyrbmspcj6zof/ufekocginaszc+2yuwkwchgoj0v+4mkmszjpkq9tumcuomk2fkzdkavafijov+oukb+nehxiou08hrngnvi/vxm4n9el1f+3ulzgcekhmsxye84uhgaz4egtfci+fqttatttyiywgitprmrzihkx14m3yg0k2xbktu31vljko+jacdwcudgqw0acannaagbfb7hgv6mb+pjedxefq0rrj5zbl9gvh8b/vys6w==</latexit> <latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafoazamzk04yzvbbzkwqlny <latexit sha1_base64="tasnsc1efest/gvdsvyv3+hx72e=">aaab9hicbvbns8naej3ur1q/qh69lbbbiygrqutbkhrxwmf+qbvkzrtpl242cxdtkaw/w4shrbz6y7z5b9ymedt6yodx3gwz8/yym6ud59pkrayurw/knwtb2zu7e8x9g6akeklog0q8km0fk8qzoa3nnkftwfic+py2/nh13g+nqvqsend6elmvxapbakawnpl3gc6ri85qymrck5yc+6jarrov5npoimvsggz1xvgj249ielkhccdkdvwn1t4us80ip7ncn1e0xmseb7rjqmahvd40pxqgtozsr0ektqmnuvxnxbshsk1c33sgwa/vx28u/ud1eh1uvsktcakpiitfqckrjta8adrnkhlnj4zgipm5fzehlphok1mhdsf7ezl8h9a8t13hdm/lpdpvfkcejuaytsgfcttgburqaal38ajp8gknrsfr1xpbtoasboyqfsf6/wkgp5ak</latexit> <latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbbu0ik0horevhygv2 <latexit sha1_base64="bjq581p2y2ddiiircxztkzqijgu=">aaab6nicbvbns8naej3ur1q/qh69lbbbu0ik0hgrevfy0x5ag8pmu2mxbjzhdyou0j/gxymix <latexit sha1_base64="pqkrp9yarzezhzuqjypjliru/py=">aaab6nicbvbns8naej3ur1q/qh69lbbbu0ik0hgrevfy0x5ag8pmu2mxbjzhdyou0j/gxymixv1f3v u<latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafo <latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbb w<latexit sha1_base64="0esugjwib4mzf14746qxlnjzwd4=">aaab6hicbvbns8naej3ur1q/qh69lbbbu0ik0horevhygv2anptndtou3wzc7kypob/aiwdfvpqtvp <latexit sha1_base64="mccqtvj35ad8cqgqotebtfqcs3y=">aaab6nicbvbns8naej3ur1o/wvxozbeinkjscq23ohepfe0htkfstpt26wytdjdccf0jxjwo4tvf5m Shading V 1 struct Intersection { float distance; int primitiveindex; v<latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbb float2 coordinates; }; Barycentric coordinates u<latexit sha1_base64="5en9wswlw/z88jnn61xxnby8qog=">aaab6hicbvdlsgnbeoynrxhfuy9ebopgadkvifew9oixafoazamzk04yzvbbzkwqlny and v<latexit sha1_base64="wlcyvdhfvigotgafrxemxnak0f0=">aaab6hicbvbns8naen3ur1q/qh69lbbbu0ik0horevhygv2 V 0 V 2 Third coordinate: w =1 u v Interpolated: uv 0 + vv 1 + wv 2

Primary Rays and Shading Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image Vertex Buffer

Primary Rays and Shading Shadow Rays Secondary Rays

Primary Rays and Shading Shadow Rays Secondary Rays

Shadow Rays Check if shading point is in shadow before adding color to image

Shadow Rays Check if shading point is in shadow before adding color to image Cast a ray from shading point to light source

Shadow Rays Check if shading point is in shadow before adding color to image Cast a ray from shading point to light source If shadow ray does not reach light, shading point is in shadow

Shadow Rays Check if shading point is in shadow before adding color to image Cast a ray from shading point to light source If shadow ray does not reach light, shading point is in shadow

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Acceleration Structure Image

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Shadow Ray Buffer

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Shadow Ray Buffer Intersection Buffer

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Shadow Rays versus Primary Rays

Shadow Rays versus Primary Rays Maximum intersection distance Maximum Distance

Shadow Rays versus Primary Rays Maximum intersection distance Don t need triangle index or barycentric coordinates Maximum Distance

Shadow Rays versus Primary Rays Maximum intersection distance Don t need triangle index or barycentric coordinates Maximum Distance Propagate color from shading kernel to final kernel

Customizing the Ray Struct Ray type is configurable struct Ray { packed_float3 origin; float mindistance; packed_float3 direction; float maxdistance; float3 color; };

Customizing the Ray Struct Ray type is configurable struct Ray { packed_float3 origin; Choose what data is provided to the intersector float mindistance; packed_float3 direction; float maxdistance; float3 color; };

Customizing the Ray Struct Ray type is configurable struct Ray { packed_float3 origin; Choose what data is provided to the intersector float mindistance; packed_float3 direction; Append app-specific data float maxdistance; float3 color; };

Customizing the Ray Struct Ray type is configurable struct Ray { packed_float3 origin; Choose what data is provided to the intersector float mindistance; packed_float3 direction; Append app-specific data float maxdistance; float3 color; }; Configure MPSRayIntersector for new Ray struct type: intersector.raydatatype =.originmindistancedirectionmaxdistance intersector.raystride = MemoryLayout<Ray>.stride

Customizing the Ray Struct Ray type is configurable struct Ray { packed_float3 origin; Choose what data is provided to the intersector float mindistance; packed_float3 direction; Append app-specific data float maxdistance; float3 color; }; Configure MPSRayIntersector for new Ray struct type: intersector.raydatatype =.originmindistancedirectionmaxdistance intersector.raystride = MemoryLayout<Ray>.stride

Customizing the Ray Struct Ray type is configurable struct Ray { packed_float3 origin; Choose what data is provided to the intersector float mindistance; packed_float3 direction; Append app-specific data float maxdistance; float3 color; }; Configure MPSRayIntersector for new Ray struct type: intersector.raydatatype =.originmindistancedirectionmaxdistance intersector.raystride = MemoryLayout<Ray>.stride

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Finding Shadow Ray Intersections Modify call to MPSRayIntersector: intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Shadow Ray Intersections Modify call to MPSRayIntersector: intersector.intersectiondatatype =.distance intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Shadow Ray Intersections Modify call to MPSRayIntersector: intersector.intersectiondatatype =.distance intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Shadow Ray Intersections Modify call to MPSRayIntersector: intersector.intersectiondatatype =.distance intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.nearest, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Finding Shadow Ray Intersections Modify call to MPSRayIntersector: intersector.intersectiondatatype =.distance intersector.encodeintersection(commandbuffer: commandbuffer, intersectiontype:.any, raybuffer: raybuffer, raybufferoffset: 0, intersectionbuffer: intersectionbuffer, intersectionbufferoffset: 0, raycount: raycount, accelerationstructure: accelerationstructure)

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Shadow Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Adding Ray Color to the Image Shadow Ray Buffer Intersection Buffer

Adding Ray Color to the Image Shadow Ray struct Ray { packed_float3 origin; float mindistance; packed_float3 direction; float maxdistance; Buffer float3 color; }; struct ShadowIntersection { }; float distance; Intersection Buffer

Adding Ray Color to the Image struct Ray { packed_float3 origin; float mindistance; packed_float3 direction; Positive float maxdistance; Shadow Ray Buffer float3 color; Yes }; struct ShadowIntersection { In Shadow float distance; }; Intersection Buffer

Adding Ray Color to the Image struct Ray { packed_float3 origin; Shadow Ray float mindistance; packed_float3 direction; float maxdistance; Positive No Not in Shadow, Add Color to Image Buffer float3 color; Yes }; struct ShadowIntersection { In Shadow float distance; }; Intersection Buffer

Adding Ray Color to the Image struct Ray { packed_float3 origin; Shadow Ray float mindistance; packed_float3 direction; float maxdistance; Positive No Not in Shadow, Add Color to Image Buffer float3 color; Yes }; struct ShadowIntersection { In Shadow Image float distance; }; Intersection Buffer

Primary Rays and Shading Shadow Rays Secondary Rays

Primary Rays and Shading Shadow Rays Secondary Rays

Secondary Rays Simulate light bouncing around the scene

Secondary Rays Simulate light bouncing around the scene Iterate, continuing in random direction at each step

Secondary Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Secondary Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Secondary Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Secondary Rays Generate Primary Rays Intersector Shading Ray Buffer Intersection Buffer Intersector Add Color to Image Shadow Ray Buffer Intersection Buffer Image

Updating Secondary Rays Update secondary rays during each iteration: ray.origin = intersectionpoint; ray.direction = getrandomdirection(surfacenormal); ray.color *= surfacecolor;

Updating Secondary Rays Update secondary rays during each iteration: ray.origin = intersectionpoint; ray.direction = getrandomdirection(surfacenormal); ray.color *= surfacecolor;

Updating Secondary Rays Update secondary rays during each iteration: ray.origin = intersectionpoint; ray.direction = getrandomdirection(surfacenormal); ray.color *= surfacecolor;

Updating Secondary Rays Update secondary rays during each iteration: ray.origin = intersectionpoint; ray.direction = getrandomdirection(surfacenormal); ray.color *= surfacecolor;

Primary Rays and Shading Shadow Rays Secondary Rays

Primary Rays and Shading Shadow Rays Secondary Rays

Demo

Sample Code This app is available as a sample Add your own geometry, light sources, camera model, shading model Refer to documentation for more information Cornell Box 2009 Morgan McGuire, https://casual-effects.com/data Ray traced with Metal, https://creativecommons.org/licenses/by/3.0/

Extending to Multiple GPUs Wayne Lister, GPU Software Engineer

Multi-GPU Ray Tracing Split work across GPUs

Multi-GPU Ray Tracing Split work across GPUs Copy data between GPUs

Multi-GPU Ray Tracing Split work across GPUs Copy data between GPUs Synchronize execution

Splitting Work Across GPUs Rendering

Splitting Work Across GPUs GPU 0 GPU 1 GPU 2 Rendering

Splitting Work Across GPUs GPU 0 GPU 1 GPU 2 Rendering Composition

Splitting Work Across GPUs Acceleration GPU 0 Structure Vertex Buffer GPU 1 GPU 2 Scene Data Rendering Composition

Splitting Work Across GPUs Acceleration Acceleration Structure Acceleration Structure Structure GPU 0 Vertex Buffer Vertex Buffer Vertex Buffer GPU 1 GPU 2 Scene Data Rendering Composition

Splitting Work Across GPUs Acceleration Acceleration Structure Acceleration Structure Structure GPU 0 Vertex Buffer Vertex Buffer Vertex Buffer GPU 1 GPU 2 Scene Data Rendering Composition

Copying Scene Data let copy = accelerationstructure.copy(with: nil, device: newdevice) copy.vertexbuffer = copybuffer(accelerationstructure.vertexbuffer, device: newdevice) Acceleration Structure Vertex Buffer GPU 0 GPU 1

Copying Scene Data let copy = accelerationstructure.copy(with: nil, device: newdevice) copy.vertexbuffer = copybuffer(accelerationstructure.vertexbuffer, device: newdevice) Acceleration Structure Acceleration Structure Vertex Buffer GPU 0 GPU 1

Copying Scene Data let copy = accelerationstructure.copy(with: nil, device: newdevice) copy.vertexbuffer = copybuffer(accelerationstructure.vertexbuffer, device: newdevice) Acceleration Structure Acceleration Structure Vertex Buffer Vertex Buffer GPU 0 GPU 1

Splitting Work Across GPUs Acceleration Acceleration Structure Acceleration Structure Structure GPU 0 Vertex Buffer Vertex Buffer Vertex Buffer GPU 1 GPU 2 Scene Data Rendering Composition

Splitting Work Across GPUs Acceleration Acceleration Structure Acceleration Structure Structure GPU 0 Vertex Buffer Vertex Buffer Vertex Buffer GPU 1 GPU 2 Scene Data Rendering Composition

Splitting Work Across GPUs Acceleration Acceleration Structure Acceleration Structure Structure GPU 0 Vertex Buffer Vertex Buffer Vertex Buffer GPU 1 GPU 2 Scene Data Rendering Composition

Creating a Shared CPU Allocation Device A Device B MTLBuffer (Private) MTLBuffer (Private)

Creating a Shared CPU Allocation Device A Device B MTLBuffer (Private) CPU Allocation MTLBuffer (Private)

Creating a Shared CPU Allocation Device A Device B MTLBuffer (Private) CPU Allocation MTLBuffer (Private)

Creating a Shared CPU Allocation Device A Device A Device B Device B MTLBuffer MTLBuffer CPU MTLBuffer MTLBuffer (Private) (Shared) Allocation (Shared) (Private)

Creating a Shared CPU Allocation Device A Device A Device B Device B Blit Blit MTLBuffer MTLBuffer CPU MTLBuffer MTLBuffer (Private) (Shared) Allocation (Shared) (Private)

Creating a Shared CPU Allocation let buffera = devicea.makebuffer(length: bufferlength, options:.shared)! let bufferb = deviceb.makebuffer(bytesnocopy: buffera.contents(), length: buffera.length, options:.shared deallocator: nil)!

Creating a Shared CPU Allocation let buffera = devicea.makebuffer(length: bufferlength, options:.shared)! let bufferb = deviceb.makebuffer(bytesnocopy: buffera.contents(), length: buffera.length, options:.shared deallocator: nil)!

Creating a Shared CPU Allocation let buffera = devicea.makebuffer(length: bufferlength, options:.shared)! let bufferb = deviceb.makebuffer(bytesnocopy: buffera.contents(), length: buffera.length, options:.shared deallocator: nil)!

Synchronizing with Metal Events GPU A Render region Blit region to shared buffer GPU B Render region Blit region from shared buffer Execution Timeline

Synchronizing with Metal Events GPU A Render region Blit region to shared buffer GPU B Render region Blit region from shared buffer Execution Timeline

Synchronizing with Metal Events GPU A Render region Blit region to shared buffer GPU B Render region Blit region from shared buffer Execution Timeline

Synchronizing with Metal Events GPU A Render region Blit region to shared buffer GPU B Render region Blit region from shared buffer Execution Timeline

Synchronizing with Metal Events GPU A Render region Blit region to shared buffer GPU B Render region Blit region from shared buffer Execution Timeline

Synchronizing with Metal Events GPU A Render region Blit region to shared buffer GPU B Render region Blit region from shared buffer Execution Timeline

Synchronizing with Metal Events GPU A Render region Blit region to shared buffer GPU B Render region Blit region from shared buffer Execution Timeline

Synchronizing with Metal Events NEW GPU A Render region Blit region to shared buffer [commandbuffer encodewaitforevent:value] GPU B Render region Wait Blit region from shared buffer Execution Timeline

Synchronizing with Metal Events NEW [commandbuffer encodesignalevent:value] GPU A Render region Blit region to shared buffer [commandbuffer encodewaitforevent:value] GPU B Render region Wait Blit region from shared buffer Execution Timeline

Load Balancing GPUs can have different performance GPU 0 GPU 1 GPU 2 Fixed Partitions

Load Balancing GPUs can have different performance GPU 0 Some regions are more complex to render GPU 1 GPU 2 Fixed Partitions

Load Balancing Adjust region sizes to keep GPUs fully utilized GPU 0 GPU 1 GPU 2 Fixed Partitions

Load Balancing Adjust region sizes to keep GPUs fully utilized GPU 0 GPU 1 GPU 2 Adaptive Partitions

Timing GPU Work commandbuffer.addcompletedhandler { commandbuffer in } completiontime[commandbuffer] = mach_absolute_time() Duration to Measure Command Command Command Command Buffer 0 Buffer 1 Buffer 2 Buffer 3 Execution Timeline

Demo

Summary Accelerates ray/triangle intersection on the GPU

Summary Accelerates ray/triangle intersection on the GPU Optimized for macos and ios

Summary Accelerates ray/triangle intersection on the GPU Optimized for macos and ios Scales across multiple GPUs

More Information https://developer.apple.com/wwdc18/606 Metal for Ray Tracing Acceleration Lab Technology Lab 6 Thursday 12:00 PM