What s New in Metal, Part 2

Transcription

1 Graphics and Games #WWDC15 What s New in Metal, Part 2 Session 607 Dan Omachi GPU Software Frameworks Engineer Anna Tikhonova GPU Software Frameworks Engineer 2015 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

2

3

4

5 MetalKit Utility functionality for Metal Apps

6 MetalKit

7 MetalKit MetalKit provides efficient implementations for commonly used scenarios Less effort to get up and rendering Increased performance and stability

8 MetalKit Overview

9 MetalKit Overview MetalKit View Unified view class for rendering Metal scenes

10 MetalKit Overview MetalKit View Unified view class for rendering Metal scenes Texture Loader Metal texture object creation from image files

11 MetalKit Overview MetalKit View Unified view class for rendering Metal scenes Texture Loader Metal texture object creation from image files Model I/O Integration Load and manage mesh data for Metal rendering

12 MetalKit View Overview Simplest way to get Metal rendering on screen

13 MetalKit View Overview Simplest way to get Metal rendering on screen Unified between ios and OS X Subclass of UIView for ios Subclass of NSView for OS X

14 MetalKit View Overview Simplest way to get Metal rendering on screen Unified between ios and OS X Subclass of UIView for ios Subclass of NSView for OS X Manages render targets Creates render pass descriptors

15 MetalKit View Overview Simplest way to get Metal rendering on screen Unified between ios and OS X Subclass of UIView for ios Subclass of NSView for OS X Manages render targets Creates render pass descriptors Multiple draw loop modes supported Timer, event, or explicitly driven draw loop

16 MetalKit View Setup Approaches to using MTKView

17 MetalKit View Setup Approaches to using MTKView A. Implement a delegate - (void)drawinview:(mtkview *)view - (void)view:(mtkview *)view willlayoutwithsize:(cgsize)size

18 MetalKit View Setup Approaches to using MTKView A. Implement a delegate - (void)drawinview:(mtkview *)view - (void)view:(mtkview *)view willlayoutwithsize:(cgsize)size

19 MetalKit View Setup Approaches to using MTKView A. Implement a delegate - (void)drawinview:(mtkview *)view - (void)view:(mtkview *)view willlayoutwithsize:(cgsize)size B. Subclass MTKView ios OS X - (void)drawrect:(cgrect)rect - (void)layoutsubviews - (void)drawrect:(cgrect)rect - (void)setframesize:(nssize)newsize

20 MetalKit View Setup Approaches to using MTKView A. Implement a delegate - (void)drawinview:(mtkview *)view - (void)view:(mtkview *)view willlayoutwithsize:(cgsize)size B. Subclass MTKView ios OS X - (void)drawrect:(cgrect)rect - (void)layoutsubviews - (void)drawrect:(cgrect)rect - (void)setframesize:(nssize)newsize

21 MetalKit View Setup Approaches to using MTKView A. Implement a delegate - (void)drawinview:(mtkview *)view - (void)view:(mtkview *)view willlayoutwithsize:(cgsize)size B. Subclass MTKView ios OS X - (void)drawrect:(cgrect)rect - (void)layoutsubviews - (void)drawrect:(cgrect)rect - (void)setframesize:(nssize)newsize

22 MetalKit View Setup Initializing view properties - (void)viewdidload { MTKView *view = (MTKView *)self.view; view.delegate = self; view.device = device; view.colorpixelformat = MTLPixelFormatBGRA8Unorm_sRGB; view.depthstencilpixelformat = MTLPixelFormatDepth32Float_Stencil8; view.samplecount = 4; view.clearcolor = MTLClearColorMake(0.8, 0.8, 0.8, 1.0); }

23 MetalKit View Setup Initializing view properties - (void)viewdidload { MTKView *view = (MTKView *)self.view; view.delegate = self; view.device = device; view.colorpixelformat = MTLPixelFormatBGRA8Unorm_sRGB; view.depthstencilpixelformat = MTLPixelFormatDepth32Float_Stencil8; view.samplecount = 4; view.clearcolor = MTLClearColorMake(0.8, 0.8, 0.8, 1.0); }

24 MetalKit View Setup Initializing view properties - (void)viewdidload { MTKView *view = (MTKView *)self.view; view.delegate = self; view.device = device; view.colorpixelformat = MTLPixelFormatBGRA8Unorm_sRGB; view.depthstencilpixelformat = MTLPixelFormatDepth32Float_Stencil8; view.samplecount = 4; view.clearcolor = MTLClearColorMake(0.8, 0.8, 0.8, 1.0); }

25 MetalKit View Setup Initializing view properties - (void)viewdidload { MTKView *view = (MTKView *)self.view; view.delegate = self; view.device = device; view.colorpixelformat = MTLPixelFormatBGRA8Unorm_sRGB; view.depthstencilpixelformat = MTLPixelFormatDepth32Float_Stencil8; view.samplecount = 4; view.clearcolor = MTLClearColorMake(0.8, 0.8, 0.8, 1.0); }

26 MetalKit View Setup Initializing view properties - (void)viewdidload { MTKView *view = (MTKView *)self.view; view.delegate = self; view.device = device; view.colorpixelformat = MTLPixelFormatBGRA8Unorm_sRGB; view.depthstencilpixelformat = MTLPixelFormatDepth32Float_Stencil8; view.samplecount = 4; view.clearcolor = MTLClearColorMake(0.8, 0.8, 0.8, 1.0); }

27 MetalKit View Setup Initializing view properties - (void)viewdidload { MTKView *view = (MTKView *)self.view; view.delegate = self; view.device = device; view.colorpixelformat = MTLPixelFormatBGRA8Unorm_sRGB; view.depthstencilpixelformat = MTLPixelFormatDepth32Float_Stencil8; view.samplecount = 4; view.clearcolor = MTLClearColorMake(0.8, 0.8, 0.8, 1.0); }

28 MetalKit View Setup Initializing view properties - (void)viewdidload { MTKView *view = (MTKView *)self.view; view.delegate = self; view.device = device; view.colorpixelformat = MTLPixelFormatBGRA8Unorm_sRGB; view.depthstencilpixelformat = MTLPixelFormatDepth32Float_Stencil8; view.samplecount = 4; view.clearcolor = MTLClearColorMake(0.8, 0.8, 0.8, 1.0); }

29 MetalKit View Drawing Simple usage - (void)drawinview:(nonnull MTKView *)view { id <MTLRenderPassDescriptor> descriptor = view.currentrenderpassdescriptor; // Create render command encoder and encode final pass... } [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

30 MetalKit View Drawing Simple usage - (void)drawinview:(nonnull MTKView *)view { id <MTLRenderPassDescriptor> descriptor = view.currentrenderpassdescriptor; // Create render command encoder and encode final pass... } [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

31 MetalKit View Drawing Simple usage - (void)drawinview:(nonnull MTKView *)view { id <MTLRenderPassDescriptor> descriptor = view.currentrenderpassdescriptor; // Create render command encoder and encode final pass... } [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

32 MetalKit View Drawing Simple usage - (void)drawinview:(nonnull MTKView *)view { id <MTLRenderPassDescriptor> descriptor = view.currentrenderpassdescriptor; // Create render command encoder and encode final pass... } [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

33 Managing Drawables

34 Managing Drawables Limited pool of drawables

35 Managing Drawables Limited pool of drawables Drawables concurrently used in many stages of the display pipeline

36 Managing Drawables Limited pool of drawables Drawables concurrently used in many stages of the display pipeline Frame 1 Application

37 Managing Drawables Limited pool of drawables Drawables concurrently used in many stages of the display pipeline Frame 1 Frame 2 Application GPU

38 Managing Drawables Limited pool of drawables Drawables concurrently used in many stages of the display pipeline Frame 1 Frame 2 Frame 3 Application GPU Display

39 Managing Drawables Limited pool of drawables Drawables concurrently used in many stages of the display pipeline Frame 1 Frame 2 Frame 3 Frame 4 Application GPU Display

40 Application Frame

41 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable

42 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable

43 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

44 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

45 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

46 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

47 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

48 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

49 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

50 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

51 Application Frame [MTKView currentrenderpassdescriptor] Reserves a drawable Encodes to the drawable [MTLCommandBuffer presentdrawable:] [MTLCommandBuffer commit] Releases the drawable

52 MetalKit View Drawing Efficient usage - (void)drawinview:(nonnull MTKView *)view { // Update app s render state and encode offscreen passes... id <MTLRenderPassDescriptor> descriptor = view.currentrenderpassdescriptor; // Create render command encoder and encode final pass... } [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

53 MetalKit View Drawing Efficient usage - (void)drawinview:(nonnull MTKView *)view { // Update app s render state and encode offscreen passes... id <MTLRenderPassDescriptor> descriptor = view.currentrenderpassdescriptor; // Create render command encoder and encode final pass... } [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

54 MetalKit View Drawing Efficient usage - (void)drawinview:(nonnull MTKView *)view { // Update app s render state and encode offscreen passes... id <MTLRenderPassDescriptor> descriptor = view.currentrenderpassdescriptor; // Create render command encoder and encode final pass... } [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

55 MetalKit View Drawing Efficient usage - (void)drawinview:(nonnull MTKView *)view { // Update app s render state and encode offscreen passes... id <MTLRenderPassDescriptor> descriptor = view.currentrenderpassdescriptor; // Create render command encoder and encode final pass... [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit]; }

56 MetalKit Texture Loader

57 MetalKit Texture Loader Texture loading made simple Give a reference, get a Metal texture

58 MetalKit Texture Loader Texture loading made simple Give a reference, get a Metal texture Fast and fully featured Asynchronously decodes files and creates textures Support for common image file formats including JPG, TIFF, and PNG Also support for PVR and KTX texture file formats

59 Texture Loader Basic usage 1. Initialize with Metal device MTKTextureLoader *textureloader = [[MTKTextureLoader alloc] initwithdevice:device]; 2. Load texture [textureloader texturewithcontentsofurl:fileurl options:nil completionhandler:mycompletionhandler];

60 Texture Loader Basic usage 1. Initialize with Metal device MTKTextureLoader *textureloader = [[MTKTextureLoader alloc] initwithdevice:device]; 2. Load texture [textureloader texturewithcontentsofurl:fileurl options:nil completionhandler:mycompletionhandler];

61 Texture Loader Basic usage 1. Initialize with Metal device MTKTextureLoader *textureloader = [[MTKTextureLoader alloc] initwithdevice:device]; 2. Load texture [textureloader texturewithcontentsofurl:fileurl options:nil completionhandler:mycompletionhandler];

62 Texture Loader Basic usage 1. Initialize with Metal device MTKTextureLoader *textureloader = [[MTKTextureLoader alloc] initwithdevice:device]; 2. Load texture [textureloader texturewithcontentsofurl:fileurl fileurl options:nil completionhandler:mycompletionhandler];

63 Texture Loader Basic usage 1. Initialize with Metal device MTKTextureLoader *textureloader = [[MTKTextureLoader alloc] initwithdevice:device]; 2. Load texture [textureloader texturewithcontentsofurl:fileurl options:nil nil completionhandler:mycompletionhandler];

64 Texture Loader Basic usage 1. Initialize with Metal device MTKTextureLoader *textureloader = 2. Load texture [[MTKTextureLoader alloc] initwithdevice:device]; [textureloader texturewithcontentsofurl:fileurl options:nil completionhandler:mycompletionhandler]; mycompletionhandler

65 Model I/O Review Model I/O introduced in ios 9 and OS X El Capitan

66 Model I/O Review Model I/O introduced in ios 9 and OS X El Capitan 3D Asset loading from various file formats Importers and exporters for proprietary formats possible

67 Model I/O Review Model I/O introduced in ios 9 and OS X El Capitan 3D Asset loading from various file formats Importers and exporters for proprietary formats possible Offline baking operations Static ambient occlusion Light map generation Voxelization of meshes

68 Model I/O Review Model I/O introduced in ios 9 and OS X El Capitan 3D Asset loading from various file formats Importers and exporters for proprietary formats possible Offline baking operations Static ambient occlusion Light map generation Voxelization of meshes Allows you to focus on your rendering code

69 MetalKit and Model I/O Utilities to efficiently use Model I/O with Metal Optimized loading of Model I/O meshes into Metal buffers Encapsulation of mesh data for Metal Functions to prepare mesh data for Metal pipelines

70 Rendering a Model I/O Asset

71 Rendering a Model I/O Asset

72 Rendering a Model I/O Asset Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing

73 Rendering a Model I/O Asset Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing

74 Rendering a Model I/O Asset Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing

75 Rendering a Model I/O Asset Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing

76 Rendering a Model I/O Asset Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing

77 Setup for Model Rendering Pipeline setup Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing

78 Setup for Model Rendering Pipeline setup Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing

79 Vertex Shader Setting up Per-Vertex Inputs struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; vertex VertexOutput vertexfunction(vertexinput current [[ stage_in ]]) {... }

80 Vertex Shader Setting up Per-Vertex Inputs struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; vertex VertexOutput vertexfunction(vertexinput current [[ stage_in ]]) {... }

81 Vertex Shader Setting up Per-Vertex Inputs struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; vertex VertexOutput vertexfunction(vertexinput current [[ stage_in ]]) {... }

82 Vertex Shader Setting up Per-Vertex Inputs struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; vertex VertexOutput vertexfunction(vertexinput current [[ stage_in ]]) {... }

83 Vertex Shader Setting up Per-Vertex Inputs struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; vertex VertexOutput vertexfunction(vertexinput current [[ stage_in ]]) {... }

84 struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; };

85 struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; };

86 struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; };

87 struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; };

88 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; MTLVertexDescriptor *metalvertexdesc = [MTLVertexDescriptor new];

89 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[0].format = MTLVertexFormatFloat3; metalvertexdesc.attributes[0].offset = 0;

90 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[0].format = MTLVertexFormatFloat3; metalvertexdesc.attributes[0].offset = 0; 0 : position Float3

91 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[1].format = MTLVertexFormatUChar4Normalized; metalvertexdesc.attributes[1].offset = 12; 0 : position Float3

92 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[1].format = MTLVertexFormatUChar4Normalized; metalvertexdesc.attributes[1].offset = 12; 0 : position Float3 1 : color UChar4

93 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[1].format = MTLVertexFormatUChar4Normalized; metalvertexdesc.attributes[1].offset = 12; 0 : position Float3 1 : color UChar4

94 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[1].format = MTLVertexFormatUChar4Normalized; metalvertexdesc.attributes[1].offset = 12; 0 : position Float3 1 : color UChar4 12

95 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[2].format = MTLVertexFormatHalfFloat2; metalvertexdesc.attributes[2].offset = 16; 0 : position Float3 1 : color UChar4

96 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[2].format = MTLVertexFormatHalfFloat2; metalvertexdesc.attributes[2].offset = 16; 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2

97 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[2].format = MTLVertexFormatHalfFloat2; metalvertexdesc.attributes[2].offset = 16; 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2

98 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.attributes[2].format = MTLVertexFormatHalfFloat2; metalvertexdesc.attributes[2].offset = 16; 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2 16

99 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.layouts[0].stride = 20; 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2

100 Vertex Descriptor struct VertexInput { float3 position [[ attribute(0) ]]; float4 color [[ attribute(1) ]]; float2 texuv [[ attribute(2) ]]; }; metalvertexdesc.layouts[0].stride = 20; 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2 20

101 Vertex Array Layout 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2

102 Vertex Array Layout 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2

103 Vertex Array Layout

104 Vertex Array Layout...

105 Building the Pipeline renderpipelinedescriptor.vertexdescriptor = metalvertexdesc; MTLRenderStatePipeline *pipeline = [device.newrenderpipelinestatewithdescriptor:renderpipelinedescriptor error:error];

106 Building the Pipeline renderpipelinedescriptor.vertexdescriptor = metalvertexdesc; MTLRenderStatePipeline *pipeline = [device.newrenderpipelinestatewithdescriptor:renderpipelinedescriptor error:error];

107 Building the Pipeline renderpipelinedescriptor.vertexdescriptor = metalvertexdesc; MTLRenderStatePipeline *pipeline = [device.newrenderpipelinestatewithdescriptor:renderpipelinedescriptor renderpipelinedescriptor error:error];

108 Setup for Model Rendering Asset initialization Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing MetalKit Mesh Buffer Allocator

109 Setup for Model Rendering Asset initialization Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing MetalKit Mesh Buffer Allocator

110 Setup for Model Rendering Asset initialization Vertex Descriptor Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing MetalKit Mesh Buffer Allocator

111 Setup for Model Rendering Asset initialization Vertex Descriptor Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing MetalKit Mesh Buffer Allocator

112 Asset Initialization Model I/O Vertex Descriptor vs. Metal Vertex Descriptor

113 Asset Initialization Model I/O Vertex Descriptor vs. Metal Vertex Descriptor Model I/O Vertex Descriptor Describes the layout of vertex attributes in a mesh

114 Asset Initialization Model I/O Vertex Descriptor vs. Metal Vertex Descriptor Model I/O Vertex Descriptor Describes the layout of vertex attributes in a mesh Metal Vertex Descriptor Describes the layout of vertex attribute inputs to a render state pipeline

115 Asset Initialization Model I/O Vertex Descriptor vs. Metal Vertex Descriptor Model I/O Vertex Descriptor Describes the layout of vertex attributes in a mesh Metal Vertex Descriptor Describes the layout of vertex attribute inputs to a render state pipeline Intentionally designed to look similar Both contain an array of attribute and buffer layout objects Simplifies translation from one type to other

116 Asset Initialization Defaults and layout efficiency Each attribute in a Model I/O Vertex Descriptor has an identifying string-based name Model I/O assigns a default name if one does not exist in the model file - Names texturecoordinate, color - Model I/O defines these with the string-based MDLVertexAttribute name constants

117 Asset Initialization Defaults and layout efficiency Each attribute in a Model I/O Vertex Descriptor has an identifying string-based name Model I/O assigns a default name if one does not exist in the model file - Names texturecoordinate, color - Model I/O defines these with the string-based MDLVertexAttribute name constants Custom MDLVertexDescriptor recommended By default, Model I/O loads vertices as high-precision floating point types Feed pipelines with the smallest type that meets your precision requirements Improves vertex bandwidth efficiency

118 Asset Initialization 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2

119 Asset Initialization 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2 MDLVertexDescriptor *modeliovertexdesc = MTKModelIOVertexFormatFromMetal(metalVertexDesc); modeliovertexdesc.attributes[0].name = MDLVertexAttributePosition; modeliovertexdesc.attributes[1].name = MDLVertexAttributeColor; modeliovertexdesc.attributes[2].name = MDLVertexAttributeTextureCoordinate;

120 Asset Initialization 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2 MDLVertexDescriptor *modeliovertexdesc = MTKModelIOVertexFormatFromMetal(metalVertexDesc); modeliovertexdesc.attributes[0].name = MDLVertexAttributePosition; modeliovertexdesc.attributes[1].name = MDLVertexAttributeColor; modeliovertexdesc.attributes[2].name = MDLVertexAttributeTextureCoordinate;

121 Asset Initialization 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2 MDLVertexDescriptor *modeliovertexdesc = MTKModelIOVertexFormatFromMetal(metalVertexDesc); metalvertexdesc modeliovertexdesc.attributes[0].name = MDLVertexAttributePosition; modeliovertexdesc.attributes[1].name = MDLVertexAttributeColor; modeliovertexdesc.attributes[2].name = MDLVertexAttributeTextureCoordinate;

122 Asset Initialization 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2 MDLVertexDescriptor *modeliovertexdesc = MTKModelIOVertexFormatFromMetal(metalVertexDesc); modeliovertexdesc.attributes[0].name = MDLVertexAttributePosition; modeliovertexdesc.attributes[1].name = MDLVertexAttributeColor; modeliovertexdesc.attributes[2].name = MDLVertexAttributeTextureCoordinate;

123 Asset Initialization 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2 MDLVertexDescriptor *modeliovertexdesc = MTKModelIOVertexFormatFromMetal(metalVertexDesc); modeliovertexdesc.attributes[0].name = MDLVertexAttributePosition; modeliovertexdesc.attributes[1].name = MDLVertexAttributeColor; modeliovertexdesc.attributes[2].name = MDLVertexAttributeTextureCoordinate;

124 Asset Initialization 0 : position Float3 1 : color UChar4 2 : texuv HalfFloat2 MDLVertexDescriptor *modeliovertexdesc = MTKModelIOVertexFormatFromMetal(metalVertexDesc); modeliovertexdesc.attributes[0].name = MDLVertexAttributePosition; modeliovertexdesc.attributes[1].name = MDLVertexAttributeColor; modeliovertexdesc.attributes[2].name = MDLVertexAttributeTextureCoordinate;

125 Asset Initialization MetalKit Mesh Buffer Allocator MTKMeshBufferAllocator *mtkbufferallocator = [[MTKMeshBufferAllocator alloc] initwithdevice:metaldevice];

126 Asset Initialization Creating the Asset Object MDLAsset *asset = [[MDLAsset alloc] initwithurl:fileurl vertexdescriptor:modeliovertexdesc bufferallocator:mtkbufferallocator];

127 Asset Initialization Creating the Asset Object MDLAsset *asset = [[MDLAsset alloc] initwithurl:fileurl vertexdescriptor:modeliovertexdesc bufferallocator:mtkbufferallocator];

128 Asset Initialization Creating the Asset Object MDLAsset *asset = [[MDLAsset alloc] initwithurl:fileurl vertexdescriptor:modeliovertexdesc bufferallocator:mtkbufferallocator];

129 Asset Initialization Creating the Asset Object MDLAsset *asset = [[MDLAsset alloc] initwithurl:fileurl vertexdescriptor:modeliovertexdesc bufferallocator:mtkbufferallocator];

130 MetalKit Meshes Initialization Setup for model rendering Vertex Descriptor Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing MetalKit Mesh Buffer Allocator

131 MetalKit Meshes Initialization Setup for model rendering Vertex Descriptor Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing MetalKit Mesh Buffer Allocator

132 Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh

133 Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh

134 Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh

135 Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh

136 NSArray<MTKMesh *> *meshes = [MTKMesh meshesfromasset:asset device:device]; Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh

137 NSArray<MTKMesh *> *meshes = [MTKMesh meshesfromasset:asset device:device]; Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh

138 NSArray<MTKMesh *> *meshes = [MTKMesh meshesfromasset:asset device:device]; Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh

139 NSArray<MTKMesh *> *meshes = [MTKMesh meshesfromasset:asset device:device]; Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh meshes Mesh Mesh Mesh Mesh Mesh

140 NSArray<MTKMesh *> *meshes = [MTKMesh meshesfromasset:asset device:device]; Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh meshes Mesh Mesh Mesh Mesh Mesh

141 NSArray<MTKMesh *> *meshes = [MTKMesh meshesfromasset:asset device:device]; Asset Camera Mesh Mesh Light Mesh Camera Mesh Light Mesh meshes Mesh Mesh Mesh Mesh Mesh

142 NSArray<MTKMesh *> *meshes = [MTKMesh meshesfromasset:asset device:device]; Asset Camera Mesh Mesh Mesh Light Mesh Camera Mesh Light Mesh meshes Mesh Mesh Mesh Mesh

143 Mesh

144 Mesh VertexBuffers

145 VertexBuffers Mesh VertexDescriptor

146 VertexBuffers Mesh VertexDescriptor

147 VertexBuffers Mesh VertexDescriptor Submesh

148 VertexBuffers Mesh VertexDescriptor IndexBuffer Submesh

149 VertexBuffers Mesh VertexDescriptor IndexBuffer Submesh

150 VertexBuffers Mesh VertexDescriptor IndexBuffer Submesh Draw Properties

151 VertexBuffers Mesh VertexDescriptor IndexBuffer Submesh Draw Properties

152 Mesh Rendering with Metal Setup for model rendering Vertex Descriptor Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing MetalKit Mesh Buffer Allocator

153 Mesh Rendering with Metal Setup for model rendering Vertex Descriptor Vertex Descriptor Metal Render State Pipeline Model I/O Asset Initialization MetalKit Mesh and Submesh Objects Metal Render State Setup and Drawing MetalKit Mesh Buffer Allocator

154 Setting Mesh State NSUInteger bufferindex = 0; for(mtkmeshbuffer *vertexbuffer in mesh.vertexbuffers) { if(vertexbuffer.buffer!= nil) { [renderencoder setvertexbuffer:vertexbuffer.buffer offset:vertexbuffer.offset atindex:bufferindex]; } bufferindex++; }

155 Setting Mesh State NSUInteger bufferindex = 0; for(mtkmeshbuffer *vertexbuffer in mesh.vertexbuffers) { if(vertexbuffer.buffer!= nil) { [renderencoder setvertexbuffer:vertexbuffer.buffer offset:vertexbuffer.offset atindex:bufferindex]; } bufferindex++; }

156 Setting Mesh State NSUInteger bufferindex = 0; for(mtkmeshbuffer *vertexbuffer in mesh.vertexbuffers) { if(vertexbuffer.buffer!= nil) { [renderencoder setvertexbuffer:vertexbuffer.buffer offset:vertexbuffer.offset atindex:bufferindex]; } bufferindex++; }

157 Setting Mesh State NSUInteger bufferindex = 0; for(mtkmeshbuffer *vertexbuffer in mesh.vertexbuffers) { if(vertexbuffer.buffer!= nil) { [renderencoder setvertexbuffer:vertexbuffer.buffer offset:vertexbuffer.offset atindex:bufferindex]; } bufferindex++; }

158 Setting Mesh State NSUInteger bufferindex = 0; for(mtkmeshbuffer *vertexbuffer in mesh.vertexbuffers) { if(vertexbuffer.buffer!= nil) { [renderencoder setvertexbuffer:vertexbuffer.buffer offset:vertexbuffer.offset atindex:bufferindex]; } bufferindex++; }

159 Setting Mesh State NSUInteger bufferindex = 0; for(mtkmeshbuffer *vertexbuffer in mesh.vertexbuffers) { if(vertexbuffer.buffer!= nil) { [renderencoder setvertexbuffer:vertexbuffer.buffer offset:vertexbuffer.offset atindex:bufferindex]; } bufferindex++; }

160 Rendering the Mesh for(mtksubmesh *submesh in mesh.submeshes) { [renderencoder drawindexedprimitives:submesh.primitivetype indexcount:submesh.indexcount indextype:submesh.indextype indexbuffer:submesh.indexbuffer.buffer indexbufferoffset:submesh.indexbuffer.offset]; }

161 Rendering the Mesh for(mtksubmesh *submesh in mesh.submeshes) { [renderencoder drawindexedprimitives:submesh.primitivetype indexcount:submesh.indexcount indextype:submesh.indextype indexbuffer:submesh.indexbuffer.buffer indexbufferoffset:submesh.indexbuffer.offset]; }

162 Rendering the Mesh for(mtksubmesh *submesh in mesh.submeshes) { [renderencoder drawindexedprimitives:submesh.primitivetype indexcount:submesh.indexcount indextype:submesh.indextype indexbuffer:submesh.indexbuffer.buffer indexbufferoffset:submesh.indexbuffer.offset]; }

163 Rendering the Mesh for(mtksubmesh *submesh in mesh.submeshes) { [renderencoder drawindexedprimitives:submesh.primitivetype submesh indexcount:submesh.indexcount indextype:submesh.indextype indexbuffer:submesh.indexbuffer.buffer indexbufferoffset:submesh.indexbuffer.offset]; submesh }

164 MetalKit Essentials

165 Metal Performance Shaders Anna Tikhonova GPU Software Frameworks Engineer

166 Metal Performance Shaders Introduction A framework of data-parallel algorithms for the GPU CPU-style library for the GPU

167 Metal Performance Shaders Introduction Optimized for ios Available in ios 9 for the A8 processor ipad Air 2 iphone 6 Plus iphone 6

168 Metal Performance Shaders Introduction Designed to integrate easily into your Metal applications As simple as calling a library function

169 Metal Performance Shaders Supported image operators Histogram, Equalization, and Specification Morphology Min, Max, Dilate, and Erode Lanczos Resampling Median Thresholding Integral Convolution General, Gaussian Blur, Box, Tent, and Sobel

170 Metal Performance Shaders Supported image operators Histogram, Equalization, and Specification Morphology Min, Max, Dilate, and Erode Lanczos Resampling Median Thresholding Integral Convolution General, Gaussian Blur, Box, Tent, and Sobel

171 Equalization

172 Equalization

173 Metal Performance Shaders Supported image operators Histogram, Equalization and Specification Morphology Min, Max, Dilate, and Erode Lanczos Resampling Median Thresholding Integral Convolution General, Gaussian Blur, Box, Tent, and Sobel

174 Lanczos Resampling

175 Lanczos Resampling

176 Metal Performance Shaders Supported image operators Histogram, Equalization and Specification Morphology Min, Max, Dilate, and Erode Lanczos Resampling Median Thresholding Integral Convolution General, Gaussian Blur, Box, Tent, and Sobel

177 Thresholding

178 Thresholding

179 Thresholding and Sobel

180 Thresholding and Sobel

181 Metal Performance Shaders Supported image operators Histogram, Equalization and Specification Morphology Min, Max, Dilate, and Erode Lanczos Resampling Median Thresholding Integral Convolution General, Gaussian Blur, Box, Tent, and Sobel

182 Gaussian Blur

183 Gaussian Blur

184 Using Metal Performance Shaders Use a blur filter // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture];

185 Using Metal Performance Shaders Use a blur filter // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture];

186 Using Metal Performance Shaders Use a blur filter // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture];

187 Using Metal Performance Shaders Use a blur filter // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture];

188 Using Metal Performance Shaders Use a blur filter // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture];

189 Using Metal Performance Shaders Use a blur filter // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; Command Buffer // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture];

190 Using Metal Performance Shaders Use a blur filter Command Buffer // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; Draw Draw Draw // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture];

191 Using Metal Performance Shaders Use a blur filter Command Buffer // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; Draw Draw Draw // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer Dispatch Kernel Dispatch Kernel source: sourcetexture destination: destinationtexture];

192 Using Metal Performance Shaders Use a blur filter Command Buffer // Create a filter object MPSImageGaussianBlur *blurfilter = [[MPSImageGaussianBlur alloc] initwithdevice: device sigma: 3]; Draw Draw Draw // Encode filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer Dispatch Kernel Dispatch Kernel source: sourcetexture destination: destinationtexture];

193 Using Metal Performance Shaders Use a blur filter Command Buffer Draw Draw Draw Dispatch Kernel Dispatch Kernel // Create a filter object MPSImageGaussianBlur *blurfilter [[MPSImageGaussianBlur alloc] initwithdevice: device Encode MPS Blur // filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture

194 Using Metal Performance Shaders Use a blur filter Command Buffer Draw Draw Draw Dispatch Kernel Dispatch Kernel // Create a filter object MPSImageGaussianBlur *blurfilter [[MPSImageGaussianBlur alloc] initwithdevice: device Encode MPS Blur // filter to the command buffer [blurfilter encodetocommandbuffer: commandbuffer source: sourcetexture destination: destinationtexture Commit

195 Download Sample Code MetalPerformanceShadersHelloWorld

196 Performance Behind the scenes Choose the right algorithm Tune for Kernel radius Pixel format Memory hierarchy Number of pixels per thread Threadgroup dimensions CPU optimizations

197 Performance Behind the scenes Choose the right algorithm Tune for Kernel radius Pixel format Memory hierarchy Number of pixels per thread Threadgroup dimensions CPU optimizations

198 Performance Behind the scenes Choose the right algorithm Tune for Kernel radius Pixel format Memory hierarchy Number of pixels per thread Threadgroup dimensions CPU optimizations

199 Performance Behind the scenes Choose the right algorithm Tune for Kernel radius Pixel format Memory hierarchy Number of pixels per thread Threadgroup dimensions CPU optimizations

200 Performance Behind the scenes Choose the right algorithm Tune for Kernel radius Pixel format Memory hierarchy Number of pixels per thread Threadgroup dimensions CPU optimizations

201 Optimized Gaussian Blur Statistics

202 Optimized Gaussian Blur Statistics 49 Metal kernels 2000 Lines of kernel code 821 Different Gaussian filter implementations

203 Demo

204 Filter Performance Not capped by screen refresh rate Smaller is better 16.6 ms = 60 FPS ms Sigma (Blur Radius)

205 A Few More Details

206 Source offset and Destination cliprect Filter properties Source Destination

207 Source offset and Destination cliprect Filter properties Source cliprect Destination

208 Source offset and Destination cliprect Filter properties offset Source cliprect Destination

209 Source offset and Destination cliprect Filter properties offset Source cliprect Destination

210 Source offset and Destination cliprect Filter properties offset Source cliprect Destination

211 In-place Operation Save memory Source == Destination

212 In-place Operation Save memory cliprect Source == Destination

213 In-place Operation Save memory offset cliprect Source == Destination

214 In-place Operation Save memory offset cliprect Source == Destination

215 In-place Operation Save memory offset cliprect Source == Destination

216 In-place Operation How? // Encode filter in-place in a Metal command buffer [blurfilter encodetocommandbuffer: commandbuffer inplacetexture: sourcetexture fallbackcopyallocator: myallocator];

217 In-place Operation How? // Encode filter in-place in a Metal command buffer [blurfilter encodetocommandbuffer: commandbuffer inplacetexture: sourcetexture fallbackcopyallocator: myallocator]; It s not always possible for Metal Performance Shaders filters to run in-place Depends on filter, filter parameters and properties Copy allocator will create a destination texture, so operation can proceed out-of-place

218 Fallback Copy Allocator Example MPSCopyAllocator myallocator = ^id <MTLTexture>( MPSKernel * filter, id <MTLCommandBuffer> commandbuffer, id <MTLTexture> sourcetexture) { MTLTextureDescriptor * desc = descriptorfromtexture(sourcetexture); id <MTLTexture> destinationtexture = [commandbuffer.device newtexturewithdescriptor: desc]; }; return destinationtexture;

219 Fallback Copy Allocator Example MPSCopyAllocator myallocator = ^id <MTLTexture>( MPSKernel * filter, id <MTLCommandBuffer> commandbuffer, id <MTLTexture> sourcetexture) { MTLTextureDescriptor * desc = descriptorfromtexture(sourcetexture); id <MTLTexture> destinationtexture = [commandbuffer.device newtexturewithdescriptor: desc]; // Use Metal encoder to initialize the destinationtexture from // sourcetexture, if necessary }; return destinationtexture;

220 Fallback Copy Allocator Example MPSCopyAllocator myallocator = ^id <MTLTexture>( MPSKernel * filter, id <MTLCommandBuffer> commandbuffer, id <MTLTexture> sourcetexture) { MTLTextureDescriptor * desc = descriptorfromtexture(sourcetexture); id <MTLTexture> destinationtexture = [commandbuffer.device newtexturewithdescriptor: desc]; // Use Metal encoder to initialize the destinationtexture from // sourcetexture, if necessary }; return destinationtexture;

221 Summary Make use of the new Metal support frameworks Robust, optimized, easy to integrate Faster bring-up of your application Less code to write and maintain Give us your feedback!

222 More Information Metal Documentation and Videos Apple Developer Forums Developer Technical Support General Inquiries Allan Schaffer, Game Technologies Evangelist

223 Related Sessions Managing 3D Assets with Model I/O Mission Tuesday 2:30PM What s New in Metal, Part 1 Presidio Tuesday 3:30PM Metal Performance Optimization Techniques Pacific Heights Friday 11:00AM

224 Related Labs Metal Lab Graphics D Friday 12:00PM

225