Metal for OpenGL Developers

Transcription

1 #WWDC18 Metal for OpenGL Developers Dan Omachi, Metal Ecosystem Engineer Sukanya Sudugu, GPU Software Engineer 2018 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

2 These legacy APIs are deprecated Still available in ios 12, macos and tvos 12 Begin transitioning to Metal

3 Choosing an Approach High-level Apple frameworks SpriteKit, SceneKit, and Core Image

4 Choosing an Approach High-level Apple frameworks SpriteKit, SceneKit, and Core Image Third-party engines Unity, Unreal, Lumberyard, etc. Update to latest version

5 Choosing an Approach High-level Apple frameworks SpriteKit, SceneKit, and Core Image Third-party engines Unity, Unreal, Lumberyard, etc. Update to latest version Focus today incrementally porting to Metal

6 Metal Concepts

7 Challenges with OpenGL OpenGL designed more than 25 years ago Core architecture reflects the origins of 3D graphics Extensions retrofitted some GPU features

8 Challenges with OpenGL OpenGL designed more than 25 years ago Core architecture reflects the origins of 3D graphics Extensions retrofitted some GPU features

9 Challenges with OpenGL OpenGL designed more than 25 years ago Core architecture reflects the origins of 3D graphics Extensions retrofitted some GPU features Fundamental design choices based on past principles GPU pipeline has changed Multithreaded operation not considered Asynchronous processing, not core

10 Challenges with OpenGL OpenGL designed more than 25 years ago Core architecture reflects the origins of 3D graphics Extensions retrofitted some GPU features Fundamental design choices based on past principles GPU pipeline has changed Multithreaded operation not considered Asynchronous processing, not core

11 Challenges with OpenGL OpenGL designed more than 25 years ago Core architecture reflects the origins of 3D graphics Extensions retrofitted some GPU features Fundamental design choices based on past principles GPU pipeline has changed Multithreaded operation not considered Asynchronous processing, not core

12 Design Goals for Metal Efficient GPU interaction

13 Design Goals for Metal Efficient GPU interaction Low CPU overhead

14 Design Goals for Metal Efficient GPU interaction Low CPU overhead Multithreaded execution

15 Design Goals for Metal Efficient GPU interaction Low CPU overhead Multithreaded execution Predictable operation

16 Design Goals for Metal Efficient GPU interaction Low CPU overhead Multithreaded execution Predictable operation Resource and synchronization control

17 Design Goals for Metal Efficient GPU interaction Low CPU overhead Multithreaded execution Predictable operation Resource and synchronization control Approachable to OpenGL developers

18 Design Goals for Metal Efficient GPU interaction Low CPU overhead Multithreaded execution Predictable operation Resource and synchronization control Approachable to OpenGL developers Built for modern and Apple-design GPUs

19 Key Conceptual Differences Expensive operations less frequent Expensive CPU operations performed less often

20 Key Conceptual Differences Expensive operations less frequent Expensive CPU operations performed less often More GPU command generation during object creation

21 Key Conceptual Differences Expensive operations less frequent Expensive CPU operations performed less often More GPU command generation during object creation Less needed when rendering

22 Key Conceptual Differences Modern GPU pipeline Reflects the modern GPU architectures This way there's much less cost to use them later on when actually rendering

23 Key Conceptual Differences Modern GPU pipeline Reflects the modern GPU architectures Closer match yields less costly translation to GPU commands This way there's much less cost to use them later on when actually rendering

24 Key Conceptual Differences Modern GPU pipeline Reflects the modern GPU architectures Closer match yields less costly translation to GPU commands State grouped more efficiently This way there's much less cost to use them later on when actually rendering

25 Key Conceptual Differences Multithreaded execution Designed for multithreaded execution

26 Key Conceptual Differences Multithreaded execution Designed for multithreaded execution Clear rules for multithreaded usage

27 Key Conceptual Differences Multithreaded execution Designed for multithreaded execution Clear rules for multithreaded usage Cross thread object usability

28 Key Conceptual Differences Execution model True interaction between software and GPU

29 Key Conceptual Differences Execution model True interaction between software and GPU Predictable operation allows efficient designs

30 Key Conceptual Differences Execution model True interaction between software and GPU Predictable operation allows efficient designs Thinner stack between application and GPU

31 Application Renderer OpenGL API OpenGL Context

32 Application Renderer Metal API Device

33 Application Renderer Metal API Device

34 Application Renderer Metal API Device GPU

35 Application Renderer Metal API Device GPU

36 Application Renderer Metal API Pipelines Textures Buffers Device GPU

37 Application Renderer Metal API Textures Buffers Device GPU Pipelines Render Objects

38 Application Renderer Metal API Textures Buffers Command Device Queue GPU Pipelines Render Objects

39 Application Renderer Metal API Textures Buffers Command Device Queue GPU Pipelines Render Objects

40 Application Renderer Metal API Textures Buffers Command Queue Device GPU GPU Pipelines Render Objects

41 Application Renderer Metal API Textures Buffers Command Queue Device GPU GPU Pipelines Render Objects

42 Application Renderer Metal API Textures Buffers Command Queue Device GPU Pipelines Render Objects

43 Application Renderer Metal API Command Buffers Command Queue Device GPU

44 Application Renderer Metal API Command Buffers Command Queue Device GPU

45 Application Renderer Metal API Command Buffers Command Queue Device GPU

46 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

47 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

48 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

49 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

50 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

51 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

52 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

53 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

54 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

55 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

56 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

57 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

58 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

59 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

60 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

61 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

62 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

63 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

64 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

65 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

66 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

67 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

68 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

69 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

70 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

71 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

72 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

73 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

74 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

75 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

76 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

77 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

78 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

79 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

80 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

81 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

82 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

83 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

84 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

85 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

86 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

87 Application Renderer Metal API Command Encoders Command Buffers Command Queue Device GPU

88 Command Encoders Render Command Encoder

89 Command Encoders Render Command Encoder Blit Command Encoder

90 Command Encoders Render Command Encoder Blit Command Encoder Compute Command Encoder

91 Render Command Encoders Commands for a render pass Encodes a series of render commands

92 Render Command Encoders Commands for a render pass Encodes a series of render commands Also called a Render Pass

93 Render Command Encoders Commands for a render pass Encodes a series of render commands Also called a Render Pass Set render object for the graphics pipeline Buffer, textures, shaders

94 Render Command Encoders Commands for a render pass Encodes a series of render commands Also called a Render Pass Set render object for the graphics pipeline Buffer, textures, shaders Issue draw commands Draw primitives, draw index primitives, instanced draws

95 Render Command Encoders Render targets Associated with a set of render targets

96 Render Command Encoders Render targets Associated with a set of render targets Textures for rendering

97 Render Command Encoders Render targets Associated with a set of render targets Textures for rendering Specify a set of render targets upon creation

98 Render Command Encoders Render targets Associated with a set of render targets Textures for rendering Specify a set of render targets upon creation All draw commands directed to these for lifetime of encoder

99 Render Command Encoders Render targets Associated with a set of render targets Textures for rendering Specify a set of render targets upon creation All draw commands directed to these for lifetime of encoder New render targets need a new encoder

100 Render Command Encoders Render targets Associated with a set of render targets Textures for rendering Specify a set of render targets upon creation All draw commands directed to these for lifetime of encoder New render targets need a new encoder Clear delineation between sets of render targets

101 Render Objects Textures

102 Render Objects Textures Buffers

103 Render Objects Textures Buffers Samplers

104 Render Objects Textures Buffers Samplers Render pipeline states

105 Render Objects Textures Buffers Samplers Render pipeline states Depth stencil states

106 Render Objects Object creation Created from a device Usable only on that device

107 Render Objects Object creation Created from a device Usable only on that device Objects state set at creation Descriptor object specifies properties for render object

108 Render Objects Object creation Created from a device Usable only on that device Objects state set at creation Descriptor object specifies properties for render object State set at creation fixed for the lifetime of the object Image data of textures and values in buffers can change

109 Render Objects Object creation Metal compiles objects into GPU state once Never needs to check for changes and recompile

110 Render Objects Object creation Metal compiles objects into GPU state once Never needs to check for changes and recompile Multithreaded usage more efficient Metal does not need to protect state from changes on other threads

111 Metal Porting

112 Build Initialize Render

113 Build Initialize Render

114 Build Initialize Render

115 Build Initialize Render

116 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

117 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

118 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

119 Metal Shading Language Based on C++ Classes, templates, structs, enums, namespaces

120 Metal Shading Language Based on C++ Classes, templates, structs, enums, namespaces Built-in types for vectors and matrices

121 Metal Shading Language Based on C++ Classes, templates, structs, enums, namespaces Built-in types for vectors and matrices Built-in functions and operators

122 Metal Shading Language Based on C++ Classes, templates, structs, enums, namespaces Built-in types for vectors and matrices Built-in functions and operators Built-in classes for textures and samplers

123 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

124 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

125 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

126 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

127 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

128 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

129 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

130 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

131 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

132 }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

133 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

134 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

135 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

136 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

137 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

138 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

139 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

140 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

141 struct VertexOutput { float4 clippos [[position]]; float2 texcoord; }; struct Vertex { float4 modelpos; float2 texcoord; }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord;

142 }; vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

143 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

144 vertex VertexOutput myvertexshader(uint vid [[ vertex_id ]], device Vertex * vertices [[ buffer(0) ]], constant Uniforms & uniforms [[ buffer(1) ]]) { VertexOutput out; out.clippos = vertices[vid].modelpos * uniforms.mvp; out.texcoord = vertices[vid].texcoord; return out; } fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

145 fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

146 [renderencoder setfragmentbuffer:myuniformbuffer offset:0 atindex:3]; [renderencoder setfragmenttexture:mycolortexture atindex:0]; [renderencoder setfragmentsampler:mysampler atindex:1]; fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

147 [renderencoder setfragmentbuffer:myuniformbuffer offset:0 atindex:3]; [renderencoder setfragmenttexture:mycolortexture atindex:0]; [renderencoder setfragmentsampler:mysampler atindex:1]; fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

148 [renderencoder setfragmentbuffer:myuniformbuffer offset:0 atindex:3]; [renderencoder setfragmenttexture:mycolortexture atindex:0]; [renderencoder setfragmentsampler:mysampler atindex:1]; fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

149 [renderencoder setfragmentbuffer:myuniformbuffer offset:0 atindex:3]; [renderencoder setfragmenttexture:mycolortexture atindex:0]; [renderencoder setfragmentsampler:mysampler atindex:1]; fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

150 [renderencoder setfragmentbuffer:myuniformbuffer offset:0 atindex:3]; [renderencoder setfragmenttexture:mycolortexture atindex:0]; [renderencoder setfragmentsampler:mysampler atindex:1]; fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

151 [renderencoder setfragmentbuffer:myuniformbuffer offset:0 atindex:3]; [renderencoder setfragmenttexture:mycolortexture atindex:0]; [renderencoder setfragmentsampler:mysampler atindex:1]; fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

152 [renderencoder setfragmentbuffer:myuniformbuffer offset:0 atindex:3]; [renderencoder setfragmenttexture:mycolortexture atindex:0]; [renderencoder setfragmentsampler:mysampler atindex:1]; fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

153 [renderencoder setfragmentbuffer:myuniformbuffer offset:0 atindex:3]; [renderencoder setfragmenttexture:mycolortexture atindex:0]; [renderencoder setfragmentsampler:mysampler atindex:1]; fragment float4 myfragmentshader(vertexoutput in [[ stage_in ]], constant Uniforms & uniforms [[ buffer(3) ]], texture2d<float> colortex [[ texture(0) ]], sampler texsampler [[ sampler(1) ]]) { return colortex.sample(texsampler, in.texcoord * uniforms.coordscale); }

154 SIMD Type Library Types for shader development Vector and matrix types

155 SIMD Type Library Types for shader development Vector and matrix types Usable with Metal shading language and application code

156 SIMD Type Library Types for shader development Vector and matrix types Usable with Metal shading language and application code struct MyUniforms { matrix_float4x4 modelviewprojection; vector_float4 sunposition; };

157 SIMD Type Library Types for shader development Vector and matrix types Usable with Metal shading language and application code MyShaderTypes.h struct MyUniforms {c matrix_float4x4 modelviewprojection; vector_float4 sunposition; };

158 SIMD Type Library Types for shader development MyShaderTypes.h struct MyUniforms {c matrix_float4x4 modelviewprojection; vector_float4 sunposition; }; MyRenderer.m MyShaders.metal #include "MyShaderTypes.h" #include "MyShaderTypes.h"

159 Shader Compilation Building with Xcode Xcode compiles shaders into a Metal library (.metallib)

160 Shader Compilation Building with Xcode Xcode compiles shaders into a Metal library (.metallib) Front-end compilation to binary intermediate representation

161 Shader Compilation Building with Xcode Xcode compiles shaders into a Metal library (.metallib) Front-end compilation to binary intermediate representation Avoids parsing time on customer systems

162 Shader Compilation Building with Xcode Xcode compiles shaders into a Metal library (.metallib) Front-end compilation to binary intermediate representation Avoids parsing time on customer systems By default, all shaders built into default.metallib Placed in app bundle for run time retrieval

163 Runtime Shader Compilation Also can build shaders from source at runtime

164 Runtime Shader Compilation Also can build shaders from source at runtime Significant disadvantages Full shader compilation occurs at runtime Compilation errors less obvious No header sharing between application and runtime built shaders

165 Runtime Shader Compilation Also can build shaders from source at runtime Significant disadvantages Full shader compilation occurs at runtime Compilation errors less obvious No header sharing between application and runtime built shaders Build time compilation recommended

166 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

167 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

168 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

169 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

170 Devices A device represents one GPU

171 Devices A device represents one GPU Creates render objects Textures, buffers, pipelines

172 Devices A device represents one GPU Creates render objects Textures, buffers, pipelines macos multiple devices may be avaliable

173 Devices A device represents one GPU Creates render objects Textures, buffers, pipelines macos multiple devices may be avaliable Default device suitable for most applications // Getting a device id<mtldevice> device = MTLCreateSystemDefaultDevice();

174 Command Queues Queue created from a device

175 Command Queues Queue created from a device Queues execute command buffers in order Create queue at initialization

176 Command Queues Queue created from a device Queues execute command buffers in order Create queue at initialization Typically one queue sufficient

177 Command Queues Queue created from a device Queues execute command buffers in order Create queue at initialization Typically one queue sufficient // Getting a device id<mtlcommandqueue> commandqueue = [device newcommandqueue];

178 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

179 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

180 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

181 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

182 Textures Buffers Pipelines

183 Textures Buffers Pipelines

184 Device

185 Descriptor Device

186 Texture descriptor Device

187 Texture descriptor Texture Type Width Height Num Mipmaps Pixel Format 2D RGBA8 Device

188 Texture descriptor Texture Type Width Height Num Mipmaps Pixel Format 2D RGBA8 Device

189 Texture descriptor Texture Object Texture Type Width Height Num Mipmaps Pixel Format 2D RGBA8 Device

190 Texture descriptor Texture Object Texture Type Width Height Num Mipmaps Pixel Format 2D RGBA8 Device Memory

191 Texture descriptor Texture Object Texture Type Width Height Num Mipmaps Pixel Format 2D RGBA8 Device Memory

192 Texture Object Device Memory

193 Texture Object Device Memory

194 Texture Object Device Memory

195 Texture Object Device Memory

196 Texture Object Device Memory

197 Storage Modes Memory

198 Storage Modes Shared storage GPU CPU Memory

199 Storage Modes Shared storage GPU CPU Memory

200 Storage Modes Private storage GPU CPU Memory

201 Storage Modes Private storage GPU CPU Memory

202 Storage Modes Private storage GPU CPU Video Memory System Memory

203 Storage Modes Managed storage GPU CPU Video Memory System Memory

204 Storage Modes Managed storage GPU CPU Video Memory System Memory

205 Storage Modes Managed storage GPU CPU Video Memory System Memory

206 Storage Modes Managed storage GPU CPU Video Memory System Memory

207 // Creating Textures MTLTextureDescriptor *texturedescriptor = [MTLTextureDescriptor new]; texturedescriptor.pixelformat = MTLPixelFormatBGRA8Unorm; texturedescriptor.width = 512; texturedescriptor.height = 512; texturedescriptor.storagemode = MTLStorageModeShared; id<mtltexture> texture = [device newtexturewithdescriptor:texturedescriptor];

208 // Creating Textures MTLTextureDescriptor *texturedescriptor = [MTLTextureDescriptor new]; texturedescriptor.pixelformat = MTLPixelFormatBGRA8Unorm; texturedescriptor.width = 512; texturedescriptor.height = 512; texturedescriptor.storagemode = MTLStorageModeShared;; id<mtltexture> texture = [device newtexturewithdescriptor:texturedescriptor];

209 // Creating Textures MTLTextureDescriptor *texturedescriptor = [MTLTextureDescriptor new]; texturedescriptor.pixelformat = MTLPixelFormatBGRA8Unorm; texturedescriptor.width = 512; texturedescriptor.height = 512; texturedescriptor.storagemode = MTLStorageModeShared; id<mtltexture> texture = [device newtexturewithdescriptor:texturedescriptor];

210 // Loading Image Data NSUInteger bytesperrow = 4 * image.width; MTLRegion region = { { 0, 0, 0 }, // Origin { 512, 512, 1 } // Size }; [texture replaceregion:region mipmaplevel:0 withbytes:imagedata bytesperrow:bytesperrow];

211 // Loading Image Data NSUInteger bytesperrow = 4 * image.width; MTLRegion region = { { 0, 0, 0 }, // Origin { 512, 512, 1 } // Size }; [texture replaceregion:region mipmaplevel:0 withbytes:imagedata bytesperrow:bytesperrow];

212 // Loading Image Data NSUInteger bytesperrow = 4 * image.width; MTLRegion region = { { 0, 0, 0 }, // Origin { 512, 512, 1 } // Size }; [texture replaceregion:region mipmaplevel:0 withbytes:imagedata bytesperrow:bytesperrow];

213 Texture Differences Sampler state never part of texture Wrap modes, filtering, min/max LOD

214 Texture Differences Sampler state never part of texture Wrap modes, filtering, min/max LOD Texture image data not flipped OpenGL uses bottom-left origin, Metal uses top-left origin

215 Texture Differences Sampler state never part of texture Wrap modes, filtering, min/max LOD Texture image data not flipped OpenGL uses bottom-left origin, Metal uses top-left origin Metal does not perform format conversion

216 Textures Buffers Shaders

217 Textures Buffers Shaders

218 Buffers Metal uses buffers for vertices, indices, and all uniform data OpenGL's vertex, element, and uniform buffers are similar Easier to port apps that have adopted these

219 // Creating Buffers id<mtlbuffer> buffer = [device newbufferwithlength:bufferdatabytesize options:mtlresourcestoragemodeshared]; struct MyUniforms *uniforms = (struct MyUniforms*) buffer.contents; uniforms->modelviewprojection = modelviewprojection; uniforms->sunposition = sunposition;

220 // Creating Buffers id<mtlbuffer> buffer = [device newbufferwithlength:bufferdatabytesize options:mtlresourcestoragemodeshared]; struct MyUniforms *uniforms = (struct MyUniforms*) buffer.contents; uniforms->modelviewprojection = modelviewprojection; uniforms->sunposition = sunposition;

221 // Creating Buffers id<mtlbuffer> buffer = [device newbufferwithlength:bufferdatabytesize options:mtlresourcestoragemodeshared]; struct MyUniforms *uniforms = (struct MyUniforms*) buffer.contents; uniforms->modelviewprojection = modelviewprojection; uniforms->sunposition = sunposition;

222 Notes About Buffer Data Pay attention to alignment Type Alignment float3, int3, uint3 16 bytes float3x3, float4x3 16 bytes half3, short3, ushore3, 8 bytes half3x3, half4x3 8 bytes structures 4 bytes

223 Notes About Buffer Data Pay attention to alignment Type Alignment float3, int3, uint3 16 bytes float3x3, float4x3 16 bytes half3, short3, ushore3, 8 bytes half3x3, half4x3 8 bytes structures 4 bytes

224 Notes About Buffer Data Pay attention to alignment Type Alignment float3, int3, uint3 16 bytes float3x3, float4x3 16 bytes half3, short3, ushore3, 8 bytes half3x3, half4x3 8 bytes structures 4 bytes

225 Notes About Buffer Data SIMD and packed types SIMD libraries vector and matrix types follow same rules as Metal shaders Special packed vector types available to shaders packed_float3 consumes 12 bytes packed_half3 consumes 6 bytes Cannot directly operate on packed types Cast to non-packed type required

226 Storage Modes for Porting Use most convient storage modes Easier access to data

227 Storage Modes for Porting Use most convient storage modes Easier access to data On ios Create all textures and buffers with MTLStorageModeShared

228 Storage Modes for Porting Use most convient storage modes Easier access to data On ios Create all textures and buffers with MTLStorageModeShared On macos Create all textures with MTLStorageModeManaged Make judicious use of MTLStorageModeShared for buffers - Separate GPU only data from CPU accessible data

229 MetalKit Texture and buffer utilities Texture Loading Textures from KTX, PVR, JPG, PNG, TIFF, etc. Model Loading Vertex buffers from USD, OBJ, Alembic, etc.

230 Textures Buffers Pipelines

231 Textures Buffers Pipelines

232 Render Pipeline Descriptor Device

233 Render Pipeline Descriptor Vertex Shader Device Fragment Shader

234 Render Pipeline Descriptor Vertex Layout Vertex Shader Device Fragment Shader

235 Render Pipeline Descriptor Vertex Layout Vertex Shader Device Fragment Shader Blend State Render Target Pixel Formats

236 Render Pipeline Descriptor Vertex Layout Vertex Shader Device Fragment Shader Blend State Render Target Pixel Formats

237 Render Pipeline Descriptor Render State Pipeline Object Vertex Layout Vertex Shader Device Fragment Shader Blend State Render Target Pixel Formats

238 Render State Pipeline Object Device

239 // Creating Render Pipeline Objects id<mtllibrary> defaultlibrary = [device newdefaultlibrary]; id<mtlfunction> vertexfunction = [defaultlibrary newfunctionwithname:@"vertexshader"]; id<mtlfunction> fragmentfunction = [defaultlibrary newfunctionwithname:@"fragmentshader"]; MTLRenderPipelineDescriptor *pipelinestatedescriptor = [MTLRenderPipelineDescriptor new]; pipelinestatedescriptor.vertexfunction = vertexfunction; pipelinestatedescriptor.fragmentfunction = fragmentfunction; pipelinestatedescriptor.colorattachments[0].pixelformat = MTLPixelFormatRGBA8Unorm; id<mtlrenderpipelinestate> pipelinestate; pipelinestate = [device newrenderpipelinestatewithdescriptor:pipelinestatedescriptor error:nil];

240 // Creating Render Pipeline Objects id<mtllibrary> defaultlibrary = [device newdefaultlibrary]; id<mtlfunction> vertexfunction = [defaultlibrary newfunctionwithname:@"vertexshader"]; id<mtlfunction> fragmentfunction = [defaultlibrary newfunctionwithname:@"fragmentshader"]; MTLRenderPipelineDescriptor *pipelinestatedescriptor = [MTLRenderPipelineDescriptor new]; pipelinestatedescriptor.vertexfunction = vertexfunction; pipelinestatedescriptor.fragmentfunction = fragmentfunction; pipelinestatedescriptor.colorattachments[0].pixelformat = MTLPixelFormatRGBA8Unorm; id<mtlrenderpipelinestate> pipelinestate; pipelinestate = [device newrenderpipelinestatewithdescriptor:pipelinestatedescriptor error:nil];

241 // Creating Render Pipeline Objects id<mtllibrary> defaultlibrary = [device newdefaultlibrary]; id<mtlfunction> vertexfunction = [defaultlibrary newfunctionwithname:@"vertexshader"]; id<mtlfunction> fragmentfunction = [defaultlibrary newfunctionwithname:@"fragmentshader"]; MTLRenderPipelineDescriptor *pipelinestatedescriptor = [MTLRenderPipelineDescriptor new]; pipelinestatedescriptor.vertexfunction = vertexfunction; pipelinestatedescriptor.fragmentfunction = fragmentfunction; pipelinestatedescriptor.colorattachments[0].pixelformat = MTLPixelFormatRGBA8Unorm; id<mtlrenderpipelinestate> pipelinestate; pipelinestate = [device newrenderpipelinestatewithdescriptor:pipelinestatedescriptor error:nil];

242 // Creating Render Pipeline Objects id<mtllibrary> defaultlibrary = [device newdefaultlibrary]; id<mtlfunction> vertexfunction = [defaultlibrary newfunctionwithname:@"vertexshader"]; id<mtlfunction> fragmentfunction = [defaultlibrary newfunctionwithname:@"fragmentshader"]; MTLRenderPipelineDescriptor *pipelinestatedescriptor = [MTLRenderPipelineDescriptor new]; pipelinestatedescriptor.vertexfunction = vertexfunction; pipelinestatedescriptor.fragmentfunction = fragmentfunction; pipelinestatedescriptor.colorattachments[0].pixelformat = MTLPixelFormatRGBA8Unorm; id<mtlrenderpipelinestate> pipelinestate; pipelinestate = [device newrenderpipelinestatewithdescriptor:pipelinestatedescriptor error:nil];

243 // Creating Render Pipeline Objects id<mtllibrary> defaultlibrary = [device newdefaultlibrary]; id<mtlfunction> vertexfunction = [defaultlibrary newfunctionwithname:@"vertexshader"]; id<mtlfunction> fragmentfunction = [defaultlibrary newfunctionwithname:@"fragmentshader"]; MTLRenderPipelineDescriptor *pipelinestatedescriptor = [MTLRenderPipelineDescriptor new]; pipelinestatedescriptor.vertexfunction = vertexfunction; pipelinestatedescriptor.fragmentfunction = fragmentfunction; pipelinestatedescriptor.colorattachments[0].pixelformat = MTLPixelFormatRGBA8Unorm; id<mtlrenderpipelinestate> pipelinestate; pipelinestate = [device newrenderpipelinestatewithdescriptor:pipelinestatedescriptor error:nil];

244 // Creating Render Pipeline Objects id<mtllibrary> defaultlibrary = [device newdefaultlibrary]; id<mtlfunction> vertexfunction = [defaultlibrary newfunctionwithname:@"vertexshader"]; id<mtlfunction> fragmentfunction = [defaultlibrary newfunctionwithname:@"fragmentshader"]; MTLRenderPipelineDescriptor *pipelinestatedescriptor = [MTLRenderPipelineDescriptor new]; pipelinestatedescriptor.vertexfunction = vertexfunction; pipelinestatedescriptor.fragmentfunction = fragmentfunction; pipelinestatedescriptor.colorattachments[0].pixelformat = MTLPixelFormatRGBA8Unorm; id<mtlrenderpipelinestate> pipelinestate; pipelinestate = [device newrenderpipelinestatewithdescriptor:pipelinestatedescriptor error:nil];

245 Pipeline Differences OpenGL Program Objects Metal OpenGL Vertex Layout Vertex Shader Vertex Shader Fragment Shader Fragment Shader Blend State Render Target Pixel Formats

246 Pipeline Building Create at intitlization Full compilation key advantage of state grouping Choose a canonical vertex layout for meshes Use a limited set of render target formats

247 Pipeline Building Lazy creation at draw time! Store pipeline state objects in a dictionary using descriptor as key Construct descriptor at draw time with current state Retrieve existing pipeline from dictionary OR build new pipeline

248 Create Render Objects at Initialization Object creation expensive

249 Create Render Objects at Initialization Object creation expensive Pipelines require backend compilation

250 Create Render Objects at Initialization Object creation expensive Pipelines require backend compilation Buffers and textures need allocations

251 Create Render Objects at Initialization Object creation expensive Pipelines require backend compilation Buffers and textures need allocations Once created, much faster usage during rendering

252 Porting the Render Loop Sukanya Sudugu, GPU Software Engineer

253 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

254 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

255 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

256 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

257 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

258 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

259 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

260 Command Buffers Explicit control over command buffer submission Start with one command buffer per frame Optionally split a frame into multiple command buffers to Submit early and get the GPU started Build commands on multiple threads

261 Command Buffers Explicit control over command buffer submission Start with one command buffer per frame Optionally split a frame into multiple command buffers to Submit early and get the GPU started Build commands on multiple threads Completion handler invoked when execution is finished

262 // Render Loop // Obtaining a command buffer at the beginning of each frame id<mtlcommandbuffer> commandbuffer = [commandqueue commandbuffer]; // Encoding commands... // Commit the command buffer to the GPU for execution [commandbuffer commit];

263 // Render Loop // Obtaining a command buffer at the beginning of each frame id<mtlcommandbuffer> commandbuffer = [commandqueue commandbuffer]; // Encoding commands... // Commit the command buffer to the GPU for execution [commandbuffer commit];

264 // Render Loop // Obtaining a command buffer at the beginning of each frame id<mtlcommandbuffer> commandbuffer = [commandqueue commandbuffer]; // Encoding commands... // Commit the command buffer to the GPU for execution [commandbuffer commit];

265 // Render Loop // Obtaining a command buffer at the beginning of each frame id<mtlcommandbuffer> commandbuffer = [commandqueue commandbuffer]; // Encoding commands... // Commit the command buffer to the GPU for execution [commandbuffer commit];

266 // Render Loop // Obtaining a command buffer at the beginning of each frame id<mtlcommandbuffer> commandbuffer = [commandqueue commandbuffer]; // Encoding commands... // Commit the command buffer to the GPU for execution [commandbuffer commit]; // Wait until the GPU has finished execution [commandbuffer waituntilcompleted];

267 // Render Loop // Obtaining a command buffer at the beginning of each frame id<mtlcommandbuffer> commandbuffer = [commandqueue commandbuffer]; // Encoding commands... // Commit the command buffer to the GPU for execution [commandbuffer commit];

268 // Render Loop // Obtaining a command buffer at the beginning of each frame id<mtlcommandbuffer> commandbuffer = [commandqueue commandbuffer]; // Encoding commands... // Add a completion hander to tell me when the GPU is done [commandbuffer addcompletedhandler:^(id<mtlcommandbuffer> commandbuffer) { // GPU is done with my buffer!... }]; // Commit the command buffer to the GPU for execution [commandbuffer commit];

269 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

270 Resource Updates Resources are explicitly managed in Metal Shared No implicit synchronization like OpenGL Allows for fine gained synchronization CPU Writes Resources Dynamic Uniforms Dynamic Vertices GPU Reads Application has complete control Best model dependent on usage Triple buffering recommended

271 Resource Updates Without synchronization CPU Write Buffer Read GPU Time

272 Resource Updates Without synchronization CPU Frame 1 Write Buffer Frame 1 Data Read GPU Time

273 Resource Updates Without synchronization CPU Frame 1 Write Buffer Frame 1 Data Read GPU Frame 1 Time

274 Resource Updates Without synchronization CPU Frame 1 Frame Wait 2 Write Buffer Frame 2 Data Frame 1 Data Read GPU Frame 1 Time

275 Resource Updates Without synchronization CPU Frame 1 Frame Wait 2 Write Buffer Frame 2 Data Frame 1 Data Read GPU Frame 1 Time

276 Temporary Solution Synchronous wait after every frame CPU Frame 1 Wait Frame 2 Write Buffer Frame 2 Data Read GPU Frame 1 Idle Time

277 Temporary Solution Synchronous wait after every frame CPU Frame 1 Wait Frame 2 Write Buffer Frame 2 Data Read GPU Frame 1 Idle Time

278 Triple Buffering Shared buffer pool CPU Frame 1 Write Frame 1 Data Buffer Read GPU Frame 1 Time

279 Triple Buffering Shared buffer pool CPU Frame 1 Frame 2 Frame 3 Write Frame 1 Data Buffer Frame 2 Data Frame 3 Data Read GPU Frame 1 Time

280 Triple Buffering Shared buffer pool CPU Frame 1 Frame 2 Frame 3 Write Frame 1 Data Buffer Frame 2 Data Frame 3 Data Read GPU Frame 1 Time

281 Triple Buffering Shared buffer pool CPU Frame 1 Frame 2 Frame 3 Write Frame 1 Data Buffer Frame 2 Data Frame 3 Data Read GPU Frame 1 Time

282 Triple Buffering Shared buffer pool CPU Frame 1 Frame 2 Frame 3 Write Frame 1 Data Buffer Frame 2 Data Frame 3 Data Read GPU Frame 1 Time

283 Triple Buffering Shared buffer pool Completion Handler CPU Frame 1 Frame 2 Frame 3 Write Frame 1 Data Buffer Frame 2 Data Frame 3 Data Read GPU Frame 1 Time

284 Triple Buffering Shared buffer pool Completion Handler CPU Frame 1 Frame 2 Frame 3 Frame 4 Write Buffer Frame 4 Data Frame 2 Data Frame 3 Data Read GPU Frame 1 Time

285 Triple Buffering Shared buffer pool Completion Handler CPU Frame 1 Frame 2 Frame 3 Frame 4 Write Buffer Frame 4 Data Frame 2 Data Frame 3 Data Read GPU Frame 1 Frame 2 Time

286 Triple Buffering Shared buffer pool Completion Handler CPU Frame 1 Frame 2 Frame 3 Frame 4 Frame 5 Write Frame 4 Data Buffer Frame 5 Data Frame 3 Data Read GPU Frame 1 Frame 2 Frame 3 Time

287 // Triple Buffering Implementation // Create FIFO queue of three dynamic data uniform buffers id <MTLBuffer> myuniformbuffers[3]; // Create a semaphore that gets signaled at each frame boundary. // The GPU signals the semaphore once it completes a frame s work, // allowing CPU To work on a new frame dispatch_semaphore_t frameboundarysemaphore = dispatch_semaphore_create(3); // Current frame Index NSUInteger currentuniformindex = 0;

288 // Triple Buffering Implementation // Create FIFO queue of three dynamic data uniform buffers id <MTLBuffer> myuniformbuffers[3]; // Create a semaphore that gets signaled at each frame boundary. // The GPU signals the semaphore once it completes a frame s work, // allowing CPU To work on a new frame dispatch_semaphore_t frameboundarysemaphore = dispatch_semaphore_create(3); // Current frame Index NSUInteger currentuniformindex = 0;

289 // Triple Buffering Implementation // Create FIFO queue of three dynamic data uniform buffers id <MTLBuffer> myuniformbuffers[3]; // Create a semaphore that gets signaled at each frame boundary. // The GPU signals the semaphore once it completes a frame s work, // allowing CPU To work on a new frame dispatch_semaphore_t frameboundarysemaphore = dispatch_semaphore_create(3); // Current frame Index NSUInteger currentuniformindex = 0;

290 // Wait until inflight frame is completed dispatch_semaphore_wait(frameboundarysemaphore, DISPATCH_TIME_FOREVER); // Grab current frame and update its buffer currentuniformindex = (currentuniformindex + 1) % 3; [self updateuniformresource: myuniformbuffers[currentuniformindex]]; // Encode commands and bind uniform buffer for GPU access // Schedule frame completion handler [commandbuffer addcompletedhandler:^(id<mtlcommandbuffer> commandbuffer) { // GPU work is complete. Signal the Semaphore to start CPU work dispatch_semaphore_signal(frameboundarysemaphore); }]; // Finalize and commit frame to GPU [commandbuffer commit];

291 // Wait until inflight frame is completed dispatch_semaphore_wait(frameboundarysemaphore, DISPATCH_TIME_FOREVER); // Grab current frame and update its buffer currentuniformindex = (currentuniformindex + 1) % 3; [self updateuniformresource: myuniformbuffers[currentuniformindex]]; // Encode commands and bind uniform buffer for GPU access // Schedule frame completion handler [commandbuffer addcompletedhandler:^(id<mtlcommandbuffer> commandbuffer) { // GPU work is complete. Signal the Semaphore to start CPU work dispatch_semaphore_signal(frameboundarysemaphore); }]; // Finalize and commit frame to GPU [commandbuffer commit];

292 // Wait until inflight frame is completed dispatch_semaphore_wait(frameboundarysemaphore, DISPATCH_TIME_FOREVER); // Grab current frame and update its buffer currentuniformindex = (currentuniformindex + 1) % 3; [self updateuniformresource: myuniformbuffers[currentuniformindex]]; // Encode commands and bind uniform buffer for GPU access // Schedule frame completion handler [commandbuffer addcompletedhandler:^(id<mtlcommandbuffer> commandbuffer) { // GPU work is complete. Signal the Semaphore to start CPU work dispatch_semaphore_signal(frameboundarysemaphore); }]; // Finalize and commit frame to GPU [commandbuffer commit];

293 // Wait until inflight frame is completed dispatch_semaphore_wait(frameboundarysemaphore, DISPATCH_TIME_FOREVER); // Grab current frame and update its buffer currentuniformindex = (currentuniformindex + 1) % 3; [self updateuniformresource: myuniformbuffers[currentuniformindex]]; // Encode commands and bind uniform buffer for GPU access // Schedule frame completion handler [commandbuffer addcompletedhandler:^(id<mtlcommandbuffer> commandbuffer) { // GPU work is complete. Signal the Semaphore to start CPU work dispatch_semaphore_signal(frameboundarysemaphore); }]; // Finalize and commit frame to GPU [commandbuffer commit];

294 // Wait until inflight frame is completed dispatch_semaphore_wait(frameboundarysemaphore, DISPATCH_TIME_FOREVER); // Grab current frame and update its buffer currentuniformindex = (currentuniformindex + 1) % 3; [self updateuniformresource: myuniformbuffers[currentuniformindex]]; // Encode commands and bind uniform buffer for GPU access // Schedule frame completion handler [commandbuffer addcompletedhandler:^(id<mtlcommandbuffer> commandbuffer) { // GPU work is complete. Signal the Semaphore to start CPU work dispatch_semaphore_signal(frameboundarysemaphore); }]; // Finalize and commit frame to GPU [commandbuffer commit];

295 // Wait until inflight frame is completed dispatch_semaphore_wait(frameboundarysemaphore, DISPATCH_TIME_FOREVER); // Grab current frame and update its buffer currentuniformindex = (currentuniformindex + 1) % 3; [self updateuniformresource: myuniformbuffers[currentuniformindex]]; // Encode commands and bind uniform buffer for GPU access // Schedule frame completion handler [commandbuffer addcompletedhandler:^(id<mtlcommandbuffer> commandbuffer) { // GPU work is complete. Signal the Semaphore to start CPU work dispatch_semaphore_signal(frameboundarysemaphore); }]; // Finalize and commit frame to GPU [commandbuffer commit];

296 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

297 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

298 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

299 Render Pass Descriptor Render Pass Descriptor Color Depth Command Buffer Stencil

300 Render Pass Descriptor Render Pass Descriptor Render Command Encoder Color Depth Command Buffer Stencil

301 Render Pass Setup // Metal Render Pass descriptor MTLRenderPassDescriptor * desc = [MTLRenderPassDescriptor new]; desc.colorattachment[0].texture = mycolortexture; desc.depthattachment.texture = mydepthtexture; id <MTLRenderCommandEncoder> encoder = [commandbuffer rendercommandencoderwithdescriptor: desc];

302 Render Pass Setup // Metal Render Pass descriptor MTLRenderPassDescriptor * desc = [MTLRenderPassDescriptor new]; desc.colorattachment[0].texture = mycolortexture; desc.depthattachment.texture = mydepthtexture; id <MTLRenderCommandEncoder> encoder = [commandbuffer rendercommandencoderwithdescriptor: desc];

303 Render Pass Load and Store Actions Render Pass Descriptor Render Command Encoder Color Depth Command Buffer Stencil Load Action Store Action

304 Render Pass Load and Store Actions Load Action Draw Store Action Color Depth

305 Render Pass Load and Store Actions Load Action Draw Store Action Clear Color Clear Depth

306 Render Pass Load and Store Actions Load Action Draw Store Action Clear Color Clear Depth

307 Render Pass Load and Store Actions Load Action Draw Store Action Clear Store Color Clear Don t care Depth

308 Render Pass Load and Store Actions // Color attachment Load and Store Actions MTLRenderPassDescriptor * desc = [MTLRenderPassDescriptor new]; desc.colorattachment[0].texture = mycolortexture; desc.colorattachment[0].loadaction = MTLLoadActionClear; desc.colorattachment[0].clearcolor = MTLClearColorMake(0.39f, 0.34f, 0.53f, 1.0f); desc.colorattachment[0].storeaction = MTLStoreActionStore; id <MTLRenderCommandEncoder> encoder = [commandbuffer rendercommandencoderwithdescriptor: desc];

309 Render Pass Load and Store Actions // Color attachment Load and Store Actions MTLRenderPassDescriptor * desc = [MTLRenderPassDescriptor new]; desc.colorattachment[0].texture = mycolortexture; desc.colorattachment[0].loadaction = MTLLoadActionClear; desc.colorattachment[0].clearcolor = MTLClearColorMake(0.39f, 0.34f, 0.53f, 1.0f); desc.colorattachment[0].storeaction = MTLStoreActionStore; id <MTLRenderCommandEncoder> encoder = [commandbuffer rendercommandencoderwithdescriptor: desc];

310 Render Pass Setup // Color attachment Load and Store Actions MTLRenderPassDescriptor * desc = [MTLRenderPassDescriptor new]; desc.colorattachment[0].texture = mycolortexture; desc.colorattachment[0].loadaction = MTLLoadActionClear; desc.colorattachment[0].clearcolor = MTLClearColorMake(0.39f, 0.34f, 0.53f, 1.0f); desc.colorattachment[0].storeaction = MTLStoreActionStore; id <MTLRenderCommandEncoder> encoder = [commandbuffer rendercommandencoderwithdescriptor: desc];

311 Rendering with OpenGL Render Targets glbindframebuffer(gl_framebuffer, myframebuffer); Shaders gluseprogram(myprogram); Vertex Buffers glbindbuffer(gl_array_buffer, myvertexbuffer); Uniforms glbindbuffer(gl_uniform_buffer, myuniforms); Textures glbindtexture(gl_texture_2d, mycolortexture); Draws gldrawarrays(gl_triangles, 0, numvertices);

312 Rendering with OpenGL Render Targets glbindframebuffer(gl_framebuffer, myframebuffer); Shaders gluseprogram(myprogram); Vertex Buffers glbindbuffer(gl_array_buffer, myvertexbuffer); Uniforms glbindbuffer(gl_uniform_buffer, myuniforms); Textures glbindtexture(gl_texture_2d, mycolortexture); Draws gldrawarrays(gl_triangles, 0, numvertices);

313 Rendering with OpenGL Render Targets glbindframebuffer(gl_framebuffer, myframebuffer); Shaders gluseprogram(myprogram); Vertex Buffers glbindbuffer(gl_array_buffer, myvertexbuffer); Uniforms glbindbuffer(gl_uniform_buffer, myuniforms); Textures glbindtexture(gl_texture_2d, mycolortexture); Draws gldrawarrays(gl_triangles, 0, numvertices);

314 Rendering with OpenGL Render Targets glbindframebuffer(gl_framebuffer, myframebuffer); Shaders gluseprogram(myprogram); Vertex Buffers glbindbuffer(gl_array_buffer, myvertexbuffer); Uniforms glbindbuffer(gl_uniform_buffer, myuniforms); Textures glbindtexture(gl_texture_2d, mycolortexture); Draws gldrawarrays(gl_triangles, 0, numvertices);

315 Rendering with OpenGL Render Targets glbindframebuffer(gl_framebuffer, myframebuffer); Shaders gluseprogram(myprogram); Vertex Buffers glbindbuffer(gl_array_buffer, myvertexbuffer); Uniforms glbindbuffer(gl_uniform_buffer, myuniforms); Textures glbindtexture(gl_texture_2d, mycolortexture); Draws gldrawarrays(gl_triangles, 0, numvertices);

316 Rendering with OpenGL Render Targets glbindframebuffer(gl_framebuffer, myframebuffer); Shaders gluseprogram(myprogram); Vertex Buffers glbindbuffer(gl_array_buffer, myvertexbuffer); Uniforms glbindbuffer(gl_uniform_buffer, myuniforms); Textures glbindtexture(gl_texture_2d, mycolortexture); Draws gldrawarrays(gl_triangles, 0, numvertices);

317 Rendering with OpenGL Render Targets glbindframebuffer(gl_framebuffer, myframebuffer); Shaders gluseprogram(myprogram); Vertex Buffers glbindbuffer(gl_array_buffer, myvertexbuffer); Uniforms glbindbuffer(gl_uniform_buffer, myuniforms); Textures glbindtexture(gl_texture_2d, mycolortexture); Draws gldrawarrays(gl_triangles, 0, numvertices);

318 Rendering with Metal Render Targets encoder = [commandbuffer rendercommandencoderwithdescriptor:descriptor]; Shaders [encoder setpipelinestate:mypipeline]; Vertex Buffers [encoder setvertexbuffer:myvertexdata offset:0 atindex:0]; [encoder setvertexbuffer:myuniforms offset:0 atindex:1]; Uniforms [encoder setfragmentbuffer:myuniforms offset:0 atindex:1]; Textures [encoder setfragmenttexture:mycolortexture atindex:0]; Draws [encoder drawprimitives:mtlprimitivetypetriangle vertexstart:0 vertexcount:numvertices]; [encoder endencoding];

319 Rendering with Metal Render Targets encoder = [commandbuffer rendercommandencoderwithdescriptor:descriptor]; Shaders [encoder setpipelinestate:mypipeline]; Vertex Buffers [encoder setvertexbuffer:myvertexdata offset:0 atindex:0]; [encoder setvertexbuffer:myuniforms offset:0 atindex:1]; Uniforms [encoder setfragmentbuffer:myuniforms offset:0 atindex:1]; Textures [encoder setfragmenttexture:mycolortexture atindex:0]; Draws [encoder drawprimitives:mtlprimitivetypetriangle vertexstart:0 vertexcount:numvertices]; [encoder endencoding];

320 Rendering with Metal Render Targets encoder = [commandbuffer rendercommandencoderwithdescriptor:descriptor]; Shaders [encoder setpipelinestate:mypipeline]; Vertex Buffers [encoder setvertexbuffer:myvertexdata offset:0 atindex:0]; [encoder setvertexbuffer:myuniforms offset:0 atindex:1]; Uniforms [encoder setfragmentbuffer:myuniforms offset:0 atindex:1]; Textures [encoder setfragmenttexture:mycolortexture atindex:0]; Draws [encoder drawprimitives:mtlprimitivetypetriangle vertexstart:0 vertexcount:numvertices]; [encoder endencoding];

321 Rendering with Metal Render Targets encoder = [commandbuffer rendercommandencoderwithdescriptor:descriptor]; Shaders [encoder setpipelinestate:mypipeline]; Vertex Buffers [encoder setvertexbuffer:myvertexdata offset:0 atindex:0]; [encoder setvertexbuffer:myuniforms offset:0 atindex:1]; Uniforms [encoder setfragmentbuffer:myuniforms offset:0 atindex:1]; Textures [encoder setfragmenttexture:mycolortexture atindex:0]; Draws [encoder drawprimitives:mtlprimitivetypetriangle vertexstart:0 vertexcount:numvertices]; [encoder endencoding];

322 Rendering with Metal Render Targets encoder = [commandbuffer rendercommandencoderwithdescriptor:descriptor]; Shaders [encoder setpipelinestate:mypipeline]; Vertex Buffers [encoder setvertexbuffer:myvertexdata offset:0 atindex:0]; [encoder setvertexbuffer:myuniforms offset:0 atindex:1]; Uniforms [encoder setfragmentbuffer:myuniforms offset:0 atindex:1]; Textures [encoder setfragmenttexture:mycolortexture atindex:0]; Draws [encoder drawprimitives:mtlprimitivetypetriangle vertexstart:0 vertexcount:numvertices]; [encoder endencoding];

323 Rendering with Metal Render Targets encoder = [commandbuffer rendercommandencoderwithdescriptor:descriptor]; Shaders [encoder setpipelinestate:mypipeline]; Vertex Buffers [encoder setvertexbuffer:myvertexdata offset:0 atindex:0]; [encoder setvertexbuffer:myuniforms offset:0 atindex:1]; Uniforms [encoder setfragmentbuffer:myuniforms offset:0 atindex:1]; Textures [encoder setfragmenttexture:mycolortexture atindex:0]; Draws [encoder drawprimitives:mtlprimitivetypetriangle vertexstart:0 vertexcount:numvertices]; [encoder endencoding];

324 Rendering with Metal Render Targets encoder = [commandbuffer rendercommandencoderwithdescriptor:descriptor]; Shaders [encoder setpipelinestate:mypipeline]; Vertex Buffers [encoder setvertexbuffer:myvertexdata offset:0 atindex:0]; [encoder setvertexbuffer:myuniforms offset:0 atindex:1]; Uniforms [encoder setfragmentbuffer:myuniforms offset:0 atindex:1]; Textures [encoder setfragmenttexture:mycolortexture atindex:0]; Draws [encoder drawprimitives:mtlprimitivetypetriangle vertexstart:0 vertexcount:numvertices]; [encoder endencoding];

325 Rendering with Metal Render Targets encoder = [commandbuffer rendercommandencoderwithdescriptor:descriptor]; Shaders [encoder setpipelinestate:mypipeline]; Vertex Buffers [encoder setvertexbuffer:myvertexdata offset:0 atindex:0]; [encoder setvertexbuffer:myuniforms offset:0 atindex:1]; Uniforms [encoder setfragmentbuffer:myuniforms offset:0 atindex:1]; Textures [encoder setfragmenttexture:mycolortexture atindex:0]; Draws [encoder drawprimitives:mtlprimitivetypetriangle vertexstart:0 vertexcount:numvertices]; [encoder endencoding];

326 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

327 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

328 Application Renderer Metal API Command Encoder Command Buffer Display Textures Buffers Pipelines Command Queue Device GPU Render Objects

329 Display Drawables and presentation Drawables Textures for on screen display Each frame MTKView provides Drawable texture Render pass descriptor setup with the drawable Render to drawables like any other texture Present drawable when done rendering

330 // Render offscreen passes... // Acquire a render pass descriptor generated from the drawable s texture MTLRenderPassDescriptor* renderpassdescriptor = view.currentrenderpassdescriptor; // encode your on-screen render passes id <MTLRenderCommandEncoder> rendercommandencoder = [commandbuffer rendercommandencoderwithdescriptor:renderpassdescriptor]; // Encode render commands... [rendercommandencoder endencoding]; // Register the drawable presentation [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

331 // Render offscreen passes... // Acquire a render pass descriptor generated from the drawable s texture MTLRenderPassDescriptor* renderpassdescriptor = view.currentrenderpassdescriptor; // encode your on-screen render passes id <MTLRenderCommandEncoder> rendercommandencoder = [commandbuffer rendercommandencoderwithdescriptor:renderpassdescriptor]; // Encode render commands... [rendercommandencoder endencoding]; // Register the drawable presentation [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

332 // Render offscreen passes... // Acquire a render pass descriptor generated from the drawable s texture MTLRenderPassDescriptor* renderpassdescriptor = view.currentrenderpassdescriptor; // encode your on-screen render passes id <MTLRenderCommandEncoder> rendercommandencoder = [commandbuffer rendercommandencoderwithdescriptor:renderpassdescriptor]; // Encode render commands... [rendercommandencoder endencoding]; // Register the drawable presentation [commandbuffer presentdrawable:view.currentdrawable]; [commandbuffer commit];

333 Build Initialize Render Shaders Devices and Queues Command Buffers Render Objects Resource Updates Render Encoders Display

334 Incrementally Porting Create shared Metal/OpenGL textures using IOSurface or CVPixelBuffer Render to texture on one API and read in the other Can enable mixed Metal/OpenGL applications Sample code available

335 Going Further Multithreading Metal is designed to facilitate Multithreading Consider multithreading if application is CPU bound Encode multiple command buffers simultaneously Split single render pass using MTLParalllelCommandEncoder

336 Going Further Staying on the GPU Metal natively supports compute Performance benefits Reduces CPU utilization Reduces GPU-CPU synchronization points Free s data bandwidth to the GPU New algorithms possible Particle systems, physics, object culling

337 More Metal Features

338 Sharable Textures Tile shaders Tessellation Resource Heaps Indirect Command Buffers Raster Order Groups Programmable Sample Positions Events Layered Rendering Argument Buffers Image Blocks Compute More Metal Features Indirect Dispatch SIMD Group operators Typed Buffers Framebuffer Fetch Metal Performance Shaders Function Specialization Multi Viewport Rendering Texture Arrays Array of Samplers Wide Color Resource Views Memoryless Render Targets

339 Developer Tools Debug and optimize your applications Xcode contains an advanced set of GPU tools Enable Metal's API validation layer On by default when target run from Xcode

340

341

342

343

344