Standards Update. Copyright Khronos Group Page 1

Transcription

1 Standards Update VR/AR, 3D, Web, Vision and Deep Learning Neil Trevett Khronos President NVIDIA VP Developer Copyright Khronos Group Page 1

2 Copyright Khronos Group Page 2 Khronos APIs Connect Software to Silicon Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access hardware acceleration for 3D graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing

3 Copyright Khronos Group Page 3 The Need for VR Standards Vertically Integrated One-off Solutions High-volume Consumer Platforms Interoperability standards so content can be easily created to run on all VR devices! Consumer platforms have intense need for content to drive demand - can t afford silo d content! Consumer confidence will my device run the content I want? Avoid Betamax vs. VHS confusion! VR/AR will build on a constellation of standards HMD Connectivity Wireless connectivity and security VR Video Formats Acceleration APIs VR in the Browser

4 Copyright Khronos Group Page 4 The Need for Standard Acceleration APIs Without API Standards With API Standards Platform Fragmentation Application Portability Everything runs on CPU Silicon Acceleration Standard Acceleration APIs provide PERFORMANCE, POWER AND PORTABILITY

5 Consequences of Not Enough Acceleration Copyright Khronos Group Page 5

6 Copyright Khronos Group Page 6 Native Acceleration APIs for VR/AR Vision sensor(s) Tracking and Positioning Geometric scene reconstruction Semantic scene understanding (Neural Networks) Download 3D augmentation object and scene data VR/AR Application Generate Low Latency 3D Content and Augmentations Interact with sensor, haptic and display devices

7 Copyright Khronos Group Page 7 Khronos 3D APIs Powering AR/VR Rendering Rich Desktop VR Functionality OpenGL 4.5 released August D on millions of mobile VR devices OpenGL ES 3.2 released August 2015 The future of explicit, high-performance, low-latency VR Rendering Vulkan 1.0 released February 2016 Available on diverse platforms Powering WebVR in browsers WebGL 2.0 Released February 2017

8 Copyright Khronos Group Page 8 Vulkan and New Generation 3D APIs Only Windows 10 Only Apple Cross Platform 7

9 Copyright Khronos Group Page 9 Vulkan Explicit GPU Control Vulkan = high performance and low latency 3D and GPU Compute Ideal for VR/AR applications Complex drivers cause overhead and inconsistent behavior across vendors Always active error handling Full GLSL preprocessor and compiler in driver OpenGL vs. OpenGL ES Application Single thread per context High-level Driver Abstraction Layered GPU Control Context management Memory allocation Full GLSL compiler Error detection GPU Application Memory allocation Thread management Synchronization Multi-threaded generation of command buffers Thin Driver Explicit GPU Control GPU Vulkan 1.0 provides access to OpenGL ES 3.1 / OpenGL 4.X-class GPU functionality but with increased performance and flexibility Multiple Front-end Compilers GLSL, HLSL etc. SPIR-V pre-compiled shaders Loadable debug and validation layers Resource management offloaded to app: low-overhead, low-latency driver Consistent behavior: no fighting with driver heuristics Validation and debug layers loaded only when needed SPIR-V intermediate language: shading language flexibility Multi-threaded command creation. Multiple graphics, command and DMA queues Unified API across all platforms with feature set flexibility

10 Copyright Khronos Group Page 10 Vulkan Multi-threading Efficiency CPU Thread Command Buffer 1. Multiple threads can construct Command Buffers in parallel Application is responsible for thread management and synch Command Buffer CPU Thread CPU Thread CPU Thread Command Buffer Command Queue GPU Command Buffer CPU Thread 2. Command Buffers placed in Command Queue by separate submission thread CPU Thread Command Buffer Applications can create graphics, compute and DMA command buffers with a general queue model that can be extended to more heterogeneous processing in the future

11 Vulkan Games and Game Engines Dota 2 on Vulkan port of Source 2 ProtoStar demo on Vulkan port of Unreal Engine 4 Talos Principle on Vulkan port of Serious Engine Vulkan support in Unity 5.6 Vulkan support since V1.8 DOOM on Vulkan port of id Tech 6 Vulkan support in V5.4 Copyright Khronos Group Page 11

12 Copyright Khronos Group Page 12 Vulkan Momentum Games Studios publicly confirming that work is ongoing on Vulkan Titles #Vulkan Games on PC = 25 #DX12 Games on PC = 20 As of July All Major GPU Companies shipping Vulkan Drivers for Desktop and Mobile Platforms Mobile, Embedded and Console Platforms Supporting Vulkan Android 7.0 Nintendo Switch Android TV Embedded Linux

13 Copyright Khronos Group Page 13 Latest Vulkan Functionality Explicit Building Blocks for VR Multiview Efficiently Render geometry to multiple surfaces Each with its own viewing parameters E.g. render stereo pairs or environment maps Resource Sharing Share memory and synchronization primitives Works across process and instance boundaries Descriptor Updates Update resource references between draw or compute dispatch calls Efficiently repeatedly update fixed set of resources Vulkan with new extension sets Vulkan released with new extension sets Explicit Building Blocks for Multi-GPU Works with NVIDIA SLI and AMD Crossfire Does NOT support dgpu/igpu Device Group is set of physical devices Acts as single logical device Access separate GPUs only for explicit control Memory allocation and binding resources Command Buffer Recording/Submission Synchronization Enables a variety of operating modes AFR (alternate frame), SFR (Sequential frame) VR SLI Stereo view rendering Shipped beta drivers on the day the specifications were released PLUS building block Vulkan extensions for VRWorks on Maxwell and Pascal

14 Copyright Khronos Group Page 14 Vulkan Future Directions Shader Functionality - Advanced compute to enable richer compute algorithms - Improved performance for capable hardware and more tightly defined behavior - Improved HLSL support - cbuffer layout and offset support - Subgroup operations - Vote, broadcast, shuffle, quad and arithmetic operations Enhanced Windows System Integration - Enhanced presentable image flexibility e.g. partial and app updates - Dedicated memory option for easier interop - Windows Full-screen ownership and memory residency control

15 Copyright Khronos Group Page 15 Vulkan Portability Bridge API Overlap Analysis No API available on all systems!! Mozilla Obisidian WebGL Next Proposal Vulkan Portability Bridge Open source libraries and tools C/C++ Portability API Library + SPIR-V Cross Translators WebGL Next - Lift Portability API to JavaScript and use in WebAssembly natve code - Nexgen graphics and GPU compute for the Web Identify Vulkan features not directly mappable to DX12 and Metal Vulkan extensions for enhanced portability (and Web security and robustness) Vulkan is non-proprietary and is already designed to be highly portable Vulkan Portability Bridge API Diff Spec Metal Shading Language HLSL SPIRV-Cross translate SPIR-V to native shading languages A Portability Solution needs to address APIs and shading languages

16 Copyright Khronos Group Page 16 Android Daydream VR Daydream uses Unity and Unreal - accelerated over Vulkan

17 Copyright Khronos Group Page 17 Native Acceleration APIs for VR/AR Vision sensor(s) Tracking and Positioning Geometric scene reconstruction Semantic scene understanding (Neural Networks) Download 3D augmentation object and scene data VR/AR Application Generate Low Latency 3D Content and Augmentations Interact with sensor, haptic and display devices

18 Copyright Khronos Group Page 18 OpenXR Solving AR/VR Fragmentation VR App 1 VR App 2 VR App 3 VR App 4 Proprietary Engine Device Discovery Device Events Sensor tracking Haptics HMD Parameters VR App 1 VR App 2 VR App 3 VR App 4 Proprietary Engine Application Interface Device Layer VR Device 1 VR Device 2 VR Device 3 VR Device 4 VR Device 5 Installable Drivers VR Device 1 VR Device 2 VR Device 3 VR Device 4 VR Device 5 Before OpenXR VR Market Fragmentation After OpenXR Wide Interoperability of VR Apps and Devices

19 Copyright Khronos Group Page 19 OpenXR Working Group Members Design work started in December 2016 Typically months to develop a V1.0 specification

20 Copyright Khronos Group Page 20 OpenXR and VR Run-times a Win-Win Access to any OpenXR Application Proprietary APIs Proprietary Driver Interfaces Extensions Application Interface VR Run-time Extensions Device Layer Any successful standard encourages and enables healthy industry competition OpenXR will not replace VR run-times or outlaw existing interfaces OpenXR will simply provide cross-vendor APIs that can be exposed by a runtime to access more apps and devices Access to any OpenXR Device OpenXR for portable AR AND VR apps and devices with initial focus on VR

21 Layered Ecosystem and VR in the Web Applications Native VR Apps 3D Web Apps VR Web Apps Web Middleware Native Middleware Browser APIs Native APIs Khronos APIs VR and AR in the Web = The Metaverse Copyright Khronos Group Page 21

22 Copyright Khronos Group Page 22 WebGL Stack Content downloaded from the Web Middleware provides accessibility for non-expert programmers E.g. three.js library Content JavaScript, HTML, CSS,... JavaScript Middleware Low-level WebGL API provide a powerful foundation for a rich JavaScript middleware ecosystem Browser provides WebGL 3D engine alongside other HTML5 technologies - no plug-in required OS Provided Drivers WebGL uses OpenGL ES 2.0 or Angle for OpenGL ES 2.0 over DX9 JavaScript CSS HTML5 Reliable WebGL relies on work by both GPU and Browser Vendors -> Khronos has the right membership to enable that cooperation

23 Copyright Khronos Group Page 23 WebGL Timeline Fixed function Pipeline Programmable Vertex and fragment shaders 32-bit integers and floats NPOT, 3D/depth textures Texture arrays Multiple Render Targets Compute Shaders Tessellation and geometry shaders ASTC Texture Compression Floating point render targets Debug and robustness for security Epic s Rivalry demo using full Unreal Engine 4on mobile Driver Update Silicon Update Pervasive OpenGL ES 2.0 OpenGL and OpenGL ES ship on every desktop and mobile OS Silicon Update 2011 WebGL Driver Update Silicon Update L N + WebGL Next using Vulkan Portability Bridge Conformance Testing is vital for Cross-Platform Reliability WebGL 2.0 conformance tests are very thorough 10x more tests than WebGL 1.0 tests Possible Compute Shader Extension GDC 2017 WebGL 2.0

24 Copyright Khronos Group Page 24 Pervasive WebGL 1.0 WebGL on EVERY major desktop and mobile browser

25 Copyright Khronos Group Page 25 WebGL 2.0 Spec Enhanced visual quality, performance, features - Instancing Multiple render targets Uniform buffers Transform feedback - Multisampled Renderbuffers 3D textures NPOT textures - More texture formats Occlusion queries Vertex array objects - Sampler objects Sync objects Fragment depth Primitive restart WebGL 2.0 now available in Chrome/Firefox! - Chrome: Releasing to desktop and Android - Firefox: Released on all platforms - Edge/Safari: plan to ship WebGL 2.0 Desktop support at 60% - rising fast WebVR Running Over WebGL 2.0

26 Copyright Khronos Group Page 26 Native Acceleration APIs for VR/AR Vision sensor(s) Tracking and Positioning Geometric scene reconstruction Semantic scene understanding (Neural Networks) Download 3D augmentation object and scene data VR/AR Application Generate Low Latency 3D Content and Augmentations Interact with sensor, haptic and display devices

27 Copyright Khronos Group Page 27 gltf Cross-Platform 3D Asset Transmission gltf enables portable, efficient, sophisticated 3D assets for all 3D applications gltf OpenGL Transmission Format Efficient transmission of 3D assets Now gltf 2.0 has PBR! Cool, portable materials Rendering API independence Coming to Microsoft Office! Laugh Engine running on Vulkan

28 Copyright Khronos Group Page 28 gltf Milestones Microsoft joins Specification Editors We Are Here! 7 Months 2012 thru 2014 Dec 2015 Oct 2016 Mar 2017 June 2017 Design Iteration and Multiple Implementations gltf 1.0 Spec Ratified and Released Industry Roadmap Meeting gltf 2.0 Provisional spec for industry freedback gltf 2.0 Spec Finalization Original motivation: standardized way to deliver 3D into WebGL applications Significant Industry Adoption Physically Based Rendering for higher-quality materials and rendering API independence

29 Copyright Khronos Group Page 29 Strong gltf Momentum Publicly Stated Support for gltf

30 Copyright Khronos Group Page 30 gltf 2.0 Scene Description Structure.gltf (JSON) Node hierarchy, PBR material textures, cameras.bin Geometry: vertices and indices Animation: key-frames Skins: inverse-bind matrices.png.jpg... Textures Geometry Texture based PBR materials

31 Copyright Khronos Group Page 31 gltf 2.0 PBR Consistency Across Engines WebGL reference implementation Laugh Engine running on Vulkan

32 Copyright Khronos Group Page 32 Native Acceleration APIs for VR/AR Vision sensor(s) Tracking and Positioning Geometric scene reconstruction Semantic scene understanding (Neural Networks) Download 3D augmentation object and scene data VR/AR Application Generate Low Latency 3D Content and Augmentations Interact with sensor, haptic and display devices

33 Copyright Khronos Group Page 33 Vision and Neural Net Acceleration Challenge Vision/AI Applications Neural Neural Net Net Training Training Neural Neural Frameworks Frameworks Net Net Training Training Frameworks Frameworks Trained Networks Vision and Neural Net Inferencing Runtime Desktop and Cloud Hardware Embedded/Mobile Vision/Inferencing Embedded/Mobile Vision/Inferencing Embedded/Mobile Embedded/Mobile Hardware Hardware Vision/Inferencing Vision/Inferencing Hardware Hardware

34 Copyright Khronos Group Page 34 NNEF - Solving Neural Net Fragmentation Before NNEF NN Training and Inferencing Fragmentation NN Authoring Framework 1 NN Authoring Framework 2 NN Authoring Framework 3 Every Tool Needs an Exporter to Every Accelerator Inference Engine 1 Inference Engine 2 Inference Engine 3 With NNEF- NN Training and Inferencing Interoperability NN Authoring Framework 1 NN Authoring Framework 2 Inference Engine 1 Inference Engine 2 NN Authoring Framework 3 Optimization and processing tools NNEF is a Cross-vendor Neural Net file format Encapsulates network formal semantics, structure, data formats, commonly-used operations (such as convolution, pooling, normalization, etc.) Inference Engine 3

35 Copyright Khronos Group Page 35 NNEF Status and Roadmap V1.0 is under development, will soon start to solicit industry comments - NNEF will form an advisory panel, you are invited today to participate First version will focus on embedded inference - But will allow training as secondary goal Support First cut range of network types - Field is moving very fast but we aim to keep up with developments NNEF Roadmap - Track development of new network types - Address a wider range of applications (outside vision apps) - Increase the expressive power of the format

36 Power Efficiency Copyright Khronos Group Page 36 OpenVX Wide range of vision hardware architectures OpenVX provides a high-level Graph-based abstraction -> Enables Graph-level optimizations! Can be implemented on almost any hardware or processor! -> Portable, Efficient Vision Processing! X100 X10 X1 Dedicated Hardware Vision DSPs GPU Compute Multi-core CPU Computation Flexibility Vision Node Vision Engines GPU Vision Node Vision Node Middleware DSP Vision Processing Graph Applications Hardware Vision Node Software Portability

37 Copyright Khronos Group Page 37 OpenVX Evolution Conformant Implementations Conformant Implementations OpenVX 1.0 Spec released October 2014 New Functionality Expanded Nodes Functionality Enhanced Graph Framework AMD OpenVX Tools - Open source, highly optimized for x86 CPU and OpenCL for GPU - Graph Optimizer looks at entire processing pipeline and removes, replaces, merges functions to improve performance and bandwidth - Scripting for rapid prototyping, without re-compiling, at production performance levels OpenVX 1.1 Spec released May 2016 New Functionality Conditional node execution Feature detection Classification operators Expanded imaging operations Extensions Neural Network Acceleration Graph Save and Restore 16-bit image operation Safety Critical OpenVX 1.1 SC for safety-certifiable systems OpenVX 1.2 Spec released May 2017 New Functionality Under Discussion NNEF Import Programmable user kernels with accelerator offload OpenVX Roadmap

38 Copyright Khronos Group Page 38 OpenVX - Graph-Level Abstraction OpenVX developers express a graph of image operations ( Nodes ) - Using a C API Nodes can be executed on any hardware or processor coded in any language - Implementers can optimize under the high-level graph abstraction Graphs are the key to run-time power and performance optimizations - E.g. Node fusion, tiled graph processing for cache efficiency etc. Camera Input OpenVX Nodes Pyr t Rendering Output RGB Frame Color Conversion YUV Frame Channel Extract Gray Frame Image Pyramid Optical Flow Array of Keypoints Harris Track OpenVX Graph Feature Extraction Example Graph Ftr t-1 Array of Features

39 Copyright Khronos Group Page 39 Simple Edge Detector in OpenVX vx_image input = vxcreateimage(1920, 1080); vx_image output = vxcreateimage(0, 0); vx_image horiz = vxcreatevirtualimage(); vx_image vert = vxcreatevirtualimage(); vx_image mag = vxcreatevirtualimage(); vx_graph g = vxcreategraph(); vxsobel3x3node(g, input, horiz, vert); vxmagnitudenode(g, horiz, vert, mag); vxthresholdnode(g, mag, THRESH, output); status = vxverifygraph(g); status = vxprocessgraph(g); Declare Input and Output Images Declare Intermediate Images Construct the Graph topology Compile the Graph Execute the Graph i h S M m T v o

40 Copyright Khronos Group Page 40 OpenVX 1.2 and Neural Net Extension Convolution Neural Network topologies can be represented as OpenVX graphs - Layers are represented as OpenVX nodes - Layers connected by multi-dimensional tensors objects - Layer types include convolution, activation, pooling, fully-connected, soft-max - CNN nodes can be mixed with traditional vision nodes Import/Export Extension - Efficient handling of network Weights/Biases or complete networks OpenVX will be able to import NNEF files into OpenVX Neural Nets Native Camera Control Vision Node Vision Node CNN Nodes Vision Node Downstream Application Processing An OpenVX graph mixing CNN nodes with traditional vision nodes

41 Copyright Khronos Group Page 41 OpenCL Low-level Parallel Programing Low-level, explicit programming of heterogeneous parallel compute resources - One code tree can be executed on CPUs, GPUs, DSPs and FPGA OpenCL C or C++ language to write kernel programs to execute on any compute device - Platform Layer API - to query, select and initialize compute devices - Runtime API - to build and execute kernels programs on multiple devices The programmer gets to control: - What programs execute on what device - Where data is stored in various speed and size memories in the system - When programs are run, and what operations are dependent on earlier operations OpenCL Kernel OpenCL Code Kernel OpenCL Code Kernel OpenCL Code Kernel Code Kernel code compiled for devices GPU DSP FPGA CPU CPU Devices Runtime API loads and executes kernels across devices CPU Host

42 Copyright Khronos Group Page 42 OpenCL 2.2 Released in May OpenCL 1.2 Becomes industry baseline for heterogeneous parallel computing OpenCL 2.0 Enables new class of hardware SVM Generic Addresses On-device dispatch OpenCL 2.1 SPIR-V 1.0 SPIR-V in Core Kernel Language Flexibility OpenCL 2.2 SPIR-V 1.2 OpenCL C++ Kernel Language Static subset of C++14 Templates and Lambdas SPIR-V 1.2 OpenCL C++ support Pipes Efficient device-scope communication between kernels Code Generation Optimizations - Specialization constants at SPIR-V compilation time - Constructors and destructors of program scope global objects - User callbacks can be set at program release time

43 Copyright Khronos Group Page 43 OpenCL Ecosystem Hardware Implementers Desktop/Mobile/Embedded/FPGA OpenCL 2.2 Top to Bottom C++ 100s of applications using OpenCL acceleration Rendering, visualization, video editing, simulation, image processing, vision and neural network inferencing Single Source C++ Programming Core API and Language Specs Portable Kernel Intermediate Language

44 OpenCL Conformant Implementations 1.0 May Jul Jun Aug Aug May Dec May Feb Mar Dec Jul14 Desktop 2.1 Jun May Jun May Feb Aug Sep13 Mobile 1.2 Mar Apr Apr Nov Apr Dec Sep Nov Jan May13 Embedded 1.2 May Jul13 FPGA 1.0 Dec Aug15 Vendor timelines are first conformant submission for each spec generation Dec08 OpenCL 1.0 Specification Jun10 OpenCL 1.1 Specification Nov11 OpenCL 1.2 Specification Nov13 OpenCL 2.0 Specification Nov15 OpenCL 2.1 Specification Copyright Khronos Group Page 44

45 Copyright Khronos Group Page 45 OpenCL as Language/Library Backend C++ based Neural network framework Language for image processing and computational photography MulticoreWare open source project on Bitbucket Single Source C++ Programming for OpenCL Java language extensions for parallelism Vision processing open source project Compiler directives for Fortran, C and C++ Open source software library for machine learning Hundreds of languages, frameworks and projects using OpenCL to access vendor-optimized, heterogeneous compute runtimes Over 4,000 GitHub repositories using OpenCL: tools, applications, libraries, languages - up from 2,000 two years ago

46 Copyright Khronos Group Page 46 SYCL Ecosystem Single-source heterogeneous programming using STANDARD C++ - Use C++ templates and lambda functions for host & device code - Layerered over OpenCL Fast and powerful path for bring C++ apps and libraries to OpenCL - C++ Kernel Fusion - better performance on complex software than hand-coding - Halide, Eigen, Boost.Compute, SYCLBLAS, SYCL Eigen, SYCL TensorFlow, SYCL GTX - trisycl, ComputeCpp, VisionCpp, ComputeCpp SDK More information at Developer Choice The development of the two specifications are aligned so code can be easily shared between the two approaches C++ Kernel Language Low Level Control GPGPU -style separation of device-side kernel source code and host code Single-source C++ Programmer Familiarity Approach also taken by C++ AMP and OpenMP

47 SPIR-V Ecosystem GLSL HLSL Khronos has open sourced these tools and translators Third party kernel and shader languages glslang MSL HLSL GLSL SPIR-V Cross OpenCL C Front-end OpenCL C++ Front-end SPIR-V Khronos defined and controlled cross-api intermediate language Native support for graphics and parallel constructs 32-bit Word Stream Extensible and easily parsed Retains data object and control flow information for effective code generation and translation SPIR-V (Dis)Assembler SPIR-V Validator Other Intermediate Forms IHV Driver Runtimes LLVM to SPIR-V Bi-directional Translator LLVM Khronos coordinating liaison with Clang/LLVM Community E.g. discussing SPIR-V as supported Clang target Copyright Khronos Group Page 47

48 OpenCL Evolution Discussions SYCL 1.2 C++11 Single source programming Single source C++ programming. Great for supporting C++ apps, libraries and frameworks SYCL 2.2 C++14 Single source programming Industry working to bring Heterogeneous compute to standard ISO C++ C++17 Parallel STL hosted by Khronos Executors for scheduling work Managed pointers or channels for sharing data Hoping to target C++ 20 but timescales are tight OpenCL 1.2 OpenCL 2.1 SPIR-V in Core 2017 OpenCL 2.2 C++ Kernel Language OpenCL 1.2++? Focus on embedded imaging, vision and inferencing Make FP32 optional for DSPs and general power efficiency OpenCL-V Converge Vulkan and OpenCL Copyright Khronos Group Page 48

49 Copyright Khronos Group Page 49 Layered Vision/ Neural Net Ecosystem Implementers may use OpenCL or Vulkan to implement OpenVX nodes on programmable processors OpenVX enables the graph to be extended to include hardware architectures that don t support programmable APIs Vulkan Roadmap Enhanced compute especially useful for vision and inferencing on mobile and Android platforms Application OpenCL Roadmap Flexible precision for widespread deployment on lowcost embedded processors Programmable Vision Processors Dedicated Vision Hardware And then developers can use OpenVX to enable a developer to easily connect those nodes into a graph The OpenVX graph abstraction enables implementers to optimize execution across diverse hardware architectures for optimal power and performance

50 Copyright Khronos Group Page 50 Safety Critical APIs OpenGL SC Fixed function graphics subset Experience and Guidelines OpenGL SC April 2016 Shader programmable pipeline subset New Generation APIs for safety certifiable vision, graphics and compute e.g. ISO and DO-178B/C OpenGL ES Fixed function graphics OpenGL ES Shader programmable pipeline OpenVX SC 1.1 Released 1st May 2017 Restricted deployment implementation executes on the target hardware by reading the binary format and executing the pre-compiled graphs Khronos SCAP Safety Critical Advisory Panel Guidelines for designing APIs that ease system certification. Open to Khronos member AND industry experts OpenCL SC TSG Formed Working on OpenCL SC 1.2 Eliminate Undefined Behavior Eliminate Callback Functions Static Pool of Event Objects

51 Copyright Khronos Group Page 51 Khronos Principles of Organization Hardware APIs need to be open standards to provide a stable foundation for significant silicon product investment and for interoperability between multiple vendors Any company is welcome to join. One company one vote Strong IP Framework to enable ROYALTY-FREE specifications: all members agree not assert patents against conformant implementations Only invest where there is strong industry momentum to ensure industry relevance let Darwinism rule! Software Silicon Any member can propose new standards initiatives Conformance Tests and Adopters Programs for defining conformance, specification integrity and crossvendor portability Non-profit organization - Membership and Adopters fees cover operating, marketing and engineering expenses

52 Copyright Khronos Group Page 52 Khronos Cooperative Framework $$ Khronos Board Strategy, budget and oversight $ API Working Groups (Industry, Academic and Associate members) One vote per industry member Advisory Panels By invitation, no fee. Provide requirements and draft spec feedback $ Adopters Programs Conformance Tests Royalty-Free Specifications Implementations, Tools Documentation, Samples Educator Guidelines Courseware Materials Open and royalty-free. Often open sourced Industry and Community Feedback and contributions on released materials Adopters Build conformant implementation and products Developers Develop applications using the APIs Educators / Certifiers Create Courses Training and Certification

53 Copyright Khronos Group Page 53 Khronos IP and Adopters Framework Working Group releases Conformance Tests for each Ratified Specification All Members Agree to NOT assert their patents against any CONFORMANT implementation of a RATIFIED Specification (no patents need be disclosed) Membership agreement contains mechanisms for members to exclude specific patents (rarely used as license grant is very narrow) Adopters Process defines how to submit Conformance Test Results for Working Group Review Implementer Executes Adopters Agreement to access Adopters Process. Adopters fee for each generation of specification (typically around $20K for unlimited number of products) When a Submission is successfully reviewed that product becomes Conformant -> The Implementation can use the API trademark AND The Implementation becomes protected from patent litigation from other members

54 Copyright Khronos Group Page 54 Khronos Membership Process Any Member Can Propose a Working Group - Board votes to establish Exploratory Group - Exploratory Group creates detailed Statement of Work (SOW) - Board votes to establish a Working Group to execute the SOW WG Passes Finalized Specification to Board - Board votes to Ratify Specification - Specification and Conformance Tests Publicly Released Working Groups - All work is discussed and documented online - One teleconference per week (in English) - Three Khronos Meetings a year - open to all members Anyone with member address can create account for full access to Khronos resources under Khronos NDA - Draft specifications and tests for all working groups - reflectors for each working group Any Company Can Join Khronos as a Member - Board - $75K/year - Contributor - $18K/year - Associate (<20 employees) - $3.5K/year - Academic - $1K/year Any Member Can Propose an Individual for an Advisory Panel - Working Group votes to approve - Invitation sent to induvial to sign Advisory Panel Agreement

55 Copyright Khronos Group Page 55 The Value of Khronos Participation See an early window into the future of the industry technology roadmap before products are developed Have a voice in how key standards evolve to suit your business needs Develop products in parallel with spec drafting for faster time to market Members can ship products faster than non-members Products are aligned with global market needs and trends Gather Industry Requirements for future silicon acceleration Draft Specifications Confidential to Khronos members Publicly Release Specifications and Conformance Tests Non-members Release Products The Khronos standardization process is proven to RAPIDLY generate industry consensus on future hardware acceleration functionality to EFFICIENTLY create new market opportunities

56 Copyright Khronos Group Page 56 Thank you for your time! These slides and further information on all these standards at Khronos - Please let us know if we can help you participate in any Khronos activities! - You are very welcome and we appreciate your input! Please contact us with any comments or questions! - Neil Trevett - ntrevett@nvidia.com