KHRONOS STANDARDS UPDATE. Neil Trevett, GTC, 26 th March 2018

Transcription

1 KHRONOS STANDARDS UPDATE Neil Trevett, GTC, 26 th March 2018

2 Khronos Mission Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standards to enable software to access hardware acceleration for 3D Graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing Copyright Khronos Group Page 2

3 Copyright Khronos Group Page 3 Some Popular Khronos Standards Run-time APIs File Formats GPU 3D and Compute Heterogenous Compute (Parallel Processing) Vision and Inferencing and AR

4 Copyright Khronos Group Page 4 New Generation GPU APIs Non-proprietary, royalty-free open standard By the industry for the industry Portable across multiple platforms - desktop and mobile Modern architecture Low overhead Multi-thread friendly EXPLICIT GPU access for EFFICIENT, LOW-LATENCY, PREDICTABLE performance Vulkan Porting Tools Vulkan is the primary platform 3D API on Android 7.0+

5 Pervasive Vulkan 1.0 Major GPU Companies supporting Vulkan for Desktop and Mobile Platforms Platforms Desktop Virtual Reality Mobile Consoles Media Players (Android 7.0+) Embedded Cloud Services Game Engines Croteam Serious Engine Copyright Khronos Group Page 5

6 Enabling Cross-Platform AAA Titles Publicly announced games as of March 2018 #Vulkan = 34 #DX12 = 27 Vulkan-only AAA Titles on PC AAA on Linux Dota 2 on PC and macos Copyright Khronos Group Page 6

7 Vulkan Launched 7th March 2018 Strengthening the Ecosystem Improved developer tools (SDK, validation/debug layers) More rigorous conformance testing Shader toolchain improvements (size, speed, robustness) Shading language flexibility HLSL and OpenCL C support Vulkan Public Ecosystem Forum February 2016 Vulkan 1.0 Vulkan 1.0 Extensions Maintenance updates plus additional functionality Explicit Building Blocks for Explicit Building Blocks for Homogeneous Multi-GPU Enhanced Windows System Integration Increased Shader Language Flexibility Enhanced Cross-Process and Cross-API Sharing March 2018 Vulkan 1.1 Building Vulkan s Future Deliver complete ecosystem not just specs Listen and prioritize developer needs - including Ray Tracing Drive GPU technology Integration of Proven and New Technology into Core Widening Platform Support Pervasive GPU vendor driver availability Port Vulkan apps to macos/ios and DX12 Open source drivers Detailed specification overview and GDC session videos: Copyright Khronos Group Page 7

8 Copyright Khronos Group Page 8 New Functionality in Vulkan 1.1 Protected Content - Restrict access or copying from resources used for rendering and display - Secure playback and display of protected multimedia content Subgroup Operations - Efficient mechanisms that enable parallel shader invocations to communicate - Wide variety of parallel computation models supported Example Subgroup Operations A subgroup is a set of invocations (tasks) running on a GPU Compute Unit (Note many GPUs typically support subgroup sizes of 32/64 invocations) Active Invocations + X Inactive Invocations Broadcast Shuffle Add Clustered Multiply

9 Copyright Khronos Group Page 9 Bringing Vulkan 1.0 Apps to Apple Platforms Dota 2 running on Mac up to 50% faster than native OpenGL Applications Very little functionality not supported. Worst Case missing functionality: Triangle fans, Separate stencil reference masks Vulkan Events, Allocation callbacks Some texture-specific swizzles Vulkan macos SDK Open source SDK to build, run, and debug applications on macos including validation layer support SPIRV-Cross Convert SPIR-V shaders to platform source formats macos / ios Run-time Maps Vulkan to Metal MoltenVK for macos and ios For macos 10.11, ios 9.0 and up Previously a paid product Now released into OPEN SOURCE Completely free to use - no fees or royalties - including for commercial applications

10 Copyright Khronos Group Page 10 Vulkan Portability Initiative Roadmap Widened Platform Support Diverse open source run-times over additional backends E.g. Mozilla helping to drive gfx-rs for Vulkan over DX12, OpenGL and Metal E.g. DX12 backends useful for porting Vulkan to Xbox and UWP Platforms Enhanced SDK Layers and Tools E.g. Simulation layer to flag use of features not present in a selected target system TODAY Free, open source tools, SDKs and libraries to bring Vulkan 1.0 applications to Apple using Metal Coordinated Subset Management Cross-platform standardized subsetted functionality queries Conformance tests tailored per target what is present must work! Supported subset features will vary per platform and typically be increased over time with underlying platform capabilities

11 Copyright Khronos Group Page 11 SPIR-V Ecosystem Open source tools and translators Third party kernel and shader languages GLSL HLSL glslang DXC SPIR-V Khronos defined cross-api IR Native graphics and parallel compute Easily parsed/extended 32-bit stream Data object/control flow retained for effective code generation/translation MSL HLSL GLSL SPIRV-Cross OpenCL C Front-end OpenCL C++ Front-end SYCL Front-end SPIR-V Optimizations Inlining (exhaustive) Store/Load Elimination Dead Code Elimination Dead Branch Elimination Common Uniform Elimination Loop Unrolling and Constant Folding Common Subexpression Elimination SPIR-V (Dis)Assembler SPIR-V Validator SPIRV-opt SPIRV-remap Additional Intermediate Forms IHV Driver Runtimes LLVM to SPIR-V Bi-directional Translator LLVM Khronos liaising with Clang/LLVM Community E.g. discussing SPIR-V as supported Clang target SPIR-V 1.3 released with Vulkan 1.1

12 Copyright Khronos Group Page 12 Clspv OpenCL C to Vulkan Compiler Experimental collaboration between Google, Codeplay, and Adobe - Successfully tested on over 200K lines of Adobe OpenCL C production code - Released in open source - Tracks top-of-tree LLVM and clang, not a fork Compiles OpenCL C s programming model to Vulkan s SPIR-V execution environment - Proof-of-concept that OpenCL compute can be brought seamlessly to Vulkan OpenCL C Source OpenCL Host Code Existing open source project Clspv Compiler Run-time API Translator Possible future open source project Runtime Increased deployment options for OpenCL developers!

13 Copyright Khronos Group Page 13 Public Vulkan Ecosystem Forum A new public forum to share ecosystem issues and opportunities - and coordinate solutions! Vulkan-related open source projects Vulkan Ecosystem Forum (open to both Khronos members and nonmembers) Vulkan Developer Community Vulkan Working Group and Advisory Panel Being chaired by NVIDIA. Join the Conversation at

14 Copyright Khronos Group Page 14 OpenXR Solving Fragmentation App 1 App 2 App 3 App 4 App 1 App 2 App 3 App 4 Proprietary Engine Proprietary Engine Application Interface Device Layer Device 1 Device 2 Device 3 Device 4 Device 5 Device 1 Device 2 Device 3 Device 4 Device 5 Before OpenXR Market Fragmentation After OpenXR Wide interoperabilityof apps and devices

15 Copyright Khronos Group Page 15 Companies Publicly Supporting OpenXR OpenXR is a collaborative design 1) for cross-platform AR/PR portability 2) Integrating many lessons from proprietary first-generation API designs

16 Copyright Khronos Group Page 16 Input and Haptics Input uses abstracted Input Actions - E.g. Move, Jump, Teleport Many advantages - Existing content can easily use new devices - Mix-and-match multiple input sources to create a unified UI - Easy optional feature support (e.g. body tracking) - Future-proofing for innovation in input devices and form factors /user/hand/left/input/trigger/click (/devices/controllercorp/fancy_controller/ input/trigger/click).../input/button_a/click.../input/trigger/click.../input/grip/value OpenXR Runtime Explode Teleport SpawnKittens XrAction: Teleport

17 Copyright Khronos Group Page 17 OpenXR Viewport Configurations Applications can: - Query for runtime supported Viewport Configurations - Applications can then set the Viewport Configurations that they plan to use - Select and change their active configuration over the lifetime of the session Camera Passthrough AR Stereoscopic / AR Projection CAVE Photo Credit: Dave Pape One Viewport Two Viewports (one per eye) Twelve Viewports (six per eye) /viewport_configuration/ar_mono/magic_window /viewport_configuration/vr/hmd /viewport_configuration/vr_cube/cave_vr

18 Copyright Khronos Group Page 18 OpenXR Frame Timing xrbeginframe - Ready to begin rendering pixels to the active image in our swap chain xrendframe - Finished rendering, and ready to hand off to the compositor for presentation - Takes a predicted display time, and layers to present xrwaitframe - Called before we begin simulation of the next frame. This is responsible for throttling xrbeginframe (make pretties) xrendframe xrwaitframe

19 Copyright Khronos Group Page 19 OpenXR Compositor Compositor is responsible re-projecting, distorting, and displaying generated images to the device - Can have multiple input layers, up to the limit of the platform runtime XrCompositionData: - Type, display time, eye position XrCompositionLayerData: - Swap chain, and current index Compositor

20 OpenXR Development Progress Call for Participation / Exploratory Group Formation Fall F2F, October 2016: Korea Statement of Work / Working Group Formation Winter F2F, January 2017: Vancouver Specification Work Spring F2F, April 2017: Amsterdam Specification Work Interim F2F, July 2017: Washington Defining the MVP Fall F2F, September 2017: Chicago Much more detailed specification overview and GDC session videos: Resolving Implementation Blockers Winterim F2F, November 2017: Washington Present Day Coming Soon Raising Implementation Issues Winter F2F, January 2018: Taipei First Public Information! GDC, March 2018 Provisional Release Implementations Underway! Spec Finalization incorporating implementation experience Conformance Testing and Implementation Ratification and Release Copyright Khronos Group Page 20

21 Copyright Khronos Group Page 21 gltf Cross-Platform 3D Asset Transmission All gltf spec development on open GitHub: OpenGL Transmission Format Efficient transmission of 3D scenes and assets gltf 1.0 Primarily for WebGL Uses GLSL for materials Released December 2015 gltf Physically Based Rendering! Metallic-Roughness and Specular-Glossiness Materials Rendering API independence Released June 2017

22 Copyright Khronos Group Page 22 gltf Recent Highlights Facebook now supports gltf drag and drop models to your feed - Driving exporter/tool demand: FBX2glTF, Modo, Sketchfab has over 150K gltf 2.0 models - Under Creative Commons license Adobe Dimension is adopting gltf - gltf publishing service for interactive marketing materials Not just for the web native engines adopting! - Unity helping open source gltf importer/exporter - Unreal 4.19 has experimental gltf import Focus on ecosystem robustness test, test, test! - gltf-vscode & gltf-validator - gltf-asset-generator - gltf-sample-models Vibrant open-source community GDC session videos:

23 Copyright Khronos Group Page 23 Google Draco Geometry Compression Extension Draco gltf Extension released for geometry compression in February Draco designed and built for compression efficiency and speed - great fit with gltf! Up to 12X Model Compression - With no loss of visual fidelity Google Draco encoders and decoders in open source - C++ source code encoders C++ and JavaScript decoders - Draco compression already in use - gltf pipeline, FBX2glTF, AMD Compressonator, three.js, and gltf sample models

24 Copyright Khronos Group Page 24 gltf Momentum Creation Tools 3D Content 150K gltf Models in Creative Commons! Discover Apps and Engines Sony 3D Creator Oculus Create Experience Paint 3D Collada2gltf gltf-vscode FBX2glTF Drive Demand Users Windows Mixed Reality Home gltf-validator gltf-asset-generator

25 NVIDIA AND KHRONOS API STANDARDS Desktop Kepler and later Nintendo Switch Shield Android TV Embedded Tegra / Jetson NVIDIA shipped Vulkan 1.1 across our product range on day of specification release Driving experimental extensions, and adding Vulkan support to tools Actively working to define and drive OpenXR to enable and complement Works Production-class OpenCL 1.2 on Linux and Windows. Active ongoing investment to further increase efficiency and performance First to ship production OpenVX 1.0. Extended OpenVX used in VisionWorks for Tegra/Jetson 25

26 Copyright Khronos Group Page 26 How to get Involved! Any company or organization is welcome to join Khronos! - For a voice and a vote in any of these standards membership starts at $3,500 OR request an invite to Vulkan, OpenCL, OpenXR Advisory Panels - No fee, execute Khronos NDA and IP Framework, provide requirements and spec feedback Bring your feedback to the Vulkan Ecosystem Forum - Or any of the Khronos open source GitHub repositories Come catch me any time during GTC - Neil Trevett - Additional Khronos-related sessions here at GTC - S8881A - NVIDIA Vulkan 2018 Update Monday 10AM Hilton Almaden 2 - S NVIDIA Update Monday 1PM - 230A - S OpenCL at NVIDIA - Recent Improvements and Future Plans Monday, 5PM 230C - S Advanced Graphics Extensions for Vulkan Tuesday, 4:30PM 230B - SE WebGL, Web and gltf Meetup Tuesday 7PM 210A!! REFRESHMENTS!!