Open Standards for Today s Gaming Industry

Transcription

1 Copyright Khronos Group Page 1 Open Standards for Today s Gaming Industry Erik Noreke VP of Business Development, Khronos Group

2 Gaming Evolution Copyright Khronos Group Page 3 DESKTOP MOBILE New high performance games continue to evolve on high-end platforms Mobile is the new platform for game innovation. Mobile APIs unlock hardware acceleration and conserve battery life, making portable gaming possible Diverse platforms mobile, TV, embedded means HTML5 will become increasingly important as a universal gaming platform

3 Courtesy of PC Gaming Alliance Copyright Khronos Group Page 4

4 Courtesy of PC Gaming Alliance Copyright Khronos Group Page 5

5 US$28.7B gaming revenues generated on platforms supporting Khronos native graphics and compute APIs Courtesy of PC Gaming Alliance Copyright Khronos Group Page 6

6 Copyright Khronos Group Page 7 OpenGL 3D API Family Tree Fixed function 3D Pipeline Programmable vertex and fragment shaders ES3 is backward compatible so new features can be added incrementally Mobile 3D OpenGL ES 1.1 Content OpenGL ES 2.0 Content WebGL 1.0 OpenGL ES 3.0 Content WebGL-Next OpenGL ES 1.1 OpenGL ES 2.0 OpenGL ES 3.0 OpenGL ES 1.0 ES-Next OpenGL 1.3 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.1 OpenGL 3.3 OpenGL 4.2 OpenGL 4.3 OpenGL 4.4 OpenGL 3.0 OpenGL 3.2 OpenGL 4.0 OpenGL 4.1 GL-Next Desktop 3D OpenGL 4.4 is a superset of DX

7 Copyright Khronos Group Page 8 Mobile OpenCL Shipping Android ICD extension released in latest extension specification - OpenCL implementations can be discovered and loaded as a shared object Multiple implementations shipping in Android NDK - ARM, Imagination, Vivante, Qualcomm, Samsung

8 Active Gamers by Platform Courtesy of PC Gaming Alliance CAMERA Copyright Khronos Group Page 9

9 Sensors & Cameras Single sensor RGB cameras are just the start of the mobile visual revolution - IR sensors LEAP Motion, eye-trackers Multi-sensors: Stereo pairs -> Plenoptic array -> Depth cameras - Stereo pair can enable object scaling and enhanced depth extraction Hybrid visual sensing solutions - Different sensors mixed for different distances and lighting conditions Dual Camera LG Electronics Plenoptic Array Pelican imaging Capri Structured Light 3D Camera PrimeSense Copyright Khronos Group Page 10

10 Copyright Khronos Group Page 11 Typical Imaging Pipeline Pre- and Post-processing can be done on CPU, GPU, DSP ISP controls camera via 3A algorithms Auto Exposure (AE), Auto White Balance (AWB), Auto Focus (AF) ISP may be a separate chip or within Application Processor Lens, sensor, aperture control Bayer Pre-processing 3A Image Signal Processor (ISP) RGB/YUV Postprocessing App CMOS sensor Color Filter Array Lens Need for advanced camera control API! CAMERA

11 Copyright Khronos Group Page 12 Low Power Environment Scanning Many sensor use cases would consume too much power to be running 24/7 - Environment aware use cases have to be very low power Scanners - very low power, always on, detect things in the environment - Trigger the next level of processing capability ARM 7 1 MIP and accelerometers can detect someone in the vicinity DSP Low power activation of camera to detect someone in field of view GPU GPU acceleration for precision gesture processing

12 Copyright Khronos Group Page 13 StreamInput Sensor Fusion Stack Applications Platforms can provide increased access to improved sensor data stream driving faster, deeper sensor usage by applications OS Sensor OS APIs (E.g. Android SensorManager or ios CoreMotion) Middleware (E.g. Augmented Reality engines, gaming engines) Middleware engines need platformportable access to native, low-level sensor data stream StreamInput implementations compete on sensor stream quality, reduced power consumption, environment triggering and context detection enabling sensor subsystem vendors to increased ADDED VALUE Sensor Sensor Low-level native API defines access to fused sensor data stream and context-awareness Sensor Hub Sensor Hub Hardware transport interfaces are defined by each system, e.g. IIO or HID sensor Mobile or embedded platforms without sensor fusion APIs can provide direct application access to StreamInput

13 Worldwide Game Users 2012 (Active and Casual) 500 WORLDWIDE GAME USERS BY REGION AND PLATFORM (IN MILLIONS) N.America Europe Asia Other Console PC Dedicated Handheld Mobile Courtesy of PC Gaming Alliance Copyright Khronos Group Page 14

14 Copyright Khronos Group Page 15 Cross-OS, Cross Platform Portability HTML/CSS HTML/CSS HTML/CSS HTML5 provides cross platform portability. GPU accessibility through WebGL available soon on ~90% mobile systems SDK Dalvik (Java) Objective C C# Preferred development environments not designed for portability C/C++ DirectX Native code is portablebut apps must cope with different available APIs and libraries

15 WebGL Availability in Browsers Much WebGL content uses three.js library: - Microsoft where you have IE11, you have WebGL turned on by default and working all the time - Microsoft - WebGL also enabled for Windows applications - web app framework and web view - Apple - WebGL must be explicitly turned on MAC Safari and only exposed on ios for iads - Chrome OS - WebGL is the only cross-platform API to program the GPU - Google IO announcement - Chrome on Android will soon launch with WebGL Copyright Khronos Group Page 16

16 WebCL Parallel Computing for the Web JavaScript bindings to hardware acceleration - Giving performance gaming in a web browser Copyright Khronos Group Page 17

17 Remember Copyright Me Khronos image Group courtesy 2013 of - Page CapCom 18

18 3D Needs a Transmission Format! Compression and streaming of 3D assets becoming essential - Mobile and connected devices need access to increasingly large asset databases 3D is the last media type to define a compressed format - 3D is more complex diverse asset types and use cases Needs to be royalty-free - Avoid an internet video codec war scenario Eventually enable hardware implementations of successful codecs - High-performance and low power but pragmatic adoption strategy is key Audio Video Images 3D MP3 H.264 JPEG?! An effective and widely adopted codec ignites previously unimagined opportunities for a media type Copyright Khronos Group Page 19

19 Copyright Khronos Group Page 20 gltf Goals Binary file format for efficient transmission for 3D assets - Reduce network bandwidth and minimize client processing overhead Run-time neutral - DO NOT IMPLY OR MANDATE ANY RUN-TIME BEHAVIOR - Can be used by any app or run-time usually WebGL accelerated Scalable to handle compression and streaming - Though baseline format does not include compression Direct load efficiency for WebGL - Little or NO processing to drop gltf data into WebGL client Carry conditioned data from any authoring format - Prototyping and optimizing efficient handling of COLLADA assets Authoring Playback A standards-based content pipeline for rich native and Web 3D applications

20 Copyright Khronos Group Page 21 Game On! Gaming and Animation is a complex and expanding application domain and coherent multiple standards across multiple domains are needed to enable the market Khronos family of standards allow you to focus your efforts on game development, not porting between platforms Now is a great time to get involved with the standards initiatives that effect your business and game development erik.noreke@khronos.org

21 Copyright Khronos Group Page 22

22 Copyright Khronos Group Page 23 WebGL First Wave Application Categories Maps and Navigation Modeling Tools and Repositories Games 3D Printing Visualization Music Videos and Promotion Education Photo Editors Music Visualizers Vision/Video Processing

23 Copyright Khronos Group Page 24 COLLADA and gltf Open Source Ecosystem OpenCOLLADA Importer/Exporter and COLLADA Conformance Tests On GitHUB Tool Interop COLLADA2GLTF Translator Other authoring formats Web-based Tools Pervasive WebGL deployment Three.js gltf Importer. Rest3D initiative

24 Copyright Khronos Group Page 25 Google Maps All rendering (2D and 3D) in Google Maps uses WebGL

25 Copyright Khronos Group Page 26 WebCL Parallel Computing for the Web JavaScript bindings to OpenCL APIs - Enables initiation of Kernels written in OpenCL C within the browser Bindings stay close to the OpenCL standard - Maximum flexibility to provide a foundation for higher-level middleware - Minimal language modifications for 100% security and app portability - E.g. Mapping of CL memory objects into host memory space is not supported API definition underway public draft released - Compelling use cases - Physics engines for WebGL games - Image and video editing in browser River Trail JavaScript binding to OpenCL APIs to enable initiation of OpenCL C kernels Data-parallel language extensions to JavaScript for offload at run-time

26 Copyright Khronos Group Page 27 Leveraging Proven Native APIs into HTML5 Khronos and W3C liaison - Leverage proven native API investments into the Web - Fast API development and deployment - Designed by the hardware community - Familiar foundation reduces developer learning curve HTML Canvas WebVX? Vision Processing WebStream? Sensor Fusion WebCAM(!) Camera control and video processing JavaScript Path Rendering Camera Control Native Native APIs shipping or Khronos working group JavaScript API shipping, acceleration being developed or work underway Possible future JavaScript APIs or acceleration

27 Copyright Khronos Group Page 32 OpenGL ES 3.0 Highlights Better looking, faster performing games and apps at lower power - Incorporates proven features from OpenGL 3.3 / 4.x - 32-bit integers and floats in shader programs - NPOT, 3D textures, depth textures, texture arrays - Multiple Render Targets for deferred rendering, Occlusion Queries - Instanced Rendering, Transform Feedback Make life better for the programmer - Tighter requirements for supported features to reduce implementation variability Backward compatible with OpenGL ES OpenGL ES 2.0 apps continue to run unmodified Standardized Texture Compression - #1 developer request!

28 Copyright Khronos Group Page 33 Accelerating OpenGL Innovation Bringing state-of-theart functionality to cross-platform graphics OpenGL 4.3 OpenGL 4.4 OpenGL 4.2 OpenGL 4.1 OpenGL 3.3/4.0 OpenGL 3.2 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 OpenGL DirectX 9.0c DirectX 10.0 DirectX 10.1 DirectX 11 DirectX 11.1

29 Copyright Khronos Group Page 34 OpenGL 4.3 Compute Shaders Execute algorithmically general-purpose GLSL shaders - Can operate on uniforms, images and textures Process graphics data in the context of the graphics pipeline - Easier than interoperating with a compute API IF processing close to the pixel Standard part of all OpenGL 4.3 implementations - Matches DX11 DirectCompute functionality Physics AI Simulation Ray Tracing Imaging Global Illumination

30 Copyright Khronos Group Page 35 OpenGL Conformance Test Suite released! Conformance submissions are required for GL 4.4 implementations encouraged for earlier driver versions Shared codebase with OpenGL ES 3.0 CTS additional desktop-specific tests Core profile functionality Enhancements underway to add more coverage

31 Copyright Khronos Group Page 36 OpenCL and OpenGL Compute Shaders OpenGL compute shaders and OpenCL support distinctly different use cases - OpenCL provides a significantly more powerful and complete compute solution 1. Fine grain compute operations inside OpenGL 2. GLSL Shading Language 3. Execute on single GPU only 1. Full ANSI C programming of heterogeneous CPUs and GPUs 2. Utilize multiple processors 3. Precisely defined IEEE accuracy Enhanced 3D Graphics apps Shaders++ Imaging Video Physics AI Pure compute apps touching no pixels Compute Shaders

32 Gaming Evolution Copyright Khronos Group Page 37 DESKTOP MOBILE New high performance games still evolve on highend platforms Mobile is the new platform for game innovation. Mobile APIs unlock hardware acceleration and conserve battery life, making portable gaming possible Diverse platforms mobile, TV, embedded means HTML5 will become increasingly important as a universal gaming platform