Acceleration Standards for Mobile Augmented Reality
|
|
- Jacob Collins
- 5 years ago
- Views:
Transcription
1 Acceleration Standards for Mobile Augmented Reality Neil Trevett Khronos President Vice President Mobile Content, NVIDIA November 2012 Copyright Khronos Group 2012 Page 1
2 Copyright Khronos Group 2012 Page 2 Accelerating AR to Meet User Expectations Mobile is an enabling platform for Augmented Reality - Mobile SOC, vision and sensor capabilities are expanding quickly But consumer expectations are not being met -.. Vision-based AR must be sub-pixel accurate, 60Hz smooth AND low power Power is now the main challenge for a quality AR user experience - Delivering the needed computation acceleration in a mobile form factor Acceleration APIs can be used to optimize AR performance AND power - OPEN STANDARD APIs can do that with minimized fragmentation and reduced developer costs SOC = System On Chip Complete compute system minus memory and some peripherals
3 Mobile SOC Performance Increases CPU/GPU AGGREGATE PERFORMANCE Tegra 2 Google Nexus 7 HTC One X 5x 2011 Tegra 3 10x WAYNE 50x LOGAN 75x Android Tablets STARK 25x perf increase in next three years Core i5 Core 2 Duo Copyright Khronos Group 2012 Page 3
4 BUT!! Mobile Thermal Design Point Leakage current in latest silicon geometries means that Moore s Law no longer delivers more performance without increasing power 4-5 Screen takes mW 2-4W 10 Screen takes 1-2W Resolution makes a difference! The ipad3 screen takes up to 8W 7 Screen takes 1W 4-7W 6-10W Typical max system power levels before thermal failure Even as battery technology improves - these thermal limits remain 30-90W Copyright Khronos Group 2012 Page 4
5 How to Save Power? Much more expensive to MOVE data than COMPUTE data Energy efficiency must now be key metric during silicon AND software design - Awareness of where data lives, where computation happens, how is it scheduled Need to use hardware acceleration - Lots of processing in parallel - Efficient caching and memory usage - Reduces data movement 32-bit Float Operation 7pJ Send 32-bits Off-chip 50pJ Send 32-bits 2mm 24pJ Write 32-bits to Memory 600pJ For 40nm, 1V process 32-bit Register Write 0.5pJ 32-bit Integer Add 1pJ Copyright Khronos Group 2012 Page 5
6 Copyright Khronos Group 2012 Page 6 Example - Typical Camera ISP Camera ISP (Image Signal Processor) has little or no programmability - Scan-line-based, Data flows through compact hardware pipe - No global memory used to minimize power BUT computational photography apps now want to mix nonprogrammable ISP processing with more flexible GPU or CPU processing - ISP pipelines will provide tap/insertion points to/from CPU/GPU at critical points ~760 math Ops ~42K vals = 670Kb 300MHz
7 Copyright Khronos Group 2012 Page 7 Engineering for Power Dark Silicon - specialized hardware - turned on only when needed - Lots of space for transistors just can t turn them all on at same time - Accelerator blocks increase locality and parallelism to save power compared to programmable processors Be power smart when using programmable processors - Instrumentation for energy-aware compilers and profilers - Dynamic and feedback-driven software power optimization - Power optimizing compiler back-end compilers / installers Smart, holistic use of sensors and peripherals - Wireless modems and networks - Motion sensors, cameras, networking, GPS Apple A5 Significant potion of die is already not CPU/GPU or IO = dedicated hardware
8 Copyright Khronos Group 2012 Page 8 Programmers View of Typical SOC c Unified memory is critical departure from PC architecture Common virtual address space for all units but typically not coherent CPU Complex 1-5 Cortex A9/A15 Cores with NEON L1 Cache L2 Cache Automatic scaling of frequency/ voltage and # cores to meet current software load HD Audio Engine / IO Software configurability 2D/3D GPU Complex Significant programmable acceleration. Fully C-programmable soon cores Graphics Cache Unified Memory Controller for 0.5-2GB DDR3L/LP-DDR3 Peripheral Busses 1-4 Display Controllers Sensor Array Image Signal Processor Video Encoder Decoder Vision Processor (Future) Typically NO or very limited programmable functionality 1-3 Cameras Software configurability Programmable? Or dedicated hardware for power efficiency?
9 Copyright Khronos Group 2012 Page 9 Khronos Connects Software to Silicon ROYALTY-FREE, OPEN STANDARD APIs for advanced hardware acceleration Graphics, video, audio, compute, visual and sensor processing Low level silicon to software interface needed on every platform Defines the forward looking roadmap for the silicon community Shipping on billions of devices across multiple operating systems Rigorous conformance tests for crossvendor consistency Khronos is OPEN for any company to join and participate Acceleration APIs BY the Industry FOR the Industry
10 Mobile Innovation Hot Spots New platform capabilities being driven by SILICON and APIs Vision - Camera as sensor Console-Class 3D Performance, Quality, Controllers and TV connectivity Computational Photography Gesture Processing Augmented Reality Media and Image Streaming Heterogeneous Parallel Processing Sensor Fusion Smart use of sensors to make devices magically context aware location, usage, position HTML5 Bringing acceleration to Web Apps that can be discovered on the Net and run on any platform Copyright Khronos Group 2012 Page 10
11 Example use of Khronos APIs in AR Audio Rendering Positional Sensors Positional and GPS Sensor Data Computer Vision and Tracking Camera Synchronization and sensor fusion Position and Tracking Semantics Synchronize processed camera frame to output displayed frame? Video TAP to Vision Subsystem Camera Processing Application on CPUs Video stream to GPU 3D Rendering and Video Composition On GPU Control Camera, Preprocess and generate video streams Copyright Khronos Group 2012 Page 11
12 3D API Family Tree ES3 is backward compatible so new features can be added incrementally Programmable vertex and fragment shaders Fixed function 3D Pipeline OpenGL ES 2.0 Content OpenGL ES 1.1 Content OpenGL ES 3.0 Content Mobile 3D WebGL 1.0 OpenGL ES 1.1 WebGL-Next OpenGL ES 2.0 OpenGL ES 3.0 ES-Next OpenGL ES 1.0 OpenGL 1.3 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.1 OpenGL 3.3 OpenGL 4.2 OpenGL 4.0 OpenGL 3.0 OpenGL 3.2 OpenGL GL-Next OpenGL 4.3 is a superset of DX11 Desktop 3D 2002 OpenGL Copyright Khronos Group 2012 Page 12
13 OpenGL ES 3.0 Highlights Better looking, faster performing games and apps at lower power - Incorporates proven features from OpenGL 3.3 / 4.x - 32-bit integers and floats in shader programs - NPOT, 3D textures, depth textures, texture arrays - Multiple Render Targets for deferred rendering, Occlusion Queries - Instanced Rendering, Transform Feedback Make life better for the programmer - Tighter requirements for supported features to reduce implementation variability Backward compatible with OpenGL ES OpenGL ES 2.0 apps continue to run unmodified Standardized Texture Compression - #1 developer request! Copyright Khronos Group 2012 Page 13
14 OpenCL Heterogeneous Computing OpenCL Kernel OpenCL Code Kernel OpenCL Code Kernel OpenCL Code Kernel Code A low-level, cross-platform, cross-vendor standard for harnessing all system compute resources C Platform Layer API - Query, select and initialize compute devices CPU Kernel Language Specification - Subset of ISO C99 with language extensions - Well-defined numerical accuracy - IEEE 754 rounding with specified max error - Rich built-in functions: cross, dot, sin, pow, log C Runtime API - Runtime or build-time compilation of kernels - Execute compute kernels across multiple devices CPU OpenCL Kernel OpenCL Code Kernel OpenCL Code Kernel OpenCL Code Kernel Code GPU GPU CPU One code tree can be executed on CPUs or GPUs Copyright Khronos Group 2012 Page 14
15 Copyright Khronos Group 2012 Page 15 OpenMAX AL Streaming Media Framework Enables key video, image stream and camera use cases - Enables optimal hardware acceleration with app portability Create Media Objects to play and process images and video with AV sync - Connect to variety of input and output objects to PLAY and RECORD media Full range of video effects and controls - Including playback rate, post processing, and image manipulation OpenMAX AL includes sophisticated camera controls Sources Analog Radio Camera Audio Input URI Efficient data routing for CPU processing DSrc Data Tap to CPU OpenMAX AL Media Object Processing DSnk Sinks EGLStream to GPU Audio Mix Display Window URI EGLStreams extension enables efficient transfer of image stream to GPU texture memory Memory Memory
16 Copyright Khronos Group 2012 Page 16 Developer Requested Camera Extensions Query camera information - Focal length (fx, fy), principal point (cx, cy), skew (s), image resolution (h, w) - Spatial information of how cameras and sensors are placed on device - Calibration and lens distortion FCAM++ for extensive exposure parameters in single or burst mode - Shutter, aperture, ISO, white balance, frame rate, focus modes, resolution - Preload parameters for each shot in a burst ROI extraction - From wide angle and fish-eye lenses Rolling shutter compensation - Using accelerometer Data output format control - Grayscale, RGB(A), YUV - Access to the raw data e.g. Bayer pattern
17 Copyright Khronos Group 2012 Page 17 OpenVX Vision Hardware Acceleration Layer - Enable hardware vendors to implement accelerated imaging and vision algorithms - For use by high-level libraries or apps Focus on enabling real-time vision - On mobile and embedded systems Diversity of efficient implementations - From programmable processors to dedicated hardware pipelines Dedicated hardware can help make vision processing performant and low-power enough for pervasive always-on use OpenCV open source library Application Open source sample implementation? Other higher-level CV libraries Hardware vendor implementations
18 Copyright Khronos Group 2012 Page 18 Current OpenVX Participants Aiming for specification first draft in early 2013 Itseez is working group chair
19 Possible Implementations of Vision Stack StreamInput combines camera vision with other sensors on device OpenVX does not need the camera, GUI or compute functionality in OpenCV use Khronos API interop instead OpenCV community can progressively port over OpenVX acceleration provided by silicon community Camera input from OpenMAX AL or other camera subsystems Data and event interop with OpenCL and OpenGL ES for display and compute processing Use OpenCL to implement OpenCV OR OpenVX with parallel execution on CPUs/GPUs OpenVX does not duplicate OpenCV functionality JUST provides essential acceleration Dedicated hardware OpenVX can be implemented using dedicated hardware Copyright Khronos Group 2012 Page 19
20 Copyright Khronos Group 2012 Page 20 Portable Access to Sensor Fusion Apps request semantic sensor information StreamInput defines possible requests, e.g. Provide Skeleton Position Am I in an elevator? Standardized Node Intercommunication Universal Timestamps E.g. align samples from camera and other sensors Input Device Input Device Input Device Advanced Sensors Everywhere RGB and depth cameras, multi-axis motion/position, touch and gestures, microphones, wireless controllers, haptics keyboards, mice, track pads Filter Node Filter Node Processing graph provides sensor data stream Utilizes optimized, smart, sensor middleware Apps can gain magical situational awareness Filter Node App Apps Need Sophisticated Access to Sensor Data Without coding to specific sensor hardware
21 Copyright Khronos Group 2012 Page 21 OpenSL ES Advanced Audio OpenSL ES does for audio what OpenGL ES does for graphics - Advanced audio functionality from simple playback to full 3D positional audio Object-based native audio API for simplicity and high performance - Same object framework as OpenMAX AL - Reduces development time Attractive alternative to open source frameworks - Tightly defined specification with full conformance tests - Robust application portability across platforms and OS
22 Copyright Khronos Group 2012 Page 22 EGLStream Streaming Images EGL originally embedded version of WGL - Abstraction layer to window systems and memory buffers Role has expanded to provide API interop data and events - EGLImages single buffers that can be passed between APIs - EGLStreams provides stream of images with AV sync - Cross process EGLStreams Producer and Consumer can be in different processes for performance or security e.g. browser compositing process Android SurfaceTexture is a Java wrapper around EGLStreams - Captures video decode or camera preview to OpenGL ES texture OpenMAX AL Media Player is the EGLStream Producer and controls production of frames. EGLStreams enables and hides details of video frame transport. Enables multiple buffering modes for different uses cases e.g.: FIFO and explicit latch/release OpenGL ES GL_TEXTURE_EXTERNAL is the EGLStream Consumer and converts video format into RGB OpenGL ES texture Camera File URL Etc. MEDIA PLAYER Object EGLStream? GL_TEXTURE_EXTERNAL
23 OS API Adoption Mobile Operating Systems OpenGL ES 2.0 Shipping - Android 2.2 OpenSL ES 1.0 (subset) Shipping Android 2.3 OpenMAX AL 1.0 (subset) Shipping - Android 4.0 EGL 1.4 Shipping under SDK -> NDK Opera and Firefox WebGL now Chrome soon OpenGL 3.2 on MacOS OpenCL 1.1 on MacOS OpenGL ES 2.0 on ios Can enable on MacOS Safari ios5 enables WebGL for iads Microsoft Windows RT: - Only Microsoft native APIs - HTML5 but not yet WebGL Copyright Khronos Group 2012 Page 23
24 Using Android Java for AR Native APIs can supplement Android capabilities now and be absorbed into platform C and Java APIs by Google over time Positional Sensors No advanced camera controls: ROI, burst parameter control, focus modes, format choices MediaSDK Camera SensorManager Positional and GPS Sensor Data Computer Vision and Tracking Camera Processing Java Callback Preview frames Limited fusion and cross sensor synch. No virtual sensors Synchronization and sensor fusion OpenCV through JNI 30FPS but 2-3 frames latency Video stream to GPU SurfaceTexture Position and Tracking Semantics Need synch between input and composited frames JetPlayer Audio Rendering Application on CPUs 3D Rendering and Video Composition On GPU GLSurfaceView Java provides access to EGL-type and OpenGL ES functionality Copyright Khronos Group 2012 Page 24
25 Leveraging Proven Native APIs into HTML5 Leverage native API investments into the Web - Faster API development and deployment - Familiar foundation reduces developer learning curve Khronos and W3C exploring liaison - Multiple potential joint projects WebMAX? Camera control and video processing WebAudio Advanced JavaScript Audio Canvas WebVX? Vision Processing Device and Sensor APIs Device Orientation Working Groups JavaScript Native Native APIs shipping or working group underway JavaScript API shipping or working group underway Possible future JavaScript APIs Copyright Khronos Group 2012 Page 25
26 Expanding Platform Reach for Graphics and Computation Desktop Mobile Production Browsers Shipping with WebGL: Desktop - Chrome, Firefox, Opera, Safari Mobile - Opera and Firefox Apple ios Safari uses WebGL for iads Web Graphics Interop Interop Interop WebGL on majority of production desktops now. WebGL pervasively available on mobile in next 12 months Typed Arrays WebCL will start deploying in next months Compute Full Profile Full Profile and Embedded Profile OpenCL pervasively available on mobile in next months Copyright Khronos Group 2012 Page 26
27 Copyright Khronos Group 2012 Page 27 Khronos and MPEG Standards COLLADA 3D Authoring Format ARAF - AR Application format 3DG 3D Graphics Coding Media Framework OpenMAX Rich Media Framework MPEG-4-2D/3D graphics MPEG-21 - Media Management/Protection MPEG--V - Sensors and actuators MPEG--U - Advanced User interaction MPEG--M - Media oriented middleware Codecs Low level Acceleration APIs for Graphics, Compute, Vision, Sensors Silicon
28 Copyright Khronos Group 2012 Page 28 Need 3D Transmission Standard? Efficient codecs for compressed binary 3D asset blobs - Geometry, textures, materials, animations, physics - Separate form scene graph description e.g. WebGL could use JSON - Perhaps use RESTful APIs and LOD management web services approach? Many initiatives under way time for communication and collaboration? - MPEG 3D Mesh Progressive Streaming (3DMC), Bones Based Animation (BBA) - Google Body compression - Delta and ZigZag encoding - COLLADA2JSON at COLLADA working group - Multiple papers presented at Web3D 2012 An effective and widely adopted codec ignites previously unimagined opportunities for a media type Audio Video Images 3D MP3 AVC PNG/JPEG??
29 Copyright Khronos Group 2012 Page 29 3D Codec Concepts - Transmission Format Specification enables reliable encoding and decoding of 3D Assets - Encoding/Decoding can be independent of server/client - Standardized, widely-used codec catalyzes optimized implementations 3D Asset Server Encode 3D Assets Decode 3D Assets 3D Client Challenges - Complexity and variety of 3D asset types to be processed - Efficiency of general codec versus application-specific encoding schema WebGL browser community suggests to try multiple techniques in open source and JavaScript to inform standardization
30 Copyright Khronos Group 2012 Page 30 Transmission Format Collaboration Exploring multi-sdo collaboration - To accelerate transmission format exploration and development Khronos has drafted liaison MOU to encapsulate this model - Feedback welcome! 4. Each SDO ratifies specification to be royalty free and publicly distributes 1. Members of Participating SDOs can attend liaison working group Liaison Working Group Problem is use of key term specification 2. Liaison Working Group drafts specification + Other SDOs! 3. Liaison Working sends draft spec to each SDO for ratification - using SDO s own ratification process for royalty-free specifications
31 Copyright Khronos Group 2012 Page 31 Summary Advances in SOC silicon processing and associated APIs are enabling significant new mobile use cases including Augmented Reality Holistic cooperation between hardware and software needed to deliver increasing computational loads in a fixed power budget Cooperative API standards enabling mobile OS and apps to tap into the potential of mobile SOC acceleration Sensor and vision processing now just as important as graphics acceleration Industry consensus needed on need and method to create 3D streaming transmission format
WebGL Meetup GDC Copyright Khronos Group, Page 1
WebGL Meetup GDC 2012 Copyright Khronos Group, 2012 - Page 1 Copyright Khronos Group, 2012 - Page 2 Khronos API Ecosystem Trends Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos
More informationSC24/WG9 Liaison Meeting
Copyright Khronos Group, 2011 - Page 1 SC24/WG9 Liaison Meeting Seoul, November 2011 Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page
More informationAR Standards Update Austin, March 2012
AR Standards Update Austin, March 2012 Neil Trevett President, The Khronos Group Vice President Mobile Content, NVIDIA Copyright Khronos Group, 2012 - Page 1 Topics Very brief overview of Khronos Update
More informationKhronos and the Mobile Ecosystem
Copyright Khronos Group, 2011 - Page 1 Khronos and the Mobile Ecosystem Neil Trevett VP Mobile Content, NVIDIA President, Khronos Copyright Khronos Group, 2011 - Page 2 Topics It s not just about individual
More informationCopyright Khronos Group, Page 1. Khronos Overview. Taiwan, February 2012
Copyright Khronos Group, 2012 - Page 1 Khronos Overview Taiwan, February 2012 Copyright Khronos Group, 2012 - Page 2 Khronos - Connecting Software to Silicon Creating open, royalty-free API standards -
More informationMobile AR Hardware Futures
Copyright Khronos Group, 2010 - Page 1 Mobile AR Hardware Futures Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Two Perspectives NVIDIA - Tegra 2 mobile processor Khronos
More informationOpen Standard APIs for Augmented Reality
Copyright Khronos Group 2014 - Page 1 Open Standard APIs for Augmented Reality Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group Copyright Khronos Group 2014 - Page 2 Khronos
More informationOpen API Standards for Mobile Graphics, Compute and Vision Processing GTC, March 2014
Open API Standards for Mobile Graphics, Compute and Vision Processing GTC, March 2014 Neil Trevett Vice President Mobile Ecosystem, NVIDIA President Khronos Copyright Khronos Group 2014 - Page 1 Khronos
More informationNext Generation OpenGL Neil Trevett Khronos President NVIDIA VP Mobile Copyright Khronos Group Page 1
Next Generation OpenGL Neil Trevett Khronos President NVIDIA VP Mobile Ecosystem @neilt3d Copyright Khronos Group 2015 - Page 1 Copyright Khronos Group 2015 - Page 2 Khronos Connects Software to Silicon
More informationOpen Standards for Today s Gaming Industry
Copyright Khronos Group 2013 - Page 1 Open Standards for Today s Gaming Industry Erik Noreke VP of Business Development, Khronos Group Gaming Evolution Copyright Khronos Group 2013 - Page 3 DESKTOP MOBILE
More informationThe State of Gaming APIs
Copyright Khronos Group, 2011 - Page 1 The State of Gaming APIs Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page 2 State of Gaming APIs
More informationWebGL, WebCL and Beyond!
Copyright Khronos Group, 2011 - Page 1 WebGL, WebCL and Beyond! Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page 2 Topics in this Session
More informationCopyright Khronos Group Page 1
Gaming Market Briefing Overview of APIs GDC March 2016 Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem ntrevett@nvidia.com @neilt3d Copyright Khronos Group 2016 - Page 1 Copyright
More informationVision Acceleration. Launch Briefing October Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group
Copyright Khronos Group 2014 - Page 1 Vision Acceleration Launch Briefing October 2014 Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group Copyright Khronos Group 2014 - Page
More informationSIGGRAPH Briefing August 2014
Copyright Khronos Group 2014 - Page 1 SIGGRAPH Briefing August 2014 Neil Trevett VP Mobile Ecosystem, NVIDIA President, Khronos Copyright Khronos Group 2014 - Page 2 Significant Khronos API Ecosystem Advances
More informationOpen Standard APIs for Embedded Vision Processing
Copyright Khronos Group 2014 - Page 1 Open Standard APIs for Embedded Vision Processing Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group Copyright Khronos Group 2014 - Page
More informationCompleting the Multimedia Architecture
Copyright Khronos Group, 2011 - Page 1 Completing the Multimedia Architecture Erik Noreke Chair of OpenSL ES Working Group Chair of OpenMAX AL Working Group Copyright Khronos Group, 2011 - Page 2 Today
More informationOpenMAX AL, OpenSL ES
Copyright Khronos Group, 2011 - Page 1 OpenMAX AL, OpenSL ES Native Multimedia in Android Erik Noreke Chair of OpenMAX AL and OpenSL ES Working Groups Copyright Khronos Group, 2011 - Page 2 Why Create
More informationOpenCL Press Conference
Copyright Khronos Group, 2011 - Page 1 OpenCL Press Conference Tokyo, November 2011 Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page
More informationOpen Standards for AR and VR Neil Trevett Khronos President NVIDIA VP Developer January 2018
Copyright Khronos Group 2018 - Page 1 Open Standards for AR and Neil Trevett Khronos President NVIDIA VP Developer Ecosystem ntrevett@nvidia.com @neilt3d January 2018 Khronos Mission E.g. OpenGL ES provides
More informationKhronos Overview The State of the Art in Open Standards for Visual Computing
Khronos Overview The State of the Art in Open Standards for Visual Computing Neil Trevett Khronos President Vice President Mobile Content, NVIDIA Copyright Khronos Group 2013 - Page 1 Copyright Khronos
More informationGraphics Technology Update
Graphics Technology Update Presented by: Erik Noreke, Khronos Group Vice President of Business Development November 2013 Copyright Khronos Group, 2013 - Page 1 Copyright Khronos Group, 2013 - Page 2 Khronos
More informationOverview and AR/VR Roadmap
Khronos Group Inc. 2018 - Page 1 Overview and AR/ Roadmap Neil Trevett Khronos President NVIDIA VP Developer Ecosystems ntrevett@nvidia.com @neilt3d Khronos Group Inc. 2018 - Page 2 Khronos Connects Software
More informationNeil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group. Copyright Khronos Group Page 1
Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group Copyright Khronos Group 2014 - Page 1 Khronos Standards 3D Asset Handling - 3D authoring asset interchange - 3D asset transmission
More informationWebGL, WebCL and OpenCL
Copyright Khronos Group, 2011 - Page 1 WebGL, WebCL and OpenCL Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page 2 Processor Parallelism
More informationOpen Standards for Vision and AI Peter McGuinness NNEF WG Chair CEO, Highwai, Inc May 2018
Copyright Khronos Group 2018 - Page 1 Open Standards for Vision and AI Peter McGuinness NNEF WG Chair CEO, Highwai, Inc peter.mcguinness@gobrach.com May 2018 Khronos Mission E.g. OpenGL ES provides 3D
More informationEnabling a Richer Multimedia Experience with GPU Compute. Roberto Mijat Visual Computing Marketing Manager
Enabling a Richer Multimedia Experience with GPU Compute Roberto Mijat Visual Computing Marketing Manager 1 What is GPU Compute Operating System and most application processing continue to reside on the
More informationCopyright Khronos Group Page 1
Open Standards and Open Source Together How Khronos APIs Accelerate Fast and Cool Applications Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem Copyright Khronos Group 2015 - Page
More informationThe Benefits of GPU Compute on ARM Mali GPUs
The Benefits of GPU Compute on ARM Mali GPUs Tim Hartley 1 SEMICON Europa 2014 ARM Introduction World leading semiconductor IP Founded in 1990 1060 processor licenses sold to more than 350 companies >
More informationStandards for WebVR. Neil Trevett. Khronos President Vice President Mobile Content,
Standards for WebVR Neil Trevett Khronos President Vice President Mobile Content, NVIDIA ntrevett@nvidia.com, @neilt3d October 2016 Copyright Khronos Group 2016 - Page 1 Khronos Open Standards Software
More informationStreaming Media. Advanced Audio. Erik Noreke Standardization Consultant Chair, OpenSL ES. Copyright Khronos Group, Page 1
Streaming Media Advanced Audio Erik Noreke Standardization Consultant Chair, OpenSL ES Copyright Khronos Group, 2010 - Page 1 Today s Consumer Requirements Rich media applications and UI - Consumer decisions
More informationCopyright Khronos Group 2012 Page 1. OpenCL 1.2. August 2012
Copyright Khronos Group 2012 Page 1 OpenCL 1.2 August 2012 Copyright Khronos Group 2012 Page 2 Khronos - Connecting Software to Silicon Khronos defines open, royalty-free standards to access graphics,
More informationBringing the Power of the GPU to the Web
Copyright Khronos Group 2013 - Page 1 Bringing the Power of the GPU to the Web Neil Trevett Vice President NVIDIA President Khronos Mobile is the New Epicenter of Innovation Copyright Khronos Group 2013
More informationCopyright Khronos Group, Page 1. OpenCL. GDC, March 2010
Copyright Khronos Group, 2011 - Page 1 OpenCL GDC, March 2010 Authoring and accessibility Application Acceleration System Integration Copyright Khronos Group, 2011 - Page 2 Khronos Family of Standards
More informationOpen Standards for Building Virtual and Augmented Realities. Neil Trevett Khronos President NVIDIA VP Developer Ecosystems
Open Standards for Building Virtual and Augmented Realities Neil Trevett Khronos President NVIDIA VP Developer Ecosystems Khronos Mission Asian Members Software Silicon Khronos is an International Industry
More informationUpdate on Khronos Open Standard APIs for Vision Processing Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem
Update on Khronos Open Standard APIs for Vision Processing Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem Copyright Khronos Group 2015 - Page 1 Copyright Khronos Group 2015 - Page
More informationOpenCL Overview. Shanghai March Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group
Copyright Khronos Group, 2012 - Page 1 OpenCL Overview Shanghai March 2012 Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2012 - Page 2 Processor
More informationPress Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem. Copyright Khronos Group Page 1
Press Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem Copyright Khronos Group 2015 - Page 1 Khronos Connects Software to Silicon Open Consortium creating ROYALTY-FREE,
More informationPress Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem. Copyright Khronos Group Page 1
Press Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem Copyright Khronos Group 2015 - Page 1 Khronos Connects Software to Silicon Open Consortium creating ROYALTY-FREE,
More informationCopyright Khronos Group Page 1. Vulkan Overview. June 2015
Copyright Khronos Group 2015 - Page 1 Vulkan Overview June 2015 Copyright Khronos Group 2015 - Page 2 Khronos Connects Software to Silicon Open Consortium creating OPEN STANDARD APIs for hardware acceleration
More informationKhronos Connects Software to Silicon
Press Pre-Briefing GDC 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem All Materials Embargoed Until Tuesday 3 rd March, 12:01AM Pacific Time Copyright Khronos Group 2015 - Page
More informationThe OpenVX Computer Vision and Neural Network Inference
The OpenVX Computer and Neural Network Inference Standard for Portable, Efficient Code Radhakrishna Giduthuri Editor, OpenVX Khronos Group radha.giduthuri@amd.com @RadhaGiduthuri Copyright 2018 Khronos
More informationAccelerating Vision Processing
Accelerating Vision Processing Neil Trevett Vice President Mobile Ecosystem at NVIDIA President of Khronos and Chair of the OpenCL Working Group SIGGRAPH, July 2016 Copyright Khronos Group 2016 - Page
More informationIntroduction to OpenGL ES 3.0
Introduction to OpenGL ES 3.0 Eisaku Ohbuchi Digital Media Professionals Inc. 2012 Digital Media Professionals Inc. All rights reserved. 12/Sep/2012 Page 1 Agenda DMP overview (quick!) OpenGL ES 3.0 update
More informationINTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES. Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp.
INTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp. Computer Vision in Mobile Tegra K1 It s time! AGENDA Use cases categories
More informationNext Generation Visual Computing
Next Generation Visual Computing (Making GPU Computing a Reality with Mali ) Taipei, 18 June 2013 Roberto Mijat ARM Addressing Computational Challenges Trends Growing display sizes and resolutions Increasing
More informationOur Technology Expertise for Software Engineering Services. AceThought Services Your Partner in Innovation
Our Technology Expertise for Software Engineering Services High Performance Computing MultiCore CPU AceThought experts will re-design your sequential algorithms or applications to execute in parallel by
More informationGPGPU on Mobile Devices
GPGPU on Mobile Devices Introduction Addressing GPGPU for very mobile devices Tablets Smartphones Introduction Why dedicated GPUs in mobile devices? Gaming Physics simulation for realistic effects 3D-GUI
More informationUnleashing the benefits of GPU Computing with ARM Mali TM Practical applications and use-cases. Steve Steele, ARM
Unleashing the benefits of GPU Computing with ARM Mali TM Practical applications and use-cases Steve Steele, ARM 1 Today s Computational Challenges Trends Growing display sizes and resolutions, richer
More informationMultimedia in Mobile Phones. Architectures and Trends Lund
Multimedia in Mobile Phones Architectures and Trends Lund 091124 Presentation Henrik Ohlsson Contact: henrik.h.ohlsson@stericsson.com Working with multimedia hardware (graphics and displays) at ST- Ericsson
More informationgltf Briefing September 2016 Copyright Khronos Group Page 1
gltf Briefing September 2016 Copyright Khronos Group 2016 - Page 1 Copyright Khronos Group 2016 - Page 2 Background and Motivation OpenGL ES and WebGL have led to a proliferation of Web 3D but no standard
More informationNavigating the Vision API Jungle: Which API Should You Use and Why? Embedded Vision Summit, May 2015
Copyright Khronos Group 2015 - Page 1 Navigating the Vision API Jungle: Which API Should You Use and Why? Embedded Vision Summit, May 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem
More informationVisual HTML5. Human Information Interaction for Knowledge Extraction, Interaction, Utilization, Decision making HI-I-KEIUD
Visual HTML5 1 Overview HTML5 Building apps with HTML5 Visual HTML5 Canvas SVG Scalable Vector Graphics WebGL 2D + 3D libraries 2 HTML5 HTML5 to Mobile + Cloud = Java to desktop computing: cross-platform
More informationDeveloping a Reference Model for Augmented Reality. 5th International AR Standards Community Meeting 19 March 2012
Developing a Reference Model for Augmented Reality 5th International AR Standards Community Meeting 19 March 2012 What are uses of a Reference Model? Basis for coordination and understanding of the AR
More informationGTC 2013 March San Jose, CA The Smartest People. The Best Ideas. The Biggest Opportunities. Opportunities for Participation:
GTC 2013 March 18-21 San Jose, CA The Smartest People. The Best Ideas. The Biggest Opportunities. Opportunities for Participation: SPEAK - Showcase your work among the elite of graphics computing - Call
More informationAdding Advanced Shader Features and Handling Fragmentation
Copyright Khronos Group, 2010 - Page 1 Adding Advanced Shader Features and Handling Fragmentation How to enable your application on a wide range of devices Imagination Technologies Copyright Khronos Group,
More informationCopyright Khronos Group, Page 1
3RUWDEOH1DWLYH0RELOH 0HGLD$SSOLFDWLRQV 1HLO7UHYHWW 9LFH3UHVLGHQW(PEHGGHG&RQWHQW19,',$ 3UHVLGHQW.KURQRV*URXS Copyright Khronos Group, 2007 - Page 1 3HUYDVLYH0RELOH0HGLD&RPSXWLQJ Handsets are becoming personal
More informationStandards for Vision Processing and Neural Networks
Copyright Khronos Group 2017 - Page 1 Standards for Vision Processing and Neural Networks Radhakrishna Giduthuri, AMD radha.giduthuri@ieee.org Agenda Why we need a standard? Khronos NNEF Khronos OpenVX
More informationVulkan 1.1 March Copyright Khronos Group Page 1
Vulkan 1.1 March 2018 Copyright Khronos Group 2018 - Page 1 Vulkan 1.1 Launch and Ongoing Momentum Strengthening the Ecosystem Improved developer tools (SDK, validation/debug layers) More rigorous conformance
More informationHardware Accelerated Graphics for High Performance JavaFX Mobile Applications
Hardware Accelerated Graphics for High Performance JavaFX Mobile Applications Pavel Petroshenko, Sun Microsystems Jan Valenta, Sun Microsystems Jerry Evans, Sun Microsystems Goal of this Session Demonstrate
More informationCopyright Khronos Group, Page 1
1HLO7UHYHWW 19,',$.KURQRV *URXS 2SHQ*/(6 Copyright Khronos Group, 2007 - Page 1 Handsets are becoming personal computing platform - not just phones - A real computer in your hand mobility, connectedness
More informationDave Shreiner, ARM March 2009
4 th Annual Dave Shreiner, ARM March 2009 Copyright Khronos Group, 2009 - Page 1 Motivation - What s OpenGL ES, and what can it do for me? Overview - Lingo decoder - Overview of the OpenGL ES Pipeline
More informationKhronos Updates GDC 2017 Neil Trevett Vice President Developer Ecosystem, NVIDIA President,
Copyright Khronos Group 2017 - Page 1 Khronos Updates GDC 2017 Neil Trevett Vice President Developer Ecosystem, NVIDIA President, Khronos ntrevett@nvidia.com @neilt3d Copyright Khronos Group 2017 - Page
More information3D Graphics in Future Mobile Devices. Steve Steele, ARM
3D Graphics in Future Mobile Devices Steve Steele, ARM Market Trends Mobile Computing Market Growth Volume in millions Mobile Computing Market Trends 1600 Smart Mobile Device Shipments (Smartphones and
More informationCopyright Khronos Group, Page 1
3RUWDEOH1DWLYH0RELOH 0HGLD$SSOLFDWLRQV 1HLO7UHYHWW 9LFH3UHVLGHQW(PEHGGHG&RQWHQW19,',$ 3UHVLGHQW.KURQRV*URXS Copyright Khronos Group, 2007 - Page 1 3HUYDVLYH0RELOH0HGLD&RPSXWLQJ Handsets are becoming personal
More informationCopyright Khronos Group Page 1
OpenCL State of the Nation Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem OpenCL Working Group Chair ntrevett@nvidia.com @neilt3d Toronto, May 2017 Copyright Khronos Group 2017
More informationCopyright Khronos Group Page 1
OpenCL State of the Nation Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem OpenCL Working Group Chair ntrevett@nvidia.com @neilt3d Toronto, May 2017 Copyright Khronos Group 2017
More informationPOWERVR MBX & SGX OpenVG Support and Resources
POWERVR MBX & SGX OpenVG Support and Resources Kristof Beets 3 rd Party Relations Manager - Imagination Technologies kristof.beets@imgtec.com Copyright Khronos Group, 2006 - Page 1 Copyright Khronos Group,
More informationCreating the Embedded Media Processing Ecosystem
Creating the Embedded Media Processing Ecosystem Neil Trevett Vice President Embedded Content, NVIDIA President, Khronos Chairman, OpenGL ES Working Group Mobile Convergence Handsets are becoming the predominant
More informationSilicon Acceleration APIs
Copyright Khronos Group 2016 - Page 1 Silicon Acceleration APIs Embedded Technology 2016, Yokohama Neil Trevett Vice President Developer Ecosystem, NVIDIA President, Khronos ntrevett@nvidia.com @neilt3d
More informationNeural Network Exchange Format
Copyright Khronos Group 2017 - Page 1 Neural Network Exchange Format Deploying Trained Networks to Inference Engines Viktor Gyenes, specification editor Copyright Khronos Group 2017 - Page 2 Outlook The
More informationGTC Interaction Simplified. Gesture Recognition Everywhere: Gesture Solutions on Tegra
GTC 2013 Interaction Simplified Gesture Recognition Everywhere: Gesture Solutions on Tegra eyesight at a Glance Touch-free technology providing an enhanced user experience. Easy and intuitive control
More informationCopyright Khronos Group Page 1
OpenCL A State of the Union Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem OpenCL Working Group Chair ntrevett@nvidia.com @neilt3d Vienna, April 2016 Copyright Khronos Group 2016
More informationGPGPU Applications. for Hydrological and Atmospheric Simulations. and Visualizations on the Web. Ibrahim Demir
GPGPU Applications for Hydrological and Atmospheric Simulations and Visualizations on the Web Ibrahim Demir Big Data We are collecting and generating data on a petabyte scale (1Pb = 1,000 Tb = 1M Gb) Data
More informationOpenGL ES 2.0 : Start Developing Now. Dan Ginsburg Advanced Micro Devices, Inc.
OpenGL ES 2.0 : Start Developing Now Dan Ginsburg Advanced Micro Devices, Inc. Agenda OpenGL ES 2.0 Brief Overview Tools OpenGL ES 2.0 Emulator RenderMonkey w/ OES 2.0 Support OpenGL ES 2.0 3D Engine Case
More informationEcosystem Overview Neil Trevett Khronos President NVIDIA Vice President Developer
Ecosystem Overview Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem ntrevett@nvidia.com @neilt3d Copyright Khronos Group 2016 - Page 1 Khronos Mission Software Silicon Khronos is
More informationThe Mobile Advantage. Erik Noreke Independent Standardization Consultant Chair, OpenSL ES. Copyright Khronos Group, Page 1
& The Mobile Advantage Erik Noreke Independent Standardization Consultant Chair, OpenSL ES Copyright Khronos Group, 2010 - Page 1 Focused on Creating an Open Market Open Standards Open Markets - Developed
More informationBringing it all together: The challenge in delivering a complete graphics system architecture. Chris Porthouse
Bringing it all together: The challenge in delivering a complete graphics system architecture Chris Porthouse System Integration & the role of standards Content Ecosystem Java Execution Environment Native
More informationOpenGL BOF Siggraph 2011
OpenGL BOF Siggraph 2011 OpenGL BOF Agenda OpenGL 4 update Barthold Lichtenbelt, NVIDIA OpenGL Shading Language Hints/Kinks Bill Licea-Kane, AMD Ecosystem update Jon Leech, Khronos Viewperf 12, a new beginning
More informationProfiling and Debugging Games on Mobile Platforms
Profiling and Debugging Games on Mobile Platforms Lorenzo Dal Col Senior Software Engineer, Graphics Tools Gamelab 2013, Barcelona 26 th June 2013 Agenda Introduction to Performance Analysis with ARM DS-5
More informationEmbedded Media Processing Ecosystem
Embedded Media Processing Ecosystem Neil Trevett Vice President Mobile Content, NVIDIA President, Khronos Group Copyright Khronos Group, 2007 - Page 1 Pervasive Mobile Media Computing Handsets are becoming
More informationWebGL. Announcements. WebGL for Graphics Developers. WebGL for Web Developers. Homework 5 due Monday, 04/16. Final on Tuesday, 05/01
Announcements Patrick Cozzi University of Pennsylvania CIS 565 - Spring 2012 Homework 5 due Monday, 04/16 In-class quiz Wednesday, 04/18 Final on Tuesday, 05/01 6-8pm David Rittenhouse Lab A7 Networking
More informationHTML5 Evolution and Development. Matt Spencer UI & Browser Marketing Manager
HTML5 Evolution and Development Matt Spencer UI & Browser Marketing Manager 1 HTML5 Ratified. finally! After 7 years of development, the HTML5 specification was ratified on 28 th October 14 urce>
More informationWhiz-Bang Graphics and Media Performance for Java Platform, Micro Edition (JavaME)
Whiz-Bang Graphics and Media Performance for Java Platform, Micro Edition (JavaME) Pavel Petroshenko, Sun Microsystems, Inc. Ashmi Bhanushali, NVIDIA Corporation Jerry Evans, Sun Microsystems, Inc. Nandini
More informationProfiling and Debugging OpenCL Applications with ARM Development Tools. October 2014
Profiling and Debugging OpenCL Applications with ARM Development Tools October 2014 1 Agenda 1. Introduction to GPU Compute 2. ARM Development Solutions 3. Mali GPU Architecture 4. Using ARM DS-5 Streamline
More informationMore performance options
More performance options OpenCL, streaming media, and native coding options with INDE April 8, 2014 2014, Intel Corporation. All rights reserved. Intel, the Intel logo, Intel Inside, Intel Xeon, and Intel
More informationARM Multimedia IP: working together to drive down system power and bandwidth
ARM Multimedia IP: working together to drive down system power and bandwidth Speaker: Robert Kong ARM China FAE Author: Sean Ellis ARM Architect 1 Agenda System power overview Bandwidth, bandwidth, bandwidth!
More informationWindowing System on a 3D Pipeline. February 2005
Windowing System on a 3D Pipeline February 2005 Agenda 1.Overview of the 3D pipeline 2.NVIDIA software overview 3.Strengths and challenges with using the 3D pipeline GeForce 6800 220M Transistors April
More informationHotChips An innovative HD video and digital image processor for low-cost digital entertainment products. Deepu Talla.
HotChips 2007 An innovative HD video and digital image processor for low-cost digital entertainment products Deepu Talla Texas Instruments 1 Salient features of the SoC HD video encode and decode using
More informationGraphics and Imaging Architectures
Graphics and Imaging Architectures Kayvon Fatahalian http://www.cs.cmu.edu/afs/cs/academic/class/15869-f11/www/ About Kayvon New faculty, just arrived from Stanford Dissertation: Evolving real-time graphics
More informationModule Introduction. Content 15 pages 2 questions. Learning Time 25 minutes
Purpose The intent of this module is to introduce you to the multimedia features and functions of the i.mx31. You will learn about the Imagination PowerVR MBX- Lite hardware core, graphics rendering, video
More informationConnecting with Tizen : An Overview & Roadmap. Mohan Rao
Connecting with Tizen : An Overview & Roadmap Mohan Rao Contents What is Tizen? Why Tizen? Device profile Tizen 2.4 Tizen 3.0 What s next? Conclusion 2 What is Tizen? (1/2) Designed for various category
More informationTake GPU Processing Power Beyond Graphics with Mali GPU Computing
Take GPU Processing Power Beyond Graphics with Mali GPU Computing Roberto Mijat Visual Computing Marketing Manager August 2012 Introduction Modern processor and SoC architectures endorse parallelism as
More informationOpenCL The Open Standard for Heterogeneous Parallel Programming
OpenCL The Open Standard for Heterogeneous Parallel Programming March 2009 Copyright Khronos Group, 2009 - Page 1 Close-to-the-Silicon Standards Khronos creates Foundation-Level acceleration APIs - Needed
More informationStreaming Media Portability
Streaming Media Portability July 2006 Copyright Khronos Group, 2006 - Page 1 Copyright Khronos Group, 2006 - Page 2 Media Portability Problem Media infrastructure portability is a multi-level industry
More informationNeil Trevett Vice President, NVIDIA OpenCL Chair Khronos President. Copyright Khronos Group, Page 1
Neil Trevett Vice President, NVIDIA OpenCL Chair Khronos President Copyright Khronos Group, 2009 - Page 1 Introduction and aims of OpenCL - Neil Trevett, NVIDIA OpenCL Specification walkthrough - Mike
More informationEmerging Vision Technologies: Enabling a New Era of Intelligent Devices
Emerging Vision Technologies: Enabling a New Era of Intelligent Devices Computer vision overview Computer vision is being integrated in our daily lives Acquiring, processing, and understanding visual data
More informationTHE LEADER IN VISUAL COMPUTING
MOBILE EMBEDDED THE LEADER IN VISUAL COMPUTING 2 TAKING OUR VISION TO REALITY HPC DESIGN and VISUALIZATION AUTO GAMING 3 BEST DEVELOPER EXPERIENCE Tools for Fast Development Debug and Performance Tuning
More informationKHRONOS STANDARDS UPDATE. Neil Trevett, GTC, 26 th March 2018
KHRONOS STANDARDS UPDATE Neil Trevett, GTC, 26 th March 2018 Khronos Mission Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standards
More informationGPGPU on ARM. Tom Gall, Gil Pitney, 30 th Oct 2013
GPGPU on ARM Tom Gall, Gil Pitney, 30 th Oct 2013 Session Description This session will discuss the current state of the art of GPGPU technologies on ARM SoC systems. What standards are there? Where are
More information