Open Standard APIs for Embedded Vision Processing
|
|
- Donald George
- 6 years ago
- Views:
Transcription
1 Copyright Khronos Group Page 1 Open Standard APIs for Embedded Vision Processing Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group
2 Copyright Khronos Group Page 2 Speakers This Morning Neil Trevett - Vice President Mobile Ecosystem, NVIDIA - President, Khronos - Chair, OpenCL Working Group Mikael Sevenier - Chair, Camera working group Jim Steele - CTO, Sensor Platforms - Chair, StreamInput
3 Copyright Khronos Group Page 3 Khronos Connects Software to Silicon Open Consortium creating ROYALTY-FREE, OPEN STANDARD APIs for hardware acceleration Defining the roadmap for low-level silicon interfaces needed on every platform Graphics, compute, rich media, vision, sensor and camera processing Rigorous specifications AND conformance tests for crossvendor portability Acceleration APIs BY the Industry FOR the Industry Well over a BILLION people use Khronos APIs Every Day
4 Copyright Khronos Group Page 4 Khronos Standards Visual Computing - 3D Graphics - Heterogeneous Parallel Computing 3D Asset Handling - 3D authoring asset interchange - 3D asset transmission format with compression Camera Control API Sensor Processing - Vision Acceleration - Camera Control - Sensor Fusion Over 100 companies defining royalty-free APIs to connect software to silicon Acceleration in HTML5-3D in browser no Plug-in - Heterogeneous computing for JavaScript
5 Copyright Khronos Group Page 5 Sensors & Vision Driving Key Mobile Use Cases Computational Photography and Videography Natural UI with Face, Body and Gesture Tracking 3D Scene and Object Reconstruction Augmented Reality Time
6 Vision Pipeline Challenges and Opportunities Growing Camera Diversity Capturing color, range and lightfields Diverse Vision Processors Driving for high performance and low power Sensor Proliferation Diverse sensor awareness of the user and surroundings Light / Proximity 2 cameras 3 microphones Touch Camera sensors >20MPix Novel sensor configurations Stereo pairs Active Structured Light Active TOF Plenoptic Arrays Camera ISPs Dedicated vision IP blocks DSPs and DSP arrays Programmable GPUs Multi-core CPUs 19 Position - GPS - WiFi (fingerprint) - Cellular trilateration - NFC/Bluetooth Beacons Accelerometer Magnetometer Gyroscope Pressure / Temp / Humidity Flexible sensor and camera control to generate required image stream Camera Control API Use best processing available for image stream processing with code portability Control/fuse vision data by/with all other sensor data on device Copyright Khronos Group Page 6
7 Copyright Khronos Group Page 7 OpenVX Power Efficient Vision Acceleration Acceleration API for real-time vision - Focus on mobile and embedded systems Enable diverse efficient implementations - From CPUs, through GPUs and DSPs to dedicated hardware Foundational API for vision acceleration - Can be used by middleware libraries or by applications directly Complementary to OpenCV - Which is great for prototyping Khronos open source sample implementation - To be released with final specification - Sample - not reference - spec remains the definitive definition of OpenVX operation OpenCV open source library Open source sample implementation Application Other higher-level CV libraries Hardware vendor implementations
8 Copyright Khronos Group Page 8 OpenVX Graphs The Key to Efficiency Vision processing directed graphs for power and performance efficiency - Each Node can be implemented in software or accelerated hardware - Nodes may be fused by the implementation to eliminate memory transfers - Processing can be tiled to keep data entirely in local memory/cache EGLStreams can provide data and event interop with other Khronos APIs - BUT use of other Khronos APIs are not mandated VXU Utility Library for access to single nodes - Easy way to start using OpenVX by calling each node independently Native Camera Control OpenVX Node OpenVX Node OpenVX Node Example OpenVX Graph OpenVX Node Heterogeneous Processing
9 Copyright Khronos Group Page 9 OpenVX 1.0 Function Overview Core data structures - Images and Image Pyramids - Processing Graphs, Kernels, Parameters Image Processing - Arithmetic, Logical, and statistical operations - Multichannel Color and BitDepth Extraction and Conversion - 2D Filtering and Morphological operations - Image Resizing and Warping Core Computer Vision - Pyramid computation - Integral Image computation Feature Extraction and Tracking - Histogram Computation and Equalization - Canny Edge Detection - Harris and FAST Corner detection - Sparse Optical Flow Widely used extensions adopted into future versions of the core OpenVX Specification Evolution OpenVX 1.0 defines framework for creating, managing and executing graphs Focused set of widely used functions that are readily accelerated Implementers can add functions as extensions
10 Copyright Khronos Group Page 10 Example Graph - Stereo Machine Vision OpenVX Graph Camera 1 Stereo Rectify with Remap Compute Depth Map (User Node) Detect and track objects (User Node) Object coordinates Camera 2 Stereo Rectify with Remap Image Pyramid Compute Optical Flow Delay Tiling extension enables user nodes (extensions) to also optimally run in local memory
11 Copyright Khronos Group Page 11 OpenVX and OpenCV are Complementary Governance Conformance Community driven open source with no formal specification No conformance tests for consistency and every vendor implements different subset Formal specification defined and implemented by hardware vendors Full conformance test suite / process creates a reliable acceleration platform Portability APIs can vary depending on processor Hardware abstracted for portability Scope Efficiency Very wide 1000s of imaging and vision functions Multiple camera APIs/interfaces Memory-based architecture Each operation reads and writes memory Tight focus on hardware accelerated functions for mobile vision Use external camera API Graph-based execution Optimizable computation, data transfer Use Case Rapid experimentation Production development & deployment
12 Copyright Khronos Group Page 12 OpenVX Participants and Timeline Provisional 1.0 specification released November 2013 for industry feedback Aiming for specification finalization and conformance tests 3Q14 Itseez is working group chair (the convener of OpenCV) Qualcomm and TI are specification editors
13 Copyright Khronos Group Page 13 OpenCL Portable Heterogeneous Computing Portable Heterogeneous programming of diverse compute resources - Targeting supercomputers -> embedded systems -> mobile devices One code tree can be executed on CPUs, GPUs, DSPs and hardware - Dynamically interrogate system load and balance work across available processors OpenCL = Two APIs and C-based Kernel language - Platform Layer API to query, select and initialize compute devices - Kernel language - Subset of ISO C99 + language extensions - C Runtime API to build and execute kernels OpenCL across multiple devices Kernel OpenCL Code Kernel OpenCL Code Kernel OpenCL Code Kernel Code GPU DSP HW CPU CPU
14 Copyright Khronos Group Page 14 OpenCL as Foundation for Parallel Compute 100+ tool chains and languages leveraging OpenCL - Heterogeneous solutions emerging for the most popular programming languages C++ AMP Shevlin Park Uses Clang and LLVM SYCL C++ syntax compiler extensions WebCL JavaScript binding for initiation of OpenCL C kernels Language for image processing and computational photography Aparapi Java language extensions for parallelism River Trail Language extensions to JavaScript PyOpenCL Python wrapper around OpenCL Harlan High level language for GPU programming Compiler directives for Fortran, C and C++ SPIR Standard Portable Intermediate Representation (extending LLVM for parallel computation) SPIR 1.2 Released in January 2014 OpenCL provides vendor optimized, cross-platform, cross-vendor access to heterogeneous compute resources
15 Copyright Khronos Group Page 15 OpenVX and OpenCL are Complementary Use Case Architecture Target Hardware Precision Ease of Use General Heterogeneous programming Language-based needs online compilation Exposed architected memory model can impact performance portability Full IEEE floating point mandated General-purpose math libraries with no built-in vision functions Domain targeted Vision processing Library-based - no online compiler required Abstracted node and memory model - diverse implementations can be optimized for power and performance Minimal floating point requirements optimized for vision operators Fully implemented vision operators and framework out of the box It is possible to use OpenCL to build OpenVX Nodes
16 Need for Camera Control API We have choice of APIs for image and vision image processing - BUT no open standard API for camera control to FEED these APIs! Need advanced control of ISP and camera subsystem - Generate sophisticated image stream for advanced imaging & vision apps No system API fulfills all developer requirements - Advanced, high-frequency burst control of camera and sensor operation - Portable support for diversity of sensors: e.g. depth sensors and sensor arrays - Tight system integration: e.g. synch of camera and MEMS sensors Scope of Camera Control API 3A - Auto Exposure (AE), Auto White Balance (AWB), Auto Focus (AF) Lens, sensor, aperture control Bayer Pre-processing Image Signal Processor (ISP) RGB/YUV Post-processing Image/Vision Applications Sensor, Color Filter Array Lens, Flash, Focus, Aperture Copyright Khronos Group Page 16
17 Copyright Khronos Group Page 17 Advanced Camera Control Use Cases High-dynamic range (HDR) and computational flash photography - High-speed burst with individual frame control over exposure and flash Subject isolation and depth detection - High-speed burst with individual frame control over focus Rolling shutter elimination - High-precision intra-frame synchronization between camera and motion sensor Augmented Reality - 60Hz, low-latency capture with motion sensor synchronization - Multiple Region of Interest (ROI) capture - Synchronized stereo sensors for scene scaling - Detailed feedback on camera operation per frame Time-of-flight or structured light depth camera processing - Aligned stacking of data from multiple sensors
18 Copyright Khronos Group Page 18 Camera API Architecture will be FCAM-based No global state - State travels with image requests - Every stage in the pipeline may have different state - Enables fast, deterministic state changes Synchronize devices - Lens, flash, sound capture, gyro - Devices can schedule Actions - E.g. to be triggered on exposure change
19 Copyright Khronos Group Page 19 Khronos Camera API Requirements Application control over ISP processing (including 3A) - Including multiple, re-entrant ISPs Control multiple sensors with synch and alignment - E.g. Stereo pairs, Plenoptic arrays, TOF or structured light depth cameras Enhanced per frame detailed control - Format flexibility, Region of Interest (ROI) selection Global timing & synchronization - E.g. Between cameras and MEMS sensors Flexible processing/streaming - Multiple input and output streams with RAW, Bayer or YUV Processing - Streaming of rows (not just frames) Enable new camera functionality not available on current platforms and align with future platform directions for easy adoption
20 Copyright Khronos Group Page 20 Camera API Design Milestones and Philosophy C-language API starting from proven designs - e.g. FCAM Design alignment with widely used hardware standards - e.g. MIPI CSI Focus on mobile, power-limited devices - But do not preclude other use cases such as automotive, surveillance, DSLR Minimize overlap and maximize interoperability with other Khronos APIs - But other Khronos APIs are not mandated Support vendor-specific extensions Apr13 Working group proposed Group charter approved Jul13 4Q13 Architectural Design First draft specification 1Q14 2Q14 Sample implementation and tests Specification ratification 3Q14
21 Copyright Khronos Group Page 21 Potential Adoption on Android Android Exposes Java camera APIs to developers - Controls underlying Camera HAL Camera HAL v1 API simplified basic point and shoot apps - Difficult or impossible to do much else Camera HAL v3 API is a fundamentally different API - Streams-based to enable more sophisticated camera applications Khronos Camera API builds on FCAM with a goal of being forward compatible with Android architecture Camera API Open source project developed by Nokia and Stanford HAL V3 adopts many FCAM ideas and can use EGL in its implementation Khronos Camera API may be used to IMPLEMENT Android Camera HAL and provide an advanced native camera API in NDK
22 Copyright Khronos Group Page 22 StreamInput Jim Steele CTO, Sensor Platforms Chair, StreamInput Working Group
23 Sensor Industry Fragmentation Copyright Khronos Group Page 23
24 Copyright Khronos Group Page 24 Low-level Sensor Abstraction API Apps request semantic sensor information StreamInput defines possible requests, e.g. Read Physical or Virtual Sensors e.g. Game Quaternion Context detection e.g. Am I in an elevator? Apps Need Sophisticated Access to Sensor Data Without coding to specific sensor hardware Sensor Discoverability Sensor Code Portability Advanced Sensors Everywhere Multi-axis motion/position, quaternions, context-awareness, gestures, activity monitoring, health and environmental sensors StreamInput processing graph provides optimized sensor data stream High-value, smart sensor fusion middleware can connect to apps in a portable way Apps can gain magical situational awareness
25 Copyright Khronos Group Page 25 Sensor Types Basic sensor data: - Acceleration, Magnetic Field, Angular Rates - Pressure, Ambient Light, Proximity, Temperature, Humidity, RGB light, UV light - Heart rate, Blood Oxygen Level, Skin Hydration, Breathalyzer Sensor fusion - Orientation (Quaternion or Euler Angles), Gravity, Linear Acceleration - Position Context awareness - Device Motion: general movement of the device: still, free-fall, - Carry: how the device is being held by a user: in pocket, in hand, - Posture: how the body holding the device is positioned: standing, sitting, step, - Transport: about the environment around the device: in elevator, in car,
26 Copyright Khronos Group Page 26 StreamInput: Potential Sensor Fusion Stack Applications Platforms can provide increased access to improved sensor data stream driving faster, deeper sensor usage by applications OS Sensor APIs (E.g. Android SensorManager or ios CoreMotion) Middleware (E.g. Context-awareness engines, gaming engines) Middleware engines need platformportable access to native, low-level sensor data streams StreamInput implementations compete on sensor stream quality, reduced power consumption, environment triggering and context detection enabling sensor subsystem vendors to increased ADDED VALUE Hardware transport interfaces are defined by each system, e.g. IIO or HID sensor Sensor Sensor Low-level native API defines access to fused sensor data stream and context-awareness Sensor Hub Sensor Hub Mobile or embedded platforms without sensor fusion APIs can provide direct application access to StreamInput Embedded processors or peripheral hardware implementing StreamInput provide a standard interface to other system processors
27 Copyright Khronos Group Page 27 Khronos APIs for Augmented Reality AR needs not just advanced sensor processing, vision acceleration, computation and rendering - but also for all these subsystems to work efficiently together Audio Rendering MEMS Sensors Sensor Fusion Application on CPUs, GPUs and DSPs Precision timestamps on all sensor samples Vision Processing Advanced Camera Control and stream generation Camera Control API EGLStream - stream data between APIs 3D Rendering and Video Composition On GPU
28 Copyright Khronos Group Page 28 Summary Khronos is building a family of interoperating APIs for portable and power-efficient vision processing OpenVX 1.0 has been provisionally released and non-members are invited to provide feedback on the forums - Khronos camera and sensor fusion APIs are currently in design and complement and integrate with OpenVX Any company is welcome to join Khronos to influence the direction of mobile and embedded vision processing! - $15K annual membership fee for access to all Khronos API working groups - Well-defined IP framework protects your IP and conformant implementations - ntrevett@nvidia.com
Vision Acceleration. Launch Briefing October Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group
Copyright Khronos Group 2014 - Page 1 Vision Acceleration Launch Briefing October 2014 Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group Copyright Khronos Group 2014 - Page
More informationOpen Standard APIs for Augmented Reality
Copyright Khronos Group 2014 - Page 1 Open Standard APIs for Augmented Reality Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group Copyright Khronos Group 2014 - Page 2 Khronos
More informationOpen API Standards for Mobile Graphics, Compute and Vision Processing GTC, March 2014
Open API Standards for Mobile Graphics, Compute and Vision Processing GTC, March 2014 Neil Trevett Vice President Mobile Ecosystem, NVIDIA President Khronos Copyright Khronos Group 2014 - Page 1 Khronos
More informationKhronos and the Mobile Ecosystem
Copyright Khronos Group, 2011 - Page 1 Khronos and the Mobile Ecosystem Neil Trevett VP Mobile Content, NVIDIA President, Khronos Copyright Khronos Group, 2011 - Page 2 Topics It s not just about individual
More informationKhronos Overview The State of the Art in Open Standards for Visual Computing
Khronos Overview The State of the Art in Open Standards for Visual Computing Neil Trevett Khronos President Vice President Mobile Content, NVIDIA Copyright Khronos Group 2013 - Page 1 Copyright Khronos
More informationWebGL Meetup GDC Copyright Khronos Group, Page 1
WebGL Meetup GDC 2012 Copyright Khronos Group, 2012 - Page 1 Copyright Khronos Group, 2012 - Page 2 Khronos API Ecosystem Trends Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos
More informationAR Standards Update Austin, March 2012
AR Standards Update Austin, March 2012 Neil Trevett President, The Khronos Group Vice President Mobile Content, NVIDIA Copyright Khronos Group, 2012 - Page 1 Topics Very brief overview of Khronos Update
More informationMobile AR Hardware Futures
Copyright Khronos Group, 2010 - Page 1 Mobile AR Hardware Futures Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Two Perspectives NVIDIA - Tegra 2 mobile processor Khronos
More informationSIGGRAPH Briefing August 2014
Copyright Khronos Group 2014 - Page 1 SIGGRAPH Briefing August 2014 Neil Trevett VP Mobile Ecosystem, NVIDIA President, Khronos Copyright Khronos Group 2014 - Page 2 Significant Khronos API Ecosystem Advances
More informationCopyright Khronos Group Page 1
Open Standards and Open Source Together How Khronos APIs Accelerate Fast and Cool Applications Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem Copyright Khronos Group 2015 - Page
More informationOpen Standards for AR and VR Neil Trevett Khronos President NVIDIA VP Developer January 2018
Copyright Khronos Group 2018 - Page 1 Open Standards for AR and Neil Trevett Khronos President NVIDIA VP Developer Ecosystem ntrevett@nvidia.com @neilt3d January 2018 Khronos Mission E.g. OpenGL ES provides
More informationUpdate on Khronos Open Standard APIs for Vision Processing Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem
Update on Khronos Open Standard APIs for Vision Processing Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem Copyright Khronos Group 2015 - Page 1 Copyright Khronos Group 2015 - Page
More informationOpen Standards for Today s Gaming Industry
Copyright Khronos Group 2013 - Page 1 Open Standards for Today s Gaming Industry Erik Noreke VP of Business Development, Khronos Group Gaming Evolution Copyright Khronos Group 2013 - Page 3 DESKTOP MOBILE
More informationOpen Standards for Vision and AI Peter McGuinness NNEF WG Chair CEO, Highwai, Inc May 2018
Copyright Khronos Group 2018 - Page 1 Open Standards for Vision and AI Peter McGuinness NNEF WG Chair CEO, Highwai, Inc peter.mcguinness@gobrach.com May 2018 Khronos Mission E.g. OpenGL ES provides 3D
More informationKhronos Connects Software to Silicon
Press Pre-Briefing GDC 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem All Materials Embargoed Until Tuesday 3 rd March, 12:01AM Pacific Time Copyright Khronos Group 2015 - Page
More informationCopyright Khronos Group Page 1. Vulkan Overview. June 2015
Copyright Khronos Group 2015 - Page 1 Vulkan Overview June 2015 Copyright Khronos Group 2015 - Page 2 Khronos Connects Software to Silicon Open Consortium creating OPEN STANDARD APIs for hardware acceleration
More informationNavigating the Vision API Jungle: Which API Should You Use and Why? Embedded Vision Summit, May 2015
Copyright Khronos Group 2015 - Page 1 Navigating the Vision API Jungle: Which API Should You Use and Why? Embedded Vision Summit, May 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem
More informationCopyright Khronos Group, Page 1. Khronos Overview. Taiwan, February 2012
Copyright Khronos Group, 2012 - Page 1 Khronos Overview Taiwan, February 2012 Copyright Khronos Group, 2012 - Page 2 Khronos - Connecting Software to Silicon Creating open, royalty-free API standards -
More informationThe OpenVX Computer Vision and Neural Network Inference
The OpenVX Computer and Neural Network Inference Standard for Portable, Efficient Code Radhakrishna Giduthuri Editor, OpenVX Khronos Group radha.giduthuri@amd.com @RadhaGiduthuri Copyright 2018 Khronos
More informationNext Generation OpenGL Neil Trevett Khronos President NVIDIA VP Mobile Copyright Khronos Group Page 1
Next Generation OpenGL Neil Trevett Khronos President NVIDIA VP Mobile Ecosystem @neilt3d Copyright Khronos Group 2015 - Page 1 Copyright Khronos Group 2015 - Page 2 Khronos Connects Software to Silicon
More informationAccelerating Vision Processing
Accelerating Vision Processing Neil Trevett Vice President Mobile Ecosystem at NVIDIA President of Khronos and Chair of the OpenCL Working Group SIGGRAPH, July 2016 Copyright Khronos Group 2016 - Page
More informationSC24/WG9 Liaison Meeting
Copyright Khronos Group, 2011 - Page 1 SC24/WG9 Liaison Meeting Seoul, November 2011 Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page
More informationAcceleration Standards for Mobile Augmented Reality
Acceleration Standards for Mobile Augmented Reality Neil Trevett Khronos President Vice President Mobile Content, NVIDIA November 2012 Copyright Khronos Group 2012 Page 1 Copyright Khronos Group 2012 Page
More informationOpenCL Press Conference
Copyright Khronos Group, 2011 - Page 1 OpenCL Press Conference Tokyo, November 2011 Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page
More informationPress Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem. Copyright Khronos Group Page 1
Press Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem Copyright Khronos Group 2015 - Page 1 Khronos Connects Software to Silicon Open Consortium creating ROYALTY-FREE,
More informationFusing Sensors into Mobile Operating Systems & Innovative Use Cases
Fusing Sensors into Mobile Operating Systems & Innovative Use Cases May 23, 2012 Tristan Joo (tristanjoo@wca.org) Board Director & Co-Chair of Mobile SIG Wireless Communications Alliance, Independent Executive
More informationCopyright Khronos Group Page 1
Gaming Market Briefing Overview of APIs GDC March 2016 Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem ntrevett@nvidia.com @neilt3d Copyright Khronos Group 2016 - Page 1 Copyright
More informationAdvanced Imaging Applications on Smart-phones Convergence of General-purpose computing, Graphics acceleration, and Sensors
Advanced Imaging Applications on Smart-phones Convergence of General-purpose computing, Graphics acceleration, and Sensors Sriram Sethuraman Technologist & DMTS, Ittiam 1 Overview Imaging on Smart-phones
More informationPress Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem. Copyright Khronos Group Page 1
Press Briefing SIGGRAPH 2015 Neil Trevett Khronos President NVIDIA Vice President Mobile Ecosystem Copyright Khronos Group 2015 - Page 1 Khronos Connects Software to Silicon Open Consortium creating ROYALTY-FREE,
More informationOverview and AR/VR Roadmap
Khronos Group Inc. 2018 - Page 1 Overview and AR/ Roadmap Neil Trevett Khronos President NVIDIA VP Developer Ecosystems ntrevett@nvidia.com @neilt3d Khronos Group Inc. 2018 - Page 2 Khronos Connects Software
More informationThe Benefits of GPU Compute on ARM Mali GPUs
The Benefits of GPU Compute on ARM Mali GPUs Tim Hartley 1 SEMICON Europa 2014 ARM Introduction World leading semiconductor IP Founded in 1990 1060 processor licenses sold to more than 350 companies >
More informationCopyright Khronos Group 2012 Page 1. OpenCL 1.2. August 2012
Copyright Khronos Group 2012 Page 1 OpenCL 1.2 August 2012 Copyright Khronos Group 2012 Page 2 Khronos - Connecting Software to Silicon Khronos defines open, royalty-free standards to access graphics,
More informationStandards for Vision Processing and Neural Networks
Copyright Khronos Group 2017 - Page 1 Standards for Vision Processing and Neural Networks Radhakrishna Giduthuri, AMD radha.giduthuri@ieee.org Agenda Why we need a standard? Khronos NNEF Khronos OpenVX
More informationWebGL, WebCL and Beyond!
Copyright Khronos Group, 2011 - Page 1 WebGL, WebCL and Beyond! Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page 2 Topics in this Session
More informationEnabling a Richer Multimedia Experience with GPU Compute. Roberto Mijat Visual Computing Marketing Manager
Enabling a Richer Multimedia Experience with GPU Compute Roberto Mijat Visual Computing Marketing Manager 1 What is GPU Compute Operating System and most application processing continue to reside on the
More informationOpen Standards for Building Virtual and Augmented Realities. Neil Trevett Khronos President NVIDIA VP Developer Ecosystems
Open Standards for Building Virtual and Augmented Realities Neil Trevett Khronos President NVIDIA VP Developer Ecosystems Khronos Mission Asian Members Software Silicon Khronos is an International Industry
More informationCopyright Khronos Group, Page 1. OpenCL. GDC, March 2010
Copyright Khronos Group, 2011 - Page 1 OpenCL GDC, March 2010 Authoring and accessibility Application Acceleration System Integration Copyright Khronos Group, 2011 - Page 2 Khronos Family of Standards
More informationCopyright Khronos Group Page 1
OpenCL State of the Nation Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem OpenCL Working Group Chair ntrevett@nvidia.com @neilt3d Toronto, May 2017 Copyright Khronos Group 2017
More informationCopyright Khronos Group Page 1
OpenCL State of the Nation Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem OpenCL Working Group Chair ntrevett@nvidia.com @neilt3d Toronto, May 2017 Copyright Khronos Group 2017
More informationNeural Network Exchange Format
Copyright Khronos Group 2017 - Page 1 Neural Network Exchange Format Deploying Trained Networks to Inference Engines Viktor Gyenes, specification editor Copyright Khronos Group 2017 - Page 2 Outlook The
More informationOpenCL Overview. Shanghai March Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group
Copyright Khronos Group, 2012 - Page 1 OpenCL Overview Shanghai March 2012 Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2012 - Page 2 Processor
More informationGraphics Technology Update
Graphics Technology Update Presented by: Erik Noreke, Khronos Group Vice President of Business Development November 2013 Copyright Khronos Group, 2013 - Page 1 Copyright Khronos Group, 2013 - Page 2 Khronos
More informationWebGL, WebCL and OpenCL
Copyright Khronos Group, 2011 - Page 1 WebGL, WebCL and OpenCL Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page 2 Processor Parallelism
More informationThe State of Gaming APIs
Copyright Khronos Group, 2011 - Page 1 The State of Gaming APIs Neil Trevett Vice President Mobile Content, NVIDIA President, The Khronos Group Copyright Khronos Group, 2011 - Page 2 State of Gaming APIs
More informationCopyright Khronos Group Page 1
OpenCL A State of the Union Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem OpenCL Working Group Chair ntrevett@nvidia.com @neilt3d Vienna, April 2016 Copyright Khronos Group 2016
More informationStandards for WebVR. Neil Trevett. Khronos President Vice President Mobile Content,
Standards for WebVR Neil Trevett Khronos President Vice President Mobile Content, NVIDIA ntrevett@nvidia.com, @neilt3d October 2016 Copyright Khronos Group 2016 - Page 1 Khronos Open Standards Software
More informationINTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES. Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp.
INTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp. Computer Vision in Mobile Tegra K1 It s time! AGENDA Use cases categories
More informationIMAGE AND VISION PROCESSING ON TEGRA K1. Elif Albuz
IMAGE AND VISION PROCESSING ON TEGRA K1 Elif Albuz IMAGE AND VISION USE CASES Driven by using camera as a sensor Computational Photography and Videography Face, Body and Gesture Tracking 3D Scene/Object
More informationSilicon Acceleration APIs
Copyright Khronos Group 2016 - Page 1 Silicon Acceleration APIs Embedded Technology 2016, Yokohama Neil Trevett Vice President Developer Ecosystem, NVIDIA President, Khronos ntrevett@nvidia.com @neilt3d
More informationCopyright Khronos Group Page 1. OpenCL BOF SIGGRAPH 2013
Copyright Khronos Group 2013 - Page 1 OpenCL BOF SIGGRAPH 2013 Copyright Khronos Group 2013 - Page 2 OpenCL Roadmap OpenCL-HLM (High Level Model) High-level programming model, unifying host and device
More informationEmerging Vision Technologies: Enabling a New Era of Intelligent Devices
Emerging Vision Technologies: Enabling a New Era of Intelligent Devices Computer vision overview Computer vision is being integrated in our daily lives Acquiring, processing, and understanding visual data
More informationEcosystem Overview Neil Trevett Khronos President NVIDIA Vice President Developer
Ecosystem Overview Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem ntrevett@nvidia.com @neilt3d Copyright Khronos Group 2016 - Page 1 Khronos Mission Software Silicon Khronos is
More informationNext Generation Visual Computing
Next Generation Visual Computing (Making GPU Computing a Reality with Mali ) Taipei, 18 June 2013 Roberto Mijat ARM Addressing Computational Challenges Trends Growing display sizes and resolutions Increasing
More informationCompleting the Multimedia Architecture
Copyright Khronos Group, 2011 - Page 1 Completing the Multimedia Architecture Erik Noreke Chair of OpenSL ES Working Group Chair of OpenMAX AL Working Group Copyright Khronos Group, 2011 - Page 2 Today
More informationNeil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group. Copyright Khronos Group Page 1
Neil Trevett Vice President Mobile Ecosystem, NVIDIA President, Khronos Group Copyright Khronos Group 2014 - Page 1 Khronos Standards 3D Asset Handling - 3D authoring asset interchange - 3D asset transmission
More informationGTC 2013 March San Jose, CA The Smartest People. The Best Ideas. The Biggest Opportunities. Opportunities for Participation:
GTC 2013 March 18-21 San Jose, CA The Smartest People. The Best Ideas. The Biggest Opportunities. Opportunities for Participation: SPEAK - Showcase your work among the elite of graphics computing - Call
More informationCopyright Khronos Group Page 1
OpenCL and Ecosystem State of the Nation Neil Trevett Khronos President NVIDIA Vice President Developer Ecosystem OpenCL Working Group Chair ntrevett@nvidia.com @neilt3d Oxford, May 2018 Copyright Khronos
More informationDeveloping a Reference Model for Augmented Reality. 5th International AR Standards Community Meeting 19 March 2012
Developing a Reference Model for Augmented Reality 5th International AR Standards Community Meeting 19 March 2012 What are uses of a Reference Model? Basis for coordination and understanding of the AR
More informationStreaming Media. Advanced Audio. Erik Noreke Standardization Consultant Chair, OpenSL ES. Copyright Khronos Group, Page 1
Streaming Media Advanced Audio Erik Noreke Standardization Consultant Chair, OpenSL ES Copyright Khronos Group, 2010 - Page 1 Today s Consumer Requirements Rich media applications and UI - Consumer decisions
More informationUnleashing the benefits of GPU Computing with ARM Mali TM Practical applications and use-cases. Steve Steele, ARM
Unleashing the benefits of GPU Computing with ARM Mali TM Practical applications and use-cases Steve Steele, ARM 1 Today s Computational Challenges Trends Growing display sizes and resolutions, richer
More informationCopyright Khronos Group Page 1
Update on Khronos Standards for Vision and Machine Learning December 2017 Neil Trevett Khronos President NVIDIA VP Developer Ecosystem ntrevett@nvidia.com @neilt3d www.khronos.org Copyright Khronos Group
More informationApril 4-7, 2016 Silicon Valley VISIONWORKS A CUDA ACCELERATED COMPUTER VISION LIBRARY S6783. Elif Albuz, April 4, 2016
April 4-7, 2016 Silicon Valley VISIONWORKS A CUDA ACCELERATED COMPUTER VISION LIBRARY S6783 Elif Albuz, April 4, 2016 Motivation Introduction to VisionWorks AGENDA VisionWorks Software Stack VisionWorks
More informationUbiquitous IoT Perspectives The Power of Connected Sensors and Actuators
Ubiquitous IoT Perspectives The Power of Connected Sensors and Actuators Andrea Onetti Vice President, General Manager MEMS Sensor Division STMicroelectronics Digital Technologies are Transforming nearly
More informationOpenMAX AL, OpenSL ES
Copyright Khronos Group, 2011 - Page 1 OpenMAX AL, OpenSL ES Native Multimedia in Android Erik Noreke Chair of OpenMAX AL and OpenSL ES Working Groups Copyright Khronos Group, 2011 - Page 2 Why Create
More informationVulkan 1.1 March Copyright Khronos Group Page 1
Vulkan 1.1 March 2018 Copyright Khronos Group 2018 - Page 1 Vulkan 1.1 Launch and Ongoing Momentum Strengthening the Ecosystem Improved developer tools (SDK, validation/debug layers) More rigorous conformance
More informationSYCL for OpenCL in a Nutshell
SYCL for OpenCL in a Nutshell Luke Iwanski, Games Technology Programmer @ Codeplay! SIGGRAPH Vancouver 2014 1 2 Copyright Khronos Group 2014 SYCL for OpenCL in a nutshell Copyright Khronos Group 2014 Why?
More informationRenderscript Accelerated Advanced Image and Video Processing on ARM Mali T-600 GPUs. Lihua Zhang, Ph.D. MulticoreWare Inc.
Renderscript Accelerated Advanced Image and Video Processing on ARM Mali T-600 GPUs Lihua Zhang, Ph.D. MulticoreWare Inc. lihua@multicorewareinc.com Overview More & more mobile apps are beginning to require
More informationCopyright Khronos Group, Page 1
3RUWDEOH1DWLYH0RELOH 0HGLD$SSOLFDWLRQV 1HLO7UHYHWW 9LFH3UHVLGHQW(PEHGGHG&RQWHQW19,',$ 3UHVLGHQW.KURQRV*URXS Copyright Khronos Group, 2007 - Page 1 3HUYDVLYH0RELOH0HGLD&RPSXWLQJ Handsets are becoming personal
More informationVulkan Launch Webinar 18 th February Copyright Khronos Group Page 1
Vulkan Launch Webinar 18 th February 2016 Copyright Khronos Group 2016 - Page 1 Copyright Khronos Group 2016 - Page 2 The Vulkan Launch Webinar Is About to Start! Kathleen Mattson - Webinar MC, Khronos
More informationStandards Update. Copyright Khronos Group Page 1
Standards Update VR/AR, 3D, Web, Vision and Deep Learning Neil Trevett Khronos President NVIDIA VP Developer Ecosystem ntrevett@nvidia.com @neilt3d www.khronos.org Copyright Khronos Group 2017 - Page 1
More informationgltf Briefing September 2016 Copyright Khronos Group Page 1
gltf Briefing September 2016 Copyright Khronos Group 2016 - Page 1 Copyright Khronos Group 2016 - Page 2 Background and Motivation OpenGL ES and WebGL have led to a proliferation of Web 3D but no standard
More informationBringing it all together: The challenge in delivering a complete graphics system architecture. Chris Porthouse
Bringing it all together: The challenge in delivering a complete graphics system architecture Chris Porthouse System Integration & the role of standards Content Ecosystem Java Execution Environment Native
More informationTake GPU Processing Power Beyond Graphics with Mali GPU Computing
Take GPU Processing Power Beyond Graphics with Mali GPU Computing Roberto Mijat Visual Computing Marketing Manager August 2012 Introduction Modern processor and SoC architectures endorse parallelism as
More informationWeb Services for Geospatial Mobile AR
Web Services for Geospatial Mobile AR Introduction Christine Perey PEREY Research & Consulting cperey@perey.com Many popular mobile applications already use the smartphone s built-in sensors and receivers
More informationThe Path to Embedded Vision & AI using a Low Power Vision DSP. Yair Siegel, Director of Segment Marketing Hotchips August 2016
The Path to Embedded Vision & AI using a Low Power Vision DSP Yair Siegel, Director of Segment Marketing Hotchips August 2016 Presentation Outline Introduction The Need for Embedded Vision & AI Vision
More informationOur Technology Expertise for Software Engineering Services. AceThought Services Your Partner in Innovation
Our Technology Expertise for Software Engineering Services High Performance Computing MultiCore CPU AceThought experts will re-design your sequential algorithms or applications to execute in parallel by
More informationCreating the Embedded Media Processing Ecosystem
Creating the Embedded Media Processing Ecosystem Neil Trevett Vice President Embedded Content, NVIDIA President, Khronos Chairman, OpenGL ES Working Group Mobile Convergence Handsets are becoming the predominant
More informationSystem Energy Efficiency Lab seelab.ucsd.edu
Motivation Embedded systems operate in, interact with, and react to an analog, real-time world Interfacing with this world is not easy or monolithic Sensors: provide measurements of the outside world Actuators:
More informationCopyright Khronos Group, Page 1
1HLO7UHYHWW 19,',$.KURQRV *URXS 2SHQ*/(6 Copyright Khronos Group, 2007 - Page 1 Handsets are becoming personal computing platform - not just phones - A real computer in your hand mobility, connectedness
More informationGPGPU on ARM. Tom Gall, Gil Pitney, 30 th Oct 2013
GPGPU on ARM Tom Gall, Gil Pitney, 30 th Oct 2013 Session Description This session will discuss the current state of the art of GPGPU technologies on ARM SoC systems. What standards are there? Where are
More informationKHRONOS STANDARDS UPDATE. Neil Trevett, GTC, 26 th March 2018
KHRONOS STANDARDS UPDATE Neil Trevett, GTC, 26 th March 2018 Khronos Mission Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standards
More informationThe Role of Standards in Heterogeneous Programming
The Role of Standards in Heterogeneous Programming Multi-core Challenge Bristol UWE 45 York Place, Edinburgh EH1 3HP June 12th, 2013 Codeplay Software Ltd. Incorporated in 1999 Based in Edinburgh, Scotland
More informationAdvantages of MIPI Interfaces in IoT Applications
Advantages of MIPI Interfaces in IoT Applications IoT DevCon Conference Hezi Saar April 27, 2017 Abstract In addition to sensors, high-resolution cameras are key enablers of IoT devices. The challenge
More informationSE 3S03 - Tutorial 1. Zahra Ali. Week of Feb 1, 2016
SE 3S03 - Tutorial 1 Department of Computer Science McMaster University naqvis7@mcmaster.ca Week of Feb 1, 2016 testing vs Software Devices and s Devices and s App Device Outline testing vs Software Devices
More informationIntroduction to OpenGL ES 3.0
Introduction to OpenGL ES 3.0 Eisaku Ohbuchi Digital Media Professionals Inc. 2012 Digital Media Professionals Inc. All rights reserved. 12/Sep/2012 Page 1 Agenda DMP overview (quick!) OpenGL ES 3.0 update
More informationHSA Foundation! Advanced Topics on Heterogeneous System Architectures. Politecnico di Milano! Seminar Room (Bld 20)! 15 December, 2017!
Advanced Topics on Heterogeneous System Architectures HSA Foundation! Politecnico di Milano! Seminar Room (Bld 20)! 15 December, 2017! Antonio R. Miele! Marco D. Santambrogio! Politecnico di Milano! 2
More informationStreaming Media Portability
Streaming Media Portability July 2006 Copyright Khronos Group, 2006 - Page 1 Copyright Khronos Group, 2006 - Page 2 Media Portability Problem Media infrastructure portability is a multi-level industry
More informationProfiling and Debugging OpenCL Applications with ARM Development Tools. October 2014
Profiling and Debugging OpenCL Applications with ARM Development Tools October 2014 1 Agenda 1. Introduction to GPU Compute 2. ARM Development Solutions 3. Mali GPU Architecture 4. Using ARM DS-5 Streamline
More informationEmbedded Software: Its Growing Influence on the Hardware world
Embedded Software: Its Growing Influence on the Hardware world ISA Vision Summit 2009, Bangalore 16 th FEB 09 V. R. Venkatesh Head, Product Engineering Services, Wipro Technologies. Wipro in Product Engineering
More informationMore performance options
More performance options OpenCL, streaming media, and native coding options with INDE April 8, 2014 2014, Intel Corporation. All rights reserved. Intel, the Intel logo, Intel Inside, Intel Xeon, and Intel
More informationKhronos Updates GDC 2017 Neil Trevett Vice President Developer Ecosystem, NVIDIA President,
Copyright Khronos Group 2017 - Page 1 Khronos Updates GDC 2017 Neil Trevett Vice President Developer Ecosystem, NVIDIA President, Khronos ntrevett@nvidia.com @neilt3d Copyright Khronos Group 2017 - Page
More informationTIOVX TI s OpenVX Implementation
TIOVX TI s OpenVX Implementation Aish Dubey Product Marketing, Automotive Processors Embedded Vision Summit, 3 May 2017 1 TI SOC platform heterogeneous cores High level processing Object detection and
More informationBringing the Power of the GPU to the Web
Copyright Khronos Group 2013 - Page 1 Bringing the Power of the GPU to the Web Neil Trevett Vice President NVIDIA President Khronos Mobile is the New Epicenter of Innovation Copyright Khronos Group 2013
More informationHSA foundation! Advanced Topics on Heterogeneous System Architectures. Politecnico di Milano! Seminar Room A. Alario! 23 November, 2015!
Advanced Topics on Heterogeneous System Architectures HSA foundation! Politecnico di Milano! Seminar Room A. Alario! 23 November, 2015! Antonio R. Miele! Marco D. Santambrogio! Politecnico di Milano! 2
More informationGPGPU Applications. for Hydrological and Atmospheric Simulations. and Visualizations on the Web. Ibrahim Demir
GPGPU Applications for Hydrological and Atmospheric Simulations and Visualizations on the Web Ibrahim Demir Big Data We are collecting and generating data on a petabyte scale (1Pb = 1,000 Tb = 1M Gb) Data
More informationKen Foust Intel, Corp. Evolving MIPI I3C SM for New Usages and Industries
Ken Foust Intel, Corp. Evolving MIPI I3C SM for New Usages and Industries Outline Introduction to MIPI I3C SM Current status Industries beyond mobile Internet of Things (IoT) High Performance Compute /
More informationSummary and Conclusions of the third Interna5onal AR Standards Mee5ng. June 15-16, 2011 Taichung, Taiwan
Summary and Conclusions of the third Interna5onal AR Standards Mee5ng June 15-16, 2011 Taichung, Taiwan Thank you to our sponsors/hosts June 17, 2011 (c) 2011 Chris5ne Perey 2 Co- Chairs & Program CommiLee
More informationIoT Market: Three Classes of Devices
IoT Market: Three Classes of Devices Typical Silicon BOM PC-Like Embedded Devices ~100 million units ATM, Retail Point of Service Intel Core $100+ Smart Things ~800 million units PLC, Edge Gateway, Thermostat
More informationPrasanna Krishnaswamy Intel Platform Architect. Imaging Systems Design for Mixed Reality Scenarios
Prasanna Krishnaswamy Platform Architect Imaging Systems Design for Mixed Reality Scenarios Agenda Mixed Reality (MR) Introduction Key Use Cases MR System Design Overview Imaging Sensors for MR Use Case
More informationCLICK TO EDIT MASTER TITLE STYLE. Click to edit Master text styles. Second level Third level Fourth level Fifth level
CLICK TO EDIT MASTER TITLE STYLE Second level THE HETEROGENEOUS SYSTEM ARCHITECTURE ITS (NOT) ALL ABOUT THE GPU PAUL BLINZER, FELLOW, HSA SYSTEM SOFTWARE, AMD SYSTEM ARCHITECTURE WORKGROUP CHAIR, HSA FOUNDATION
More information