Enable AI on Mobile Devices

Transcription

1 Enable AI on Mobile Devices Scott Wang 王舒翀 Senior Segment Manager Mobile, BSG ARM Tech Forum th June 2017, Shenzhen

2 AI is moving from core to edge Ubiquitous AI Safe and autonomous Mixed reality Hyper-efficient Secure private compute Redefining computing to deliver the possibilities of distributed intelligence 2

3 AI in your pocket 3 Source:

4 Mobile AI Use Cases Enhanced Human Interface Enhanced System / Security 4

5 From Use Cases, We Learned Comprehensive portfolio of solutions to address AI Cortex-M for voice and always-on Cortex-A for imaging and inference Mali GPU Compute, to complement the CPU with further acceleration, CV hardware CV/ML and AI is real World Facing Fish-Eye Cam Depth Cam RGB Cam User Facing Depth/Gesture Sensing 1x IR Cam for Security Far Field Audio Sensors 5

6 ARM build the foundation Enabling new applications The right platforms Distributed intelligence Security in intelligence 6

7 A system-level approach for Mobile AI Boosting AI performance with ARM new CPUs, GPU and compute library Mali GPU Cortex-A CPUs Accelerator 1.4x performance 17% GEMM efficiency gain More than 50x AI performance boost on the CPU in the next 3-5 years Up to 10x quicker response for accelerators 7

8 Yes, Heterogeneous! ARM Mobile Products for AI/ML Cortex-A Cortex-M Mali-GPU Mali-Vision SW Lib Security DynamIQ Push for higher ST performance ISA for ML Always-on. DSP capabilities with M4/M7 Best in class mobile ML efficiency gain with G72 Accelerating DNN for image recognition 5x - 15x in ML performance uplift Built in Security with CryptoCell product CoreLink 8

9 Expanding CV&ML capability in CPU&GPU Highly parallelized workload Serial control workloads CV Image pre-processing Feature extraction Feature classification& tracking/flow Recognizing Object & Decision Making ML Convolutional NN, Recurrent NN Nearest neighbors Knowledge Systems ISP/GPU DSP/accelerators CPU Acceleration enablement Expanding AI capability with new ISA 9

10 Enabling growth opportunities in mobile CV, ML, AI, MR bring unprecedented opportunities for innovation and business growth across the whole ecosystem and in all market segments Developers should be able to focus on innovation not on re-implementation of common technologies and optimizations Chip manufacturers should be spending their time on features and differentiation, not on enabling To help, ARM has recently released the Compute Library 10

11 ARM Compute Library Optimized low-level functions for CPU and GPU Key imaging, vision and ML functions Ongoing integration with ML frameworks Enable faster deployment of CV and ML on ARM Targeting CPU (NEON) and GPU (OpenCL) Significant performance uplift compared to OSS alternatives Publically available Open-source (GitHub), MIT license, free Key Functions categories Basic arithmetic Convolutions Colour manipulation Feature detection Pyramids Filters Image reshaping Mathematical functions Neural network GEMM 11

12 Library Functions Choosing what to include Standard APIs Developers ARM Product Roadmaps CV & ML functions OEMs and SiPs Machine Learning frameworks 12

13 Sample of users of ARM Compute Library 13 UPDATED: 25 APRIL 2017

14 ARM Compute Library Faster than OpenCV Speed-up compared to equivalent NEON functions from OpenCV Tested on Huawei Mate 8 (Kirin 950, Octa-core, 2300 MHz, ARM Cortex-A72 and ARM Cortex-A53) Workload is single-core, single-thread, 64b - DVFS disabled 14

15 Benchmarks cycle consumption breakdown 15

16 ARM Compute Library Faster than OpenCV Speed-up compared to equivalent NEON functions from OpenCV Tested on Huawei Mate 8 (Kirin 950, Octa-core, 2300 MHz, ARM Cortex-A72 and ARM Cortex-A53, 64-bit) Workload is single-core single-thread - DVFS disabled 16

17 Demo: real-time food calories count 17

18 Mobile Application from ThunderView Real time food calorie counting application on smartphone Segments the food from its background and works out its volume using computer vision Identifies the food name using a machine learning trained model Estimates the calories count based on an on-device database Works with mono, stereo and hybrid sensors Accelerated using ARM Compute Library on ARM CPU and GPU New use-case enabled on existing smartphone technology 18

19 ThunderView: Compute Library improvements 10% faster 6% faster 126% faster Lower is better The ARM Compute Library is an excellent tool. Our engineers were able to improve performance of the critical path of our software, in particular convolution, using the ARM optimised CPU and GPU routines, and this has helped us achieve our performance targets and getting our solution to market sooner - Yu Yang, CEO of ThunderView 19

20 Software optimizations for TensorFlow TensorFlow was designed with mobile and embedded platforms in mind Google [1] ARM is collaborating with Google Optimization of low-precision GEMM primitives Porting and optimization of TensorFlow to ARM CPU and ARM Mali GPU Integration of the ARM Compute Library with TensorFlow Adding support for the new asynchronous graph backend Optimizations and porting of Eigen via SYCL [2] [1] [2] C++ Single-source Heterogeneous Programming for OpenCL 20

21 Software optimizations for TensorFlow Improve ML on shipping cores Optimized matrix-multiplication FP32 Low-precision arithmetic Targeting Cortex-A53 and Cortex- A57/A72 Accelerating Google Translate ondevice performance Up-streamed into gemmlowp Image recognition Machine translation 21

22 HOG and SVM Histogram of Gradients (HOG) is a dense feature extraction algorithm useful to determine shapes within an image Support Vector Machine (SVM) is a supervised machine learning algorithm that can be used for binary classification Person Hyperplane Not a person 22

23 Evaluating HOG/SVM performance OpenCV Compute Library 213.4ms Lower is better 95.4ms Image resolution 640x480 Speed-up 7.1 x 73.4ms 800x x 47.6ms 960x x 33ms 6.7ms 7.9ms 13.2ms 15.1ms 1280x x 640x x x x960 Speed-up compared to equivalent NEON functions from OpenCV Tested on Firefly board - CPU: ARM 4x Cortex-A17 1.8GHz, GPU: ARM Mali-T760 (MP4) 600 MHz 23

24 Conclusion Mobile is at the heart of the AI revolution ARM is aggressively investing in AI Heterogeneous mobile platforms, software libraries and frameworks enable leading-edge AI use-cases today ARM Compute Library provides performance benefit out-of-thebox compared with existing open source solutions 24

25 Start your AI journey on mobile with ARM compute library 25 Confidential

26 The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited (or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners. Copyright 2017 ARM Limited