Machine learning for the Internet of Things

Transcription

1 Machine learning for the Internet of Things Chris Shore Director of Embedded Solutions Arm 2018 Arm Limited April 2018

2 More Intelligence at the Edge Arm Cortex-M Expanding opportunity for the embedded intelligence market Sensor fusion Contextual awareness Voice activation Biometrics More data More processing More algorithms More compute Arm Limited

3 Scalable and distributed compute needed Artificial intelligence Machine learning Deep learning Artificial intelligence (AI) components have different processing needs High throughput tasks Low latency tasks Large scale adoption of AI Increase network bandwidth efficiency Increase energy-efficiency Optimize for specific system requirements Algorithms: CNNs, RNNs, etc. Usage of machine learning (ML) Rules and decisions depend on too many factors Scalability: lots of data, complexity of data Arm Limited

4 Range of Edge Applications Increasing performance (Ops/second) Arm Cortex-M systems Keyword detection Pattern training ~1-10mW Object detection Voice & image recognition ~1W Image enhancement ~10W Autonomous drive ~100W Arm Limited Increasing power and cost (silicon)

5 Training and inference are different Cloud side Train the model Throughput oriented High performance High Performance Compute, servers, GPU Large data sets Device side Apply the model Low/mid performance Low latency Efficiency Security Privacy Embedded systems, heterogeneous processing Arm Limited

6 Machine learning trade-offs in embedded systems Considerations Neural networks Number of outputs Data type HW/SW Machine learning Number of inputs Transfer function Neurons per hidden layer Software Number of hidden layers Accuracy Memory Hardware System System considerations Latency Communication bandwidth Energy efficiency Privacy Security Scalability Optimization of system and machine learning requirements Arm Limited

7 Embedded intelligence with neural networks Inputs Input layer m hidden layers Output layer x 1 x 2 x 3 x n-1 x n w 2 w 1 w 3 y = H ( z) w n-1 w n n z = å wi xi i= 1 Output Transfer function HW and SW support for matrix multiplication and matrix-vector multiplication Arm Limited

8 Arm: Powering embedded intelligence for ML Right-sized compute Arm provides the right computational engine Scalable platforms Scalable SoCs tailored for intelligent applications Libraries SW libraries enabling the ecosystem Arm Limited

9 Right-sized processing at the right node System requirements push for distributed intelligence Data gets cleaned (filtered), processed, analyzed and anonymized Improving efficiency and securing data at its source heterogeneous solution Datacenters Network infrastructure Edge compute & gateways Rich devices Constrained devices Distributed intelligence across heterogeneous platforms for energy-efficiency Cortex-A powered products Cortex-R powered products Cortex-M powered products Arm Limited

10 Flexible and scalable processors for end nodes Cortex-M23 Cortex-M3 Cortex-M0+ Cortex-M0 Cortex-M33 Cortex-M4 SP FPU SIMD Cortex-M7 SP/DP FPU SIMD Cortex-A32 Cortex-A7 NEON Optimized machine learning libraries available Arm Limited

11 Heterogeneous systems enable efficient ML Compute performance Advanced ML using Cortex-A subsystem Highly efficient Cortex-M subsystem filters and processes data using ML Time Sophisticated end node based on a heterogeneous architecture Cortex-A subsystem Shared L2 DMC DDR Interconnect Sensor Cortex-M subsystem AHB/AXI interconnect Local memory AHB interconnect Timer Always-on SRAM Arm Limited

12 Accelerated machine learning with coprocessors Simple end node utilizing coprocessors Sense Decide Actuate Coprocessor Coprocessor #8 Cortex-M33 TrustZone for ARMv8-M untrusted trusted - Gather information - Pre-filter data - Apply ML - Accelerate on coprocessor - Take actions - Communicate AHB5 interconnect Sensor Radio SRAM Arm Limited

13 Ultra High Performance with ML and OD Processors Use OD and ML Processors together to achieve high resolution real-time identification and recognition 1. Arm OD Processor detects faces and only passes on these areas of interest 2. Arm ML Processor runs fine-grain machine learning on area of interest to determine identity Combined solution ü OD Processor processes more than 1 TOPs/s ü ML Processor focuses on recognition through CNN ü Allows both face detection and recognition at the edge ü Enables a new class of smart camera and other vision-based devices Image Metadata stream (Regions of interest) stream Arm OD Processor Arm ML Processor Beth Ben Arm Limited

14 Mobile Experiences: Insights From Advanced Compute ML and OD processors enable smartphone linking to any screen for awareness/protection (e.g. sunglasses, ski goggles, dive masks) Blue shark ü Anti-shark suit electric current active ü Dive boat alerted Sea anemone ü Poisonous, only touch with gloves Beware hole ü Could be Moray eel hideaway Bigeye snapper ü Not protected Arm Limited

15 Living: Interpreting Data for Smart City Planning ML and OD processors embedded in city camera systems for real-time information and control Detecting pedestrian impedance, congestion, safety issues (e.g. abandoned bag) Road-obstruction recognition, linked to GPS network to look for specific information (e.g. lost child) By 2030, a projected 27 percent of people worldwide will be concentrated in cities with at least 1 million inhabitants. - United Nations Arm Limited

16 Artificial audio intelligence makes a home smarter Machine learning designed specifically for task of sound recognition Extraction of hundreds of sound features - Scalable platform capable On-device sound recognition of: Smoke and CO alarm Baby cry Window breaking Dog barking Low memory and power footprint for maximum OEM design freedom Ai3 - ARTIFICIAL AUDIO INTELLIGENCE SOUNDS IN HOME ACOUSTIC FEATURE ANALYSER ADVANCED MACHINE LEARNING DECISION ENGINE Runs on ARM Cortex-M & Cortex-A Arm Limited Courtesy of Audio Analytic

17 Prevent respiratory attacks CoughAware More than 40 different types of cough Respiratory disease monitoring Providing peace of mind for families & improved health for patients Providing cost effective disease management for CCGs Cough as an early warning system Asthma attacks are avoidable Analyse cough frequency and duration Runs on Cortex-M4 and Cortex-M Arm Limited Courtesy of Quvium and DSP Concepts

18 2018 Arm Limited It is easy to recognize speech It is easy to wreck a nice beach

19 Energy-efficient speech recognition Listening Always-on Triggered processing Cloudbased compute Cortex-M Powered MCU Cortex-A Powered SoC High performance compute Pre-filter data Voice activity detection Keyword spotting ~1K neurons Speech recognition ~10K neurons Natural language processing Simultaneous processing of speech Arm Limited

20 CMSIS-DSP library optimized for machine learning Keyword spotting algorithm using fully connected deep neural networks 9 KWS network time to process 10s of on Cortex-M4 based MCU 8 Optimizing code for 16-bit fixed-point shows only minimal accuracy impact Significant gains from optimized CMSIS- DSP library Matrix multiplication Matrix-vector multiplication Execution time (s) 1.3x 2.6x 4.8x 1 0 Float32 Float32 (CMSIS) 16-bit fixed point (CMSIS) 16-bit fixed point (Improved CMSIS library) Arm Limited

21 Optimum ML Performance on Arm for Any Application Arm NN software translates existing NN frameworks: TensorFlow, Caffe, Android NNAPI, MXNet etc. Developers maintain existing workflow and tools Reduces overall development time Abstracts away the complexities of underlying hardware CMSIS-NN Cortex-M CPU Compute Library Cortex-A CPU CMSIS-NN 5x better efficiency and performance for NN functions NN Frameworks Arm NN Compute Library Mali GPU Compute Library Arm ML processor Compute Library 15x faster than other opensource software (OSS) Partner IP driver and SW functions 3 rd party IP Arm Limited Arm Limited

22 Summary Heterogeneous processing - Right-sized processing at the right node Machine learning in embedded systems demand low latency and data security Energy-efficient machine learning on ARM Cortex processors Scalable and configurable platform addressing various performance points Optimized libraries TrustZone Arm Limited

23 Thank You Danke Merci Gracias Kiitos 감사합니다 ध"यव द ה ד ו ת Arm Limited

24 Resources CMSIS-NN paper: CMSIS-NN blog: CMSIS-NN Github link: KWS (Keyword Spotting) paper: KWS blog: KWS Github link: ArmNN: ArmNN SDK blog: Arm Limited

25 The Arm trademarks featured in this presentation are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners Arm Limited