An introduction to Machine Learning silicon November 28 2017
Insight for Technology Investors
AI/ML terminology Artificial Intelligence Machine Learning Deep Learning Algorithms: CNNs, RNNs, etc. Additional terms Location Cloud processing done in data farms Edge processing done in local devices Types of machine learning Model a mathematical approximation of a collection of input data Training in deep learning, data-sets are used to create a model Inference using a model to check against new data 3
Classification error Neural Networks (NNs) outperform humans 28% 26% AlexNet, 8 layers ZF, 8 layers VGG, 19 layers GoogleNet, 22 layers Data for ImageNet Large Scale Visual Recognition Challenge 16% 12% 7.3% 6.7% 3.6% 3% ResNet, 152 layers CUImage Human error Deep networks, introduced in 2012, resulted in big improvements 2010 2011 2012 2013 2014 2015 2016 shallow deep (Image source: Synopsys) Error rates have now stabilized at ~3% 4
Machine Learning training Training data Model For each piece of data used to train the model, millions of model parameters are adjusted. The process is repeated many times until the model delivers satisfactory performance. 5
Machine Learning inference Input Model Output 97.4% confidence 96.4% confidence When new data is presented to the trained model, large numbers of multiply-add operations are performed using the new data and the model parameters. The process is performed once. 6
Why is on-device ML driving AI to the Edge? Bandwidth Power Cost Latency Privacy 7
Inference everywhere Mobile Automotive Robotics Drones IoT Surveillance Augmented reality Shipping & logistics 8
Processor options for Machine Learning workloads 9
A System-on-Chip contains multiple compute engines Main processor (CPU) A versatile compute engine for running rich software. The main CPU runs device s operating system, applications and user interface. It also manages the flow of data to specialist processors in the device. Graphics processor (GPU) Used for generating 2D/3D images and executing highly-parallelised workloads such as neural network arithmetic Digital signal processors (DSPs) A specialist form of CPU, optimised for analysing waveforms. Useful for radio control, sensor readings, audio and image processing Accelerators Heavily-optimised data processors for frequently-used tasks, e.g. encryption, video, computer vision 10
Comparing processor options for Machine Learning CPU DSP Training Inference Usability Hardware cost Power efficiency Hardware cost Power efficiency Flexibility Programmability GPU 1 2 3 1 2 Accelerator FPGA 1 = High volume, evolving workload 2 = High volume, stable workload 3 = Low volume, evolving workload 1 = A client device that requires a GPU for graphics 2 = A device that uses a GPU for ML work only 11 Weak, relative to alternatives Good, relative to alternatives
Performance Processor options for various sizes of chip Machine Learning demands (accuracy, response time) vary by use case All use cases can default to a CPU A GPU is often a good all-rounder solution Accelerators are useful when it is essential to either maximize response speed or minimize power consumption Cortex-M Accelerator Cortex-A (little CPU) Accelerator Cortex-A (big CPU) Keyword detection GPU Speech recognition Visual object recognition Visual object detection 12 Silicon area / power consumption
Arm s ML computing platform AI Applications: ML, CV, speech recognition etc. Applications Neural network frameworks (e.g. Tensorflow, Caffe, AndroidNN) Optional Spirit libraries & model sets Stable SW interfaces Compute library Arm DS-5 / Keil tools / compilers / drivers Spirit metadata library 13 SVE CPU CPU GPU Partner IP: DSPs, FPGAs, accelerators Spirit Computer Vision Provided by Arm Provided by third-party Edge devices
Machine Learning is driving all of Arm s technology roadmap Processor design Software support Computer vision 14
The Arm trademarks featured in this presentation are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners. www.arm.com/company/policies/trademarks 15