Fast Hardware For AI

Size: px

Start display at page:

Download "Fast Hardware For AI"

Dayna Golden
5 years ago
Views:

1 Fast Hardware For AI Karl Freund Sr. Analyst, AI and HPC Moor Insights & Strategy Follow my blogs covering Machine Learning Hardware on Forbes: 9/30/2018 1

2 My background (Or, why am I here?) >30 years Marketing Exec for Servers and GPUs Industry Analyst with Moor: HPC and AI 50 articles on Forbes Dozens of research projects & papers My Clients include most silicon vendors Due Diligence projects for Private Equity Investors 9/30/2018 2

3 Why are YOU here? 1. AI is projected to add $25.7 Trillion to GDP by 2030 OR 2. AI may displace over 25% of US Workers by /30/2018 3

4 AGENDA MARKET INTRODUCTION HW TECHNOLOGIES FOR DEEP LEARNING OVERVIEW OF MAJOR PLAYERS SAMPLE STARTUPS And a little surprise.. 9/30/2018 4

5 -- OR?? 9/30/2018 5

6 Why this event? Why now? 1. AI will impact every company in every industry over the next decade 2. AI demands silicon that is > 100X faster 3. Combined, these factors create tremendous opportunities in a $25-$30B market 9/30/2018 6

Let s Define the Space AI Machine Learning Deep

can be used to classify / process new data (AKA

7 Let s Define the Space AI Machine Learning Deep Learning Includes all applications where the computer mimics human intelligence. Applications which use known data to create models that can be used to classify / process new data (AKA Statistics) Discover and model complex unstructured data for perception and cognition. 9/30/2018 7

8 DEEP LEARNING 9/30/2018 8

9 Deep Learning Requirements or, What is different this time? 1. Data: Massive Sample Datasets w/ known categories 2. Software Algorithms: Open Source Frameworks 3. Compute: Supercomputing-class performance is required It can still take hours or days for each training run, even with fast GPUs Linear Algebra (matrix multiplications) underpins all Deep Learning algorithms Low precision is often acceptable (16-bit vs. 32-bit, even 8 bit for inf.) Memory capacity and bandwidth improves performance 9/30/2018 9

Deep Learning HW 101: The Basics AI development (training) requires massive processing power Accelerators are required to achieve reasonable performance NVIDIA GPUs are today s gold standard for

10 Deep Learning HW 101: The Basics AI development (training) requires massive processing power Accelerators are required to achieve reasonable performance NVIDIA GPUs are today s gold standard for training DNN s Google TPU, Intel Nervana, and startups seek to challenge NVIDIA here AI deployment (inference) will be a large diverse market CPU s, FPGA s, GPU s, and ASICs will all have roles in Inference applications In 2018, the market is probably 80%/20% Training/Inference This could flip to 20%/80% in the future 9/30/

A Machine Learning Application Landscape Consumer/Entertainment/Retail Personal VR/Gaming Smart Displays Personal Assistants Ad Targeting and E- Commerce Transportation/Infrastructure Autonomous Cars

11 A Machine Learning Application Landscape Consumer/Entertainment/Retail Personal VR/Gaming Smart Displays Personal Assistants Ad Targeting and E- Commerce Transportation/Infrastructure Autonomous Cars &Trucks Transportation & Grid Control Traffic & Network Analytics Enterprise Operations Delivery Drones, Warehouse Robots Cyber Security Sales, Marketing & Customer Service Oil & Gas/Agriculture Field Drones & Robots Climate, Water, Energy & Flow Control Field Sensor Data Analytics Industrial/Military Robots/Cobots, UAV, Inspection Factory Control & Surveillance Factory & Operations Analytics Medical/Healthcare Medical Imaging & Surgical Robots Medical Diagnostics Clinical Analytics & Recommendations Source: Machine Learning Landscape from Moor Insights & Strategy Edge Resident Apps Hybrid Solutions Cloud Hosted Apps 9/30/

12 Hardware for Machine Learning CPUs: Good for Inference & large-model training FPGA s: Good for Inference GPU s: Great for training and demanding inference workloads ASIC s: Great for inference, and training, but Requires significant R&D investment and chip volumes to break even Entails long development cycles in a fast-moving world 9/30/

13 Datacenter Inference Embedded Training Hardware Technologies Used in Machine Learning AMD VEGA GPU Google Cloud TPU (2&3) Nvidia Tesla V100 FPGA s (Intel & Xilinx) FPGA SOCs Edge TPU Nvidia Jetson Nvidia Drive PX2 FPGA (Xilinx & Intel) Google TPU Nvidia Tesla P40 & P4 Performance & Functionality 9/30/

14 NVIDIA: Training leader also targeting inference Volta delivers industry leading 125 TOPS today Supported by All clouds, all Networks, all OEMs NVSwitch creates a virtual pool of GPUs with 512 GB HBM2 memory Pascal GPUs (P4 and P40) are being deployed in clouds for complex and large-batch inference Recently replaced by the Turing-based T4 ( Watts) Xavier SOC provides flexible platform for robotics and Autonomous Vehicles DLA is open-source sleeper ASIC for CNNs More to come in 7nm Volta successor? 9/30/

Intel has Lots of Irons in the AI Fire 2 new Xeons w/ ~11X AI performance DLBoost ( Casade and Cooper Lake in 18 & 19) Nervana Neural Network Processors (training) ships late 2019 Intel has

15 Intel has Lots of Irons in the AI Fire 2 new Xeons w/ ~11X AI performance DLBoost ( Casade and Cooper Lake in 18 & 19) Nervana Neural Network Processors (training) ships late 2019 Intel has acquired Mobileye for AEVs and Movidius for embedded vision Intel s Exascale Dataflow Engine could be a game changer in the 2022/3 timeframe. Intel claims $1B in AI in past year 9/30/

16 9/30/

17 FPGA s: Flexible, Programmable Inference HW programmable accelerators from Intel and Xilinx Microsoft champions Intel FPGAs in datacenter apps Xilinx and AWS offer pre-built solutions for AI, others Xilinx 7nm Everest ACAP will offer Engines for AI. Could be a game changer Xilinx Claim orders of Magnitude performance increases More to come at XDF, Oct /30/

18 Microsoft Hardware and Software for Intel FPGAs 9/30/

19 AMD: Laying the Foundation AMD has been building a software story for >2 years Can up-compile CUDA for AI and HPC New libraries improve performance Strong DOE partnership agreements in place AMD s Vega GPU offers ~ NVIDIA Pascal performance 7nm part expected by end of year. SC 18? Long-term potential for a Big APU with Infinity Fabric Lots of interest, especially in Asia, for AMD to succeed 9/30/

20 Google s TensorFlow Processor Google TPU is an ML ASIC Available now in the Google Cloud TPU1: Used as an inference engine TPU 2 (now) & 3 (future): Used for training and complex inference Edge TPU recently announced for inference Limited Availability TPU1-3 are only available in Google Cloud Only supports the TensorFlow framework Does not (currently?) support direct HW access for tuning & optimization 9/30/

21 Startups (Big and Small) Could Disrupt Market Over 40 Startups from US, UK, India, and China: $25B Opportunity Most claim X GPU Performance for AI Some will focus on Inference, others on Training Some will focus on datacenter, others on embedded and Autonomous Vehicles Some are accelerators (PCIe), others are complete systems All will need to address the ecosystem requirements dominated by NVIDIA 9/30/

22 Introducing... Habana!! 9/30/

Accelerating your Embedded Vision / Machine Learning design with the revision Stack. Giles Peckham, Xilinx

Accelerating your Embedded Vision / Machine Learning design with the revision Stack Giles Peckham, Xilinx Xilinx Foundation at the Edge Vision Customers Using Xilinx >80 ADAS Models From 23 Makers >80