Accelerating your Embedded Vision / Machine Learning design with the revision Stack. Giles Peckham, Xilinx

Transcription

1 Accelerating your Embedded Vision / Machine Learning design with the revision Stack Giles Peckham, Xilinx

2 Xilinx Foundation at the Edge Vision Customers Using Xilinx >80 ADAS Models From 23 Makers >80 ProAV & Broadcast Suppliers >60 Smart Camera & Visualization Suppliers >50 Industrial Vision Equipment Makers >10 Medical Diagnostic Suppliers >5 VR/AR Equipment Makers >5 Drone Suppliers Page 2

3 Machine Learning: From the Edge to the Cloud Consumer/Entertainment/Retail Personal VR/Gaming Smart Displays Personal Assistants Ad Targeting and E-Commerce Transportation/Infrastructure Autonomous Cars &Trucks Transportation & Grid Control Traffic & Network Analytics Enterprise Operations Delivery Drones, Warehouse Robots Cyber Security Sales, Marketing & Customer Service Oil & Gas/Agriculture Field Drones & Robots Climate, Water, Energy & Flow Control Field Sensor Data Analytics Industrial/Military Robots/Cobots, UAV, Inspection Factory Control & Surveillance Factory & Operations Analytics Medical/Healthcare Medical Imaging & Surgical Robots Medical Diagnostics Clinical Analytics & Recommendations Source: Machine Learning Landscape from Moor Insights & Strategy Edge Resident Apps Page 3 Hybrid Solutions Cloud Hosted Apps

4 Applications: Wide Range of Rapidly Changing Vision Guided Systems Embedded Vision Systems Factory Robotics Vision Guided Autonomous Systems Vision Guided Cobots Camera Equipped Aircraft Sense and Avoid & Autonomous Drones Physical Displays and HMI Augmented Reality and HUDs Forward Auto Camera Autonomous Vehicles Video Security Cams Automated Surveillance Medical Imaging and Human Eye Automated Medical Diagnostics Page 4

5 Must Keep Up with Sensor Fusion Evolution Sensor Types Velocity Displacement Position Level Flow Angle Fluid Speed Photon Density Gas Distance Light Force Humidity Proximity Imaging Magnetic Chemical Vibration Acoustic Acceleration Pressure Sound Temperature Solid State 3D Lidar Lidar Radar Infrared HI Res CCD Ultrasonic Multispectral RF Gyroscopic Accelerometer Imagers IMU Multispectral Cameras Multi-mode Lidar GPS Traditional Growth AI Expansion Sensor Categories Page 5

6 Must Keep Up with Neural Network Evolution Perceptron ANN Perceptron Belief Net TDNN ResNET CNN LeNet5 DCNN Deep Belief Neural Net ZFNet Alex Net VGG Net Microsoft ResNet GoogLeNet Spatial Transformer DCCN Net WaveNet Fractal Net Madaline Inception Net Spike NN Fast RCNN Faster RCNN DTNN 40 Years 5 Years 2 Years Future Back Propagation FINN VDCNN DRQN SGAN HashNet DenseNet SSD QuickNet YOLO ROLO StuffNet Squeeze Net Floating Point 8-bit to 1-bit and Variable Precision Inference Page 6

7 Xilinx Unique Application Advantages Responsive Reconfigurable Optimized from Sensors to Inference & Control Reconfigurable for Latest Networks & Sensors Connected Any-to-Any Connectivity Mandates: From Embedded Vision to Autonomous Systems Page 7

8 Xilinx Unique Application Advantages Responsive Reconfigurable Connected Optimized from Sensors to <8-bit Inference & Control Barrier to Broad Adoption: Reconfigurable for Latest Networks & Sensors Software Defined Programming, Libraries and Frameworks Any-to-Any Connectivity Mandates: From Embedded Vision to Autonomous Systems Page 8

9 Application Development DNN CNN GoogLeNet SSD FCN Algorithm Development Platform Development Page 9

10 Development Time Removing the Barrier to Broad Adoption: revision Stack 20% Xilinx/80% User ML Apps OpenCV Apps A subsystem design used to take 3 weeks I ve done it in 4 days with SDSoC - DSP Engineer Algorithm to RTL 80% Xilinx/20% User System Integration Bitstream Generation SDSoC C/C++ revision will shorten our development cycle for new products and upgrades by up to 12 months - System Architect Ease of Use Traditional RTL flow OpenCV Machine Learning Page 10

11 Machine Learning Inference Page 11

12 The Divergence of Training and Inference in Machine Learning TRAINING Input cat =? labels dog Training: Process for machine to learn and optimize model from data Many Error Input Inference: Using trained models to predict/estimate outcomes from new observations in efficient deployments INFERENCE dog Inference now 8 bit and below for maximum efficiency Fewer FP-32 FIXED-16 (INT16) FIXED-8 (INT8) Difference vs FP32 Top-5 Accuracy VGG % 866% 864% (02%) GoogLeNet 886% 885% 857% (29%) SqueezeNet 814% 814% 803% (11%)

13 Inference Precisions Moving to Lower and Variable Precision # of weight bits # of weight bits Citation: Page 13

14 Xilinx: Future Proof Architecture for Any Precisions Beyond 8 bit FP32 FP/INT16 INT8 INT6 INT4 INT2 INT1 CPU Limited to 32 bit operations GPU New devices required to support change in precision efficiently Xilinx 16 2 x Reconfigurable to scale and optimize for different precisions Page 14

15 Low Latency Inference by Layer to Layer Dataflow On Chip Xilinx: Maximum local data reuse layer merging between layers GPU/CPU: Dataflow using Off chip memory On-chip Memory Nvidia Tegra X1 (GPU L2 Cache) Xilinx ZU7 (BRAM + URAM) 2 Mb 38 Mb Up to 19x More On-chip Memory than SoCs and egpus *Nvidia TX1 spec: Page 15

16 xfdnn: Direct Deep Learning Inference from Caffe Import prototxt and trained weights Call prototxt runtime API in your application Cross-compile for Cortex-A53 and run on a board Compiles only ARM software code in minutes No hardware compilation Page 16

17 Deep Learning Design Examples batch = 1 batch = 1 batch = 1 Mar 2017 Roadmap Images/s Power (W) Images/s/watt Mar 2017 Roadmap Images/s 63 Coming Soon Power (W) 65 Coming Soon Images/s/watt 10 Coming Soon Mar 2017 Roadmap Images/s 70 Coming Soon Power (W) 65 Coming Soon Images/s/watt 11 Coming Soon Page 17

18 Performance: More efficient than Nvidia Tegra X1 6x Images/sec/watt 42x Frames/sec/watt batch = 1 Xilinx ZU9 Xilinx ZU5 Nvidia TX1 Images/s Power (W) Images/s/watt Machine Learning Inference Computer Vision CV:: Xilinx ZU9 Xilinx ZU5 nvidia TX1 Frames/s Power (W) Frames/s/watt CV:: LK Dense Optical Xilinx ZU9 Xilinx ZU5 nvidia TX1 Frames/s Power (W) Frames/s/watt Xilinx Benchmark Xilinx Benchmark nvidia GoogLeNet performance: ML based on Xilinx GoogleNet performance roadmap in 2H2017: 180 img/s in May 2017, 370 img/s in 2H17 LK Dense Optical Flow using pyramid = 5, iteration = 5 All benchmarks utilize as much resources as possible on GPU (~99%) and programmable logic (~70%) Page 18

19 Latency: Xilinx Provides Fastest Response Time for Any Batch Size 1/5 Latency (ms) Real Time Applications Latency batch = 1 batch = 8 Xilinx ZU9 Xilinx ZU5 Nvidia TX1 Images/s Latency (ms) Xilinx ZU9 Xilinx ZU5 Nvidia TX1 Images/s Latency (ms) Xilinx Benchmark For large batch, Nvidia latency increases significantly *nvidia GoogLeNet performance: Page 19

20 Removing the Barriers for Expansion into a Wide Range of Vision Guided Machine Learning Applications Broadening the Development and Deployment of Machine Learning Applications from the Edge to the Cloud Page 20