在数据中心中加速 AI - Xilinx 机器学习套件 (Xilinx ML Suite )

Transcription

1 赛灵思高级主任 DSP/ 机器学习专家赛灵思高级主任 DSP/ 机器学习专家 赛灵思技术日 XILINX TECHNOLOGY DAY 在数据中心中加速 AI - Xilinx 机器学习套件 (Xilinx ML Suite ) 王宏强赛灵思资深主任 DSP/ 机器学习专家 2019 年 3 月 19 日

2 机器学习推断是赛灵思的长项 TRAINING Input cat =? labels dog Training: Process for machine to learn and optimize model from data Many Error INFERENCE Input dog Focus Inference: Using trained models to predict/estimate outcomes from new observations in efficient deployments Fewer

3 从云到端加速 AI 应用 Featuring the Most Powerful FPGA in the Cloud Deep Learning Applications Virtex Ultrascale+ VU9P Zynq Ultrascale+ MPSoC Cloud On Premises Edge

4 深度学习模型 Multi-Layer Perceptron Classification Universal Function Approximator Autoencoder Convolutional Neural Network Feature Extraction Object Detection Image Segmentation Recurrent Neural Network Sequence and Temporal Data Speech to Text Language Translation Classification Object Detection Segmentation Dog

5 使用开源软件进行无缝部署 Deploy xfdnn Middleware, Tools and Runtime From Xilinx From Community xdnn CNN Processing Engine *TensorFlow Q4 2017

6 Xilinx ML Suite Supported Frameworks: Caffe / MxNet / Tensorflow / Darknet Python Support Jupyter Notebooks available: Image Classification with Caffe Using the xfdnn Compiler w/ a Caffe Model Using the xfdnn Quantizer w/ a Caffe Model Pre-trained Models Caffe 8/16-bit GoogLeNet v1 / ResNet50 / Flowers102 / Places365 Python 8/16-bit Yolov2 MxNet 8/16-bit GoogLeNet v1 xfdnn Tools Compiler Quantizer

7 xfdnn 推断工具箱 (Toolbox) Graph Compiler Network Optimization xfdnn Quantizer Python tools to quickly compile networks from common Frameworks Caffe, MxNet and Tensorflow Automatic network optimizations for lower latency by fusing layers and buffering on-chip memory Quickly reduce precision of trained models for deployment Maintains 32bit accuracy at 8 bit within 2%

8 基于 xdnn 处理引擎的 ML Suite 套件 Adaptable AI algorithms are changing rapidly Adjacent acceleration opportunities Realtime 10x Low latency than CPU and GPU Data flow processing xdnn PE xdnn PE xdnn PE xdnn PE FPGA Platform DDR Efficient Performance/watt CPU Low Power

9 Overlay Architecture 基于赛灵思 FPGA 灵活多变特性的定制化处理器 Customized overlays with ISA architecture for optimized implementation Easy plug and play with Software Stack MLP Engine Scalable sparse and dense implementation xdnn CNN Engine for Large 16 nm Xilinx Devices Deephi DPU Flexible CNN Engine with Embedded Focus CHaiDNN HLS based open source offering Deephi ESE LSTM Speech to Text engine Random Forest Configurable RF classification

10 快速提升功能和性能 xdnn-v1 Q4CY17 Array of Accumulator Int16 (Batch=1) and Int8 (Batch=2) support Instructions: Convolution, ReLU, Pool, Elementwise Flexible kernel size(square) and strides 500 MHz xdnn-v2 Q2CY18 All xdnn-v1 Features DDR Caching: Larger Image size New Instructions: Depth-wise Convolution, De-convolution, Up-sampling Rectangular Kernels 500 MHz xdnn-v3 Q4CY18 New Systolic Array Implementation: 2.2x lower latency Instruction Level Parallelism non-blocking data movement Batch=1 for Int8 lower latency Feature compatible with xdnn-v MHz

11 XDNN v3 特性集 Features Description Kernel Sizes W: 1-15; H:1-15 Convolution / Deconvolution / Convolution Transpose Strides W: 1,2,4,8; H: 1,2,4,8 Padding Same, Valid Dilation Factor: 1,2,4 Activation ReLU/pReLU Bias Scaling Value Per Channel Scale & Shift Value Per Channel Supported Operations Max Pooling Kernel Sizes W: 1-15; H:1-15 Strides W: 1,2,4,8; H: 1,2,4,8 Padding Same, Valid Kernel Sizes W: 1-15; H:1-15 Avg Pooling Strides W: 1,2,4,8; H: 1,2,4,8 Padding Same, Valid Element-wise Add Memory Support Width & Height must match; Depth can mismatch. On-Chip Buffering, DDR Caching Expanded set of image sizes Square, Rectangular Upsampling Strides Factor: 2,4,8,16 Miscellaneous Precision Int16-bit or Int8-bit

12 Xilinx DNN (xdnn) 处理器 Image Queue Instruction Buffer Spill / Restore DMA Controller Execution Controller Weights DMA Controller Systolic Array Configurable Overlay Processor DNN Specific Instruction Set Convolution, Max Pool etc. Any Network, Any Image Size High Frequency & High Compute Efficiency Compile and run new networks Bias Bias Bias Bias Pooling/ EWA ReLU ReLU ReLU ReLU Pooling Pooling Pooling Pooling Cross Bar

13 xfdnn 流程 CNTK Caffe2 PyTorch Tensorflow MxNet Caffe F R O N T E N D Framework Tensor Graph to Xilinx Tensor Graph xfdnn Tensor Graph Optimization ONNX xfdnn Compiler xfdnn Compression Model Weights Calibration Set Image xfdnn Runtime (python API) CPU Layers FPGA Layers

14 xdnn v3 在 Alveo U200 上的实现 3 Large 96x16 PEs 1 in each SLR 5.2 ML Shell 720 MHz/360MHz Resource Count Utilization LUTs 658k 52% DSPs % BRAM % URAM %

15 xdnn v3 在 Alveo U250 上的实现 4 Large 96x16 PEs 1 in each SLR standard 5.2 Shell Kernels at 700 MHz/350 MHz Resource Count Utilization LUTs 876k 51% DSPs % BRAM % URAM %

16 灵活多变 : 多网络配置 Application: Image Classification Application: Speech Application: Localization Application: Object Detection Model: Resnet50 Model: Googlenet v1 Model: FaceNet Model: Yolo v2 Framework: Caffe Framework: MxNet Framework: TesnorFlow Framework: Caffe xfdnn PCIe SDx Runtime Host 1 FPGA Provides 4 Virtual Accelerators For Real Time Deep Learning

17 灵活多变 : 部署您自己的 IP! Application: Localization Application: Object Detection Custom Application Model: FaceNet Framework: TensorFlow xfdnn Model: Yolo v2 Framework: Caffe PCIe xdnn PE Custom FPGA Platform Infrastru cture SDx Runtime Host Integrate Custom Applications Directly with xdnn Processing Engines

18 自定义的深度学习流程 xdnn Video Decode + Processing Video + ML Genomics + ML xdnn XDNN Risk Modelling + ML Database + ML xdnn Video Processing + Encode Network IPS + ML Storage + ML Integrate Custom Applications with xdnn. Lower end-to-end latency

19 xdnn GoogLeNet v1 性能 图像尺寸为 224x , ,124 3, Images/s , Latency (ms) Alveo U200 Latency Mode (INT8) Alveo U200 Throughput Mode (INT8) Alveo U250 Latency Mode (INT8) Alveo U250 Throughput Mode (INT8) 0

20 xdnn YOLO v2 性能 图像尺寸为 608x Images/s Latency(ms) Alveo U200 Latency Mode (INT8) Alveo U250 Latency Mode (INT8) 0

21 ML Suite: 赛灵思和深鉴技术的完美集成 Edge/Embedded Cloud/DC Models 20+ pruned / customized / basic models Deephi Pruning Coming to ML Suite at XDF Software Stack Deephi Quantizer Deephi Compiler SDSoC SDAccel xfdnn Quantizer xfdnn Compiler Deephi Runtime xfdnn Runtime FPGA IP Deephi DPU Deephi LSTM xdnn Platforms Z7020 Board Z7020 SOM ZU2/3 SOM ZU2/3 Card ZU9 Card ZCU102 ZCU104 Ultra96 Xilinx U200, U250, U280

22 Adaptable. Intelligent. 赛灵思技术日 XILINX TECHNOLOGY DAY