赛灵思技术日 XILINX TECHNOLOGY DAY 用赛灵思 FPGA 加速机器学习推断 张帆资深全球 AI 方案技术专家

Transcription

1 赛灵思技术日 XILINX TECHNOLOGY DAY 用赛灵思 FPGA 加速机器学习推断 张帆资深全球 AI 方案技术专家

2 Who is Xilinx? Why Should I choose FPGA? Only HW/SW configurable device 1 2 for fast changing networks High performance / low power with custom internal memory hierarchy Future proof to lower 3 4 Low latency end-to-end 5 precisions Scalable device family for different applications

3 Xilinx AI 解决方案 Surveillance Data center ADAS/AD Algorithm Model Zoo Pruning Quantization Reference solutions Deep learning libraries Cross-compiling tools BSP for boards DNNDK Xilinx AI SDK DPU and Accelerating IPs Dense Neural Network Pruned Neural Network Compressed Sparse Neural Network Compiler Core API Compilation Runtime Assembler Loader Driver Tracer Zynq-7020 Zynq-7020 ZU2 ZU2 ZU9

4 算法模型集合 (Zoo) 69 Models in the zoo, in which 34 models are already deployed. Network Backbone DPU Deployment Input Size OPs Paras Training Set Val Set Eva Norm Float Eva Norm Fixed Resnet50_v1 Resnet Yes 224* G 25.6M ImageNet ImageNet Inception_v3 Inception No 299* G 23.8M ImageNet ImageNet SSD VGG16 No 300* G 26.3M Voc07+12 Voc % RefineDet VGG-16 Support 480* G 29.6M Coco2014 Coco % Densebox Yes 320* M Private Private 97.92% 97.50% Yolo_v3 Yolo Yes 512* G 61.8M Cityscape Cityscape 53.7% 53.1% DPU deployment - Yes: the model is successfully deployed on DPU. - Support: the model is supported but not deployed. Similar model structure is deployed and test successfully. - No: the model is not supported by DPU right now mainly due to some special operations or layers.

5 DNNDK 深度神经网络开发套件 DECENT DNNC Flt32 to Int8 quantization with one line command Automatic layer fusion to avoid frequently data read and write Runtime N 2 Cube Various APIs to facilitate specific application DECENT DNNC DNNDK N 2 Cube DSight Profiler Dsight Powerful tool as failure analysis and optimization Customer Platform (Board, OS)

6 DNNDK 开发流程 01 Model Compression 02 Model Compilation 03 Programming 04 Hybrid Compilation Five Steps with DNNDK 05 Execution

7 第 1 步 : 使用 DECENT 进行模型压缩 decent Deep compression Tool decent_q Quantization Tool decent_p Pruning Tool Consists of two separate tools Quantization Tool Pruning Tool Effects Compress model size 5x 100x Compress running time 1.5x 10x Platform Caffe, Darknet TensorFlow Quantization Tool Beta version Pruning Tool Internal version

8 第 2 步 : 使用 DNNC 进行模型编译 Neural Network Model Parser Optimizer Code-generator DNNC DPU Instruction Code Programmable tensor-level DPU instruction set - Compatible for Caffe/TensorFlow frameworks - Flexible & scalable for various CNN layers

9 硬件加速 - SSD Operations(G) / map Pruning Speedup on Hardware (2xDPU-4096@ZU9) VGG_SSD 4 classes surveillance data x baseline Pruning Iterations Operations(G) map(%) fps 5.7x FPS

10

11 Xilinx AI SDK Xilinx AI SDK to enable low touch engagements for customers Encourage top customers to use SDK to build applications and solutions Only use generic SDK release to support low priority customers Xilinx AI SDK Libraries and reference solutions Xilinx AI Suite Algorithm Model Zoo Cross-compiling tools BSP for boards DNNDK DPU and accelerating IPs

12 DPU 可扩展能力 Peak Perf INT8 (OPS) 6.8T ZU15 5.5T ZU11 4.1T ZU9 3.5T ZU7 2.9T 2.8T Z7100 ZU6 2.4T 1.7T 1.6T Z7035 Z7045 ZU4 ZU5 DPU Configuration 1.2T ZU3 700G 576G 230G Z7020 Z7030 ZU2 115G 102G 56G Z7014S/Z7015 Z7012S Z7010 * B256/288/512/3136 work in progress

13 DPU IP 集成 The DPU TRD (Targeted Reference Design) has been published in Xilinx.com.

14 重点总结 1 赛灵思提供先进的 Edge ML 解决方案 2 DNNDK 让 FPGA 上神经网络的部署更容易 3 DPU IP 已在赛灵思官方网站开放免费下载

15 Adaptable. Intelligent. 赛灵思技术日 XILINX TECHNOLOGY DAY