绝佳的并行处理 - FPGA 加速的根本基石

Transcription

1 赛灵思技术日 XILINX TECHNOLOGY DAY 绝佳的并行处理 - 加速的根本基石 朱勇赛灵思大中华区业务拓展总监 2019 年 3 月 19 日

2 加速 : 大幅提升应用的性能 Without acceleration CPU func1 func2 func3 func4 With acceleration CPU func1 func3 func4 func2 handles compute-intensive, deeply pipelined, massively parallel operations. CPU handles the rest

3 定制化的架构 的优势 CPUs and GPUs Fixed instruction set and rigid memory hierarchy Developer adapts the program to the architecture s Flexible, fully customizable architecture Developer adapts the architecture to the program

4 绝佳的并行处理器件 No predefined instruction set or underlying architecture Developer customizes the architecture to his needs Custom datapaths Custom bit-width Custom memory hierarchies Excels at all types of parallelism Deeply pipelined (e.g. Video codecs) Bit manipulations (e.g. AES, SHA) Wide datapath (e.g. DNN) Custom memory hierarchy ( e.g: Data analytics) Adapts to evolving algorithms and workload needs 2.5 million system logic cells 6,800 DSP engines 345 Mb on-chip memory

5 定制化的架构 加速的关键 Custom datapaths, parallel and pipelined User-defined bitwidths Custom dataflow pipelines Multiple stages executing simultaneously Streaming programming model Custom memory architectures Double-buffers, FIFOs, Shift-registers

6 高性能且适应性广泛 Speech Recognition Song Han et al. "ESE: Efficient Speech Recognition Engine with Sparse LSTM on. International Symposium on [Link] Database Analytics Jian Ouyang et al. "SDA: Software-Defined Accelerator for General- Purpose Big Data Analysis System." Hotchips [Link] Pattern Matching Shreyas G Singapura et al. " Based Accelerator for Pattern Matching in YARA Framework. CENG [Link] Genomic Analysis Edico Genome. "DRAGEN Genome Pipeline." Last accessed April 6, [Link] 43x 25x 18x 80x

7 加速应用的架构 x86 CPU Host Application Accelerated Functions User Application Code Acceleration API Runtime Library Drivers AXI Interfaces Global Memory DMA Engine Acceleration Platform PCIe

8 初始化应用 When the application starts, the device only contains the Shell The Shell will be managing the communications with the host buf2 Memory buf0 buf1 buf2 buf0 buf1 Memory Shell Custom Logic Region x86 CPU

9 初始化应用 When the application starts, the device only contains the Shell The Shell will be managing the communications with the host First, the host initializes the runtime buf2 Memory buf0 buf1 buf2 buf0 buf1 Memory Runtime Library Shell Custom Logic Region x86 CPU

10 初始化应用 When the application starts, the device only contains the Shell The Shell will be managing the communications with the host First, the host initializes the runtime Then programs the device with the desired binary buf2 Memory buf0 buf1 buf2 buf0 buf1 Memory.xclbin Runtime Library x86 CPU PCIe Shell Custom Acceleration Logic Kernels Region

11 分配缓冲区并将数据迁移到器件 Host allocates input and output buffers in the device Buffers are used to transfer data from the CPU to the and back buf2 buf0 buf1 buf2 buf0 buf1 Runtime Library x86 CPU Shell Acceleration Kernels

12 分配缓冲区并将数据迁移到器件 Host allocates input and output buffers in the device Buffers are used to transfer data from the CPU to the and back Host migrates data to be processed by the accelerator to the buffer buf2 buf0 buf1 buf2 buf0 buf1 Runtime Library Shell Acceleration Kernels x86 CPU

13 运行加速器 Host schedules execution of the desired kernel The Runtime is responsible for starting the kernel at the right moment Kernel reads data from the input buffer, processes it and writes results in the output buffer previously allocated buf2 buf0 buf1 buf2 buf0 buf1 Runtime kernel Library start Shell Acceleration Kernels x86 CPU

14 运行加速器 Host schedules execution of the desired kernel The Runtime is responsible for starting the kernel at the right moment Kernel reads data from the input buffer, processes it and writes results in the output buffer previously allocated After the kernel finishes processing the data, it notifies the host buf2 buf0 buf1 buf2 buf0 buf1 Runtime Library Shell kernel done x86 CPU

15 将数据迁移回主机 The host retrieves the results by scheduling a copy of the desired buffer back to host memory buf2 buf0 buf1 buf2 buf0 buf1 Runtime Library Shell Acceleration Kernels x86 CPU

16 加速 : 简单理解 Without acceleration CPU func1 func2 func3 func4 With acceleration CPU func1 func3 func4 func2

17 加速 : 更准确的观点 Without acceleration CPU func1 API calls to the interact with func2 func3 Data transfers between the func4 the accelerator host and the device With acceleration CPU func1 func3 func4 func3 func4 API API PCIe func2 WR RD func2

18 规则 1 牢记阿姆达尔定律 (Amdahl s Law ) It s better to accelerate func2 by 2x than to accelerate func3 by 50x! CPU func1 func2 func3 func4 Consider overall performance, not just individual functions When working top-down, identify performance bottlenecks in the application Use profiling tools, analyze the roof line of a Flame Graph Target accelerators that will impact end-to-end performance of the application

19 规则 2 针对大型 计算绑定的任务 CPU func1 func3 func4 API API PCIe Accelerators run very fast WR func2 RD But data transfers and API calls add latency Look for functions with where {compute time} is much greater than {data transfer time} Good: Monte-Carlo a few inputs, a lot of computations Bad: Vector-addition 2x more inputs than computations Prefer functions that perform a lot of processing per invocation to small functions which get called many times Minimizes API calls and event management overhead

20 规则 3 了解你的天花板 CPU func1 func3 func4 API API PCIe WR RD func2 Acceleration performance can t exceed PCIe throughput Compute-bound problems (on the CPU) are good for acceleration But maximum throughput will be limited by PCIe Maximum acceleration potential : {PCIe throughput} / {SW throughput}

21 规则 4 考虑吞吐量, 而不仅仅是时延 CPU PCIe func1 func3 func4 Aim to maximize WR RD kernel utilization API API Aim to reduce CPU idle time func2 Customize datapath for best performance Target applications with inherent SW and HW parallelism Task -level, Data-level, Instruction-level, Bit-level parallelism Adapt programming model to exploit parallelism Threads, asynchronous programming, dataflow model

22 加速 When to USE Algorithm allows for parallelization Many similar tasks When May Not be beneficial Small problem size Cost of Host to Device transfers outweighs benefit When NOT beneficial Little to no parallelism Algorithm is highly sequential over multiple data Tasks are highly dependent

23 开发者 vs 终端用户 Accelerators Binary Host Executable Data Center EXE Developers Require development tools and skills Can use 3 rd party libraries of accelerated functions Develop for own use or for external customers End Users No need for development tools and skills Run Apps on -equipped servers Cloud or on-premise

24 SDAccel 开发环境 Fully integrated Eclipse-based IDE Develop host applications in C/C++ Develop accelerators in RTL, C/C++ and OpenCL Develop Debug Debug, profiling and performance analysis tools Supports both GUI and command-line users Profile Optimize

25 SDAccel 支持并指导优化流程 I want to achieve SDAccel Application Timeline View Start with the end in mind conceptualize desired results Use visualization and guidance tools confirm and converge

26 Xilinx 使 可以便携应用 Develop once U200, U250, U280 U200, U250, U280 F1 2x.large, 16x.large FP1 Build for different target platforms Deploy on-premise or in the cloud F2, F3

27 访问 china.xilinx.com/alveo 立刻启动 Alveo 加速之旅吧! Learn and practice how to accelerate applications with s

28 Adaptable. Intelligent. 赛灵思技术日 XILINX TECHNOLOGY DAY