FPGA Acceleration and Virtualization Technology in DPDK ROSEN XU TIANFEI ZHANG

Size: px

Start display at page:

Download "FPGA Acceleration and Virtualization Technology in DPDK ROSEN XU TIANFEI ZHANG"

Jennifer Martin
5 years ago
Views:

1 x FPGA Acceleration and Virtualization Technology in DPDK ROSEN XU TIANFEI ZHANG

2 Agenda FPGA in Networking Acceleration Partial Reconfiguration (PR) FPGA Acceleration on DPDK DPDK High Level Design Scan and Probe Work Flow Intel FPGA Acceleration Environment Intel FPGA Acceleration Stack OPAE Intro Intel FPGA Acceleration on DPDK Port Representor and Virtualization Scenario Status & WIP Acknowledgement 2

3 FPGA in Networking Acceleration CPU VM VM Opportunities Enhancing Performance: Provide NIC ASIC liked performance Changing dynamically: Flexible enough for adding new feature by replace Bit Stream vswitch Slow Path Acc IP 1 FPGA Networking Acceleration Acc IP 2 Acc IP 3 Problems Longer design cycle than software: Compilation, Analysis & Synthesis, Fitter(Place & Router), Assembler, Timing Update Bit Stream affecting business: PCIe rescan and driver re-probe How to share One FPGA resource with many users 3

4 Partial Reconfiguration (PR) vswitch Slow Path Portion 1 (Acc IP 1) CPU VM Portion 2 (Acc IP 2) FPGA Networking Acceleration VM Portion 3 (Acc IP 3) With Partial Reconfigure(PR) a portion of the FPGA dynamically, FPGA not only provides one kinds of accelerator but also provides many types of accelerators at the same time All FPGA vendor support PR function How DPDK fully support FPGA? Which type of DPDK Device can provide FPGA PR? How can we bind DPDK Driver to FPGA Partial Bit Stream? 4

5 DPDK High Level Design Application(s) DPDK Framework(APIs) FPGA is divided to many portions, each portion has its own Partial Bit Stream PMD AFU Dev Eth PMD Eth PMD Eth AFU Eth Dev AFU Dev Crypto PMD Crypto PMD Crypto Crypto AFU Dev AFU Dev RawDev OPS FPGA RawDev Driver RawDev Driver Vdev Cfg takes configuration for each AFU IFPGA Bus is a new bus for AFU devices scan and drivers probe New added AFU Device for each portion s Acceleration function BUS IFPGA Bus PCI Bus All AFUs PMD are based on AFU Acceleration Hot-plugin Vdev Cfg vdev_fpga_cfg vdev_fpga_cfg driver vdev_fpga_cfg driver vdev_fpga_cfg driver driver FPGA AFU FPGA AFU FPGA AFU FPGA AFU FPGA Note: AFU(Accelerated Function Unit): Partial-Bitstream for a portion of the FPGA 5

6 Scan and Probe Work Flow constructor rte_bus_register INSERT to rte_bus_list rte_ifpga_driver_ register INSERT to afu_driver_list rte_bus_list ifpga pci ixgb 2 afu_ crypto afu_list afu_ eth afu_driver_list drv_ crypto pci_device_list drv_ eth i40e mcp_ fpga 1 rawdev.ops Ixgb _drv ENUMERATE & PR INSERT AFU Device raw_dev_arra y rte_eal_hotplug_ add PROBE AFU Driver pci_driver_list I40e _drv dev_ fpga mcp_ drv Rawdev probed as PCI Driver takes FPGA Configuration(Download/PR) 2 scans: FPGA PCI Device Scan(1 st Scan) and AFU Scan(2 nd Scan) OPAE Provides Common lib and API for low level FPGA management & accelerator access 6

Intel FPGA Acceleration Environment CPU User Application & Libraries FPGA Accelerator Function (Developer created or provided by Intel) Accelerator Function Interfaces Intel Acceleration Engine with

7 Intel FPGA Acceleration Environment CPU User Application & Libraries FPGA Accelerator Function (Developer created or provided by Intel) Accelerator Function Interfaces Intel Acceleration Engine with OPAE 1 Technology Hypervisor & OS CPU OPAE FPGA Interface Manager (FIM) FPGA HSSI 3 UPI 2 /PCIe* FPGA Management Engine(FME) Provides: power and thermal management, error reporting, partial reconfiguration, performance reporting, and other infrastructure functions. Each FPGA has one FME, accessible through the physical function. Accelerated Function Unit(AFU) Implements: one Acceleration, can be partial reconfiguration. Each FPGA can support Multiple AFUs. 7

Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform API Minimal software overhead and latency Supports virtual machines and

8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform API Minimal software overhead and latency Supports virtual machines and bare metal platforms Open source code licensing and developer community Intel FPGA drivers being upstreaming to Linux kernel Intel FPGA user space drivers have merged into DPDK 8

9 Intel FPGA Acceleration on DPDK DPDK Application Bare metal Application crypto event DPDK Device PMD FPGA Mgmt (partial reconfiguration) Network Path (NIC) Custom User space driver FPGA Accelerator librte_rawdev librte_ethdev OPAE High Level APIs fpga bus Eth driver (AFU) DPDK PCI Bus DPDK FPGA Device Driver OPAE Hardware Layer API Intel fpga base code User Space VFIO DPDK Device PMD For defined DPDK Device(Ethdev/Cryptodev/Eventdev) Non-DPDK User-Space Driver For Customized Device, Transparent DPDK Rawdev Support PR Rawdev is submited in Kernel Space Note: DPDK and NON-DPDK mode will not run at the same time 9

10 Port Representor and Virtualization Scenario FPGA Rawdev Driver Control Plan Application ETH DEV API ETH DEV API ETH DEV API ETH DEV API ETH DEV REPRESENTOR API REPRESENTOR REPRESENTOR PMD REPRESENTOR PMD PF PMD PMD PMD VM APP VM Socket App BSD Socket Syscall Host OS Kernel HyperVisor FME PF AFU VF AFU VF AFU VF AFU VF AFU FPGA Networking Acceleration Port Representor Each VF port binging to one representor port Control Plan Application take perception of VF port by its representor port Virtualization FPGA can support both DPDK APP and Socket APP in VM FPGA Rawdev Driver take FPGA configuration 10

11 Status & Working in Progress 2017 Q3 OPAE 0.9 Release [DONE] Fully support Intel FPGA Acceleration Environment Support FIM Q1 OPAE 0.13 Release, [DONE] Support FIM Q2 OPAE 1.0 Release [DONE] 2018 Q1 DPDK with OPAE User Space Driver PoC [DONE] 2018 Q1 IFPGA Bus RFC patch [DONE] 2018 Q2 IFPGA Bus patch set upstream to DPDK [DONE] DPDK supports FPGA Acceleration is ready. Welcome on board! 11

12 Acknowledgement Song Liu Zhang Zhang Hao Wu Yilun Xu Heqing Zhu Cunming Liang Helin Zhang Zhihong Wang Yanglong Wu Jingjing Wu Qi Zhang Yuwei Zhang Declan Doherty Bruce Richardson Ferruh Yigit Roy Fan Zhang 12

13 Thanks!

Zhang Tianfei. Rosen Xu

Zhang Tianfei. Rosen Xu Zhang Tianfei Rosen Xu Agenda Part 1: FPGA and OPAE - Intel FPGAs and the Modern Datacenter - Platform Options and the Acceleration Stack - FPGA Hardware overview - Open Programmable Acceleration Engine