Offloading Collective Operations to Programmable Logic

Transcription

1 Offloading Collective Operations to Programmable Logic Martin Swany (With thanks to Omer Arap) Center for Research in Extreme Scale Computing (CREST) Department of Intelligent Systems Engineering Indiana University, Bloomington, IN

2 FPGAs Should Be Used As MPI Accelerators I know MPI accelerators. We have the best MPI acclerators. - DT Martin Swany Center for Research in Extreme Scale Computing (CREST) Department of Intelligent Systems Engineering Indiana University, Bloomington, IN

3 Overview We have developed an architecture and implementation for offloading collective operations to programmable logic in the communication substrate Not the first to use FPGAs for collectives It is clear that there is a tradeoff between performance and generality FPGAs are at one end of that spectrum Some folks live at the other end Programmable logic is burgeoning Xilinx Zynq Intel Xeon+Altera (HARP) Mellanox ConnectX-4 + FPGA FPGAs on other NICs and in switches

4 Programming FPGAs The programmable logic provided by FPGAs is a powerful option for creating task-specific functionality for applications Programming with FPGAs is not easy Many agree: Accelerate common functionality Others want to make it easy to use generalpurpose toolchains to deploy arbitrary kernels Jeff and OpenACC Franck s workshop New NSF call for ways to make it easy It may not ever be easy

5 Things that go bump in the wire We have been exploring a (relatively) generic approach for scenarios in which there is programmable logic in the communication pipeline Addresses the problem of how to get data to and from the device A bump in the wire Rich Graham clearly got this phrase from us In teaching how to use FPGAs, we have found this model to be quite useful Students who haven t done Verilog can get to this in a semester Important topic in Indiana U. s new engineering program This also maps to a growing use case

6 Collective Offload Offloading collective logic is a common technique in various platforms to improve performance Mellanox s CORE-Direct is one such state of the art collective operation offload framework. Defines primitive tasks: send, receive, wait, binary calculations. Collective algorithm defines list of tasks and tasks are performed by the NIC without CPU involvement Collective offload systems have been done many times, but ours is easy to use and is a starting point

7 Goals of FPGA-based Offload Reduce collective operation latency and variance Provide selection of collective algorithms that are implemented in hardware Implement collective algorithms independent of end-to-end communication protocols Utilize hardware-level, protocol-independent multicasting to optimize, and when possible, redesign major collective operation algorithms Support collective operation offload for different classes of collective operations

8 NetFPGA A line-rate, flexible, open networking platform for teaching and research (a) NetFPGA SUME Board Fig. 1. NetFPGA SUME Board and Block Diagram

9 Xilinx Zynq Processing System Flash Controller NOR, NAND, SRAM, Quad SPI Multiport DRAM Controller DDR3, DDR3L, DDR2 2x SPI AMBA Interconnect AMBA Interconnect Processor I/O Mux EMIO 2x I2C 2x CAN 2x UART GPIO 2x SDIO with DMA 2x USB with DMA 2x GigE with DMA XADC 2x ADC, Mux, Thermal Sensor ARM CoreSight Multi-Core Debug and Trace NEON DSP/FPU Engine Cortex - A9 MPCore 32/32 KB I/D Caches General Purpose AXI Ports 512 Kbyte L2 Cache General Interrupt Controller Configuration Timers AMBA Interconnect Watchdog Timer DMA Snoop Control Unit Security AES, SHA, RSA NEON DSP/FPU Engine Cortex- A9 MPCore 32/32 KB I/D Caches ACP Programmable Logic (System Gates, DSP, RAM) 256 Kbyte On-Chip Memory AMBA Interconnect High Performance AXI Ports PCIe Gen2 1-8 Lanes Multi-Standard I/Os (3.3V & High-Speed 1.8V) Multi-Gigabit Transceivers

10 Zedboard & Ethernet FMC

11 Collective Offload Design Ease implementation with the bump in the wire model Host offloads collective to the Programmable Logic (PL) by sending a specially crafted UDP packet Simplest solution Host blocks until the PL generates a release message, which is also a UDP packet

12 Collective Offload Packet dst_mac src_mac_1 src_mac_2 type ver IHL Diff_Serv Total_Length Identification flags fragment_offset TTL Protocol Hdr_Cksum src_ip dst_ip_1 dst_ip_2 UDP_Source_Port UDP_Dest_POrt Length UDP_checksum coll_id comm_size coll_type algo_type node_type msg_type rank local_rank_count operation data_type count

13 AXI-Based Platforms Advanced extensible Interface (AXI) standards define intermodule communication in programmable logic Widely utilized in recent Xilinx FPGA designs Recent designs depend on AXI-based protocols AXIS (AXI-Stream) is for streaming data between modules. We employ the AXIS protocol in our design Support for multiple ranks per host Provides insight how our design scales

14 Implementation Modules inherited from the NetFPGA package Input Arbiter, Output Queues Modules from Xilinx IP catalog AXI Ethernet, AXI DMA, AXI Width Converter, FPU, AXIS Data FIFO New Modules Stream Separator Stream Aggregator Collective Forwarding Engine (CFE) Collective Instruction Processing Engine (CIPE) Implements MPI_Allreduce, MPI_Reduce, MPI_Barrier, MPI_Scan, MPI_Alltoall

15 Data Path AXI DMA AXI Ethernet AXI Ethernet AXI Ethernet AXI Ethernet 32-b ctrl 32-b data 32-b ctrl 32-b data 32-b ctrl 32-b data 32-b ctrl 32-b data 32-b ctrl 32-b data Stream Aggregator Stream Aggregator Stream Aggregator Stream Aggregator Stream Aggregator 32-bit 32-bit 32-bit 32-bit 32-bit Width Converter Width Converter Width Converter Width Converter Width Converter 64-bit 64-bit 64-bit 64-bit 64-bit Input Arbiter 64-bit CFE (Collective Forwarding Engine) 64-bit CIPE (Collective Instruction Processing Engine) 64-bit Output Queues 64-bit 64-bit 64-bit 64-bit 64-bit Width Converter Width Converter Width Converter Width Converter Width Converter 32-bit 32-bit 32-bit 32-bit 32-bit Stream Separator Stream Separator Stream Separator Stream Separator Stream Separator 32-b ctrl 32-b data 32-b ctrl 32-b data 32-b ctrl 32-b data 32-b ctrl 32-b data 32-b ctrl 32-b data AXI DMA AXI Ethernet AXI Ethernet AXI Ethernet AXI Ethernet

16 Collective Forwarding Engine (CFE) Heart of the design Responsible for: Assessing the offload request header fields. Detecting collective operation and algorithm to run Perform state transitions and save the state for algorithms Based on the state, make forwarding decisions Determine the instructions which the CIPE will apply

17 Collective Instruction Processing Engine (CIPE) CFE attaches 64-bit instruction header CIPE processes the instruction Example Instructions Forward packet, do nothing Buffer packet in specified buffer Perform reduction operations on incoming packet and specified buffers Buffer or not buffer the result Perform reduction operations on specified buffers and ignore incoming packet Buffer or not buffer the result Ignore incoming packet and forward data on specified buffer

18 Collective Instruction Processing Engine (CIPE) Interfaces with FPU and FIFO cores FPU cores for streaming and applying floating point arithmetic FIFO cores for streaming intermediate results of single cycle arithmetic operations MIN, MAX, BOR, LOR, BAND, LAND, BXOR, LXOR, Integer SUM Internal BRAM buffers

19 Collective Instruction Processing Engine (CIPE) Op1_1 Op1_2 FPU 1 Op2_1 Op2_2 FPU 2 Op3_1 Op3_2 FPU 3 Op4_1 CFE Instruction Processor Op4_2 RSLT1 RSLT2 RSLT3 RSLT4 FPU 4 FIFO 1 FIFO 2 FIFO 3 FIFO 4 Output Queues Buffer 1 Buffer 2 Buffer 3 Buffer 4

20 Collective Instruction Processing Engine (CIPE) Fully pipelined design FPU cores can be configured to produce results in 1 cycle Due to observed instabilities, we increased the first result cycle to 11 (the maximum) Each extra FPU core introduces extra cycle Each FIFO core employed introduces an extra cycle

21 Evaluation Experimental Setup 8 Zynq Zedboards with EthernetFMC adapters Directly connected Come see it at SC

22 Zynq Cluster MPI_Allreduce ARM&Host&vs&PL&Offloaded&(Msg&Size&=&8&byte)& Speedup"Ave" Speedup"Min" ARM"Ave" ARM"Min" PL"Ave" PL"Min" 12000" 12" ARM&Host&vs&PL&Offloaded&(Msg&Size&=&128&byte)& Speedup"Ave" Speedup"Min" ARM"Ave" ARM"Min" PL"Ave" PL"Min" 12000" 12" 10000" 10" 10000" 10" Time&(usecs)& 8000" 6000" 4000" 8" 6" 4" Speedup& Time&(usecs)& 8000" 6000" 4000" 8" 6" 4" Speedup& 2000" 2" 2000" 2" 0" 8" 16" 32" 64" 128" Communicator&Size& 0" 0" 8" 16" 32" 64" 128" Communicator&Size& 0" ARM&Host&vs&PL&Offloaded&(Msg&Size&=&1024&Byte)& Speedup"Ave" Speedup"Min" ARM"Ave" ARM"Min" PL"Ave" PL"Min" 12000" 12" 10000" 10" Time&(usecs)& 8000" 6000" 4000" 8" 6" 4" Speedup& 2000" 2" 0" 8" 16" 32" 64" 128" Communicator&Size& 0"

23 Zynq Cluster MPI_Allreduce Ave,)Min)Difference:)ARM)vs)PL)Msg)(Size)=)8B)) Ave,)Min)Difference:)ARM)vs)PL)Msg)(Size)=)128B)) ARM"%"Diff" PL"%"Diff" ARM"Diff" PL"Diff" ARM"%"Diff" PL"%"Diff" ARM"Diff" PL"Diff" 1800" 30" 1800" 30" 1600" 1400" 25" 1600" 1400" 25" Time)(usecs)) 1200" 1000" 800" 600" 20" 15" 10" Percentage) Time)(usecs)) 1200" 1000" 800" 600" 20" 15" 10" Percentage) 400" 200" 5" 400" 200" 5" 0" 8" 16" 32" 64" 128" Communicator)Size) 0" 0" 8" 16" 32" 64" 128" Communicator)Size) 0" Time)(usecs)) 1800" 1600" 1400" 1200" 1000" 800" 600" 400" 200" 0" Ave,)Min)Difference:)ARM)vs)PL)Msg)(Size)=)1024B)) ARM"%"Diff" PL"%"Diff" ARM"Diff" PL"Diff" 8" 16" 32" 64" 128" Communicator)Size) 30" 25" 20" 15" 10" 5" 0" Percentage)

24 Zynq Cluster MILC Perentage( 20" 18" 16" 14" 12" 10" 8" 6" 4" 2" 0" MILC(4(Clover(Dynamical( %"speedup"(min"results)" %"speedup"(average"results)" 8" 16" 32" 64" Communicator(Size( 45.00# %#speedup#(min#results)# MILC(4(Schroed_cl_inv( %#speedup#(average#results)# Perentage( 40.00# 35.00# 30.00# 25.00# 20.00# 15.00# 10.00# 5.00# 0.00# 8# 16# 32# 64# Communicator(Size(

25 Conclusions and Future Work Empirical validation of the efficacy of collective offload Promising performance Increasing availability of programmable logic stands to increase impact The use of UDP allowed straightforward implementation, but is limiting Clearly, this the right way to use FPGAs 25

26