Using FPGAs as Microservices

Transcription

1 Using FPGAs as Microservices David Ojika, Ann Gordon-Ross, Herman Lam, Bhavesh Patel, Gaurav Kaul, Jayson Strayer (University of Florida, DELL EMC, Intel Corporation) The 9 th Workshop on Big Data Benchmarks, Performance Optimization and Emerging Hardware ASPLOS 2018 March 25, 2018

2 Contents Motivation Challenges of FPGA integration Proposed solutions Case study with Spark Key design elements Conclusions & future work 2

3 Motivation FPGAs are increasingly being used in low-latency / high throughput computing (e.g., big data, machine learning) Cloud service providers (CSPs) continue to introduce FPGAs in their datacenters (e.g., Amazon, Microsoft) to meet computing demands Cloud Emerging cloud computing trends (microservices, serverless) show promise in hyperscale datacenters, increased developer productivity Developer 3

4 FPGA + Microservices? 4

5 Overview of FPGAs FPGA: Field-programable gate array A reconfigurable chip composed of CLBs Basic structure consists of: Look-up table (LUT), Flip-Flop (FF), Routing wires, I/O pads CLB: configurable logic block Contemporary FPGAs architectures incorporate more advanced resources: Multiply-accumulate (MAC) Off-chip memory controllers High-speed serial transceivers Embedded, distributed memories Phase-locked loops (PLLs) Up to 2 million logic cells 5

6 Overview of FPGAs FPGA: Field-programable gate array A reconfigurable chip composed of CLBs Basic structure consists of: Look-up table (LUT), Flip-Flop (FF), Routing wires, I/O pads Contemporary FPGAs architectures incorporate more advanced resources: Multiply-accumulate (MAC) Off-chip memory controllers High-speed serial transceivers Embedded, distributed memories Phase-locked loops (PLLs) Up to 2 million logic cells Key benefits: Flexibility, Power, Performance CLB: configurable logic block 6

7 What are Microservices Microservices: a collection of loosely-coupled software services - Unlike Monolithic, provide lightweight (shared) services - Scale consistently with changing workloads Monolith / Layered Microservice / Disaggregated Our approach: use Microservices (as collection of loosely-coupled FPGA accelerator functions) to support FPGAs as software services 7

8 What are Microservices Microservices: a collection of loosely-coupled software services - Unlike Monolithic, provide lightweight (shared) services - Scale consistently with changing workloads Key benefits: scalability, fault-tolerance, dynamic configuration, auto-deployment Monolith / Layered Microservice / Disaggregated Our approach: use Microservices (as collection of loosely-coupled FPGA accelerator functions) to support FPGAs as software services 8

9 Challenges of FPGA integration 9

10 1. Intricate software interface JVM-to-FPGAinterfacing is cumbersome How to keep FPGA accelerator fully-utilized Expose all FPGA functions? 10

11 2. Non-trivial FPGA sharing and data movement FPGA and CPU thread co-existence is complicated Data transfer overheads can hurt performance FPGA mgt. & data movement? 11

12 3. FPGA reconfiguration Reconfiguration can take milliseconds to a few seconds Certain applications may be intolerable to downtime FPGA App CPU Hiding reconfiguration time CPU fallback? 12

13 4. Challenging programming model Requirement on hardware-specific knowledge Long synthesis (compile) time Programmer / Developer HLS HDL binary hours of development / design iterations Programmer vs Developer? FPGA 13

14 Proposed Solution FaaM: a runtime framework for deploying FPGA accelerators as microservices Leverage optimized hardware libraries Share accelerator across threads, applications, users Platform-agnostic (OpenCL, etc.) Seamless integration with Java, support wide range of apps/frameworks Takes care of accelerator management (init./setup, buffer mgt., etc.) 14

15 FPGA Microservices Overview Accelerator Manager /w HW-accelerated function FPGA Library FaaM 15

16 Experimenting with Spark 1. Map convert raw input into key/value pairs. Output to memory (or disk) 2. Shuffle/Sort All reduces retrieve all records from all mappers over network 3. Reduce For each distinct key, do something with all the corresponding values. Output to disk Data Shuffle/Sort in Spark data movement can hurt performance 16

17 Experiment Setup Platform: Xeon+FPGA with Arria10 FPGA Hardware-accelerated function: DEFLATE compression algorithm (level 9) Workload: Sort Evaluation technique: Focus on compression of Spark Output RDDs Spark Worker (Xeon+FPGA Server) CPU FPGA With FaaM (FPGA Microservice): No change to application code Only update Spark config. files 17

18 Performance Results FPGA as drop-in replacement via FaaM 4X memory saving 3.2X speedup Functional Evaluation Performance Evaluation 18

19 System-level optimization #1: enable RDD caching + 2X improvement *CPU used only as baseline. RDD: Resilient Distributed Disk 19

20 System-level optimization #2: increase CPU count Increased FPGA accelerator utilization, resulting in improved application performance 20

21 Key design elements 21

22 1. FPGA accelerator abstraction Driver Arria 10 FPGA AFU: Accelerator Functional Unit 22

23 1. FPGA accelerator abstraction New Library Driver Arria 10 FPGA AFU: Accelerator Functional Unit 23

24 2. Task scheduler Disk Task scheduler Library FPGA-to-Host Memory Communication JVM and native memory data transfers can incur significant overhead Leverage non-blocking I/O; access data in streaming fashion No garbage collection Buffer re-use among groups of SW threads Hde data transfer latency by overlapping comm. with comp. Pinned buffer (NIO) 24

25 3. JVM runtime system Runtime system Task scheduler CPU-FPGA Thread Co-existence Keep FPGA busy all time with tasks Stateless Accelerator management, (buffer size, int./clean-up, etc.) Library 25

26 Buffer R/W performance Sweet spot resulting in almost no performance loss (next slide) for the runtime system. 26

27 Compression: Raw Performance equal almost equal Functional Evaluation Performance Evaluation 27

28 Conclusions & Future Work FPGAs compliment CPUs in certain compute-intensive task 3.2X speedup, 4X memory footprint reduction in Spark (single-node deployment!) Potentially increased benefit in larger-scale scenarios FPGA accelerators can leverage microservices architecture Key to efficiency is CPU-memory-FPGA interaction; accelerator management Optimizations to both low-level and high-level stack help improve performance Future work: Performance evaluation with cloud and machine learning workloads 28