Providing Multi-tenant Services with FPGAs: Case Study on a Key-Value Store

Transcription

1 Zsolt István *, Gustavo Alonso, Ankit Singla Systems Group, Computer Science Dept., ETH Zürich * Now at IMDEA Software Institute, Madrid Providing Multi-tenant Services with FPGAs: Case Study on a Key-Value Store

2 FPGAs in the Cloud Wider adoption of FPGAs (e.g., Amazon F1, Microsoft Catapult, ) Many promising use-cases but often singe-tenant designs Clouds built on sharing and multi-tenancy High utilization Flexible provisioning Load isolation and QoS guarantees 2

3 Providing multi-tenancy with FPGAs FPGA FPGA Virtualization General purpose (PR) Few tenants Trades off functionality Course grained resource alloc. Tenants bring applications Multi-tenant applications Domain-specific Many tenants Trades off performance (?) Fine grained resource alloc. Provider brings application 3

4 Multi-tenant application as a service Key-value store Widely deployed in the cloud and datacenters Different tradeoffs but similar interfaces, e.g.: Memcached caching, no replication, latencyoptimized, main-memory Amazon S3 BLOB store, replicated, BW-optimized, needs large capacity 4

5 Building a multi-tenant KVS (Multes) Area well studied in related work Several pipelined designs, all saturate network link Caribou: Interfaces and functionality similar to SW [VLDB17] FPGA can provide replication for fault-tolerance [NSDI16] Requirements for multi-tenancy: Performance isolation Data isolation Flexibility in resource allocation (focus on network bandwidth) Efficient use of resources regardless of number of tenants [VLDB17] Z. István, D. Sidler, G. Alonso Caribou: Intelligent Distributed Storage. [NSDI16] Z. István, D. Sidler, G. Alonso, M. Vukolic: Consensus in a Box: Inexpensive Coordination in Hardware. 5

6 Designing for multi-tenancy Caribou is composed of four modules Requests can take various routes Some traffic is inter-node Hard to reason about load interactions! Multes: Reorganized pipeline to ensure all requests take same path (1) Hash table implements parts of the replication log features (multi-version) More coupling between modules (opcodes) Replication messages Client messages Multes Caribou (single pipeline) Network Stack (TCP) Value Value Access Access + + Processing Processing Traffic Shaper Traffic Shaper Memory Replication + Replication Log Manager Multivers. Hash Table + Hash Table Allocator + Allocator 6

7 Token buckets Traffic Shaper Per-tenant limits (D,C,T) Configuration Round -robin Commonly used in networking scenarios Max. number of tokens (D), adding C tokens every T cycles Limits data rate, burst size Buffer space on the FPGA? Queue commands before data movement Input packets/commands Extract tenant ID Token Bucket Token Bucket Token Bucket Output packets/commands Token buckets can be configured with no overhead at runtime (2) Per-tenant allocations controlled by Metadata software 7 Body Request/command Encodes the real cost of the request

8 Replicated KVS Caribou implements inter-fpga replication (leader based algorithm) Tenant 1 replicated group Tenant 2 Leader FPGA node Replica FPGA node Leader Replica FPGA node Replica FPGA node Replica Tenant 3 replicated group 8

9 Having multiple roles Control state machine at heart of replication protocol Data and control handled separately Multiple copies not an option Complex logic + plumbing SM extended to store state for each tenant can context switch per each packet (3) Not all states need tenant context Latency inside SM not on critical path Now in registers, but could use BRAMs to store state Input message Tenant 1 State List of peers, Role in protocol, Outstanding proposals, etc. Tenant 2 State Tenant 3 State Replication controller (atomic broadcast protocol) Encodes key, data op., socket numbers, etc. 9 Out. command

10 Replication protocol Evaluation of Multes Client Client Client Tenant 1 Client Client Network Multes Memory/ Storage Client Tenant 2 Multiple Xilinx VC709s connected to a 10Gbps switch 9 load generating machines, Go-based benchmarking tool Tenants connect to different TCP port numbers (e.g. 2880, 2881, ) Multes offers flexible multi-tenancy while efficiently using the network link 10

11 No performance loss due to multi-tenancy Read-only throughput on a single node 11

12 Load isolation Replicated write latency of Tenant 0 (group = 3) Additional tenants using their full read bandwidth (1/8 of 10Gbps) Replicated write latency [us] (without client overhead) 12

13 % of VC709 resources Resource Usage: Small cost for sharing Logic 2x Caribou BRAM Logic BRAM Multes T=2 Caribou No. of max. supported tenants in Multes The FPGA part on the VC709 is XC7VX690T-2FFG1761C 13

14 Thoughts on the future Platform-as-a-service Multes Customize KVS with tenant-defined processing for different flavors Network Stack (TCP) Traffic Shaper Replication Combining multi-tenant application with small PR regions Simple streaming interfaces can use HLS, OpenCL, etc. Misbehaving PR region does not impact others Value Access + Processing Traffic Shaper Memory Multivers. Hash Table + Allocator 14

15 Conclusion Multes: multi-tenant KVS service that doesn t sacrifice performance Project on Github: Relied on three techniques: 1) Single-pipeline architecture and traffic shapers no load interaction 2) Runtime-parameterization of control modules flexible allocations 3) Contexts in controlling state machines no overhead when switching between tenants Applicable to many network-facing applications on FPGAs 15