NetChain: Scale-Free Sub-RTT Coordination

Transcription

1 NetChain: Scale-Free Sub-RTT Coordination Xin Jin Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, Ion Stoica

2 Conventional wisdom: avoid coordination NetChain: lightning fast coordination enabled by programmable switches Open the door to rethink distributed systems design 1

3 Coordination services: fundamental building block of the cloud Applications Coordination Service Chubby 2

4 Provide critical coordination functionalities Applications Configuration Management Group Membership Distributed Locking Barrier Coordination Service 3

5 The core is a strongly-consistent, fault-tolerant key-value store Applications Configuration Management Group Membership Distributed Locking Barrier Coordination Service Strongly-Consistent, Fault-Tolerant Key-Value Store This Talk Servers 4

6 Workflow of coordination services request Can we do better? client reply coordination servers running a consensus protocol Ø Throughput: at most server NIC throughput Ø Latency: at least one RTT, typically a few RTTs 5

7 Opportunity: in-network coordination client request reply Distributed coordination is communication-heavy, not computation-heavy. coordination servers running a consensus protocol Server Switch Example [NetBricks, OSDI 16] Barefoot Tofino Packets per second 30 million A few billion Bandwidth Gbps 6.5 Tbps Processing delay us < 1 us 6

8 Opportunity: in-network coordination request client reply coordination switches running a consensus protocol Ø Throughput: switch throughput Ø Latency: half of an RTT 7

9 Design goals for coordination services Ø High throughput Ø Low latency Directly from high-performance switches Ø Strong consistency Ø Fault tolerance How? 8

10 Design goals for coordination services Ø High throughput Ø Low latency Directly from high-performance switches Ø Strong consistency Ø Fault tolerance Chain replication in the network 9

11 What is chain replication Read Request Read Reply S 0 S 1 S 2 Head Replica Tail Ø Storage nodes are organized in a chain structure Ø Handle operations Ø Read from the tail 10

12 What is chain replication Write Request Read Request Read/Write Reply S 0 S 1 S 2 Head Replica Tail Ø Storage nodes are organized in a chain structure Ø Handle operations Ø Read from the tail Ø Write from head to tail Ø Provide strong consistency and fault tolerance Ø Tolerate f failures with f+1 nodes 11

13 Division of labor in chain replication: a perfect match to network architecture Chain Replication Storage Nodes Optimize for high-performance to handle read & write requests Provide strong consistency Auxiliary Master Handle less frequent reconfiguration Provide fault tolerance Network Architecture Network Data Plane Handle packets at line rate Network Control Plane Handle network reconfiguration 12

14 NetChain overview NetChain Handle read & write requests at line rate Handle reconfigurations (e.g., switch failures) S 0 S 1 Network Controller S 2 S 3 S 4 S 5 Host Racks 13

15 How to build a strongly-consistent, fault-tolerant, in-network key-value store Ø How to store and serve key-value items? Ø How to route queries according to chain structure? Data Plane Ø How to handle out-of-order delivery in network? Ø How to handle switch failures? Control Plane 14

16 PISA: Protocol Independent Switch Architecture Ø Programmable Parser Ø Convert packet data into metadata Ø Programmable Mach-Action Pipeline Ø Operate on metadata and update memory state Match + Action Memory ALU Programmable Parser Programmable Match-Action Pipeline 15

17 PISA: Protocol Independent Switch Architecture Ø Programmable Parser Ø Parse custom key-value fields in the packet Ø Programmable Mach-Action Pipeline Ø Read and update key-value data at line rate Match + Action Memory ALU Programmable Parser Programmable Match-Action Pipeline 16

18 Control plane (CPU) Network Management NetChain Switch Agent NetChain Controller PCIe Run-time API Data plane (ASIC) Network Functions Key-Value Store Match + Action Memory ALU Programmable Parser Programmable Match-Action Pipeline 17

19 How to build a strongly-consistent, fault-tolerant, in-network key-value store Ø How to store and serve key-value items? Ø How to route queries according to chain structure? Data Plane Ø How to handle out-of-order delivery in network? Ø How to handle switch failures? Control Plane 18

20 NetChain packet format Existing Protocols NetChain Protocol ETH IP UDP SC S 0 S 1 S k OP SEQ KEY VALUE L2/L3 routing reserved port # NetChain routing read, write, delete, etc. inserted by head switch Ø Application-layer protocol: compatible with existing L2-L4 layers Ø Invoke NetChain with a reserved UDP port 19

21 In-network key-value storage Match Key = X Key = Y Key = Z Default Match-Action Table Action Read/Write RA[0] Read/Write RA[5] Read/Write RA[2] Drop() Register Array (RA) Ø Key-value store in a single switch Ø Store and serve key-value items using register arrays [SOSP 17, NetCache] Ø Key-value store in the network Ø Data partitioning with consistent hashing and virtual nodes 20

22 How to build a strongly-consistent, fault-tolerant, in-network key-value store Ø How to store and serve key-value items? Ø How to route queries according to chain structure? Data Plane Ø How to handle out-of-order delivery in network? Ø How to handle switch failures? Control Plane 21

23 NetChain routing: segment routing according to chain structure Write Request dstip = S 0 SC = 2 S 1 S 2 H 0 Client Write Reply dstip = H 0 SC = 0 S 0 S 1 S 2 Head Replica Tail dstip = S 1 SC = 1 S 2 dstip = S 2 SC = 0 22

24 NetChain routing: segment routing according to chain structure H 0 Read Request Client dstip = S 2 SC = 2 S 1 S 0 Read Reply dstip = H 0 SC = 2 S 1 S 0 S 0 S 1 S 2 Head Replica Tail 23

25 How to build a strongly-consistent, fault-tolerant, in-network key-value store Ø How to store and serve key-value items? Ø How to route queries according to chain structure? Data Plane Ø How to handle out-of-order delivery in network? Ø How to handle switch failures? Control Plane 24

26 Problem of out-of-order delivery Concurrent Writes W 1 : foo=b W 2 : foo=c time Head Replica Tail S 0 S 1 S 2 foo=a foo=a foo=a foo=b foo=c foo=c foo=b foo=b foo=c Inconsistent Serialization values withbetween sequence three number replicas 25

27 How to build a strongly-consistent, fault-tolerant, in-network key-value store Ø How to store and serve key-value items? Ø How to route queries according to chain structure? Data Plane Ø How to handle out-of-order delivery in network? Ø How to handle switch failures? Control Plane 26

28 Before failure: tolerate f failures with f+1 nodes Handle a switch failure S 0 S 1 S 2 Fast Failover Failure Recovery S 0 S 2 S 0 S 3 S 2 Ø Failover to remaining f nodes Ø Tolerate f-1 failures Ø Efficiency: only need to update neighbor switches of failed switch Ø Add another switch Ø Tolerate f+1 failures again Ø Consistency: two-phase atomic switching Ø Minimize disruption: virtual groups 27

29 Protocol correctness Invariant. For any key k that is assigned to a chain of nodes [S 1, S 2,, S n ], if 1 i < j n (i.e., S i is a predecessor of S j ), then State +, k. seq State + 2 k. seq. Ø Guarantee strong consistency under packet loss, packet reordering, and switch failures Ø See paper for TLA+ specification 28

30 Implementation Ø Testbed Ø 4 Barefoot Tofino switches and 4 commodity servers Ø Switch Ø P4 program on 6.5 Tbps Barefoot Tofino Ø Routing: basic L2/L3 routing Ø Key-value store: up to 100K items, up to 128-byte values Ø Server Ø 16-core Intel Xeon E5-2630, 128 GB memory, 25/40 Gbps Intel NICs Ø Intel DPDK to generate query traffic: up to 20.5 MQPS per server 29

31 Evaluation Ø Can NetChain provide significant performance improvements? Ø Can NetChain scale out to a large number of switches? Ø Can NetChain efficiently handle failures? Ø Can NetChain benefit applications? 30

32 Evaluation Ø Can NetChain provide significant performance improvements? Ø Can NetChain scale out to a large number of switches? Ø Can NetChain efficiently handle failures? Ø Can NetChain benefit applications? 31

33 Orders of magnitude higher throughput 1etCKaiQ(Pax) 1etCKaiQ(4) ZooKeeSer 1etCKaiQ(Pax) 1etCKaiQ(4) ZooKeeSer TKrougKSut (0436) MQPS 82 MQPS 0.15 MQPS alue 6ize (Byte) TKrougKSut (0436) MQPS 82 MQPS 0.15 MQPS 0 20K 40K 60K 80K 100K 6tore 6ize 32

34 Orders of magnitude lower latency /ateqcy (µs) us 170 us 9.7 us ZooKeeSer (ZrLte) ZooKeeSer (read) 1etCKaLQ (read/zrlte) TKrougKSut (043S) 33

35 Handle failures efficiently ThroughSut (0QPS) failover failure recovery TiPe (s) (a) 1 Virtual Group. ThroughSut (0QPS) failover reduce throughput drop with virtual groups failure recovery TiPe (s) (b) 100 Virtual Groups. 34

36 Conclusion Ø NetChain is an in-network coordination system that provides billions of operations per second with sub-rtt latencies Ø Rethink distributed systems design Ø Conventional wisdom: avoid coordination Ø NetChain: lightning fast coordination with programmable switches Ø Moore s law is ending Ø Specialized processors for domain-specific workloads: GPU servers, FPGA servers, TPU servers Ø PISA servers: new generation of ultra-high performance systems for IO-heavy workloads enabled by PISA switches 35

37 Thanks! 36