Dataplane Programming

Transcription

1 Dataplane Programming 1

2 2 Outline Example use case Introduction to data plane programming P4 language

3 3 Example Use Case: Paxos in the Network

4 4 The Promise of Software Defined Networking Increased network programmability allows ordinary programs to manage the network Applications can leverage SDNs to improve performance through data plane configuration (e.g., route selection and QoS) Can application logic be moved into the network? This work focuses on the widely-deployed Paxos protocol

5 5 Why Paxos? Paxos is a fundamental building block for distributed applications e.g., Chubby, OpenReplica, and Ceph There exists extensive work on optimizing Paxos (e.g., Fast Paxos) Paxos operations can be efficiently implemented in hardware

6 6 Outline of This Talk Motivation Paxos Background Consensus in the Network Paxos in SDN Switches (and required OpenFlow extensions) Alternative consensus protocol (without OpenFlow changes) Evaluation Conclusions

7 Paxos Background 7

8 8 Paxos Protocol Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

9 8 Paxos Protocol Issues requests Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

10 8 Paxos Protocol Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

11 8 Paxos Protocol Advocates requests Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

12 8 Paxos Protocol Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

13 8 Paxos Protocol Chooses a value / provides memory Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

14 8 Paxos Protocol Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

15 8 Paxos Protocol Provides replication Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

16 8 Paxos Protocol Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Goal: Have a group of participants agree on one result (i.e., consensus) Protocol proceeds in rounds, each round has two phases Performance is measured in message hops Classic Paxos requires 3 hops

17 9 Fast Protocol Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner Key idea: optimize for the case when proposals don t collide Optimistically uses fast rounds that bypass coordinator Only 2 message hops Requires 1 more acceptor If there is collision, revert to Classic Paxos

18 10 Observations Performance metric (hops) does not account for network topology 1 classic message hop has to travel through multiple switches Coordinators and acceptors are typically bottlenecks Must aggregate or multiplex messages Fault tolerance does not include the network devices

19 11 Moving Paxos into the Network

20 12 Paxos on Network Devices Proposer Proposer Proposer Coordinator Key idea: Move coordinator and acceptor (phase 2) logic into switches OpenFlow API not sufficient Acceptor Acceptor Acceptor Learner Learner

21 13 Required Extensions Proposer Proposer Proposer Coordinator Acceptor Acceptor Acceptor Learner Learner

22 13 Required Extensions Proposer Proposer Proposer Coordinator 1. Generate round and sequence numbers Acceptor Acceptor Acceptor Learner Learner

23 13 Required Extensions Proposer Proposer Proposer Coordinator 1. Generate round and sequence numbers Acceptor Acceptor Learner Learner Acceptor 2. Persistent storage 3. Stateful comparisons 4. Storage cleanup

24 14 Hardware Feasibility Requirement Device Capability Round & Sequence Generator Stateful Comparisons Netronome NFP-6xxx, NetFPGA, Arista 7124FX Stateful flow processing Persistent Storage Storage Cleanup Arista 7124FX 50 GB SSD logging

25 15 Consensus without OpenFlow Extensions

26 16 Can We Avoid Extensions? Proposer Proposer Proposer Coordinator Make optimistic assumptions about message order Like Fast Paxos, have fast and classic rounds Acceptor Acceptor Acceptor Learner Learner

27 17 No Sequence Numbers Needed Proposer Proposer Proposer Rely on optimistic message ordering Use switch to increase probability Serializer Acceptor Acceptor Acceptor Learner Learner

28 18 No On-Device State Needed Proposer Proposer Proposer Serializer Acceptor Acceptor Acceptor Learner Learner Use additional servers as storage

29 19 No On-Device Logic Needed Proposer Proposer Proposer No coordinator Acceptors always accept Need an additional minion Serializer Minion Minion Minion Minion Learner Learner

30 20 Performance Assumption Proposer Proposer Proposer Messages from serializer to minion are in the same order If not, execute slow round Serializer Minion Minion Minion Minion Learner Learner

31 21 Correctness Assumption Proposer Proposer Proposer Messages from minion to servers are in same order If not, protocol breaks Serializer Minion Minion Minion Minion Learner Learner

32 22 NetPaxos Summary Latency: Fewer true network hops, including switches Throughput: Avoids potential bottlenecks, reduced logic Fault tolerance: Serializer can easily be made redundant, other devices can fail, as with Classic Paxos

33 Evaluation 23

34 24 Experiments Focus on two questions: Do our ordering assumptions hold? What is the potential benefit of NetPaxos? Testbed: Three Pica8 Pronto 3290 switches, 1Gbps links Send messages of 1 MTU size, with sequence numbers

35 25 Assumptions Mostly Hold Percentage of Packets Resulting in Disagreement or Indecision Indecisive Disagree 10,000 30,000 50,000 Performance assumption held up to 70% link capacity Correctness assumption was never violated Traffic should not be bursty Messages / Second

36 26 High Potential for Performance Latency (ms) Basic Paxos NetPaxos Disclaimer: Best case scenario 9x increase in throughput 90% reduction in latency 10,000 30,000 50,000 Messages / Second

37 27 Outlook Formalizing protocol with Spin model checker Implementing a NetFPGA-based prototype Investigating root causes for packet reordering

38 28 Conclusions SDNs enable tight integration with the network, which can improve distributed application performance Proposed two approaches to moving Paxos logic into the network Paxos is a fundamental protocol. Performance improvements would have a great impact on data center applications

39 29

40 30 Introduction to Data Plane Programming

41 31 SDN is Not Enough SDN allows you to program the control plane Many large data centers program host network stacks (hypervisors), roughly edge-based SDN Not yet able to program the data plane

42 32 Data Plane Opportunities Simplify and improve network management Extensions for debugging and diagnostics Dynamic resource allocation Enable critical new features Improved robustness Port-knocking Load balancing, enhanced congestion control

43 33 Suppressing Innovation OpenFlow provides an (intentionally) limited interface No state No computation Restricted to a fixed set of headers May need to customize hardware support Match tables usually have fixed width, depth, and execution order

44 34 Demand for New Features SDNs and white boxes set the stage Large private networks want new features Rate of new feature arrivals exceeds rate of hardware evolution

45 35 Getting New Features No DIY solution. Must work with vendors at the feature level Hard to get consensus on the feature Long time to realize the feature Need to buy the new hardware What you get is not usually what you want

46 36 Extensions to OpenFlow OpenState project, G. Bianchi et al. Mealy machine abstraction ONF Working Groups EXT-WG focused on extensions FAWG focused on forwarding abstractions

47 37 Vision What about the next version of OpenFlow? Or custom protocol? We all know how to program CPUs Supporting tools and infrastructure Allows fast iteration and differentiation Let s you quickly realize your own ideas Challenge: How can we replicate this in the network?

48 38 Networking is Late to the Game Domain Target Hardware Language Computers CPU C, Java, OCaml, JavaScript, etc. Graphics GPU CUDA, OpenCL Cellular Base Station DSP C, MATLAB, Data Flow languages Networks??

49 39 Two Questions What do we need at the hardware level? Once we have that, how do we program it?

50 40 Hardware Trend PISA (Protocol Independent Switch Architecture) Fundamental departure from switch ASICS Programmable parsing Protocol independence (i.e., generic match-action units) Parallelism across multiple match-action units, and within each stage Power, size, or cost penalty is negligible

51 41 Why Now? I/O, memory, and bus dominate chip size Logic is getting proportionally smaller Programmability means larger logic. The rest stays the same. Very little power, area, or performance penalty for programmability.

52 42 PISA Chip PISA)Chip) Logical4Dataplane4View4 Switch4Pipeline4 L24 IPv44 IPv64 ACL4 Parser4 Match4Table4 Ac:on4Macro4 Match4Table4 Ac:on4Macro4 Match4Table4 Ac:on4Macro4 Fixed4Ac:on4 Match4Table4 Ac:on4Macro4 Queues4 84

53 43 PISA Chip Mapping)Logical)Dataplane)Design)to) PISA)Chip) L24 Logical4Dataplane4View4 Switch4Pipeline4 IPv44 IPv64 ACL4 Parser4 Match4Table4 L24Table4 L24Ac:on4Macro4 Match4Table4 IPv44Table4 v44ac:on4macro4 Match4Table4 IPv64Table4 Ac:on4Macro4 v64ac:on44 Match4Table4 ACL4Table4 ACL4Ac:on4Macro4 Ac:on4Macro4 Queues4 94

54 44 PISA Chip ReMconfigurability) L24 MyEncap4 IPv44 IPv64 ACL4 Logical4Dataplane4View4 Switch4Pipeline4 Parser4 L24Table4 L24Ac:on4Macro4 IPv44 IPv44Table4 MyEncap4 v44ac:on4macro4 IPv64 IPv64Table4 IPv44 v64ac:on4macro4 ACL4Table4 ACL4Ac:on4Macro4 Queues4 104

55 45 PISA Chip Match4 Memory4 Ac:on4 ALU4 Parser4 L24Table4 L24Ac:on4Macro4 IPv44Table4 v44ac:on4macro4 IPv64Table4 v64ac:on4macro4 ACL4Table4 ACL4Ac:on4Macro4 PISA)(Protocol4Independent4Switch4Architecture)4 Q Parallelism4across4pipelined4stages4 Q Parallelism4within4each4stage4 114

56 46 Key Players Hardware manufacturers Proto-PISA chips already available: FlexPipe (Intel) and others (Cisco and Cavium) Full-fledged PISA chips on the horizon (Barefoot) High-level language P4 (p4.org) Compiler and development tools Hour-glass design with IR, profilers, debuggers

57 47 Abstract Forwarding Model Eth4 Parser4 VLAN4 Hdr4 Fields4 Match Ac:on4 Match Ac:on4 Ingress4Stages4 Match Ac:on4 Match Ac:on4 Queues4 Egress4Stages4 Match Ac:on4 Match Ac:on4 IPv44 IPv64 TCP4 Match Ac:on4 Match Ac:on4 Match Ac:on4

58 48 P4 Language Components Parser) Program) StateQmachine;44 Field4extrac:on4 Match)Tables) +)Ac:ons) Control)Flow) Table4lookup4and4update;4 Field4manipula:on;44 Control4flow4 Assembly) ( deparser )) Program) Field4assembly4 No:4memory4(pointers),4loops,4recursion,4floa:ng4point4

59 49 P4 Examples Header Fields Parsing Tables Actions Control Flow

60 50 Header Field and Parsing header_type ethernet_t { fields { dstaddr : 48; srcaddr : 48; ethertype : 16; } } Eth4 IPv44 VLAN4 IPv64 parser parse_ethernet { extract(ethernet); return select(latest.ethertype) { 0x8100 : parse_vlan; 0x800 : parse_ipv4; 0x86DD : parse_ipv6; } } TCP4 New4

61 51 Table (Match) table ipv4_lpm { reads { ipv4.dstaddr : lpm; } actions { set_next_hop; drop; } } Lookup4key4 ipv4.dstaddr) ac:on) 0.*4 drop *4 set_next_hop 224.*4 drop *4 drop *4 set_next_hop

62 52 Actions ipv4.dstaddr) ac:on) 0.*4 drop *4 set_next_hop 224.*4 drop *4 drop *4 set_next_hop nhop_ipv4_addr) port) action set_next_hop(nhop_ipv4_addr, port) { modify_field(metadata.nhop_ipv4_addr, nhop_ipv4_addr); modify_field(standard_metadata.egress_port, port); add_to_field(ipv4.ttl, -1); }

63 53 Control Flow Control)Flow) Match Ac:on4 Match Ac:on4 Match Ac:on4 Match Ac:on4 Match Ac:on4 Match Ac:on4 control ingress { apply(port); if (valid(vlan_tag[0])) { apply(port_vlan); } apply (bridge_domain); if (valid(mpls_bos)) { apply(mpls_label); } retrieve_tunnel_vni(); if (valid(vxlan) or valid(genv) or valid(nvgre)) { apply(dest_vtep); apply(src_vtep); } }

64 54 Compilation Parser4Program4 Match4Tables4+4 Ac:ons4 Control4Flow4 Compiler4 PISA4 Chip4

65 55 Protocol API MPLS4(source)4 MPLSQTE4 Forwarding) Table4Popula:on4 Parser) Program) Match)Tables) +)Ac:ons) Control)Flow) Compiler4 My4(running)4switch4 Forwarding) Table4Popula:on4 Switch4Program4API4 Switch4Run:me4API4 Linux4 Switch4Driver4 Add/delete4 Program4 PISA4 Chip4 MPLSQTE4

66 56