VPP Host Stack. Transport and Session Layers. Florin Coras, Dave Barach, Keith Burns, Dave Wallace

Transcription

1 Host Stack Transport and Layers Florin Coras, Dave Barach, Keith Burns, Dave Wallace

2 - A Universal Terabit Network Platform For Native Cloud Network Services Most Efficient on the Planet EFFICIENCY Superior Performance PERFORMANCE Flexible and Extensible SOFTWARE DEFINED NETWORKING Cloud Native CLOUD NETWORK SERVICES Open Source LINUX FOUNDATION Breaking the Barrier of Software Defined Network Services 1 Terabit Services on a Single Intel Xeon Server!

3 Motivation: Container networking PID 1234 PID 4321 glibc send() recv() FIFO kernel FIFO IP (routing) IP (routing) device device

4 Motivation: Container networking PID 1234 PID 4321 send() etc etc etc ACL, SR, VXLAN, LISP IP4/6 MPLS Ethernet recv() FIFO af_packet dpdk af_packet FIFO IP (routing) FIFO dpdk FIFO IP (routing) device device device device device

5 Why not this? PID 1234 PID 4321 send() recv() FIFO FIFO IP DPDK

6 Host Stack App rx tx shm segment

7 Host Stack: Layer App Maintains per app state and conveys to/from session events Allocates and manages sessions/segments/fifos Isolates network resources via namespacing lookup tables (5-tuple) and local/global session rule tables (filters) Support for pluggable transport protocols Binary/native C API for external/builtin applications rx tx shm segment

8 Host Stack: SVM FIFOs App rx tx shm segment Allocated within shared memory segments Fixed position and size Lock free enqueue/dequeue but atomic size increment Option to dequeue/peek data Support for out-of-order data enqueues

9 Host Stack: App Clean-slate implementation Complete state machine implementation Connection management and flow control (window management) Timers and retransmission, fast retransmit, SACK NewReno congestion control, SACK based fast recovery Checksum offloading Linux compatibility tested with IWL protocol tester rx tx shm segment

10 Host Stack: Comms Library (VCL) Comms library (VCL) apps can link against LD_PRELOAD library for legacy apps epoll App rx tx shm segment

11 Application Attachment attach bind (server) connect (client) App shm segment

12 Establishment Client Server attach bind listen

13 Establishment attach connect Client Server attach bind open listen

14 Establishment Client Server handshake

15 Establishment Client Server connect succeeded handshake new client

16 Establishment Client Server shm segment rx tx connect reply accept notify rx tx shm segment

17 Data Transfer write Client Server read tx write evt rx write evt rx tx rx tx copy to buffer copy to fifo Congestion control Reliable transport

18 Data Transfer write Client Server read tx write evt rx write evt rx tx rx tx copy to buffer copy to fifo Congestion control Reliable transport Not yet part of CSIT but some rough numbers on a E2690: ~200k CPS and ~12Gbps/core!

19 Redirected Connections (Cut-through) Client Server bind

20 Redirected Connections (Cut-through) Client Server connect redirect

21 Redirected Connections (Cut-through) Throughput is memory bandwidth constrained: ~120Gbps! Client Server connect redirect

22 Multi-threading App1 rx tx rx tx Connections/sessions pinned to a thread Per-thread data structures/state IP IP Core 0 DPDK Core 1

23 Features: Namespaces App Request access to vpp ns + secret Namespaces are configured independently and associate applications to network layer resources like interfaces and fib tables IP IP IP fib1 fib2 ns1 ns2 ns3

24 Features: Tables App1 Request access to global and/or local scope NS Local Table NS Local Table Global Table fib1 ns1 ns2

25 Features: Tables App1 Both table have rules table that can be used for filtering Local tables are namespace specific and can be used for egress filtering Global tables are fib table specific and can be used for ingress filtering NS Local Table NS Local Table Global Table fib1 ns1 ns2

26 Ongoing work Overall integration with k8s Istio/Envoy Rx policer/tx pacer TSO New congestion control algorithms PMTU discovery Optimization/hardening/testing VCL/LD_PRELOAD Iperf, nginx, wget, curl

27 Next steps Get involved Get the Code, Build the Code, Run the Code layer: src/vnet/session : src/vnet/tcp SVM: src/svm VCL: src/vcl Read/Watch the Tutorials Read/Watch Tutorials Join the Mailing Lists

28 Thank you!? Florin Coras irc: florinc