VPP Host Stack. TCP and Session Layers. Florin Coras, Dave Barach, Keith Burns, Dave Wallace

Transcription

1 Host Stack and Layers Florin Coras, Dave Barach, Keith Burns, Dave Wallace

2 - A Universal Terabit Network Platform For Native Cloud Network Services Most Efficient on the Planet EFFICIENCY Superior Performance PERFORMANCE Flexible and Extensible SOFTWARE DEFINED NETWORKING Cloud Native CLOUD NETWORK SERVICES Open Source LINUX FOUNDATION Breaking the Barrier of Software Defined Network Services 1 Terabit Services on a Single Intel Xeon Server!

3 How does it work? Compute Optimized SW Network Platform 1 Packet processing is decomposed into a directed graph of nodes 2 packets move through graph nodes in vector 3 graph nodes are optimized to fit inside the instruction cache vhost-userinput af-packetinput dpdk-input Packet 0 Packet 1 Microprocessor ethernetinput Packet 2 Packet 3 3 Instruction Cache arp-input cdp-input lldp-input mpls-input ip4-lookupmulitcast l2-input ip4-lookup* ip4-input...-nochecksum ip6-input Packet 4 Packet 5 Packet 6 Packet 7 4 Data Cache Packet 8 interfaceoutput ip4-loadbalance mpls-policyencap ip4-rewritetransit ip4- midchain Packet 9 Packet 10 4 packets are pre-fetched into the data cache. * Each graph node implements a micro-nf, a micro-networkfunction processing packets. Makes use of modern Intel Xeon Processor micro-architectures. Instruction cache & data cache always hot è Minimized memory latency and usage.

4 Motivation: Container networking PID 1234 PID 4321 glibc send() recv() FIFO kernel FIFO IP (routing) IP (routing) device device

5 Motivation: Container networking PID 1234 PID 4321 send() etc etc etc ACL, SR, VXLAN, LISP IP4/6 MPLS Ethernet recv() FIFO af_packet dpdk af_packet FIFO IP (routing) FIFO dpdk FIFO IP (routing) device device device device device

6 Why not this? PID 1234 PID 4321 send() recv() FIFO FIFO IP DPDK

7 Host Stack App rx tx shm segment

8 Host Stack: Layer App Maintains per app state and conveys to/from session events Allocates and manages sessions/segments/fifos Isolates network resources via namespacing lookup tables (5-tuple) and local/global session rule tables (filters) Support for pluggable transport protocols Binary/native C API for external/builtin applications rx tx shm segment

9 Host Stack: SVM FIFOs App rx tx shm segment Allocated within shared memory segments Fixed position and size Lock free enqueue/dequeue but atomic size increment Option to dequeue/peek data Support for out-of-order data enqueues

10 Host Stack: App Clean-slate implementation Complete state machine implementation Connection management and flow control (window management) Timers and retransmission, fast retransmit, SACK NewReno congestion control, SACK based fast recovery Checksum offloading Linux compatibility tested with IWL protocol tester rx tx shm segment

11 Host Stack: Comms Library (VCL) Comms library (VCL) apps can link against LD_PRELOAD library for legacy apps epoll App rx tx shm segment

12 Application Attachment attach bind (server) connect (client) App shm segment

13 Establishment Client Server attach bind listen

14 Establishment attach connect Client Server attach bind open listen

15 Establishment Client Server handshake

16 Establishment Client Server connect succeeded handshake new client

17 Establishment Client Server shm segment rx tx connect reply accept notify rx tx shm segment

18 Data Transfer write Client Server read tx write evt rx write evt rx tx rx tx copy to buffer copy to fifo Congestion control Reliable transport

19 Data Transfer write Client Server read tx write evt rx write evt rx tx rx tx copy to buffer copy to fifo Congestion control Reliable transport Not yet part of CSIT but some rough numbers on a E2690: 200k CPS and 8Gbps/core!

20 Redirected Connections (Cut-through) Client Server bind

21 Redirected Connections (Cut-through) Client Server connect redirect

22 Redirected Connections (Cut-through) Throughput is memory bandwidth constrained: ~120Gbps! Client Server connect redirect

23 Ongoing work Overall integration with k8s Istio/Envoy Rx policer/tx pacer TSO New congestion control algorithms PMTU discovery Optimization/hardening/testing VCL/LD_PRELOAD Iperf, nginx, wget, curl

24 Next steps Get involved Get the Code, Build the Code, Run the Code layer: src/vnet/session : src/vnet/tcp SVM: src/svm VCL: src/vcl Read/Watch the Tutorials Read/Watch Tutorials Join the Mailing Lists

25 Thank you!? Florin Coras irc: florinc

26

27 Multi-threading App1 rx tx rx tx IP IP Core 0 DPDK Core 1

28 Features: Namespaces App Request access to vpp ns + secret IP IP IP fib1 fib2 ns1 ns2 ns3

29 Features: Tables App1 Request access to global and/or local scope NS Local Table NS Local Table Global Table fib1 ns1 ns2

30 Features: Tables App1 Both table have rules table that can be used for filtering Local tables are namespace specific and can be used for egress filtering Global tables are fib table specific and can be used for ingress filtering NS Local Table NS Local Table Global Table fib1 ns1 ns2