libvnf: building VNFs made easy

Transcription

1 libvnf: building VNFs made easy Priyanka Naik, Akash Kanase, Trishal Patel, Mythili Vutukuru Dept. of Computer Science and Engineering Indian Institute of Technology, Bombay SoCC th October, 2018

2 NFV ecosystem Orchestrator Firewall VNF VNF VNF VNF Load balancer VM VM VM VM Router Hypervisor Network address translator NFV: Network Function Virtualization VNF: Virtual Network Function 2

3 NFV ecosystem Will they give good performance? Is it easy to build them? Orchestrator Firewall VNF VNF VNF VNF Load balancer VM VM VM VM Router Hypervisor Network address translator NFV: Network Function Virtualization VNF: Virtual Network Function 3

4 How to build VNF? VNF code developed by VNF developer 4

5 How to build VNF? VNF code developed by VNF developer 38% EPC code read/write packets CORD Intel EPC: 4

6 How to build VNF? VNF Processing logic VNF code developed by VNF developer VNF Framework 38% EPC code read/write packets CORD Intel EPC: 4

7 How to build VNF? VNF Processing logic netbricks VNF Framework StatelessNF OpenNF VPP 4

8 How to build VNF? VNF Processing logic netbricks VNF Framework StatelessNF OpenNF VPP What is missing in these frameworks? 4

9 What is required from VNF frameworks? " Requirement 1: Support for both L3 and Transport VNF " Requirement 2: Flexibility of network stack " Requirement 3: Support for distributed state management 5

10 What is required from VNF frameworks? " Requirement 1: Support for both L3 and Transport VNF " Requirement 2: Flexibility of network stack " Requirement 3: Support for distributed state management 6

11 Support for L3 and transport VNFs Layer 3 VNFs Network address translator Layer 3 Load balancer 7

12 Support for L3 and transport VNFs Layer 3 VNFs Network address translator Layer 3 Load balancer Header manipulations N/W layer Data link layer 7

13 Support for L3 and transport VNFs Layer 3 VNFs Network address translator Layer 3 Load balancer Header manipulations N/W layer Data link layer Frameworks: netbricks, YANFF 7

14 Support for L3 and transport VNFs Layer 3 VNFs Transport Layer VNFs vepc internet Network address translator Layer 3 Load balancer Header manipulations N/W layer Data link layer Frameworks: netbricks, YANFF 7

15 Support for L3 and transport VNFs Layer 3 VNFs Transport Layer VNFs vepc internet Network address translator Layer 3 Load balancer Header manipulations N/W layer Connection termination Request processing Transport Layer N/W layer Connection initiation Data link layer Data link layer Frameworks: netbricks, YANFF 7

16 Support for L3 and transport VNFs Layer 3 VNFs Transport Layer VNFs vepc internet Network address translator Layer 3 Load balancer Header manipulations N/W layer Connection termination Request processing Transport Layer N/W layer Connection initiation Data link layer Data link layer Frameworks: netbricks, YANFF Frameworks: mtcp, TLDK Netbricks: Taking the v out of nfv. In Proc. of OSDI 16 YANFF: mtcp: A highly scalable user-level tcp stack for multicore systems. In Proc. of NSDI 14 TLDK: 7

17 Support for L3 and transport VNFs Layer 3 VNFs Transport Layer VNFs vepc internet Network address translator Layer 3 Load balancer Header manipulations N/W layer Connection termination Request processing Transport Layer N/W layer Connection initiation Data link layer Data link layer Frameworks: netbricks, YANFF Frameworks: mtcp, TLDK Netbricks: Taking the v out of nfv. In Proc. of OSDI 16 YANFF: mtcp: A highly scalable user-level tcp stack for multicore systems. In Proc. of NSDI 14 TLDK: Are these frameworks enough? 7

18 Event driven I/O Existing transport-layer frameworks are event-driven 8

19 Event driven I/O Pros: Existing transport-layer frameworks are event-driven 8

20 Event driven I/O Existing transport-layer frameworks are event-driven Pros: " Efficient for multi-core scalability 8

21 Event driven I/O Existing transport-layer frameworks are event-driven Pros: " Efficient for multi-core scalability 8

22 Event driven I/O Existing transport-layer frameworks are event-driven Pros: " Efficient for multi-core scalability Cons: 8

23 Event driven I/O Existing transport-layer frameworks are event-driven Pros: " Efficient for multi-core scalability Cons: " Needs explicit request state storage 8

24 Need to maintain request state 1 2 A B C 4 3 9

25 Need to maintain request state 1 2 A B C 4 3 State at B to process A s request A s request C s reply Connection identifiers 9

26 Need to maintain request state VNF processing layer (abstraction?) 1 2 A B C 4 3 State at B to process A s request network stack (mtcp) (connection) A s request C s reply Connection identifiers DPDK and netmap layer (packet) 9

27 Need to maintain request state VNF processing layer (abstraction?) 1 2 A B C 4 3 State at B to process A s request network stack (mtcp) (connection) A s request C s reply Connection identifiers DPDK and netmap layer (packet) Existing frameworks do not provide this support 9

28 What is required from VNF frameworks? " Requirement 1: Support for both Layer 3 and Transport VNF " Requirement 2: Flexibility of network stack " Requirement 3: Support for distributed state management 10

29 Flexibility of network stack Kernel Stack Application VNF Kernel network stack vnic 11

30 Flexibility of network stack Kernel Stack Kernel Bypass Stack Application VNF Kernel network stack vnic Application VNF + userspace stack DPDK/netmap vnic 11

31 Flexibility of network stack Kernel Stack Kernel Bypass Stack Application VNF Kernel network stack vnic Application VNF + userspace stack DPDK/netmap vnic Easy switch between stacks 11

32 What is required from VNF frameworks? " Requirement 1: Support for both L3 and Transport VNF " Requirement 2: Flexibility of network stack " Requirement 3: Support for distributed state management 12

33 Support for distributed state management Orchestrator VNF 1 VNF 2 VNF 2 VNF 3 VM VM VM VM Hypervisor 13

34 Support for distributed state management VNF 2 VNF 2 State Synchronization 13

35 Support for distributed state management VNF 2 VNF 2 State Synchronization State Migration opennf, split/merge 13

36 Support for distributed state management VNF 2 Data Store VNF 2 State Synchronization Remote store statelessnf State Migration opennf, split/merge Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI 17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI 13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM 14 13

37 Support for distributed state management VNF 2 Data Store VNF 2 State Synchronization Remote store statelessnf State Migration opennf, split/merge None of above support transport layer VNFs Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI 17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI 13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM 14 13

38 Summary of VNF Frameworks Requirement/ Framework netbricks Flick StatelessNF Split-Merge/ OpenNF libvnf Layer 3 + Applayer support Flexibility of network stack Distributed State Management no yes no no yes no no no no yes no no yes yes yes Netbricks: Taking the v out of nfv. In Proc. of OSDI 16 Flick: Developing and running application-specific network services. In Proc. of USENIX ATC 16 Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI 17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI 13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM 14 14

39 libvnf Design Goals VNF processing logic Handled by VNF developer R3 R2 Distributed State Management Support for network and transport layer VNF Handled by libvnf R1 Flexibility of network stack R: Requirement 15

40 libvnf overview VNF code API Calls libvnf API 16

41 libvnf overview VNF code API Calls libvnf API mtcp+ netmap/dpdk initialization Kernel stack initialization Stack initialization 16

42 libvnf overview VNF code API Calls libvnf API mtcp+ netmap/dpdk initialization Kernel stack initialization Stack initialization Per-core threads 16

43 libvnf overview VNF code API Calls libvnf API mtcp+ netmap/dpdk initialization Kernel stack initialization Stack initialization Per-core threads Per-core data structures Lock-free Cache optimized 16

44 libvnf API libvnf API Communication Request state State Management 17

45 libvnf API libvnf API Request state State Management 18

46 libvnf API libvnf API Communication Request state State Management 18

47 Communication API VNF code libvnf API Per-core data structures Per-core packet pool Pre-allocated memory pools (Per-core packet pools) 19

48 Communication API VNF code registercallback(socket, fn) Store mapping libvnf API Per-core data structures Per-core packet pool Pre-allocated memory pools (Per-core packet pools) 19

49 Communication API VNF code registercallback(socket, fn) Store mapping libvnf API Per-core data structures fn(packet) Packet arrives on socket Per-core packet pool Pre-allocated memory pools (Per-core packet pools) 19

50 Communication API VNF code getpktbuf libvnf API Per-core data structures Per-core packet pool Pre-allocated memory pools (Per-core packet pools) 19

51 Communication API VNF code getpktbuf Buffer to write packet libvnf API Per-core data structures Per-core packet pool Pre-allocated memory pools (Per-core packet pools) 19

52 libvnf API VNF Design Requirements Communication State Management 20

53 libvnf API VNF Design Requirements Communication Request state State Management 20

54 Need for request state VNF (abstraction processing?) layer (abstraction?) 1 2 A B C 4 3 State at B to process A s request A s request C s reply Connection identifiers network stack (mtcp) (connection) DPDK and netmap layer (packet) 21

55 Need for request state VNF (abstraction processing?) layer REQUEST OBJECT 1 2 A B C 4 3 State at B to process A s request A s request C s reply Connection identifiers network stack (mtcp) (connection) DPDK and netmap layer (packet) Request object 22

56 Request Object API 1 2 A B C 4 3 libvnf API Per-core request pool 23

57 Request Object API 1 2 A B C 4 3 allocreqobj(a connection_id) Allocate request object block for A s request libvnf API Per-core request pool A s request C s reply Connection identifiers 23

58 Request Object API 1 2 A B C 4 3 allocreqobj(a connection_id) Allocate request object block for A s request libvnf API Per-core request pool A s request Per-core packet pool C s reply Connection identifiers 23

59 Request Object API 1 2 A B C 4 3 linkreqobj(c connection_id) Link to the existing A request object libvnf API Per-core request pool A s request Per-core packet pool C s reply Connection identifiers 23

60 libvnf API VNF Design Requirements Communication Request state 24

61 libvnf API VNF Design Requirements Communication Request state State Management 24

62 State Management API VNF code libvnf API 25

63 State Management API VNF code libvnf API Local data store pool 25

64 State Management API VNF code setdata( LOCAL ) libvnf API Store in local datastore Local data store pool 25

65 State Management API VNF code setdata( LOCAL ) libvnf API Store in local datastore Local data store pool libvnf data store wrapper Redis KV store Remote Data store 25

66 State Management API VNF code setdata( REMOTE ) libvnf API Cache locally Local data store pool Store in remote data store libvnf data store wrapper Redis KV store Remote Data store 25

67 Evaluation " Overhead of libvnf " Scalability with cores " Benefits of libvnf 26

68 Setup 1 2 A B C

69 Setup 1 2 A B C 4 3 VNF A VNF C S/W switch (on kernel) 27

70 Setup 1 2 A B C 4 3 VNF A VNF C VNF B S/W switch (on kernel) S/W switch (like netmap-vale) NIC Queue Physical NIC 27

71 Setup 1 2 A B C 4 3 VNF A VNF C VNF B S/W switch (on kernel) S/W switch (like netmap-vale) NIC Queue VNF A, C: 4 core, 4GB RAM VNF B: 4 GB RAM, cores varied Physical NIC 27

72 Evaluation " Overhead of libvnf " Scalability with cores " Benefits of libvnf 28

73 Overhead check 29

74 Overhead check <5% overhead of libvnf DPDK~ netmap performance 29

75 Evaluation " Overhead of libvnf " Scalability with cores " Benefits of libvnf 30

76 Core scalability 31

77 Core scalability scales linearly with cores 31

78 Evaluation " Overhead of libvnf " Scalability with cores " Benefits of libvnf 32

79 Building VNFs VNF Performance Overhead of libvnf LoC Saved IMS (IP Multimedia Subsystem) EPC (LTE-Evolved Packet Core ) 3.4% 42% 5.5% 38% Layer 3 Load Balancer 14% 52% 33

80 Building VNFs VNF Performance Overhead of libvnf LoC Saved IMS (IP Multimedia Subsystem) EPC (LTE-Evolved Packet Core ) 3.4% 42% 5.5% 38% Layer 3 Load Balancer 14% 52% Low overhead in app-layer VNF Higher overhead in L3 VNF 33

81 Summary " Library to ease building of VNFs " Expressive to build L3 and App-layer VNF " Supports multiple network stacks " Low performance overhead 34

82 Thank You 35

83 Setup 1 2 A B C 4 3 VNF A VNF C LB VNF VNF B VNF B Data store VM S/W switch (on kernel) S/W switch (vale on netmap) NIC Queue Physical NIC 36

84 Setup 1 2 A B C 4 3 VNF A VNF C LB VNF VNF B VNF B Data store VM S/W switch (on kernel) S/W switch (vale on netmap) VNF A, C: 4 core, 4GB RAM VNF B: 4 GB RAM, cores varied Data Store VM: 6 core, 16GB RAM LB: 1 core, 4GB RAM NIC Queue Physical NIC 36