lecture 18: network virtualization platform (NVP) 5590: software defined networking anduo wang, Temple University TTLMAN 401B, R 17:30-20:00

Transcription

1 lecture 18: network virtualization platform (NVP) 5590: software defined networking anduo wang, Temple University TTLMAN 401B, R 17:30-20:00

2 Network Virtualization in multi-tenant Datacenters Teemu Koponen., et al. Network Virtualization in Multi-tenant 2 Datacenters

3 (partial) server virtualization server virtualization -managing computational resources by exposing the software abstraction of a server to users partially realized -new application / environment requires an associated change in the network 3

4 (partial) server virtualization partially realized -new application / environment requires an associated change in the network and services -different workloads demand different topology - flat L2, L3, L4-L7 -virtualized workloads operate in the physical address space - arbitrary location, address type, server virtualization requires network virtualization -no single unifying abstraction, invoked in a global manner 4

5 (partial) server virtualization computation is virtualized, the network is not network virtualization primitives -VLAN virtualize L2 domain -VRFs virtualize L3 FIB -NAT virtualize IP space -MPLS virtualize path 5

6 (partial) server virtualization computation is virtualized, the network is not network virtualization primitives -VLAN virtualize L2 domain -VRFs virtualize L3 FIB -NAT virtualize IP space -MPLS virtualize path but -traditional configured box-by-box -no single unifying abstraction, invoked in a global manner 6

7 solution network virtualization virtual networks over the same physical network -each with independent service models -topologies -addressing architectures the creation and management -done through global abstractions, rather than pieced together through box-by-box configuration 7

8 solution network virtualization virtual networks over the same physical network -each with independent service models -topologies -addressing architectures the creation and management -done through global abstractions, rather than pieced together through box-by-box configuration NVP -deployed in dozens of production environments -last few years -tens of thousands of virtual networks and virtual machines -enterprise network 8

9 multi-tenant datacenter (MTD) CP CP CP Control Abstraction VM L2 L3 L2 VM Packet Abstraction Network Hypervisor Physical Forwarding Infrastructure Figure 1: A network hypervisor sits on top of the service provider hosts connected by physical network -each host hosts many VMs, connected by a virtual switch 9

10 MTD control abstraction CP CP CP Control Abstraction VM L2 L3 L2 VM Packet Abstraction logical datapath Network Hypervisor Physical Forwarding Infrastructure Figure 1: A network hypervisor sits on top of the service provider -set of logical network elements -= a packet forwarding pipeline interface, -= a sequence of lookup tables -= resulting in a forwarding decision 10

11 MTD packet abstraction CP CP CP Control Abstraction VM L2 L3 L2 VM Packet Abstraction packets sent by endpoints in the MTB have the same switching, routing, and filtering services as in a physical network Network Hypervisor Physical Forwarding Infrastructure Figure 1: A network hypervisor sits on top of the service provider 11

12 control- and packet- abstractions CP CP CP Control Abstraction VM L2 L3 L2 VM Packet Abstraction Network Hypervisor Physical Forwarding Infrastructure Figure 1: A network hypervisor sits on top of the service provider hypervisor implements the abstractions by implementing tenant-specific logical data paths over the provider s physical network 12

13 implementing control- and packet- abstractions Logical Forwarding Logical Ingress Port Logical Datapath 1 Logical Datapath 2 Logical Egress Port Source vnic Physical Datapath (OVS) Physical Fabric (ECMP) Dest. vnic Figure 2: Source Hypervisor OVS on the sender VM -implements the logical data path -(after forwarding decision) tunnels to the receiving host hypervisor the receiving hypervisor -decapsulates the packet and sends it the destination VM 13 Tunneling The virtual switch of the originating host hypervisor Dst

14 implementing control- and packet- abstractions Logical Forwarding Logical Ingress Port Logical Datapath 1 Logical Datapath 2 Logical Egress Port Source vnic Physical Datapath (OVS) Physical Fabric (ECMP) Dest. vnic Figure 2: OVS on the sender VM Source Hypervisor Tunneling Dst The virtual switch of the originating host hypervisor -configured by a centralized SDN controller 14

15 implementing control- and packet- abstractions Physical, Nonvirtualized Workloads Gateway Cluster Service Node Cluster Controller Cluster Hypervisor Hypervisor VM 1 VM 2 VM 3 VM 4 Hypervisor VM 5 VM 6 Figure 3: In NVP, controllers manage the forwarding state at all tunnels between every pair of host-hypervisors -logical point-to-point -logical broadcast, multicast implemented by service nodes 15

16 virtualization architecture Physical, Nonvirtualized Workloads Gateway Cluster Service Node Cluster Controller Cluster Hypervisor Hypervisor VM 1 VM 2 VM 3 VM 4 Hypervisor VM 5 VM 6 Figure 3: In NVP, controllers manage the forwarding state at all transport nodes -service nodes, hypervisors, gateways 16

17 virtualization at the edge 17

18 logical datapath Logical Datapath 1 Logical Datapath 2 Logical Datapath 3 VM ACL L2 ACL ACL L2 L3 ACL ACL L2 ACL Logical Physical Map Map Map Map Tunnel Figure 4: Processing steps of a packet traversing through two logical switches interconnected by a logical router (in the middle). Physical flows flow tables -similar to OVS flow-tables -metadata registers (identifier of a logical path) 18

19 forwarding performance problem: fast packet classification with wildcards -TCAM not available on OVS OVS solution: traffic locality -kernel module: sends first packet of a new flow to userspace - follow-up packets quickly matched by kernel -user module: matched against the full flow table - install on kernel exact match 19

20 fast failovers hypervisor failures -hypervisor-to-hypervisor tunnel cannot survive service node failure -controller load-balance traffic across many service nodes gateway nodes -many gateway nodes for each physical network -failover to backup (leader/backup to prevent loop) 20

21 forwarding state computation on a single controller 21

22 forwarding state computation Provisioned Configuration (2) Controller nlog Logical Control Planes Logical Datapaths Network Hypervisor Location information (1) and Forwarding State (3) inputs -(1) location of vnics - through OVS, update as VM migrates Hypervisor Hypervisor Gateway VM 1 VM 2 VM 3 VM 4 VLAN VLAN Figure 5: Inputs and outputs to the forwarding state computation process 22

23 forwarding state computation Provisioned Configuration (2) Controller nlog Logical Control Planes Logical Datapaths Network Hypervisor Location information (1) and Forwarding State (3) inputs Hypervisor Hypervisor Gateway VM 1 VM 2 VM 3 VM 4 VLAN VLAN Figure 5: Inputs and outputs to the forwarding state computation process -(1) location of vnics - through OVS, update as VM migrates -(2) service provider configuration - through NVP API, update as tenant s (virtual) network and/or physical network change 23

24 forwarding state computation Provisioned Configuration (2) Controller nlog Logical Control Planes Logical Datapaths Network Hypervisor Location information (1) and Forwarding State (3) Hypervisor Hypervisor Gateway VM 1 VM 2 VM 3 VM 4 VLAN VLAN Figure 5: Inputs and outputs to the forwarding state computation process output -logical lookup tables -(3) (transformed by the hypervisor into the physical) forwarding states, pushed to transport nodes through OpenFlow and OVS 24

25 forwarding state computation Provisioned Configuration (2) Controller nlog Logical Control Planes Logical Datapaths Network Hypervisor Location information (1) and Forwarding State (3) Hypervisor Hypervisor Gateway VM 1 VM 2 VM 3 VM 4 VLAN VLAN Figure 5: Inputs and outputs to the forwarding state computation process (proactive) output -(3) proactively compute forwarding states, and push to transport nodes -do not process any packets -benefits: scaling, failure isolation 25

26 computation challenge large total input size -123 types of input - eg., a particular type of logical ACL, location of a vnic -81 types of output - eg., a single type of attribute being configured by OVS frequent, localized changes 26

27 computation challenge incremental computation with hand-written state machine infeasible -number of event types -arbitrary interleaving 27

28 incremental computation with nlog head_table :- joined_table, joined_table, -maps controller input to output types output types (or immediate results) input types (or immediate results) tables head table joined table 28

29 incremental computation with nlog head_table :- joined_table, joined_table, -maps controller input to output types output types (or immediate results) input types (or immediate results) tables head table joined table -change in the joined tables results in (incremental) reevaluation - inserting into or removing from head table 29

30 incremental computation with nlog -non-recursive declarations and 900 tables (all three types) benefits -separate (logic) spec from the (state machine) implementation -incremental evaluation without worrying about state transition, input event ordering 30

31 extending nlog with functions 31

32 extending nlog with functions datalog re-structure column data 31

33 extending nlog with functions datalog re-structure column data nlog adds function table -certain columns of a row is a stateless function of others -NVP primitive function tables -match over flow, sequence of actions 31

34 extending nlog with functions datalog re-structure column data nlog adds function table -certain columns of a row is a stateless function of others -NVP primitive function tables -match over flow, sequence of actions hook up output and input table by arbitrary C++ -output table tuples C++ implementation -C++ implementation processing -C++ implementation tuples - input table 31

35 distribution controller cluster 32

36 distribution of computation API Universal Flows Physical Flows (OpenFlow) & OVS config Logical Controllers Location Information Physical Controllers Physical State Transport nodes distribute computation of logical data path among controllers -sharding logical datapath using its identifier -each controller computes - lookup tables + tunnels (universal flows) -universal flows published over RPC to physical controllers 33

37 distribution of computation API Universal Flows Physical Flows (OpenFlow) & OVS config Logical Controllers Location Information Physical Controllers Physical State Transport nodes distribute universal-to-physical translation among physical controllers -sharding the translation for transport nodes among controllers - translation independent for each transport node 34

38 distribution of computation API Universal Flows Physical Flows (OpenFlow) & OVS config Logical Controllers Location Information Physical Controllers Physical State Transport nodes logical-/physical- controller failover by hot standbys -one highly-available controller acts as sharding coordinator - elected using Zookeeper -maintain a (master, standby) pair - if master fails, promotes standby to master, find new standby - if standby fails, coordinator assigns a new standby 35

39 extended onix-distributed services NVP uses Onix -replicated transactional database to persist configuration state (extend Onix by) Zookeeper -elects sharding coordinator -assigns globally unique label (for logical egress port) among the many controllers 36