Hypervisors networking: best practices for interconnecting with Cisco switches

Transcription

1

2 Hypervisors networking: best practices for interconnecting with Cisco switches Ramses Smeyers Customer Support Engineer

3 Agenda

4 What is this session about? Networking virtualization concepts Hypervisor overview VMware vsphere ESXi 5.5 Microsoft HyperV 3.0 XenServer 6.2 4

5 Networking virtualization concepts

6 Hypervisor networking virtualization VM-to-VM and VM to physical host traffic handled via software switch that lives in the hypervisor VM-to-VM: memory transfer VM-to-physical: physical adapter 6

7 Hypervisor networking virtualization vsphere ESXi example VMs don t directly control networking hardware x86 hw designed to be handled by only one device driver! When a VM communicates with the outside world, it: passes the packet to its local device driver which in turns hands it to the hypervisor which in turns passes it to the physical NIC ESX gives VMs several device driver options: Strict emulation of Intel s e1000 / e1000e VMware vmxnet/vmxnet3: paravirtualized! VMs have MAC addresses that appear on the wire 7

8 Gratuitous ARP 8

9 Gratuitous ARP 9

10 spanning-tree? Enable? spanning-tree port type edge [ trunk ] spanning-tree portfast With spanning-tree link-failure/fail back 30s block on port network disconnections 10

11 Switch independent / dependent Independent no specific switch protocol / configuration is required Dependent specific switch configuration is required (port-channel) 11

12 Hypervisor overview VMware vsphere ESXi 12

13 vnetwork standard switch Nic: Virtual machine NIC vmnic: Physical server NIC Portgroup!= VLAN vmnic {0,1} vswitch 13

14 vnetwork distributed switch Multiple distributed switches can exist on 1 server Enterprise plus license is required 14

15 VMware vsphere ESXi Route based on originating virtual port Route based on IP hash Route based on IP hash + LACP (vds) Route based on source MAC hash Route based on physical NIC load (vds) Use explicit failover order 15

16 Uplink connectivity possibility's vss vds vds (extra): Route based on physical NIC load Route based on IP hash + LACP 16

17 Load balancing Per portgroup: Route based on originating virtual port Route based on IP hash Route based on source MAC hash Route based on physical NIC load (vds) Use explicit failover order Per DVUplink: Route based on IP hash + LACP (vds) 17

18 Load balancing: vss Override vswitch0 properties 18

19 Load balancing: vds Not configurable on the DVUplinks 19

20 Active / Standby / Unused vmnic0 vminc1 vmnic2 line Ethernet1/17 is up Ethernet1/17 is up GigabitEthernet1/0/5 is up, protocol is up (connected) 20

21 Route based on originating virtual port VSS + VDS Default configuration Even distribution of traffic if the number of veth adapters > physical adapters 1 veth can only be on 1 physical interface Can go to 1 or multiple switches 21

22 Route based on originating virtual port Switch configuration interface GigabitEthernet1/0/5 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/6 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/5 switchport trunk encapsulation dot1q switchport trunk allowed vlan 49 switchport mode trunk spanning-tree portfast interface GigabitEthernet1/0/6 switchport trunk encapsulation dot1q switchport trunk allowed vlan 49 NXOS: spanning-tree port type edge trunk switchport mode trunk spanning-tree portfast 22

23 Route based on originating virtual port - ESXi 23

24 Route based on IP hash VSS + VDS Uplink based on a hash of source and destination IP address Evenness of traffic distribution depends on the number of TCP/IP sessions and their characteristics Physical switch sees the client MAC address on multiple ports. Use of 802.3ad is required mode on Needs to connect to 1 port-channel Route based on IP hash does not work with Cisco UCS-B Fabric Interconnects cannot do VPC 24

25 Route based on IP hash no port-channel 25

26 Route based on IP hash Switch configuration interface GigabitEthernet1/0/5 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 1 mode on interface GigabitEthernet1/0/6 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 1 mode on interface Port-channel1 switchport access vlan 49 switchport mode access spanning-tree portfast bdsol (config-if)#channel-group 1 mode? active Enable LACP unconditionally auto Enable PAgP only if a PAgP device is detected desirable Enable PAgP unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected bdsol (config-if)#channel-group 1 mode 26

27 Route based on IP hash ESXi 27

28 Route based on IP hash + LACP (vds) VDS Uplink based on a hash of source and destination IP address Evenness of traffic distribution depends on the number of TCP/IP sessions and their characteristics Physical switch sees the client MAC address on multiple ports. Use of 802.3ad is required active or passive Needs to connect to 1 port-channel 28

29 Route based on IP hash + LACP (vds) - ESXi Active The port is in an active negotiating state, in which the port initiates negotiations with remote ports by sending LACP packets. Passive The port is in a passive negotiating state, in which the port responds to LACP packets it receives but does not initiate LACP negotiation. 29

30 Route based on IP hash + LACP (vds) - ESXi Extra LACP_MgmtPort 30

31 Route based on IP hash + LACP (vds) Switch configuration interface Port-channel1 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/5 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 1 mode active interface GigabitEthernet1/0/6 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 1 mode active channel-group 1 mode passive channel-group 1 mode active ESXi Switch Active Active Yes Yes Passive Yes No Passive 31

32 Route based on source MAC hash VSS + VDS Uplink chosen based on a hash of the source MAC address (VM) 1 veth 1 physical adapter Even distribution of traffic if random MAC addresses Be careful with manual MAC address assignments Can go to 1 or multiple switches 32

33 Route based on source MAC hash - ESXi 33

34 Route based on source MAC hash Switch configuration interface GigabitEthernet1/0/5 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/6 switchport access vlan 49 switchport mode access spanning-tree portfast 34

35 Route based on physical NIC load (vds) VDS Initial placement based on Route based on originating virtual port If RX or TX > 75% rebalance VMkernel check runs every 30 seconds 35

36 Route based on physical NIC load (vds) VDS Initial placement based on Route based on originating virtual port If RX or TX > 75% rebalance VMkernel check runs every 30 seconds MAC moves on switch layer Can go to 1 or multiple switches 36

37 Route based on physical NIC load (vds) 37

38 Route based on physical NIC load (vds) interface GigabitEthernet1/0/5 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/6 switchport access vlan 49 switchport mode access spanning-tree portfast No port-channel as asymmetric return traffic will be dropped 38

39 Use explicit failover order VSS + VDS Highest order uplink from the list of active adapters Can go to 1 or multiple switches 39

40 Use explicit failover order - ESXi All MAC s on 1 port 40

41 Use explicit failover order Switch configuration interface GigabitEthernet1/0/5 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/6 switchport access vlan 49 switchport mode access spanning-tree portfast 41

42 VMware conclusion vss Switch independent Route based on originating virtual port Switch dependent Route based on IP hash vds Switch independent Route based on physical NIC load Switch dependent Route based on IP hash + LACP 42

43 Hypervisor overview Microsoft Hyper-V 3.0 Windows

44 Uplink connectivity possibility's Single physical NIC or Teaming 44

45 Enable NIC Teaming Pre Windows 2012 Teaming provided by vendor device drivers Windows 2012 Teaming provided native by Windows 45

46 Microsoft Network Adapter Multiplexor Driver 46

47 Uplink connectivity details All Address hash modes Hyper-V Port mode Switch Independent Scenario 1 Scenario 2 Switch Dependent (Static and LACP) Address Hash comes in 3 flavors: Scenario 3 and 4 Scenario 5 and 6 4-tuple hashing (Source and destination TCP ports, usually with IP addresses, this is the default hashing mode) 2-tuple: If the ports aren t available (encrypted traffic such as IPsec) then it ll go to 2-tuple where it uses Source and destination IP address, with or without MAC addresses MAC address hash: If not IP traffic, then MAC addresses are hashed 47

48 Static / LACP / Independent Static teaming 802.3ad draft v1 LACP 802.1ax LACP, also referred to as 802.3ad Independent No switch configuration required 48

49 Standby adapter bdsol #show int GigabitEthernet1/0/3 GigabitEthernet1/0/3 is up, line protocol is up (connected) bdsol #show int GigabitEthernet1/0/4 GigabitEthernet1/0/4 is up, line protocol is up (connected) bdsol # 49

50 Single physical NIC Normal server connection Possibility of trunk Multiple MAC s on 1 switch port 50

51 Switch Independent - All Address hash modes Outbound traffic is spread across all active members Can go to 1 or multiple switches Observation: traffic sticks to 1 interface 51

52 Switch Independent - All Address hash modes Observation: All traffic sticks to 1 interface 52

53 Switch Independent - All Address hash modes Switch configuration interface GigabitEthernet1/0/3 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/4 switchport access vlan 49 switchport mode access spanning-tree portfast For Your Reference 53

54 Switch Independent - Hyper-V Port mode Outbound traffic is tagged with the port on the Hyper-V switch All traffic with that port tag is sent on the same team member 1 MAC address will be fixed to 1 physical switch port Can go to 1 or multiple switches 54

55 Switch Independent - Hyper-V Port mode 55

56 Switch Independent - Hyper-V Port mode Switch configuration interface GigabitEthernet1/0/3 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/4 switchport access vlan 49 switchport mode access spanning-tree portfast For Your Reference 56

57 Switch Dependent - All Address hash modes Outbound traffic is spread across all active members Inbound traffic will be distributed by the switch s load distribution algorithm Needs to connect to 1 portchannel 57

58 Switch Dependent - All Address hash modes 58

59 Switch Dependent - All Address hash modes Switch configuration Static Teaming interface GigabitEthernet1/0/3 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 2 mode on interface GigabitEthernet1/0/4 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 2 mode on interface Port-channel2 switchport access vlan 49 switchport mode access bdsol (config-if)#channel-group 2 mode? active Enable LACP unconditionally auto Enable PAgP only if a PAgP device is detected desirable Enable PAgP unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected bdsol (config-if)#channel-group 2 mode 59

60 Switch Dependent - All Address hash modes Switch configuration LACP interface GigabitEthernet1/0/3 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 2 mode active interface GigabitEthernet1/0/4 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 2 mode active interface Port-channel2 switchport access vlan 49 switchport mode access bdsol (config-if)#channel-group 2 mode? active Enable LACP unconditionally auto Enable PAgP only if a PAgP device is detected desirable Enable PAgP unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected bdsol (config-if)#channel-group 2 mode 60

61 Switch Dependent - Hyper-V Port mode Outbound traffic is tagged with the port on the Hyper-V switch All traffic with that port tag is sent on the same team member 1 MAC address will be fixed to 1 physical switch port Inbound traffic will be distributed by the switch s load distribution algorithm Needs to connect to 1 portchannel 61

62 Switch Dependent - Hyper-V Port mode 62

63 Switch Dependent - Hyper-V Port mode Switch configuration Static Teaming interface GigabitEthernet1/0/3 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 2 mode on interface GigabitEthernet1/0/4 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 2 mode on interface Port-channel2 switchport access vlan 49 switchport mode access bdsol (config-if)#channel-group 2 mode? active Enable LACP unconditionally auto Enable PAgP only if a PAgP device is detected desirable Enable PAgP unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected bdsol (config-if)#channel-group 2 mode 63

64 Switch Dependent - Hyper-V Port mode Switch configuration LACP interface GigabitEthernet1/0/3 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 2 mode active interface GigabitEthernet1/0/4 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 2 mode active interface Port-channel2 switchport access vlan 49 switchport mode access bdsol (config-if)#channel-group 2 mode? active Enable LACP unconditionally auto Enable PAgP only if a PAgP device is detected desirable Enable PAgP unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected bdsol (config-if)#channel-group 2 mode 64

65 Hyper-V conclusion Switch independent Switch Independent - Hyper-V Port mode Switch dependent Switch Dependent - Hyper-V Port mode 65

66 Hypervisor overview XenServer

67 Uplink connectivity possibilities Single physical NIC or Bonding LACP as of XenServer

68 Single physical NIC Normal server connection Possibility of trunk Multiple MAC s on 1 switch port 68

69 Active-active balance-slb Server Load Balancing 1 VM is fixed to 1 physical adapter Every 10 second traffic is evaluated and balanced over all physical adapters MAC move Can go to 1 or multiple switches 69

70 Active-active 70

71 Active-active interface GigabitEthernet1/0/1 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/2 switchport access vlan 49 switchport mode access spanning-tree portfast For Your Reference 71

72 Active-passive 1 NIC is active, other NIC s are passive waiting for adapter failure Can goto 1 or multiple switches 72

73 Active-passive 73

74 Active-passive interface GigabitEthernet1/0/1 switchport access vlan 49 switchport mode access spanning-tree portfast interface GigabitEthernet1/0/2 switchport access vlan 49 switchport mode access spanning-tree portfast For Your Reference 74

75 LACP with load balancing based on IP and port of source and destination Outgoing traffic will be hashed over multiple uplinks based on src/dest IP and port Inbound traffic will be distributed by the switch s load distribution algorithm Needs to connect to 1 port-channel 75

76 LACP with load balancing based on IP and port of source and destination 76

77 LACP with load balancing based on IP and port of source and destination interface GigabitEthernet1/0/1 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 3 mode active interface GigabitEthernet1/0/2 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 3 mode active interface Port-channel3 switchport access vlan 49 switchport mode access bdsol (config-if)#channel-group 3 mode? active Enable LACP unconditionally auto Enable PAgP only if a PAgP device is detected desirable Enable PAgP unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected bdsol (config-if)#channel-group 3 mode 77

78 LACP with load balancing based on source MAC address 1 VIF will use 1 outgoing physical interface (1 MAC per link) 1-1 mapping between VIF (virtual interface) and physical interface/switch port Needs to connect to 1 port-channel 78

79 LACP with load balancing based on source MAC address 79

80 LACP with load balancing based on source MAC address interface GigabitEthernet1/0/1 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 3 mode active interface GigabitEthernet1/0/2 switchport access vlan 49 switchport mode access spanning-tree portfast channel-group 3 mode active interface Port-channel3 switchport access vlan 49 switchport mode access bdsol (config-if)#channel-group 3 mode? active Enable LACP unconditionally auto Enable PAgP only if a PAgP device is detected desirable Enable PAgP unconditionally on Enable Etherchannel only passive Enable LACP only if a LACP device is detected bdsol (config-if)#channel-group 3 mode 80

81 XenServer conclusion Switch independent Active-Active Switch dependent LACP with load balancing based on source MAC address 81

82 Nexus 1000V 82

83 Nexus 1000V Virtual Supervisor Module(VSM) CLI interface into the Nexus 1000V Leverages NX-OS Controls multiple VEMs as a single network device Virtual Ethernet Module(VEM) Replaces Vmware s virtual switch Enables advanced switching capability on the hypervisor Provides each VM with dedicated switch ports 83

84 Load balance possibilities 84

85 Nexus 1000V Multi hypervisor switch (VMware, Hyper-V* and KVM*) Layer 2 switching: VLANs, private VLANs, VXLAN, loop prevention, multicast, virtual PortChannels, LACP, ACLs Network management: SPAN, ERSPAN, Netflow 9, vtracker, vcenter Server Plug-in Enhanced QOS features Cisco vpath Security: DHCP Snooping, IP Source Guard, Dynamic ARP inspection, Cisco TrustSec SGA support Cisco Virtual Security Gateway Other virtual services (Cisco ASA 1000V, Cisco vwaas, etc..) 85

86 Nexus 1000V Sessions BRKVIR-2011 BRKVIR-2017 BRKVIR-2044 BRKDCT-3444 Nexus 1000v LTRVIR-2963 Deploying Services in a Virtualized Environment The Nexus 1000V on Microsoft Hyper-V: Expanding the Virtual Edge Multi-Hypervisor Networking Compare and Contrast Advanced - Configuration of features common to both UCS and Deploying virtual services in vpath service chain with nexus 1000v 86

87 Key take-aways Understand the hypervisor s load-balancing mechanisms Align the configuration on hypervisor and networking layer All adapters are always up MAC moves are possible depending on the load balancing algorithm Use the correct port-channel configuration (on/active/passive) 87

88 Call to Action Visit the World of Solutions:- Cisco Campus Walk-in Labs Technical Solutions Clinics Meet the Engineer Lunch Time Table Topics, held in the main Catering Hall Recommended Reading: For reading material and further resources for this session, please visit 88

89 Complete Your Online Session Evaluation Complete your online session evaluation Complete four session evaluations and the overall conference evaluation to receive your Cisco Live T-shirt 89

90