Virtual Machine Fabric EXtension (VM-FEX) Extending the Network directly to the VM s

Transcription

1 Virtual Machine Fabric EXtension (VM-FEX) Extending the Network directly to the VM s BRKCOM-2005 Follow us on Twitter for real time updates of the #CLEUR

2 Housekeeping We value your feedback- don't forget to complete your online session evaluations after each session & the Overall Conference Evaluation which will be available online from Thursday Visit the World of Solutions and Meet the Engineer Visit the Cisco Store to purchase your recommended readings Please switch off your mobile phones After the event don t forget to visit Cisco Live Virtual: Follow us on Twitter for real time updates of the #CLEUR 2

3 Agenda What we will be Sharing Fabric Extender Technology Overview Virtual Machine Fabric Extender (VM-FEX) Introduction VM-FEX Operational Model VM-FEX General Details on UCS and Nexus 5500 VM-FEX Implementation with VMware on UCS and Nexus 5500 VM-FEX Implementation with KVM on UCS VM-FEX Implementation with Hyper-V on UCS Easy-VMFEX tool usage Performance Review Summary 3

4 Fabric Extender Technology Overview

5 Logical Switch Fabric Extension (FEX) Concept Virtualising the Network Port Legacy multi-tier architecture LAN FEX architecture LAN Switch port extended over Fabric Extender Switch Switch Switch FEX Collapse network tiers, fewer network management points 5

6 Nexus Fabric Extender Single Access Layer LAN SAN Nexus 5000 Parent Switch MDS + Cisco Nexus 2000 FEX = N7000/ C6500 N5000 Access Layer N2232 N2232 Distributed Modular System 1 12 Distributed Modular System Nexus 2000 FEX is a Virtual Line Card to the Nexus 5000 Nexus 5000 maintains all management & configuration No Spanning Tree between FEX & Nexus 5000 Over 6000 production customers Over 5 million Nexus 2000 ports deployed 6

7 Key Architectural Component #1: VNTAG Intra-Chassis Bus Header FEX architecture Switch LAN Frame Application Payload TCP IP VNTAG Ether type FEX VNTAG Frame VNTAG Ethernet D P L R Destination Virtual Interface ver Source Virtual Interface VNTAG mimics forwarding vectors inside a switch D: Direction, P: Unicast/Multicast, L: Loop Policy associated with the Virtual Interface NOT port VLAN member ship, QoS, MTU, Rate limit etc 7

8 FEX Data Forwarding Revisiting Traditional Modular Switches (Example Catalyst 6500) Constellation Bus had 32 byte header for fabric switching - Vast majority of modular switch vendors have an internal Tag for fabric communications Originally, Centralized forwarding ASICs - Line cards fed into these ASICs directly When we needed higher performance we added more ASICs in the form of Faster Switch Fabrics, and Distributed Forwarding Cards 8

9 FEX Data Forwarding Decoupling the Modular Switch Think the original C6k Satellite Program for VSL and RSL The Constellation Bus now is smaller header 6 Byte VNtag header - Core to FEX technology Keep the ASICs for high performance but put them on the Central controlling switch instead of all these line cards - Latency adds were more a function of the layers of ASICs to traverse in a tree rather than the location of these ASICs Add protocols for configuration and firmware management of these remote cards (Satellite Control Protocol, Satellite Discovery Protocol) - Allows us to get away from manual firmware code management per (remote) linecard Move from Store-and-Forward behavior to Cut-Through switching to make latency actually better 9

10 Parent Switch + FEX Decoupling the Modular Switch LAN SAN Parent Switch FEX 10

11 FEX Technology for Unified I/O Virtual Switch Ports, Cables, and NIC Ports Mapping of Ethernet and FC Wires over Ethernet Service Level enforcement Multiple data types (jumbo, lossless, FC) Individual link-states DCB Ethernet Blade Management Channels (KVM, USB, CDROM, Adapters) Individual Ethernets Fewer Cables Multiple Ethernet traffic co-exist on same cable Fewer adapters needed Overall less power Interoperates with existing Models Management remains constant for system admins and LAN/SAN admins Possible to take these links further upstream for aggregation Individual Storage (iscsi, NFS, FC) 11

12 Virtual Machine Fabric Extender (VM-FEX) Introduction

13 Logical Switch Logical Switch Extending FEX Architecture to the VMs Cascading of Fabric Extenders Virtualized Deployment VM-FEX architecture LAN LAN Switch Switch Switch port extended over cascaded Fabric Extenders to the Virtual Machine FEX FEX Hypervisor vswitch Hypervisor VM-FEX App App App App App App OS OS OS OS OS OS 13

14 Key Architectural Component #2: UCS VIC UCS Virtual Interface Card Family 256 PCIe devices Devices can be vnics or vhbas Each device has a corresponding switch interface Bandwidth 2x4x10 Gb Uses 4x10 Ether Channel, HW 40Gb Capable vnics/vhbas NOT limited to 10Gb PCIe Gen-2 x 16 Mezzanine and PCIe Form Factors 14

15 UCS VM-FEX Distributed Modular System Removing the Virtual Switching Infrastructure to a FEX UCS Fabric Interconnect Parent Switch LAN SAN MDS + UCS IOM-FEX + Cisco UCS VIC = N7000/ C6500 UCS 6100 UCS IOM Access Layer UCS IOM Distrbuted Modular System UCS VIC UCS VIC VM-FEX: Single Virtual-Physical Access Layer Collapse virtual and physical switching into a single access layer VM-FEX is a Virtual Line Card to the parent switch Parent switch maintains all management & configuration Virtual and Physical traffic treated the same App OS App OS App OS App OS App OS App OS App OS App OS App OS App OS App OS App OS 15

16 Nexus 5000/2000 VM-FEX Distributed Modular System Removing the Virtual Switching Infrastructure to a FEX Nexus 5500 Parent Switch LAN SAN + Nexus 2000 FEX + Cisco UCS VIC = N7000/ C6500 Nexus 5500 MDS Access Layer Nexus 2000 Nexus 2000 Distrbuted Modular System UCS VIC UCS VIC VM-FEX: Single Virtual-Physical Access Layer Collapse virtual and physical switching into a single access layer VM-FEX is a Virtual Line Card to the parent switch Parent switch maintains all management & configuration Virtual and Physical traffic treated the same App OS App OS App OS App OS App OS App OS App OS App OS App OS App OS App OS App OS 16

17 VM-FEX Operational Model

18 Hypervisor Hypervisor Software Based Virtual Access Layer Out of the Box Deployments Physical Network VETH VNIC VM VM VM VM VM VM VM VM Virtual Network 18

19 Hypervisor Hypervisor VM-FEX Operations Model Pre-Boot Configuration Step 1: Preboot - UCS defined PCIe devices and enumerations - Host discovers PCIe devices 19

20 Hypervisor Hypervisor VM-FEX Operational Model Defining Port Profiles on the UCS or Nexus 5000 Step 1: Preboot - UCS defined PCIe devices and enumerations - Host discovers PCIe devices Step 2: Port Profile - Folder of Network Policy on UCS or Nexus 5500 defined VLAN Web VLAN HR VLAN DB VLAN Comp Port Profiles Definition WEB Apps HR DB Compliance UCSM or Nexus

21 Hypervisor Hypervisor VM-FEX Operational Model Pushing Port Profiles to the Hypervisor System Step 1: Preboot - UCS defined PCIe devices and enumerations UCSM or Nexus 5500 exports Port Profiles - Host discovers PCIe devices Step 2: Port Profile Hypervisor Manager VLAN Web VLAN HR VLAN DB VLAN Comp UCSM or Nexus Folder of Network Policy on UCS or Nexus 5500 defined Step 3: Port Profile Export - Port Profile name list exported to virtualization manager 21

22 Hypervisor Hypervisor VM-FEX Operational Model Mapping of Port Profiles to VM Virtual Adapters Step 1: Preboot UCSM or Nexus 5500 exports Port Profiles - UCS defined PCIe devices and enumerations - Host discovers PCIe devices Hypervisor Manager VLAN Web VLAN HR VLAN DB VLAN Comp Network Manager Step 2: Port Profile - Folder of Network Policy on UCS or Nexus 5500 defined VM Step 3: Port Profile Export - Port Profile name list exported to virtualization manager Step 4: VM Definition - Named Policy in VM VM VM VM 22

23 VM-FEX Modes of Operation Enumeration vs. Hypervisor Bypass Emulated Mode PCIe Pass-Thru or VMDirectPath Standard (Emulated) Mode Each VM gets a dedicated PCIe device 12%-15% CPU performance improvement Appears as distributed virtual switch to hypervisor LiveMigration supported High Performance Mode Co-exists with Standard mode Bypasses Hypervisor layer 30% improvement in I/O performance Appears as distributed virtual switch to hypervisor Currently supported with ESX 5.0 only vmotion supported 23

24 Bandwidth (Gbps) VM-FEX Operational Model vmotion with Hypervisor Bypass (VMDirectPath with VM-FEX) 12 Temporary transition from VMDP to standard I/O vmotion to secondary host Time (sec) 8GB VM, sending UDP stream using pktgen (1500MTU) UCS B200 blades with UCS VIC card vsphere

25 Hypervisor Hypervisor VM-FEX Operational Model Simplifying the Access Infrastructure Unify the virtual and physical network - Same Port Profiles for various hypervisors and bare metal servers Physical Network Consistent functions, performance, management VETH VNIC VM VM VM VM VM VM VM VM Virtual Network 25

26 VM-FEX Operational Model Traffic Management Programmable Access Layer - Software Definable Networking (SDN) Standardize Troubleshooting - SPAN usage, Performance Trending Traffic Engineering Physical Network VLANs VM VM VM VM VM VM VM VM Virtual Network 26

27 Hypervisor Hypervisor VM-FEX Operational Model Traffic Forwarding Removing performance dependencies from VM location Physical Network Offloading software switching functionalities from host CPU More on this in upcoming slides VETH VNIC VM VM VM VM VM VM VM VM 27

28 VM-FEX Operational Model Contrasting VM-FEX to Virtualised Switching Layers Simpler Deployments - Unifying the virtual and physical network - Consistency in functionality, performance and management Robustness - Programmability of the infrastructure - Troubleshooting, traffic engineering virtual and physical together Performance - Near bare metal I/O performance - Improve jitter, latency, throughput and CPU utilization Security - Near bare metal I/O performance - Improve jitter, latency, throughput and CPU utilization 28

29 VM-FEX General Details on UCS and Nexus 5500

30 UCS System Components Foundation UCS Manager Embedded Manages entire system UCS Fabric Interconnect UCS x 10GE Ports 1 RU 40x 10GE Ports 2 RU Ethernet or FC Expansion Modules UCS Fabric Interconnect UCS x Unified Ports (Eth/FC) 1 RU 32x base and 16x expansion UCS 6100 UCS 6200 UCS Fabric Extender UCS x 10GE Downlinks to Servers 4x 10GE Uplinks to FIs UCS Fabric Extender UCS x 10GE Downlinks to Servers 8x 10GE Uplinks to FIs UCS 2104 IOM UCS 2208 IOM UCS Blade Server Chassis Flexible Bay Configuration Houses blades based on Industry-standard architecture Adapters - M81KR VIC, M71KR, etc. Up to 2x 10GE ports M81KR: Up to 128 virtual interfaces Adapter - UCS VIC 1280 Up to 8x 10GE ports Up to 256 virtual interfaces 30

31 UCS System Architecture Foundation 2x 1 Link 20 Gbps per Chassis 2x 2 Link 40 Gbps per Chassis 2x 4 Link 80 Gbps per Chassis 2x 8 Links 160 Gbps per Chassis Wire once for bandwidth, not connectivity Policy-driven bandwidth allocation Integrates as a single system into your data center 31

32 Nexus 5500 and 2200 Components for VM-FEX Foundation Nexus x 10GE Ports 1 RU 40x 10GE Ports 2 RU Ethernet or FC Expansion Modules Nexus x Unified Ports (Eth/FC) 1 RU 32x base and 16x expansion Nexus 5548 Nexus 5596 Nexus x 10GE Downlinks to Servers 4x 10GE Uplinks to FIs Nexus 2232 FEX Adapters - M81KR VIC, M71KR, etc. Up to 2x 10GE ports M81KR: Up to 128 virtual interfaces Adapter - UCS VIC 1280 Up to 8x 10GE ports Up to 256 virtual interfaces 32

33 UCS VM-FEX System View Deploying on a UCS B or C Series Infrastructure Mgmt Uplink 0 veth10 UCS Fabric Interconnect A (port profiles) veth1 veth2 veth3 veth4 Fiber Channel Uplink Ports vfc0 Virtual Interface Control Logic VN 10Gbe Internal Connections UCS 6x00 Physical Ports UCS Fabric Interconnect B (port profiles) Fiber Channel Uplink Ports 1 2 Virtual Interface Control Logic vfc veth1 veth2 veth3 veth4 veth10 7 UCS 6x00 Physical Ports 8 0 Mgmt Uplink VM-FEX Chassis IOM Ports Server Ports Cisco Adapter vhba0 HBA Chassis IO Module A vnic1(s) ESX 4.0u1+ / RHEL KVM 6.1+ / MS Windows 8 Server VIC CPU 1 2 vcenter Controlled interfaces on VMs with forwarding rules enforced on dynamic adapters and signaled on Private Interfaces 0 1 CIMC KVM etc. 3 4 Chassis IOM Ports Chassis IO Module B ESX Kernel Module / Libvirt / HyperV Extendable Switch vnic2(s) Server Ports UCS B or C Series Server vhba1 HBA 1 33

34 UCS VM-FEX System View Deploying on a UCS B or C Series Infrastructure Mgmt Uplink 0 veth10 UCS Fabric Interconnect A (port profiles) veth1 veth2 veth3 veth4 Fiber Channel Uplink Ports vfc0 Virtual Interface Control Logic VN 10Gbe Internal Connections UCS 6x00 Physical Ports UCS Fabric Interconnect B (port profiles) Fiber Channel Uplink Ports 1 2 Virtual Interface Control Logic vfc veth1 veth2 veth3 veth4 veth10 7 UCS 6x00 Physical Ports 8 0 Mgmt Uplink VM-FEX Chassis IOM Ports Server Ports Cisco Adapter vhba0 HBA Chassis IO Module A ESX 4.0u1+ / RHEL KVM 6.1+ / MS Windows 8 Server VIC CPU 1 2 vcenter Controlled interfaces on VMs with forwarding rules enforced on dynamic adapters and signaled on Private Interfaces Chassis IO Module B CIMC vnic1(s) d-vnic1 d-vnic2 KVM etc. d-vnic3 d-vnic4 vnic2(s) ESX Kernel Module / Libvirt / HyperV Extendable Switch Kernel Service Console Chassis IOM Ports Attempts by Guest OS to improperly mark traffic blocked Server Ports UCS B or C Series Server vhba1 HBA 1 34

35 UCS VM-FEX System View Deploying on a UCS C Series with Nexus 5500 Infrastructure Mgmt Uplink 0 VM-FEX veth Fabric Ports 2232 Server Ports vhba0 HBA 0 Nexus 55xx A (port profiles) Fiber Channel Uplink Ports veth1 veth2 veth3 veth4 Cisco P81E Adapter 1 2 vfc FEX A VN 10Gbe Internal Connections vnic1(s) Virtual Interface Control Logic Nexus 55xx Physical Ports vpc Connections (veth s not a vpc at FCS) VIC CPU Virtual Interface Control Logic 1 2 vcenter Controlled interfaces on VMs with forwarding rules enforced on dynamic adapters and signaled on Private Interfaces 0 1 CIMC KVM etc. ESX Kernel Pass Through Module Nexus 55xx B (port profiles) Fiber Channel Uplink Ports 1 2 vfc FEX B veth1 veth2 veth3 veth4 veth Nexus 55xx Physical Ports 2232 Fabric Ports vnic2(s) Server Ports UCS C Series Server 8 vhba1 HBA 1 0 Mgmt Uplink ESX 4.0u1+ 35

36 UCS VM-FEX System View Deploying on a UCS C Series with Nexus 5500 Infrastructure Mgmt Uplink 0 veth10 Nexus 55xx A (port profiles) Fiber Channel Uplink Ports veth1 veth2 veth3 veth4 vfc0 Virtual Interface Control Logic VN 10Gbe Internal Connections Nexus 55xx Physical Ports vpc Connections (veth s not a vpc at FCS) Nexus 55xx B (port profiles) Fiber Channel Uplink Ports 1 2 Virtual Interface Control Logic vfc veth1 veth2 veth3 veth4 veth10 Nexus 55xx Physical Ports Mgmt Uplink VM-FEX 2232 Fabric Ports 2232 Server Ports Cisco P81E Adapter vhba0 HBA 0 ESX 4.0u FEX A vnic1(s) 8 VIC CPU 1 2 vcenter Controlled interfaces on VMs with forwarding rules enforced on dynamic adapters and signaled on Private Interfaces FEX B CIMC d-vnic1 d-vnic2 KVM etc. d-vnic3 d-vnic4 ESX Kernel Pass Through Module Kernel Service Console 36 8 Attempts by Guest OS to improperly mark traffic blocked 2232 Fabric Ports vnic2(s) 2232 Server Ports UCS C Series Server vhba1 HBA 1

37 UCS General Baseline #1: Creating Dynamic vnics Setting a Dynamic Adapter Policy Up Policies are to automatically provision dynamics on Servers Dependent on the number of Fabric Interconnect to IO Module connections - (# IOM to FI links * 63)

38 UCS General Baseline #2: Building Service Profile Adding the Dynamic Policy and Static Adapters 2 Statics 1 to each UCS Fabric Change dynamic vnic connection policy to setup dynamics 38

39 UCS General Baseline #3: Building Port Profiles Creating Folders of Network Access Attributes Creating Port Profiles Includes: - VLAN(s) - Native and/or Tagging allowed - QoS Weights and Flow Rates - Upstream Ports to always use 39

40 UCS General Baseline #4: Building Port Profiles Enhanced Options like VMDirectPath with VM-FEX Selecting High Performance will only Impact VMware deployment today No problem if selected and used on other hypervisors 40

41 UCS General Baseline #5: Communication with Manager Establishing Communication to Hypervisor Manager Same Plug-in Method used in Nexus 1000v Tool discussed later to simplify the whole integration process 8 Separate managers today 41

42 UCS General Baseline #6: Publishing Port Profiles Exporting Port Profiles to these to Hypervisor Manager Publish Port Profiles to Hypervisors and virtual switches within 4 Separate virtual switch per manager today 42

43 UCS C2xx / N5500 General Baseline #1: Dynamic vnics Setting a Group of Dynamics on C2xx Servers Enable NIV (Network Interface Virtualization) on the P81E Adapter in the CIMC - CIMC is the management Interface for the Cisco C2xx servers - Choose the number of dynamics to configure (next slide) 43

44 UCS C2xx / N5500 General Baseline #1: Dynamic vnics Setting a Group of Statics and Dynamics on C2xx Servers Enable vnics then view the VM- FEXs Tab in the CIMC - CIMC is the management Interface for the Cisco C2xx servers - UCS Standalone CIMC version 1.4 or greater required - Minimum of 2 static vnics defined - Numbers of VM FEX s (dynamic vnics) are dependent on links from 5500 to 2232 if using FEX (Limit at 96 today) - Nexus 5500 version 5.1(3)N1(1) or later required 44

45 UCS C2xx / N5500 General Baseline #2: N5k Port Profiles Enabling VMFEX, and Creating Folders of Network Access Attributes nexus5500-1(config)# install feature-set virtualization nexus5500-1(config)# feature-set virtualization nexus5500-1(config)# feature vmfex nexus5500-1(config)# feature npiv nexus5500-1(config)# vethernet auto-create nexus5500-1(config)# interface Ethernet100/1/27 << P81E Port nexus5500-1(config-if)# switchport mode vntag nexus5500-1(config)# port-profile type vethernet VM_VLAN_6 nexus5500-1(config-port-prof)# switchport access vlan 6 nexus5500-1(config-port-prof)# high-performance host-netio nexus5500-1(config-port-prof)# port-binding dynamic nexus5500-1(config-port-prof)# dvs-name all << To publish to the VM-FEX DVS nexus5500-1(config-port-prof)# state enabled 45

46 UCS C2xx / N5500 General Baseline #3: P81E Statics Configuring Adapter FEX interfaces for Startup/Shutdown Operations Configuring a static profile for the fixed interfaces One to each N5k in a pair Select the vnic, and the port profile to assign to it These will be initially in the vswitch in the out of box VMware configuration 46

47 UCS C2xx / N5500 General Baseline #4: vcenter plugin Enabling VMFEX, and Creating Folders of Network Access Attributes nexus5500-1(config)# feature http-server Downloading and registration of plug-in per the other VM- FEX topologies 47

48 UCS C2xx / N5500 General Baseline #5: Comm with Manager Establishing Communication to Hypervisor Manager nexus5500-1(config)# svs connection vcenter1 nexus5500-1(config-svs-conn)# protocol vmware-vim nexus5500-1(config-svs-conn)# extension-key UCSTME-Nexus5000-VMFEX-DVS nexus5500-1(config-svs-conn)# remote ip address port 80 vrf management nexus5500-1(config-svs-conn)# dvs-name UCS_C2XX_VMFEX_DVS_1 nexus5500-1(config-svs-conn)# vmware dvs datacenter-name UCSTMELAB nexus5500-1(config-svs-conn)# connect Do this on Both Nexus 5500 s in the vpc Pair nexus5500-1(config-svs-conn)# show svs connections 48

49 VM-FEX Implementation with VMware on UCS and Nexus 5500

50 VMware VM-FEX: Infrastructure Requirements Versions, Licenses, etc. 8 vcenters can be joined from UCS Each vcenter can have 4 DVS within Plug-In download and install method (unless Easy VM-FEX tool is used) Hosts then use VUM Depot s to install ESX module when bringing host into UCS DVS (unless Easy VM-FEX tool is used) Enterprise+ required (as is for any DVS) on Host Standard and above is required for vcenter VMotion fully supported VMDirectPath with VM-FEX is possible (Hypervisor Bypass) - Cisco UCS with VM-FEX linked to in vsphere 5 networking guide page 42 - vcenter VM->Properties->Resources need to reserve all guest memory 50

51 VMware VM-FEX: vcenter View View from the Administrator Uplinks from ESX hosts shown on right - These are the statics for overhead VM vnics shown in port groups on left - Port Groups are from Port Profiles sent in from UCSM or Nexus 5500 Normal view of VM vnics, MAC, Port numbers, etc. 51

52 VMware VM-FEX: vcenter View View from the VM Settings to get DirectPath with VM-FEX Match the Memory Reservation with the Limit Fundamental requirement of all DirectPath deployments Only Supported Guests will get DP with VM-FEX - Windows Server 2008 SP2, Windows Server 2008 R2, RHEL 6.x, SLES11 SP1 52

53 VMware VM-FEX: vcenter View View from the VM Settings Simply add vnics Define normal VMXNET3 type Select Port Group to put adapter into Displays if the DirectPath with VM- FEX is active Other Adapters can remain in emulated mode if desired 53

54 VMware VM-FEX: vcenter View View from the vcenter 54

55 VMware VM-FEX: Administrator View View from the UCSM 55

56 VMware VM-FEX: Administrator View View from the Nexus 5500 User can see where the VM s map to the Veth ports on the 5500 s Normal NX-OS command set to view/manage/span etc as in physical ports 56

57 VM-FEX Implementation with KVM on UCS

58 RHEL KVM VM-FEX: Infrastructure Requirements Versions, Licenses, etc. VM-FEX is available for KVM on only UCS Managed Deployments Today Install Red Hat as Virtualization Host Unlike VMware no VEM to load (utilizes libvirt) Works with live migration feature for VMs Scripted nature of configuration at FCS - No current RHEV-M for RHEL KVM 6.x - RHEV-M 3.0 will have RHEL 6.2 hooks for VM-FEX configuration assistance MacVTap has 3 distinct modes - Bridge mode for normal end points connecting together within host (Adapter FEX has value here) Qbg VEPA for traffic hair-pinning on next upstream device Qbh private-mode for traffic always passing to controlling bridge (UCS FI) VM-FEX uses private-mode Virtual Machine interface management via editing of VM domain XML file 58

59 RHEL KVM VM-FEX: virt-manager View View from the Administrator RHEL virt-manager to do simple VM operations Can start, stop, open, migrate VM s with VM-FEX connections also RHEV-M will be able to present VM-FEX port profiles natively with 3.x 59

60 RHEL KVM VM-FEX: CLI View View from the Administrator Virsh set of commands to control VM s Create, Start, Stop, Migrate, etc. of VM s with VM-FEX included 60

61 RHEL KVM VM-FEX: VM vnic View View from the Administrator Adapters can be created in virtmanager wizard and MAC assigned Edit the domain.xml file to make the adapters VM-FEX Bring in the port profile here Then VM will operate as normal No reliance on bridge domains 61

62 VMware VM-FEX: Administrator View View from the UCSM Same port profiles can be used to KVM VM s Live Migration fully supported 62

63 VM-FEX Implementation with Hyper-V on UCS

64 HyperV VM-FEX: Infrastructure Requirements Versions, Licenses, etc. VM-FEX is available for HyperV on only UCS Managed Deployments at shipment of Windows 8 HyperV Role Enabled on Windows 8 Servers For Live Migration, MS Cluster built with shared storage - VM-FEX with Live Migration fully supported HyperV Networks defined as shown here - Through HyperV Manager GUI - Through PowerShell Applets Cisco Extension to the HyperV Extensible Switch infrastructure Systems Center Virtual Machine Manager (SCVMM) 2012 version with Windows 8 Server support will be needed for manager integration - Fully supported via PowerShell until the SCVMM ships 64

65 HyperV VM-FEX: VM vnic View Key PowerShell Components Install the VIC device Install the Csco VmFex Forwarding Extension -.\vmfextool.exe -i E:\VmFex\ReleaseDrivers\cscovmfext\cscovmfext.inf" "Csco_FEXT" Create a IOV Switch - New-VMSwitch -Name "Cisco-Network" -NetAdapter $externalnic -SwitchType External -AllowManagementOS 1 -Notes "Cisco Private Network Traffic Switch" - EnableIov 1 - Enable-VMSwitchExtension -VMSwitchExtensionName "Cisco VmFex Forwarding Extension Insert the Cluster UUID and Name to match the UCSM -.\Cisco_SwitchProp-Add.ps1 "Cisco-Network" "38c79463-e5d3-46f6-ba47- b35719f15c70" "Msft-clus" Add an SR-IOV adapter to the VM with the command -.\PortProfileSetting-Add.ps1 "VMName" "PortProfileId" "PortProfileName" "NicID" 65

66 HyperV VM-FEX: Failover Cluster Manager View View from the Administrator 66

67 HyperV VM-FEX: VM vnic View View from the Administrator Virtual Network Manager to configure the switch PS scripts to configure the Cisco extension to the HyperV switch 67

68 HyperV VM-FEX: VM vnic View View from the Administrator VM is created first PS scripts add the network policy from our portprofiles behind Adapters 68

69 VMware VM-FEX: Administrator View View from the UCSM 69

70 Easy-VMFEX tool usage

71 Easy VM-FEX Tool Tool Usage VMware solution only today with UCS Quick System Bringups Assumption of 1 management interface per ESX host - Optional vmotion / FT logging also handled All supported versions of VMware that VM-FEX supports - Enterprise+ or Evaluation Can define some defaults in text file vcenter folders OK Server needs Dynamic vnics on Service Profile (will check) Deployment name limited to 8 characters in tool UCSM respository for ESX kernel model, or separate tool to pull from VMware online to a dedicated directory locally 71

72 Easy VM-FEX Tool Tool View 72

73 VM-FEX Performance Review Test of 10 VM s running HTTP and FTP gets with IxLoad Virtual Switch, CPU at ~65% VM-FEX, CPU at ~ 37% 73

74 VM-FEX Performance Review Test of 4 IxVM s Virtual Switching 88%, VM-FEX 52% 3 IxVM s, sending fixed 3.33G flows to a 4 th VM All on same ESX host to maximize the differences in operations - Virtual switch with VMs on same host flows contained within ESXi 5 host - VM-FEX with VMs on same host flows traverse the chassis IOM/FI, or N2k/5k - Higher Bandwidth AND Lower Latency between VM s 74

75 Summary

76 Summary What we have covered FEX technologies can reduce managed device count FEX technologies will greatly reduce cabling overhead VM-FEX is terminating these virtual links directly on the VMs Closely maps to the physical server model for operations and management Multiple Hypervisors are supported with advanced features Bandwidth can be engineered identically to physical infrastructures today Latency can surpass local virtualized switching by moving away from virtual switching store and forward buffering, tree s of ASIC traversals, to a uniform port controller and switch fabric model 76

77 Recommended Reading BRKCOM- 2005

78 Please complete your Session Survey We value your feedback Don't forget to complete your online session evaluations after each session. Complete 4 session evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt Surveys can be found on the Attendee Website at which can also be accessed through the screens at the Communication Stations Or use the Cisco Live Mobile App to complete the surveys from your phone, download the app at 1. Scan the QR code (Go to for QR code reader software, alternatively type in the access URL above) 2. Download the app or access the mobile site 3. Log in to complete and submit the evaluations 78

79 79

80 Thank you. 80