Integrating Cisco UCS with Cisco ACI Marian Klas, mklas@cisco.com Systems Engineer Data Center February 2015
Agenda: Connecting workloads to ACI Bare Metal Hypervisors UCS & APIC Integration and Orchestration Converged Stack Examples 2
Connecting workloads to ACI: Bare Metal Servers 3
Connecting Physical Workloads to a Port Infrastructure configuration (Access Policy) Switch Profile Interface Profile Policy Group = port configuration Switch 101 Switch 102 Etc Interface 1/1 Interface 1/2 Etc Aggregates port configurations It can be of type Interface, Portchannel, vpc etc 4 4
Enabling VLANs on a set of ports What is it? It s just a way to tell the fabric which VLAN ranges are expected where, it s useful for VLAN reuse Attach Entity Profile Aggregates Domains and has a reference with the policy-group Physical Domain Can be made of multiple VLAN pools VLAN Pool Defines a range of VLANs 5 5
Mapping between Tenant View and Infrastructure view Tenant View Infrastructure View EPG maps to port, VLAN Port (i.e. portprofile, switch profile) Policy Group = port configuration Attach Entity Profile VLAN must be part of AEP 6
How to Connect Servers Tenant leaf portchannel virtual machine virtual machine EPG, bridge domain, router 8
Mapping an EPG to a Server Port 9
Endpoints are discovered Under Operational Client Endpoints 11
EPG mapped to multiple VLANs on different leafs Spines Leafs Border Leafs vlan30 vlan10 vlan20 vlan40 12
ACI Fabric Endpoint Connectivity Rack Mount Servers Policy Enforcement 9300 9300 1/10G 40G 10G 9300 9300 N2K N2K No Host vpc 9396PX/TX or 93128TX Leaf Direct Connect, FEX Supported 9300 up to 6 x FEX (as of Jan 2015) 9300 Leaf w/ FEX Flexible teaming options including: Active/Active w/ LACP; Active/Active with AVS and FEX Enhanced vpc and vpc w/ FEX not Supported (as of Jan 2015) Active/Standby 13 13
FEX Topology Support Roadmap Active/Standby Teaming Straight Through (Single Homed) vpc (Dual Homed) EvPC Nexus 9300 Standalone 6.1(2)I2(3) 6.1(2)I2(3) Future Future Nexus 9300 ACI Leaf 11.1(x) - 1HCY15 11.0(1d) - Shipping Future Future 14
FEX Support Nexus 9300 FEX Support 6.1(2)I2(3) N2224TP, N2248TP, N2248TP-E, N2232TM, N2232PP, B22HP 6.1(2)I3(1) B22-Dell, 2232TM-E, 2248PQ FEX Q1 CY15 B22-IBM, B22-Fujitsu, 2348UPQ 11.0(1d) - Shipping N2K-C2248PQ, N2K-C2248TP-E, N2K-C2248TP-1GE, N2K- C2232PP-10GE, N2K-C2232TM-E 11.1(x) - Q1CY15 B22HP, B22-DEL, B22-IBM Q2CY15 ACI FEX Support 2348UPQ 15
ACI Fabric Endpoint Connectivity 3 rd Party Blade Servers & Ethernet northbound connectivity Policy Enforcement L2 Switching ACI Fabric Virtual Switching N9K N9K N9K N9K N9K N9K N9K N9K N2K N2K PT PT B22 B22 HPVC HPVC SW SW No Host vpc Passthrough Blade Servers HP, IBM, Dell. (Q1CY15 MR2 release) 3 rd Party Switches STP Interoperability Leaf Port Type Network/Edge Edge Port BPDU Guard Network Port BPDU Pass-Through No Host vpc B22-HP B22-Dell B22-IBM Flex-10 FlexFabric Blade Switches from: Cisco HP Force-10 IBM/BNT 17 17
Connecting workloads to ACI: Hypervisor Integration 18
Hypervisors Integration General concepts Integration with Vmware Cisco Application Virtual Switch (AVS) VMware DVS Integration with Hyper-V Integration with KVM / Openstack 19 19
Hypervisor Interaction with ACI Two modes of Operation Non-Integrated Mode Integrated Mode VLAN 10 VLAN 10 VXLAN 10000 APP WEB DB DB ACI Fabric as an IP-Ethernet Transport Encapsulations manually allocated Separate Policy domains for Physical and Virtual ACI Fabric as a Policy Authority Encapsulations Normalized and dynamically provisioned Integrated Policy domains across Physical and Virtual 20 20
Hypervisor Integration with ACI Control Channel - VMM Domains Relationship is formed between APIC and Virtual Machine Manager (VMM) Multiple VMMs likely on a single ACI Fabric Each VMM and associated Virtual hosts are grouped within APIC vcenter DVS vcenter AVS SCVMM Called VMM Domain There is 1:1 relationship between a Virtual Switch and VMM Domain VMM Domain 1 VMM Domain 2 VMM Domain 3 21 21
Hypervisor Integration with ACI F/W EPG WEB APIC Application Network Profile L/B EPG APP WEB PORT GROUP APP PORT GROUP DB PORT GROUP VM VM VM EPG DB ACI Fabric implements policy on Virtual Networks by mapping Endpoints to EPGs Endpoints in a Virtualized environment are represented as the vnics VMM applies network configuration by placement of vnics into: Port Groups (VMWare), VM Networks (Hyper-V) Networks (OpenStack) EPGs are exposed to the VMM as a 1:1 mapping to Port Groups, VM Networks or OpenStack Networking. 22 22
Hypervisor Integration with ACI Endpoint Discovery Virtual Endpoints are discovered for reachability & policy purposes via 2 methods: APIC Control Plane Learning: - Out-of-Band Handshake: vcenter APIs - Inband Handshake: OpFlex-enabled Host (AVS, Hyper-V, etc.) Data Path Learning: Distributed switch learning Control (vcenter API) LLDP/CDP used to resolve Virtual host ID to attached port on leaf node (non-opflex Hosts) Control (OpFlex) Data Path Data Path VMM DVS Host OpFlex Host 26 26
Hypervisors Integration General concepts Integration with Vmware Cisco Application Virtual Switch (AVS) VMware DVS Integration with Hyper-V Integration with KVM / Openstack 27 27
VMWare Integration Three Different Options Distributed Virtual Switch (DVS) vcenter + vshield Application Virtual Switch (AVS) + Encapsulations: VLAN Installation: Native VM discovery: LLDP/ CDP Software/Licenses: vcenter with EnterprisePlus License Encapsulations: VLAN, VXLAN Installation: Native VM discovery: LLDP/ CDP Software/Licenses: vcenter with EnterprisePlus License, vshield Manager with vshield License Encapsulations: VLAN, VXLAN Installation: VIB through VUM or Console VM discovery: OpFlex Software/Licenses: vcenter with EnterprisePlus License 28 28
ACI Hypervisor Integration VMware DVS/vShield APIC 5 Create Application Policy F/W Application Network Profile EPG WEB L/B EPG APP EPG DB APIC Admin 9 ACI Fabric Push Policy 1 Cisco APIC and VMware vcenter Initial Handshake 6 Automatically Map EPG To Port Groups 4 Learn location of ESX Host through LLDP 2 Create VDS VIRTUAL DISTRIBUTED SWITCH WEB PORT GROUP APP PORT GROUP DB PORT GROUP VI/Server Admin vcenter Server / vshield 8 Instantiate VMs, Assign to Port Groups 7 3 Create Port Groups Attach Hypervisor to VDS Web App HYPERVISOR DB Web Web HYPERVISOR DB 29 29
ACI Hypervisor Integration VMware DVS Name of VMM Domain Type of vswitch (DVS or AVS) Associated Attachable Entity Profile (AEP) VLAN Pool vcenter Administrator Credentials vcenter server information 30 30
ACI Hypervisor Integration VMware DVS 31 31
ACI Hypervisor Integration AVS APIC 5 Create Application Policy F/W Application Network Profile EPG WEB L/B EPG APP EPG DB APIC Admin 9 ACI Fabric Push Policy 1 Cisco APIC and VMware vcenter Initial Handshake 6 Automatically Map EPG To Port Groups 4 Learn location of ESX Host through OpFlex OpFlex Agent OpFlex Agent VI/Server Admin vcenter Server 8 Instantiate VMs, Assign to Port Groups 2 7 3 Create AVS VDS Create Port Groups Attach Hypervisor to VDS Application Virtual Switch (AVS) WEB PORT GROUP APP PORT GROUP DB PORT GROUP Web App HYPERVISOR DB Web Web HYPERVISOR DB 33 33
Extending ACI to Existing Virtual & Physical Network AVS supports OpFlex to integrate with APIC Supports a Full multi-hop Layer 2 Network between Nexus 9k and AVS: Investment Protection Phase 1: Layer 2 Existing Network/Local Switching VMware DVS can only support a single L2 switch between N9k and DVS LLDP/CDP and NOT OpFlex Integration OpFlex OpFlex AVS OpFlex AVS Layer 2 network is required to support OpFlex bootstrapping in this phase AVS Layer 2 Network 34
Hypervisors Integration General concepts Integration with Vmware Cisco Application Virtual Switch (AVS) VMware DVS Integration with Hyper-V Integration with KVM / Openstack 36 36
Microsoft Interaction with ACI Two modes of Operation Integration with SCVMM Integration with Azure Pack APIC APIC + Policy Management: Through APIC Software / License: Windows Server with HyperV, SCVMM VM Discovery: OpFlex Encapsulations: VLAN, NVGRE (Future) Plugin Installation: Manual Superset of SCVMM Policy Management: Through APIC or through Azure Pack Software / License: Windows Server with HyperV, SCVMM, Azure Pack (free) VM Discovery: OpFlex Encapsulations: VLAN, NVGRE (Future) Plugin Installation: Integrated 37 37
Microsoft Azure Pack Integration Integration with Microsoft requires: - Windows Server 2012 - Systems Center 2012 R2 with SPF - Windows Azure Pack Azure Pack provides single pane of glass for Definition, creation, management of their cloud service Divided into Provider (Admin) portal and Consumer Self-Service (Tenant) portal Cisco ACI Service Plugin enables management of Network Infrastructure through APIC REST API Service Plans Users Web Sites Service Provider Provider Portal VMs SQL Web Sites Apps Database VMs ACI Service Bus Customer Consumer Self-Service Portal R2 w/ Service Provider Foundation 39 39
ACI Azure Pack Integration 1 APIC APIC Admin (Basic Infrastructure) 7 ACI Fabric 3 Get VLANs allocated for each EPG 2 Push Network Profiles to APIC Pull Policy on leaf where EP attaches Create Application Policy 1 Create VM Networks 4 4 5 Instantiate VMs 6 Indicate EP Attach to attached leaf when VM starts APIC Plugin SCVMM Plugin OpFlex Agent OpFlex Agent OpFlex Agent HYPERVISOR HYPERVISOR HYPERVISOR Azure Pack \ SPF Azure Pack Tenant Web App Web App DB Web Web DB 41 41
Hypervisors Integration General concepts Integration with Vmware Cisco Application Virtual Switch (AVS) VMware DVS Integration with Hyper-V Integration with KVM / Openstack 42 42
ACI OpenStack Integration Phase 1 APIC 3 Create Application Policy APIC Admin (Performs Steps 3) 5 ACI Fabric Push Policy 2 Automatically Push Network Profiles to APIC Create Network, Subnet, Security Groups, Policy 1 NETWORK ROUTING SECURITY OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH NEUTRON NOVA 4 Web App Web App DB Web Web DB OpenStack Tenant (Performs Steps 1,4) Instantiate VMs HYPERVISOR HYPERVISOR HYPERVISOR 46 46
ACI OpenStack Integration Phase 2 (Group-based policy) Create Application Network Profile 1 F/W L/B Application Network Profile EPG WEB L/B EPG APP EPG DB NEUTRON NOVA 4 Web App Web App DB Web Web DB OpenStack Tenant (Performs step 1,4) Instantiate VMs HYPERVISOR HYPERVISOR HYPERVISOR 2 Automatically Push Network Profiles to APIC APIC 3 Create Application Policy F/W L/B Application Network Profile EPG WEB L/B EPG APP EPG DB ACI Admin (manages physical network, monitors tenant state) 5 ACI Fabric Push Policy 47
UCS & APIC Integration 48
ACI Management Overview with UCS End-Host Mode UCS Director SAN A SAN B UCSM / UCS Central Ethernet FC Ethernet and FCOE FCOE 51 51
ACI Management Overview with UCS End-Host Mode UCS Director SAN A SAN B UCS Stand-alone EPG UCS Director UCSM/ UCS Central SP Binding - VLANs UCSM / UCS Central Ethernet FC Ethernet and FCOE FCOE 52 52
UCSD Application Provisioning Lifecycle UCS Director Application Catalog Create VMs Assign to EPGs Assign Baremetal Servers to EPGs VM VM VM VM VM VM Web Create Additional Storage for DB Tier Create SharePoint App Policy in APIC VM L4-7 App L4-7 DB VM VM Storage Manager A B C Virtualization Fabric Compute Baremetal Server Load Balancer L4-7 Services Application Fully Instantiated Storage 57
Common Policy Based Infrastructure Programmability Flexibility, Performance, and Visibility Consistent Virtualized and Physical Server Deployment Models Multi-Phase Approach All phases show a Cisco together value add First 2 on existing UCS deployments Later one on 3 rd Generation UCS FI Phase 1 (2014): UCS Director to orchestrate workloads over interlinked UCS and ACI Phase 2 (2015): UCS and ACI sharing policy and state for a better together story Phase 3 (Planning): 3 rd Gen UCS Fabric Interconnect is a leaf with expanded policy and state interaction Automated Configuration via UCS Director Policy and State Exchange via Software Agent Integrated Management as Native ACI Leaf UCS FI s 58
Phases and Tradeoffs UCSM Managed Servers Feature Phase 1 Phase 2a Phase 2b Phase 3 UCS Fabric Interconnects UCS to ACI Leaf vpc Pair Ratio 6100/6200 6100/6200 6300 6300 N:1 (10GE) N:1 (10GE) N:1 (40GE) 1:1 (Can be a Leaf) Port Types/Speeds 10GE SFP, 8GFC SFP 10GE SFP, 8GFC SFP 40GE QSFP, 16GFC SFP 40GE SFP, 16GFC SFP UCS Manager Location Within FI Within FI Within FI Decoupled and containerized Policy Integration Agent UCS Director Over the Top Bridge OpFlex Control Path OpFlex Control Path between DME s UCS Physical Attachment Connected to ACI Leaf Connected to ACI Leaf Connected to ACI Leaf Is ACI Leaf Connected to ACI Spine Health Scoring To ACI Leaf Intra UCS fabric data feed to Observer End Point Group Mechanisms Construction of EPG to VLAN within UCSD Auto-Population of EPG to VLAN data Inside OpFlex Path Auto-Population of EPG to VLAN data Direct from UCS FI ASIC and OS EPG assignment and encapsulation per UCS server interface 61
Converged stack examples 62
ACI-ready Vblock and FlexPOD http://newsroom.cisco.com/press-release-content? type=webcontent&articleid=1421361 63 63
FlexPod with ACI - Cisco and Partner Technologies Key Cisco and Partner Technologies Unified Compute System Blade and Rack mount Servers Nexus 9000 Top of Rack and Modular switches Application Policy Infrastructure Controller APIC ACI Fabric / Nexus 9000 spine / leaf Key Components of FlexPod with ACI design Nexus 9500/9300 Spine & Leaf Switches 3 node Cisco Application Policy Infrastructure Controller (APIC) UCS Manager 2.2 release NetApp Clustered Data ONTAP utilizing storage virtual machines vsphere 5.5 update 2 L4-L7 Services, ASA Firewall and F5 Load balancer VMware Hypervisor and vcenter management Hypervisor Managers NetApp FAS 8000 and CDOT 8.2.1 Firewall and Load Balancer Services Load Balancer OnCommand System Manager NetApp Snap Manager Hypervisor NetApp Clustered ONTAP Integrated L4-L7 Services Cisco Unified Computing System 64
FlexPod Data Center with ACI FlexPod Data Center pre-validated Integration with ACI Configuration management using GUI in the current release UCS Director to be incorporated in upcoming designs Application Validation Microsoft Exchange 2013 Microsoft SharePoint 2013 Microsoft SQL Server 2012 SP1 Additional Applications to be validated in future 65
Thank you.