with ACI Any workload anywhere.

Transcription

1

2 Cisco IT: Scalable Enterprise UCS with ACI Any workload anywhere. Hugh Flanagan, Senior IT Engineer Jason Stevens, IT Engineer BRKCOC-0

3 Agenda Introduction Challenges of Large Scale UCS Deployments in the Datacenter How ACI and UCS Central Allow us to Enhance UCS Infrastructure Operational Gains Conclusion

4 Introduction

5 Impact of ACI on Cisco Datacenters BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

6 Cisco IT s Internal Cloud Journey Reduce Cost Deliver Services Faster Virtualization Home Grown Automation Private Cloud (Iaas + PaaS) Application Centric Cloud Hybrid Cloud We Are HERE UCS APIC VM VM VM ESX VMware Virtualization Cisco Intelligent Automation for Cloud (CIAC) Virtual Security Gateway Prime Network Services Controller Application Centric Infrastructure Cisco Intercloud Open Source BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

7 Challenges of Large Scale UCS Deployments in the Datacenter

8 Cisco IT s Legacy Design Up to Chassis per UCS Domain Fault / Migration Domain Limited to Single UCS Domain Hardware Maintenance Set Aside = % BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

9 Physical Connectivity BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

10 Datacenter Physical Layout BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

11 Legacy UCS Build Work Flow BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

12 Legacy UCS Build Work Flow Continued BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

13 Domain Upgrade Maintenance Summary Cisco IT has 0 UCS Domains Upgrades Complicated by: Change Freeze Windows Maintenance Windows Critical Client Workloads This Results in UCS Upgrades Taking up to Years to Complete BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

14 How ACI and UCS Central Allow us to Enhance UCS Infrastructure

15 End to End Programmable Infrastructure Resources Controllers Resource Managers Orchestration (Cloud) Client Application Code PaaS Compute CIAC Portal ASA Network Integrated Security eaclm Security Admin IP File/ IP Block/ IP Object Storage Block Storage Compute Network/Storage Admins BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

16 Fabric Infrastructure Requirements Flexible Topology Multi-Tenant Virtual Boundaries Physical and Virtual Services Easier to Manage Components ACI: 0, PX, APIC UCS: Bm, B0m, B0m ESXi + AVS Openstack + KVM + OVS NetApp CDoT (.) Citrix SLB ASA CIAC (estore & automation) BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

17 Next Generation Virtual Compute Design vcenter Domain- vcenter Domain- vcenter Domain- vcenter `Domain- Logical Pod- Logical Pod- Logical Pod- NKV Domain- NKV Domain- NKV Domain- NKV Domain- Logical Pod- Physical Leaf pair- Physical Leaf pair- Physical Leaf pair- Physical Leaf pair- Legacy Virtual Compute Design Virtual Compute Design with ACI vmotion vmotion vmotion vmotion vmotion Across VMM Domain BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

18 Striped Cluster Architecture Spine Switches Leaf Switches UCS FI Hypervisor Stripe Bare Metal Stripe BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

19 Challenges with Next Generation Design Increased Capacity. Terahertz of CPU Capacity (not including hyper-threading) 0 Terabytes of Memory Per UCS Domain (0 Blade UCS Domain) VLANs are not Able to Support Number of Possible Tenants Keeping UCS Domains Configuration in Sync BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

20 Cisco UCS Central BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

21 UCS Central Architecture XML API Web Client SNMP UCS Central Virtual Appliance DME Services Framework PostgreSQL DB UCS Manager # UCS Manager #n Hardened Guest OS BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

22 Service Profile Overview RTP ALLN BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

23 UCS Central Org. Structure Service Profiles UCS Domains BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

24 UCS Central Domain Stripe BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

25 VXLAN Extends Layer networks across Layer infrastructure MAC-to-UDP encapsulation VXLAN Tunnel Endpoint (VTEP) 0-byte overhead to original frame Can support up to million segments Cisco IT is using AVS (Application Virtual Switch) for VXLAN connectivity VXLAN traffic uses the Infrastructure VLAN configured during initial fabric turn-up BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

26 VXLAN Packet Format BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

27 UCS Central Ethernet Adapter Policy BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

28 Client VMM (VxLAN) BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

29 Management/Network Services VMM (VLAN) BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

30 OpFlex: An Open Source Framework OpFlex changes have been upstreamed as part of OpenDaylight. OpFlex library and MODB are reusable across any device Rendering agent must be rewritten to specific device OpFlex Agent OpFlex (Policy Manager) Managed Objects Store (MODB) OVS Render Plugin (Policy Enforcement) Open vswitch changes are all upstream and included in OVS. Open vswitch OVSDB OpenFlow Flow Table Datapath and Linux kernel (.) include support for VXLAN-GBP Datapath SW/HW Datapath BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

31 OpFlex Hypervisor Switch Integration OpFlex Proxy runs as part of ACI leaf managing interaction with APIC OpFlex Agent runs on hypervisor in userspace and will be packaged as a separate component OpFlex Proxy OpFlex Proxy OpFlex Proxy OpFlex Proxy OpFlex Proxy OpFlex Proxy It speaks to OVS through OpenFlow / OSDB interfaces Available as part of FCS+ (BMR release of ACI) OpFlex Agent OPEN VSWITCH OpFlex Agent CISCO AVS OpFlex Agent MICROSOFT HYPER-V OpFlex Agent THIRD PARTY VSWITCH OpFlex Agent OPEN VSWITCH OpFlex Agent CISCO AVS BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

32 ID Cisco Nexus T GE Fabric Extender STAT ID Cisco Nexus T GE Fabric Extender STAT ID Cisco Nexus T GE Fabric Extender STAT ID Cisco Nexus T GE Fabric Extender STAT Fabric Design Bare Metal Border Storage Virtual Compute DMZ Core Internal Core Clustered Data on Tap ESXi Openstack BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

33 Operational Gains

34 Hardware Specifications Legacy Deployment - B0 M X0 CPU x ( Cores per CPU) GB RAM VIC CNA Modern Deployment - B0 M E- v CPU x ( Cores per CPU) GB RAM VIC 0 CNA (Supports VXLAN Offload) BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

35 Blade Consolidation Legacy DC Environment has,00 Hypervisor Blades,00 of which are B0 M ACI, Leveraging B0 M, Consolidates this to 00 Blades : Consolidation on Average, Achieved Through Higher Density Blades and Larger Migration Domains : Consolidation Seen when Consolidating B0 M Environments BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

36 Domain Upgrade Maintenance Time Expended for Legacy hours Determining Maintenance Windows hours negotiating with clients minutes running health checks and verifications hour fixing issues to ensure no downtime. hours executing upgrade Time Expended for ACI minutes evacuating. hours executing upgrade BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

37 Datacenter Physical Layout Revisited BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

38 Capacity Additions Under ACI Each UCS Domain is built with a minimal amount of chassis and blades for Day workloads Chassis additions are performed when additional capacity is needed for quick turnaround Automation built leveraging CIAC, UCS Central, and APICs configures the blades for use BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

39 UCS Domain Evacuation Demo

40 BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

41 Stretched Fabric Use Case Single DC Facility Data Hall Data Hall Transit Leaf Switches APIC Cluster APIC APIC APIC BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

42 Best Practices / Lessons Learned

43 Cisco and/or its affiliates. All rights reserved. Cisco Public BRKCOC-0 Hitless Upgrades and Failover Testing All failovers and upgrades: max - sec connectivity loss ACI Switch Application Virtual Switch UCS Compute (B-Series) ESXi/Vmware multiple IP Storage Leaf Pairs Border Leafs Leaf Pairs Multiple Compute & Services Leaf Pairs NAS DMZ Internal Services (physical or virtual) IP Network ACI Fabric Services (physical or virtual) Compute Compute Internet Cisco+Corp APIC APIC APIC vcenter NAS

44 Best Practices and Lessons Learned Test/Certify new Features and Code Prior to Production Deployment Utilize lab environment for testing Create certification process with standard must have capabilities and verification Document/track issues found Start off with basic functionality and add new features as you go Build with Automation in Mind Create standard and reusable constructs Document naming conventions for various objects to make readability and troubleshooting easier Scripting skills will help you on your journey BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

45 Best Practices and Lessons Learned AVS for vswitch Use AVS and VXLAN where possible L-L services required to be on VLAN VMM Reduced configuration overhead (No need to trunk down huge amounts of VLANs) Better management and visibility when troubleshooting connectivity issues BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

46 Best Practices and Lessons Learned Upgrades Test upgrade process in lab prior to production rollout and check release notes for any important changes Maintenance Groups Border Leaf Even / Border Leaf Odd Separate Compute Leaf Pairs as Capacity Allows (Still into Evens / Odds) Config backups Schedule secure configuration backups/archives on daily basis BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

47 Conclusion

48 Cisco and/or its affiliates. All rights reserved. Cisco Public Fabric Pod Pod Pod Pod Resource Stripe - ACI Greater Resiliency Easier Maintenance Resource pools and applications striped across multiple pods BRKCOC-0

49 Flexible Assets Storage or Compute Unit Unit Unit Vertical Lock-in Initial placement of applications results in storage and compute resources physically locked in Reserve (future & failover) capacity can t be flexibly used Stranded Storage and Compute Capacity Today Horizontal Scaling Applications can flexibly tap into any available compute or storage resources ACI Unit Unit Unit Initially: ACI + VMware + NetApp CDOT ACI Policy Based Control Infrastructure Maintenance BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

50 Any Application, Anywhere ACI allows any EPG to be leveraged on any compute resource within fabric Using VXLAN allows any compute resource to be immediately capable of running any VM with no further network configuration Stretched Fabric allows for logically combining multiple datacenter halls into a single fabric. UCS Central allows massively scaling compute capacity while keeping policies and configuration consistent throughout BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public 0

51 Additional Reading Cisco IT ACI Storage Deployment /Cisco_IT_ACI_Storage_Deployment.html Cisco IT ACI Design /Cisco_IT_ACI_Design.html Application Virtual Switch AVS vs DVS Convergence BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

52 Additional Reading Cisco APIC Controller with Openstack Integration Demo UCS Central Demo VXLAN Whitepaper Design Guide for L-L Services on AVS nce-guide/l-l-services/cisco_avs_l-l_white_paper.html BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

53 Complete Your Online Session Evaluation Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $0 Amazon gift card. Complete your session surveys through the Cisco Live mobile app or from the Session Catalog on CiscoLive.com/us. Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

54 Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Lunch & Learn Meet the Engineer : meetings Related sessions BRKCOC-0 Cisco and/or its affiliates. All rights reserved. Cisco Public

55 Please join us for the Service Provider Innovation Talk featuring: Yvette Kanouff Senior Vice President and General Manager, SP Business Joe Cozzolino Senior Vice President, Cisco Services Thursday, July th, : am - : pm, In the Oceanside A room What to expect from this innovation talk Insights on market trends and forecasts Preview of key technologies and capabilities Innovative demonstrations of the latest and greatest products Better understanding of how Cisco can help you succeed Register to attend the session live now or watch the broadcast on cisco.com

56 Thank you

57