Cisco Virtual Topology System Release 2.6.2 Service Provider Data Center Cisco Knowledge Network Phil Lowden (plowden) October 9, 2018
Cisco VTS is a standards-based, open software-overlay management and provisioning system. It automates data center network fabric provisioning for virtual and physical infrastructure
Cisco Data Centre Networking Strategy: Providing Choice in Automation and Programmability Application Centric Infrastructure Programmable Fabric Programmable Network Connection Creation Reporting Expansion Fault Mgmt VTS DB DB Web Web App Web App Turnkey integrated solution with security, centralized management, compliance and scale Automated application centric-policy model with embedded security Broad and deep ecosystem VxLAN-BGP EVPN standard-based 3 rd party controller support Cisco Controller for software overlay provisioning and management across N2K-N9K Modern NX-OS with enhanced NX- APIs DevOps toolset used for Network Management (Puppet, Chef, Ansible etc.) Nexus 9400 & 9600 (line cards), 9200, 3100, 3200 Nexus 9700EX + 9300EX 3
Cisco Virtual Topology System (VTS) Open standards based Overlay Provisioning and Management System Automates Overlay provisioning across Cisco Datacenter Top of Rack Nexus switches (Nexus 2000- Nexus 9000), Virtual Switches & DCI routers Automates fabric provisioning for both virtual and bare metal workloads. Cisco Network Services Orchestrator VMware vcenter GUI Custom Orchestrator Openstack vcenter VTS GUI Container Virtual Topology System Service and Infrastructure Policy Resource Management Device Management Inventory Database REST API Policy Plane REST API Cisco Virtual Topology System Service Routing Route Reflector IOS XRv Control Plane Control Plane Federation MP-BGP Programmable using North Bound REST APIs Tighter Integration with Orchestration systems such as Openstack, vcenter and Cisco NSO Cisco Nexus 2000, 3000, 5000, and 7000 Series Cisco Nexus 9000 Series Cisco ASR 9000 Series Virtual Compute Environment Cisco Nexus 2000, 3000, Cisco Nexus 9000 Series Cisco ASR 9000 Series 5000, and 7000 Series VTF DVS Virtual Compute Environment VTF YANG CLI NX-API BGP-EVPN NETCONF/YANG CLI NXAPI SNMP REST API DVS Bare Metal VM OS VM OS Virtualized Automated DCI / WAN Simplified Management for Ease of Operations
Network Virtualization & Overlays Network virtualization: ability to separate, abstract and decouple the physical topology from a logical or virtual topology. This logical topology is called overlay networks Simplified workload provisioning Multi-tenancy at scale Flexible workload placement/mobility NX-API, CLI, YANG 5 SDN Based Overlays introduces agility and automation to Network Orchestration
MPBGP-EVPN &VXLAN based Overlays Overlay Forwarding Table EVPN T1,S1 T1,S2 T2,S3 MAC, IP Address MAC, IP Address MAC, IP Address P1/2 VTEP2 VTEP3 Layer-2 MAC and Layer-3 IP information distribution by Control- Plane (BGP) VXLAN T2,S4 MAC, IP Address VTEP4 Built in multi-tenancy (at scale) Integrated Routing/Bridging (IRB) for Optimized Forwarding Minimize flooding through ARP suppression Fast convergence upon network failures and host movements Security through VTEP peer-authentication IP routing proven, stable, scalable ECMP utilize all available network paths Flexible placement of multitenant segments Better utilization of network paths Scalable network domain (16M VNI vs. 4K VLANs) BGP-EVPN/VXLAN based overlays provides flexibility, manageability, isolation, multi-tenancy, scalability & convergence6
VTS Architecture Multiple workload types and multiple orchestration systems Border Leaf & DCI (Integrated or Separated) DC POD Custom Orchestrator Cisco VTS DC Fabric (OSPF or BGP as the Underlay Protocol) SPINE vcenter VTEP VTEP VTEP VTEP LEAF VTS GUI OVS/DVS Tenant VM Tenant VM Service VM Host Tenant Bare Metal Workload or Service Host VTF Appliance VM Tenant VM Tenant VM Host VTEP VTF Appliance Container Tenant Container Tenant Container Host VTEP For containers, integrated with Cisco Container Networking Container Virtual Machines Physical Appliance or Bare Metal VM with SW Overlay Containers VTS offers a single overlay networking solution for any type of data center workload enabling customers streamline their operational workflows
VTS Flexible Network Overlays Hardware-Based Overlays Software (VTF) Based Overlays VTS Hybrid Overlays Hardware VTEP (TOR Leaf Switch) Software VTEP (Virtual Topology Forwarder / Fd.io) VTS provides architectural and infrastructure independence through a multi-vendor, multi-hypervisor, SW and HW overlay solution
VTS Integrate VXLAN with WAN IP/MPLS WAN For Disaster recovery, High Availability Integrate EVPN/VXLAN to MPLS-L3VPN
Cisco VTS Operational models VTS GUI based VMM Initiated New Multi VMM vcenter VTS vcenter VTS VTS Network and Compute groups work in Silos Port-group and vlan information are exchanged offline as the VMs are attached. VTS Plugin in VMM initiated workflow. Network objects creation is initiated in VMM Degree of Automation The Network segments are shared across VMMs Network objects can be created at VMMs or at VTS
VTS Use Cases Multi-Tenant Data Centers Network-Function Virtualization
Customer Proof Points Tier 1 US Service Provider Workload Agnostic Overlay Support for both VM and Bare Metal Workloads Versatile Support for Multiple VMMs (openstack and VMWare) Dual Stack Enabled Tenancy based on IPv4/IPv6(dual stack) capable overlay networks Custom Service Integrated Redirect select traffic to the services connected to the Border Leaf
Customer Proof Points Middle East Service Provider Multi-Tenant Support Colocation of Tenants in Common Environment Services Internet and VPN as Service Offerings Firewall and Load Balancing within the Fabric Resilient Ability to Connect the Same Customers across Multiple Data Centers Support Bare Metal Bare Metal attach to Fabric 13
Customer Proof Point NFV deployment at Service Provider in Asia Orchestration & Controllers Layer Admin Tools Portal OSS/BSS REST API NSO NSO: Network Service Orchestrator VTS: Virtual Topology System VTS: Virtual Topology System ToR: Top of Rack switch PNF: Physical Network Function VNF: Virtual Network Function dvs: distributed Virtual Switch PE: Provider Edge ESC vcenter VTS Plug-in VTS Virtual Overlay Networking Layer BGP-EVPN MPLS VPN Network VXLAN VPN PE & VXLAN Gateway Nexus 9300 (ToR) Nexus 9300 (ToR) VLANs VLANs VLANs VLANs Virtual Infrastructure, VNF & PNF Layer PNF1 dvs dvs PNF2 VNF1 VNF2 VNF1 VNF2 14
Cisco NFV Integration with VTS 15
Cisco VTS comprise of the following: Virtual Topology Controller (VTC) VTS Service Routing (VTSR) with XRv9000 Virtual Topology Forwarder (VTF)* VTC and VTSR are typically installed on Controller Node(s) *Only if virtual vtep is required - VM mode on vcenter, vhost user mode on OpenStack/KVM.
Virtual Topology Controller (VTC) Also known as VTS policy plane. VTC is a specialized application of NSO ++ WebUI VTS specific YANG models and Fastmap Resource Pool Manager VTC / VTS Policy Plane Embedded Tail-f NSO NEDs
VTS Service Routing (VTSR) Also known as VTS control plane. (Optional) BGP EPVN route reflector using XRv9000 Centralized control plane for the VTFs For L3 HA Deployments Embedded IOS XRv9000 VTF Driver (DL) VTF VTF VTF VTF VTF VTSR
Virtual Topology Forwarder (VTF) Provides virtual VTEP data-plane as part of Cisco VTS solution Runs as a self-contained virtual-machine / process on compute servers Programmed by VTSR (DL) Leverages on Intel DPDK and VPP Technology. Full multi-tenancy support Supports trunk, VLAN, DHCP relay, etc. Supports Ingress & Multicast replication Supports L2 mode (replaces OVS on compute hosts for higher performance)
What is VPP? VTF is based on Vector Packet Processor (VPP) technology Open source project: FD.io (Feb 2016) VPP is a user-mode packet processing stack for commodity hardware - High performance: ~10 MPPS on a single core, > 40 gbit per system - Same bits run on physical hosts, in VMs, or in Linux containers Control-plane / orchestration-plane via standards-based APIs - Integrated w/ tail-f confd High speed patch panel for co-located VMs, outperforms OVS Layer 2 and Layer 3 functionality, multiple kinds of tunneling Leverages best-of-breed open source technology: Intel DPDK Extensible by use of plug-ins Cisco US Patent 7,931,636 (filed 2004)
VPP vs OVS Performance Benchmarking
VTS 2.6.2 New Features
VTS 2.6.2 New Features MPLS/SR Support Multi-hop upgrade RedHat OSP 13 and OSP Director support Device objects enhancement Multi-site support Route-Reflector Functional Group Host Inventory optimization LDAP authentication support Port scoped static routes w/bfd on VTF Data-Plane Learning in L3 VTF
MPLS/SR Support
SDN-Enabled Network as a Fabric for Service Creation End-to-End Service Provisioning Access Network Domain: Cloud Scale Networking Central Office Access VTS (Service Provisioning) SDN Network Domain XTC + WAE (Centralized PCEP) Aggregation VNF VNF Compute PE LSR EVPN L2/L3 Segment Routing VNF VNF Centralized Services Delivery VTS provides service provisioning for L2/L3 EVPN services VTS provisions SR policy via device templates XTC + WAE provide PCEP solution for SR policy (e.g. lowest IGP metric, lowest TE metric, disjoint path, bandwidth path, etc.)
MPLS/SR - What do we support in 2.6.2 Support for NCS55xx series (Fretta) for MPLS Segment Routing (MPLS SR) Fabric Type. Network fabric configuration (i.e. IGP, SR) is required as part of Day0 configuration (not handled by VTS) VTS performs service configuration, including creation of Bridge Domains, L2 Sub interfaces/vlans on NCS55xx series. L2 EVPN - Multi-point service within and across CO sites L2 EVPN - P2P VPWS via L2 Service Template L3 EVPN - MPLS VPN service within and across CO sites Device Template Multi-homing (ESI) VM Migration Besides OVS on compute, VTF (L2), Cisco s high performance virtual forwarder is supported. Multi-site One VTS instance may be used to manage multiple MPLS SR fabric sites. Seamless provisioning workflow Virtual Machine Manager (VMM) integration RedHat OSP10, OSP13 with support for VM Mobility Resource Pool Management (e.g. Global EVI resource pool, VLAN resource pool) Additional configuration (e.g. SR policy) may be performed through the use of VTS device templates. Supported Scale: 2000 VMs, 2000 VLANs, 255 Tenants, 2000 Tenant Networks, 255 VRFs.
MPLS SR Supported use cases
Central Office/Micro DC POD Segment Routing Transport BGP-EVPN based Layer-3 and Layer-2 Service VPWS EVPN via Service templates Distributed Anycast Gateway on Leafs ML2 VTS Plugin Spine + RR VTS Segment Routing Transport Leaf Host Host BGP-EVPN based L2 and L3 Service
Unified Forwarding with E2E EVPN Single BGP Admin domain End to End EVPN Layer-2 Services between Central Offices End to End Segment Routing Transport DC-01 VTS ML2 VTS Plugin BGP-EVPN RRs AS 65001 AS 65001 AS 65001 ibgp-evpn RRs ML2 VTS Plugin VTS BGP-EVPN DC-n Host CO Fabric Core DC Fabric Host Leaf Node Leaf Node ISIS/BGP SR ISIS/BGP SR ISIS/BGP SR
Collapsed TOR L3 VPN (VPN4/VPN6) MPLS based service RR One of the PEs DC-01 ML2 VTS Plugin AS 65001 AS 65001 AS 65001 DC-01 ML2 VTS Plugin DC-02 VTS VTS Host LACP CO Fabric Core DC Fabric Leaf Node Host Leaf Node Host ISIS/BGP -SR IGP LDP BGP-VPNv4/6 ISIS/BGP - SR
Multi-site support Prior to VTS 2.6.2, each VTS (VTC) instance supports a single site Multi-site supports provides site-scoped Admin domain Inventory Resource Pool Tenant Overlay Device Object N.B.: Auth-group is global-scoped. For a given VTS instance, all sites has to be either all VxLAN or MPLS SR fabric type. Mix of VxLAN and MPLS/SR fabrics type is planned to be supported in the future. Changes introduces site-hierarchical data-model. Benefit: Ease of deployment, simplified management. Note: Multi-site support is different from support for NX-OS VXLAN EVPN Multi-Site which is not supported.
Multi-site support (Con t) Host and Network Inventory for sites must be globally unique (i.e. unique hostname and management IP for underlay) VTSR required per-site for VTF deployment within site. Each site is independently managed, without any logical connectivity to other site unless otherwise configured (no change from current behavior) VMM deployment is site-scoped - VMM deployment spanning multiple sites is not supported. Multi-VMM within site is supported. Existing mechanisms may be used to extend network across sites L2VNI Extension for L2 connectivity across sites. L3 DC-Gateway+DCI or Integrated DCI for L3 connectivity across sites. Resource pool depends on Fabric Type (e.g. VNI for EVPN VxLAN vs EVI for MPLS SR). REST API base URI changes (more on this later)
Multi-site support Supported Deployment model Site 1 Admin-domain Inventory Resource pool Tenant Overlay Site 1 Fabric MPLS SR Site 3 Admin-domain Inventory Resource pool Tenant Overlay Site 3 Fabric MPLS SR VTSR VTSR VMM1 VTC VMM2 Site 2 Admin-domain Inventory Resource pool Tenant Overlay Site 2 Fabric MPLS SR Site 4 Admin-domain Inventory Resource pool Tenant Overlay Site 4 Fabric MPLS SR VTSR VTSR VMM3 VMM4
Multi-site support Supported Deployment model Site 1 Admin-domain Inventory Resource pool Tenant Overlay Site 1 Fabric EVPN VXLAN Site 3 Admin-domain Inventory Resource pool Tenant Overlay Site 3 Fabric EVPN VXLAN VTSR VTSR VMM1 VTC VMM2 Site 2 Admin-domain Inventory Resource pool Tenant Overlay Site 2 Fabric EVPN VXLAN Site 4 Admin-domain Inventory Resource pool Tenant Overlay Site 4 Fabric EVPN VXLAN VTSR VTSR VMM3 VMM4
LDAP Authentication Support VTS 2.6.2 introduces LDAP authentication support. Initial Phase features Only authentication is supported; accounting leverages on existing VTS local accounting logs Valid user on LDAP server assumed to be admin or operator (no RBAC). Verified with OpenLDAP. Multiple LDAP servers (with priorities) are supported.
Demo