SP Datacenter fabric technologies. Brian Kvisgaard System Engineer CCIE SP #41039

Transcription

1 SP Datacenter fabric technologies Brian Kvisgaard System Engineer CCIE SP #41039

2 VMDC 2.1 DC Container Architecture Simplified architecture Services on the stick design modification (Core/Agg handoff) Core Cisco Nexus 7000 Enterprise centric services integration Enterprise multi-tenancy SLA with QoS and alignment with WAN/Campus QoS requirements Functional multicast integration with multi-tenancy Services Aggregation Cisco Nexus 7000 Nexus 1010 integration and Network analysis and monitoring (NAM) capability validation Access vpc Cisco Nexus 5000 Jumbo MTU support and jumbo frame validation Compute and Storage Components UCS Blade Server Compute Nexus 1010 VMware vcenter VMware vsphere 4x10GE 4x10G E 4x10G E 4x10G E Cisco UCS 6100 Fabric Interconnect UCS 5100 Blade Server NAS Storage vpc to N5K

3 Traditional Networking Management options: CLI Cut/Paste Limited automation Disparate management platforms Limitations: Box by box approach Lack of consistent configuration (no network wide policies) Leftover/unknown configuration Open any to any connectivity* Separate virtual and physical networks Separate L4-7 device management

4 ACI Networking APIC APIC APIC Management options: GUI (basic/advanced) CLI XML/JSON Scripting Open API Automation Benefits: Distributed, Centralised Management Full traffic visibility* Self documenting Integrated virtual and physical network Integrated L4-7 device management Policy defined network

5 New Concept: Endpoint Groups Endpoint Groups are quite simply groups of endpoints on the network. The endpoints are identified by their connectivity Domain (virtual/physical/outside) and their connectivity method e.g. Virtual machine portgroups (VLAN, VXLAN) Physical interfaces / VLANs External VLANs External subnets Devices within the same Endpoint group can communicate irrespective of their VLAN/VXLAN backing/id, provided that they have IP reachability. Communication between Endpoint groups is, by default, not permitted (similar to PVLAN).

6 Secure Networking with ACI End Point Groups APIC APIC APIC VRF: 01 (Anycast gateway) BD: storage Hardware Proxy: No ARP Flooding: Yes Unknown Unicast Flooding: Yes IP Routing: No BD: vmotion Hardware Proxy: No ARP Flooding: Yes Unknown Unicast Flooding: Yes IP Routing: No BD: Host-Mgmt Hardware Proxy: No ARP Flooding: Yes Unknown Unicast Flooding: Yes IP Routing: No Endpoints in EPG identified by Interface and VLAN ID vpc_to_ucs_a vlan-12 vpc_to_ucs_b vlan-12 vpc_to_ucs_a vlan-10 vpc_to_ucs_b vlan-10 vpc_to_ucs_a vlan-8 vpc_to_ucs_b vlan-8 ANP: ESXi-Hosts EPG: vmk-storage Security Zone EPG: vmotion Security Zone EPG: Host-Mgmt Security Zone Tenant: ESXi-Hosts Communication allowed within EPG Communication allowed within EPG Communication allowed within EPG

7 Hypervisor Integration APIC Network Admin APIC ACI Fabric Integrated gateway for VLAN, VxLAN, NVGRE networks from virtual to physical Normalization for NVGRE, VXLAN, VLAN VXLAN VLAN NVGRE VLAN VXLAN VLAN and VLAN networks ESX Hyper-V KVM Customer not restricted by a choice of hypervisor PHYSICAL SERVER Fabric is ready for multi-hypervisor Application Admin Hypervisor Management

8 New concept: Contracts (ACLs) Contracts are directional Access Lists between Provider and Consumer EPGs. They comprise of one or more Filters (ACEs) to identify traffic, e.g: Contract: Any-to-Any Filter: Any-Traffic Contract: Web Filter: 80, 443, 8000 Contract: DNS Filter: 53 ANP: My-Web-App Provider EPG: Web Filter: 80, 443 etc Contract: Clients-to-Web Any-to-Any Flags : IP Protocol Ports Stateful Etc. Filter: none Flags : Consumer External Subnet EPG: Clients L3out: Clients Apply in both directions (single contract which allows return traffic) Reverse filter ports (dynamically permits return flow based on src/dst ports)

9 Contracts are Required for Inter EPG Connectivity APIC APIC APIC VRF: 01 (Anycast gateway) BD: ESXi Hardware Proxy: Yes ARP Flooding: No Unknown Unicast Flooding: No IP Routing: /24 : /24 Primary Gateway: /24 Secondary Gateway: /24 vpc Node104_105/1/50 vlan-40 vpc_to_ucs_a vlan-30 vpc_to_ucs_b vlan-30 vpc_to_ucs_a vlan-8 vpc_to_ucs_b vlan-8 ANP: ESXi-Storage EPG: Shared-storage ANP: ESXi-Hosts EPG: vmk-storage EPG: Host-Mgmt Tenant: ESXi-Hosts Contract = Allow Communication No Contract = No Communication

10 Contracts Scope Contracts are scoped at: Global Tenant Context (aka Private Network, aka VRF) Web_to_App Application Profile App_to_DB ANP: 01 EPG: Web EPG: Web EPG: App EPG: App EPG: DB EPG: DB BD: 01 Hardware Proxy: Yes IP Routing: Yes ANP: 02 VRF: 01 Tenant: Web_Hosting

11 Application Centric Infrastructure DB DB Web Web App Web App Turnkey integrated solution with security, centralized management, compliance and scale Automated application centric-policy model with embedded security Broad and deep ecosystem Mass Market (commercial, enterprises, public sector)

12 Programmable Network across the Nexus portfolio Starting with programmability boost on N3K /N9K Programmable Open APIs 3 rd Party DevOps Automation Tools Custom Application Development Managing Switch with Linux Tools DC Repository 3 rd party/custom apps integration Nexus Open, Modular Operating System Toolset Integration in Open NX-OS Extensible Open NX-OS Leverage Linux Toolchain for Switch Management Enhancements to existing NX-API to support objectbased, model driven APIs (RESTful XML/JSON) Pre-developed RPMs from Cisco and Partners Leverage same software tools and expertise across different IT departments New SDK enables custom application development with option for securelxc containers CPU, memory, priority controls Leverage tcpdump, ifconfig ethtool, iproute, BASH shell commands for config and troubleshooting *Deliverables and Timelines for Nexus platforms varies*

13 Application Centric Infrastructure Programmable Network DB DB Web Web App Web App Turnkey integrated solution with security, centralized management, compliance and scale Automated application centric-policy model with embedded security Broad and deep ecosystem Modern NX-OS with enhanced NX-APIs Automation Ecosystem (Puppet, Chef, Ansible etc.) Common NX-API across N2K-N9K Mass Market (commercial, enterprises, public sector) Mega Scale Datacenters

14 VTE P IP Transport Network VTE P VXLAN VNI Local LAN Local LAN Segment Local LAN Local LAN Underlay Network: IP routing proven, stable, scalable ECMP utilize all available network paths Overlay Network: Standards-based overlay Layer-2 extensibility and mobility Expanded Layer-2 name space (16M) Scalable network domain Multi-Tenancy

15 Dst. MAC Addr. Src. MAC Addr. VLAN Type 0x8100 VLAN ID Tag Ether Type 0x0800 IP Header Misc Data Protocol 0x11 Header Checksum Outer Src. IP Outer Dst. IP UDP Src. Port UDP Dst Port UDP Length Checksu m 0x0000 VXLAN RRRR1RRR Reserved VNID Reserved Outer Mac Header Outer IP Header UDP Header VXLAN Header Original L2 Frame FCS FCS 10 or 14 Bytes 20 Bytes 8 Bytes 8 Bytes For next-hop transport in the underlay network Source and Destination addresses, allowing transport across the underlay IP network Allows for possible The well known VXLAN port Indicates a 16M segments VXLAN packet Hash of the internal L2/L3/L4 header of the original frame. Can be used as entropy for better ECMP/LACP load sharing

16 VXLAN terminates its tunnels on s (Virtual Tunnel End Point). Each has two interfaces, one is to provide bridging function for local hosts, the other has an IP identification in the core network for VXLAN encapsulation/decapsulation. Transport IP Network IP Interface IP Interface Local LAN Segment Local LAN Segment End System End System End System End System

17 No VXLAN control plane Data driven flood-&-learn Multicast transport for VXLAN BUM (Broadcast, Unknown Unicast and Multicast) traffic. End System End System -3 3 IP-3 End System A MAC-A IP-A -1 1 IP-1 Multicast Group IP Network -2 2 IP-2 End System B MAC-B IP-B

18 The Secret Sauce is the Control Plane, not the Encapsulation

19 MP-BGP with MPLS VPN Route Distribution Exchange of VPN Policies Among PE Routers Full mesh of BGP sessions among all PE routers BGP Route Reflector PE-CE Link BGP Route Reflector PE-CE Link Multi-Protocol BGP extensions (MP-iBGP) to carry VPN policies PE-CE routing options Static routes ebgp OSPF IS-IS CE CE Blue VPN Policy Red VPN Policy PE PE P P P P PE PE CE BlueVPN Policy` Red VPN Policy CE Label Switched Traffic

20 VPN Control Plane Processing VRF Parameters Make customer routes unique: Route Distinguisher (RD): 8-byte field, VRF parameters; unique value to make VPN IP routes unique VPNv4 address: RD + VPN IP prefix Selective distribute VPN routes: Route Target (RT): 8-byte field, VRF parameter, unique value to define the import/export rules for VPNv4 routes MP-iBGP: advertises VPNv4 prefixes + labels

21 Ethernet VPN Highlights Next generation solution for Ethernet multipoint connectivity services Leverage similarities with L3VPN Data-plane address learning from Access Control-plane address advertisement / learning over Core PEs run Multi-Protocol BGP to advertise & learn MAC addresses over Core Learning on PE Access Circuits via dataplane transparent learning VID 100 SMAC: M1 DMAC: F.F.F CE1 PE1 PE3 CE3 No pseudowire full-mesh required Unicast: use MP2P tunnels Multicast: use ingress replication over MP2P tunnels or use LSM MPLS Under standardization at IETF draft-ietfl2vpn-evpn PE2 BGP MAC adv. Route E-VPN NLRI MAC M1 via PE1 PE4

22 EVPN Ethernet VPN VXLAN Evolution Control- Plane EVPN MP-BGP draft-ietf-l2vpn-evpn Data- Plane Multi-Protocol Label Switching (MPLS) draft-ietf-l2vpn-evpn Provider Backbone Bridges (PBB) draft-ietf-l2vpn-pbb-evpn Network Virtualization Overlay (NVO) draft-sd-l2vpn-evpn-overlay EVPN over NVO Tunnels (VXLAN, NVGRE, MPLSoE) for Data Center Fabric encapsulations Provides Layer-2 and Layer-3 Overlays over simple IP Networks 22

23 DC Core 3-Tier Design Fabric Design DC Spine DC Aggregation DC Access DC Leaf Collapsed Core/Aggregation 2-Tier Design DC-1 DC Interconnect DC-2 DC Core/ Aggregation DC Access WAN

24 Flood-&-Learn EVPN Control Plane Overlay Services L2+L3 L2+L3 Underlay Network IP network with ECMP IP network with ECMP Encapsulation MAC in UDP MAC in UDP Peer Discovery Data-driven flood-&-learn MP-BGP Peer Authentication Not available MP-BGP Host Route Learning Local hosts: Data-driven flood-&-learn Remote hosts: Data-driven flood-&-learn Host Route Distribution No route distribution. MP-BGP Local Host: Data-driven Remote host: MP-BGP L2/L3 Unicast Forwarding Unicast encap Unicast encap BUM Traffic forwarding Multicast replication Unicast/Ingress replication Multicast replication Unicast/Ingress replication

25 MP-BGP for EVPN MP-BGP is the routing protocol for EVPN Multi-tenancy construct using VRF (Rout Distinguisher, Route Targets) New address-family l2vpn evpn for distributing EVPN routes EVPN routes = [MAC] + [IP] ibgp or ebgp support vrf context evpn-tenant-1 vni rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn evpn vni l2 rd auto route-target import auto route-target export auto router bgp 100 router-id log-neighbor-changes address-family ipv4 unicast address-family l2vpn evpn neighbor remote-as 100 update-source loopback0 address-family ipv4 unicast address-family l2vpn evpn send-community extended vrf evpn-tenant-1 address-family ipv4 unicast advertise l2vpn evpn

26 C Install host info to RIB/FIB: H-MAC-1 MAC table H-IP-1 VRF IP host table Host IP VNI AC- H-IP-1 VNII BGP Update: H-MAC-1 H-IP-1-1 VNI Route Reflector 2 3 BGP Update: H-MAC-1 H-IP-1-1 VNI-1 BGP Update: H-MAC-1 H-IP-1-1 VNI Install host info to RIB/FIB: H-MAC-1 MAC table H-IP-1 VRF IP host table MAC Host IP VNI H-MAC-1 H-IP-1 VNII-1-1 MAC Host IP VNI H-MAC-1 H-IP-1 VNII-1-1 Local learning of host info: H-MAC-1 (MAC table) H-IP-1 (VRF IP host table ) 1-1 H-MAC-1 H-IP-1 VLAN-1 /VNI-1 BGP Update RD: Route distinguisher MAC address length: 6 bytes MAC address: Host MAC address IP address length: 32 or 128 IP address: Host IP address (IPv4 or IPv6) L2 VNI: VNI of the bridge domain to which the end host belongs L3 VNI: VNI associated with the tenant VRF routing instance

27 VXLAN BGP Control Plane EVPN Control Plane --- Host Movement NLRI: Host MAC1, IP1 NVE IP 1 VNI 5000 Next-Hop: -1 NLRI: Host MAC1, IP1 NVE IP 1 VNI 5000 Next-Hop: -3 Ext. Community: Encapsulation: VXLAN Cost/Sequence: 1 Ext. Community: Encapsulation: VXLAN Cost/Sequence: 0 Host 1 MAC1 IP 1 VNI MAC IP VNI Next-Hop Encap Seq MAC-1 IP VXLAN 0 MAC IP VNI Next-Hop Encap Seq MAC-1 IP VXLAN detects Host1 and advertise an EVPN route for Host1 with seq# 0 2. Host1 Moves behind detects Host1 and advertises an EVPN route for Host1 with seq # sees more recent route and withdraws its advertisement

28 SVI GW IP GW MAC Host 1 MAC1 IP 1 VLAN A VXLAN A Host 2 MAC2 IP 2 VLAN A VXLAN A Host 3 MAC3 IP 3 VLAN A VXLAN A Host 4 MAC4 IP 4 VLAN A VXLAN A

29 # VLAN to VNI mapping vlan 200 vn-segment 5200 # Anycast Gateway MAC, identically configured on all s fabric forwarding anycast-gateway-mac The same anycast gateway virtual IP address and MAC address need to be configured on all s in the VNI # Distributed IP Anycast Gateway (SVI) # Gateway IP address needs to be identically configured on all s interface vlan 200 no shutdown vrf member Tenant-A ip address /24 fabric forwarding mode anycast-gateway SVI GW IP GW MAC SVI GW IP GW MAC SVI GW IP GW MAC SVI GW IP GW MAC Host 1 MAC1 IP 1 VLAN A VXLAN A Host 2 MAC2 IP 2 VLAN A VXLAN A Host 3 MAC3 IP 3 VLAN A VXLAN A Host 4 MAC4 IP 4 VLAN A VXLAN A

30 ARP Suppression in MP-BGP EVPN ARP suppression reduces network flooding due to host learning IP Address MAC Address VLAN Physical Interface Index (ifindex) Flags IP-1 MAC-1 10 E1/1 Local IP-2 MAC-2 10 Null Remote IP-3 MAC-3 10 Null Remote -1 intercepts the ARP request and checks in its ARP suppression cache. It finds a match for IP-2 in its ARP suppression cache.* sends an ARP response back to Host-1 with MAC-2.* Host-1 learns the IP-2 and MAC-2 mapping. 3 4 Host 1 MAC1 IP 1 VLAN 10 VXLAN Host 1 MAC1 IP 2 VLAN 10 VXLAN 5000 Host-1 in VLAN 10 sends an ARP request for Host-2 s IP-2 address. * If -1 doesn t have a match for IP-2 in its ARP suppression cache table, it will flood the ARP request to all other s in this VNI 30

31 ARP Suppression in MP-BGP EVPN (Cont ed) ARP Suppression can be enabled on a per-vni basis under the interface nve1 configuration interface nve1 no shutdown source-interface loopback0 host-reachability protocol bgp member vni suppress-arp mcast-group member vni suppress-arp mcast-group member vni associate-vrf member vni associate-vrf n9396-vtep-1.sakommu-lab.com# sh ip arp suppression topo-info ARP L2RIB Topology information Topo-id ARP-suppression mode 100 L2 ARP Suppression 200 L2/L3 ARP Suppression 201 L2/L3 ARP Suppression

32 Head-end Replication Head-end Replication (aka. Ingress replication): Eliminate the need for underlay multicast to transport overlay BUM traffic Multicast-Free Spine Underlay 2-1 receives the overlay BUM traffic, encapsulates the packets into unicast VXLAN packets, sends one copy to each remote peer in the same VXLAN VNI Leaf 1 Host-1 sends BUM traffic into the VXLAN VNI 32

33 Different integrated Route/Bridge (IRB) Modes VXLAN Routing Overlay Networks do follow two slightly different integrated Route/Bridge (IRB) semantics Asymmetric Uses different path from Source to Destination and back Symmetric Uses same path from Source to Destination and back Cisco follows Symmetric IRB SVI A -1 Host 1 H-MAC-1 H-IP-1 VNI-A -2 Routing? IP Transport Network -3 SVI B -4 Host 2 H-MAC-2 H-IP-2 VNI-B

34 Asymmetric Routing and Bridging on the ingress Bridging on the egress Both source and destination VNIs need to reside on the ingress Ingress routes packets from source VNI to destination VNI. D- MAC in the inner header is the destination host MAC VNI A 1 VNI B S-IP: -1 D-IP: -4 VNI: VNI-B S-MAC: H-MAC-1 D-MAC: H-MAC-2 S-IP: H-IP-1 D-IP: H-IP-2 VNI A VNI B S-MAC: H-MAC-1 D-MAC: H-MAC-2 S-IP: H-IP-1 D-IP: H-IP-2 S-MAC: H-MAC-1 D-MAC: H-MAC-2 S-IP: H-IP-1 D-IP: H-IP-2-1 Host 1 H-MAC-1 H-IP-1 VNI-A Host 2 H-MAC-2 H-IP-2 VNI-B 2 Egress bridges packets in the destination VNI

35 VXLAN BGP Control Plane VNI Membership Asymmetric IRB Every needs to be in all VNIs Every needs to maintain MAC tables for all VNIs, including those they don t have local hosts for. SVI 100 SVI 200 SVI 100 SVI 200 SVI 100 SVI 200 SVI 100 SVI 200 Host 1 MAC1 IP 1 VLAN 100 VXLAN 5100 Host 2 MAC2 IP 2 VLAN 100 VXLAN 5100 Host 3 MAC3 IP 3 VLAN 200 VXLAN All s in a VNI can be the virtual IP gateway for the local hosts 2. Optimized south-north bound forwarding for routed traffic without hair-pinning

36 Routing on both ingress and egress s Layer-3 VNI Tenant VPN indicator One per tenant VRF Router MAC Ingress routes packets onto the Layer-3 VNI Egress routes packets to the destination Layer-2 VNI

37 Ingress routes packets from source VNI to L3 VNI. D-MAC in the inner header is the egress router MAC VNI A 1 L3 VNI S-IP: -1 D-IP: -4 VNI: L3 VNI S-MAC: Router-MAC-1 D-MAC: Router-MAC-4 S-IP: H-IP-1 D-IP: H-IP-2 L3 VNI 2 VNI B Egress routes packets from L3 VNI to the destination VNI/VLAN S-MAC: H-MAC-1 D-MAC: H-MAC-2 S-IP: H-IP-1 D-IP: H-IP-2-1 Router MAC Router MAC-4 S-MAC: H-MAC-1 D-MAC: H-MAC-2 S-IP: H-IP-1 D-IP: H-IP-2 Host 1 H-MAC-1 H-IP-1 VNI-A Host 2 H-MAC-2 H-IP-2 VNI-B

38 VXLAN BGP Control Plane VNI Membership Symmetric IRB Every only needs to be in VNIs that it has local hosts for. s don t need to maintain MAC tables for VNIs that they don t have local hosts for. SVI 100 SVI 100 SVI 200 Host 1 MAC1 IP 1 VLAN 100 VXLAN 5100 Host 2 MAC2 IP 2 VLAN 100 VXLAN 5100 Host 3 MAC3 IP 3 VLAN 200 VXLAN Optimal utilization of ARP and MAC tables 2. A only needs to be in the VNIs which it has local hosts for.

39 -1 IP Transport Network S-MAC: Router-MAC-1 D-MAC: Router-MAC-2 S-IP: H-IP-1 D-IP: H-IP-2 S-IP: -1 D-IP: -2 VNI: L3-VNI-A -2 Use addresses in the outer header to route encapsulated packets to the egress S-IP: -1 D-IP: -2 VNI: L3 VNI-A S-MAC: Router-MAC-1 D-MAC: Router-MAC-4 S-IP: H-IP-1 D-IP: H-IP-2 Use L3-VNI to identify the tenant VRF S-MAC: H- MAC-1 D-MAC: H- MAC-2 S-IP: H-IP-1 D-IP: H-IP-2 Host 1 H-MAC-1 H-IP-1 VNI-A L3-VNI-A VRF-A S-MAC: H- MAC-1 D-MAC: H- MAC-2 S-IP: H-IP-1 D-IP: H-IP-2 Host 2 H-MAC-2 H-IP-2 VNI-B L3-VNI-A VRF-A Tenant A VRF-A L3-VNI-A H-IP-2 Tenant B VRF-B L3-VNI-B Tenant C VRF-C L3-VNI-C

40 Symmetric IRB has optimal utilization of ARP and MAC tables on a Symmetric IRB scales better for end hosts Symmetric IRB scales better in terms of the total number of VNIs a VXLAN overlay network can support Multi-vendor interoperability: Some vendors implemented Asymmetric IRB It s been agreed upon among multiple vendors that Symmetric IRB is the ultimate solution Cisco implemented Symmetric IRB Cisco will introduce backward compatability with asymmetric IRB by adding the support for it.

41 Local Scoping of VLANs ToR Local 16 million possible VNIs global scope VNI 5000 maps to VLAN 10 VLANS are Locally Scoped at Top of Rack/ Gateway Possible VLAN IDs 1-4K VNI 5000 maps to VLAN 60 VLANS are Locally Scoped at Top of Rack/ Gateway Possible VLAN IDs 1-4K 41

42 Local Scoping of VLANs Port Local* * Available in Q2CY million possible VNIs global scope (Eth1/1, Vlan10) => VNI (Eth1/2, Vlan10) => VNI (Eth1/2, Vlan11) => VNI VNI 5000 maps to (E1/1, VLAN 10) VNI 5000 maps to (E1/2, VLAN 60 VLANS are Locally Scoped VLAN to VNI mapping is per-port significant Possible VLAN IDs 1-4K VLANS are Locally Scoped VLAN to VNI mapping is per-port significant Possible VLAN IDs 1-4K 42

43 Underlay IP Network BGP Router ID 1 BGP Router ID 2 vpc with Anycast Address vpc -1 Virtual PortChannel vpc -2 interface loopback0 ip address /32 ip address /32 secondary Layer 2 Link Layer 3 Link

44 EVPN Control Plane Advantages A multi-tenant fabric solution with host-based forwarding Industry standard protocol for multi-vendor interoperability Build-in multi-tenancy support Leverage MP-BGP to deliver VXLAN with L3VPN characteristics Truly scalable with protocol-driven learning Host MAC/IP address advertisement through EVPN MP-BGP Fast convergence upon host movements or network failures MP-BGP protocol driven re-learning and convergence Upon host movement, the new will send out a BGP update to advertise the new location of the host

45 EVPN Control Plane Advantages (Cont ed) A multi-tenant fabric solution with host-based forwarding Optimal traffic forwarding supporting host mobility Anycast IP gateway for optimal forwarding for host generated traffic No need for hair-pinning to to reach the IP gateway ARP suppression Minimize ARP flooding in overlay Head-end Replication with dynamically learned remote- list Head-end replication enables multicast-free underlay network Dynamically learned remote- list minimizes the operational overhead of head-end replication peer authentication via MP-BGP authentication Added security to prevent rogue s or spoofing

46 Application Centric Infrastructure Programmable Fabric Programmable Network DB DB Web Web App Web App Turnkey integrated solution with security, centralized management, compliance and scale Automated application centric-policy model with embedded security Broad and deep ecosystem VxLAN-BGP EVPN standard-based 3 rd party controller support Modern NX-OS with enhanced NX-APIs Automation Ecosystem (Puppet, Chef, Ansible etc.) Common NX-API across N2K-N9K Mass Market (commercial, enterprises, public sector) Service Providers Mega Scale Datacenters

47 Application Centric Infrastructure Programmable Fabric Programmable Network VTS DB DB Web Web App Web App Turnkey integrated solution with security, centralized management, compliance and scale VxLAN-BGP EVPN standard-based Modern OS with enhanced APIs Integrated Overlay and Underlay optimizations Overlay optimizations Mass Market (commercial, enterprises, public sector) Service Providers Mega Scale Datacenters

48 Cisco Virtual Topology System (VTS) Overlay Provisioning & Management System Cisco Network Services Orchestrator VMware vcenter GUI Flexible Overlays Physical and Virtual Overlays Bare-metal and Virtualized Workloads Service Chaining REST API Automated Seamless Integration with Orchestrators Automated Overlay Provisioning Automated DCI/WAN Integration Open and Programmable REST-Based Northbound APIs Multi-protocol Support Multi-hypervisor Support Cisco Virtual Topology System YANG CLI NX-API BGP-EVPN Scalable VXLAN Mgmt. MP-BGP EVPN Control Plane Virtual Tenant Networks High Performance Virtual Forwarding Nexus Portfolio Nexus 2k 9k

49 VTS Architecture Cisco Network Services Orchestrator (Tail-f) VMware vcenter GUI Unified Information Model (REST API) Virtual Topology System Service and Infrastructure Policy Resource Management Device Management Inventory Database IOS XRv Policy Plane Control Plane Control Plane Federation MP-BGP YANG CLI NX-API BGP-EVPN Virtual Compute Environment Cisco Nexus 2000, 3000, 5000, and 7000 Series Cisco Nexus 9000 Series Cisco ASR 9000 Series OVS VTF DVS

50 EVPN Control Plane S1 MAC, IP Address 1 S2 MAC, IP Address 2 S3 MAC, IP Address 3 S4 MAC, IP Address 4 Industry standard protocol for multi-vendor support IP Transport Network MP-BGP EVPN RR Built in multi-tenancy support VXLAN VNI Restconf/YANG Scalable, protocol driven control plane architecture Fast convergence upon network failures and host movements VTF 4 Local LAN Local LAN LAN Segment Local LAN Local LAN Minimize flooding through ARP suppression S1 S2 S3 S4 Overlay Forwarding Table Security through peer-authentication S1 MAC, IP Address P1/2 S2 MAC, IP Address 2 S3 MAC, IP Address 3 S4 MAC, IP Address 4

51 VTS Architecture Hardware Switches Spine Spine REST API Cisco VTS NX-API, CLI, YANG ToR ToR ToR VMware vcenter Hypervisor Hypervisor Hypervisor VM VM VM VM x86 Server x86 Server x86 Server 51

52 VTS Architecture Integrated DCI Simpler Configuration Single MP-BGP session for all tenants DCI REST API Cisco VTS NX-API, CLI, YANG Spine Spine L3 VNIs (Route) VRF Route-Leaking L3PVN Stitching ToR ToR ToR VMware vcenter Hypervisor Hypervisor Hypervisor VM VM VM VM x86 Server x86 Server x86 Server 52

53 VTS Architecture - VTF User space packet forwarder, Multi tennant DCI Uses Cisco Vector Packet Processing technology Border Leaf Integrated with Intel DPDK Supports VXLAN, extend to e.g SR, MPLS, MPLSoGRE, L2TPv3.. VTF (VM) Programmed by VTS using Restconf/YANG Tenant VM Tenant VM Spine VTF (VM) Spine Tenant VM Tenant VM REST API Cisco VTS NX-API, CLI, YANG vswitch vswitch vswitch ESXi ToR ToR ToR KVM NIC NIC VMware vcenter Hypervisor Hypervisor Hypervisor VM VM VM VM x86 Server x86 Server x86 Server 53

54 VTS Hardware and Software overlay management and provisioning NX-OS mode based VXLAN fabric with MP-BGP EVPN & ToR-based anycast gateway BGP-EVPN VXLAN Overlay Hardware Underlay (standards-based) Hardware-based Overlay (standards-based) VTS ESX Bare Metal ESX Bare Metal ESX Software-based Overlay (standards-based)

55 VTS OpenStack Workflow VTS provisions, VLAN for each and EVPN on ToR/VTF 7 1 Create Tenant Networks 2 Tenent and Tenant Networks Created NX-API, CLI, YANG Spine Spine REST API Cisco VTS OpenStack Tenant View 3 VNID assigned for each network ToR VXLAN ToR VXLAN ToR 4 Attach VM to Network 5 VM Host info captured by VTS and mapped to the right ToR & ToR port using topology database 6 Neutron agent modified to request VLAN information from VTS before programming vswitch Hypervisor 55 VLAN VM x86 Server VLAN Hypervisor VM VM x86 Server VLAN Hypervisor VM x86 Server VLAN

56 VTS OpenStack Workflow 9 VTS provisions L3 VXLAN (distributed L2/L3), Anycast gateway with EVPN Spine Spine NX-API, CLI, YANG REST API Cisco VTS VXLAN VXLAN OpenStack Tenant View ToR ToR ToR VLAN VLAN VLAN Hypervisor Hypervisor Hypervisor VLAN 8 Create router and attach interfaces to tenant networks VM x86 Server VM x86 Server VM VM x86 Server 56

57 Admin Domain d1

58 Administration BGP Route Reflector The Administrator can choose to install BGP RR configuration on 1. Virtualized XR 2. Inline RR on Nexus9k Spine

59 View Virtual Forwarding Group compute2 compute1 BOTH XRv and VTFs register w/ VTS automatically Control Plane is xrv02 running IOS-XR

60 What does VTS provide Infrastructure Providers Tenant selfprovisioning Neutron Abstracted view of a network-wide topology Automate VM discovery in topology and provision virtual network attachment. Make it simple for the end-user SW Forwarder Seamless P2V HW Forwarder SW forwarder for brownfield deployment HW forwarder for performance Virtual Appliance inter-working w/ Physical appliance WA N Connect Tenant networks to Provider Networks Stitch Provider L3VPN to Tenant DC virtual network(s) Tenants attach to External networks via Provider Network VTE P VTE P VTE P VTE P VTE P VTE P

61 Application Centric Infrastructure Programmable Fabric Programmable Network DB DB Web Web App Web App Turnkey integrated solution with security, centralized management, compliance and scale Automated application centric-policy model with embedded security Broad and deep ecosystem VxLAN-BGP EVPN standard-based 3 rd party controller support VTS for software overlay provisioning and management across N2K-N9K Modern NX-OS with enhanced NX-APIs Automation Ecosystem (Puppet, Chef, Ansible etc.) Common NX-API across N2K-N9K Mass Market (commercial, enterprises, public sector) Service Providers Mega Scale Datacenters

62