Cisco Dynamic Fabric Automation Architecture. Miroslav Brzek, Systems Engineer

Transcription

1

2 Cisco Dynamic Fabric Automation Architecture Miroslav Brzek, Systems Engineer

3 Agenda DFA Overview Optimized Networking Fabric Properties Control Plane Forwarding Plane Virtual Fabrics Fabric Management Workload Automation Hardware Support

4 STP The Data Center Fabric Journey MC-LAG vpc L2MP FabricPath DFA MAN/WA N MAN/WAN 4

5 Cisco Dynamic Fabric Automation Architecture Innovative Building Blocks Optimized Fabric Management Network Virtual Workload Fabrics Automation Optimized Fabric Management Network Virtual Workload Fabrics Automation 5

6 Agenda DFA Overview Optimized Networking Fabric Properties Control Plane Forwarding Plane Virtual Fabrics Fabric Management Workload Automation Hardware Support

7 Cisco Dynamic Fabric Automation Scale, Resiliency and Efficiency Advantages Any subnet, anywhere, rapidly Reduced Failure Domains Extensible Scale & Resiliency Profile Controlled Configuration Network Config Profile Network Services Policies n1000v# show port-profile name WebProfile port-profile WebServer-PP description: status: enabled system vlans: port-group: WebServers config attributes: switchport mode access switchport access vlan 110 no shutdown security-profile Protected-Web-Srv evaluated config attributes: switchport mode access switchport access vlan 110 no shutdown assigned interfaces: Veth10 Full Bi-Sectional Bandwidth (N Spines) Any/All Leaf Distributed Default Gateways Any/All Subnets on Any Leaf 7

8 Cisco Dynamic Fabric Automation Flexible Topologies MAN/WAN Traditional Aggregation/Access Folded CLOS MAN/WAN Three Tiers / Fat Tree Full Mesh MAN/WAN MAN/WAN 8

9 Cisco Dynamic Fabric Automation Architecture Device Roles Virtual Machines Physical Machines FEXes 3rd Party Switches UCS FIs Blade Switches Storage Services Firewalls Load Balancers 3rd Party Appliances MAN/WAN Routers Switches 3rd Party Devices Spine Leaf Border Leaf Services Leaf Virtual Leaf* Note: the different Leaf roles are logical and not physical. The same Leaf Switch could perform all three functions (Regular, Services and Border Leaf) *Virtual Leaf: being a light participant on the control plane protocol (supporting VDP) 9

10 Cisco Dynamic Fabric Automation Fabric Properties High Bi-Sectional Bandwidth Wide ECMP: Unicast or Multicast Uniform Reachability, Deterministic Latency High Redundancy: Node/Link Failure Line rate, low latency, for all traffic Fabric properties applicable to all topologies 10

11 Agenda DFA Overview Optimized Networking Fabric Properties Control Plane Forwarding Plane Virtual Fabrics Fabric Management Workload Automation Hardware Support

12 Fabric Control Plane IS-IS as Fabric Control Plane ISIS for fabric link state distribution Fabric node reachability for overlay encapsulation Building multi-destination trees for multicast and broadcast traffic Quick reaction to fabric link/node failure (Layer-2 BFD) Enhanced for mesh topologies Fabric Control Protocol doesn t distribute Host Routes Host originated control traffic Server subnet information IS-IS Control Plane IS-IS Adjacencies 12

13 Fabric Control Plane Host and Subnet Route Distribution RR RR Fabric Host/Subnet Route Injection MP-BGP Control Plane External Subnet Route Injection MAN/WAN Host Route Distribution decoupled from the Fabric link state protocol Use MP-BGP on the leaf nodes to distribute internal host/subnet routes and external reachability information ibgp Adjacencies MP-BGP enhancements to carry up to 1.2 Million routes and reduce convergence time Note: Route-Reflectors deployed for scaling purposes 13

14 Fabric Control Plane Host Originated Protocols Containment Controls External Subnet Route Injection ARP, ND, IGMP, DHCP with endpoints MAN/WAN PIM, IGP, ebgp toward the External Network Domain ARP, ND, IGMP, DHCP originated on servers are terminated on Leaf nodes Contain floods and failure domains, distribute control packet processing Terminate PIM, OSPF, ebgp from external networks on Border Leafs 14

15 Fabric Control Plane Host Detection and Deletion In order to advertise host reachability information, a leaf must first discover locally connected devices Detection of local hosts Based on VDP* or ARP/ND/DHCP *VDP (VSI Discovery and Configuration Protocol) is IEEE 802.1Qbg Clause 41 ARP/ND/DHCP VDP Virtual Switch ARP/ND/DHCP Detection of remote hosts Received MP-BGP notifications Note: Discovered IP address information from ARP/ND-Table get redistributed into MP-BGP Control Plane for End-Host reachability 15

16 Fabric Control Plane Host Detection and Deletion (Detail on Data Plane Trigger) Data packet from Server reaches Leaf Leaf detects new MAC learn event pull config interface eth1/1 switchport mode trunk switchport trunk native vlan 10 pull config interface eth1/32 switchport mode trunk VLAN detected based on: IEEE 802.1q tag used between Server and Leaf VLAN configured on Leaf port ARP/ND/DHCP Virtual Switch ARP/ND/DHCP switch# show mac-address-table Based on learned VLAN and Leaf local configuration parameters, logical configuration get pulled, instantiated and applied on Leaf VLAN MAC Address Type Age Port b967.3cd0 dynamic 10 Eth1/ c.b05a.5380 dynamic 200 Eth1/32 16

17 Fabric Control Plane Host Detection and Deletion (Detail on VDP* Trigger) VDP* session gets established between virtual Leaf and physical Leaf pull config Segment ID (VNI) gets sent from virtual Leaf based on Virtual Machine configuration switch# show evb host interface eth1/21 switchport mode trunk VDP Host Name VNI Vlan BD Mac-address IP-Address Interface Server ac.1f Eth1/21 Physical Leaf responds with next available VLAN defined in Pool (system fabric dynamic-vlans) Based on learned Segment ID (VNI), logical configuration get pulled, instantiated and applied on Leaf *VDP (VSI Discovery and Configuration Protocol) is IEEE 802.1Qbg Clause 41 17

18 Agenda DFA Overview Optimized Networking Fabric Properties Control Plane Forwarding Plane Virtual Fabrics Fabric Management Workload Automation Hardware Support

19 Optimized Network Distributed Gateway at the Leaf GW IP: GW MAC: 0011:2222:3333 GW IP: GW MAC: 0011:2222:3333 Virtual Switch MAN/WAN Anycast Gateway Any Subnet anywhere => Any Leaf can instantiate ANY Subnet All Leafs share gateway IP and MAC for a Subnet (No HSRP) ARPs are terminated on Leafs, No Flooding beyond Leaf Facilitates VM Mobility, workload distribution, arbitrary clustering Seamless Layer-2 or Layer-3 communication between physical hosts and virtual machines 19

20 Optimized Networking Distributed Gateway Mode Distributed Gateway exists on all Leafs where VLAN/Segment-ID is active There are different Forwarding Modes for the Distributed Gateway: Proxy-Gateway (Enhanced Forwarding) Leverages local proxy-arp Intra and Inter-Subnet forwarding based on Routing Contain floods and failure domains to the Leaf Anycast-Gateway (Traditional Forwarding) Intra-Subnet forwarding based on Bridging Data-plane based conversational learning for endpoints MAC addresses ARP is flooded across the fabric vlan 123 vn-segment 30000! interface vlan 123 vrf member OrgA:PartA fabric forwarding mode proxy-gateway ip address /24 no shutdown no ip redirects vlan 145 vn-segment 31000! interface vlan 145 vrf member OrgA:PartA fabric forwarding mode anycast-gateway ip address /24 no shutdown 20

21 Optimized Networking IP Forwarding within the Same Subnet 1. H1 sends an ARP request for H The ARP request is intercepted at Leaf1 (L1) and punted to the Supervisor 3. Assuming a valid route to H3 does exist in the Unicast RIB, Leaf1 (L1) sends the ARP reply with G_MAC so that H1 can build its ARP cache Note: the ARP request is NOT flooded across the Fabric nor out of other local interfaces belonging to the same Layer-2 domain L1 RIB /32 -> NH L4_IP L1 ARP Table L4_IP -> L4_MAC 1 H CPU 2 3 H1 ARP Cache > G_MAC L1 L4 Virtual Switch H

22 Optimized Networking IP Forwarding within the Same Subnet (2) 4. H1 generates a data packet with G_MAC as destination MAC 5. Leaf1 (L1) receives the packet and performs Layer-3 lookup for the destination 6. Leaf1 (L1) adds the Layer-2 and the FabricPath headers and forwards the FabricPath frame across the Fabric, picking one of the equal cost paths available via the multiple Spines 7. Leaf4 (L4) receives the packet, strips off the FabricPath header and performs Layer-3 lookup and forwarding toward H3 5 L1 RIB /32 -> NH L4_IP L1 ARP Table L4_IP -> L4_MAC 4 DMAC: G_MAC SMAC: H1_MAC DIP: SIP : H H1 ARP Cache > G_MAC L1 6 DSID: L4 SSID: L1 DMAC: L4_MAC SMAC: L1_MAC DIP: SIP : L4 e1/1 Virtual Switch H L4 RIB /32 -> e1/1 SIP : DIP: SMAC: G_MAC DMAC: H3_MAC 7 22

23 Optimized Networking IP Forwarding Across Different Subnet 1. H1 sends an ARP request for Default Gateway The ARP request is intercepted at the Leaf1 (L1) and punted to the Supervisor 3. Leaf1 (L1) acts as a regular Default Gateway and sends ARP reply with G_MAC CPU 2 L1 L4 1 3 Virtual Switch H H H1 ARP Cache > G_MAC 23

24 Optimized Networking IP Forwarding Across Different Subnet (2) 4. H1 generates a data packet destined to H2 IP with G_MAC as destination MAC 5. Leaf1 (L1) receives the packet and performs Layer-3 lookup for the destination 6. If valid routing information for H2 is available in the unicast routing table, Leaf1 (L1) adds the Layer-2 and the FabricPath headers and forwards the FabricPath frame across the Fabric, picking one of the equal cost paths available via the multiple Spines 7. Leaf4 (L4) receives the packet, strips off the FabricPath header and performs Layer-3 lookup and forwarding toward H2 5 L1 RIB /32 -> NH L4_IP 4 DMAC: G_MAC SMAC: H1_MAC DIP: SIP : L1 ARP Table L4_IP -> L4_MAC H H1 ARP Cache > G_MAC L1 6 DSID: L4 SSID: L1 DMAC: L4_MAC SMAC: L1_MAC DIP: SIP : L4 e1/1 Virtual Switch H L4 RIB /32 -> e1/1 SIP : DIP: SMAC: G_MAC DMAC: H2_MAC 7 24

25 Optimized Networking Introducing Layer-3 Conversational Learning Use of /32 host routes may lead to scaling issues if all the routes are installed in the Hardware tables of all Leaf nodes Layer-3 conversational learning is introduced to alleviate this concern Disabled by default -> all host routes are programmed in the Hardware With Layer-3 conversational learning, host routes for remote endpoints will be programmed into the Hardware FIB (from the Software RIB) upon detection of an active conversation from a local endpoint Default Behavior (No Layer-3 Conversational Learning) After Enabling Layer-3 Conversational Learning L1 RIB /32 -> NH L2_IP /32 -> NH L3_IP L1 FIB /32 -> NH L2_IP /32 -> NH L3_IP DFA Fabric L1 L2 L3 L1 RIB /32 -> NH L2_IP /32 -> NH L3_IP L1 FIB /32 -> NH L2_IP DFA Fabric L1 L2 L3 Virtual Switch Virtual Switch H H H H1 H H

26 Encapsulation and Forwarding What about Encapsulation? Layer-2 IS-IS Fabric FabricPath Encapsulation Layer-3 IS-IS Fabric VXLAN Encapsulation End-Host Reachability and Distribution DFA + FabricPath Multi-Protocol BGP DFA + VXLAN Multi-Protocol BGP End-Host Detection ARP/ND & VDP ARP/ND & VDP Fabric Multicast Control-Protocol FabricPath IS-IS PIM Bi-Dir Fabric Topology Control-Plane FabricPath IS-IS Layer-3 IS-IS Leaf IP/MAC binding distribution within Fabric FabricPath IS-IS Multi-Protocol BGP 26

27 Agenda DFA Overview Optimized Networking Fabric Properties Control Plane Forwarding Plane Virtual Fabrics Fabric Management Workload Automation Hardware Support

28 Cisco Dynamic Fabric Automation Virtual Fabrics for Public or Private Cloud Environments Advantages Any workload, any Virtual Fabric, rapidly Scalable Secure Virtual Fabrics Virtual Fabric Tenant Visibility Routing/Switching Segmentation HR Manufacturing Finance Sales 28

29 Virtual Fabrics Introducing Segment-ID Support Traditionally VLAN space is expressed over 12 bits (802.1Q tag) Limits the maximum number of segments in a Data Center to 4096 VLANs 29

30 Virtual Fabrics Introducing Segment-ID Support Traditionally VLAN space is expressed over 12 bits (802.1Q tag) Limits the maximum number of segments in a Data Center to 4096 VLANs DFA leverages a double 802.1Q tag for a total address space of 24 bits Support of ~16M segments Segment-ID is hardware-based innovation offered by Leaf and Spine nodes part of the DFA Fabric Cisco DFA Frame (FP Encap) Segment- ID Segment-ID = 802.1Q 802.1Q 30

31 Virtual Fabrics 802.1Q Tagged Traffic to Segment-ID Mapping Access VLANs vlan 123 mode fabricpath vn-segment Segment-ID vlan 123 mode fabricpath vn-segment q 802.1q VLANs Virtual Switch VLANs Segment-IDs (Global) vlan 500 mode fabricpath vn-segment Segment-IDs are utilized for providing isolation at Layer-2 and Layer-3 across the Integrated Fabric 802.1Q tagged frames received at the Leaf nodes from edge devices must be mapped to specific Segments The VLAN-Segment mapping can be performed on a leaf device level VLANs become locally significant on the leaf node and 1:1 mapped to a Segment- ID Segment-IDs are globally significant, VLAN IDs are locally significant 31

32 Virtual Fabrics How are Segment-IDs Utilized? Each IP Subnet, defined at the edge of the Fabric is associated to a Layer-2 domain, which is represented by a Segment-ID A Segment-ID will be used to uniquely identify a VRF within the Fabric Multiple Layer-2 domains can be defined for a given Tenant and are mapped to a Layer-3 VRF Tenant: Blue VRF: OrgA:PartA Segment-ID: Segment-ID: /24 Tenant: Green VRF: OrgA:PartB Segment-ID: Segment-ID: /24 Segment-ID: /24 Segment-ID: /24 32

33 Virtual Fabrics Fabric Routed Flows 1. H1 sends a packet to H3 traffic between the Server and the Leaf is tagged with a local VLAN-ID Layer-3 lookup is performed by Leaf1 (L1) in the context of the BLUE VRF 3. Leaf1 (L1) adds the Layer-2 and FabricPath headers before sending the packet into the fabric. The Segment-ID identifying the BLUE VRF is added inside the Layer-2 header 4. Leaf4 (L4) receives the frame and associates it to the BLUE VRF by looking at the Segment-ID value. It then sends it to H3 using a local VLAN-ID Blue VRF <-> Segment-ID <-> NH L4_IP 1 DMAC: GW_MAC SMAC: L1_MAC VLAN 123 DIP: SIP : H L1 3 DSID: L4 SSID: L1 DMAC: L4_MAC SMAC: L1_MAC Segment-ID = DIP: SIP : L4 e1/1 Virtual Switch H Blue VRF <-> Segment-ID <-> e1/1 SIP : DIP: VLAN 123 SMAC: GW_MAC DMAC: L4_MAC 4 Note: this behavior applies to all Fabric routed flows (intra-subnet or inter-subnet) 33

34 Virtual Fabrics Fabric Routed Flows 1. H1 sends a packet to H3 traffic between the Server and the Leaf is tagged with a local VLAN-ID Layer-3 lookup is performed by Leaf1 (L1) in the context of the BLUE VRF 3. Leaf1 (L1) adds the Layer-2 and FabricPath headers before sending the packet into the fabric. The Segment-ID identifying the BLUE VRF is added inside the Layer-2 header 4. Leaf4 (L4) receives the frame and associates it to the BLUE VRF by looking at the Segment-ID value. It then sends it to H3 using a local VLAN-ID Blue VRF <-> Segment-ID <-> NH L4_IP 1 DMAC: GW_MAC SMAC: L1_MAC VLAN 123 DIP: SIP : H L1 3 DSID: L4 SSID: L1 DMAC: L4_MAC SMAC: L1_MAC Segment-ID = DIP: SIP : Switch Local significance L4 e1/1 Virtual Switch H Blue VRF <-> Segment-ID <-> e1/1 SIP : DIP: VLAN 555 SMAC: GW_MAC DMAC: L4_MAC 4 Note: this behavior applies to all Fabric routed flows (intra-subnet or inter-subnet) 34

35 Agenda DFA Overview Optimized Networking Fabric Properties Control Plane Forwarding Plane Virtual Fabrics Fabric Management Workload Automation Hardware Support

36 Simplifying Fabric Management & Optimizing Fabric Visibility Advantages Device Auto-Configuration Cabling Plan Consistency Check Automated Network Provisioning Common point of fabric access Tenant, Virtual Fabric & Host Visibility SCP/SFTP Services TFTP Services DHCP Services XMPP Services LDAP Message Broker 36

37 Agenda DFA Overview Optimized Networking Fabric Properties Control Plane Forwarding Plane Virtual Fabrics Fabric Management Workload Automation Hardware Support

38 Workload Automation & Open Environment Advantages Any workload, Anywhere, Anytime Open Integration: Orchestration Automated Scalable Provisioning Workload aware fabric UCS Director Open APIs Fabric Management Network Services Controller Published Schema Compute & Storage Policies Network & Network Services Policies 38

39 Workload Automation & Open Environment Compute & Storage Orchestration UCS Director Network & Services Orchestration Compute/Virtualization Controllers e.g. UCS Manager, vcenter, SCVMM, etc. Fabric Management Here is the Glue! MAN/WAN Network Services Controller Virtual Switch *VDP (VSI Discovery and Configuration Protocol) is IEEE 802.1Qbg Clause 41 39

40 Workload Automation Leveraging VDP for Leaf Auto-Configuration Fabric Management Segment-ID, IP information (GWY, Mask, Org, etc.) Orchestrator Network Services Controller Tenants Segment Information Download (Push) Create Logical Org Network Communicate Org Network to Fabric

41 Workload Automation Leveraging VDP for Leaf Auto-Configuration (2) Fabric Management Query the DCNM Network DB (Segment-ID as key) Network Services Controller Orchestrator Tenants VDP Segment-ID from the Virtual-Switch Create Logical Org Network New VM gets created in Blue Network Instantiate Blue Network Communicate Org Network to Fabric VDP information exchange

42 Workload Automation Leveraging VDP for Leaf Auto-Configuration (3) Fabric Management Configuration Download (Pull) Network Services Controller Orchestrator Tenants SVI, VRF Creation VDP VLAN-ID to the Virtual-Switch Create Logical Org Network New VM gets created in Blue Network Instantiate Blue Network Communicate Org Network to Fabric VDP information exchange

43 Workload Automation Leveraging VDP for Leaf Auto-Configuration (4) Fabric Management Leaf receives 802.1q tagged frames and associates them to the Segment-ID Network Services Controller Orchestrator Tenants Create Logical Org Network New VM gets created in Blue Network Instantiate Blue Network Communicate Org Network to Fabric VDP information exchange

44 Agenda DFA Overview Optimized Networking Fabric Properties Control Plane Forwarding Plane Virtual Fabrics Fabric Management Workload Automation Hardware Support

45 Cisco Dynamic Fabric Automation Platform Support Compute & Storage Orchestration UCS Director Spine Nexus 7x00 (F2/F2e/F3) Nexus 6000 Leaf Border Leaf Nexus 5500 Nexus 5600 Nexus 6000 Cisco Prime DCNM Nexus 2000 (FEX) Nexus 7x00 (F3) Cisco Prime NSC Nexus 1000v/OVS 45

46 Cisco Dynamic Fabric Automation Platform Support Compute & Storage Orchestration UCS Director Spine Nexus 7x00 (F2/F2e/F3) Nexus 6000 FabricPath VXLAN Leaf Border Leaf Nexus 5500 Nexus 5600 Nexus 6000 Cisco Prime DCNM Nexus 2000 (FEX) Nexus 7x00 (F3) Cisco Prime NSC Nexus 1000v/OVS Shipping NX-OS 7.1 (roadmap) 46

47 Cisco Dynamic Fabric Automation Simplify Optimize - Automate Enabling the Unified Fabric with Automation, Optimization and Simplification using Orchestration Integration with UCS Director, Openstack & vcd for Workload Automation Optimized Networking for Extensible Resiliency and for Workload distribution and VM Mobility Secure Virtual Fabrics CPoM for Multi-Device Configuration, Automated Network Provisioning and Network & Host Visibility 47

48 Prosíme, ohodnoťte tuto přednášku Děkujeme

49 Q & A

50