XLAN the Cisco Way: hy, Where, When, What, How. arren Marinko DC Consulting Systems Engineer hil Lowden DC Consulting Systems Engineer

Transcription

1 XLAN the Cisco Way: hy, Where, When, What, How arren Marinko DC Consulting Systems Engineer hil Lowden DC Consulting Systems Engineer

2 Why?

3 LAN provides a Network with gmentation, IP Mobility, and Scale tandards based overlay verages Layer-3 ECMP with all links forwarding creased name space from 4K to 16M Layer-2 identifiers gmentation and multi-tenancy tegration of physical and virtual SDN: centralized controller separate from data plane

4 siness Outcomes exibility: Layer-2 everywhere; physical and virtual implicity: Centralized controller (irtual Topology Syste calability: Increased name space from 4K to 16M yer-2 identifiers; one spine or leaf at a time ecurity: Segmentation and multi-tenancy

5 Where?

6 rge Cloud Provider (Top 10) Case Customer Ask : Any workload anywhere in their large fabric Scale to support a 188 Rack Data Center! Workload Placement using XLAN Extension Connect to External Network through WAN

7 rge Cloud Provider (Top 10) lementation L3 Boundary N9508 N9508 N9508 N9508 In-Rack LANs XLAN Extended LANs Routed Link Distributed Anycast Gateway SIs N9300 N9300 N9300 N9300 N9300 N9300 Rack 1 LAN 10 Rack 188 LAN 10 IT Core App OS App OS App OS App OS

8 When?

9 thernet PN is Standards-Based l- EPN MP-BGP - RFC 7432 (draft-ietf-l2vpn-evpn) Multi-Protocol Label Switching (MPLS) draft-ietf-l2vpn-evpn Provider Backbone Bridges (PBB) draft-ietf-l2vpn-pbb-evpn Network irtualization Overlay (NO) draft-sd-l2vpn-evpn-overla EPN over NO Tunnels (XLAN, NGRE, MPLSoE) for Data Center Fabric encapsulations Provides Layer-2 and Layer-3 Overlays over simple IP Networks

10 co Supported XLAN IETF Drafts Title er Category Comments 8 irtual extensible Local Area Network Data Plane Flood and Learn based implemented 2 BGP MPLS based Ethernet PNs EPN Control Plane Control Plane is based on this RFC bess-evpn- A Network irtualization Overlay Solution using EPN 0 EPN Control Plane Implemented bess-evpnetg dan-l2vpnn-prefixment Integrated Routing and Bridging in EPN 0 EPN Control Plane Implemented IP Prefix Advertisement in E-PN 2 EPN Control Plane Implemented a-nvo3-oam- NO3 Fault Management 1 Mgmt Plane How much of this are we implementing in s ICMP based? s Jain draft?

11 AN Evolution icast Independent External Connectivity Protocol Learning IP Services y Control-Plane es dynamic TEP ery End Replication es Unicast-only mode ngress Replication) XLAN Hardware Gateway Redundancy (PC) Integrated physical and virtual Overlays (Hybrid Overlays) Inter-Pod Connectivity XLAN Gateway to other Encaps/Networks Workload MAC and IP Addresses learnt by XLAN Edge Devices (NEs) Advertises Layer-2 and Layer-3 Address-to-TEP Association (Overlay Control-Plane) Flood Prevention Optimized ARP forwarding XLAN Routing Hardware Gate Distributed Redundancy Anycast (PC) Integrated Gateway (requires physical Ove and virtual Control-Plane) Overlays (Hybrid Overlays) Multi-Tenancy Inter-Pod Connectivity XLAN Gateway to oth Encaps/Networks 11

12 XLAN/EPN - Interoperability & Feasibility XLAN/EPN interoperability demonstrated during MPLS/SDN World Congress in Paris Participating endors are Cisco, Juniper, Alcatel Lucent & Ixia Independently Tested at EANTC with public available Whitepaper

13 o XLAN Portfolio Scale Secure Multi-tenancy Cisco XLAN Solutions Workload Mobility Workload Anywhere 0 0 Nexus 1000 Nexus 2000 Nexus 3100 Nexus 5600 Nexus 7000 Nexus 9000 ASR ateway L3 Gateway BGP EPN Control Plane Anycast Gateway Head End Replication

14 What?

15 XLAN with BGP EPN Control-Pl still requires a technology for Broadcast/Unknown Unicast/Multic (BUM) replication LAN/EPN = ress Replication Cisco provides 2 Options Multicast Efficient way of send multi-destination traffic to all participating TEP (1 Copy per Link) Unicast Sending a copy of mu destination traffic to all participat TEP (1 copy per TEP)

16 LAN/EPN was ild for Data nter erconnect (DCI) XLAN with BGP EPN Control- Plane uses the control-plane for integrated Route and Bridge (IRB decisions XLAN has now a Control-Plane like OT but XLAN was built for Local Are Network (LAN) There are missing features/functions which are n there (we will talk about it) XLAN could be used for Inte Pod communication

17 hat is XLAN/EPN? XLAN is a network overlay technology Layer-2 & Layer-3 overlay network on top of an IP routed network MAC in UDP encapsulation (UDP destination port 4789) Standards based overlay (XLAN) with standards based Controlplane (BGP) Layer-2 MAC and Layer-3 IP information distribution by Control- Plane (BGP) Forwarding decision based on Control-Plane (minimizes flooding) Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay

18 LAN Gateway Types N Taxonomy AN to LAN Bridging ayer-2 Gateway) Ingress XLAN packet on RED segment XLAN Layer-2 Gateway Egress packet is IEEE 802.1q tagged interface. packet is BRIDGED to new LAN AN-to-XLAN Routing ayer-3 Gateway) Ingress XLAN packet on RED segment Egress XLAN packet is ROUTED to new segment XLAN Router AN-to-LAN Routing ayer-3 Gateway) Ingress XLAN packet on RED segment Egress packet is IEEE 802.1q tagged interface. packet is ROUTED to new LAN XLAN Router 18

19 LAN Overview (1) Edge Device Edge Device Local LAN Segment IP Interface Local LAN Segment Physical Host Edge Device Local LAN Segment Physical Host irtual Switch irtual Hosts 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 19

20 LAN Overview (2) TEP TEP Local LAN Segment Encapsulation Local LAN Segment Physical Host TEP Local LAN Segment Physical Host XLAN Tunnel End-Point D XLAN Network Identifier 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 irtual Switch irtual Hosts

21 LAN Frame Format Dest. MAC Address 48 Next-Hop MAC Address Src TEP MAC Address -in-ip Encapsulation Outer MAC Header Outer IP Header UDP Header XLAN Header Underlay Src. MAC Address LAN Type 0x8100 LAN ID Tag Ether Type 0x0800 Source Port XLAN Port UDP Length Checksum 0x Bytes (4 Bytes Optional) 8 Bytes UDP 4789 IP Header Misc. Data Protocol 0x11 (UDP) Header Checksum Source IP Dest. IP Bytes Src and D of the TE Hash of the inner L2/L3/L4 headers of the original frame. Enables entropy for ECMP Load balancing in the Network. Original Layer-2 Frame Overlay XLAN Flags RRRRIRRR 8 Reserved 24 NI 24 8 Bytes Allows for 16M possible Segments Reserved 8 21

22 AN Flood&Learn MAC NI TEP MAC_A E1/12 MAC NI TEP MAC_B E1/4 1 2 MAC NI TEP MAC_C E1/8 Host A MAC_A / IP_A 3 Host B MAC_B / IP_B irtual Switch 22 Host C MAC_C / IP_C

23 LAN Packet Forwarding N Flood&Learn MAC NI TEP MAC NI TEP MAC_A E1/12 MAC_B E1/4 MAC_B MAC_A : MAC_A : MAC_B _A _B Host A MAC_A / IP_A 1 SIP: IP_1 DIP: IP_2 1 2 SMAC: MAC_1 DMAC: hop-by-hop UDP XLAN NID: Underlay 3 SIP: IP_1 DIP: IP_2 3 SMAC: hop-by-hop DMAC: MAC_2 UDP 2 XLAN NID: Underlay SMAC: MAC_A DMAC: MAC_B SIP: IP_A DIP: IP_B 4 Host B MAC_B / IP_B SMAC: MAC_A DMAC: MAC_B SIP: IP_A DIP: IP_B Overlay SMAC: MAC_A DMAC: MAC_B SIP: IP_A DIP: IP_B Overlay 23

24 EP Discovery (1) ast Independent RR RR Host A MAC_A / IP_A 3 Host B MAC_B / IP_B EPs advertise their NI mbership within the Control-Plane irtual Switch 24 Host C MAC_C / IP_C Host Y MAC_Y / IP_Y

25 EP Discovery (2) ast Independent Peer NI TEP RR RR 3 2 Peer NI TEP Peer NI TEP Host A MAC_A / IP_A Host B MAC_B / IP_B P consolidates and pagates TEP list for NI EP obtains list of TEP ighbors for each NI 25 Host C MAC_C / IP_C irtual Switch Host Y MAC_Y / IP_Y

26 ad-end Replication ast Independent ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF Host A MAC_A / IP_A 1 SIP: IP_1 DIP: IP_3 2 Peer NI TEP SMAC: MAC_1 DMAC: MAC_3 UDP XLAN NID: ARP Request SMAC: MAC_A DMAC: FF:FF:FF:FF:FF:FF 3 3 Underlay Overlay 3 SIP: IP_1 DIP: IP_2 RR Peer NI TEP SMAC: MAC_1 DMAC: MAC_2 RR UDP XLAN NID: ARP Request SMAC: MAC_A DMAC: FF:FF:FF:FF:FF:FF Underlay Overlay ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF irtual Switch Peer NI TEP ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 5 3 Host B MAC_B / IP_B 26 Host C MAC_C / IP_C

27 How?

28 ployment Considerations rlay U and Overlays cast Routing Protocol and IP Addressing lticast for BUM* Traffic Replication 28

29 U and XLAN rlay Outer MAC Header XLAN adds 50 Bytes to the Original Ethernet Frame 50 (54) Bytes of Overhead Outer IP Header UDP Header XLAN Header Original Layer-2 Frame Underlay Overlay Avoid Fragmentation by adjusting the IP Networks MTU Data Centers often require Jumbo MTU; most Server NIC do support up to 9000 Bytes Using a MTU of 9216* Bytes accommod XLAN Overhead plus Server max. MTU *Cisco Nexus 5600/6000 switches only support 9192 Byte for Layer-3 29

30 ilding your IP Network Interface Principles erlay ow your IP addressing and IP ale requirements est to use single aggregate for all nderlay Links and Loopbacks Pv4 only or each Point-2-Point (P2P) onnection, minimum /31 required oopback requires / uted Ports/Interfaces ayer-3 Interfaces between Spine and eaf (no switchport) EP uses Loopback as Sourceerface Cisco and/or its affiliates. All rights reserved. Cisco Public 30

31 ilding your IP Network Routing Protocols; OSP erlay PF watch your Network type etwork Type Point-2-Point (P2P) Preferred (only LSA type-1) No DR/BDR election Suits well for routed interfaces/ports (optimal from a LSA Database perspective) Full SPF calculation on Link Change etwork Type Broadcast Suboptimal from a LSA Database perspective (LSA type-1 & 2) DR/BDR election Additional election and Database Overhead Cisco and/or its affiliates. All rights reserved. Cisco Public 31

32 ilding your IP Network Routing Protocols; IS-IS erlay -IS what was this CLNS? ndependent of IP (CLNS) ell suited for routed interfaces/ports o SPF calculation on Link change; nly if Topology changes ast Re-convergence ot everyone is familiar with it Cisco and/or its affiliates. All rights reserved. Cisco Public 32

33 ilding your IP Network Routing Protocols; ibgp erlay P + IGP = The Routing Protocol mbo GP for underlay topology & eachability (e.g. IS-IS, OSPF) BGP for TEP (loopback) reachability BGP route-reflector for simplification nd scale equires two routing protocols eparates Links (IGP) from TEPs ibgp) End-Host information are still in ibgp but different address-family 1 ibgp RR RR RR RR Cisco and/or its affiliates. All rights reserved. Cisco Public 33

34 ilding your IP Network Routing Protocols; ebgp erlay GP BGP Peer is IP interface Loopback would require additional IGP and ebgp multi-hop ultiple Autonomous-Systems (AS) Minimum amount of AS is two any BGP Neighbors For each neighboring p2p interface S Path Src and Dst AS might be same o Route-Reflector But next-hop needs to be unchanged 1 AS #65501 ebgp 3 AS #65503 AS #65500 RR RR RR RR 1 ebgp 2 AS #65502 AS #65500 RR RR RR RR 2 3 AS # Cisco and/or its affiliates. All rights reserved. Cisco Public 34

35 lticast Enabled Underlay erlay use PIM-ASM or PIM-BiDir (Different hardware has different capabilities) Nexus 1000v Nexus 3000 Nexus 5600 Nexus 7000/F3 Nexus 9000 ASR 1000 CSR 1000 ASR 90 t IGMP v2/v3 PIM ASM PIM BiDir PIM ASM / PIM BiDir PIM ASM PIM BiDir PIM ASM / PI ine and Aggregation Switches make good Rendezvous-Point (RP) Locations in pologies serve a range of Multicast Groups (Destination Groups/DGroups) to service the Overl d optimize for diverse NIs Spine/Leaf topologies with lean Spine se multiple Rendezvous-Point across the multiple Spines ap different NIs to different Rendezvous-Point for simple load balancing measure se Redundant Rendezvous-Pint sign a Multicast Underlay for a Network Overlay, Host TEPs will leverage this Netwo Cisco and/or its affiliates. All rights reserved. Cisco Public 35

36 lticast Enabled Underlay PIM ASM erlay Sparse-Mode (ASM) dundant Rendezvous-Point using PIM ycast-rp or MSDP RP RP urce-tree or Unidirectional Sharede (Source-Tree shown) hared-tree will always use RP for orwarding ource-tree per Multicast-Group per EP (each TEP is Source & Receiver) 1 2 ample from depicted topology TEPs sharing same NI and Multicastroup mapping (single Multicast-Group) x Source-Tree (1 per TEP per Multicastroup) 3 RP Rendezvous-Po Cisco and/or its affiliates. All rights reserved. Cisco Public 36

37 lticast Enabled Underlay PIM ASM erlay Sparse-Mode (ASM) dundant Rendezvous-Point using PIM ycast-rp or MSDP RP RP urce-tree or Unidirectional Sharede (Source-Tree shown) hared-tree will always use RP for orwarding ource-tree per Multicast-Group per EP (each TEP is Source & Receiver) 1 2 ample from depicted topology TEPs sharing same NI and Multicastroup mapping (single Multicast-Group) x Source-Tree (1 per TEP per Multicastroup) 3 RP Rendezvous-Po Cisco and/or its affiliates. All rights reserved. Cisco Public 37

38 tributed IP Anycast Gateway* AN/EPN tributed Routing with IP Anycast teway (Integrated Route/Bridge B) outing between NI (Different ubnet) ridging within NI (Same Subnet) er-xlan Routing Leaf/Access yer ll Leafs share gateway IP and MAC or a Subnet (No HSRP) Host will always find its Gateway irectly attached anywhere it moves s EPN Control-Plane. Host A NI Host Y NI Cisco and/or its affiliates. All rights reserved. Cisco Public 38

39 LAN/EPN Fabric External Routing rlay e Border Leaf/Spine provides yer-2 and Layer-3 connectivity to ternal Network GP and OSPF are Layer-3 uting Protocols available for ternal routing day, RF-lite allows to extend F outside of the fabric th Nexus 7000/7700 and F3, P & MPLS L3PN becomes ailable for fabric extension BL WAN Cisco and/or its affiliates. All rights reserved. Cisco Public

40 us 9300/9500 ernal Connectivity core Nexus 9300/9500 RF-lite (Layer-3 Classic Ethe PC (Layer-2 Classic Etherne AS X Border Leaf Border Spine RR BS RR BS XLAN FABRIC exus 9300/9500 as Border Leaf or Border Spine

41 us 7000/7700 (F3) ible External Connectivity core Nexus 7000/7700 (F3) RF-lite (Layer-3 Classic Ethe PC (Layer-2 Classic Etherne LISP* MPLS L3PN* AS X Border Leaf Border Spine RR BS RR BS XLAN FABRIC exus 7000/7700 as Border Leaf or Border Spine

42 LAN/EPN Fabric External Routing rlay RF for External Routing need to exist on Border Leaf RF A BL RF B RF C 2 Interface-Type Options: Physical Routed Ports 1 Sub-Interfaces LAN SIs over Trunk Ports Peering Interface can be in Global or Tenant R 3 WAN Cisco and/or its affiliates. All rights reserved. Cisco Public 42

43 rconnecting Classic Ethernet Networks (Layer-2 -Pod Connectivity erconnecting Classic Ethernet Pods h XLAN/EPN is possible PN Control-Plane provides ARP uppression and avoids Flood&Learn Core (Layer-3) XLAN/EPN NI ress Replication to avoid Multicast uirement in Underlay epends on various communication equirement factors (traffic requirements, mount of Sites, etc) ebgp lti-homing requires PC(!) op in one Classic Ethernet Pod can uence the other Pod(s) 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 43

44 rconnecting XLAN Networks (Layer-2) -Pod Connectivity rconnecting XLAN/EPN ds with XLAN/EPN is sible ontrol-plane Domains (EPN) an be separated (ibgp/ebgp) Core (Layer-3) ta-plane Encapsulation is d-to-end! eaf/tor knows about all TEP cross the two Data Centers UM Traffic is across Pods XLAN/EPN NI ebgp XLAN/EP NI cision on Ingress Replication icast) or Multicast is across Pods being interconnected ibgp e Shared Route-Target to ure Route Exchange! 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 44

45 rconnecting XLAN Networks (Layer-3) -Pod Connectivity erconnecting XLAN/EPN Pods with LAN/EPN is possible ontrol-plane Domains (EPN) can be eparated (ibgp/ebgp) Core (Layer-3) XLA/EPN NI th Layer-3 interconnect, Data-Plane capsulation is separated outing decision at DC-Edge results in ecapsulation equires a Transit NI between Sites Layer-2 Interconnect! XLAN or XLAN/EPN NI XLAN XLAN/E NI Cisco and/or its affiliates. All rights reserved. Cisco Public 45

46 AN/EPN Designs OSPF/PIM for Underlay Control-P ibgp for Overlay Control-Plane ebgp for External Connectivity (vr AS X Spine RR RR RR RR XLAN Overlay PC L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 BL BL exus 9300 or 9500 as Spine (RP/RR) exus 9300 or 9500 as Leaf (TEP) exus 9300 or 9500 as Border Leaf (TEP) e B G P core Bo

47 AN/EPN Designs OSPF/PIM for Underlay Control-P ibgp for Overlay Control-Plane ebgp for External Connectivity (vr AS X Spine RR RR RR RR XLAN Overlay PC L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 BL BL exus 7000/7700 as Spine (RP/RR) exus 9300 or 9500 as Leaf (TEP) exus 7000/7700 with F3 as Border Leaf (TEP) e B G P core Bo

48 AN/EPN Designs AS X OSPF/PIM for Underlay Control-P ibgp for Overlay Control-Plane ebgp for External Connectivity (vr Spine RR RR RR RR XLAN Overlay PC L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 BL BL exus 5600 as Spine (RP/RR) exus 5600 as Leaf (TEP) exus 7000/7700 with F3 as Border Leaf (TEP) e B G P core Bo

49 AN/EPN Designs core OSPF/PIM for Underlay Control-P ibgp for Overlay Control-Plane ebgp for External Connectivity (vr AS X Border Spine RR BS RR BS PC XLAN Overlay L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 exus 7000/7700 as Border Spine (TEP/RP/RR) exus 9300 or 9500 as Leaf (TEP)

50 AN/EPN Designs core OSPF/PIM for Underlay Control-P ibgp for Overlay Control-Plane ebgp for External Connectivity (vr AS X Border Spine RR BS RR BS PC XLAN Overlay L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 exus 7000/7700 as Border Spine (TEP/RP/RR) exus 5600 as Leaf (TEP)

51 AN/EPN Designs OSPF/PIM for Underlay Control-P ibgp for Overlay Control-Plane ebgp for External Connectivity (vr AS X Spine RR RR RR RR XLAN Overlay PC L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 BL BL exus 7000/7700 as Spine (RP/RR) exus 9300 or 9500 as Leaf (TEP) exus 7000/7700 with F3 as Border Leaf (TEP) e B G P core Bo

52 AN/EPN Designs AS X OSPF/PIM for Underlay Control-P ibgp for Overlay Control-Plane ebgp for External Connectivity (vr Spine RR RR RR RR XLAN Overlay PC L2/L3 L2/L3 L2/L3 L2/L3 L2/L3 BL BL exus 5600 as Spine (RP/RR) exus 5600 as Leaf (TEP) exus 7000/7700 with F3 as Border Leaf (TEP) e B G P core Bo

53

54 Questions?