Deployment Considerations with Interconnecting Data Centers

Transcription

1

2 Deployment Considerations with Interconnecting Data Centers Patrice Bellagamba Distinguished SE Cisco Europe

3 Session Objectives The main goals of this session are: Highlighting the main business requirements driving Data Center Interconnect (DCI) deployments Understand the functional components of the holistic Cisco DCI solutions Get a full knowledge of Cisco LAN extension technologies and associated deployment considerations Integrate routing aspect induced by the emerging application mobility offered by DCI This session does not include: Network services integration (Firewall / Load Balancer) This is the objective of BRK session Storage extension considerations associated to DCI deployments 3

4 Related Cisco Live 2011 Events DCI Sessions Session-ID TECIPM-3191 BRKDCT-2049 BRKDCT-2081 BRKDCT-2131 Session Name Advanced LISP Techtorial Overlay Transport Virtualization Cisco FabricPath Technology and Design Mobility and Virtualization in the Data Center with LISP and OTV An important companion to this session is: BRKDCT Design consideration for Network and Security services stretched over multiple locations 4

5 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions IP Based Solutions LISP for DCI Deployments LISP and Path Optimization LISP as L3 DCI Summary and Q&A 5

6 Data Center Interconnect Business Drivers Data Centers are extending beyond traditional boundaries Virtualization applications are driving DCI across PODs (aggregation blocks) and Data Centers DCI Drivers Business Solution Constraints IT Technology Business Continuity Disaster Recovery HA Framework Stateless Network Service Sync Process Sync GSLB Geo-clusters HA Cluster Operation Cost Containment Data Center Maintenance / Migration / Consolidation Host Mobility Distributed Virtual Data Center Business Resource Optimization Disaster Avoidance Workload Mobility VLAN Extension Statefulness Bandwidth & Latency VM Mobility Cloud Services Inter-Cloud Networking XaaS Flexibility Application mobility VM Mobility Automation 6

7 Data Center Interconnect LAN Extension Model Path Optimization Any type of links Dual-Homing STP Domain isolation + Storm-control GW STP domain STP domain STP domain ALT GW ALT GW ALT Si Si Si Si DC1 DC2 DC3 Storage extension 7

8 LAN Extension for DCI VLAN Types Type T0 Limited to a single access layer Type T1 Extended inside an aggregation block (POD) Type T2 Extended between PODs part of the same DC site OTV/VPLS Type T3 Extended between twin DC sites connected via dedicated dark fiber links Type T4 Extended between twin DC sites using non 5*9 connection Type T5 Extended between remote DC sites T0 T1 T4 T3 T2 Fabric-path / vpc OTV/VPLS Fabric-path / vpc 8

9 LAN Extension for DCI Technology Selection Criteria Ethernet Over dark fiber or protected D-WDM VSS & vpc Dual site interconnection FabricPath (TRILL) Campus style MPLS MPLS Transport EoMPLS Transparent point to point A-VPLS Enterprise style MPLS H-VPLS Large scale & Multi-tenants SP style IP IP Transport OTV Enterprise style Inter-site MAC Routing IP style 9

10 LAN Extension for DCI Technology Selection Criteria Transport Fiber LOS report / Protected DWDM L2 SP offer (HA= ) IP Scale Site VLAN (10 2 or 10 3 or 10 4 ) MAC (10 3 or 10 4 or 10 5 ) Multi-tenants Tagging (VLAN / 2Q / VRF) Overlapping / Translation Multi-point or point to point Greenfield vs. Brownfield Ethernet only for 5*9 HA link MPLS/IP for WAN quality link Ethernet for medium scale IP for low scale MPLS for high scale MPLS for multi-tenancy features 10

11 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions VSS, vpc and FabricPath MPLS Based Solutions IP Based Solutions LISP for DCI Deployments LISP and Path Optimization LISP as L3 DCI Summary and Q&A 11

12 Dual Site Interconnection Leveraging EtherChannel between Sites On DCI Etherchannel: STP Isolation (BPDU Filtering) Broadcast Storm Control FHRP Isolation Primary Root Primary Root interface port-channel10 desc DCI point to point connection switchport switchport mode trunk vpc 10 switchport trunk allowed vlan spanning-tree port type edge trunk spanning-tree bpdufilter enable storm-control broadcast level 1 storm-control multicast level x Si Si L 2 L 3 WAN L 3 L 2 Link utilization with Multi- Chassis EtherChannel DCI port-channel - 2 or 4 links Requires protected DWDM or Direct fibers Server Cabinet Pair 1 Server Cabinet Pair N Server Cabinet Pair 1 Server Cabinet Pair N vpc does not support L3 peering: Use dedicated L3 Links for Inter-DC routing! Validated design: 200 Layer 2 VLANs VLAN SVIs 1000 VLAN SVI (static routing) 12

13 FabricPath is primarily positioned for Clos-based architectures L2 DCi is NOT LAN Switching! Is FabricPath a valid solution for DCi? DC Site 1 DC Site 2 DC Site 3 DC Site 4 Perception on FabricPath DCi Plug and play No Spanning Tree events shared between DC sites Can do IP routing over FP DCi One single protocol to manage end to end One single Fabric end to end Works also with N5K only scenarios 13

14 FabricPath DCI - Lessons learned Dependencies with L1 WAN links - Requires point to point high quality connections - Golden rule : WAN links must support Remote Port Shutdown and micro flapping protection Multidestination traffic impacts - Must tune multicast tree to avoid local traffic to fly over root tree site - Cannot avoid multicast to fly over root tree site for DCI multicast IP routing over FabricPath - Ship in the night effect - OSPF hellos are multicast and will fly over root site S1 R1 Site A Root MDT1 Site C Site B R2 STP interactions with FabricPath DCI - The Fabric becomes STP root for all propagated VLAN, means that twin site vpc will be blocking FabricPath & HSRP Localization - HSRP Control-plane can be isolated with mismatching authentication key - But HSRP data-plane cannot be isolated when DC is also FP, leading to flapping vmac High Availability - L2 ISIS fine tuning is required: allocate-delay timer, transition-delay, linkup-delay, spf-interval, lsp-gen-interval - Sub second convergence, except node recovery in 3s

15 FabricPath DCI - Key Takeaways FabricPath Customer references Operations simplicity Domino effect prevention DCi link quality mgmt 3+ Sites optimization High Availability L2 functions L3 Unicast functions Multicast functions Scalability On DCi, FabricPath is not so Plug and Play actually No specific DCI functions compared to OTV, VPLS Several designs gotchas but do not impact all customers Multidestination Trees capacity planning may be very complex Multiple Topologies will enhance the overall solution By default, OTV/VPLS should be the first solutions to promote Cisco Validated Designs (CVDs) Specific DCi features Offer an efficient independence between DC FabricPath is a valid DCi solution when : Short distances between DCs (tromboning is not a issue) Multicast is not massively used 15

16 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions EoMPLS VPLS H-VPLS IP Based Solutions LISP for DCI Deployments LISP and Path Optimization LISP as L3 DCI Summary and Q&A 16

17 EoMPLS Port Mode xconnect interface PE1 T-LDP PE2 interface g1/1 description EoMPLS port mode connection no switchport no ip address xconnect vcid 1 encapsulation mpls interface LSP Label VC Label Ethernet Header Ethernet Payload DA SA 0x8847 FCS

18 EoMPLS Usage for DCI End-to-End Loop Avoidance using Edge to Edge LACP On DCI Etherchannel: STP Isolation (BPDU Filtering) Broadcast Storm Control FHRP Isolation Active PW MPLS Core Aggregation Layer DC1 DCI Active PW DCI Aggregation Layer DC2 Encryption Services with 802.1AE Requires a full meshed vpc 4 PW 18

19 EoMPLS Usage for DCI Over IP Core Active PW IP Core Aggregation Layer DC1 DCI Active PW DCI Aggregation Layer DC2 crypto ipsec profile MyProfile set transform-set MyTransSet interface Tunnel100 ip address ip mtu 9216 mpls ip tunnel source Loopback100 tunnel destination tunnel protection ipsec profile MyProfile 19

20 Dealing with PseudoWire (PW) Failures Remote Ethernet Port Shutdown PE receives the PW down notification and shutdown its transmit signal toward aggregation X Active PW X MPLS Core X Si Si Aggregation Layer DC1 DCI Active PW DCI Aggregation Layer DC2 ASR1000 / ASR903 feature configuration: Failover (msec) Fallback (msec) interface GigabitEthernet1/0/0 xconnect pw-class eompls Bridged traffic remote link failure notification! (default) 20

21 EoMPLS Deployment on VSS Point to Point EoMPLS with Port-Channel xconnect Local LACP Local LACP Si One PW MPLS Si Si Si Aggregation Layer DC1 VSS VSS Aggregation Layer DC2 Instead of xconnecting physical port, xconnect port-channel LACP is kept local, no more extended over EoMPLS PW is virtual on both VSS members SSO protection in 12.2(33)SXJ Requires VSS or Nexus as DC device Limited support of L3 routing with vpc 21

22 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions EoMPLS VPLS H-VPLS IP Based Solutions LISP for DCI Deployments LISP and Path Optimization LISP as L3 DCI Summary and Q&A 22

23 Multi-Point Topologies What is VPLS? VLAN PW VFI VLAN SVI VFI MPLS Core PW PW SVI One extended bridge-domain built using: VFI = Virtual Forwarding Instance ( VSI = Virtual Switch Instance) PW = Pseudo-Wire SVI = Switch Virtual Interface xconnect VFI SVI VLAN Mac address table population is pure Learning-Bridge 23

24 VPLS Cluster Solutions Using clustering mechanism Two devices in fusion as one VSS Sup720 VSS Sup2T ASR9K nv virtual cluster One control-plane / two data-planes Dual node is acting as one only device Native redundancy (SSO cross chassis) Native load balancing Capability to use port-channel as attachment circuit SUP720+ES SUP2T ASR9K nv 24

25 VPLS Redundancy Making Usage of Clustering Si X Si Si mpls ldp session protection Si mpls ldp router-id Loopback100 force VSS Bridged traffic Failover (msec) Fallback (msec) LDP session protection & Loopback usage allows PW state to be unaffected LDP + IGP convergence in sub-second Fast failure detection on Carrier-delay / BFD Immediate local fast protection Traffic exit directly from egress VSS node 25

26 VPLS Redundancy Making Usage of Clustering X Si Si mpls ldp graceful-restart Si Si VSS Bridged traffic Failover (msec) Fallback (msec) If failing slave node: PW state is unaffected If failing master node: PW forwarding is ensured via SSO PW state is maintained on the other side using Graceful restart Edge Ether-channel convergence in sub-second Traffic is directly going to working VSS node Traffic exits directly from egress VSS node Quad sup SSO for SUP2T in 1QCY13 26

27 VPLS Deployment Considerations Symmetry is Good Problem / Solution sh ip route Known via "ospf 2 via GigabitEthernet1/3/0/1 Route metric is 2 via GigabitEthernet2/3/0/1 Route metric is 2 X Remote VSS are having two un-equal cost path to others, so one only route is put in RIB Stops forwarding traffic for 2mn when primary route is removed (there is no control-plane to insert backup route) Build a symmetric core with two ECMP paths between each VSS 27

28 VSS - A-VPLS CLI SUP2T in 15.1SY Si #sh mpls l2 vc Local intf Local circuit Dest address VC ID Status VFI VFI_610_ VFI UP VFI VFI_610_ VFI UP VFI VFI_611_ VFI UP VFI VFI_611_ VFI Si UP Si Rem: One PW per VLAN per destination Si interface Virtual-Ethernet1 switchport switchport mode trunk switchport trunk allowed vlan transport vpls mesh neighbor pw-class Core neighbor pw-class Core pseudowire-class Core encapsulation mpls Si Si Any card type facing edge SUP720 + SIP-400 facing core (5Gbps) or SUP720 + ES-40 (40Gbps) support with 12.2(33)SXJ SUP2T 28

29 VSS - A-VPLS over GRE SUP2T in 15.1SY Si One GRE tunnel per site Si Si Si int tunnel 1 tunnel mode gre ip mpls ip tunnel source tunnel destination interface virtual-ethernet 1 transport vpls mesh neighbor pw-class cl1 switchport switchport mode trunk switchport trunk allowed vlan 10, 20 ip route Tunnel1 Si Si Native on SUP2T 29

30 ASR9K VPLS Set-up PW l2vpn router-id bridge group BG bridge-domain BD interface TenGigE0/0/0/4 interface TenGigE0/0/0/5! vfi VFI vpn-id 4003 neighbor pw-id

31 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions EoMPLS VPLS H-VPLS IP Based Solutions LISP for DCI Deployments LISP and Path Optimization LISP as L3 DCI Summary and Q&A 31

32 DC Access Multi-Homing Inter Chassis Communication Protocol - ICCP draft-ietf-martini-pwe3-iccp C7600 SRE with ES facing edge ASR9K XR4.0 ASR903 Q3CY13 Redundancy Group Active POA DHD ICCP MPLS ICCP synchronizes event/states between multiple chassis in a redundancy group ICCP runs over reliable LDP / TCP ICCP relies on BFD/IP route-watch as keepalive ICCP message to synch state Ex: LACP, IGMP query Standby POA Terminology: mlacp : Protocol MC-LAG : DHD : DHN : POA : Multi-Chassis Link Aggregation Control Multi-Chassis Link Aggregation Group Dual Homed Device (Customer Edge) Dual Homed Network (Customer Edge) Point of Attachment (Provider Edge) 32

33 DC Access Multi-Homing Inter Chassis Communication Protocol - ICCP Multi-Chassis LACP synchronization: LACP BPDUs (01:80:C2:00:00:00) are exchanged on each Link System Attributes: Priority + bundle MAC Address Port Attributes: Key + Priority + Number + State Redundancy Group Active POA redundancy iccp group <ig-id> mlacp node <node id> mlacp system mac <system mac> mlacp system priority <sys_prio> member neighbor <mpls device> DHD ICCP Standby POA MPLS interface <bundle> mlacp iccp-group <ig-id> mlacp port-priority <port prio> interface <physical interface> bundle id <bundle id> mode active Terminology: mlacp : Protocol MC-LAG : DHD : DHN : POA : Multi-Chassis Link Aggregation Control Multi-Chassis Link Aggregation Group Dual Homed Device (Customer Edge) Dual Homed Network (Customer Edge) Point of Attachment (Provider Edge) 33

34 MC-LAG to VPLS Testing Si MPLS core Si Only error 2/3/4 are leading to ICCP convergence Rem: 2 & 4 are dual errors 500 VLAN Unicast: Link error sub-1s & Node error sub-2s 1200 VLAN unicast: Link error sub-2s & Node error sub-4s 34

35 Flexible VLAN Handling Ethernet Virtual Circuit - EVC 1. Selective Trunk Support Group multiple VLAN in one only core bridge domain QinQ model or PBB Model VLAN overlapping 2. VLAN translation 121 / 222 / Inter-DC VLAN numbering independency 3. Scale to 4000 * 4000 VLAN Scale above 4000 VLAN 4. Routing for multi-tag Multi-tenant default gateway IRB - IP routing / VRF routing for QinQ tagged frames 35

36 E-VPN (aka Routed VPLS) Main Principles Control-Plane Distribution of Customer MAC- Addresses using BGP PE continues to learn C-MAC over AC BGP When multiple PEs announce the same C-MAC, hash to pick one PE PE PE MP2MP/P2MP LSPs for Multicast Traffic Distribution MP2P (like L3VPN) LSPs for Unicast Distribution PE PE Full-Mesh of PW no longer required!! 36

37 Nexus Data Center Interconnect with VPLS Q3 CY 2013 Layer 2 switchport Trunk Portchannel Primary N7K WAN Edge VLAN tied to Active VFI with neighbors to remote DC sites Vlan X VLAN X VFI Virtual Port Channel (vpc) MCT VLAN tied to Standby VFI with neighbors to remote DC sites Vlan X VLAN X VFI Secondary N7K WAN Edge MCT = Multi-chassis Trunk Interface VFI = Virtual Forwarding Instance

38 Data Center Interconnect with VPLS Sample Configuration Nexus 7000 PE EVEN VLANs PE Primary VFI owner for EVEN vlans Secondary owner for PE 1 PE 2 vlan 80-81! ODD vlans vlan configuration 80 member vfi vpls-80! vlan configuration 81 member vfi vpls-81! l2vpn vfi context vpls-80 vpn id 80 redundancy primary member encapsulation mpls member encapsulation mpls! l2vpn vfi context vpls-81 vpn id 81 redundancy secondary member encapsulation mpls member encapsulation mpls! interface port-channel50 switchport mode trunk switchport trunk allowed vlan 80,81 Note: Virtual Port Channel (vpc) configuration not shown vlan 80-81! vlan configuration 80 member vfi vpls-80! vlan configuration 81 member vfi vpls-81! l2vpn vfi context vpls-80 vpn id 80 redundancy secondary Primary VFI owner for ODD vlans Secondary owner for EVEN vlans member encapsulation mpls member encapsulation mpls! l2vpn vfi context vpls-81 vpn id 81 redundancy primary member encapsulation mpls member encapsulation mpls! interface port-channel50 switchport mode trunk switchport trunk allowed vlan 80,81 vpc vpc VFI VFI PE PE VFI VFI PE ODDVLANs VFI VFI PE PE VFI VFI PE

39 Nexus Layer 3 + Layer 2 Extension Dual VDC or additional VPLS PE layer required for L3 and L2 extension for the same vlan Double-sided vpc design (dual vpc peer links) No VPLS and IRB support vpc Operational Primary Active VFI Vlan 100 PE1 VFI 100 VFI 200 Peer Link VFI 100 VFI 200 PE2 vpc Operational Secondary Standby VFI Vlan 100 AGG A vpc Operational Primary Primary Root Vlan 100 AGG A DP DP CFS RP Peer Link DP AGG B AGG B vpc Operational Secondary Secondary Root Vlan , flow 1 100, flow 2 RP Access X 39

40 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions IP Based Solutions OTV Technology Overview OTV Deployment Considerations LISP for DCI Deployments LISP and Path Optimization LISP as L3 DCI Summary and Q&A 40

41 Overlay Transport Virtualization Technology Pillars OTV is a MAC in IP technique to extend Layer 2 domains OVER ANY TRANSPORT Dynamic Encapsulation No Pseudo-Wire State Maintenance Optimal Multicast Replication Multipoint Connectivity Point-to-Cloud Model Nexus 7000 First platform to support OTV (since 5.0 NXOS Release) ASR 1000 Now also supporting OTV (since 3.5 XE Release) Protocol Learning Preserve Failure Boundary Built-in Loop Prevention Automated Multi-homing Site Independence 41

42 Overlay Transport Virtualization OTV Control Plane Edge Device (ED): connects the site to the (WAN/MAN) core and responsible for performing all the OTV functions Internal Interfaces: L2 interfaces (usually 802.1q trunks) of the ED that face the site Join Interface: L3 interface of the ED that faces the core Overlay Interface: logical multi-access multicast-capable interface. It encapsulates Layer 2 frames in IP unicast or multicast headers OTV Overlay Interface Internal Interfaces L2 L3 Join Interface Core 42

43 OTV Data Plane Inter-Site Packet Flow MAC TABLE VLAN MAC IF 100 MAC 1 Eth 2 Transport Infrastructure OTV OTV OTV OTV 100 MAC 2 Eth 1 Encap 100 MAC 2 IP A MAC 1 MAC 3 IP A IP B 100 MAC 3 IP B MAC 1 MAC 3 IP A IP B 100 MAC 3 Eth MAC 4 IP B IP A 3 4 Decap 5 IP B MAC TABLE VLAN MAC IF 100 MAC 1 IP A 100 MAC 4 Eth 4 6 Layer 2 Lookup MAC 1 MAC 3 1 MAC 1 MAC 3 West East Server 1 Site Site Server

44 Overlay Transport Virtualization OTV Control Plane Neighbor discovery and adjacency over Multicast (Nexus 7000 and ASR 1000) Unicast (Adjacency Server Mode currently available with Nexus 7000 from 5.2 release) OTV proactively advertises/withdraws MAC reachability (control-plane learning) IS-IS is the OTV Control Protocol - No specific configuration required VLAN MAC IF 4 3 New MACs are learned on VLAN 100 Vlan 100 MAC A 1 OTV updates exchanged via the L3 core MAC A IP A 100 MAC B IP A 100 MAC C IP A Vlan 100 Vlan 100 MAC B MAC C 2 East West IP A 3 VLAN MAC IF 100 MAC A IP A 100 MAC B IP A MAC C IP A South 44

45 OTV Failure Domain Isolation Spanning-Tree Site Independence Site transparency: no changes to the STP topology Total isolation of the STP domain Default behavior: no configuration is required BPDUs sent and received ONLY on Internal Interfaces OTV OTV The BPDUs stop here The BPDUs L3 stop here L2 45

46 OTV Failure Domain Isolation Preventing Unknown Unicast Storms No requirements to forward unknown unicast frames Assumption: end-host are not silent or uni-directional Default behavior: no configuration is required MAC TABLE VLAN MAC IF OTV OTV 100 MAC 1 Eth1 100 MAC 2 IP B L3 L No MAC 3 in the MAC Table MAC 1 MAC 3 46

47 OTV Multi-homing VLANs Split Across AEDs Automated and deterministic algorithm (not configurable) In a dual-homed site: Lower IS-IS System-ID (Ordinal 0) = EVEN VLANs Higher IS-IS System-ID (Ordinal 1) = ODD VLANs Future functionality will allow to tune the behavior Remote OTV Device MAC Table VLAN MAC IF 100 MAC 1 IP A 101 MAC 2 IP B OTV-a# show otv vlan OTV Extended VLANs and Edge Device State Information (* - AED) VLAN Auth. Edge Device Vlan State Overlay East-b inactive(non AED) Overlay * East-a active Overlay East-b inactive(non AED) Overlay100 OTV-b# show otv vlan OTV Extended VLANs and Edge Device State Information (* - AED) IP A AED ODD VLANs OTV OTV-a Overlay Adjacency Site Adjacency* OTV OTV-b Internal peering for AED election IP B AED EVEN VLANs VLAN Auth. Edge Device Vlan State Overlay * East-b active Overlay East-a inactive(non AED) Overlay * East-b active Overlay100 *Supported from 5.2 NX-OS release 47

48 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions IP Based Solutions OTV Technology Overview OTV Deployment Considerations LISP for DCI Deployments LISP and Path Optimization LISP as L3 DCI Summary and Q&A 48

49 OTV and SVI Routing Introducing the OTV VDC Guideline: The current OTV implementation on the Nexus 7000 enforces the separation between SVI routing and OTV encapsulation for any extended VLAN This separation can be achieved with having two separate devices to perform these two functions An alternative cleaner and less intrusive solution is the use of Virtual Device Contexts (VDCs) available with Nexus 7000 platform: A dedicated OTV VDC to perform the OTV functionalities The Aggregation-VDC used to provide SVI routing support L3 L2 OTV VDC OTV VDC Aggregation 49

50 Placement of the OTV Edge Device OTV in the DC Aggregation L2-L3 boundary at aggregation DC Core performs only L3 role STP and L2 broadcast Domains isolated between PODs Intra-DC and Inter-DCs LAN extension provided by OTV Requires the deployment of dedicated OTV VDCs Ideal for single aggregation block topologies Recommended for Green Field deployments Nexus 7000 required in aggregation SVIs SVIs SVIs SVIs vpc vpc 50

51 Single Homed OTV VDC Simple Model OTV VDC OTV VDC Link-1 Link-3 Link-1 N7K-A Routing VDC N7K-A Po1 Po1 Routing VDC Logical View Physical View N7K-B N7K-B Link-2 OTV VDC OTV VDC Link-2 Link-4 May use a single physical link for Join and Internal interfaces Minimizes the number of ports required to interconnect the VDCs Single link or physical node (or VDC) failures lead to AED re-election 50% of the extended VLANs affected Failure of the routed link to the core is not OTV related Recovery is based on IP convergence Layer 3 Layer 2 51

52 Dual Homed OTV VDC Improving the Design Resiliency N7K-A N7K-B OTV VDC OTV VDC Links 1-2 Routing VDC N7K-A Po1 Logical View OTV VDC Links 1-2 Po1 Links 3-4 Link 5 Link 7 Link 6 Link 8 Routing VDC Physical View Layer 3 Layer 2 N7K-B Links 3-4 OTV VDC Logical Port-channels used for the Join and the Internal interfaces Increases the number of physical interfaces required to interconnect the VDCs Traffic recovery after single link failure event based on port-channel re-hashing No need for AED re-election Physical node (or VDC) failure still requires AED re-election In the current implementation may cause few seconds of outage (for 50% of the extended VLANs) 52

53 OTV in the DC Aggregation Site Based Per-VLAN Load Balancing OTV VDC Simple Appliance Model Aggregation OTV VDC AED role negotiated between the two OTV VDCs (on a per VLAN basis) Internal IS-IS peering on the site VLAN AED Recommended to carry the site VLAN on vpc links and vpc peer-link For a given VLAN all traffic must be carried to the AED Device Part of the flows carried across the vpc peer-link Most Resilient Model Optimized traffic flows is achieved in the most resilient model leveraging Port-Channels as Internal Interfaces OTV VDC Aggregation OTV VDC The AED encapsulates the original L2 frame into an IP packet and send it back to the aggregation layer device AED The aggregation layer device routes the IP packet toward the DC Core/WAN edge L3 routed traffic bypasses the OTV VDC 53

54 OTV in the DC Aggregation Using F-Series Linecards F1 and F2 linecards do not support OTV natively As of today, the OTV VDC must use only M-series ports for both Internal and Join Interfaces Recommendation is to allocate M1 only interfaces to the OTV VDC Native OTV support on F-series is targeted for 6.2 release (Q2CY13) 55

55 OTV in the DC Aggregation Configuration (Multicast Transport) PIM enabled interfaces Routing VDC hostname routing-vdc! interface Ethernet1/1 switchport switchport mode trunk switchport trunk allowed vlan 100, ! interface Ethernet2/1 ip address /24 ip router ospf 1 area ip ospf passive-interface ip pim sparse-mode ip igmp version 3! ip pim rp-address group-list /4 ip pim ssm range /8 OTV VDC Routing VDC N7K-Agg1 L3 Link L2 Link Routing VDC e2/1 OTV VDC e2/2 e1/1 e1/2 N7K-Agg2 Establish L3 peering on a dedicated VLAN OTV VDC hostname otv-vdc feature otv! otv site-vlan 100! interface Ethernet1/2 description Internal Interface switchport switchport mode trunk switchport trunk allowed vlan 100, ! interface Ethernet2/2 description Join Interface ip address /24 ip igmp version 3! interface Overlay100 otv join-interface Ethernet2/2 otv control-group otv data-group /24 otv extend-vlan ! ip route Cisco and/or its affiliates. ** All Could rights use reserved. static default route or ospf stub 56

56 OTV in the DC Aggregation Configuration (Unicast Transport) Routing VDC hostname routing-vdc! interface Ethernet1/1 switchport switchport mode trunk switchport trunk allowed vlan 100, ! interface Ethernet2/1 ip address /24 ip router ospf 1 area ip ospf passive-interface Release 5.2 and above OTV VDC Routing VDC N7K-Agg1 L3 Link L2 Link Routing VDC e2/1 OTV VDC e2/2 e1/1 e1/2 N7K-Agg2 Establish L3 peering on a dedicated VLAN OTV VDC hostname otv-vdc feature otv! otv site-vlan 100! interface Ethernet1/2 description Internal Interface switchport switchport mode trunk switchport trunk allowed vlan 100, ! interface Ethernet2/2 description Join Interface ip address /24! interface Overlay100 otv join-interface Ethernet2/2 otv adjacency-server* otv use-adjacency-server otv extend-vlan ! ip route * Needed only on the Adjacency Server 57

57 Placement of the OTV Edge Device Connecting Brownfield and Greenfield Data Centers OTV OTV Greenfield Greenfield ASR 1K Brownfield L3 OTV Nexus 7K OTV L2 Si Si OTV OTV OTV OTV Nexus 7K Nexus 7K L3 L2 Nexus 7K Si Si L3 L2 L3 L2 FabricPath OTV Virt. Link Leverage OTV capabilities on Nexus 7000 (Greenfield) and ASR 1000 (Brownfield) Build on top of the traditional DC L3 switching model (L2-L3 boundary in Agg, Core is pure L3) Possible integration with the FabricPath/TRILL model 60

58 6.2 (Q2CY13) OTV New functionality Key features for larger DCI usage Selective Unicast Flooding (for unidirectional MACs & silent hosts) OTV VLAN translation Dedicated Data Broadcast Multicast Group Multiple Uplinks / Loopback ED IP Scalability to large deployment Fast convergence AED synchronization Fast remote convergence using Site-ID Fast local convergence using pre-population Fast ED failure detection using BFD & route tracking F1 and F2e as internal interfaces (proxy mode)

59 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions IP Based Solutions LISP for DCI Deployments LISP and Path Optimization LISP as L3 DCI Summary and Q&A 62

60 Path Optimization and DCI Avoid Suboptimal Traffic Path After Workload Motion /25 & /25 EEM or RHI can be used to get very granular ISP A DC A Layer 3 Core Ingress Path Optimization: Clients-Server ISP B DC B Agg Server-Server Path Optimization Public Network VLAN A Agg Access DB Move the whole application tier Optimize the whole path: Client to Server Server to Server Server to Client Access Front-End Data-Base Egress Path Optimization: Server-Client L2 Links (GE or 10GE) L3 Links (GE or 10GE) Egress Path Optimization: Server-Client 63

61 Inbound Path Optimization LISP Host Mobility 1 DNS Entry: D.abc.com A Mapping Cache Entry EID-prefix: /32 Locator-set: , priority: 1, weight: 50 (D1) , priority: 1, weight: 50 (D2) > /24 LISP Site S ITR IP Network > > > ETR West-DC D LAN Extension /24 Mapping DB East-DC 64

62 Inbound Path Optimization LISP Host Mobility DNS Entry: D.abc.com A EID-prefix: /32 Locator-set: , , priority: 1, weight: 50 (D1) , , priority: 1, weight: 50 (D2) 7 Mapping Cache Entry Update /24 LISP Site S ITR > > IP Network Mapping DB ETR West-DC D LAN Extension / Workload Move East-DC 65

63 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions IP Based Solutions LISP for DCI Deployments L3 Host Mobility using LISP LISP as L3 DCI Summary and Conclusions Q&A 67

64 Use Case Focus for LISP as L3 DCI Data-Center migration Capability to install a device in a brownfield DC to ensure subnet continuity for migration to a greenfield DC VM migration / Physical migration with no change on workload IP address Hybrid Cloud Insertion of SP resource in customer local subnet (like SaaS) Cloud bursting (provisioning of resource in Cloud) Migration Backup services Partial Disaster Recovery This require capability of moving back resource to the original site

65 Enable LISP on a stick MS/MR PiTR with Mobility on a stick NOT default gateway Does not receive any traffic before move LISP ETR (using M1-32) on a stick Default gateway for the moved traffic Does not receive any traffic before the move ETR PxTR / /24 Brownfield DC Greenfield DC This is ASM mode with same subnet value both side Home subnet is Greenfield (registers with MS /24), Dynamic part is Brownfield (detects and registers any /32) 69

66 Packet Flow from Client & Server in Brownfield North-South Traffic Traffic to a non moved resource does not reach LISP nodes on a stick 70

67 Symmetric Packet Flow from Client & Server in Greenfield North-South Traffic PeTR Existence of a Firewall between WAN edge & PxTR requires symmetrical flow Use PeTR PeTR allows return flow to go thru LISP Path, nevertheless it requires ETR to work with default routing 71

68 Convergence Considerations All Failures are Leading to Sub 3s Convergence with IGP/BGP tuning There are three mechanisms to handle convergence Route watch / Route notification Mandate RLOC /32 to be received remotely This /32 must not be part of an aggregated route As fast convergence as the routing protocol The one used in this solution EID/RLOC probing Probes every EID 60s convergence LSB bits Data-plane bits indicating local RLOC status Not supported with M

69 Agenda DCI Business Drivers and Solutions Overview LAN Extension Deployment Scenarios Ethernet Based Solutions MPLS Based Solutions IP Based Solutions LISP for DCI Deployments L3 Host Mobility using LISP LISP as L3 DCI Summary and Q&A 73

70 Data Center Interconnect - DCI Model Connecting Virtualized Data Centers L2 Domain Elasticity - LAN Extension STP Isolation is the key element Multipoint Loop avoidance + Storm-Control Unknown Unicast & Broadcast control Link sturdiness Scale & Convergence OTV OTV Path Optimization - Optimal Routing - Route Portability Considerations Network and Security services deployment Server-Client Flows Server-Server Flows Path Optimization Options Egress Addressed by FHRP Filtering Ingress: Addressed by LISP OTV Storage Elasticity - SAN Extensions Sync or Async replication modes are driven by the applications, hence the distance/latency is a key component to select the choice Localization of Active Storage is key Distance can be improved using IO accelerator or caching Virtual LUN is allowing Active/Active OTV VN-link notifications 74

71 Data Center Interconnect Where to Go for More Information

72 Recommended Reading for 76

73 Call to Action Visit the Cisco Campus at the World of Solutions to experience Cisco innovations in action Get hands-on experience attending one of the Walk-in Labs Schedule face to face meeting with one of Cisco s engineers at the Meet the Engineer center Discuss your project s challenges at the Technical Solutions Clinics

74

75