Flexible Data Centre Fabric - FabricPath/TRILL, OTV, LISP and VXLAN

Transcription

1 Flexible Data Centre Fabric - FabricPath/TRILL, OTV, LISP and VXLAN Ron Fuller CCIE #5851 (R&S/Storage) Technical Marketing Engineer, Nexus 7000 rfuller@cisco.com

2 Agenda The Evolving Data Centre Fabric FabricPath VXLAN LISP 1K Cisco Nexus x8 6 LISP Host Mobility OTV LAN Extension Mobility with Extended Subnets Nexus Fabric 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3

3 Goals of the Fabric Addressing Concurrent Workloads, Mobility and Latency Port Density Adequate Buffer Capacity Adequate Table Sizes Low Latency Switching Cut-through Switching : : Priority Flow Control Early Congestion Notification FabricPath Multiple Trees ECMP L2 & L3 Multi-tenancy : : Architecture is evolving Rapidly in the next 24 months L2/L3 Boundary becomes less relevant Clos Topologies dominate new implementations HA models shift Server Edge becomes more intelligent DC Fabric becomes more scalable 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4

4 Goals of the Fabric Addressing High Availability and Fate Sharing L3 L3/L2 L3/L2 L2 L2 East-West traffic Fate Sharing Domain STP is the protocol of choice 1+1 redundancy limited forwarding paths East-West across L3 boundaries OSPF/EIGRP are protocols of choice N+1 redundancy Broad forwarding Paths North-South traffic OSPF/EIGRP are protocols of choice N+1 redundancy Broad forwarding paths Larger POD East-West Traffic Fate Sharing Domain N+1 redundancy IS-IS is the protocol of choice Broad forwarding paths Broader Adjacency Support Same number of physical boxes and links Protocol behavior is L3-like Multi-pathing over L2 and L3 More flexible L2 adjacency, better scale capacity Better latency consistency within POD 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5

5 Goals of the Fabric Not a L2 vs. L3 debate L2/L3 The traditional L2 vs. L3 debate has been based on a number of issues Scalability Availability Requirements for the scalable design moving forward is a scalable, highly available switching fabric with the advantages of both L2 and L Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6

6 Plug-and-Play and Mobility vs. Availability and Scaling Advantages of Layer 2 Practically plug-n-play No user configuration is required to build forwarding database It makes it simple to support teaming or L2 multicast for clusters Easy to segment traffic with VLANs Very fast movement of end station addresses (ability to update MAC address tables after a vmotion-type event) MAC Table A Disadvantages of Layer 2 MAC address consumption BPDU generation is CPU intensive with increasing number of VLANs VLAN sprawl causes flooding and broadcasts to propagate even where they are not needed Half of the links in the topology are blocking Misconfigurations can cause Layer 2 loops which may make switches unmanageable MAC Table A MAC Table A MAC Table A MAC Table 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7 A Layer 2 Domain MAC Table A

7 Availability and Scaling vs. Restricted Workload Flexibility Layer 3 Routed Topologies alleviate the consumption of L2 tables via route summarization Layer 3 Routed topologies provide for a degree of fault isolation and Routed Access provides the logical extension of the design philosophy Scaling Up of the Access Switch via such mechanism as the FEX provide a degree of workload mobility L2 domain extension of some form is required for most workload mobility requirements 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8 L3 L2 Workload Domain for most Hypervisor and Clustering based solutions is restricted by the Traditional Layer 2/3 boundary

8 Segment-ID: Scaling Logical Groupings of Connectivity S1 Web Server S2 App Server S3 Database Server S Q VLAN ID 12-bits VLAN ID 12-bits 802.1Q SegmentId 24-bits VLAN ID 12-bits VLAN ID 12-bits 802.1ad standardized frame format 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9

9 Location Identity Separation Location Identity L2/L3 Fabric Location reachability determined by traditional routing mechanisms in the Fabric Identity is mapped to location addresses All these technologies leverage Location/Identity Mapping Location Identity Multi-tenancy FabricPath / TRILL Switch-ID (IS-IS) Client MAC (Flooding) VXLAN OTV LISP IP address (IP protocols) Client MAC (Flooding) IP address (IP protocols) Client MAC (IS-IS) 24-bit Segment Identifier IP address (IP protocols) Client IP/MAC (Mapping DB) 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10

10 Scale FabricPath, LISP, VXLAN & OTV Requirement Intra-DC Inter-DC Layer 2 connectivity FabricPath/TRILL/VXLAN OTV/VPLS IP Mobility LISP LISP Secure Segmentation VXLAN / Segment-ID VPNs (LISP/MPLS) DC-west LISP IP mobility IP Network DC-east POD POD POD POD App OS App OS App OS OTV/VPLS (Inter-DC x-l3) App OS App OS App OS Fabric Path (Intra-DC L2) VXLAN/OTV (Intra-DC x-l3) Fabric Path (Intra-DC L2) VXLAN/OTV (Intra-DC x-l3) 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11

11 Agenda The Evolving Data Centre Fabric FabricPath VXLAN LISP 1K Cisco Nexus x8 6 LISP Host Mobility OTV LAN Extension Mobility with Extended Subnets Nexus Fabric 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12

12 Cisco FabricPath NX-OS Innovation Enhancing L2 with L3 Switching Easy Configuration Plug & Play Provisioning Flexibility Routing Multi-pathing (ECMP) Fast Convergence Highly Scalable FabricPath FabricPath brings Layer 3 routing benefits to flexible Layer 2 bridged Ethernet networks 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13

13 MAC-in-MAC Creates hierarchical layer 2 address scheme with additional MAC header Source and destination Switch_ID written into outer MAC header at L2MP edge Forwarding inside L2MP core network is based on destination Switch_ID Embedded path selector (FTAG) provides multi-pathing for even broadcast and multicast Built-in protections (TTL and multicast RPF) minimize impact of transient network issues Optimal MAC Learning Prevent potential MAC table overflow in large scale L2 domain Traditional source-learning only on Edge port for locally connected MAC addresses Learning is disabled on Core port to reduce MAC table utilization Non-local source-mac only learned if destination-mac is already learned as local entry IS-IS Scalable routing protocol with proven implementation for fast convergence upon network changes Link-state protocol ensures optimal path between any 2 nodes Built-in authentication mechanism enhances network security and stability Inherent support for ECMP and multi-topology maximize link utilization 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14

14 New Control Plane Plug-n-Play L2 IS-IS manages forwarding topology IS-IS assigns addresses to all FabricPath switches automatically Compute shortest, pair-wise paths Support equal-cost paths between any FabricPath switch pairs FabricPath Routing Table S10 S20 S30 S40 Switch S10 S20 S30 S40 S200 S400 IF L1 L2 L3 L4 L1, L2, L3, L4 L1, L2, L3, L4 FabricPath L1 L2 L3 L4 S100 S200 S300 S Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15

15 New Data Plane The association MAC address/switch ID is maintained at the edge S10 S20 S30 S40 Switch ID space: Routing decisions are made based on the FabricPath routing table A B S100 S300 FabricPath (FP) S100 S200 S300 S300: FabricPath Routing Table Switch S100 IF L1, L2, L3, L4 MAC adress space: Switching based on MAC address tables A 1/1 Classical Ethernet (CE) 1/2 B S300: CE MAC Address Table MAC IF B 1/2 A S100 Core fabric leverages an independent routing topology from the edge Scales MAC learning Scales Core topology state 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16

16 New Control and Data Plane Edge switches maintain both MAC address table and Switch ID table Ingress switch uses MAC table to determine destination Switch ID Egress switch uses MAC table (optionally) to determine output switchport S10 S20 S30 S40 FabricPath MAC Table on S100 Local MACs point to switchports Remote MACs point to Switch IDs MAC IF/SID A e1/1 B e1/2 C S101 D S200 S100 S101 FabricPath S200 MAC A MAC B MAC C MAC D 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17

17 New Control and Data Plane FabricPath IS-IS manages Switch ID (routing) table All FabricPath-enabled switches automatically assigned Switch ID (no user configuration required) Algorithm computes shortest (best) paths to each Switch ID based on link metrics Equal-cost paths supported between FabricPath switches S10 S20 S30 S40 One best path to S10 (via L1) FabricPath Routing Table on S100 Switch IF S10 S20 S30 L1 L2 L3 L1 L2 L3 L4 Four equal-cost paths to S101 S40 S101 L4 L1, L2, L3, L4 FabricPath S200 L1, L2, L3, L Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18 S100 S101 S200

18 Scaling Conversational Learning MAC IF A e1/1 B s8, e1/2 A FabricPath s3 s5 s8 e1/1 e1/2 MAC IF MAC IF A s1,e1/1 B e1/2 B Edge switch only learn the MAC of remote hosts when there are two way communications between remote hosts and local hosts Unknown unicast flooding alone won t have all switches within VLAN learn the source MAC Intermediate switches don t learn the MAC Hardware based MAC learning 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19

19 Cisco FabricPath Terminology FP Core Ports Interface connected to another FabricPath device Sends/receives traffic with FabricPath header Does not run spanning tree Does not perform MAC learning! Exchanges topology info through L2 ISIS adjacency Forwarding based on Switch ID Table S10 S20 S30 S40 Spine Switch FabricPath (FP) S100 S200 S300 Leaf Switch 1/1 1/2 Classical Ethernet (CE) A B CE Edge Ports Interface connected to traditional network device Sends/receives traffic in standard Ethernet frame format Participates in STP domain Forwarding based on MAC table 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20

20 Configuration Simplicity Automatically handled by IS-IS FabricPath V10 V20 V30 V30 V10 V20 V10 V30 V10 V20 V Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21

21 Root for Root for Tree 1 S10 S20 S30 Tree 2 S40 Multidestination traffic constrained to loop-free trees touching all FabricPath switches Root switch assigned for each multidestination tree in FabricPath domain Loop-free tree built from each Root and assigned a network-wide identifier (Ftag) S100 S101 FabricPath S200 S100 S20 Support for multiple multidestination trees provides multipathing for multidestination traffic Two trees supported in NX-OS release 5.1 S100 S10 S10 S101 S30 S40 S101 S20 Root Logical Tree 1 S200 S40 Logical Tree 2 S200 S Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22 Root

22 Multi-Topology Support FabricPath Topology 0 VLAN 20 (DC Wide) Common across entire Data Center FabricPath Topologies FabricPath Topology 1 VLAN 20 DC Wide VLAN 30 POD Local (and non-unique) VLAN 10 POD Local (and unique) FabricPath Topology 2 VLAN 20 DC Wide VLAN 30 POD Local (and non-unique) VLAN 40 POD Local (and unique) Extending FabricPath to the edge switches without requiring a redesign of the VLAN topology Each FP switch can have up to 2 Topology ID s defined (Topology ID s does not have to be unique). Each Topology will have 2 Multi-Destination Trees defined 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23

23 Mac-in-Mac Header Classical Ethernet Frame 16 bytes DMAC SMAC 802.1Q Etype Payload CRC Original CE Frame Cisco FabricPath Frame Outer DA (48) Outer SA (48) FP Tag (32) DMAC SMAC 802.1Q Etype Payload CRC (new) 6 bits bits bits 8 bits 16 bits 16 bits 10 bits 6 bits Endnode ID (5:0) U/L I/G Endnode ID (7:6) RSVD OOO/DL Switch ID Sub Switch ID LID Etype 0x8903 Ftag TTL Switch ID Unique number identifying each FabricPath switch Sub-Switch ID Identifies devices/hosts connected via VPC+ LID Local ID, identifies the destination or source interface Ftag (Forwarding tag) Unique number identifying topology and/or distribution tree TTL Decremented at each switch hop to prevent frames looping infinitely 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24

24 4 Ftag Putting it all together Host A to Host B (1) Broadcast ARP Request Multidestination Trees on Switch 10 Tree IF 1 po100,po200,po300 2 po100 DA FF Ftag 1 SA DMAC FF Root for Root for Tree 1 S10 S20 S30 Tree 2 S40 po100 po200 po300 DA FF Ftag 1 3 Broadcast FabricPath MAC Table on S100 MAC A Multidestination Trees on Switch 100 Tree IF/SID e1/13 (local) IF 1 po10 2 po10,po20,po30,po40 2 DMAC FF SMAC A Payload SMAC A Payload S100 S200 S300 1 MAC A po10 po20 e1/13 po30 po40 5 Ftag Multidestination Trees on Switch 300 po20 po30 po Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25 Tree FabricPath MAC Table on S200 MAC IF 1 po10,po20,po30,po40 2 po40 IF/SID po10 e2/29 MAC B SA DMAC FF SMAC A Payload 6 Payload SMAC A DMAC FF

25 Putting it all together Host A to Host B (1) Broadcast ARP Request S100: S100# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID * a dynamic 0 F F Eth1/13 MAC A learned as local entry on e1/13 S100# S10 (and S20, S30, S40, S200, S300): S10# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID MAC A not learned on other switches 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26

26 Ftag Ftag (2) Broadcast ARP Reply Multidestination Trees on Switch 10 Tree Multidestination Trees on Switch 100 Tree IF 1 po100,po200,po300 2 po100 IF 1 po10 2 po10,po20,po30,po40 FabricPath MAC Table on S100 MAC A B IF/SID e1/13 (local) (remote) DA MC1 Ftag 1 SA DMAC A SMAC B Payload Payload SMAC B 12 DMAC A MAC A Root for Root for Tree 1 S10 S20 S30 Tree 2 S40 po100 po10 po20 e1/13 po200 po30 po Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27 po300 S200 9 Unknown 8 FabricPath MAC Table on S300 MAC B Multidestination Trees on Switch 300 Tree MISS IF 1 po10,po20,po30,po40 2 po40 IF/SID e2/29 (local) po20 po30 po40 po10 e2/29 MAC B DA MC1 Ftag 1 SA DMAC A SMAC B Payload S300 7 DMAC A SMAC B Payload

27 Putting it all together Host A to Host B MAC Address Table after the first ARP frame S100: S100# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID * a dynamic 90 F F Eth1/ b dynamic 60 F F S100 learns MAC B as remote entry reached through S300 S100# S300: S300# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID b dynamic 0 F F Eth2/29 MAC B learned as local entry on e2/ Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28

28 FabricPath Routing Table on S30 Switch IF S10 S20 S30 S40 S300 B S300 FabricPath Routing Table on S100 FabricPath MAC Table on S100 MAC A B Switch S10 S20 S30 S40 S200 S300 S300 po IF po10 po20 po30 po40 po10, po20, po30, po40 po10, po20, po30, po40 IF/SID e1/13 (local) (remote) 14 DA Ftag 1 SA DMAC B SMAC A Payload 15 DMAC B SMAC A Payload 13 S100 Hash MAC A po10 po20 e1/13 po30 po40 If DMAC is known, then learn remote MAC 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29 S200 S300 FabricPath Routing Table on S300 Switch FabricPath MAC Table on S300 MAC A B po300 IF S300 Use LID (64) IF/SID e2/29 (local) po20 po30 po40 po10 17 S (remote) e2/29 18 MAC B DA Ftag 1 SA DMAC B SMAC A Payload S300 Payload SMAC A DMAC B

29 Putting it all together Host A to Host B Unicast forwarding S100# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID * a dynamic 90 F F Eth1/ b dynamic 60 F F S300# sh mac address-table dynamic Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID a dynamic 30 F F b dynamic 90 F F Eth2/29 S100 learns MAC A as remote entry reached through S Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30

30 Putting it all together Host A to Host B Unicast Forwarding S100# sh fabricpath route FabricPath Unicast Route Table 'a/b/c' denotes ftag/switch-id/subswitch-id '[x/y]' denotes [admin distance/metric] ftag 0 is local ftag subswitch-id 0 is default subswitch-id FabricPath Unicast Route Table for Topology-Default 0/100/0, number of next-hops: 0 via ----, [60/0], 0 day/s 04:43:51, local 1/10/0, number of next-hops: 1 via Po10, [115/20], 0 day/s 02:24:02, isis_fabricpath-default 1/20/0, number of next-hops: 1 via Po20, [115/20], 0 day/s 04:43:25, isis_fabricpath-default 1/30/0, number of next-hops: 1 via Po30, [115/20], 0 day/s 04:43:25, isis_fabricpath-default 1/40/0, number of next-hops: 1 via Po40, [115/20], 0 day/s 04:43:25, isis_fabricpath-default 1/200/0, number of next-hops: 4 via Po10, [115/40], 0 day/s 02:24:02, isis_fabricpath-default via Po20, [115/40], 0 day/s 04:43:06, isis_fabricpath-default via Po30, [115/40], 0 day/s 04:43:06, isis_fabricpath-default via Po40, [115/40], 0 day/s 04:43:06, isis_fabricpath-default 1/300/0, number of next-hops: 4 via Po10, [115/40], 0 day/s 02:24:02, isis_fabricpath-default via Po20, [115/40], 0 day/s 04:43:25, isis_fabricpath-default via Po30, [115/40], 0 day/s 04:43:25, isis_fabricpath-default via Po40, [115/40], 0 day/s 04:43:25, isis_fabricpath-default FabricPath 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31 S100 Topology (ftag), Switch ID, Sub-Switch ID Administrative distance, routing metric Route age Client protocol Next-hop interface(s) S10 S20 S30 S40 po10 po20 po30 po40 A B S200 C S300

31 FabricPath Design STP Interaction FabricPath (no STP) FabricPath Classical Ethernet (STP) STP Domain 1 STP Domain BPDU FabricPath domain appears as single Spanning-Tree bridge All FabricPath bridges share a common (static) bridge ID Cisco reserved MAC c84c.75fa.6000 STP BPDUs are not carried through the FabricPath network 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32 BPDU STP Domain 2 CE Edge Ports Configure all FabricPath edge switches using spanning-tree vlan <x> root primary (or manually configure bridge priority lower than any STP bridge) Each FabricPath edge switch must be the root for all connected STP domains Strongly recommended to use the same bridge priority on all FabricPath edge switches

32 FabricPath L2/L3 Boundary Location Layer 3 Boundary at the Spine Straightforward with two spine switches Considerations with more than two spines: HSRP: Traffic polarized to spines on a per VLAN basis (South-North) GLBP to distribute servers to different default gateways Anycast FHRP future solution Layer 3 Integration at the Leaf/Edge Provides a cleaner spine design Traffic distributed equally across spines (no hot spot) Increased number of hops to reach gateway (latency) L3 FabricPath FabricPath L3 L Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33

33 FabricPath L2/L3 Boundary Location Classic Two Switch Spine Switch-id based forwarding + MAC learning for routed traffic Simplest migration from most existing designs The spine is also used for routing with M1/F1 in the same VDC Consideration MAC Learning and Scaling Compared to classic ethernet designs you gain: Ease of configuration MAC address table increased scalability and more efficient learning Traffic distribution on all uplinks Possibility to offload the spine by providing direct communication paths between the edge layer devices [ ] L3 edge/spine s M1+F1 Conversational Learning L3 Domain M1+F1 Conversational Learning edge 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34

34 Switch-id based forwarding FabricPath L2/L3 Boundary Location Leaf/Spine/Boundary Architecture By separating the L3 function from the spine, the F1 card in the spine performs pure switchid forwarding The L3 edge will need both M1/F1 in order to connect with Fabricpath ports to the spine The M1/F1 L3 edge will need to perform learning for the remote mac addresses FP port spine L3 edge M1/F1 L3 Domain M1/F1 FP port spine L3 edge and spine can be combined in the same chassis by means of VDCs Conversational Learning 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35 edge Conversational Learning

35 Nexus Edge, Core & Boundary Nodes Large Scale Fabric 4K VLAN s, 128K MAC Address, 512K Routes blade1 blade1 blade1 blade1 blade1 blade1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade1 blade1 blade1 blade1 blade1 blade1 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 slot 8 slot 8 slot 8 slot 8 slot 8 slot 8 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 slot 7 slot 7 blade8 slot 7 slot 7 blade8 slot 7 blade8 slot 7 blade8 blade8 slot 8 blade8 slot 8 slot 8 slot 8 slot 8 slot 8 blade1 blade1 blade1 blade1 blade1 blade1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade1 blade1 blade1 blade1 blade1 blade1 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 slot 7 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade6 slot 5 blade8 slot 8 blade8 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 slot 8 slot 8 slot 8 slot 8 slot 8 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade8 slot 7 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 slot 8 slot 8 blade1 blade1 blade1 blade1 blade1 blade1 slot 8 blade8 slot 8 slot 8 slot 8 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade1 blade1 blade1 blade1 blade1 blade1 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade2 slot 1 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade3 slot 2 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade4 slot 3 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 blade5 slot 4 slot 7 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade6 slot 5 blade8 slot 8 blade8 blade6 slot 5 blade6 slot 8 slot 8 slot 8 slot 8 slot 8 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade7 slot 6 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 blade8 slot 7 slot 8 slot 8 slot 8 slot 8 slot 8 slot Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36

36 Standards Based + Cisco Extensions Nexus 5500, F1, F2 and all future HW are capable of IETF standards TRILL Support for TRILL in NX-OS is pending completion of extensions to the baseline protocol 3 1 Cisco Forwarding Outer CDCE DA ET = DTAG Inner MAC DA Outer CDCE DA Outer CDCE SA Inner MAC DA Inner MAC SA Outer CDCE SA FTAG TTL Inner MAC SA TRILL Forwarding Outer MAC DA Outer MAC DA Outer MAC SA Outer MAC SA ET = 802.1Q Outer VLAN ET = TRILL V/R/M, HopCnt Egress RB Ingress RB Inner MAC DA 0 NextHop Header TRILL Header Multi-topology, VRRP interaction, ET = 802.1Q Payload Inner VLAN Inner MAC DA Inner MAC SA Inner MAC SA ET = 802.1Q Inner VLAN Ethernet Header Payload Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37

37 Flexibility in the Fabric - Layer 2 Routing L3 Core L2+L3 FabricPath Core FabricPath POD vpc POD vpc+ POD vpc+ POD Fabric Path Site 1 FabricPath Site 4 FabricPath FabricPath Site 2 FabricPath Site 3 Fabric Path 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38

38 Agenda The Evolving Data Centre Fabric FabricPath VXLAN LISP 1K Cisco Nexus x8 6 LISP Host Mobility OTV LAN Extension Mobility with Extended Subnets Nexus Fabric 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39

39 Customer Requirement Secure movement of vapps across cloud infrastructure Tenant Network (VLAN) Solution: VXLAN Millions of dedicated LAN segments Web VM vapp1 vapp2 Web VM Security at Scale vapp mobility across data centers & clouds VXLAN is network friendly App VM DB VM App VM DB VM Efficient load sharing of links (port channel) Supports NAT; better security controls VXLAN IETF Draft: Cisco and/or its affiliates. All rights reserved. Cisco Confidential 40

40 Ethernet in IP overlay network Entire L2 frame encapsulated in UDP 50 bytes of overhead Include 24 bit VXLAN Identifier 16 Million logical networks VXLAN can cross Layer 3 (IPv4 currently) Tunnel between VEMs VMs do NOT see VXLAN ID IP multicast used for L2 broadcast/multicast, unknown unicast Technology submitted to IETF for standardization (Cisco, VMware, Citrix, Red Hat, Broadcom, Arista, and Others) VXLAN Encapsulation Original Ethernet Frame Outer MAC DA Outer MAC SA Outer 802.1Q Outer IP DA Outer IP SA Outer UDP VXLAN Header (8 bytes) Inner MAC DA InnerM AC SA Optional Inner 802.1Q Original Ethernet Payload CRC Flags8 bits VXLAN Networker Identifier (VIN) 24 bits Reserved 24 bits 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41 Res. 8 bits

41 The Nexus 1000V VEMs act as the VXLAN Tunnel Endpoints (VTEP) Nexus 1000V uses a VMKNIC to terminate VTEP traffic VM to VM traffic on different access switches is encapsulated in a VXLAN header + UDP + IP VTEPs use multicast to deliver unknown destination VM MAC addresses to all VTEPs participating in a given VXLANs VM MAC to VTEP IP address mappings are gleaned from encapsulated packets Similar to Ethernet bridge flood and learn behavior Known destination VM MAC addresses are carried over point to point tunnels between VTEPs 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 42

42 Access Switch Access Switch End System End System Bridge Domain Switch VTEP IP Multicast Enabled Underlying Network VTEP Bridge Domain Switch End System End System Direct Unicast tunnels between VTEPs (Carries known unicast frames) VTEP = VXLAN Tunnel End Point VNI = VXLAN Network Identifier VTEP VTEP VXLAN s IP Any Source Multicast Group (*,G) acts as a bus for delivery to all relevant VTEPs for a given VNI (Carries unknown/broadcast/multicast frames) VTEP VTEP 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43 43

43 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 44

44 VTEP Use Of IGMP IGMP Used to Join Each VXLANs Assigned Multicast Group on Demand Web VM DB VM DB VM Web VM Join Multicast Group Join Multicast Group Join Multicast Group Join Multicast Group Cisco and/or its affiliates. All rights reserved. Cisco Confidential 45

45 VXLAN Example Data Flow VM1 Communicating with VM2 in a VXLAN MAC: VM 1 VM 2 MAC: VM 3 abc xyz VXLAN VMKNIC VXLAN VMKNIC VXLAN VMKNIC VEM 1 VEM 2 VEM 3 Multicast Multicast Multicast 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 46

46 VXLAN Example Data Flow VM1 Communicating with VM2 in a VXLAN MAC: VM 1 VM 2 MAC: VM 3 abc xyz VXLAN VMKNIC VXLAN VMKNIC VXLAN VMKNIC Unicast Layer 3 MAC Table: VEM 2 VM Source MAC Remote Host VXLAN IP VM1:abc Cisco and/or its affiliates. All rights reserved. Cisco Confidential 47

47 VXLAN Example Data Flow VM1 Communicating with VM2 in a VXLAN MAC: VM 1 VM 2 MAC: VM 3 abc xyz VXLAN VMKNIC VXLAN VMKNIC VXLAN VMKNIC VEM 1 VEM 2 VEM 3 MAC Table: VEM 1 VM Source MAC Remote Host VXLAN IP VM2:xyz MAC Table: VEM 2 VM Source MAC Remote Host VXLAN IP VM1:abc Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48

48 VXLAN Example Data Flow VM1 Communicating with VM2 in a VXLAN MAC: VM 1 VM 2 MAC: VM 3 abc xyz VXLAN VMKNIC VXLAN VMKNIC VXLAN VMKNIC Unicast MAC Table: VEM 1 VM Source MAC Remote Host VXLAN IP VM2:xyz MAC Table: VEM 2 VM Source MAC Remote Host VXLAN IP VM1:abc Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49

49 Multiple VXLANs Can Share One Multicast Group Blue & Red VXLANs Share The Multicast Group Web VM App VM DB VM App VM Encapsulate with Blue VXLAN ID Multicast to Servers Registered for Multicast Group VEM Discards Since No VM with Blue VXLAN ID VM Broadcast Frames Sent to More Servers But Broadcast Domain Respected Within VXLAN Segment 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 50

50 Agenda The Evolving Data Centre Fabric FabricPath VXLAN LISP 1K Cisco Nexus x8 6 LISP Host Mobility OTV LAN Extension Mobility with Extended Subnets Nexus Fabric 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51

51 Single Network Architecture Delivers: VM Mobility (topology independent addressing) Security: VPNs/Multi-tenancy Route Scalability (on demand routing) IPv6 enablement, Routing Policy simplification Benefits Services integrated in a single architecture Services can be offered across organizational boundaries (multiple providers) Very large scale Open model to integrate with cloud orchestrators Use-Cases DCI route optimization/mobility Workload Portability to Cloud Secure Multi-tenancy across organizations Rapid IPv6 Deployment Route scaling 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 52

52 LISP Use Cases Consolidated Architecture with Multiple Applications Efficient Multi-Homing LISP Site LISP Routers Internet IPv6 Transition Support v6 Services LISP Router v6 IPv4 Internet v4 v6 LISP Router v6 IPv6 Internet IP Portability Ingress Traffic Engineering without BGP Multi-Tenancy and VPNs LISP Site v6-over-v4, v6-over-v6 v4-over-v6, v4-over-v4 Host-Mobility LISP Site IP Network IP Network West-DC East-DC West-DC East-DC Reduced CapEx/OpEx Large scale Segmentation Cloud / Layer 3 VM moves Segmentation 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53

53 Location Identity Separation Protocol What Do We Mean by Location and Identity? IP core Today s IP Behavior Loc/ID Overloaded Semantic When the Device Moves, It Gets a New IPv4 or IPv6 Address for Its Device IPv4 or IPv6 New Identity and Location Address Represents Identity and Location Device IPv4 or IPv6 Address Represents Identity Only. Its Location Is Here! IP core Only the Location Changes LISP Behavior Loc/ID Split When the Device Moves, Keeps Its IPv4 or IPv6 Address. It Has the Same Identity 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 54

54 A LISP Packet Walk How Does LISP Operate? 1 DNS Entry: D.abc.com A > > > > /24 LISP Site S ETR West-DC D 3 Mapping Entry ITR /24 EID-prefix: /24 Locator-set: Non-LISP site Non-LISP , site priority: 1, weight: 50 (D1) , priority: 1, weight: 50 (D2) IP Network /24 PITR EID-to-RLOC mapping East-DC This Policy Controlled by Destination Site 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55

55 A LISP Packet Walk How About Non-LISP Sites? 1 DNS Entry: D.abc.com A Non-LISP Site S Non-LISP Site 3 Mapping Entry EID-Prefix: /24 Locator-Set: , priority: 1, weight: 50 (D1) , priority: 1, weight: 50 (D2) > ETR West-DC D > > > /24 IP Network /24 PITR East-DC EID-to-RLOC mapping Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56

56 LISP Roles and Address Spaces What Are the Different Components Involved? LISP Roles Tunnel Routers - xtrs Edge devices in charge of encap/decap Ingress/Egress Tunnel Routers (ITR/ETR) EID to RLOC Mapping DB Contains RLOC to EID mappings Distributed across multiple Map Servers (MS) MS may connect over an ALT network Proxy Tunnel Routers - PxTR Coexistence between LISP and non-lisp sites Ingress/Egress: PITR, PETR Non-LISP PxTR EID Space EID = End-point Identifier Host IP or prefix RLOC = Routing Locator Prefix Next-hop w.x.y.1 x.y.w.2 z.q.r.5 z.q.r.5 e.f.g.h e.f.g.h e.f.g.h e.f.g.h ITR Address Spaces IP address of routers in the backbone Mapping DB EID a.a.a.0/24 b.b.b.0/24 c.c.c.0/24 d.d.0.0/16 RLOC w.x.y.1 x.y.w.2 z.q.r.5 z.q.r.5 EID ALT RLOC Space ETR EID Space EID RLOC a.a.a.0/24 w.x.y.1 b.b.b.0/24 x.y.w.2 c.c.c.0/24 z.q.r.5 d.d.0.0/16 z.q.r.5 RLOC a.a.a.0/24 w.x.y.1 b.b.b.0/24 x.y.w.2 c.c.c.0/24 z.q.r.5 d.d.0.0/16 z.q.r Cisco and/or its affiliates. All rights reserved. Cisco Confidential 57

57 LISP Mapping Database The Basics Registration and Resolution LISP Site ITR Mapping Cache Entry (on ITR): /16-> ( , ) Map Server / Resolver: Database Mapping Entry (on ETR): /16 -> ( , ) Map-Reply /16 -> ( , ) West-DC East-DC / /16 X ETR Y ETR Cisco and/or its affiliates. All rights reserved. Cisco Confidential 58 ETR Y ETR Z Database Mapping Entry (on ETR): /16 -> ( , )

58 Basic LISP Configuration Servers ip lisp map-resolver ip lisp map-server lisp site west-dc authentication-key 0 s3cr3t eid-prefix /24 Border Routers Between Backbones ip lisp proxy-itr ip lisp ITR map-resolver Branch Routers ip lisp itr-etr ip lisp ITR map-resolver DC Aggregation Routers ip lisp itr-etr ip lisp database-mapping / p1 w50 ip lisp database-mapping / p1 w50 ip lisp ETR map-server key s3cr3t ip lisp ETR map-server key s3cr3t Usually Devices Will Be Configured as ITRs and ETRs to Handle Traffic in Both Directions; We Illustrate Only One Direction for Simplicity LISP Site ETR West-DC /24 Non-LISP Sites IP Network East-DC 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 59 ITR RLOC EID PITR Mapping DB LISP Encap/Decap

59 Agenda The Evolving Data Centre Fabric FabricPath VXLAN LISP 1K Cisco Nexus x8 6 LISP Host Mobility OTV LAN Extension Mobility with Extended Subnets Nexus Fabric 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 60

60 LISP Host-Mobility Needs: Global IP-Mobility across subnets Optimized routing across extended subnet sites LISP Solution: Automated move detection on xtrs Dynamically update EID-to-RLOC mappings Traffic Redirection on ITRs or PITRs Benefits: Direct Path (no triangulation) Connections maintained across move No routing re-convergence No DNS updates required Transparent to the hosts Global Scalability (cloud bursting) IPv4/IPv6 Support LISP Site West-DC Non-LISP Sites IP Network East-DC 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 61 xtr LAN Extensions LISP-VM (xtr) RLOC EID PxTR Mapping DB LISP Encap/Decap

61 Host-Mobility Scenarios Moves Without LAN Extension Moves With LAN Extension LISP Site xtr Non-LISP Site LISP Site xtr Mapping DB Internet or Shared WAN DR Location or Cloud Provider DC LAN Extension IP Network Mapping DB LISP-VM (xtr) West-DC East-DC LISP-VM (xtr) West-DC East-DC IP Mobility Across Subnets Routing for Extended Subnets Disaster Recovery Active-Active Data Centers Cloud Bursting Distributed Clusters Application Members in One Location Application Members Distributed (Broadcasts across sites) 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 62

62 LISP Host-Mobility - Move Detection Monitor the Source of Received Traffic The new xtr checks the source of received traffic Configured dynamic-eids define which prefixes may roam lisp dynamic-eid roamer database-mapping /24 <RLOC-C> p1 w50 database-mapping /24 <RLOC-D> p1 w50 map-server key abcd interface vlan 100 lisp mobility roamer LISP-VM (xtr) Mapping DB A B C D Received a Packet It s from a New Host It s in the Dynamic-EID Allowed Range It s a Move! Register the /32 with LISP West-DC East-DC / /16 Y X Y Z Cisco and/or its affiliates. All rights reserved. Cisco Confidential 63

63 LISP Host-Mobility - Traffic Redirection Update Location Mappings for the Host System Wide When a host move is detected, updates are triggered: The host-to-location mapping in the Database is updated to reflect the new location The old ETR is notified of the move ITRs are notified to update their Map-caches Ingress routers (ITRs or PITRs) now send traffic to the new location LISP Site xtr A B C D Mapping DB /16 RLOC A, B /32 RLOC C, D LISP-VM (xtr) West-DC East-DC / /16 Y X Y Z Cisco and/or its affiliates. All rights reserved. Cisco Confidential 64

64 LISP Host-Mobility - First Hop Routing Across Different Subnets SVI (Interface VLAN x) and HSRP configured as usual (Consistent GWY-MAC configured across all dynamic subnets) The lisp mobility <dyn-eid-map> command enables proxy-arp functionality on the SVI The LISP-VM router services first hop routing requests for both local and roaming subnets Hosts can move anywhere and always talk to a local gateway with the same MAC interface Ethernet2/4 lisp mobility roamer ip address /24 ip proxy-arp lisp mobility roamer hsrp 101 ip proxy-arp hsrp 101 mac-address e1d.010c ip HSRP Active interface vlan 100 ip address /24 mac-address e1d.010c ip LISP-VM (xtr) A B C D interface vlan 100 interface vlan 200 ip address /24 ip address /24 lisp mobility roamer lisp mobility roamer ip proxy-arp ip proxy-arp hsrp 201 hsrp 201 mac-address e1d.010c mac-address e1d.010c ip ip West-DC East-DC / /24 HSRP HSRP ARP ARP GWY-MAC GWY-MAC HSRP Active 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 65

65 Null0 host routes indicate the host is away /16 RLOC A, B /32 RLOC C, D Map-Notify /32 <C,D> Mapping DB Map-Register /32 <C,D> Routing Table: /16 Local /32 Null0 10 Map-Notify /32 <C,D> A Routing Table: /16 Local / / /32 Null0 1 East-DC West-DC Y X B Routing Table: /16 Local /32 Local Cisco and/or its affiliates. All rights reserved. Cisco Confidential 66 Y C 4 D Routing Table: /16 Local /32 Local Map-Notify /32 <C,D>

66 Map ITR 1. ITRs and PITRs with cached mappings continue to send traffic to the old locators The old xtr knows the host has moved (Null0 route). 2. Old xtr sends Solicit Map Request (SMR) messages to any encapsulators sending traffic to the moved host LISP site ITR /16 RLOC A,B /32 RLOC C,D Mapping DB 3. The ITR then initiates a new map request process 4. An updated map-reply is issued from the new location A B C D LISP-VM (xtr) 5. The ITR Map Cache is updated Traffic is now re-directed SMRs are an important integrity measure to avoid unsolicited map responses and spoofing West-DC East-DC / /16 Y X Y Z 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 67

67 LISP Host-Mobility Configuration Across Subnets (No LAN Extensions) ip lisp ITR-ETR ip lisp database-mapping /16 <RLOC-A> ip lisp database-mapping /16 <RLOC-B> lisp dynamic-eid roamer database-mapping /24 <RLOC-A> database-mapping /24 <RLOC-B> map-server key abcd map-notify-group interface vlan 100 ip address /16 lisp mobility roamer ip proxy-arp hsrp 101 mac-address e1d.010c ip LISP-VM (xtr) A B C D ip lisp ITR-ETR ip lisp database-mapping /16 <RLOC-C> ip lisp database-mapping /16 <RLOC-D> lisp dynamic-eid roamer database-mapping /24 <RLOC-C> database-mapping /24 <RLOC-D> map-server key abcd map-notify-group interface vlan 100 ip address /16 lisp mobility roamer ip proxy-arp hsrp 201 mac-address e1d.010c ip West-DC East-DC / /16 Mapping DB X 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 68 Y Z

68 Agenda The Evolving Data Centre Fabric FabricPath VXLAN LISP 1K Cisco Nexus x8 6 LISP Host Mobility OTV LAN Extension Mobility with Extended Subnets Nexus Fabric 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 69

69 Simplifying LAN Extensions Ethernet LAN Extension over any Network Works over dark fiber, MPLS, or IP Multi-data center scalability Many Physical Sites One Logical Data Center Simplified Configuration & Operation Seamless overlay - No network re-design Single touch site configuration High Resiliency Failure domain isolation Seamless Multi-homing Maximizes available bandwidth Automated multi-pathing Optimal multicast replication Any Workload, Anytime, Anywhere Unleashing the Full Potential of Compute Virtualization 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 70

70 OTV Data Plane Inter-Site Packet Flow 1. Layer 2 lookup on the destination MAC. MAC 3 is reachable through IP B 2. The Edge Device encapsulates the frame 3. The transport delivers the packet to the Edge Device on site East 4. The Edge Device on site East receives and decapsulates the packet 5. Layer 2 lookup on the original frame. MAC 3 is a local MAC 6. The frame is delivered to the destination 1 Layer 2 Lookup MAC TABLE VLAN MAC IF Transport Infrastructure 100 MAC 1 Eth 2 IP A IP B 100 MAC 1 IP A OTV OTV OTV OTV 100 MAC 2 Eth 1 Encap 100 MAC 2 IP A MAC 1 MAC 3 IP A IP B 100 MAC 3 IP B MAC 1 MAC 3 IP A IP B 100 MAC 3 Eth MAC 4 IP B 2 3 Decap 4 MAC TABLE VLAN MAC IF 100 MAC 4 Eth 4 5 Layer 2 Lookup MAC 1 MAC 3 MAC 1 West Site MAC 1 MAC Cisco and/or its affiliates. All rights reserved. Cisco Confidential 71 East Site MAC 3 6

71 The OTV Control Plane OTV proactively advertises MAC reachability (control-plane learning) MAC addresses advertised in the background once OTV has been configured IS-IS is the OTV Control Protocol running between the Edge Devices No specific configuration is required OTV MAC Addresses Advertisements OTV West IP A IP B East IP C OTV South 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 72

72 Agenda The Evolving Data Centre Fabric FabricPath VXLAN LISP 1K Cisco Nexus x8 6 LISP Host Mobility OTV LAN Extension Mobility with Extended Subnets Nexus Fabric 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 73

73 Ingress Routing Challenge in DCI Extending Subnets Creates a Routing Challenge A subnet usually implies location Yet we use LAN extensions to stretch subnets across locations Location semantics of subnets are lost LISP site xtr Traditional routing relies on the location semantics of the subnet Can t tell if a server is at the East or West location of the subnet More granular (host level) information is required LISP provides host level location semantics LAN Extension West-DC IP Network East-DC 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 74

74 Host-Mobility and Multi-homing ETR updates Extended Subnets Null0 host routes indicate the host is away /16 RLOC A, B /32 RLOC C, D /24 is the dyn-eid 4 Routing Table: /16 Local /24 Null /32 Null0 West-DC Y X Map-Notify /32 <C,D> Mapping DB Map-Register /32 <C,D> 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 75 Y Routing Table: /16 Local /24 Null /32 Local Routing Table: /16 Local /24 Null0 4 A B /32 Local C D Routing Table: /16 Local /24 Null /32 Null / /16 OTV 1 East-DC Map-Notify /32 <C,D>

75 Refreshing the map caches Map ITR /16 RLOC A,B 1. ITRs and PITRs with cached mappings continue to send traffic to the old locators 1. The old xtr knows the host has moved (Null0 route). LISP site ITR /32 RLOC C,D 2. Old xtr sends Solicit Map Request (SMR) messages to any encapsulators sending traffic to the moved host Mapping DB 3. The ITR then initiates a new map request process 4. An updated map-reply is issued from the new location 5. The ITR Map Cache is updated Traffic is now re-directed SMRs are an important integrity measure to avoid unsolicited map responses and spoofing A B C D LISP-VM (xtr) West-DC OTV East-DC / /16 Y X Y Z 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 76

76 LISP Host-Mobility - First Hop Routing With Extended Subnets Consistent GWY-IP and GWY-MAC configured across all sites Consistent HSRP group number across sites consistent GWY-MAC Servers can move anywhere and always talk to a local gateway with the same IP/MAC interface vlan 100 interface vlan 100 ip address /24 interface vlan ip 200 address /24 interface Ethernet2/4 lisp mobility roamer ip address lisp /24 mobility roamer ip address /24 lisp extended-subnet-mode lisp mobility lisp roamer extended-subnet-mode lisp mobility hsrp roamer 101 lisp extended-subnet-mode hsrp 101 LAN Ext. lisp extended-subnet-mode ip hsrp 101 ip ip hsrp 101 A B C D ip HSRP Active LISP-VM (xtr) West-DC East-DC / /24 HSRP HSRP ARP ARP GWY-MAC GWY-MAC HSRP Active 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 77

77 LISP VM-Mobility Configuration With Extended Subnets Use Extended-Subnet-Mode ip lisp ITR-ETR ip lisp database-mapping /16 <RLOC-A> ip lisp database-mapping /16 <RLOC-B> ip lisp database-mapping /16 <RLOC-C> ip lisp database-mapping /16 <RLOC-D> ip lisp ITR-ETR ip lisp database-mapping /16 <RLOC-A> ip lisp database-mapping /16 <RLOC-B> ip lisp database-mapping /16 <RLOC-C> ip lisp database-mapping /16 <RLOC-D> lisp dynamic-eid roamer database-mapping /24 <RLOC-A> database-mapping /24 <RLOC-B> map-server key abcd map-notify-group interface vlan 100 ip address /16 lisp mobility roamer lisp extended-subnet-mode hsrp 101 ip lisp dynamic-eid roamer database-mapping /24 <RLOC-C> database-mapping /24 <RLOC-D> map-server key abcd map-notify-group interface vlan 100 ip address /16 lisp mobility roamer lisp extended-subnet-mode hsrp 101 ip LISP-VM (xtr) LAN Ext A B C D Mapping DB West-DC /16 East-DC X Y Z 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 78

78 Off-Subnet Client-Server Traffic All Off-Subnet/Off-Site Traffic Is LISP Encapsulated Clients ( & communicate with Server Client-server traffic is LISP encapsulated at the ITRs or PITRs Client-to-server: to ETRs C or D Server-to-client: to ETR (F) for LISP sites to PETR (G) for non-lisp sites Server-Server off-subnet traffic across sites is also LISP encapsulated CLIENT Non-LISP Sites CLIENT LISP Site xtr PxTR G F F C G D A B C D Mapping DB LISP-VM (xtr) West-DC East-DC / / Y X Y Cisco and/or its affiliates. All rights reserved. Cisco Confidential 79

79 On-Subnet Server-Server Traffic On Subnet Traffic Across L3 boundaries With LAN Extension Live moves and cluster member dispersion Traffic between X & Y uses the LAN Extension Link-local-multicast handled by the LAN Extension LAN Ext. A B C D Without LAN Extensions Cold moves, no application dispersion X- Y traffic is sent to the LISP-VM router & LISP encapsulated Need LAN extensions for link-local multicast traffic A B C B C D Mapping DB LISP-VM (xtr) LISP-VM (xtr) West-DC /16 East-DC West-DC /16 East-DC / Y Y X Y Z X Y Z Cisco and/or its affiliates. All rights reserved. Cisco Confidential 80

80 Agenda The Evolving Data Centre Fabric FabricPath VXLAN LISP 1K Cisco Nexus x8 6 LISP Host Mobility OTV LAN Extension Mobility with Extended Subnets Nexus Fabric 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 81

81 Enhance application availability by distributing Cluster members across PODs and across locations Distance limited by application latency budget and storage replication Two types of traffic specific to the cluster: Non-routable heartbeats: FabricPath (Intra-DC) & OTV (Inter-DC) provide LAN connectivity Front-end IP connectivity: LISP provides mobility for cluster virtual-ip failover DC-west LISP IP mobility IP Network DC-east OTV POD POD (Inter-DC) POD POD App Cluster OS OS Distributed App (GeoCluster) OS Fabric Path (Intra-DC) 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 82

82 VXLAN & FP provide elasticity within the DC within a L2 POD and across PODs OTV extends the LAN across DC sites without compromising network stability LISP integrates with SLBs and balances traffic across the SLBs (Future) Intra-DC Inter-DC Virtual Machines VXLAN (x-l3), FabricPath (L2) OTV (x-l3) Physical Machines FabricPath (L2), VXLAN GWY (future) OTV (x-l3) DC-west LISP IP mobility IP Network DC-east POD POD POD POD App OS App OS App OS OTV (Inter-DC x-l3) App OS Fabric Path (Intra-DC L2) VXLAN (Intra-DC x-l3) 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 83 SLB

83 Reduce Disaster Recovery Bring-up times - Less Network Changes/Operations = Faster recovery times Preserve IP addressing with LISP host mobility No reconfiguration of applications or network service policies No routing re-convergence Automatic routing re-localization (upon application bring-up at DR) VXLAN segments move along with the applications (vapps) DC-west LISP IP mobility IP Network DC-east POD POD POD POD App App App App App App OS OS OS OS OS OS VXLAN (Intra-DC x-l3) VXLAN (Intra-DC x-l3) 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 84

84 IP1 GWY vxlan 1 Move virtual Applications (vapps) to private cloud PODs Move VMs and virtual Segments (VXLANs) LISP host mobility allows the vapp GWY to roam Maintain GWY IP address and optimal reachability VXLAN segments move along with the applications (vapps) Very large scale of virtual segments can move and extend across L3 boundaries DC-west LISP IP mobility IP Network web app db DC-east V M V M V M vxlan 2 VSG vxlan 3 vapp = Collection of VMs and segments with a GWY POD POD POD POD GWY GWY GWY vxlan 1 GWY GWY vxlan 1 VM vxlan 1 vxlan 1 vxlan 1 VM web VM vxlan 2 VM web VM vxlan 2 web VM vxlan 2 web vxlan 2 web vxlan 2 VM app VM vxlan 3 VM app VM vxlan 3 app VM vxlan 3 app vxlan 3 app vxlan 3 db VM 2010 Cisco db and/or its VM affiliates. All rights reserved. VM VM Cisco Confidential 85 db db db

85 Scale Complimentary Capabilities FabricPath, VXLAN, LISP Requirement Intra-DC Inter-DC Layer 2 connectivity FabricPath/TRILL/VXLAN OTV/VPLS IP Mobility LISP LISP Secure Segmentation VXLAN / Segment-ID VPNs (LISP/MPLS) DC-west LISP IP mobility IP Network DC-east POD POD POD POD App OS App OS Fabric Path (Intra-DC L2) VXLAN/OTV (Intra-DC x-l3) App OS OTV/VPLS (Inter-DC x-l3) 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 86 App OS App OS Fabric Path (Intra-DC L2) VXLAN/OTV (Intra-DC x-l3) App OS

86 Q&A #CiscoPlusCA

87 We value your feedback. Please be sure to complete the Evaluation Form for this session. Access today s presentations at cisco.com/ca/plus and join the #CiscoPlusCA conversation