Advanced OTV Configure, Verify and Troubleshoot OTV in Your Network

Transcription

1

2 Advanced OTV Configure, Verify and Troubleshoot OTV in Your Network Andy Gossett, Customer Support Engineer, Cisco Services

3 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 3

4 Introduction Overlay Transport Virtualization (OTV) in a Nutshell OTV is a MAC-in-IP method that extends Layer 2 connectivity across a transport network infrastructure OTV supports both multicast and unicast-only transport networks OTV uses ISIS as the control protocol OTV on Nexus7000 does not encrypt encapsulated payload 4

5 Introduction Terminology: Edge Device Performs OTV functions Multiple OTV Edge Devices can exists at each site OTV requires the Transport Services (TRS) license Creating non default VDC s requires Advanced Services license Edge Devices 5

6 Introduction Terminology: Internal Interfaces Regular layer 2 interfaces facing the site No OTV configuration required Supported on M-series modules Support for F1 and F2e starting in 6.2 Internal Interfaces 6

7 Introduction Terminology: Join Interface Uplink on Edge device that joins the Overlay Forwards OTV control and data traffic Layer 3 interface Currently supported only on M-series modules Join Interfaces 7

8 Introduction Terminology: Overlay Interface Virtual Interface where the OTV configurations are applied Multi-access multicast-capable interface Encapsulates Layer 2 frames 8

9 Introduction Terminology: Authoritative Edge Device OTV supports multiple edge devices per site A single OTV device is elected as AED on a per-vlan basis The AED is responsible for advertising MAC reachability and forwarding traffic into and out of the site for its VLANs AED for odd VLANs 9

10 Introduction Terminology: Authoritative Edge Device OTV supports multiple edge devices per site A single OTV device is elected as AED on a per-vlan basis The AED is responsible for advertising MAC reachability and forwarding traffic into and out of the site for its VLANs AED for even VLANs 10

11 Introduction Terminology: Site VLAN Prior to 5.2(1) OTV used only communication on the site vlan for AED election I m AED for Even VLANs I m AED for Odd VLANs OTV Hello OTV Hello 11

12 Introduction Terminology: Site VLAN Prior to 5.2(1) OTV used only communication on the site vlan for AED election Misconfiguration or connectivity issues on the site vlan could result in active/active AED mode I m AED for All VLANs I m AED for All VLANs OTV Hello OTV Hello 12

13 Introduction Terminology: Site VLAN and Site Identifier 5.2(1) added Dual Site Adjacency 1. Site Adjacency established across the site vlan 2. Overlay Adjacency established via the Join interface across Layer 3 network I m AED for Even VLANs OTV Hello Site-ID Full Adjacency I m AED for Odd VLANs OTV Hello Site-ID OTV Hello Site-ID OTV Hello Site-ID

14 Introduction Terminology: Site VLAN and Site Identifier If a communication problem occurs on the site vlan, each OTV device can still advertise AED capabilities across overlay to prevent an active/active scenario I m AED for Even VLANs OTV Hello Site-ID I m AED for Odd VLANs OTV Hello Site-ID Partial Adjacency OTV Hello Site-ID OTV Hello Site-ID

15 Introduction Terminology: Site VLAN and Site Identifier Dual Site Adjacency also has mechanism for advertising AED capabilities on local failure to improve convergence Join interface down I m not AED capable I m AED for Even VLANs Partial Adjacency I m AED for Odd VLANs I m now AED ALL VLANs I m not AED capable 15

16 Introduction Terminology: Site VLAN and Site Identifier Dual Site Adjacency also has mechanism for advertising AED capabilities on local failure to improve convergence Join interface down Internal Vlans down I m not AED capable I m AED for Even VLANs I m not AED capable Partial Adjacency I m AED for Odd VLANs I m now AED ALL VLANs 16

17 Introduction Terminology: Site VLAN and Site Identifier Dual Site Adjacency also has mechanism for advertising AED capabilities on local failure to improve convergence Join interface down Internal Vlans down AED down or initializing I m now AED EVEN VLANs Initializing I m now AED capable Full Adjacency I m AED for ALL VLANs I m now AED ODD VLANs I m now AED capable 17

18 Introduction Overlay Transport Multicast Transport vs. Unicast Transport Packet Type ISIS Hello, Unicast Transport ISIS Hello, Multicast Transport OTV Header for User Data IP Length 1450B 1442B 42B 18

19 Introduction MTU Reference Slide OTV adds a 42 Byte header to both ISIS control traffic and encapsulated data traffic. OTV sets the DF (Don t Fragment) bit on all packets For OTV adjacencies to come up, transport must support MTU: 1442 Bytes for multicast transport 1450 Bytes for unicast transport To successfully forward standard 1500 Byte packets for host applications, ensure that transport supports at least 1542 Byte MTU 19

20 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 20

21 Configuration Enable OTV Feature feature West East 21

22 Configuration Site VLAN and Site Identifier Site VLAN needs to be configured and active even if you do not have multiple OTV devices in the same site site VLAN should not be extended across overlay Site Identifier can be any number between and ffff.ffff.ffff. Value will always be displayed in MAC format Site Identifier must be unique for each site Site Identifier is required in 5.2(1) and above for overlay to come up. This must be kept in mind when performing an ISSU from a pre-5.2(1) release Service Impacting 22

23 Configuration Site VLAN and Site Identifier site-vlan 210 site-identifier site-vlan 210 site-identifier West East 23

24 Configuration Internal Interface interface port-channel 103 switchport switchport mode trunk West East 24

25 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 25

26 Configuration Multicast Transport: Overlay Multicast Transport requires the configuration of a control-group and datagroup Adjacencies are built and MAC reachability information is exchanged over the control-group The data-group is an source specific multicast (SSM) delivery group for extending multicast traffic across the overlay. It can be configured as any subnet within the transport s SSM range. The data-group range should not overlap with the control-group 26

27 Configuration Multicast Enabled West feature pim ip pim rp-address <rp> interface port-channel y ip pim sparse-mode ip igmp version 3 East 27

28 Configuration Multicast Transport West interface Overlay1 join-interface port-channel100 control-group data-group /24 extend-vlan East interface port-channel 100 mtu 9216 ip address /30 ip igmp version 3 28

29 Configuration Multicast Transport Full Picture West WEST_OTVA feature site-vlan 210 site-identifier interface Overlay1 join-interface port-channel100 control-group data-group /24 extend-vlan no shutdown EAST_OTVA feature site-vlan 210 site-identifier interface Overlay1 join-interface port-channel100 control-group data-group /24 extend-vlan no shutdown East interface port-channel100 mtu 9216 ip address /30 ip igmp version 3 interface port-channel100 mtu 9216 ip address /30 ip igmp version 3 29

30 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 30

31 Configuration Unicast Transport: Overlay OTV can run across a unicast only transport Unicast Transport requires the configuration of one or more adjacency servers. OTV devices register with the adjacency server which in turn provides each with an OTV Neighbor List (onl). Think of the adjacency server as a special process running on a generic OTV edge device A primary and secondary adjacency server can be configured for redundancy 31

32 Configuration Unicast Transport: Adjacency Server Reference Slide Primary and Secondary Adjacency servers are stateless; every OTV client must register with both servers OTV client will not process the onl from the secondary server while the primary server is still active OTV uses graceful exit of Adjacency Servers. If the primary server is rebooted or reconfigured, it can notify the OTV clients allowing them to immediately use the secondary. 32

33 Configuration Unicast Transport: Primary Adjacency Server Overlay West interface Overlay1 join-interface port-channel100 extend-vlan adjacency-server unicast-only East 33

34 Configuration Unicast Transport: Secondary Adjacency Server Overlay West interface Overlay1 Primary Server join-interface port-channel100 extend-vlan use-adjacency-server unicast-only adjacency-server unicast-only East 34

35 Configuration Unicast Transport: Client Overlay West Primary Server interface Overlay1 join-interface port-channel100 extend-vlan use-adjacency-server unicast-only Secondary Server East 35

36 Configuration Unicast Transport: Full Picture WEST_OTVA feature site-vlan 210 site-identifier EAST_OTVA feature site-vlan 210 site-identifier interface Overlay1 join-interface port-channel100 extend-vlan use-adjacency-server unicast-only adjacency-server unicast-only no shutdown West interface Overlay1 join-interface port-channel100 extend-vlan adjacency-server unicast-only no shutdown interface port-channel100 mtu 9216 ip address /30 East interface port-channel100 mtu 9216 ip address /30 36

37 Configuration Unicast Transport: Full Picture WEST_OTVB feature site-vlan 210 site-identifier interface Overlay1 join-interface port-channel100 extend-vlan use-adjacency-server West unicast-only no shutdown EAST_OTVB feature site-vlan 210 site-identifier interface Overlay1 join-interface port-channel100 extend-vlan use-adjacency-server unicast-only East no shutdown interface port-channel100 mtu 9216 ip address /30 interface port-channel100 mtu 9216 ip address /30 37

38 Configuration Authentication Reference Slide OTV supports authentication of Hello messages along with authentication of PDU s! Configure OTV key chain key chain OTVKeys key 1 key-string 0 cisco! Apply md5 authentication to OTV Hellos interface Overlay1 isis authentication-type md5 isis authentication key-chain OTVKeys! Apply md5 authentication to OTV PDUs -isis default vpn Overlay1 authentication-check authentication-type md5 authentication key-chain OTVKeys 38

39 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 39

40 New Features Targeted 6.2 F1/F2E used as Internal Interfaces Selective Unicast Flooding Dedicated Data Broadcast Forwarding Multicast Group Source-Interface with Loopback Tunnel Depolarization with Secondary IP OTV VLAN Translation flood mac AC9.00A2 vlan 202 interface Overlay1 source-interface loopback0 broadcast-group control-group data-group /24 extend-vlan vlan mapping 203 to

41 New Features Source-Interface with Loopback Targeted 6.2 Join Interface Limitations Bandwidth to the core is limited to one physical link or port-channel Changes to join-interface will churn all OTV overlay states, since the overlay encapsulation for all routes need to be updated PIM cannot be enabled on the join-interface, since the OTV solution assumes it's an IGMP host interface Unable to utilize the redundancy of multiple uplinks when available, and the flexibility of dynamic unicast routing convergence on uplink failures If join-interface goes down, the connectivity to the core is broken. User intervention is needed to provide alternate core connectivity 41

42 New Features Source-Interface with Loopback Targeted 6.2 Overlay uses source-interface command to source OTV traffic from the edge device and source joins to receive multicast traffic from the core West East Multiple L3 uplinks connected to OTV edge device, each running PIM and an IGP with IPv4. OTV device can now act as first-hop/last-hop multicast router instead of IGMP client 42

43 New Features Source-Interface with Loopback West WEST_OTVA OTV Config feature site-vlan 210 site-identifier interface Overlay1 source-interface loopback0 control-group data-group /24 extend-vlan no shutdown WEST_OTVA PIM and IGP Config feature pim feature ospf ip pim rp-address group-list /32 ip pim ssm range /24 router ospf 1 interface loopback0 ip address /32 ip router ospf 1 area ip pim sparse-mode interface Ethernet3/2 mtu 9216 ip address /24 ip router ospf 1 area ip pim sparse-mode interface Ethernet3/3 mtu 9216 ip address /24 ip router ospf 1 area ip pim sparse-mode Targeted 6.2 East multiple L3 uplinks 43

44 New Features Tunnel Depolarization with Secondary IP Targeted 6.2 All encapsulated traffic between AED s have same source and destination IP s limiting the advantages of Etherchannel and ECMP load-balancing West East A1->B1 44

45 New Features Tunnel Depolarization with Secondary IP Targeted 6.2 Secondary IPs allows OTV to forward traffic between multiple endpoints to prevent polarization along the path West Interface loopback0 ip address /32 ip address /32 secondary East Interface loopback0 ip address /32 ip address /32 secondary WEST_OTVA# show (output omitted) Source interface : Lo0 ( ) Secondary IP Addresses: :

46 New Features Tunnel Depolarization with Secondary IP Targeted 6.2 West A4->B4 A2->B2 A3->B3 A1->B1 WEST_OTVA# show tunnel internal implicit detail Tunnel16392 is up Tunnel source , destination packets output, 1 minute output rate 7730 packets/sec packets input, 1 minute input rate 8033 packets/sec Tunnel16393 is up Tunnel source , destination packets output, 1 minute output rate 7432 packets/sec packets input, 1 minute input rate 6843 packets/sec Tunnel16394 is up Tunnel source , destination Last clearing of "show interface" counters never packets output, 1 minute output rate 6244 packets/sec packets input, 1 minute input rate 7437 packets/sec Tunnel16395 is up Tunnel source , destination packets output, 1 minute output rate 8325 packets/sec packets input, 1 minute input rate 7437 packets/sec East 46

47 New Features OTV VLAN Translation Targeted 6.2 VLAN translation allows OTV to map a local VLAN to a remote VLAN. West interface Overlay0 extend-vlan 202, 303 East Vlan 203 Vlan 203 Vlan 303 Vlan 303 interface Overlay0 extend-vlan vlan mapping 203 to 303 WEST_OTVA# show vlan-mapping Original VLAN -> Translated VLAN >

48 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 48

49 Verification Adjacency: IP Connectivity For both multicast and unicast transports, adjacencies cannot be formed without IP connectivity between each OTV edge device WEST_OTVA#! Ping EAST_OTVA join interface WEST_OTVA# ping count 1 PING ( ): 56 data bytes 64 bytes from : icmp_seq=0 ttl=251 time=1.287 ms ping statistics packets transmitted, 1 packets received, 0.00% packet loss WEST_OTVA#! Ping EAST_OTVB join interface WEST_OTVA#! Ping WEST_OTVB join interface 49

50 Verification Adjacency: Overlay Verify overlay and site-vlan are up WEST_OTVA# show Multicast Transport WEST_OTVA# show Unicast Transport OTV Overlay Information Site Identifier Overlay interface Overlay1 VPN name : Overlay1 VPN state : UP Extended vlans : (Total:10) Control group : Data group range(s) : /24 Join interface(s) : Po100 ( ) Site vlan : 210 (up) AED-Capable : Yes Capability : Multicast-Reachable OTV Overlay Information Site Identifier Overlay interface Overlay1 VPN name : Overlay1 VPN state : UP Extended vlans : (Total:10) Join interface(s) : Po100 ( ) Site vlan : 210 (up) AED-Capable : Yes Capability : Unicast-Only Is Adjacency Server : Yes Adjacency Server(s) : [None] / [None] 50

51 Verification Adjacency: ISIS Hello (IIH) statistics WEST_OTVA# show isis site statistics begin PDU OTV-IS-IS PDU statistics for site-vlan: PDU Received Sent RcvAuthErr OtherRcvErr ReTransmit LAN-IIH n/a CSNP n/a PSNP n/a LSP WEST_OTVA# show isis traffic OTV-IS-IS process: default VPN: Overlay1 OTV-IS-IS Traffic: PDU Received Sent RcvAuthErr OtherRcvErr ReTransmit LAN-IIH n/a CSNP n/a PSNP n/a LSP

52 Adjacency: ISIS Hello over Multicast Transport ISIS Hellos have packet size of 1442 Bytes through multicast transport 1400B ISIS + 42B OTV header OTV ISIS sent on multicast control-group, sourced from join interface System ID and Site Identifier included in Hello 52

53 Adjacency: ISIS Hello over Unicast Transport ISIS Hellos have packet size of 1450 Bytes through unicast transport 1400B ISIS + 42B OTV header + 8B UDP OTV ISIS via unicast, sourced and destined between join interfaces OTV ISIS Hello sent encapsulated in UDP unicast on port 8472 System ID and Site Identifier included in Hello 53

54 Verification Adjacency: Multicast Transport (join-interface) OTV join interfaces are configured with IGMPv3. Therefore, from the transport's perspective, the OTV edge devices appear as host sending and requesting traffic from the control-group. Ensure (S,G) and (*,G) state is created at first-hop router for each edge device West East CORE#show ip mroute summary [output omitted] (*, ), 01:15:56/00:03:06, RP , OIF count: 4, flags: S ( , ), 00:47:13/00:03:25, OIF count: 3, flags: T ( , ), 00:47:06/00:03:15, OIF count: 3, flags: T ( , ), 00:56:40/00:03:25, OIF count: 3, flags: T ( , ), 00:57:17/00:03:25, OIF count: 3, flags: T 54

55 Verification Adjacency: Multicast Transport (source-interface) Reference Slide OTV Edge device is acting as the first-hop/last-hop router, therefore it should have (*,G) for the control group and (S,G) for each OTV edge device Targeted 6.2 Interface Overlay1 source-interface loopback 0 Interface West loopback0 ip address /32 Interface Overlay1 source-interface loopback 0 Interface loopback0 East ip address /32 Interface Overlay1 source-interface loopback 0 Interface loopback0 ip address /32 Interface Overlay1 source-interface loopback 0 Interface loopback0 ip address /32 55

56 Verification Adjacency: Multicast Transport (source-interface) Reference Slide OTV Edge device is acting as the first-hop/last-hop router, therefore it ( /32, /32), uptime: 17:30:05, ip mrib pim should have (*,G) for the control group Incoming interface: and (S,G) loopback0, for RPF each nbr: OTV edge device West WEST_OTVA# show ip mroute IP Multicast Routing Table for VRF "default" (*, /32), uptime: 17:30:09, ip pim Incoming interface: Ethernet3/2, RPF nbr: Outgoing interface list: (count: 1) loopback0, uptime: 17:30:09, Outgoing interface list: (count: 3) Ethernet3/4, uptime: 17:27:03, pim Ethernet3/2, uptime: 17:27:49, pim loopback0, uptime: 17:30:05, mrib,, (RPF) ( /32, /32), uptime: 17:27:49, ip mrib pim Incoming interface: Ethernet3/3, RPF nbr: Outgoing interface list: (count: 2) Overlay1, uptime: 17:27:49, loopback0, uptime: 17:27:49, mrib Targeted 6.2 East ( /32, /32), uptime: 17:27:49, pim mrib ip Incoming interface: Ethernet3/4, RPF nbr: Outgoing interface list: (count: 2) Overlay1, uptime: 17:27:49, loopback0, uptime: 17:27:49, mrib ( /32, /32), uptime: 17:25:04, ip mrib pim Incoming interface: Ethernet3/5, RPF nbr: Outgoing interface list: (count: 2) Overlay1, uptime: 17:25:04, loopback0, uptime: 17:25:04, mrib 56

57 Verification Adjacency: ISIS Overlay Adjacencies West WEST_OTVA# show isis adjacency OTV-IS-IS process: default VPN: Overlay1 OTV-IS-IS adjacency database: System ID SNPA Level State Hold Time Interface Site-ID EAST_OTVB 64a0.e741.c841 1 UP 00:00:11 Overlay WEST_OTVB 64a0.e741.c842 1 UP 00:00:09 Overlay EAST_OTVA 6c9c.ed UP 00:00:13 Overlay East 57

58 Verification Adjacency: OTV Overlay Adjacencies WEST_OTVA# show adjacency Overlay Adjacency database West Overlay-Interface Overlay1 : Hostname System-ID Dest Addr Up Time State EAST_OTVA 6c9c.ed :34:34 UP EAST_OTVB 64a0.e741.c :34:30 UP WEST_OTVB 64a0.e741.c :34:30 UP East 58

59 Verification Adjacency: ISIS Site Adjacencies WEST_OTVA# show isis site OTV-ISIS site-information for: default West Level Metric CSNP Next CSNP Hello Multi Next IIH :00: Level Adjs AdjsUp Pri Circuit ID Since WEST_OTVA.01 * 23:57:51 East [output omitted] Neighbor SystemID: 64a0.e741.c842 [WEST_OTVB] IPv4 site groups:

60 Verification Adjacency: OTV Site Adjacencies WEST_OTVA# show site West Dual Adjacency State Description Full - Both site and overlay adjacency up Partial - Either site/overlay adjacency down Down - Both adjacencies are down (Neighbor is down/unreachable) (!) - Site-ID mismatch detected Local Edge Device Information: Hostname WEST_OTVA System-ID 6c9c.ed Site-Identifier Site-VLAN 210 State is Up Site Information for Overlay1: Local device is AED-Capable Neighbor Edge Devices in Site: 1 Hostname System-ID Adjacency- Adjacency- AED- State Uptime Capable WEST_OTVB 64a0.e741.c842 Full 23:57:51 Yes East 60

61 Verification Authoritative Edge Device (AED) WEST_OTVA# show vlan OTV Extended VLANs and Edge Device State Information (* - AED) West VLAN Auth. Edge Device Vlan State Overlay WEST_OTVB inactive(non AED)Overlay1 201* WEST_OTVA active Overlay1 East 61

62 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 62

63 Verification Unicast Forwarding West Host on vlan 201 IP MAC 001b.d AED vlan 201 AED vlan 201 Host on vlan 201 IP MAC 001f.6c75.1d42 East 63

64 Verification Unicast Forwarding Simplified topology based on AED for vlan 201 Let s assume that ARP has already resolved between hosts West East Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 64

65 Verification Unicast Forwarding: User on Site West Sends Unicast Packet to Site East Packet is received on internal interface on OTV AED Verify CAM entry and OTV route West vlan b.d WEST_OTVA# show mac address-table address 001f.6c75.1d42 vlan 201 Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID O f.6c75.1d42 dynamic 0 F F Overlay1 WEST_OTVA# show route 001f.6c75.1d42 Po103 Po100 Po100 Po101 OTV Unicast MAC Routing Table For Overlay1 VLAN MAC-Address Metric Uptime Owner Next-hop(s) f.6c75.1d :02:25 overlay EAST_OTVA East vlan f.6c75.1d42 65

66 Verification Unicast Forwarding: User on Site West Sends Unicast Packet to Site East Verify next hop IP address for MAC WEST_OTVA# show adjacency Overlay Adjacency database West Overlay-Interface Overlay1 : Hostname System-ID Dest Addr Up Time State EAST_OTVA 6c9c.ed :34:34 East UP EAST_OTVB 64a0.e741.c :34:30 UP WEST_OTVB 64a0.e741.c :34:30 UP Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 66

67 Verification Unicast Forwarding: User on Site West Sends Unicast Packet to Site East Verify hardware adjacency and label OTV shim includes an MPLS label to identify the encapsulated VLAN. MPLS Label = 32 + vlan = 233 = 0xe9 West GRE Tunnel built between WEST_OTVA and EAST_OTVA vlan b.d WEST_OTVA# show system internal forwarding overlay 1 vlan 201 detail Dev No: 1 MPLS Label: 0xe9 VPN id: 10 TCAM Idx: 0x1ffa9 Adj Idx: 0x4302a Type: EoMPLS rc: 1 ccc: 6 pv: 0 l2_fwd: 1 Packets: 0 Bytes: 0 (BD: 21, Vlan Id: 201, Peer Id: 1) DevNo: 1 TCAM Idx: 0x1ff03 Adj Idx: 0x4303d Egress Lif: 0x8004 RDT: 1 Egress Lif East Base: 5 ccc: 0 pv: 1 l2_fwd: 1 rc: 1 label hi: 0x0 label low: 0xe90 Packets: 0 Bytes: 0 Tunnel adj index: 0x43037 GRE: YES Po103 Po100 Po100 Po101 DIP: SIP: LIF: 0x4074 DI: 0x421 ccc: 6 L2_FWD: NO RDT: YES Packets: 0 Bytes: 0 zone enforce: 0 vlan f.6c75.1d42 67

68 Verification Unicast Forwarding: User on Site West Sends Unicast Packet to Site East Verify incrementing counters on tunnel interface West WEST_OTVA# show tunnel internal implicit detail Tunnel16397 is up MTU 9178 bytes, BW 9 Kbit Transport protocol is in VRF "default" Tunnel protocol/transport GRE/IP Tunnel source , destination Last clearing of "show interface" counters never Tx packets output, 1 minute output rate 171 packets/sec Rx packets input, 1 minute input rate 171 packets/sec Po103 Po100 Po100 Po101 East vlan b.d vlan f.6c75.1d42 68

69 Unicast Forwarding: Encapsulated Packet 42 Byte OTV header overhead Increases packet size to 142 Bytes Packet sent unicast through transport with endpoint IP s of AED Join interface Label to identify vlan = 233 Original IP header maintained 69

70 Verification Unicast Forwarding: User on site West sends unicast packet to site East Encapsulated packet is received on Join interface of East AED For Decapsulation, verify hits against internal ACL EAST_OTVA# show system internal access-list output statistics begin Tcam Tcam 0 resource usage: Label_a = 0x802 Bank IPv4 West Class Policies: Tunnel Decap(Tunnel Decap on VRF default) Entries: [Index] Entry Po103 [Stats] Po100 Po100 Po [0006] redirect(0x4307d) / /32 [0] [0007] redirect(0x4307b) / /32 [0] vlan 201 [0009] redirect(0x43079) / /32 [18505] [0010] 001b.d permit ip / /0 [1444] East vlan f.6c75.1d42 70

71 Verification Unicast Forwarding: User on site West sends unicast packet to site East Verify OTV route for entry points to local site Verify CAM entry for destination MAC points out internal interface EAST_OTVA# show route 001f.6c75.1d42 vlan 201 OTV Unicast MAC Routing Table For Overlay1 VLAN MAC-Address Metric Uptime Owner Next-hop(s) West f.6c75.1d :01:50 site port-channel101 EAST_OTVA# show mac address-table address 001f.6c75.1d42 vlan 201 Po103 Po100 Po100 Po101 Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link vlan VLAN 201 MAC Address Type age Secure NTFY Ports/SWID.SSID.LID * b.d f.6c75.1d42 dynamic 0 F F Po101 East vlan f.6c75.1d42 71

72 Verification Unicast Forwarding: User on site West sends unicast packet to site East Packet will be sent out internal interface at site East and L2 switched to the host Return path from East to West will be the same West East Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 72

73 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 73

74 Verification Multicast Forwarding: IGMP Join from Client Client at West site requests traffic for multicast group There is no server sending traffic for this group Reference Slide West East Po103 Po100 Po100 Po101 vlan 201 Multicast Client vlan 201 Multicast Server

75 Verification Multicast Forwarding: IGMP Join from Client Multicast Transport Reference Slide WEST_OTVA# show ip igmp snooping groups vlan 201 Type: S - Static, D - Dynamic, R - Router port, F - Fabricpath core port Vlan Group Address Ver Type Port list v2 D Po103 WEST_OTVA# show mroute vlan 201 detail West vlan 201 Multicast Client OTV Multicast Routing Table For Overlay1 (201, *, ), metric: 0, uptime: 00:00:21, igmp Outgoing interface list: (count: 1) Po103 Po100 Po100 Po101 Po103, uptime: 00:00:21, igmp East vlan 201 Multicast Server

76 Verification Multicast Forwarding: IGMP Join from Client Multicast Transport Reference Slide (r) means there is a receiver that exists across the overlay EAST_OTVA# show mroute vlan 201 OTV Multicast Routing Table For Overlay1 (201, *, ), metric: 0, uptime: 00:00:01, overlay(r) Outgoing interface list: (count: 1) Overlay1, uptime: 00:00:01, isis_-default West East Po103 Po100 Po100 Po101 vlan 201 Multicast Client vlan 201 Multicast Server

77 Verification Multicast Forwarding: IGMP Join from Client Unicast Transport Reference Slide EAST_OTVA# show mroute vlan 201 detail OTV Multicast Routing Table For Overlay1 (201, *, ), metric: 0, uptime: 00:00:46, overlay(r) Outgoing interface list: (count: 1) Overlay1, WEST_OTVA, uptime: 00:00:46, isis_-default West East vlan 201 Multicast Client Po103 Po100 Po100 Po101 OIL contains Overlay and OTV neighbor in unicast mode as edge device is responsible for replications vlan 201 Multicast Server

78 Verification Multicast Forwarding: Multicast Server Discovery Server at East site starts sending traffic to group There are no clients requesting the stream Reference Slide West East Po103 Po100 Po100 Po101 vlan 201 Multicast Client vlan 201 Multicast Server

79 Verification Multicast Forwarding: Multicast Server Discovery Multicast Transport Reference Slide EAST_OTVA# show mroute vlan 201 detail OTV Multicast Routing Table For Overlay1 (201, , ), metric: 0, uptime: 00:00:30, site Outgoing interface list: (count: 0) Local Delivery: s = , g = EAST_OTVA# show data-group vlan 201 West Local Active Sources for Overlay1 East VLAN Active-Source Po103 Active-Group Po100 Delivery-Source Delivery-Group Join-IF State Po100 Po Po100 Local vlan 201 Multicast Client vlan 201 Multicast Server

80 Verification Multicast Forwarding: Multicast Server Discovery Multicast Transport Reference Slide (s) means there is a source that exists across the overlay WEST_OTVA# show mroute vlan 201 detail OTV Multicast Routing Table For Overlay1 (201, , ), metric: 0, uptime: 00:00:53, overlay(s) Outgoing interface list: (count: 0) Remote Delivery: s = , g = West vlan 201 Multicast Client WEST_OTVA# show data-group vlan 201 Remote Active Sources for Overlay1 Po103 Po100 Po100 Po101 VLAN Active-Source Active-Group Delivery-Source Delivery-Group Joined-I/F vlan 201 Multicast Server East 80

81 Verification Multicast Forwarding: Multicast Server Discovery Unicast Transport Reference Slide EAST_OTVA# show mroute vlan 201 detail OTV Multicast Routing Table For Overlay1 (201, , ), metric: 0, uptime: 00:00:11, site Outgoing interface list: (count: 0) Local Delivery: s = , g = West EAST_OTVA# show data-group vlan 201 Local Active Sources for Overlay1 Po103 Po100 Po100 Po101 VLAN Active-Source Active-Group Delivery-Source Delivery-Group Join-IF State vlan Multicast Client Local No delivery group in unicast mode East vlan 201 Multicast Server

82 Verification Multicast Forwarding: Multicast Server Discovery Unicast Transport Reference Slide No state created on West site WEST_OTVA# show mroute vlan 201 WEST_OTVA# show data-group vlan 201 West East Po103 Po100 Po100 Po101 vlan 201 Multicast Client vlan 201 Multicast Server

83 Verification Multicast Forwarding: Source and Client Present Multicast Transport (VLAN, *, G) created on West Side on receiving IGMP join from client and advertised to all edge devices WEST_OTVA knows of remote (VLAN,S,G) based on advertisement from EAST_OTVA West Po103 Po100 Po100 Po101 WEST_OTVA# show data-group Delivery group allocated by Remote Active Sources for Overlay1 EAST_OTVA for (VLAN,S,G) vlan 201 Multicast Client WEST_OTVA# show mroute vlan 201 detail OTV Multicast Routing Table For Overlay1 (201, *, ), metric: 0, uptime: 00:01:50, igmp Outgoing interface list: (count: 1) Po103, uptime: 00:01:50, igmp WEST_OTVA sends an IGMPv3 SSM join for the Delivery Source and the Delivery Group on its Join interface (201, , ), metric: 0, uptime: 00:04:47, overlay(s) Outgoing interface list: (count: 0) Remote Delivery: s = , g = East VLAN Active-Source Active-Group Delivery-Source Delivery-Group vlan Joined-I/F Multicast Server Po100 83

84 Verification Multicast Forwarding: Source and Client Present Multicast Transport EAST_OTVA# show data-group Local Active Sources for Overlay1 VLAN Active-Source Active-Group Delivery-Source Delivery-Group Join-IF State Po100 Local Local delivery group is allocated for directly connected source (VLAN, *, G) known on EAST_OTVA based off advertisement from WEST_OTVA (r) means there is a receiver that exists across the overlay EAST_OTVA# show mroute vlan 201 detail OTV Multicast West Routing Table For Overlay1 On receiving multicast traffic East from the local host, EAST_OTVA creates a (VLAN,S,G) (201, *, ), metric: 0, uptime: 00:01:42, overlay(r) and advertises to all edge devices Outgoing interface Po103 list: (count: Po100 1) Po100 Po101 Overlay1, uptime: 00:01:42, isis_-default Since there is a receiver across the overlay, Overlay1 is added in outgoing interface list (201, , ), metric: 0, uptime: 00:04:42, site vlan Outgoing 201 interface list: (count: 1) vlan 201 Multicast Overlay1, Client uptime: 00:01:42, Multicast Server Local Delivery: s = , g =

85 Multicast Encapsulated Packet, Multicast Transport Sourced from Join interface, and destined to first address in data-group Label to identify vlan = 233 Original IP header maintained 85

86 Verification Multicast Forwarding: Source and Client Present Unicast Transport Since core does not support multicast, West site cannot send SSM join for group. Instead, West needs only to communicate to East that it has a receiver and it will receive the group via unicast. Only the *,G is created in unicast mode on client site. West vlan 201 Multicast Client WEST_OTVA# show mroute detail OTV Multicast Routing Table For Overlay1 Po103 Po100 (201, *, ), metric: 0, uptime: Po100 00:00:17, igmp Po101 Outgoing interface list: (count: 1) Po103, uptime: 00:00:17, igmp East vlan 201 Multicast Server

87 Verification Multicast Forwarding: Source and Client Present Unicast Transport (VLAN,*,G) known on EAST_OTVA with Outgoing interface list containing Overlay1 WEST_OTVA EAST_OTVA# show mroute vlan 201 OTV Multicast Routing Table For Overlay1 (201, *, ), metric: 0, uptime: 00:00:46, overlay(r) Outgoing interface list: (count: 1) Overlay1, WEST_OTVA, uptime: 00:00:46, isis_-default Local (VLAN,S,G) created on EAST_OTVA on receiving directly connected source. WEST_OTVA copied into OIL based off (VLAN,*,G) (201, , ), metric: 0, uptime: 00:01:54, site Outgoing interface list: (count: 1) West East Overlay1, WEST_OTVA, uptime: 00:00:46, multicast group is encapsulated and sent via unicast to WEST_OTVA EAST_OTVA# show forwarding multicast route vlan 201! Some output omitted Po103 Po100 Po100 Po101 ( /32, /32), RPF Interface: NULL, flags: Received Packets: Bytes: Number of Outgoing Interfaces: 1 vlan 201 vlan 201 Outgoing Interface List Index: 25 Multicast Client Multicast Server Tunnel16407 Outgoing Packets:41096 Bytes: OTV unicast tunnel end-points: ( , ) vlan:

88 Multicast Encapsulated Packet, Unicast Transport Source and Destination IP between Join Interfaces Label to identify vlan = 233 Original IP header maintained 88

89 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 89

90 Verification Address Resolution Protocol (ARP) We will assume that none of the devices in the topology have ARP or CAM entries for the hosts in vlan 201 West East Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 90

91 Verification ARP: Host at West Site Sends ARP Request for Host at East 1. Since it s a broadcast packet, it is forwarded to both the OTV devices at West site 2. Non AED at West site drops the broadcast packet (loop prevention) 3. AED learns the MAC address on its internal interface West vlan b.d WEST_OTVA# show mac address-table vlan 201 Po103 Po100 Legend: Po100 Po101 * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID vlan 201 * b.d dynamic 0 F F Po f.6c75.1d42 East 91

92 Verification ARP: Host at West Site Sends ARP Request for Host at East 4. On learning new MAC, West AED sends ISIS update to all OTV devices Single packet on multicast control group (Multicast Transport) Or, unicast to each adjacency (Unicast Transport) 5. Only AED at remote sites program new MAC into OTV route and CAM tables EAST_OTVA# show isis database detail i Host Vlan MAC Hostname West : WEST_OTVA Length : 9 Vlan : 201 : Metric : 1 MAC Address : 001b.d Po103 Po100 Po100 Po101 EAST_OTVA# show route vlan 201 VLAN MAC-Address Metric Uptime Owner Next-hop(s) vlan 001b.d :00:08 overlay WEST_OTVA EAST_OTVA# 001b.d show mac address-table vlan 201 VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID O b.d dynamic 0 F F Overlay1 East vlan f.6c75.1d42 92

93 Verification ARP: Host at West Site Sends ARP Request for Host at East 6. West AED performs lookup in ARP-ND (ARP- IPv6 Neighbor Discover) cache for East Host IP If an entry were present, West could send ARP reply (proxy) to local host without forwarding packet across overlay West WEST_OTVA# show arp-nd-cache OTV ARP/ND L3->L2 Address Mapping Cache! empty East Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 93

94 Verification ARP: Host at West Site Sends ARP Request for Host at East 7. Since there is no entry present in cache, West encapsulates ARP broadcast and sends to all OTV devices Single packet on multicast control group (Multicast Transport) Or, unicast to each adjacency (Unicast Transport) West East Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 94

95 Verification ARP: Encapsulated ARP Request Reference Slide Sourced from IP of Join Interface at West an destined to control group (or unicast to each neighbor) Broadcast ARP Request 95

96 Verification ARP: Host at West Site Sends ARP Request for Host at East 8. AED at East site receives packet on Join interface, decapsulates and sends it on internal interface toward host Non AED at East will also receive packet but will not forward West East Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 96

97 Verification ARP: Host at East Site Sends ARP Reply for Host at West 1. AED at East receives unicast Reply on its internal interface 2. East updates its CAM table with the MAC address pointing out its internal interface West East Po103 Po100 Po100 Po101 EAST_OTVA# show mac address-table vlan 201 Legend: vlan 201 * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN 001b.d MAC Address Type age Secure NTFY Ports/SWID.SSID.LID O b.d dynamic 0 F F Overlay1 * f.6c75.1d42 dynamic 0 F F Po101 vlan f.6c75.1d42 97

98 Verification ARP: Host at East Site Sends ARP Reply for Host at West 3. On learning new MAC, East sends ISIS update to all OTV devices Single packet on multicast control group (Multicast Transport) Or, unicast to each adjacency (Unicast Transport) 4. Only AED at remote sites program new MAC into OTV route and CAM tables West East Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 98

99 Verification ARP: Host at East Site Sends ARP Reply for Host at West 3. On learning new MAC, East sends ISIS update to all OTV devices Single packet on multicast control group (Multicast Transport) WEST_OTVA# show isis database detail i Host Vlan MAC Hostname : WEST_OTVA Length : 9 Vlan : 201 : Metric : 1 MAC Address : 001b.d Or, unicast to each adjacency (Unicast Transport) 4. Only AED at remote sites Hostname program : EAST_OTVA new MAC into Length OTV : 9 route and CAM tables Vlan : 201 : Metric : 1 MAC Address : 001f.6c75.1d42 West vlan b.d WEST_OTVA# show route vlan 201 VLAN MAC-Address Metric Uptime Owner Next-hop(s) b.d :00:16 site Po103 Po103 Po100 Po100 Po f.6c75.1d :00:08 overlay EAST_OTVA East WEST_OTVA# show mac address-table vlan 201 VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID vlan * b.d dynamic 0 F F Po f.6c75.1d42 O f.6c75.1d42 dynamic 0 F F Overlay1 99

100 Verification ARP: Host at East Site Sends ARP Reply for Host at West 5. East performs lookup in its CAM for unicast destination. Because of previous ISIS update, East finds an entry pointing out overlay toward West 6. East encapsulates ARP reply and sends via unicast to West West East Po103 Po100 Po100 Po101 vlan b.d vlan f.6c75.1d42 100

101 Verification ARP: Encapsulated ARP Reply Reference Slide Sourced from IP of Join Interface at East destined to Join interface at West Unicast ARP Reply 101

102 Verification ARP: Host at East Site Sends ARP Reply for Host at West 7. West receives packet on Join Interface, decapsulates packet and sends out internal interface toward host 8. West updates ARP-ND cache for East Host from ARP received on Overlay WEST_OTVA# show arp-nd-cache OTV ARP/ND L3->L2 Address Mapping Cache West vlan b.d Overlay Interface Overlay1 VLAN MAC Address Layer-3 Address Age Expires In f.6c75.1d :00:04 00:07:55 Po103 Po100 Po100 Po101 East vlan f.6c75.1d42 102

103 Agenda OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 103

104 Troubleshooting MTU Verify appropriate MTU via ping between OTV join interfaces packet-size in NxOS represents size of data in ICMP packet To test MTU, must account for 8 Byte ICMP header, 20 Byte IP header Example: = , use packet-size of = , use packet-size of = , use packet-size of 1514 WEST_OTVA#! Verify transport supports MTU of 1542 WEST_OTVA# ping packet-size 1514 df-bit PING ( ): 1514 data bytes 1522 bytes from : icmp_seq=0 ttl=251 time=2.333 ms 104

105 Troubleshooting Partial Adjacency (!) Flag implies a mismatch site-id due 1. Receiving same site-id across overlay without site adjacency 2. Receiving different site-id across site adjacency WEST_OTVA# show site Dual Adjacency State Description Full - Both site and overlay adjacency up Partial - Either site/overlay adjacency down Down - Both adjacencies are down (Neighbor is down/unreachable) (!) - Site-ID mismatch detected! Some output omitted Hostname System-ID Adjacency- Adjacency- AED- State Uptime Capable WEST_OTVB 64a0.e741.c842 Partial (!) 00:15:16 No 105

106 Troubleshooting Partial Adjacency Partial Adjacency implies either the site or overlay adjacency is down. Both are required to maintained a full adjacency Most common reason for down overlay adjacency is PIM misconfiguration in transport (multicast transport) or insufficient MTU Ensure 1. Matching site-id configuration at each site 2. Sufficient MTU through transport 3. IP connectivity between each edge device 4. Correct PIM configuration through core for multicast transport 106

107 Troubleshooting ARP and CAM timer issue Asymmetrical routing with mis-match ARP timers can cause traffic to blackhole across OTV West N7k Po103 Po100 Po100 Po East Vlan 201: Vlan 202: c9c.ed Vlan 201: Vlan 202: f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46 107

108 Troubleshooting ARP and CAM timer issue Reference Slide Host1 on vlan 201 sends packet toward Host2 who sits on vlan 202 Since hosts are on different subnets, Host1 sends unicast packet toward its gateway to route the packet West N7k Po103 Po100 Po100 Po East Vlan 201: Vlan 202: c9c.ed Vlan 201: Vlan 202: f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46

109 Troubleshooting ARP and CAM timer issue Reference Slide The gateway on West site will use normal ARP procedure and resolve the address of Host2 on vlan 202 This process will update CAM entries on each OTV edge device West N7k Po103 Po100 Po100 Po East Vlan 201: Vlan 202: c9c.ed Vlan 201: Vlan 202: f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46 109

110 Troubleshooting ARP and CAM timer issue Reference Slide Similar process will occur on East site as Host2 on vlan 202 sends packet toward its local gateway to reach Host1. Gateway at East will ARP for Host1, OTV edge devices will learn MAC of East gateway and Host1 West N7k Po103 Po100 Po100 Po East Vlan 201: Vlan 202: c9c.ed Vlan 201: Vlan 202: f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46 110

111 Troubleshooting ARP and CAM timer issue Reference Slide At this point, each OTV device learns the Hosts and gateway MACs on each vlan West Host 1, vlan , GW: b.d WEST_OTVA# show mac add vlan 201 Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link East VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID Po103 Po100 O f179.b640 dynamic 0 Po100 F F Overlay1 Po101 * b.d dynamic 400 F F Po103 * 201 6c9c.ed dynamic 400 F F Po103 N7k 6500 Vlan 201: Vlan 202: c9c.ed WEST_OTVA# show mac add vlan 202 Vlan 201: Legend: Vlan 202: * - primary entry, G - Gateway MAC, (R) - Routed 0014.f179.b640 MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID Host 2, vlan 202 O f179.b640 dynamic 0 F F , Overlay1 GW: O f.6c75.1d46 dynamic 0 F F 001f.6c75.1d46 Overlay1 * 202 6c9c.ed dynamic 0 F F Po

112 Troubleshooting ARP and CAM timer issue Reference Slide At this point, each OTV device learns the Hosts and gateway MACs on each vlan EAST_OTVA# show mac add vlan 201 Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC West age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID * f179.b640 Po103 dynamic Po100 0 F F Po101 Po100 Po101 O b.d dynamic 0 F F Overlay1 O 201 6c9c.ed dynamic 0 F F Overlay1 N7k 6500 Vlan 201: EAST_OTVA# show mac add vlan 202 Vlan 202: Legend: 6c9c.ed * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID Host 1, vlan 201 * , 0014.f179.b640 GW: dynamic 410 F F Po b.d * f.6c75.1d46 dynamic 410 F F Po101 O 202 6c9c.ed dynamic 0 F F Overlay1 Vlan 201: Vlan 202: f179.b640 East Host 2, vlan , GW: f.6c75.1d46 112

113 Troubleshooting Since the traffic flow between Host1 and Host2 is routed traffic, OTV will only see source MAC of the gateways and destination of the Hosts ARP 25 min West DA: Host2 SA: N7k1 CAM 30 min DA: Host2 SA: N7k1 N7k Po103 Po100 Po100 Po DA: Host2 SA: N7k1 CAM 30 min DA: Host2 SA: N7k1 ARP 4 hours East DA: N7k SA: Host1 Vlan 201: Vlan 202: c9c.ed Vlan 201: DA: Host2 Vlan 202: SA: N7k f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46 113

114 Troubleshooting Since the traffic flow between Host1 and Host2 is routed traffic, OTV will only see source MAC of the gateways and destination of the Hosts ARP 25 min West DA: Host1 SA: 6500 CAM 30 min DA: Host1 SA: 6500 N7k Po103 Po100 Po100 Po DA: Host1 SA: 6500 CAM 30 min DA: Host1 SA: 6500 ARP 4 hours East DA: Host1 SA: 6500 Vlan 201: Vlan 202: c9c.ed Vlan 201: DA: 6500 Vlan 202: SA: Host f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46 114

115 Troubleshooting ARP and CAM timer issue Reference Slide The Nexus7000 gateway at West will refresh its ARP entry for Host2 after 25 minutes. This process will also update the CAM timer for the OTV edge devices West N7k Po103 Po100 Po100 Po East Vlan 201: Vlan 202: c9c.ed Vlan 201: Vlan 202: f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46 115

116 Troubleshooting ARP and CAM timer issue Reference Slide The 6500 will not refresh ARP for 4 hours by default After 30 minutes, OTV West will age out Host1 MAC from its CAM. West will send ISIS update to all OTV neighbors to remove route to Host1 West N7k Po103 Po100 Po100 Po East Vlan 201: Vlan 202: c9c.ed Vlan 201: Vlan 202: f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46 116

117 Troubleshooting ARP and CAM timer issue OTV does not send unknown unicast traffic across Overlay Subsequent packets from East toward Host1 will be dropped until Host1 MAC is relearned on West West N7k Po103 Po100 Po100 Po East EAST_OTVA# show mac add vlan 201 Legend: Vlan 201: * - primary Vlan 202: entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - 6c9c.ed seconds since last seen,+ - primary entry using vpc Peer-Link VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID * 201 Host 1, vlan 0014.f179.b dynamic 0 F F Po101 O , 6c9c.ed GW: dynamic 0 F F Overlay1 001b.d ! Host1 (001b.d ) MAC has been removed Vlan 201: Vlan 202: f179.b640 Host 2, vlan , GW: f.6c75.1d46 117

118 Troubleshooting ARP and CAM timer issue - Solution Solution: Change 6500 ARP timer to be less than OTV CAM timer ARP 25 min West CAM 30 min CAM 30 min ARP 25 min N7k Po103 Po100 Po100 Po East Vlan 201: Vlan 202: c9c.ed Vlan 201: Vlan 202: f179.b640 Host 1, vlan , GW: b.d Host 2, vlan , GW: f.6c75.1d46 118

119 Troubleshooting Network Load Balancer Services Some network load balancer services (NLBS) rely on flooding to reach all devices in the cluster Clusters that rely on a unicast IP to multicast MAC will be forwarded across overlay in same fashion as a broadcast packet without any additional configurations Encapsulated within the control group (multicast transport) Unicast to each OTV neighbor (unicast transport) Clusters that rely on a unicast IP to unicast MAC will be dropped 119

120 Troubleshooting Network Load Balancer Services Single Site Static MAC Solution Another Option: Force Flooding for Virtual MAC EAST_OTVA# show run i static mac address-table static ac9.00a2 vlan 201 interface port-channel101 EAST_OTVA# show route vlan 201 OTV Unicast MAC Routing Table For Overlay1 West VLAN MAC-Address Metric Uptime Owner Next-hop(s) ac9.00a2 1 00:00:11 site port-channel101 East WEST_OTVA# show route vlan 201 Po103 Po100 Po100 Po101 OTV Unicast MAC Routing Table For Overlay1 NLB Cluser ac9.00a2 VLAN MAC-Address Metric Uptime Owner Next-hop(s) ac9.00a :04:34 overlay EAST_OTVA 120

121 Troubleshooting Network Load Balancer Services Selective Unicast Flooding Solution Targeted 6.2 EAST_OTVA# show run i flood flood mac ac9.00a2 vlan 201! Same configuration on WEST_OTVA EAST_OTVA# show route vlan 201 i static VLAN MAC-Address Metric Uptime Owner Next-hop(s) ac9.00a2 0 00:00:57 static West East NLB Cluser ac9.00a2 Po103 Po100 Po100 Po101 Cluster can now exists at both sites NLB Cluser ac9.00a2 121

122 Troubleshooting Common Misconfiguration - First Hop Redundancy Protocols (FHRP) Isolation FHRP Isolation allows for local egress routing at each site to reduce traffic sent across the overlay. I.e., active gateways with the same Virtual IP and Virtual MAC at each site. Active Standby Active Standby OTV Block FHRP Traffic 122

123 Troubleshooting Common Misconfiguration - First Hop Redundancy Protocols (FHRP) Isolation 1. VLAN Access List (VACL) to drop Hellos! IP ACL's to drop HSRP Hellos,! forward other traffic ip access-list HSRP_IP 10 permit udp any /32 eq permit udp any /32 eq 1985 ip access-list ALL_IPs 10 permit ip any any! MAC ACL's to drop non-ip HSRP traffic,! forward other traffic mac access-list HSRP_VMAC 10 permit c07.ac ff any 20 permit c9f.f fff any mac access-list ALL_MACs 10 permit any any! Create the VACL vlan access-map HSRP_Localization 10 match mac address HSRP_VMAC match ip address HSRP_IP action drop vlan access-map HSRP_Localization 20 match mac address ALL_MACs match ip address ALL_IPs action forward! Apply the VACL to each extended vlan vlan filter HSRP_Localization vlan-list <OTV_Extended_VLANs> 123

124 Troubleshooting Common Misconfiguration - First Hop Redundancy Protocols (FHRP) Isolation 2. ARP Inspection Filter to drop ARP sourced from the Virtual MAC (preventing duplicate IP messages between Active Devices at each site)! Feature dhcp required for ARP inspection feature dhcp! Create the ARP access-list to deny traffic from Virtual MAC arp access-list HSRP_VMAC_ARP 10 deny ip any mac c07.ac00 ffff.ffff.ff00 20 deny ip any mac c9f.f000 ffff.ffff.f permit ip any mac any! Apply ARP ACL to each extended VLAN ip arp inspection filter HSRP_VMAC_ARP vlan <OTV_Extended_VLANs> 124

125 Troubleshooting Common Misconfiguration - First Hop Redundancy Protocols (FHRP) Isolation 3. Apply Route-Map to each Overlay to filter Virtual MAC (prevents virtual MAC from flapping between sites)! mac-list to deny advertising virtual MAC mac-list OTV_HSRP_VMAC_deny seq 10 deny c07.ac00 ffff.ffff.ff00 mac-list OTV_HSRP_VMAC_deny seq 11 deny c9f.f000 ffff.ffff.f000 mac-list OTV_HSRP_VMAC_deny seq 20 permit ! create the route-map route-map OTV_HSRP_filter permit 10 match mac-list OTV_HSRP_VMAC_deny! apply route-map to each overlay -isis default vpn Overlay1 redistribute filter route-map OTV_HSRP_filter 125

126 Summary OTV Introduction Configuration Multicast Transport Unicast-only Transport New Features Verification Adjacency Unicast Forwarding Multicast Forwarding ARP Troubleshooting 126

127 Further Reading and Resources Recommended Sessions: BRKDCT-2049 Overlay Transport Virtualization BRKDCT-2131 Mobility and Virtualization in the Data Center with LISP and OTV BRKDCT-3060 Deployment challenges with Interconnecting Data Centers BRKRST-3046 Advanced LISP - What's In It For Me? OTV Technology Introduction and Deployment Considerations: Configuring OTV IOS XE Wireshark 127

128 Complete Your Online Session Evaluation Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Cisco Daily Challenge points for each session evaluation you complete. Complete your session evaluation online now through either the mobile app or internet kiosk stations. Maximize your Cisco Live experience with your free Cisco Live 365 account. Download session PDFs, view sessions on-demand and participate in live activities throughout the year. Click the Enter Cisco Live 365 button in your Cisco Live portal to log in. 128

129 Final Thoughts Get hands-on experience with the Walk-in Labs located in World of Solutions, booth 1042 Come see demos of many key solutions and products in the main Cisco booth 2924 Visit after the event for updated PDFs, on-demand session videos, networking, and more! Follow Cisco Live! using social media: Facebook: Twitter: LinkedIn Group: 129

130

131 Appendix ASR 1000 Support beginning in 3.5S Advance Enterprise Image or Advance IP Service (AES or AIS) to have the cli enabled Extended and site VLANs configured via EFP s and bridge-domains Multi-homing ASR and N7k OTV at same site is not supported (must be located at different sites) Support for multicast transport Support for unicast transport starting in 3.9S 131

132 Appendix ASR 1000 Configuration Internal Interface Site-ID and Site Bridge-Domain required Bridge-Domain must be forwarding on internal interface before adjacencies will be built site bridge-domain 210 site-identifier Bridge-domain for an extended VLAN Site Bridge-domain must be active on internal interface interface GigabitEthernet1/0/2 no ip address cdp enable service instance 201 ethernet encapsulation dot1q 201 bridge-domain 201 service instance 210 ethernet encapsulation dot1q 210 bridge-domain

133 Appendix ASR 1000 Configuration Join Interface Join Interface must be configured with IGMPv3 for multicast transport. Multicast routing must be enabled Enable IGMP snooping querier Configure PIM Passive mode on Join Interface ip multicast-routing distributed ip igmp snooping querier version 3 ip igmp snooping querier interface GigabitEthernet1/0/1 mtu 9000 ip address ip pim passive ip igmp version 3 133

134 Appendix ASR 1000 Configuration Overlay Configure control and data-groups Specify join-interface Create service instance for each bridge-domain that should be extended across overlay Do not extend site bridge-domain interface Overlay1 no ip address control-group data-group /24 join-interface GigabitEthernet1/0/1 service instance 201 ethernet encapsulation dot1q 201 bridge-domain

135 Appendix ASR 1000 Verify Overlay is UP SOUTH_OTVA#show Overlay Interface Overlay1 VPN name : None VPN ID : 1 State : UP AED Capable : Yes IPv4 control group : Mcast data group range(s): /24 Join interface(s) : GigabitEthernet1/0/1 Join IPv4 address : Tunnel interface(s) : Tunnel0 Encapsulation format : GRE/IPv4 Site Bridge-Domain : 210 Capability : Multicast-reachable Is Adjacency Server : No Adj Server Configured : No Prim/Sec Adj Svr(s) : None 135

136 Appendix ASR 1000 Verify Site Adjacency and AED SOUTH_OTVA#show site Site Adjacency Information (Site Bridge-Domain: 210) Overlay1 Site-Local Adjacencies (Count: 2) Hostname System ID Last Change Ordinal AED Enabled Status SOUTH_OTVB 001D.707E.1B00 01:02:23 0 site overlay * SOUTH_OTVA 001D.707E.3A00 00:42:08 1 site overlay SOUTH_OTVA#show vlan Key: SI - Service Instance Overlay 1 VLAN Configuration Information Inst VLAN Bridge-Domain Auth Site Interface(s) yes Gi1/0/2:SI201 Total VLAN(s): 1 Total Authoritative VLAN(s): 1 136

137 Appendix ASR 1000 Verify Overlay Adjacencies SOUTH_OTVA#show adjacency Overlay 1 Adjacency Database Hostname System-ID Dest Addr Up Time State EAST_OTVB 64a0.e741.c :06:21 UP WEST_OTVB 64a0.e741.c :06:21 UP WEST_OTVA 6c9c.ed :06:21 UP EAST_OTVA 6c9c.ed :06:21 UP SOUTH_OTVB 001d.707e.1b :29:48 UP Peering between ASR and N7k between sites is supported. 137

138 Appendix ASR 1000 Verify Locally and Remotely Learned Routes SOUTH_OTVA#show bridge-domain 201 mac dynamic address Port MAC Address Gi1/0/2 ServInst a.e2be.52cd SOUTH_OTVA#show route vlan 201 Codes: BD - Bridge-Domain, AD - Admin-Distance, SI - Service Instance, * - Backup Route Locally Learned MAC from Bridge-domain 201 OTV Unicast MAC Routing Table for Overlay1 Inst VLAN BD MAC Address AD Owner Next Hops(s) a.e2be.52cd 40 BD Eng Gi1/0/2:SI b.d ISIS WEST_OTVA f.6c75.1d42 50 ISIS EAST_OTVA MAC learned via ISIS from OTV peer 138

139 Appendix ASR 1000 Multicast Local Receiver (Bridge-domain, *,G) programmed based on IGMP join from client (Bridge-domain, S,G) created to deliver to local receiver once received from overlay SOUTH_OTVA#show mroute OTV Multicast Routing Table for Overlay1 Bridge-Domain = 201, s = *, g = * Outgoing interface list: Default, NoRedist Incoming interface count = 0, Outgoing interface count = 1 Bridge-Domain = 201, s = *, g = Outgoing interface list: Service Instance 201, GigabitEthernet1/0/2 Incoming interface count = 0, Outgoing interface count = 1 Bridge-Domain = 201, s = , g = Incoming interface list: Service Instance 201, Overlay1, 001f.6c75.1d42 Incoming interface count = 1, Outgoing interface count = 0 139

140 Appendix ASR 1000 Multicast Local Receiver SOUTH_OTVA#show data-group Flags: D - Local active source dynamically detected S - Local active source statically configured J - Data group has been joined in the core U - Data group has not been joined in the core IGMPv3 join sent to core for delivery group Remote Active Sources for Overlay1 BD Active-Source Active-Group Delivery-Source Delivery-Group Flags J 140