Troubleshooting End-to-End MPLS

Transcription

1

2 Troubleshooting End-to-End MPLS Vinit Jain - CCIE# Twitter BRKMPL-3124

3 Coming this year BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 3

4 Agenda Fundamentals Troubleshooting LDP Issues Troubleshooting MPLS LSP Troubleshooting MPLS L3 VPNs Troubleshooting 6VPE Inter-AS MPLS VPNs Conclusion

5 Introduction Housekeeping Who am I? Who are you? Service Provider Enterprise Enterprises using MPLS Studying for CCIE Advanced Class Assume MPLS Operational Experience Basic configuration Show commands Understand basic MPLS concepts BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 5

6 MPLS Fundamentals

7 MPLS Fundamentals MPLS Architecture MPLS has two major components: 1. Control plane: Exchanges Layer 3 routing information and labels 2. Forwarding plane: Forwards packets based on labels Control plane contains complex mechanisms to exchange routing information, such as OSPF, EIGRP, IS-IS, and BGP, and to exchange labels, such as TDP, LDP, BGP, and RSVP. Forwarding plane forwards packets based on CEF BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 7

8 MPLS Fundamentals Terminologies RIB is the Routing Information Base that is analogous to the IP routing table. FIB aka CEF is Forwarding information base that is derived from the IP routing table. LIB is Label Information Base that contains all the label bindings learned via LDP LFIB is Label Forwarding Information Base that is derived from FIB entries and corresponding LIB entries. FEC ( Forwarding Equivalence Class) Group of IP packets forwarded in the same manner (e.g. over same forwarding path) A FEC can represent a: Destination IP prefix, VPN ID, ATM VC, VLAN ID, Traffic Engineering tunnel, Class of Service. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 8

9 MPLS Fundamentals MPLS Architecture Incoming IP Packet Control Plane Data Plane Routing updates from peer router s Routing Protocol Database Label Bindings via LDP peering Routing Information Base (RIB) Label Information Base (LIB) Forwarding Information Base (FIB) Label Forwarding Information Base (LFIB) Outgoing MPLS/IP Packet Incoming MPLS Packet BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 9

10 MPLS Fundamentals MPLS Label: Label Format MPLS uses a 32-bit label field that is inserted between Layer 2 and Layer 3 headers Label COS S TTL Label = 20 bits COS/EXP = Class of Service, 3 bits S = Bottom of Stack, 1 bit TTL = Time to Live (Loop detection) BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 10

11 MPLS Fundamentals MPLS: Ethertype Ethertype 0x0800 refers to IP Ethertype 0x8847 refers to MPLS Based on the Ethertype, the packet is handed over to the appropriate processing engine on the router BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 11

12 MPLS Fundamentals MPLS Label: The Label Stack \ An MPLS packet may have more than one label Frame Mode can handle a stack of two or more labels, depending on the platform Bottom most label has the S-bit set to 1 LSRs label switch packets are based ONLY on the label at the top of the stack BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 12

13 MPLS Fundamentals MPLS Label: The Label Stack The following scenarios may produce more than one label: MPLS L3 VPNs (two labels: The top label points to the egress router and the second label identifies the VPN.) MPLS TE with Fast Reroute (FRR) (two or more labels: The top label is for the backup tunnel and the second label points to the primary tunnel destination.) MPLS VPNs combined with MPLS TE / FRR (three labels) Carrier Supporting Carrier (CSC) with MPLS TE / FRR (four labels) BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 13

14 MPLS Fundamentals Label Switch Path (LSP) IGP domain without a label distribution protocol LSP follows IGP shortest path IGP domain with a label distribution protocol LSP diverges from IGP shortest path LSPs are derived from IGP routing information LSPs may diverge from IGP shortest path LSP tunnels (explicit routing) with TE LSPs are unidirectional BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 14

15 MPLS Fundamentals Facts Check - Question Which protocols have signaling and labeling capabilities? OSPF / IS-IS RSVP LDP / TDP BGP BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 15

16 Troubleshooting LDP Issues

17 Troubleshooting LDP Issues MPLS LDP Configuration IOS / IOS XE IOS XR NX-OS mpls label protocol ldp! interface Gig 0/0 mpls ip mpls label protocol ldp exit! mpls ldp router-id loopback0 force mpls ldp router-id x.x.x.x interface gi 0/0/0/0 interface gi 0/0/0/1 install feature-set mpls feature-set mpls feature mpls mpls ldp configuration router-id x.x.x.x! interface ethernet 2/1 mpls ip BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 17

18 Troubleshooting LDP Issues LDP Neighborship LDP neighborship is formed on TCP port 646 Discovery Mechanism: Basic Discovery Multicast UDP hellos for directly connected neighbors Extended Discovery Targeted Unicast UDP hellos for non-directly connected neighbors Parameters Session Keepalive = 60 sec. & Hold time = 180 Sec. Discover Hello interval = 5 sec. and Hold Time = 15 sec. Can be viewed using the command show mpls ldp parameters BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 18

19 Troubleshooting LDP Issues LDP Neighborship Negotiation BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 19

20 Troubleshooting LDP Issues Verifying LDP Neighborship PE1#sh mpls ldp neighbor Peer LDP Ident: :0; Local LDP Ident :0 TCP connection: State: Oper; Msgs sent/rcvd: 58/60; Downstream Up time: 00:39:27 LDP discovery sources: Ethernet0/0, Src IP addr: Ethernet1/0, Src IP addr: Addresses bound to peer LDP Ident: PE1#show tcp brief i ABB ESTAB PE1# BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 20

21 Troubleshooting LDP Issues Reachability and ACL verification Ensure reachability between the LDP router ID s PE1#ping source lo0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to , timeout is 2 seconds: Packet sent with a source address of Success rate is 0 percent (0/5) Check Routing Configuration Verify no ACL in path blocking TCP port 646 and other Multicast traffic for LDP Hello s. PE1#telnet /source-interface lo0 Trying , % Destination unreachable; gateway or host down Verify ACLs in the path or on the routers itself BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 21

22 Troubleshooting LDP Issues LDP Router-id If router-id is not set manually, router checks all operational interfaces on the router(including loopbacks) and chooses the highest IP address as the LDP router-id. LDP_ID should be hardcoded via mpls ldp router-id <interface> The above configuration will not help unless: <interface> is UP when LDP gets started Existing LDP_ID (usually an interface) is shut Following avoids both shortcomings mpls ldp router-id <interface> force BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 22

23 Troubleshooting LDP issues Verifying LDP Connection show mpls ldp discovery [detail] Must show xmit/recv on LDP enabled interface PE1#show mpls ldp discovery Local LDP Identifier: :0 Local LDP_ID Discovery Sources: Interfaces: GigabitEthernet0/1 (ldp): xmit/recv LDP Id: :0 Xmited and Recvd Hellos on that interface Discovered Neighbors LDP_ID BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 23

24 Troubleshooting LDP issues Problem with xmit / recv Lo0= Lo0= PE1 PE1#show mpls ldp discovery Local LDP Identifier: :0 Discovery Sources: Interfaces: GigabitEthernet0/1 (ldp): xmit P1 P1#show mpls ldp discovery Local LDP Identifier: :0 Discovery Sources: Interfaces: GigabitEthernet0/1 (tdp): xmit R1#debug mpls ldp transport connections 07:00:06.106: ldp: Scan listening TCBs 07:01:06.106: ldp: Scan listening TCBs 07:02:06.106: ldp: Scan listening TCBs Label Protocol is TDP PE1(config-if)#mpls label protocol ldp BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 24

25 Troubleshooting LDP issues LDP No Route Problem Lo0= Lo0= PE1 PE1#show mpls ldp discovery Local LDP Identifier: :0 Discovery Sources: Interfaces: Gi0/1 (ldp): xmit/recv LDP Id: :0; no route P1 P1#show mpls ldp discovery Local LDP Identifier: :0 Discovery Sources: Interfaces: Gi0/1 (ldp): xmit/recv LDP Id: :0 PE1#show ip route % Network not in table Problem: Default route towards the peering router BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 25

26 Troubleshooting LDP issues Problem due to Summarization PE1 PE1#show mpls ldp neighbor PE1#show mpls ldp discovery Local LDP Identifier: :0 GigabitEthernet0/1 (ldp): xmit/recv LDP Id: :0 PE1#show ip route Routing entry for /32 Known via "ospf 100", distance 110, metric 2, type intra area Last update from on Gi0/1, 00:04:34 ago Routing Descriptor Blocks: * , from , 00:04:34 ago, via GigabitEthernet0/1 Route metric is 2, traffic share count is 1 P1 PE2#sh mpls ldp neighbor PE2#show mpls ldp discovery Local LDP Identifier: :0 GigabitEthernet0/1 (ldp): xmit/recv LDP Id: :0 PE2#show ip route Routing entry for /24 Known via "bgp 100", distance 200, metric 0 Tag 1, type internal Last update from :10:38 ago Routing Descriptor Blocks: * , from , 20:10:38 ago Route metric is 0, traffic share count is 1 AS Hops 5 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 26

27 Troubleshooting LDP Issues MPLS LDP Trace on IOS XR Also good to check show mpls ldp trace discovery RP/0/0/CPU0:PE2#show mpls ldp trace peer last 20 0/0/CPU0 t1 [PEER]:506: VRF(0x ): Peer( :0): Peer FSM: Stepped, pp=0x102d9548, event=0, state 0 -> 1 0/0/CPU0 t1 [PEER]:581: VRF(0x ): Peer( :0): DOWN - reason 'TCP connection closed' 0/0/CPU0 t1 [PEER]:3262: VRF(0x ): Release Peer( :0): rsn 'TCP connection closed' ('Success') 0/0/CPU0 t1 [PEER]:3625: Peer( :0): Unsupported/Unknown TLV (type 0x506, U/F 1/0) rcvd in INIT msg 0/0/CPU0 t1 [PEER]:506: VRF(0x ): Peer( :0): Peer FSM: Stepped, pp=0x102d9520, event=0, state 0 -> 1 0/0/CPU0 t1 [PEER]:575: VRF(0x ): Peer( :0): DOWN - reason 'Received Notification message from peer' (more_info 'KeepAlive Timer Expired') 0/0/CPU0 t1 [PEER]:3262: VRF(0x ): Release Peer( :0): rsn 'Received Notification message from peer' ('KeepAlive Timer Expired') 0/0/CPU0 t1 [PEER]:3625: Peer( :0): Unsupported/Unknown TLV (type 0x506, U/F 1/0) rcvd in INIT msg BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 27

28 Troubleshooting LDP Issues LDP & IGP Sync When a link comes up, LDP and IGP compete to converge; Labeled traffic drops if IGP wins. When LDP session on a link drops, IGP may continue forwarding labeled traffic to that link and cause traffic dropped. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 28

29 Troubleshooting LDP Issues LDP & IGP Sync Solution Link up: If LDP peer is reachable (alternate route exists), defer IGP adjacency on the link. If LDP peer is not reachable (no alternate route), IGP advertise max-metric to reach neighbor through the link. LDP session down: IGP advertises max-metric to reach neighbor through the link. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 29

30 Troubleshooting LDP Issues LDP & IGP Sync LDP IGP Sync feature is enabled under IGP (OSPF/ISIS) - sync-igp-shortcuts for TE tunnel interfaces, sync for all other types. router (config-isis-if-af) # mpls ldp sync [ level <1-2> ] router (config-ospf) # mpls ldp sync + (config-ospf-ar), (config-ospf-ar-if) router (config-ospf) # mpls ldp sync-igp-shortcuts + (config-ospf-ar) BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 30

31 Troubleshooting LDP Issues LDP & IGP Sync LDP IGP Sync delays are configured under LDP router (config-ldp) # igp sync delay on-session-up <sec> router (config-ldp) # igp sync delay on-proc-restart <sec> BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 31

32 Troubleshooting LDP Issues LDP Session Protection Problem: I. When a link flaps (for a short time), II. LDP hello adjacency over the link flaps III. LDP session is torn down then re-setup IV. LDP re-exchanges label bindings when LDP session is setup (i.e. LDP re-convergence). Solution: When LDP session supported by link hello is setup, create a targeted hello to protect the session. When link is down, the targeted hello remains through other path and keeps the LDP session up. When link restores, re-discover neighbors, re-program forwarding. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 32

33 Troubleshooting LDP Issues LDP Session Protection router (config-ldp) # session protection [ for <peer-acl> ] [ duration { <sec> infinite } ] router (config-ldp) # log session-protection BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 33

34 Troubleshooting MPLS LSP

35 Troubleshooting MPLS LSP Reasons for LSP to Break MP-IBGP VPNv /24 LDP + IGP / / /24 CE1 PE1 P1 PE2 CE2 Lo0= / / / /32 Lo0= /32 Broken LDP adjacency MPLS not enabled Mismatch labels Software/hardware corruption BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 35

36 Troubleshooting MPLS LSP Label Information Base (LIB) LIB stores local and remote bindings Local Binding: Prefix in own routing table + local label One binding Remote Binding: Prefix + remote label received from LDP neighbor Holds LDP router-id One binding per LDP neighbor LIB stores all labels from all LDP (BGP) neighbors, even the ones that are not used for packet forwarding (now) BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 36

37 Troubleshooting MPLS LSP Looking at the LIB RTR#show mpls ldp bindings detail tib entry: /30, rev 10 local binding: tag: imp-null Advertised to: : : :0 remote binding: tsr: :0, tag: imp-null remote binding: tsr: :0, tag: remote binding: tsr: :0, tag: BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 37

38 Troubleshooting MPLS LSP Label Forwarding Information Base (LFIB) The LFIB stores local and remote labels for prefixes that are used to forward packets Prefixes that are used = prefixes in routing table (RIB) Labels are derived from LIB LDP TDP LIB prefix, next-hop and inlabel, out-label LFIB prefix + next-hop RIB (prefix, LDP Ident, label) get in- and out-label for (prefix, next-hop) (prefix,next-hop, in-label, out-label) (prefix, next-hop) BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 38

39 Troubleshooting MPLS LSP Building the LFIB P1#show ip route Routing entry for /32 * , from , 13:28:32 ago, via Ethernet0/0 P1#show mpls ldp neighbor Peer LDP Ident: :0; Local LDP Ident :0 P1#show mpls ldp binding lib entry: /32, rev 18 remote binding: lsr: :0, label: imp-null P1#show mpls forwarding Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 20 Pop Label /32 0 Et0/ BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 39

40 Troubleshooting MPLS LSP MPLS OAM Defined in RFC 4379 LSP Ping and Traceroute provide ability to monitor MPLS Label Switched Paths and quickly isolate MPLS forwarding problems. Two messages MPLS Echo Request: MPLS labeled IPv4 or IPv6 UDP packet MPLS Echo Reply IPv4 or IPv6 UDP packet Ping mode: Connectivity check of an LSP Test if a particular FEC ends at the correct egress LSR Traceroute mode: Hop by Hop fault localization Packet follows data path BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 40

41 Troubleshooting MPLS LSP FEC Types Supported ping mpls? ipv4 Target specified as an IPv4 address pseudowire Target VC specified as an IPv4 address and VC ID traffic-eng Target specified as TE tunnel interface traceroute mpls? ipv4 Target specified as an IPv4 address multipath LSP Multipath Traceroute pseudowire Target VC specified as an IPv4 address and VC ID traffic-eng Target specified as TE tunnel interface BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 41

42 Troubleshooting MPLS LSP LSP Ping (ping mpls... ) Simple and efficient mechanism to detect data plane failures in MPLS LSPs Verify data plane against the control plane Sending echo request and receiving echo reply Verify that packets belonging to a FEC exit the LSP on the correct egress LSR Modelled after the well known IP ping and traceroute Ping verifies connectivity, traceroute verifies path LSP Ping/trace leave the LSR with the correct label stack for the LSP to be tested BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 42

43 Troubleshooting MPLS LSP Packet Format Version Number Must Be Zero Message Type Reply Mode Return Code Return Subcode Sender s Handle Sequence Number Timestamp Sent (seconds) Timestamp Sent (microseconds) Timestamp Received (seconds) Timestamp Received (microseconds) TLV BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 43

44 Troubleshooting MPLS LSP Packet Format Version number: 1 Message Type MPLS Echo Request MPLS Echo Reply Reply Mode 1 Do not reply 2 Reply via an IPv4/IPv6 UDP packet 3 Reply via an IPv4/IPv6 UDP packet with Router Alert 4 Reply via application level control channel Timestamp Time-of-day in seconds and microseconds BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 44

45 Troubleshooting MPLS LSP Reply Modes Reply Mode Do Not Reply This mode is useful for a keepalive application running at the remote end Such an application would trigger state changes if it does not receive a LSP ping packet within a predefined time An MPLS echo request with do not reply may also be used by the receiving router to log gaps in the sequence numbers and/or maintain delay/jitter statistics BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 45

46 Troubleshooting MPLS LSP Reply Modes Reply Mode Reply via an IPv4 UDP Packet The Reply via UDP packet implies that an IP V4 UDP packet should be sent in reply to an MPLS echo request This will be the most common reply mode for simple LSP pings sent to periodically poll the integrity of an LSP This is the default reply mode BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 46

47 Troubleshooting MPLS LSP Reply Modes Reply Mode Reply via an IPv4 UDP Packet with Router Alert In this mode when the destination router replies it appends a label of 1 to the packet This forces all the intermediate routers, on the way back, to process switch the reply This mode is CPU intensive and should generally be used if the reply fails for reply with IPv4 UDP packet This mode is useful when we have inconsistency between IP and MPLS BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 47

48 Troubleshooting MPLS LSP Return Codes Value Meaning 0 The Error Code Is Contained in the Error Code TLV 1 Malformed Echo Request Received 2 One Or More of the TLVs Was Not Understood 3 Replying Router Is an Egress for the FEC 4 Replying Router Has No Mapping for the FEC 5 Replying Router Is Not One of the Downstream Routers 6 Replying Router Is one of the Downstream Routers, and Its Mapping for this FEC on the Received Interface Is the Given Label BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 48

49 Troubleshooting MPLS LSP MPLS Echo Request R1#ping mpls ipv /32 verbose destination repeat 1 exp 7 pad 0xFFFF Sending 1, 100-byte MPLS Echos to /32, timeout is 2 seconds, send interval is 0 msec: Codes: '!' - success, 'Q' - request not transmitted, '.' - timeout, 'U' - unreachable, 'R' - downstream router but not target Type escape sequence to abort.! Reply address , return code 3 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 49

50 Troubleshooting MPLS LSP MPLS Ping (Operational Theory) We use the same label stack as used by the LSP and this makes the echo to be switched inband of LSP The IP header destination address field of the echo request is a 127/8 address An Echo reply, which may or may not be labelled, has the egress interface IP address as the source; destination IP address/port are copied from the echorequest s source address/port Presence of the 127/8 address in the IP header destination address field causes the packet to be consumed by any routers trying to forward the packet using the ip header In this case P1 would not forward the echo-req to PE1 but rather consumes the packet and sends a reply to PE2 accordingly BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 50

51 Troubleshooting MPLS LSP MPLS Ping Packet Capture BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 51

52 Operation MPLS OAM Caveats For LSP ping we generate an MPLS echo request The payload includes the LDP/RSVP/L2 Circuit sub-tlv depending on the LSP we use Echo request is appropriately labelled and sent out Ping mode: MPLS TTL = 255 Traceroute mode: TTL = 1, 2,3 etc. MPLS Echo Request always has FEC Stack TLV The LSP ping sender sets the return code to 0. The replying router would set it accordingly based on the table shown previously BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 52

53 Troubleshooting MPLS LSP TTL Field in Labels Only the TTL field in the label at the top of the stack counts The outgoing TTL value is only a function of the incoming TTL value Outgoing TTL is one less than incoming TTL If outgoing TTL = 0, packet is not forwarded (not even stripped and forwarded as an IP packet) When an IP packet is first labelled, the TTL field is copied from the IP header to the MPLS header (after being decremented by 1) When the label stack is removed, the outgoing TTL value is copied to the TTL field in the IP header Unless MPLS TTL > IP TTL BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 53

54 Troubleshooting MPLS LSP Operation Receiving LSR checks that label stack of received packet matches with the received FECs in FEC Stack MPLS Echo Reply is sent in response to MPLS Echo Request Destination IP address is source IP address of Echo Request IP TTL = 255 Reply Mode: (You do not control if return packet is sent over IP or MPLS) IPv4 IPv4 with Router Alert (IP Option) If over MPLS, then Router Alert Label as topmost label is added in the label stack Hardware forwarding bypassed; packet is sent to RP process level forwarding BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 54

55 Traceroute in MPLS Network In Label Prefix Output Interface Out Label In Label Prefix Output Interfac e Out Label /32 Y /32 X /32 Y /32 X Y Y PE1 X P1 X PE / /32 CE1 In Label Prefix Output Interface Out Label /32 X pop / / /32 Y pop CE2 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 55

56 Troubleshooting MPLS LSP Traceroute in MPLS Network TTL=2 UDP port Label 19, TTL=1 Label TTL=1 UDP port Label 24008, TTL= TTL=255, ICMP TTL Exceeded Aggregate Outgoing Label, IP Lookup done in CEF for VRF CE /32 PE1 P1 PE2 CE / / /32 Label 22, TTL= TTL=252 ICMP TTL Exceeded Label 16, TTL= TTL=254 ICMP TTL Exceeded Label TTL=254 ICMP TTL Exceeded BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 56

57 Troubleshooting MPLS LSP MPLS Trace The ICMP messages TTL exceeded are forwarded along the LSP until the end of the LSP. So, the router does not lookup the source ip address in the global routing table to return the ICMP message. Reason : P routers do not have knowledge of VPN prefixes : all traceroutes initiated from within a VPN would fail ICMP messages are forwarded with EXP bits = 6 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 57

58 Troubleshooting MPLS LSP MPLS Trace Hiding This command prohibits the copying of the TTL from the IP header to the MPLS shim header and vice versa (TTL is set to 255) It should be configured on the routers that do the label imposement (LSR edge routers), which is the PE routers. Providers like to use it so that the customers see the MPLS network as one hop when tracerouting no mpls ip propagate-ttl forwarded BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 58

59 Troubleshooting MPLS LSP MPLS Trace Hiding CE1#traceroute source (mpls ip propagate-ttl forwarded) Type escape sequence to abort. Tracing the route to [AS 100] 3 msec 3 msec 3 msec local PE [MPLS: Labels 19/24008 Exp 0] 122 msec 25 msec 19 msec [MPLS: Label Exp 0] 21 msec 16 msec 23 msec [AS 100] 23 msec * 22 msec remote CE P remote PE CE1#traceroute source (no mpls ip propagate-ttl forwarded) Type escape sequence to abort. Tracing the route to VRF info: (vrf in name/id, vrf out name/id) [AS 100] 4 msec 3 msec 3 msec local PE [MPLS: Label Exp 0] 25 msec 25 msec 31 msec [AS 100] 24 msec * 28 msec remote CE remote PE BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 59

60 Troubleshooting MPLS LSP MPLS Trace with no mpls ip propagate-ttl on PE routers TTL=2 UDP port Label 19, TTL=1 Label TTL=1 UDP port Label 24008, TTL= TTL=1 UDP port Aggregate Outgoing Label TTL=1 UDP port udp port 35678? CE / TTL=254, ICMP Port Unreachable PE1 P1 PE2 Label 16, TTL= TTL=254, ICMP Port Unreachable Label 22, TTL=255 Label TTL=254, ICMP Port Unreachable TTL=255, ICMP Port Unreachable CE /32 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 60

61 Troubleshooting MPLS LSP MPLS Forwarding Plane With MPLS, the idea is to de-couple the forwarding from the IP header The forwarding decision is based on the MPLS header, not the IP header The above is true once the packet is inside the MPLS network Forwarding is still based on the IP header at the edge where the packet first enters the MPLS network CEF must be configured on all the routers in a MPLS network. CEF takes care of the crucial recursion and resolution operations BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 61

62 Troubleshooting MPLS LSP What happens when CEF disabled? PE1#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 No Label /32 0 drop 17 No Label /32 0 drop 20 No Label /32 0 drop 21 No Label /24 0 drop 22 No Label /24 0 drop 23 No Label /32 0 drop 24 No Label /24 0 drop 25 No Label /24 0 drop BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 62

63 Troubleshooting MPLS LSP MPLS Forwarding Plane Outgoing Labels PE1#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing NextHop Label Label or Tunnel Id Switched interface /32 0 Gi0/ PE1# Outgoing label also conveys what treatment the packet is going to get. It could also be: I. Pop - Pops the topmost label II. Untagged - Untag the incoming MPLS packet III. Aggregate - Untag and then do a FIB lookup Label values 0-15 are reserved. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 63

64 Troubleshooting MPLS LSP MPLS Forwarding Plane: Outgoing Labels PE1#sh mpls forwarding-table Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface /32 0 Et0/ /32 0 Et1/ Pop tag /32 0 Et1/ Pop tag /32 0 Et0/ Pop tag /30 0 Et1/ Pop tag /30 0 Et0/ Untagged /32[V] 0 Se2/0 point2point 21 Pop tag /30 0 Et1/ Pop tag /30 0 Et0/ Aggregate /30[V] 0 26 Untagged /32[V] 0 Se2/0 point2point PE1# BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 64

65 Troubleshooting MPLS LSP MPLS Forwarding Plane: Outgoing Labels Untagged Convert the incoming MPLS packet to an IP packet and forward it. Pop Pop the top label from the label stack present in an incoming MPLS packet and forward it as an MPLS packet. If there was only one label in the stack, then forward it as an IP packet. SAME as imp-null label. Aggregate Convert the incoming MPLS packet to an IP packet and then do a FIB lookup for it to find out the outgoing interface. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 65

66 Troubleshooting MPLS LSP MPLS Forwarding Plane - Lookup Three cases in the MPLS forwarding: 1) Label Imposition - IP to MPLS conversion 2) Label swapping - MPLS to MPLS 3) Label disposition - MPLS to IP conversion So, depending upon the case, we need to check: 1) FIB - For IP packets that get forwarded as MPLS 2) LFIB - For MPLS packets that get forwarded as MPLS 3) LFIB - For MPLS packets that get forwarded as IP BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 66

67 Troubleshooting MPLS LSP MPLS Forwarding Plane: Loadsharing MPLS Loadsharing (due to multiple paths to a prefix) is no different from that of IP Hashing-algorithm is still the typical FIB based i.e per-dest loadsharing by default ** So the show commands are still relevant Show ip cef exact-route <source> <dest> etc. But the <dest> must be known in the FIB table, otherwise the command won t work. Won t work on P routers for the VPN prefixes. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 67

68 Troubleshooting MPLS LSP MPLS Forwarding Plane: MTU Setting mpls mtu <bytes> can be applied to an interface to change the MPLS MTU size on the interface MPLS MTU size is checked by the router while converting an IP packet into a labeled packet or transmitting a labelled packet Label imposition(s) increases the packet size by 4 bytes/label, hence the outgoing packet size may exceed interface MTU size, hence the need to tune MTU mpls mtu <bytes> command has no effect on interface or IP MTU size. By default, MPLS MTU = interface MTU MPLS MTU setting doesn t affect MTU handling for IP-to-IP packet switching BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 68

69 Troubleshooting MPLS LSP MPLS Forwarding Plane: MTU Setting If the label imposition makes the packet bigger than the MPLS MTU size of an outgoing interface, then: - If the DF bit set, then discard the packet and send ICMP reply back (with code=4) - If the DF bit is not set, then fragment the IP packet (say, into 2 packets), and then impose the same label(s) on both the packets, and then transmit MPLS packets BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 69

70 Troubleshooting MPLS LSP MPLS Forwarding Plane: Show Commands show mpls forwarding Shows all LFIB entries (vpn, non-vpn, TE etc.) show mpls forwarding <prefix> LFIB lookup based on a prefix show mpls forwaring label <label> LFIB lookup based on an incoming label show mpls forwarding <prefix> detail Shows detailed info such as L2 encap etc BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 70

71 Troubleshooting MPLS LSP MPLS Forwarding Plane: Show Commands R2#show mpls forwarding detail Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface /32 0 Fa1/1/ MAC/Encaps=14/18, MRU=1500, Tag Stack{51} 0003FD1C E No output feature configured Per-packet load-sharing R2# 14/18 means that the L2 header is of 14 bytes, but L2+label header is 18 bytes (one label is 4 bytes) BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 71

72 Troubleshooting MPLS L3 VPNs

73 Troubleshooting MPLS L3 VPNs Nodes and their Roles PE Provider Edge router, connects to P and CE routers Maintains separate routing table per VRF (RD) Uses MP-BGP to exchange VRF routing information (RD + RT) Performs LFIB and FIB lookups, label imposition and disposition Exchanges IGP and LDP labels with the core P Provider core router, connects to P and PE routers Does not need to run BGP with the PE s Performs LFIB MPLS forwarding, label swap or PHP Exchanges IGP and LDP labels with other P routers and the PE s CE Customer edge router, connects to the CE network and the PE Forwards only IP packets no awareness of the MPLS network is needed Routes between the CE internal network and the PE router BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 73

74 Troubleshooting MPLS L3 VPNs L3VPN by Parts The Core: BGP between PEs LDP IGP (mainly to get between PEs) The Edge: Any routing protocol between the PE and CE PE-CE Protocol MP-iBGP LDP + IGP PE-CE Protocol CE PE P PE CE BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 74

75 Troubleshooting MPLS L3 VPNs VRF Overview VRF = VPN Routing Forwarding instance Isolated routing table, kind of like a VM Easiest to think of each VRF like a different physical box Interfaces are assigned to a VRF Everything not in a VRF is in the global (routing table) In MPLS-VPN each customer has a VRF VRFs for customers, global for the Provider Customer Network vrf mpls ISP BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 75

76 Troubleshooting MPLS L3 VPNs VRF Overview Because each RIB is isolated, overlapping address are allowed VRF-aware features add vrf <name> to commands Commands without VRF keyword reference the global RIB e0 ip vrf forwarding red ip address /24 e1 ip vrf forwarding red ip address /24 e2 ip address /24 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 76

77 Troubleshooting MPLS L3 VPNs VRF Overview e0 ip vrf forwarding red ip address /24 e1 ip vrf forwarding red ip address /24 PE1#show ip route % Network not in table e2 ip address /24 PE1#show ip route vrf red Routing Table: red Routing entry for /24 Known via "connected" * directly connected, via Ethernet1 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 77

78 Troubleshooting MPLS L3 VPNs MP-BGP (Multi Protocol BGP) MP-BGP extends BGP to carry more than just IPv4 prefixes Introduced address family style configuration Allows for IPv6, MPLS and other information in same BGP session When session is established the capabilities are negotiated No new rules, still requires full mesh or RRs RRs need to support additional capabilities For MPLS only PEs need to speak BGP or know CE routes L3VPN Relies on Extended Communities Extended Communities are arbitrary TLVs attached to BGP prefixes BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 78

79 Troubleshooting MPLS L3 VPNs MP-BGP: Address-Families Address-family vpnv4, ipv4 unicast vrf introduced vpnv4 AFI for PE to PE (label information) ipv4 unicast vrf for PE to CE Neighbor must be activated for each AFI supported router bgp 100 neighbor remote-as 100! address-family vpnv4 neighbor activate neighbor send-community extended! address-family ipv4 unicast vrf red neighbor remote-as 400 neighbor activate Remote PE Local CE BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 79

80 Troubleshooting MPLS L3 VPNs MP-BGP: Advertising CE Routes BGP maintains a table for each AFI (vpnv4, ipv4, vrf ) CE routes are placed into the vpnv4 BGP table BGP routes in a vrf AFI are automatically turned into vpnv4 routes If BGP is not PE-CE protocol routes must be redistributed into ipv4 vrf AFI All vpnv4 routes get an assigned label vpnv4 routes are exchanged between vpnv4 peers (PEs) BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 80

81 Troubleshooting MPLS L3 VPNs RTs and RDs: Creating the VRF VRFs have 3 parts: 1. VRF name (case sensitive) 2. Route Distinguisher (RD) 3. Route Target(s) (RT) RD and RT are for MPLS; RD must always be defined RD must be unique to the VRFs on the local PE ip vrf red rd 100:100 route-target import 200:200 route-target export 201:201 If there is no MPLS, called VRF-lite BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 81

82 Troubleshooting MPLS L3 VPNs Understanding RDs Route Distinguisher Every CE route from all VRFs are placed in a single VPNv4 table How are routes from one VRF distinguished from another VRF? By prepending the RD to the route to create a VPNv4 route Only used to make routes unique VPNv4 prefixes IPv4 Route: /24 RD: 100:100 VPNv4 Route: 100:100: /24 ip vrf red rd 1:1 route-target import 200:200 route-target export 201:201 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 82

83 Troubleshooting MPLS L3 VPNs Understanding the RT Route Target RT is a BGP extended community (extra information on the update) route-target export adds the community to the outbound update route-target import defines which routes to bring into the VRF Multiple imports and exports allowed ip vrf red rd 1:1 route-target import 100:100 route-target import 200:200 route-target export 201:201 route-target export 44:313 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 83

84 Troubleshooting MPLS L3 VPNs RT in Action ip vrf red rd 1:1 route-target import 100:100 route-target export 201:201 66:66: /24 RT: 100:100 55:55: /24 RT: 201:201 44:44: /24 RT: 100:100 BGP Update VRF Red RIB / /24 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 84

85 Troubleshooting MPLS L3 VPNs MP-BGP: Advertising CE Routes ip vrf test rd 1:1 route-target export 123:456 Route Target Locally Assigned Label RD Prefix BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 85

86 Troubleshooting MPLS L3 VPNs Example Topology MP-IBGP VPNv /24 LDP + IGP / / /24 CE1 IOS PE P1 XR PE CE2 Lo0= /32 Lo0= /32 Lo0= /32 Lo0= /32 Lo0= /32 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 86

87 Troubleshooting MPLS L3 VPNs Verify VPNv4 Neighborship IOS-PE#sh bgp vpnv4 unicast all summary BGP router identifier , local AS number 100 Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd :04: :24:28 1 RP/0/0/CPU0:XR-PE#sh bgp vpnv4 unicast summary BGP router identifier , local AS number 100 Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd :35:10 2 RP/0/0/CPU0:XR-PE#show bgp vrf ABC summary BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 87

88 Troubleshooting MPLS L3 VPNs Verify PE to PE LSP IOS-PE#ping mpls ipv Sending 5, 100-byte MPLS Echos to /32, timeout is 2 seconds, send interval is 0 msec: Type escape sequence to abort.... Success rate is 0 percent (0/5) RP/0/0/CPU0:XR-PE(config)#mpls oam RP/0/0/CPU0:XR-PE(config-oam)#commit IOS-PE#ping mpls ipv Sending 5, 100-byte MPLS Echos to /32, timeout is 2 seconds, send interval is 0 msec: Type escape sequence to abort.!!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 4/5/7 ms BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 88

89 Troubleshooting MPLS L3 VPNs Verify VPN Labels and Prefix IOS-PE#show bgp vpnv4 unicast vrf ABC BGP routing table entry for 1:1: /32, version 23 Paths: (1 available, best #1, table ABC) Advertised to update-groups: 5 Refresh Epoch (via vrf ABC) from ( ) Origin IGP, metric 0, localpref 100, valid, external, best Extended Community: RT:1:1 Local VPN Label mpls labels in/out 24/nolabel rx pathid: 0, tx pathid: 0x0 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 89

90 Troubleshooting MPLS L3 VPNs Verifying VPN Label on Remote PE RP/0/0/CPU0:XR-PE#show bgp vpnv4 unicast vrf ABC BGP routing table entry for /32, Route Distinguisher: 2:2 Last Modified: May 30 16:57: for 00:18: (metric 3) from ( ) Received Label 24 Remote VPN Label Origin IGP, metric 0, localpref 100, valid, internal, best, group-best, import-candidate, imported Received Path ID 0, Local Path ID 1, version 36 Extended community: RT:1:1 Originator: , Cluster list: Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1:1 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 90

91 Troubleshooting MPLS L3 VPNs Verifying Labels (The Easy Way) IOS-PE#show bgp vpnv4 unicast vrf ABC labels Network Next Hop In label/out label Route Distinguisher: 1:1 (ABC) / /nolabel / nolabel/ / /nolabel(ABC) / nolabel/24005 In Label represents local label and Out Label represents remote label BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 91

92 Troubleshooting MPLS L3 VPNs Verifying CEF (FIB, and LFIB) - IOS IOS-PE#show ip cef vrf ABC detail /32, epoch 0, flags [rib defined all labels] recursive via label 24006() nexthop GigabitEthernet0/2 label 17() IOS-PE#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface /32 0 Gi0/ IOS-PE#show ip cef detail /32, epoch 0 dflt local label info: global/19 [0x0] 1 RR source [no flags] nexthop GigabitEthernet0/2 label 17() BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 92

93 Troubleshooting MPLS L3 VPNs Verifying CEF (FIB, and LFIB) - IOS P1#show mpls forwarding-table labels 17 Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 17 Pop Label / Gi0/ Implicit- Null P1#show ip cef detail /32, epoch 0 dflt local label info: global/17 [0x0] nexthop GigabitEthernet0/2 P1#show ip cef detail /32, epoch 0 dflt local label info: global/16 [0x0] nexthop GigabitEthernet0/1 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 93

94 Troubleshooting MPLS L3 VPNs Verifying CEF (FIB, and LFIB) IOS XR RP/0/0/CPU0:XR-PE#show cef vrf ABC detail... via /32, 3 dependencies, recursive [flags 0x6000] path-idx 0 NHID 0x0 [0xa14fd474 0x0] recursion-via-/32 next hop VRF - 'default', table - 0xe next hop /32 via 24000/0/21 next hop /32 Gi0/0/0/0 labels imposed {16 24} Load distribution: 0 (refcount 1) Hash OK Interface Address 0 Y Unknown 24000/0 Local Label for PE1 Lo0 IGP Label from P1 VPN Label from PE1 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 94

95 Troubleshooting MPLS L3 VPNs Verifying CEF (FIB, and LFIB) IOS XR RP/0/0/CPU0:XR-PE#show mpls forwarding labels Mon May 30 18:39: UTC Local Outgoing Prefix Outgoing Next Hop Bytes Label Label or ID Interface Switched /32 Gi0/0/0/ RP/0/0/CPU0:XR-PE#show cef /32... via /32, GigabitEthernet0/0/0/0, 5 dependencies, weight 0, class 0 [flags 0x0] path-idx 0 NHID 0x0 [0xa0ed91a8 0x0] next hop /32 local adjacency local label labels imposed {16} BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 95

96 Troubleshooting MPLS L3 VPNs Verifying Hardware Programming IOS XR RP/0/0/CPU0:XR-PE#show cef vrf ABC hardware egress location 0/0/CPU /32, version 18, internal 0x x0 (ptr 0xa13f20f4) [1], 0x0 (0x0), 0x208 (0xa ) Updated May 30 16:57: Prefix Len 32, traffic index 0, precedence n/a, priority 3 via /32, 3 dependencies, recursive [flags 0x6000] path-idx 0 NHID 0x0 [0xa14fd474 0x0] recursion-via-/32 next hop VRF - 'default', table - 0xe next hop /32 via 24000/0/21 next hop /32 Gi0/0/0/0 labels imposed {16 24}... BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 96

97 Troubleshooting MPLS L3 VPNs Case Study MPLS Traffic Not Forwarded Customer reported traffic forwarding issue to the VRF s attached to a newly configured PE2 router The PE1 router has the VPN label which is being shared with the remote PE2 router MP-IBGP VPNv4 LDP + IGP / / / /24 CE1 PE1 P1 PE2 Lo0= /32 Lo0= /32 Lo0= /32 Lo0= /32 On PE1, the CEF shows the correct forwarding output. CE2 Lo0= /32 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 97

98 Troubleshooting MPLS L3 VPNs Troubleshooting Approach The first step in MPLS deployment is to verify if the LSP is complete or not. Use ping mpls ipv4 <dest-pe-loopback> <subnet_mask> to verify LSP Path Use traceroute mpls ipv4 <dest-pe-loopback> <subnet_mask> to verify what is the path and see the point where MPLS packet is getting dropped The other option is to check the labeling and LFIB information hop by hop or at least on the node where the MPLS trace is dropped. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 98

99 Troubleshooting MPLS L3 VPNs Findings The MPLS PING failed MPLS Trace dropped on P-1 router Show mpls forwarding <PE2-loopback> output shows no label as outgoing label P-1# show mpls forwarding Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 17 No Label / Et0/ Verified that LDP was enabled between the two routers but there was no bindings BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 99

100 Troubleshooting MPLS L3 VPNs Resolution P-1(config)#no mpls ldp advertise-labels P-1(config)#mpls ldp advertise-labels for LOOPBACK_ACL The P-1 router had an ACL to limit the allocation of labels for certain prefixes Sometimes, there are too many prefixes in the core due to which the labels get exhausted To prevent such situations, LDP is configured to allocate labels for certain prefixes but not all. PE2 loopback address was added in the ACL which fixed the problem BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 100

101 6VPE Troubleshooting

102 Troubleshooting 6VPE Reference Topology IPv /32 IPv6 2001:DB8::1/128 PE1 AS 100 Service Provider Core IPv4 IGP MPLS IPv6 2001:DB8::6/128 IPv /32 IPv6 2001:DB8::2/128 IPv /32 IPv6 2001:DB8::5/128 IPv6 2001:DB8::7/128 CE1 RR-P PE5 CE2 AS 200 PE2 IPv /32 AS 300 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 102

103 Troubleshooting 6VPE VRF Configuration IPv6 enabled VRF s are configured in the same way as IPv4 VRF s On Cisco IOS, use command vrf definition to configure both IPv4 and IPv6 capable VRF s vrf definition ABC rd 1:1 address-family ipv6 unicast route-target import 1:1 route-target export 1:1 route-target import 2:2 address-family ipv4 unicast... interface Gi0/0 vrf forwarding ABC ipv6 address xx:xx:xx::y/64 vrf ABC address-family ipv6 unicast import route-target 1:1 2:2 export route-target 1:1 address-family ipv4 unicast... interface Gi0/0/0/0 vrf ABC ipv6 address xx:xx:xx::y/64 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 103

104 6VPE Configuration Cisco IOS router bgp 100 bgp router-id bgp log-neighbor-changes no bgp default ipv4-unicast neighbor remote-as 100 neighbor update-source Loopback0! address-family vpnv6 neighbor activate neighbor send-community extended neighbor next-hop-self! address-family ipv6 vrf red neighbor 2001:DB8:0:16::6 remote-as 200 neighbor 2001:DB8:0:16::6 activate exit-address-family BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 104

105 6VPE Configuration IOS XR router bgp 100 bgp router-id address-family vpnv6 unicast! neighbor remote-as 100 update-source Loopback0 address-family vpnv6 unicast next-hop-self! vrf red rd 100:1 address-family ipv6 unicast! neighbor 2001:db8:0:26::6 remote-as 200 address-family ipv6 unicast route-policy pass in route-policy pass out BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 105

106 Troubleshooting 6VPE Verifying Control Plane Since both control plane and data plane works in opposite direction, verify the IPv6 VPN prefix on PE5. PE5#show ipv6 route vrf red! Output omitted for brevity B 2001:DB8::6/128 [200/0] via %default, indirectly connected B 2001:DB8::7/128 [20/0] via FE80::7, GigabitEthernet0/2 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 106

107 Troubleshooting 6VPE Verifying Control Plane Verify the VPNv6 prefix in BGP along with the local label PE5#show bgp vpnv6 unicast vrf red 2001:db8::7/128 BGP routing table entry for [100:5]2001:DB8::7/128, version 38 Paths: (1 available, best #1, table red) Advertised to update-groups: 2 Refresh Epoch :DB8:0:57::7 (FE80::7) (via vrf red) from 2001:DB8:0:57::7 ( ) Origin IGP, metric 0, localpref 100, valid, external, best Extended Community: RT:100:1 mpls labels in/out 23/nolabel rx pathid: 0, tx pathid: 0x0 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 107

108 Troubleshooting 6VPE Verifying Control Plane The remote IOS PE - PE1, receives the VPNv6 prefix as the out label of 23. PE1#show bgp vpnv6 unicast vrf red 2001:db8::7/128 BGP routing table entry for [100:1]2001:DB8::7/128, version 7 Paths: (1 available, best #1, table red) Advertised to update-groups: 1 Refresh Epoch 1 300, imported path from [100:5]2001:DB8::7/128 (global) ::FFFF: (metric 3) (via default) from ( ) Origin IGP, metric 0, localpref 100, valid, internal, best Extended Community: RT:100:1 Originator: , Cluster list: mpls labels in/out nolabel/23 rx pathid: 0, tx pathid: 0x0 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 108

109 Troubleshooting 6VPE Verifying Control Plane RP/0/0/CPU0:PE2#show bgp vpnv6 unicast vrf red 2001:db8::7/128 BGP routing table entry for 2001:db8::7/128, Route Distinguisher: 100:1 Last Modified: Feb 4 22:46: for 1d05h Paths: (1 available, best #1) Not advertised to any peer Path #1: Received by speaker 0 Not advertised to any peer (metric 3) from ( ) Received Label 23 Origin IGP, metric 0, localpref 100, valid, internal, best, group-best, import-candidate, imported Received Path ID 0, Local Path ID 1, version 5 Extended community: RT:100:1 Originator: , Cluster list: Source VRF: default, Source Route Distinguisher: 100:5 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 109

110 Troubleshooting 6VPE Verifying Data Plane PE1#show ipv6 cef vrf red 2001:db8::7/128 detail 2001:DB8::7/128, epoch 0, flags [rib defined all labels] recursive via label 23 nexthop GigabitEthernet0/2 label 19 PE1#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface /32 0 Gi0/ BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 110

111 Troubleshooting 6VPE Verifying Data Plane on IOS XR RP/0/0/CPU0:PE2#show cef vrf red ipv6 2001:db8::7/ :db8::7/128, version 7, internal 0x x0 (ptr 0xa140c5f4) [1], 0x0 (0x0), 0x208 (0xa14db230) Updated Feb 4 22:46: Prefix Len 128, traffic index 0, precedence n/a, priority 3 via ::ffff: , 3 dependencies, recursive [flags 0x6000] path-idx 0 NHID 0x0 [0xa176b0bc 0x0] recursion-via-/128 next hop VRF - 'default', table - 0xe next hop ::ffff: via ::ffff: :0 next hop /32 Gi0/0/0/1 labels imposed {19 23} RP/0/0/CPU0:PE2#show mpls forwarding-table prefix /32 Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface /32 0 Gi0/0/0/ BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 111

112 Verifying Ingress Hardware Programming IOS XR PE2#show cef vrf red ipv6 2001:db8::7/128 hardware ingress detail loc0/0/cpu0 2001:db8::7/128, version 7, internal 0x x0 (ptr 0xa140c5f4) [1], 0x0 (0x0), 0x208 (0xa14db230) Updated Feb 4 22:46: [1 type 1 flags 0x48089 (0xa14f5398) ext 0x0 (0x0)] LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0] gateway array update type-time 1 Feb 4 22:46: LDI Update time Feb 4 22:46: via ::ffff: , 3 dependencies, recursive [flags 0x6000] path-idx 0 NHID 0x0 [0xa176b0bc 0x0] recursion-via-/128 next hop VRF - 'default', table - 0xe next hop ::ffff: via ::ffff: :0 next hop /32 Gi0/0/0/1 labels imposed {19 23} Ingress platform showdata is not available. Load distribution: 0 (refcount 1) Hash OK Interface Address 0 Y Unknown ::ffff: :0 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 112

113 Verifying Egress Hardware Programming IOS XR PE2#show cef vrf red ipv6 2001:db8::7/128 hard egr det loc 0/0/CPU0 2001:db8::7/128, version 7, internal 0x x0 (ptr 0xa140c5f4) [1], 0x0 (0x0), 0x208 (0xa14db230) [1 type 1 flags 0x48089 (0xa14f5398) ext 0x0 (0x0)] LW-LDI[type=0, refc=0, ptr=0x0, sh-ldi=0x0] gateway array update type-time 1 Feb 4 22:46: LDI Update time Feb 4 22:46: via ::ffff: , 3 dependencies, recursive [flags 0x6000] path-idx 0 NHID 0x0 [0xa176b0bc 0x0] recursion-via-/128 next hop VRF - 'default', table - 0xe next hop ::ffff: via ::ffff: :0 next hop /32 Gi0/0/0/1 labels imposed {19 23} Egress platform showdata is not available. Load distribution: 0 (refcount 1) Hash OK Interface Address 0 Y Unknown ::ffff: :0 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 113

114 Troubleshooting 6VPE / MPLS Verifying Counters on Interface Verify the interface counters for mpls forwarding If there is forwarding problem, check the counters and ensure they are not increasing. Initiate the VPNv6 prefix ping and verify the counters again to see if they increased RP/0/0/CPU0:PE2#show interface gigabitethernet0/0/0/1 accounting GigabitEthernet0/0/0/1 Protocol Pkts In Chars In Pkts Out Chars Out IPV4_UNICAST IPV6_UNICAST MPLS ARP IPV6_ND BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 114

115 Inter-AS MPLS VPNs

116 Inter-AS MPLS VPNs Flavors Previous section VPNs within Single-AS boundary Inter-AS MPLS VPN VPNs spanning across multiple AS boundaries Types: Option 1 Back to Back VRF Option 2 Inter-Provider VPNs using ASBR-to-ASBR approach A. Next-Hop-Self Method B. Redistribute Connected Method C. Multi-hop EBGP between ASBRs Option 3 MP-EBGP between RR and EBGP between ASBR BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 116

117 Inter-AS MPLS VPNs Option 1 - Back-to-Back VRF Method RR-P1 VRF- ABC VRF- XYZ RR-P2 PE1 AS100 PE-ASBR1 IPv4 + IGP/BGP PE-ASBR2 AS200 PE2 Lo /32 Lo /32 CE1 CE2 BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 117

118 Inter-AS MPLS VPNs Option 2a ASBR-to-ASBR with Next-Hop-Self Method v1 RR-P1 RR-P2 MP-eBGP PE1 AS100 PE-ASBR1 PE-ASBR2 AS200 PE2 Lo /32 Lo /32 neighbor x.x.x.x next-hop-self CE1 CE2 No LDP or IGP required on the link between the two ASBRs Configure no bgp default route-target filter on ASBRs BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 118

119 Inter-AS MPLS VPNs Option 2a ASBR-to-ASBR with Next-Hop-Self Method Both ASBRs allocate VPN labels for prefixes received from the other AS. When MP-eBGP peering is configured between ASBRs, below configuration is done to complete LSP mpls bgp forwarding on Cisco IOS devices no bgp default route-target filter configured on ASBR not having VRF configured. Default behavior deny vpnv4 prefixes that are not imported in any local VRF On XR retain route-target all BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 119

120 Inter-AS MPLS VPNs Option 2b ASBR-to-ASBR with Redistribute Connected Method v1 RR-P1 RR-P2 MP-eBGP PE1 AS100 PE-ASBR1 PE-ASBR2 AS200 PE2 Lo /32 Lo /32 CE1 No LDP or IGP required on the link between the two ASBRs. CE2 Configure no bgp default route-target filter on ASBRs BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 120

121 Inter-AS MPLS VPNs Option 2b ASBR-to-ASBR with Redistribute Connected Method Redistribute the link between ASBR into IGP in local AS Required on both ASBR routers. Both ASBRs allocate VPN labels for prefixes received from the other AS. VPN label V1 is advertised from AS100 towards ASBR-PE2 in AS200. Since the NH changes on ASBR-PE2, ASBR-PE2 swaps that label with V2 and advertises it towards the core. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 121

122 Inter-AS MPLS VPNs Option 2c ASBR-to-ASBR with Multi-Hop EBGP between ASBRs Method v1 RR-P1 RR-P2 PE1 AS100 PE-ASBR1 MP-eBGP PE-ASBR2 AS200 PE2 Lo /32 Lo /32 CE1 Loopback to loopback peering between ASBRs CE2 Configure no bgp default route-target filter on ASBRs BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 122

123 Inter-AS MPLS VPNs Option 2c ASBR-to-ASBR with Multi-Hop EBGP between ASBRs Method Loopback to loopback MP-EBGP peering between ASBRs. IGP or static route required between the ASBR link Both ASBRs allocate VPN labels for prefixes received from the other AS. VPN label V1 is advertised from AS100 towards ASBR-PE2 in AS200. Since the NH changes on ASBR-PE2, ASBR-PE2 swaps that label with V2 and advertises it towards the core. BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 123

124 Inter-AS MPLS VPNs Option 3 Multi-Hop MP-EBGP between RR and EBGP between ASBRs MP-eBGP RR-P1 RR-P2 ebgp + PE1 AS100 PE-ASBR1 Send-label PE-ASBR2 AS200 PE2 Lo /32 Lo /32 CE1 Neighbor send-label required on ebgp peers on ASBR. CE BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 124

125 Inter-AS MPLS VPNs Option 3 Multi-Hop MP-EBGP between RR and EBGP between ASBRs RR & ASBR loopbacks are advertised via EBGP on ASBR The remote ASBR redistributes the received loopbacks into local IGP MP-EBGP peering configured between RR s on each AS Configure neighbor next-hop-unchanged BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 125

126 Complete Your Online Session Evaluation Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card. Complete your session surveys through the Cisco Live mobile app or from the Session Catalog on CiscoLive.com/us. Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online BRKMPL Cisco and/or its affiliates. All rights reserved. Cisco Public 126