Troubleshooting EIGRP Networks

Transcription

1

2 Troubleshooting EIGRP Networks Don Slice

3 Troubleshooting EIGRP Networks Avoiding Common Problems Troubleshooting Tips Problems with Links > 1G IPv6 Unique Issues Troubleshooting Tools 3

4 Avoiding Common Problems Summary Problems Route Propagation Problems Redistribution Problems 4

5 Summary Problems Summary Metrics Summary Admin Distance Summary Black Holes 5

6 Summary Basics Summarization is an information hiding technique to send lessspecific routes to represent block of prefixes /24, /24, and /24 can be aggregated to /22 Rather than advertising three networks with each representing 255 addresses (253 hosts), RtrA advertises a single network, representing 1024 addresses A / / /22 1 Network 1024 Addresses 3 Networks 255 Addresses Each / Hosts 6

7 Summary Basics Summarization is an information-hiding technique used to minimize the number of prefixes advertised while still maintaining full reachability summarization will be most effective if the network is designed in a hierarchical way so that multiple prefixes can be represented at some point in the network by a single, less specific prefix; one typical place of summarization is from distribution routers toward spokes that only need to know a default route (or at least some subset of total routes) in order to reach the remainder of the network When summarization is used in EIGRP networks, scalability is greatly enhanced both because of the fewer number of prefixes known throughout the network as well as the decreased query scope that summarization brings; the query scope aspect will be explained in more detail later in this presentation 7

8 /24 Cost /24 Cost /24 Cost /24 Cost 20 Summary Metrics In EIGRP, the metric of a summary is based on the metrics of its components EIGRP chooses the metric of the lowest cost component route as the metric of the summary /23 Cost 10 B A C /23 Cost 10 8

9 Summary Metrics When EIGRP creates a summary route, it has to determine the metric to include with the route advertisement EIGRP examines every entry in the database (topology table) looking for components of the summary that will be suppressed (thus represented by) the summary; EIGRP finds the component with the best composite metric and then copies the metric details from it (bandwidth, delay, etc.) into the summary topology table entry Note that it does not take the best delay, best bandwidth, etc., but takes the best composite metric and grabs the attributes from it. This works fine except for the fact that components of the summary may come and go, which means EIGRP has to continually make sure the summary is still using the lowest metric contained in a summary component 9

10 /24 Cost /24 Cost /24 Cost /24 Cost 20 Summary Metrics If the component the metric was derived from flaps, then summary updates are required as well! The summary is used to hide reachability information, yet changes to the metric information causes the routers beyond the summary to perform work to keep up with the metric changes /23 Cost 10 Cost 20 B A C /23 Cost 10 There is also processing overhead for EIGRP to recalculate the summary metric each time a component changes 10

11 Summary Metrics This recalculation of the summary metric when components change causes two significant things to happen: Every time the component with the best metric changes, the summary needs to be readvertised to all of it s peers thus the desire to hide topology changes behind the summary is only partially functional; while it hides the changes for each component prefix, it still causes updates and processing if the best component is the one that changed Even if the best component isn t the one that changed, EIGRP internally has to look at every topology table entry to make sure the summary metric wasn t affected; with large numbers of components or large numbers of summaries, this can be significant processing 11

12 /24 Cost /24 Cost 20 Summary Metrics Solutions Use a loopback interface to force the metric to remain constant Create a loopback interface within the summary range with a lower metric than any other component Generally best to use a /32 for the prefix and use delay to force the metric value The summary will use the metric of the loopback, which will never go down A B /23 Cost 10 loopback 0 ip address delay 1 12

13 Summary Metrics One way to minimize/remove the first problem (metric changing downstream due to component changes) is to create a loopback on the router doing the summarization and ensure that it has the best metric of any component of the summary; since it will remain up unless administratively shut down, the metric of the summary will not change in its updates to upstream peers Note that this approach does nothing to change the second summary metric issue; i.e., router cpu processing required to recalculate on the router doing the summarization that s next 13

14 /24 Cost /24 Cost 20 Summary Metrics Solutions In the latest EIGRP code, you can define the summary-metric command in router mode in order to specify the metric to be used on the summary, regardless of the metrics of the component routes This is similar to defining the metric on redistribution statements in router mode This eliminates metric churn downstream as well as local processing A B /23 Cost 10 router eigrp ROCKS address-family ipv4 autonomous-system 1 network topology base summary-metric

15 Summary Metrics The recent implementations of EIGRP (release five and newer) contain the new summary-metric command under the router prompt which allows you to specify the metric to use on the summary so that learning the metric from summary components is unnecessary; since the metric is fixed, both the route churn problem for downstream peers and local database searching processing are removed This new command will greatly improve scalability in networks using summarization with large topology tables, which is where summarization is most useful! 15

16 Admin Distance Basics Route Source Default Distance Connected Interface 0 Static Route 1 EIGRP Summary Route 5 ebgp 20 Internal EIGRP 90 IGRP 100 OSPF 110 IS-IS 115 RIP 120 On Demand Routing (ODR) 160 External EIGRP 170 ibgp 200 Unknown 255 The administrative distance has nothing to do with distance but instead should be considered preference or believability If a route is being installed in the RIB from multiple sources, the admin distance defines which one wins The chart supplied here shows the default distances; important to this discussion is the fact that EIGRP summary routes are preferred (AD 5) over routes learned from EIGRP peers (AD 90 for internals, 170 for externals) Note: Lower = better 16

17 Summary Admin Distance Original summary Admin Distance problem Default summary is defined on distribution routers to spokes Internet access point provides external for default route to the Internet AD of five for the summary is better than the AD of 170 from the external EIGRP route, so the Internet default route is rejected! A C B ip summary-address A eigrp So let s just add the AD to the summary! RtrA#sh ip route Routing entry for /0, supernet Known via "eigrp 1", distance 5, metric 25600, candidate default path, type internal 17

18 Summary Admin Distance A common implementation of summaries in an EIGRP network is the generation of a default route ( /0) from the distribution layer to the access routers; this installs a /0 Null0 summary (discard) route on the distribution layer router The problem occurs when there is an actual /0 default route generated elsewhere in the network that the distribution layer needs to receive in order to do proper routing; since the /0 summary route has an AD of 5 and the /0 learned across the EIGRP network has an AD of 90 for internals or 170 for externals, the local summary wins and the received route is discarded In order to solve this problem, we added the ability to define a manual admin distance to the summary several years ago this turned out to be a good idea only in limited deployment scenarios 18

19 Summary Admin Distance Since the summaries created on RtrA and RtrB now have a worse AD than the external route received, the external route wins and is propagated to RtrC Components of the summary are still suppressed RtrC has equal cost paths to /0 But what happens if RtrA loses the path to the Internet? A C B ip summary-address A eigrp D*EX /0 [170/409600] via , 00:00:10, Serial1/0 [170/409600] via , 00:00:10, Serial0/0 19

20 Summary Admin Distance When the summary is defined with an admin distance of 200 to make it worse than the external route learned across the EIGRP network, it works just like we intended; the summary is created internally to EIGRP so that the components are still suppressed to the access routers, but the external route received across the EIGRP network wins the installation into the RIB and is advertised to the access layer routers The problem occurs if the distribution layer router loses the /0 from the EIGRP network for some reason since it s the receipt of this route that keeps the local summary from being installed in the RIB and being advertised to the access layer peers, it s disappearance changes things 20

21 Summary Admin Distance Since RtrA no longer has the external route, it creates the local summary route and advertises it to RtrC Now RtrC receives an external route from RtrB and an internal route from A The route from RtrA wins! Now RtrC s default route points to A, who doesn t have access to the Internet and maybe not even the company s intranet! A B C ip summary-address eigrp D* /0 [90/409600] via , 00:00:10, Serial0/0 21

22 Summary Admin Distance When RtrA loses the /0 it received from the EIGRP network, it installs it s local summary route into the RIB with an admin distance of 200 and happily advertises it to the access layer peers; this is the interesting part summary routes only appear as a special route type on the router that generates the summary on peers that receive the summary, it appears like any other internal route! That means that RtrC will receive an external route from RtrB with an AD of 170 and an internal route from RtrA with and AD of 90 the route from RtrA wins! The problem is that now RtrC points at RtrA for all of it s traffic, yet RtrA no longer has access to the Internet and maybe not even the internal company network drats! This problem doesn t occur on single-homed access layer routers; if there s only one path out from the access router, it doesn t really matter if it s internal or external 22

23 Summary Admin Distance How do you resolve this problem? Define the Admin Distance on the summary as 255 instead of something lower The route will only be advertised if the external exists! Note: Don t use 255 for single-homed remotes! A C B ip summary-address eigrp B D*EX /0 [170/409600] via , 00:00:10, Serial1/0 C 23

24 Summary Admin Distance For dual-homed sites, using the summary admin distance in a slightly different way can provide a decent solution; if the summary is defined with an AD of 255, it will lose to the external route and allow that route to be propagated through to the access routers as before if the external route disappears, however, the AD of 255 keeps the local summary from being installed in the RIB and thus it won t be advertised to the access routers; the only remaining route will be the external through RtrB Note: that if you put an AD of 255 on a single-homed remote and the external default route goes away, the access router will not receive the route at all! You may need a floating static on the access router to avoid this side-effect 24

25 Summary Admin Distance Another way to resolve the problem Don t use summary admin distance if sending to dual-homed remotes Instead, use distribute-list to permit to remotes with floating static on remotes A B C router eigrp 1 distribute-list 1 out Serial0/0. access-list 1 permit A B ip route ip route C 25

26 Summary Admin Distance An alternative solution is to not use summarization at all on the distribution layer and instead use a distribute-list out to permit only the /0 route to the access layer; of course, that means that if the /0 route disappears, the access layer routers are stranded and can t reach the internal, company network not good To avoid this problem, the distribute-list out on the distribution layer should be used in tandem with floating static routes on the access-layer routers 26

27 Summary Admin Distance Another situation encountered when adding the admin distance to a summary Same summary can be defined on multiple interfaces Only one topology table/routing table entry is actually created The last defined Admin distance wins! ip summary-address eigrp ip summary-address eigrp Router#show run int e0/0 interface Ethernet0/0 ip address ip summary-address eigrp Router#show run int e0/1 interface Ethernet0/1 ip address ip summary-address eigrp Router#sh ip route Routing entry for /0, supernet Known via "eigrp 1", distance 240, metric , candidate default path, type internal A C B 27

28 Summary Admin Distance One other interesting aspect that may be unexpected when using the admin distance option on the summary commands is that while a summary can be entered on many different interfaces, only one summary topology entry and one routing table entry is actually created This means that there is really only one distance associated with the summary, regardless of how many different distances you enter for the same summary on different interfaces If you enter the same summary on different interfaces, it s really only adding interfaces to a single summary queue entry; if each of these summaries has a different admin distance, the last one entered will be the one used 28

29 / / /24 Summary Black Holes This network implements manual summarization from the distribution routers toward the core These summaries represent all spoke networks and links to the spokes It normally doesn t matter whether RtrA or RtrB is used to reach an address on a spoke from X X A C D B E ip summary-address eigrp RtrX#sh ip route sec D /16 [90/307200] via , 00:02:01, Ethernet0/0 [90/307200] via , 00:02:01, Ethernet0/0 29

30 Summary Black Holes In this example network, the designer implemented manual summarization to hide the specific routes located at the remote sites by summarizing from the distribution layer toward the core; on each of the interfaces of RtrA and RtrB toward router X is the summary statement ip summary-address eigrp this blocks the specific prefixes from being advertised to X, and instead only advertises the /16 prefix there Normally, this works great; minimal info is known at the core and proper routing takes place just fine transitions in the remotes are hidden form the core, which makes the core more stable but what happens if a problem occurs? 30

31 / / /24 Summary Black Holes What happens if the RtrA to RtrC Link fails? RtrX is still receiving the summary from both RtrA and RtrB and may chose RtrA as its path for packets going to /24 A builds a discard route to null0 with an administrative distance of five when the summary is configured The traffic will be dropped at RtrA! RtrA#sh ip route Routing entry for /16 Known via "eigrp 1", distance 5, * directly connected, via Null0 X A B C D E RtrX#ping Success rate is 0 percent (0/5) RtrX# ip summary-address eigrp

32 Summary Black Holes The problem is that a summary will be installed and sent if any component of the summary exists; if a router doing summarization loses access to one or more of the components of the summary, it will still advertise reachability to the entire summary, even though packets destined to the missing component(s) cannot be delivered This isn t a problem if the summarizing router is the only path the lost network, but often it s not; in the diagram above, both RtrA and RtrB have access to the remotes and are summarizing them toward the core of the network if RtrA loses access to /24, routers downstream (like rtrx) could send packets to RtrA that it cannot deliver; if the packets went to RtrB, however, they would have been delivered successfully Note that this problem is common to all routing protocols which can do summarization. Information hiding is a good thing, but there are possible down-sides, as well. 32

33 / / /24 Summary Black Holes Possible Solutions Add a new link between RtrA and RtrB without summarization configured Add a GRE tunnel between RtrA and RtrB without summarization configured RtrA#sh ip ei to all sec P /24, 1 successors, FD is , serno 19 via (307200/281600), Ethernet0/1 via ( /307200), Ethernet1/0 A C D X New Link, No Summarization GRE Tunnel, No Summarization B E ip summary-address eigrp

34 Summary Black Holes The normal method to avoid/resolve this problem is to have another link between summarizing routers (another fast/gigethernet, PVC, etc.) and not summarize across this link; in that way, RtrA would be getting component routes from RtrB and would know how to deliver packets to through RtrB Another approach used if the cost of another link is too high is to put a GRE tunnel between RtrA and RtrB and allow all component routes to be advertised across this tunnel There s also some discussion inside of EIGRP development of ways to solve this problem dynamically. A sys-wish bug has been filed against it (CSCdw68502) but we haven t started the work yet; we re still discussing the best way to solve it 34

35 Route Propagation Problems Zero successor routes Duplicate Router-ID Resource depletion 35

36 Zero Successor Routes Zero successor routes happen when EIGRP attempts to install a route in the RIB and it is rejected This normally occurs when there is a route in the RIB with a better admin distance than EIGRP EIGRP cannot propagate zero successor routes to peers /32 A B RtrA#show ip eigrp topology P /32, 1 successors, FD is via Connected, Loopback0 RtrB#show ip eigrp topology P /32, 0 successors, FD is Inaccessible via (409600/128256), Ethernet0/0 RtrB#show ip route static /8 is variably subnetted S /32 [1/0] via C RtrC#show ip eigrp topology incl RtrC# 36

37 Zero Successor Routes A route that shows up in the topology table with 0 successors and an FD of inaccessible is unusable by EIGRP for some reason; typically that means that we received the prefix from a peer and when we tried to install it into the RIB, the RIB rejected the installation Normally, the RIB rejects a route installation when there is already a better route in the table remember our discussion about admin distance earlier in this presentation? Here s what happens when we re the loser with a worse admin distance One of the side-effects of a route being flagged as 0 successor/inaccessible is that we aren t permitted to advertise a route to peers that we didn t succeed at installing in the RIB; if we couldn t put the route in the RIB, we can t verify that the destination will be reachable, thus we can t tell our peers about it 37

38 Zero Successor Routes Another case of zero successor routes happens with overlapping EIGRP Autonomous Systems If a prefix is known in both AS with the same AD, only one AS can install it in the RIB The EIGRP AS that failed to install it will not propagate the route to its peers /32 A B C RtrA#show ip eigrp 1 topology P /32, 1 successors, FD is via Connected, Loopback0 RtrA#show ip eigrp 2 topology P /32, 1 successors, FD is via Connected, Loopback0 RtrB#show ip eigrp 1 topology P /32, 1 successors, FD is via (409600/128256), Ethernet0/0 RtrB#show ip eigrp 2 topology P /32, 0 successors, FD is Inaccessible via (409600/128256), Ethernet0/0 D RtrD#show ip eigrp 2 topology incl RtrD# RtrC#sh ip eigrp 1 topology P /32, 1 successors, FD is via (435200/409600), Ethernet0/0 38

39 Zero Successor Routes Another case of the zero successor route problem is when there are overlapping EIGRP Autonomous Systems. Sometimes a network designer will define two EIGRP Autonomous Systems with the same network statement, covering the same interfaces and expect both of them to propagate the associated routes; this is often done during AS transition time when they re trying to combine Autonomous Systems The problem is that a prefix cannot be installed in the RIB by both Autonomous Systems at the same time, so one will be accepted and one will be rejected; the one that is rejected will not be sent to peers in the losing Autonomous System 39

40 Duplicate Router-ID A problem previously limited to redistributed routes happens when there are duplicate router-ids Please note that this problem is no longer limited to external routes! In this example, RtrB sees the route redistributed from RIP on RtrA just fine, but RtrC does not see it A B C /24 via RIP router eigrp 100 redistribute rip default-metric... RtrB#show ip route /24 via [RtrA] RtrC#show ip route RtrC# 40

41 Duplicate Router-ID Looking at RtrB s topology table, we can see the originating router ID field in the external route is set to But, that s RtrC s loopback address! A B RtrB#show ip eigrp topology IP-EIGRP (AS 1) topology entry for /24... External data: Originating router is C

42 Duplicate Router-ID In the example above, we have a problem where routes are being redistributed into the network, but one router elsewhere in the network is refusing to install the routes into its topology table or routing table As the slide shows, the problem is that the router-id of the redistributing router matches the router-id of the router refusing to install the route To block routing loops, a router doing redistribution will not accept a route from a neighbor if it is the one that originated it via redistribution; this is known by the originating router field in the external data section of the topology table entry Since the router-ids are the same on RtrA and RtrC, RtrC thinks RtrA s external routes originated on RtrC and it rejects them 42

43 Duplicate Router-ID Impact on route installation EIGRP includes the router ID of the originating router in external routing information in older code, and both internal and external routes in newer code If a router receives an route with a router ID matching its own local router ID, it discards the route This prevents routing loops/sia for routes originated locally but also learned form others You need to make sure your router-ids are unique by either not duplicating addresses on loopback interfaces or explicitly defining the router-id! 43

44 Duplicate Router-ID The EIGRP router ID is derived from The router-id command Highest IP address on a loopback interface Highest non-loopback interface IP address if no loopbacks NOTE: Interface used for router-id must reside in the same routing table as the EIGRP process creating the router-id Once the router ID is set, it won t be changed without manual intervention, even if the interface from which it s taken is removed from the router 44

45 Duplicate Router-ID Environments containing mixed versions of IOS can create interesting problems if duplicate router-ids exist in your network If a prefix is learned through a path that runs code that doesn t support the internal router-id, the information gets stripped out This could cause inconsistent results! 45

46 Duplicate Router-ID The EIGRP router ID was added to internal routes as part of release 5 EIGRP Code before release 5 exchanges older TLV types which do not contain the router-id for Internal routes If a router running release 5 or later sends updates to an older peer (pre-release 5), it sends the older TLV type without the router-id in the packet This can cause inconsistencies in your network 46

47 Duplicate Router-ID In the network shown here, RtrA receives updates for /24 through two paths Because the internal router-id is not supported by RtrC, RtrA would install the prefix learned through him The prefix learned through RtrB would be rejected due to the duplicate router-id This could also mean that the prefix works sometimes and not others! Rel A B C Rel 5 Pre-Rel 5 D Rel /24 47

48 Duplicate Router-ID In the preceding slide, a prefix could be learned via two different paths. If the prefix is learned from one peer, it doesn t contain the router-id and is accepted and installed. If the prefix is learned from the other peer, the router-id is included in the update and the prefix is rejected due to the duplicate router-id. This example shows a straightforward result, with one path not allowed stopping the equal cost load-balancing that should be going on based on the topology. In some cases, it may happen that at times a prefix could be installed if certain links were up (or down) but not if other links are up (or down.) This inconsistency could be extremely difficult to troubleshoot due to the apparent random nature of the symptoms. Just remember, router-ids need to be unique! 48

49 Duplicate Router-ID In older versions of Cisco IOS software, the only way to find out a router s router ID was to go to an adjacent router and look for some redistributed route from that router router# show ip eigrp topology IP-EIGRP (AS 7): topology entry for /24 State is Passive, Query origin flag is 1, 1 Successor(s), FD is Routing Descriptor Blocks: (Ethernet0/0), via , Send flag is 0x0 Composite metric is ( /0), Route is External... External data: Originating router is AS number of route is 0 External protocol is RIP, external metric is 1 Administrator tag is 0 (0x ) 49

50 Duplicate Router-ID In current versions of Cisco IOS software, the router ID is listed in the output of show ip eigrp topology router-1# show ip eigrp topology IP-EIGRP Topology Table for AS(7)/ID( )... If your event log is large enough, or things are happening slowly enough, you might also see the problem indicated in your event log 1 02:30: Ignored route, metric: :30: Ignored route, neighbor info: /24 Serial0/3 3 02:30: Ignored route, dup router:

51 Duplicate Router-ID To determine the EIGRP release to see if it supports the internal router-id, use the command show eigrp plugin If the command is rejected, you re definitely running older code If the command is accepted, look at the version of EIGRP in the output to see if it s before or after release 5 RtrA#show eigrp plugin EIGRP feature plugins::: eigrp-release : : Portable EIGRP Release : : Source Component Release(Portable EIGRP Release(rel5_1)) igrp2 : : Reliable Transport/Dual Database external-client : : Service Distribution Client Support Snip 51

52 Duplicate Router-ID To determine the release of EIGRP you re running on a router, use the command show eigrp plugin or show eigrp plugin detail. These commands are explained a little later in this presentation but I thought you could use the info here, as well. Older code (Release3 and earlier, IIRC) did not support the show eigrp plugin command and will reject it as invalid. Newer code will show you the version of EIGRP you re running in addition to which subsystems/features are loaded into EIGRP. 52

53 EIGRP Resource Depletion EIGRP by default will use up to 50% of the link bandwidth for EIGRP packets This parameter is manually configurable by using the command: ip bandwidth-percent eigrp <AS-number> <nnn> 53

54 EIGRP Resource Depletion Prior to CSCdi36031 (roughly 10.3), EIGRP had huge problems with bandwidth depletion EIGRP would use all of the link at the expense of layer 2 keep alives! With CSCdi36031, there was a significant change in the way we build and transmit packets at the time that still effects low-speed links; occasionally the TAC still gets cases or questions about the behavior so it s worth explaining here so you can design accordingly The biggest part of the change was to implement packet pacing based on the defined bandwidth of the interface. This packet pacing puts enough gaps between the packets to ensure that we don t overwhelm the interface with EIGRP packets causing the layer two keepalives to be missed and the interface to drop There are some circumstances where the bandwidth on the interface isn t a good measure of what pacing should be, however 54

55 Bandwidth over Multipoint Interfaces EIGRP over multipoint interfaces such as DMVPN and mgre has to share the available bandwidth among peers EIGRP uses the bandwidth on the main interface divided by the number of neighbors on that interface to get the bandwidth available per neighbor 55

56 EIGRP Resource Depletion Some interface types appear to EIGRP to be a shared interface but in reality they re provided by a point-to-point mechanism (like DMVPN Dynamic Multipoint Virtual Private Networks which uses MGRE for transport) and the ability of the underlying network may not match up with the bandwidth defined on the interface; for example, if an mgre outbound interface is Gigabit Ethernet but the tunnels traverse an ISPs network, we can t actually send at Gigabit rates and expect all of the packets to be delivered at that rate EIGRP divides the defined bandwidth by the number of peers seen, giving each approximately equal shares of the available bandwidth; this may not be exactly right, but it s the best we can guess 56

57 Bandwidth over Multipoint Interfaces Set the bandwidth on the multipoint interface to a value that most closely defines the actual ability to deliver packets across the interface to the peers For DMVPN, other resources can also be depleted (and often are) Several processes are involved, each of which has overhead and can run into resource depletion nhrp, ipsec, etc. Make sure buffers are tuned to minimize/eliminate drops 57

58 EIGRP Resource Depletion DMVPN/mGRE is a very popular technology which presents some interesting challenges in troubleshooting and avoiding problems; from an EIGRP perspective, the mgre Tunnel interface is multicast and our sending process (and pacing) is based on the assumption that we send a multicast packet out to all of the peers on the Tunnel interface in reality, the multicast packet is replicated by the NHRP code (Next Hop Resolution Protocol), which then delivers the replicated packets to (normally) ipsec for encryption before delivering on the Tunnel Each step along the way has queues and buffers and other resources required to do the job of packet delivery each of these resources are potential places of resource depletion; the following slides give you a few commands we ve found useful when troubleshooting large scale DMVPN environments 58

59 EIGRP Resource Depletion Many commands are useful to see how DMVPN/mGRE are behaving resource-wise Show ip nhrp summary Show ip nhrp multicast Show ip nhrp traffic Show buffers include failures Show interface tunnel 1 include nput Show interface Gig 0/1 include nput (for outbound interface used by tunnel) Show buffer input-interface Gig 0/1 header Show ip eigrp topo summary Show ip eigrp traffic Show ip eigrp interface detail tunnel 1 59

60 EIGRP Resource Depletion hub2#show ip nhrp summary IP NHRP cache 800 entries, bytes 0 static 800 dynamic 0 incomplete hub2#show ip nhrp /32 via Tunnel2 created 1w6d, expire 00:14:37 Type: dynamic, Flags: unique registered NBMA address: /32 via Tunnel2 created 1w6d, expire 00:14:37 Type: dynamic, Flags: unique registered NBMA address:

61 EIGRP Resource Depletion hub2#show ip nhrp multicast I/F NBMA address Tunnel Flags: static Tunnel Flags: dynamic Tunnel Flags: dynamic Tunnel Flags: dynamic Tunnel Flags: dynamic Tunnel Flags: dynamic hub2#show ip nhrp traffic Tunnel2: Max-send limit:65535pkts/10sec, Usage:0% Sent: Total Resolution Request 0 Resolution Reply 0 Registration Request Registration Reply 0 Purge Request 0 Purge Reply 0 Error Indication 0 Traffic Indication Rcvd: Total Resolution Request 0 Resolution Reply Registration Request 0 Registration Reply 0 Purge Request 0 Purge Reply 0 Error Indication 0 Traffic Indication 61

62 EIGRP Resource Depletion hub2#show buffers include failures 0 failures (0 no memory) 0 failures (0 no memory) 0 failures (0 no memory) 0 failures (0 no memory) 0 failures (0 no memory) hub2#show interface tunnel 2 include nput Last input 00:00:00, output 00:00:01, output hang never Input queue: 0/4096/0/0 (size/max/drops/flushes); Total output drops: 0 30 second input rate bits/sec, 189 packets/sec packets input, bytes, 0 no buffer 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 62

63 EIGRP Resource Depletion hub2#show interface gig 0/3 include nput output flow-control is XON, input flow-control is unsupported Last input 00:00:00, output 00:00:01, output hang never Input queue: 1/4096/0/0 (size/max/drops/flushes); Total output drops: 0 5 minute input rate bits/sec, 173 packets/sec packets input, bytes, 0 no buffer 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, multicast, 0 pause input 63

64 EIGRP Resource Depletion hub2#show buffer input-interface gig 0/3 header Buffer information for Middle buffer at 0x7A59A7C data_area 0x7890CFE4, refcount 1, next 0x0, flags 0x280 linktype 7 (IP), enctype 1 (ARPA), encsize 14, rxtype 1 if_input 0x5A9DA04 (GigabitEthernet0/3), if_output 0x0 (None) inputtime 1w6d (elapsed 00:00:00.004) outputtime 00:00: (elapsed never), oqnumber datagramstart 0x7890D02A, datagramsize 108, maximum size 756 mac_start 0x7890D02A, addr_start 0x7890D02A, info_start 0x0 network_start 0x7890D038, transport_start 0x7890D04C, caller_pc 0x22DCC58 source: , destination: , id: 0xCA33, ttl: 252, prot: 47 64

65 EIGRP Resource Depletion hub2#show ip eigrp topology summary EIGRP-IPv4 Topology Table Summary for AS(1)/ID( ) Head serial 1, next serial routes, 0 pending replies, 0 dummies Enabled on 877 interfaces, 800 neighbors present on 1 interfaces Quiescent interfaces: Tu2 hub2#show ip eigrp traffic EIGRP-IPv4 Traffic Statistics for AS(1) Hellos sent/received: / Updates sent/received: 99630/12123 Queries sent/received: 0/0 Replies sent/received: 0/0 Acks sent/received: 2609/ SIA-Queries sent/received: 0/0 SIA-Replies sent/received: 0/0 Hello Process ID: 260 PDM Process ID: 259 Socket Queue: 0/2000/864/0 (current/max/highest/drops) Input Queue: 0/2000/864/0 (current/max/highest/drops) 65

66 DMVPN with Multiple Next Hops In DMVPN phase 2 setup, a hub router can learn two or more equalcost paths to a site. However, the hub router will only advertise one of the paths to other spokes in the DMVPN network. Implication: Spoke to spoke tunnels will only be established to a single router and cannot leverage this multi-routers setup [90/18600] via , Tunnel1 via , Tunnel / [90/32600] via , Tunnel1 66

67 DMVPN with Multiple Next Hops While this isn't a route propagation problem, per se, it's still a situation that may take you by surprise and therefore may be useful to understand One of the designs being implemented with DMVPN uses multiple paths from the hub to reach spoke subnets. This could be two paths to the same spoke or through two spokes (as shown on the previous slide) The problem is that EIGRP still uses normal distance vector rules and sends updates based on the top topology table entry. Even if there are two equal cost paths, EIGRP sends updates based on the top entry. Since historically distance vector protocols only report how far they are metric-wise from a destination, this has been enough. Now it s not quite enough information 67

68 DMVPN with Multiple Next Hops One way to avoid this situation is to use a different hub for each preferred spoke path Each hub could use either a metric type command (offset-list) or distance (if all internals) to prefer one path or the other in order to propagate the desired next-hop information to the other spokes [90/18600] via [90/18600] via [90/32600] via via /24 68

69 DMVPN with Multiple Next Hops There isn't necessarily a great solution to this problem at the moment, though it's not that hard to design your network so that the two paths to the prefix at the spoke is learned through two different hubs rather than through one You could then use metric (offset-list possibly) or the distance command to have each hub prefer a different path to the spoke prefix. It could then advertise the next-hop value associated with it's choice, avoiding the problem. There is also work in progress to allow EIGRP to send updates in the DMVPN case using all equal cost next-hops seen on that interface. This would be the most elegant solution but not quite ready for prime-time yet 69

70 Redistribution Problems Redistribution metrics Static route to connected interface Multiple points of redistribution 70

71 Redistribution Basics Redistribution is used to advertise routes learned in another routing protocol (or another EIGRP AS) into the EIGRP network; routes that are redistributed into EIGRP are considered less trustworthy then native EIGRP routes because of the loss of specific topology information/metrics because they re less trustworthy, they re given a worse admin distance so that routes that are learned internally within the AS are preferred Redistribution is a fact of life in many networks, with the foreign route sources coming from suppliers, other divisions, other companies when there are mergers, etc.; redistribution isn t evil, but it needs to be controlled so that the EIGRP network remains stable Another use of redistribution is for MPLS/VPN over BGP using PE-CE support; this creates its own interesting troubleshooting challenges 71

72 Redistribution Metrics One of the most common problems with redistribution into EIGRP is when a redistribution metric isn t defined RtrA is redistributing /24 from RIP, but RtrB and RtrC do not have the route installed The first thing to check is whether RtrA has a redistribution metric configured via either Default-metric <metric> Redistribute rip metric <metric> EIGRP can t directly turn a hop count or cost into an EIGRP metric, so it won t redistribute routes unless it knows what metric to assign to them A B C /24 via RIP router eigrp 100 redistribute rip What Metric Should I Use? RtrB#show ip route B# RtrC#show ip route RtrC# 72

73 Redistribution Metrics As demonstrated, EIGRP must have a redistribution metric to use for the routes being redistributed; the two forms of supplying this redistribution metric serve similar, but not identical purposes make sure you use the one that matches your requirements If you want to set the metric to be used for routes from a particular redistribution source, use the metric keyword on the redistribution statement router eigrp 1 redistribute rip metric redistribute ospf 1 metric If you want all redistribution sources to have the same metric applied to the redistributed routes, you can use the default-metric command instead router eigrp 1 redistribute rip redistribute ospf 1 default-metric

74 Redistribution Metrics EIGRP will automatically derive the redistribution metrics from: A connected interface for redistribute connected The interface through which a static route is reached for redistribute static (note: this isn t always reliable; you re better off specifying the redistribution metric!) The metric of an IGRP route in the same AS The metric of an EIGRP route from another AS If none are those are true, you must supply the metric for redistribution 74

75 Static Route to Connected Interface Another surprise you could hit isn t really a problem but could be unexpected A static route with the next-hop pointing to a local interface may be automatically redistributed This happens only if the destination network is covered by a network statement 75 r32#show ip route /24 is subnetted, 3 subnets C is directly connected, Ethernet0/0 D [90/281600] via , 00:19:20, Ethernet0/0 D [90/307200] via , 00:24:19, Ethernet0/0 r31 r32 router eigrp 1 network ! Ip route null0 r31#show ip eigrp topology /24 IP-EIGRP (AS 1): Topology entry for /24 State is Passive, Query origin flag is 1, 1 Successor(s), FD is 256 Routing Descriptor Blocks: , from Rstatic, Send flag is 0x0 Composite metric is (256/0), Route is Internal Vector metric: Minimum bandwidth is Kbit Total delay is 0 microseconds Reliability is 0/255 Load is 0/255 Minimum MTU is 1500 Hop count is 0

76 Static Route to Connected Interface One situation that isn t necessarily a problem but creates calls to the TAC and questions on our internal aliases is the apparently bizarre behavior that EIGRP will automatically redistribute a static route even if redistribute static isn t configured in some circumstances how could this not be a bug? When a static route is configured and the next-hop is a local interface (including null0) it sets a bit on the route identifying that it is pseudo-connected which requires things like ARP to reach hosts within the subnet Since the connected bit is set on the route, EIGRP picks it up if the destination of the route is also covered by a network statement; this has always been the case and is operating as designed One really unusual aspect of the route that redistributed is that it shows up as a redistributed internal so peers will see the route as an internal even though it s redistributed confusing, huh? 76

77 OSPF EIGRP Multiple Points of Redistribution A route is injected into EIGRP as an external; this route is redistributed into OSPF by RtrB The route is transmitted through OSPF to RtrA, who redistributes it back into EIGRP Depending on the manually set metrics, RtrB may prefer this redistributed route, building a routing loop Depending on the timing, the loop can be persistent or transient. Either way, a bad thing! A Metric 25 Metric Metric 10 Metric B Metric /24 77

78 Multiple Points of Redistribution As mentioned before, redistribution isn t evil in itself, but a network designer needs to be particularly careful if there are multiple points of redistribution between routing protocols; also as mentioned before, a redistribution metric is normally supplied manually at the redistribution point this artificial setting of the metric hides where the redistributed routes actually exist in the network Because of the loss of specific topology information due to resetting the metrics, suboptimal routing is likely if there are multiple points of redistribution how can you know which redistribution point is closest to the actual destination? You can t Not only that, it s possible to create routing loops if the redistribution metrics at some places is better than the original metric; in the above example, a route that originates in EIGRP as an external that is then redistributed into OSPF and back into EIGRP could have a better metric at the inbound OSPF redistribution point than at the original redistribution point broken 78

79 Redistribution Design There are three primary methods used to prevent this routing loop: Redistributing live routing information in only one direction Filtering routes based on the prefixes advertised to prevent feedback loop Filtering routes using routing tags to prevent feedback 79

80 Multiple Points of Redistribution The possible routing loop previously discussed occurs because of the mutual redistribution between protocols at multiple points; if the redistribution occurs in only one direction, the invalid improvement in metric cannot occur One way to change this from a mutual redistribution scenario to one-way redistribution is to provide the routes in one direction either through summarization or through a redistributed static One direction uses dynamic redistribution and other direction is static 80

81 OSPF / /16 EIGRP Multiple Points of Redistribution Live Routing Information in Only One Direction Redistribute a static in one direction, and between protocols in the other direction ip route serial 0/0 A route is injected into EIGRP as an external; this route is then redistributed into OSPF by RtrB Metric 25 A The route is transmitted to RtrA through OSPF; the route is not redistributed back into EIGRP, since redistribution between OSPF and EIGRP is not configured Metric 10 Metric B router ospf 100 redistribute eigrp 100 metric 10 router eigrp 100 redistribute static metric /24 Metric

82 Multiple Points of Redistribution Another way to eliminate the routing loop is by filtering routes that originate in EIGRP from being relearned back into EIGRP from the OSPF-EIGRP redistribution point; in other words, if the route originated in EIGRP, we have no need to accept the route back into EIGRP after it s been redistributed into OSPF This filtering can be done with distribute-lists if the prefixes involved are easily identified blocks if not, we have other techniques 82

83 OSPF / /16 EIGRP Multiple Points of Redistribution Filtering Based on Prefixes Configure access/prefix lists which match the address ranges used in each section of the network and filter based on these ACLs The route is injected into EIGRP as an external; this route is then redistributed into OSPF by RtrB The route is transmitted through OSPF and reaches RtrA The route is now blocked by distribute list 20, which breaks the routing loop access-list 10 permit access-list 20 permit Metric 25 router ospf 100 redistribute eigrp 100 metric 10 distribute-list 10 out A Metric 10 Metric B router eigrp 100 redistribute ospf 100 metric distribute-list 20 out /24 Metric

84 Multiple Points of Redistribution If the prefixes generated in each routing protocol are not in well-defined blocks that are easily specified in an access-list, filtering can also be done using routetags The tags can be applied as a route is redistributed from EIGRP into OSPF (as well as OSPF into EIGRP) Filters can be set to deny the routes from re-entering the domains in which they originated This is a far more flexible filtering method since it doesn t require that the routes being filtered be in well-defined blocks; any route that has the tag set will be matched in the filtering process 84

85 OSPF / /16 EIGRP Multiple Points of Redistribution Filtering Based on Tags Set route tags when redistributing between the protocols; deny tagged routes at the redistribution point The route is injected into EIGRP as an external; it is redistributed into OSPF by RtrB and a tag is set The route is transmitted to RtrA through OSPF The route is blocked from being redistributed into EIGRP because of the route tag route-map usetags deny 10 match tag 1000 route-map usetags permit 20 set tag 1000 Metric 25 router ospf 100 redistribute eigrp 100 metric 10 route-map usetags router eigrp 100 redistribute ospf 100 metric route-map usetags A Metric 10 Metric B /24 Metric

86 Troubleshooting Tips

87 Troubleshooting Tips Neighbor Stability Problems Stuck In Active Routes 87

88 Checking Neighbor Status RtrA#show ip eigrp neighbors IP-EIGRP neighbors for process 1 H Address Interface Hold Uptime SRTT RTO Q Seq (sec) (ms) Cnt Num Et0 12 6d16h Et1 13 2w2d Et1 10 2w2d Seconds Remaining Before Declaring Neighbor Down How Long Since the Last Time Neighbor Was Discovered How Long It Takes for This Neighbor to Respond to Reliable Packets How Long We ll Wait Before Retransmitting if No Acknowledgement 88

89 Checking Neighbor Status The most useful command for checking neighbor status is show ip eigrp neighbors Some of the important information provided by this command are Hold time time left that you ll wait for an EIGRP packet from this peer before declaring him down Uptime how long it s been since the last time this peer was initialized SRTT (Smooth Round Trip Time) average amount of time it takes to get an Ack for a reliable packet from this peer RTO (Retransmit Time Out) how long to wait between retransmissions if Acks are not received from this peer 89

90 Checking Neighbor Status EIGRP Log-Neighbor-Changes is on by default since 12.2(12) Turn it on and leave it on Best to send to buffer log RtrA# config terminal Enter configuration commands, one per line. End with CNTL/Z. RtrA(config) # router eigrp 1 RtrA(config-router) # eigrp log-neighbor-changes RtrA(config-router) # logging buffered RtrA(config) # service timestamps log datetime msec 90

91 Checking Neighbor Status EIGRP log-neighbor-changes is the best tool you have to understand why neighbor relationships are not stable. It should be enabled on every router in your network CSCdx67706 (12.2(12)) made it the default behavior; as explained on the previous slide, the uptime value from show ip eigrp neighbors will tell you the last time a neighbor bounced, but not how often or why with log-neighbor-changes on and logging buffered, you keep not only a history of when neighbors have been reset, but the reason why absolutely invaluable Logging buffered is also recommended, because logging to a syslog server is not bulletproof; for example, if the neighbor bouncing is between the router losing neighbors and the syslog server, the messages could be lost it s best to keep these types of messages locally on the router, in addition to the syslog server It may also be useful to increase the size of the buffer log in order to capture a greater duration of error messages you would hate to lose the EIGRP neighbor messages because of flapping links filling the buffer log; if you aren t starved for memory, change the buffer log size using the command logging buffered in configuration mode The service timestamps command above puts more granular timestamps in the log, so it s easier to tell when the neighbor stability problems occurred 91

92 Log-Neighbor-Changes Messages So this tells us why the neighbor is bouncing but what do they mean? Hint: peer restarted means you have to ask the peer; he s the one that restarted the session Neighbor (Ethernet0) is down: peer restarted Neighbor (Ethernet0) is up: new adjacency Neighbor (Ethernet0) is down: holding time expired Neighbor (Ethernet0) is down: retry limit exceeded Others, but not often 92

93 Log-Neighbor-Changes Messages Peer restarted the other router reset our neighbor relationship; you need to go to him to see why it thought our relationship had to be bounced New adjacency established a new neighbor relationship with this neighbor; happens at initial startup and after recovering from a neighbor going down Holding time expired we didn t hear any EIGRP packets from this neighbor for the duration of the hold time; this is typically 15 seconds for most media (180 seconds for low-speed NBMA) Retry limit exceeded this neighbor didn t acknowledge a reliable packet after at least 16 retransmissions (actual duration of retransmissions is also based on the hold time, but there were at least 16 attempts) 93

94 Holding Time Expired The holding time expires when an EIGRP packet is not received during hold time Typically caused by congestion or physical errors A Ping Ping the multicast address ( ) from the other router If there are a lot of interfaces or neighbors, you should use extended ping and specify the source address or interface Hello Hello B Neighbor (Ethernet0) is down: holding time expired 94

95 Holding Time Expired When an EIGRP packet is received from a neighbor, the hold timer for that neighbor resets to the hold time supplied in that neighbor s hello packet, then the value begins decrementing The hold timer for each neighbor is reset back to the hold time when each EIGRP packet is received from that neighbor (long ago and far way, it needed to be a hello received, but now any EIGRP packet will reset the timer) Since hellos are sent every five seconds on most networks, the hold time value in a show ip eigrp neighbors is normally between 10 and 15 (resetting to hold time (15), decrementing to hold time minus hello interval or less, then going back to hold time) Why would a router not see EIGRP packets from a neighbor? He may be gone (crashed, powered off, disconnected, etc.) He (or we) may be overly congested (input/output queue drops, etc.) Network between us may be dropping packets (CRC errors, frame errors, excessive collisions) 95

96 Holding Time Expired RtrA# debug eigrp packet hello EIGRP Packets debugging is on (HELLO) 19:08:38.521: EIGRP: Sending HELLO on Serial1/1 19:08:38.521: AS 1, Flags 0x0, Seq 0/0 idbq 0/0 iidbq un/rely 0/0 19:08:38.869: EIGRP: Received HELLO on Serial1/1 nbr :08:38.869: AS 1, Flags 0x0, Seq 0/0 idbq 0/0 iidbq un/rely 0/0 19:08:39.081: EIGRP: Sending HELLO on FastEthernet0/0 19:08:39.081: AS 1, Fags 0x0, Seq 0/0 idbq 0/0 iidbq un/rely 0/0 Remember Any Debug Can Be Hazardous 96

97 Holding Time Expired Another troubleshooting tool available is to do the command debug eigrp packet hello ; this will produce debug output to the console or buffer log (depending on how you have it configured) that will show the frequency of hellos sent and received You should make sure you have the timestamps for the debugs set to a value that you can actually see the frequency; something like: service timestamps debug datetime msec Remember that any time you enable a debug on a production router, you are taking a calculated risk; it s always better to use all of the safer troubleshooting techniques before resorting to debugs sometimes they re necessary, however 97

98 Retry Limit Exceeded EIGRP sends both unreliable and reliable packets Hellos and Acks are unreliable Updates, Queries, Replies, SIA-queries and SIA-replies are reliable Reliable packets are sequenced and require an acknowledgement Reliable packets are retransmitted up to 16 times if not acknowledged 98

99 Retry Limit Exceeded Exceeding the retry limit means that we re sending reliable packets which are not getting acknowledged by a neighbor when a reliable packet is sent to a neighbor, it must respond with a unicast acknowledgement; if a router is sending reliable packets and not getting acknowledgements, one of two things are probably happening The reliable packet is not being delivered to the neighbor The acknowledgement from the neighbor is not being delivered to the sender of the reliable packet These errors are normally due to problems with delivery of packets, either on the link between the routers or in the routers themselves congestion, errors, and other problems can all keep unicast packets from being delivered properly; look for queue drops, errors, etc., when the problem occurs, and try to ping the unicast address of the neighbor to see if unicasts in general are broken or whether the problem is specific to EIGRP 99

100 Retry Limit Exceeded Reliable packets are re-sent after Retransmit Time Out (RTO) Typically 6 x Smooth Round Trip Time (SRTT) Minimum 100 ms Maximum 5000 ms (five seconds) 16 retransmits takes between roughly 40 and 80 seconds If a reliable packet is not acknowledged before 16 retransmissions and the hold timer duration has passed, re-initialize the neighbor B Neighbor (Ethernet0) is down: retry limit exceeded A Ack Packet 100

101 Retry Limit Exceeded The Retransmit Timeout (RTO) is used to determine when to retry sending a packet when an Ack has not been received, and is (generally) based on 6 X Smooth Round Trip Time (SRTT); the SRTT is derived from previous measurements of how long it took to get an Ack from this neighbor the minimum RTO is 100 Msec and the maximum is 5000 Msec; each retry backs off 1.5 times the last interval The minimum time required for 16 retransmits is approximately 40 seconds (minimum interval of 100 ms with a max interval of 5000 ms); for example, If there isn t an acknowledgement after 100 ms, the packet is retransmitted and we set a timer for 150 ms if it expires, we send it again and set the timer for 225 ms, then 337 ms, etc., until 5000 ms is reached; 5000 ms is then repeated until a total of 16 retransmissions have been sent The maximum time for 16 retransmits is approximately 80 seconds, if the initial retry is 5000 ms and all subsequent retries are also 5000 ms 101

102 Retry Limit Exceeded If a reliable packet is retransmitted 16 times without an acknowledgement, EIGRP checks to see if the duration of the retries has reached the hold time, as well Since the hold time is typically 15 sec on anything but low-speed NBMA, it normally isn t a factor in the retry limit; NBMA links that are T1 or less, however, wait an additional period of time after re-trying 16 times, until the hold-time period (180 seconds) has been reached before declaring a neighbor down due to retry limit exceeded This was done to give the low-speed NBMA networks every possible chance to get the Acks across before downing the neighbor Remember this if you modify the hold times! 102

103 Unidirectional Links RtrB#show ip eigrp neighbors IP-EIGRP neighbors for process 1 H Address Interface Hold Q Seq Cnt Num Et A Update Hello %DUAL-5-NBRCHANGE: IP-EIGRP 1: Neighbor (Serial1) is down: retry limit exceeded RtrA#show ip eigrp neighbors IP-EIGRP neighbors for process 1 RtrA# B 103

104 Unidirectional Links In this example, we see what happens when a link is only working in one direction; unidirectional links can occur because of a duplicate IP address, a wedged input queue, link errors, or any other reason you can think of that would allow packets to be delivered only in one direction on a link RtrB doesn t even realize that RtrA exists RtrA is sending out his hellos, waiting for a neighbor to show up on the network; what it doesn t realize is that the RtrB is already out there and trying to bring up the neighbor relationship RtrB, on the other hand, sees the hellos from RtrA, sends his own hellos and then sends an update to RtrA to try to get their topology tables/routing tables populated unfortunately, since the updates are also not being received by RtrA, it of course isn t sending acknowledgements; RtrB tries it 16 times and then resets his relationship with RtrA and starts over You ll spot this symptom by the retry limit exceeded messages on RtrB, RtrB having RtrA in his neighbor table with a continual Q count, and RtrA not seeing RtrB, at all CSCdy45118 has been implemented to create a reliable neighbor establishment process (threeway handshake) and reliable neighbor maintenance (neighbor taken down more quickly when unidirectional link encountered). 12.2T, 12.3 and up 104

105 Retry Limit Exceeded Ping the neighbor s unicast address Vary the packet size Try large numbers of packets This ping can be issued from either neighbor; the results should be the same Common causes Mismatched MTU Unidirectional link Dirty link RtrB# ping Protocol[ip]: Target IP address: Repeat count [5]: 100 Datagram Size: 1500 Timeout in seconds[2]: Extended commands[n]: y... A B 105

106 Manual Changes Some manual configuration changes can also reset EIGRP neighbors, depending on the Cisco IOS version Summary changes (manual and auto) Route filter changes Stub setting changes This is normal behavior for older code CSCdy20284 removed many of these neighbor resets Implemented in 12.2S, 12.3T, and 12.4 (approximately 2005) 106

107 Manual Changes Summary changes When a summary changes on an interface, components of the summary may need to be removed from any neighbors reached through that interface; neighbors through that interface are reset to synch up topology entries Route filter changes Similar to summary explanation above; neighbors are bounced if a distribute-list is added/removed/changed on an interface in order to synch up topology entries In the past, we also bounced neighbors when interface metric info changed (delay, bandwidth), but we no longer do that (CSCdp08764) CSCdy20284 was implemented to stop bouncing neighbors when many manual changes occur; in late 12.2S, 12.3T, and 12.4, summary and filter changes no longer bounce neighbors Changing Stub setting or router-ids still resets peers! Remember to make these changes during maintenance windows 107

108 Stuck-in-Active Routes (SIA) Indicates at least two problems A route went active It got stuck %DUAL-3-SIA: Route stuck-in-active state in IP-EIGRP 100. Cleaning up 108

109 The Active Process RtrA loses its route to /24 RtrA has no other path to this destination, so it marks the route as Active and sends a Query to RtrB RtrB receives this Query from its successor and has no other paths to reach the destination RtrB marks /24 as Active, and sends a Query to RtrC /24 A Query B Query C No other path 109

110 The Active Process RtrC receives the Query and has no more neighbors to Query and no alternate paths to /24 RtrC marks the route as unreachable, and sends a Reply to RtrB RtrB receives the Reply, marks /24 as unreachable, and sends a Reply to RtrA RtrA receives the Reply and since it didn t learn any viable paths to reach /24, it deletes the route from the topology and routing tables /24 A Query Reply B Query C Reply No other path 110

111 The Active Process What happens is RtrC s Reply isn t sent, or doesn t make it tortrb? While RtrC is trying to send the Reply, RtrA s Active timer is running After 90 seconds, RtrA sends an SIA query to RtrB If RtrB is still waiting on RtrC, it sends an SIA reply to RtrA /24 Active Timer A Query SIA Query SIA Reply No other path B Query C Reply 111

112 The SIA Query Process Sometimes the active process doesn t complete normally; this can be due to a number of different problems which are covered later in this presentation what happens when things go wrong? If RtrB doesn t respond to RtrA within 1.5 minutes because it s still waiting for a Reply from RtrC, RtrA will send an SIA-query to RtrB checking the status if RtrB is still waiting for a Reply itself, it will respond to RtrA with an SIA-reply; this resets the SIA timer on RtrA so it will wait another 1.5 minutes Eventually, the problem keeping RtrC from responding to RtrB will take the neighbor relationship down between RtrB and RtrC, which will cause RtrB to reply to A, ending the Query process 112

113 The SIA Query Process SIA-queries are sent to a neighbor up to three times May attempt to get a reply from a neighbor for a total of six minutes If a Reply is not received by the end of this process, the route is considered stuck through this neighbor On the router that doesn t get a reply after three SIA-queries Reinitializes neighbor(s) who didn t answer Goes active on all routes known through bounced neighbor(s) Re-advertises to bounced neighbor all routes that were previously advertised 113

114 Troubleshooting SIAs Two (probably) unrelated causes of the problem stuck and active Need to troubleshoot both parts Cause of active often easier to find Cause of stuck more important to find 114

115 Troubleshooting SIAs If routes never went active in the network, we would never have to worry about any getting stuck; unfortunately, in a real network there are often link failures and other situations that will cause routes to go active one of our jobs is to minimize them, however If there are routes that regularly go active in the network, you should absolutely try to understand why they are not stable; while you cannot ensure that routes will never go active on the network, a network manager should work to minimize the number of routes going active by finding and resolving the causes Even if you reduce the number of routes going active to the minimum possible, if you don t eliminate the reasons that they get stuck you haven t fixed the most important part of the problem; the next time you get an active route, you could again get stuck The direct impact of an active route is small; the possible impact of a stuck-in-active route can be far greater 115

116 Troubleshooting the Active Part of SIAs Determine what is common to routes going active Knows network problems? Flapping link(s)? From the same region of the network? Resolve whatever is causing them to go active (if possible) 116

117 Troubleshooting the Active Part of SIAs The syslog may tell you which routes are going active, causing you to get stuck. Since the SIA message reports the route that was stuck, it seems rather straight forward to determine which routes are going active. This is only partially true once SIAs are occurring in the network, many routes will go active due to the reaction to the SIA; you need to determine which routes went active early in the process in order to determine the trigger Additionally, you can do show ip eigrp topology active on the network when SIAs are not occurring and see if you regularly catch the same set of routes going active If you are able to determine which routes are regularly going active, determine what is common to those routes are links flapping (bouncing up and down) causing the routes (and everything behind it) to regularly go active? Are most or all of the routes coming from the same area of the network? If so, you need to determine what is common in the topology to them so that you can determine why they are not stable 117

118 Troubleshooting the Stuck Part of SIAs Show ip eigrp topology active Useful only while the problem is occurring If the problem isn t occurring at the time, it is very difficult to find the reason the routes are getting stuck 118

119 Troubleshooting the Stuck Part of SIAs Our best weapon to use to find the cause of routes getting stuck-in-active is the command show ip eigrp topology active; it provides invaluable information about routes that are in transition examples of the output of this command and how to evaluate it will be in the next several slides Unfortunately, this command only shows routes that are currently in transition; it isn t useful after the fact when you are trying to determine what happened earlier if you aren t chasing it while the problem is occurring, there aren t really any tools that will help you find the cause 119

120 /24 Chasing Active Routes A B C D / / / Why Is RtrA Reporting SIA Routes? Let s Look at a Problem in Progress RtrA#show ip eigrp topology active IP-EIGRP Topology Table for AS(1)/ID( ) A /24, 1 successors, FD is Inaccessible 1 replies, active 00:01:17, query-origin: Local origin via Connected (Infinity/Infinity), Ethernet1/0 Remaining replies: via , r, Ethernet0/0 RtrA Is Waiting on RtrB 120

121 Chasing Active Routes In our example network, we ve noticed dual-3-sia messages in the log of RtrA and we know the trigger is an unstable network off of this router; instead of just shutting down the unstable link, we decide to try to determine the cause of the stuck part of stuck-in-active In the above output, we see that RtrA is active on the route /24 (note the A in the left column) and has been waiting for an answer from (RtrB) for one minute and 17 seconds we know that we are waiting on RtrB because of the lower case r after the IP address; sometimes, the lower case r comes after the metric in the upper part of the output (not under remaining replies) don t be fooled the lower case r is the key, not whether it s under the remaining replies are or not Since we know why we are staying active on the route because RtrB hasn t answered us, we need to go to him (RtrB) to see why he s taking so long to answer 121

122 /24 Chasing Active Routes A B C D / / / So Why Hasn t RtrB Replied? RtrB#show ip eigrp topology active IP-EIGRP Topology Table for AS(1)/ID( ) A /24, 1 successors, FD is Inaccessible 1 replies, active 00:01:26, query-origin: Successor Origin via (Infinity/Infinity), Ethernet0/0 Remaining replies: via , r, Ethernet1/0 RtrB is Waiting on RtrC 122

123 Chasing Active Routes (Step 1) We repeat the show ip eigrp topology active command on RtrB and we get the results seen above We see that RtrB probably isn t the cause of our stuck-in-active routes, since it is also waiting on another router downstream to answer his query before it can reply; again, the lower case r beside the IP address of tells us it is the neighbor slow to reply We now need to go to (RtrC) and see why it isn t answering RtrB 123

124 /24 Chasing Active Routes A B C D / / / What s RtrC s Problem? RtrC#show ip eigrp topology active IP-EIGRP Topology Table for AS(1)/ID( ) A /24, 1 successors, FD is Inaccessible, Qqr 1 replies, active 00:01:33, query-origin: Successor Origin, retries(1) via (Infinity/Infinity), Ethernet0/0, serno 20 via (Infinity/Infinity), rs, q, Ethernet1/0, serno 19, anchored RtrC Is Waiting on RtrD 124

125 Chasing Active Routes (Step 2) On RtrC we repeat the show ip eigrp topology active command and see what it thinks of the route Again, he s waiting on another neighbor downstream to answer him before it can answer RtrB you are probably getting the idea of how exciting this process can be; of course, in a real network you probably have users/managers breathing down your neck making it a bit more interesting As I m sure you suspect our next step should be to see why (RtrD) isn t answering RtrC s query 125

126 /24 Chasing Active Routes A B C D / / / Why Isn t RtrD Answering? RtrD#show ip eigrp topology active IP-EIGRP Topology Table for AS(1)/ID( ) RtrD# 126

127 Chasing Active Routes (Step 3) And again, we look at the active topology table entries, this time on RtrD Wait RtrD isn t waiting on anyone for any routes; did the replies finally get returned and the route is no longer active? We need to go back to RtrC and see if it is still active on the route 127

128 /24 Chasing Active Routes A B C D / / / No; RtrC Is Still Waiting on RtrD; What s the Deal? RtrC#show ip eigrp topology active IP-EIGRP Topology Table for AS(1)/ID( ) A /24, 1 successors, FD is Inaccessible, Qqr 1 replies, active 00:01:52, query-origin: Successor Origin, retries(1) via (Infinity/Infinity), Ethernet0/0, serno 20 via (Infinity/Infinity), rs, q, Ethernet1/0, serno 19, anchored RtrC is waiting on RtrD 128

129 Chasing Active Routes (Step 4) Hmmm RtrC still thinks the route is active and it s gotten even older There appears to be a problem, Houston. RtrC thinks it needs a reply from RtrD, yet RtrD isn t active on the route; we need to take a look at the neighbor relationship between these two routers to try to identify what is going wrong 129

130 /24 Chasing Active Routes A B C D / / / Let s see why they don t seem to agree about the active route RtrC#show ip eigrp neighbors IP-EIGRP neighbors for process 1 H Address Interface Hold Uptime SRTT RTO Q Seq (sec) (ms) Cnt Num Et1/ :00: Et0/ :22: Looks like something s broken between RtrC and RtrD 130

131 Chasing Active Routes (Step 5) It appears that RtrC is having a bit of a problem communicating with RtrD the neighbor relationship isn t even making it completely up based on the Q count on RtrC; we also notice in the log that the neighbor keeps bouncing due to retry limit exceeded Now we need to use our normal troubleshooting methodology to determine why these two routers can t talk to each other properly 131

132 /24 Chasing Active Routes A B C D / / / RtrC#ping Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to , timeout is 2 seconds:... Success rate is 0 percent (0/5) Okay we can t ping; we need to fix this before EIGRP stands a chance of working 132

133 Chasing Active Routes (Step 6) How does basic connectivity look? A ping between RtrC and RtrD isn t succeeding either; we ll need to find out why they can t talk to each other Whatever is causing them to not talk to each other is undoubtedly a contributing factor to the SIAs we re seeing in the network; we need to find and fix the problem with this link and remove the cause of the SIA routes 133

134 Troubleshooting the Stuck Part of SIAs It s not always this easy to find the cause of an SIA Sometimes you chase the waiting neighbors in a circle If so, summarize and simplify Easier after CSCdp33034 SIA should happen closer to the location of the cause of the problem 134

135 Troubleshooting the Stuck Part of SIAs Our example of chasing SIA routes was intentionally made very easy in order to demonstrate the tools and techniques in a real event on a network, there would probably be many more routes active, and many more neighbors replying; this can make chasing the waiting neighbors significantly more challenging Usually, you will be able to succeed at tracking the waiting neighbors back to the source of the problem occasionally, you can t on highly redundant networks, in particular, you can find yourself chasing neighbors in circles without reaching an endpoint cause of the waiting; if you run into this case, you may need to temporarily reduce the redundancy in order to simplify the network for troubleshooting and convergence 135

136 Likely Causes for Stuck-in-Active Bad or congested links Query range is too long Excessive redundancy Overloaded router (high CPU) Router memory shortage Software defects (seldom) 136

137 Likely Causes for SIAs Remember that the cause of the SIA route could be a different location than where the SIA message and bounced neighbors happened; this is particularly true with code older than CSCdp33034 Some of the possible causes of SIAs are: Links that are either experiencing high CRC or other physical errors or are congested to the point of dropping a significant number of frames queries, replies, or acknowledgements could be lost The time it takes for a query to go from one end of the network to the other is too long and the active timer expires before the query process completes; I don t think I ve ever seen a network where this is true, by the way The complexity in the network is so great due to excessive redundancy that EIGRP is required to work so hard at sending and replying to queries that it cannot complete them in time A router is low on memory so that it is able to send hellos, which are very small, but be unable to send queries or replies There have occasionally been software defects that caused SIAs (CSCdi83660, CSCdv85419, CSCtc31545) 137

138 Excessive Redundancy What is excessive redundancy? Isn t redundancy a good thing, not something to avoid? I categorize excessive redundancy as alternative paths that exist in the network that provide little if any real benefit of improved reliability, and are often unplanned and unexpected In the above example, the four subnets on the left (which could be VLANs through a switch or any other media, for that matter) are there to provide users with access to the network; there are two routers connected to each VLAN in order to provide redundancy (probably via HSRP) so that the users will have failover capability if there is a problem Unfortunately, the designer may have created a network topology a little different than what it intended 138

139 Excessive Redundancy RtrA#show ip route begin C is directly connected, Loopback1...snip... RtrA#show ip eigrp topo begin P /24, 1 successors, FD is via Connected, Loopback1 P /24, 1 successors, FD is snip... RtrA#show ip eigrp topo all begin P /24, 1 successors, FD is , serno via Connected, Loopback1 via ( / ), FastEthernet6/0.19 via ( / ), FastEthernet6/0.20 via ( / ), FastEthernet6/0.13 via ( / ), FastEthernet6/0.45 via ( / ), FastEthernet6/0.27 via ( / ), FastEthernet6/0.28 via ( / ), FastEthernet6/0.22 via ( / ), FastEthernet6/ snip /24 A RtrA B RtrB Wow, Where did all of these alternative paths come from! 139

140 Excessive Redundancy If you just define network statements under EIGRP covering all of your interfaces, each of the user subnets will be treated by EIGRP as possible alternative paths to reach every destination in the network! It is rarely the network designers goal to have these user subnets used as transit paths to reach other parts of the network for anyone other than the users that reside on that segment, but EIGRP doesn t know what the designer intended, only what he/she configured As the output above shows, when something changes in the network EIGRP has to converge over each of the user subnets as part of the query path 140

141 Excessive Redundancy /24 A RtrA router eigrp 1 passive-interface fastethernet6/0.1 passive-interface fastethernet6/0.2 Or router eigrp 1 passive-interface default no passive-interface fastethernet0/0 B 141

142 Excessive Redundancy A simple solution to this particular problem is to use the passive-interface command. In EIGRP, defining an interface as passive means that the subnet on that interface is included in the EIGRP topology table and propagated to the rest of the network, but no peers will be formed across these interfaces; this means that they will not be in the transit path and will greatly simplify EIGRP s apparent topology and the associated complexity of convergence If you don t plan to have a link as a transit path, make it passive! Note that if you didn t want those interfaces to show up in EIGRP at all, you could define more specific network statements to only cover the interfaces you re interested in. Our assumption above is that those interfaces are needed in EIGRP, just not for peer formation. 142

143 Minimizing SIA Routes Decrease query scope (involve fewer routers in the query process) Summarization Route filters Define spoke/edge routers as stubs Run a Cisco IOS which includes CSCdp

144 Minimizing SIA Routes We ve now talked about the impact that SIA routes can have on your network and how to track down the root cause of SIA events; while you may not be able to completely rid your network of SIA routes, there are techniques you can use to minimize your exposure Decrease query scope in our example network, you saw the queries sent to each router in a chain; if a router receives a query on a route that it doesn t have in its topology table, it immediately answers and doesn t send the query onward this is a very good thing; you do this through: Summarization auto-summary (seldom used) or manual summary to summarize within a major network or to summarize external routes Route filtering used to limit knowledge of routes; particularly on dual-homed remotes, which tend to reflect all routes back to the other leg of the dual home connection Use hierarchy if the network doesn t have hierarchy, the two techniques above cannot adequately be used Define spoke/edge routers as stubs so they aren t queried at all Run a Cisco IOS with CSCdp33034 included 144

145 Problems with Links > 1G

146 Problems with Links > 1G Scaled Bandwidth Problem Delay Granularity Problem Path Selection Problem The Solution Wide Metrics Remaining Issues 146

147 Problems with Links > 1G Scaled Bandwidth Problem EIGRP was created when FastEthernet was considered fast In order to simplify metric calculation and comparison, bandwidth was scaled Scaled Bandwidth is 10^7/BW (with BW in kbps) This worked fine until the BW was 10^7 or greater 10G is 10^7 in kbps, so the scaled value becomes 10^7/10^7 or a value of 1 If the interface bandwidth is > 10G, the value becomes 0! 10^7/11^7 is < 1 therefore value becomes 0 (Integer math) While the code is smart enough to discount a BW value of 0 for the calculation, it can still cause invalid routing decisions 147

148 Problems with Links > 1G Scaled Bandwidth Problem EIGRP was created in the early 1990s and was an enhancement to IGRP, which dated back to the 1980s In order to scale the metrics larger for EIGRP, the IGRP was increased from 24 to 32 bits in size. This wasn t enough. Similar to IGRP, we scaled the Bandwidth value in order to ease transport and comparisons. By making the stored bandwidth value 10^7/BW, larger bandwidth values creating a smaller number, making comparisons easier. (Bigger bandwidth = smaller value/better metric) Of course, when you go beyond 10^7 as a bandwidth value, the formula breaks down. 10G is 10^7 (since it s measured in kbps). Links above 10G weren t envisioned in the early 1990s and that lack of foresight cause us to have to deal with it now. 148

149 Problems with Links > 1G Scaled Bandwidth Problem C6K-MCORE-1#show interface port-channel 12 Port-channel12 is up, line protocol is up (connected) MTU 9216 bytes, BW Kbit, DLY 10 usec, C6K-MCORE-1#show ip eigrp topology /30 EIGRP-IPv4 Topology Entry for AS(1)/ID( ) for /30 State is Passive, Query origin flag is 1, 1 Successor(s), FD is 256 Descriptor Blocks: (Port-channel12), from Connected, Send flag is 0x0 Composite metric is (256/0), route is Internal Vector metric: Minimum bandwidth is 0 Kbit Total delay is 10 microseconds 149

150 Problems with Links > 1G Delay Granularity Problem Interface delay values are defined by the core code EIGRP just pulls in the value provided by the infrastructure Just like EIGRP with scaled BW, the core code didn t foresee such high BW links 1 Gigabit Delay is 10 microseconds 10 Gigabit Delay is 10 microseconds 20 Gigabit Delay is 10 microseconds Etc. Since EIGRP uses aggregate delay to make path selection decision, this is trouble A 1G, 10G, and 20G link all look the same value and the 20G isn t preferred 150

151 Problems with Links > 1G Delay Granularity Problem While this problem wasn t really EIGRP s fault, the cause is similar to the Bandwidth problem described above. The Interface Delay values were determined back in the 1980s and the range of Delay values in microseconds became too small too quickly. Since a 1G link had a delay value of 10 microseconds and the value is specified in 10s of microseconds, you can t get any smaller delay values on the interface. Since EIGRP uses this Delay value in the metric calculations, we were unable to tell a 1G link from a 2G link from a 10G link, etc. Not our fault, but our problem to fix! 151

152 Problems with Links > 1G Path Selection Due to the problem with Scaled BW and Delay granularity, path selection is negatively impacted Links could be seen as equal cost when they re not Higher BW links are not preferred over lower BW links While this will not create routing loops, sub-optimal routing could easily occur 152

153 Problems with Links > 1G Path Selection The lack of Delay granularity and the inability of EIGRP to store and compare BW values > 10G combined to cause EIGRP to make invalid or suboptimal routing decisions when links above 1G are used. Routing loops will not occur since all routers make the same (wrong) decision, but you can t use the higher speed links in your network in the way you want. There s a reason you put in a 10G link (or higher) and we should pay attention to that! Unfortunately, the standard formula, interface delay values, and transport wouldn t allow us to fix the problem using the old metric components. 153

154 Problems with Links > 1G Path Selection RtrD sees two equal cost paths to reach /24 One path definitely better, but not preferred Underutilize 10G path? Overrun 1G path? /24 A 10G 1G B C 10G 1G D 154

155 Problems with Links > 1G Path Selection In the preceding diagram, EIGRP on RtrD will be unable to distinguish the difference in metric for the path to /24 through RtrB and RtrD. Since the 10G link will have the same scaled bandwidth value as the 1G link, and both will have a delay value of 10 msecs, both paths are seen as being the same. This is either inefficient or terrible. EIGRP on RtrD will attempt to install equal cost paths through RtrB and RtrC, using both equally in the forwarding plane. Since the path through RtrC cannot handle nearly as much traffic as the path through RtrD, it s likely either the path through RtrC will be overwhelmed or the path through RtrB will be underutilized. 155

156 Problems with Links > 1G Path Selection An even worse example: RtrD sees one best path to reach /24 through RtrC The path taken is definitely worse than the other, but EIGRP is not aware and thus makes the wrong routing decision /24 A 10G 1G B C 10G E 1G 10G D 156

157 Problems with Links > 1G The Solution In the past, the only solution to this problem was to manually set the delay values so that you could impact the path selection to make sense in your network This wasn t a good answer, mainly because it was administratively burdensome and hard to design/implement We also saw cases where customers built in routing loops by changing the delay values differently on different ends of the links! OSPF solved this a few years ago by defining a reference bandwidth that must be set the same throughout an area EIGRP doesn t have areas and we couldn t make customers upgrade all routers in the network in order to support a different metric method How can we solve this elegantly? 157

158 Problems with Links > 1G The Solution EIGRP has implemented a feature called Wide Metric support, which no longer scales the BW and dynamically creates the delay in picoseconds This feature is backward compatible (though mixed environments can have suboptimal routing) Implemented in EIGRP Release 8.0 and later NOTE: only available when configured in Named Mode! On by default when configured in Named Mode 158

159 Problems with Links > 1G The Solution Router# show eigrp address-family ipv4 topology EIGRP-IPv4 VR(WideMetric) Topology Entry for AS(4453)/ID( ) for /16 State is Passive, Query origin flag is 1, 1 Successor(s), FD is , RIB is 2048 Descriptor Blocks: (Ethernet0/2), from , Send flag is 0x0 Composite metric is (262144/196608), route is Internal Vector metric: Minimum bandwidth is Kbit Total delay is picoseconds Reliability is 255/255 Load is 1/255 Minimum MTU is 1500 Hop count is 2 Originating router is

160 Problems with Links > 1G Remaining Issues Possible Routing Loop with IOS and IOS/XR PEs Granularity Problem for Offset-lists between Classic and Wide Metrics 160

161 Problems with Links > 1G Possible Routing Loop with IOS and IOS/XR PEs PE - Provider Edge router CE - Customer Edge router VPN - Virtual Private Network VRF - Virtual Routing and Forwarding instance (routing table) Backdoor link - link between sites which doesn t use the VPN EIGRP Site 1 Backdoor link Service Provider VPN PE PE CE CE EIGRP Site 2 161

162 Problems with Links > 1G Possible Routing Loop with IOS and IOS/XR PEs In a MPLS/VPN PE/CE environment with both IOS and IOS/XR routers running wide metrics, IOS/XR evaluates the cost community attribute of the prefixes differently than IOS This can cause the IOS and IOS/XR PE to make different decisions about the exit point for a prefix, potentially leading to a routing loop The workaround is to use 32 bit metrics on the IOS/XR router Test metric-type 32-bit before CSCul96943 Metric-type 32-bit after CSCul

163 Problems with Links > 1G Possible Routing Loop with IOS and IOS/XR PEs One PE is running an IOS/XR version with Wide Metric support and another is running IOS with Wide Metric support Service Provider There is also a backdoor link between the two EIGRP sites IOS PE1 PE2 IOS/XR PE1 could prefer PE2 and PE2 could prefer PE1 CE EIGRP Site 1 This can cause a routing loop! Define metric-type 32-bit on IOS/XR! Backdoor link CE EIGRP Site 2

164 Problems with Links > 1G Granularity Problem for Offset-lists between Classic and Wide Metrics Wide metrics and Classic metrics use different scales for delay value Classic 10s of microseconds Wide picoseconds Offset-lists use changes in delay to offset the metric If the change is made on a wide-metric router and then sent to a classic router, granularity is lost and a different value than expected may be received Workaround verify changes to metric after implementing offset-list and adjust as needed! Note that this could impact match statements if you re looking for a specific value! 164

165 Problems with Links > 1G Granularity Problem for Offset-lists between Classic and Wide Metrics Before: R2#show ip eigrp topology /24 EIGRP-IPv4 Topology Entry for AS(100)/ID( ) for /24 State is Passive, Query origin flag is 1, 1 Successor(s), FD is Descriptor Blocks: (Ethernet0/0), from , Send flag is 0x0 Composite metric is (307200/281600), route is Internal /24 Release 12 R1 After: R2#show ip eigrp topology /24 EIGRP-IPv4 Topology Entry for AS(100)/ID( ) for /24 State is Passive, Query origin flag is 1, 1 Successor(s), FD is Descriptor Blocks: (Ethernet0/0), from , Send flag is 0x0 Composite metric is (307968/282368), route is Internal R1#show run sec router eigrp router eigrp 100 network offset-list 10 out 1000 R1#show run sec access-list access-list 10 permit any Release 6 R2 165

166 IPv6 Unique Issues

167 IPv6 Unique Issues IPv6 Router-ID IPv6 Interfaces IPv6 Peer Addresses IPv6 Shutdown 167

168 IPv6 Router-id EIGRP IPv6 topology table entries include the router-id of the originating router, just like IPv4 In previous versions, external routes only Now true for Internal routes, as well The router-id used by IPv6 is a four-byte number Actually uses IPv4 address, just like IPv4! Why use an IPv4 address for EIGRP IPv6 router-id? Seemed overkill to use 128-bit number for router-id Routers may not have a global IPv6 address defined 168

169 IPv6 Router-id EIGRP IPv6 topology table entries have router-id fields, just like their IPv4 equivalents; this router-id is used to identify the place in the network that the prefix was originated to help eliminate routing/redistribution loops When designing the EIGRP IPv6 implementation, we discussed whether to use one of the IPv6 addresses for the router-id or whether to use an IPv4 address as we did for IPv4 EIGRP Why use an IPv4 address for the router-id? First, the router-id is only used as a label to identify where a prefix originated an IPv4 address is 32 bits and an IPv6 address is 128 bits; to sacrifice 128 bits for the router-id for every prefix would significantly decrease the number of prefixes we could fit into an Update packet, all for a field that is effectively a label Additionally, IPv6 has the interesting characteristic that a router isn t required to have any globally reachable addresses; since a router could contain only link-local addresses, the usefulness of an IPv6 address as a router-id could be minimal it may or may not give you any indication of where the originating router exists in the network 169

170 IPv6 Router-id EIGRP IPv6 will not work without a router-id EIGRP uses the same router-id selection process used by IPV4 Highest IPv4 address on a Loopback interface If no Loopbacks, highest IPv4 address on non-loopback interface If no IPv4 address is available to use, manually set the router-id under the ipv6 router eigrp x configuration eigrp router-id Note that in some older versions, the leading eigrp in the command above wasn t required. 170

171 IPv6 Router-id IPv6 uses exactly the same router-id selection criteria as IPv4 If there are one or more loopback interfaces, use the highest IP address on a loopback interface If no loopback interfaces, use the highest IP address on whatever interfaces you have Interesting limitations exist, however The interfaces must exist in the same table (IPv4 version) as the IPv6 instance; in other words, if the interfaces belong to a VRF (Virtual Routing and Forwarding table), then they aren t eligible to be used as router-ids for IPv6 Note that once a router-id is selected, it isn t changed to reflect changes in addresses; in other words, if a no ip address is issued for the address used as a router-id, it won t change the router-id used until the next reload If there aren t any interfaces available with IPv4 addresses, manually specify the address using the eigrp router-id x.x.x.x command Some versions don t use the leading eigrp we haven t been terribly consistent with this, but we will be from now on really 171

172 IPv6 Interfaces EIGRP does not use a network statement to specify its IPv6 interfaces Interfaces may not have a globally routable address (may have only link-local) Need some other way to identify which should run EIGRP Two different methods exist, depending on the configuration method Classic mode uses the ipv6 eigrp <AS> command on each interface Named mode defaults to having all interfaces enabled Will need to shutdown under af-interface in order to not include some interfaces 172

173 IPv6 Interfaces Classic Mode R2#conf t Enter configuration commands, one per line. End with CNTL/Z. R2(config)#ipv6 router eigrp 1 R2(config-rtr)#interface s4/0 R2(config-if)#ipv6 eigrp 1 R2#sh run interface s4/0 Interface Serial4/0 ipv6 address 1:2::2/64 ipv6 eigrp 1 end R2#sh run section ipv6 router ipv6 router eigrp 1 R2#sh ipv6 eigrp topology P 1:2::/64, 1 successors, FD is via Connected, Serial4/0 Named mode R1# conf t Enter configuration commands, one per line. End with CNTL/Z. R1(config)#router eigrp foo R1(config-router)#address-family ipv6 unicast auto 1 R1#sh run section router eigrp router eigrp foo! address-family ipv6 unicast autonomous-system 1! topology base exit-af-topology exit-address-family R1#sh eigrp address-family ipv6 topology P 1:1::/64, 1 successors, FD is via Connected, Serial4/0 173

174 IPv6 Peer Addresses EIGRP sends hellos sourced from the link-local interface address Note that IPv6 interfaces are not required to have globally routable addresses Many routers will NOT have a global address on an interface that is facing other routers! A couple of interesting differences due to this use of link-local addresses Address in neighbor tables are only useful if you know which routers are reachable on each interface already Since a router can assign the same link-local address to multiple interfaces, you may see multiple peers with the same address if you peer across multiple links! 174

175 IPv6 Peer Addresses Output of show ipv6 eigrp neighbors with neighbor seen through multiple interfaces: EIGRP-IPv6 Neighbors for AS(1) H Address Interface Hold Uptime SRTT RTO Q Seq (sec) (ms) Cnt Num 3 Link-local address: Gi1/ :00: FE80::216:9CFF:FE6E:2C40 2 Link-local address: Gi1/ :00: FE80::216:9CFF:FE6E:2C40 1 Link-local address: Gi1/ :00: FE80::216:9CFF:FE6E:2C40 0 Link-local address: Gi1/ :00: FE80::216:9CFF:FE6E:2C40 175

176 IPv6 Peer Addresses EIGRP IPv6 sends hellos (and actually all EIGRP packets) with the source address of the link-local address on the interface Note for those not familiar: IPv6 has two primary address classes Link-local - not routable and significant only on the link (broadcast domain) on which they exist Global - advertised and used for remote reachability Since a peer relationship is limited to the link the peers see each other on, routers use the link-local address for communication An interesting aspect of this is that some platforms will assign the same link-local address to multiple interfaces This works just great since the address is only meaningful on the local link This does cause the above behavior, however note that all four peers seen on this router have exactly the same address! Not a problem, but probably unexpected 176

177 IPv6 Shutdown When EIGRP was originally coded, the router process was defined to be Shutdown by default This was done because of the lack of network statement and the fact that interface commands could start EIGRP before filtering was defined This default behavior has confused users and testers so we ve changed it in the latest code Just be warned that the default behavior for shutdown in EIGRP IPv6 is different in different versions! 177

178 IPv6 Shutdown Many, many years ago (in an IOS far, far away) we made the decision to have EIGRP IPv6 start off shutdown by default; since there are no network statements in EIGRP IPv6, as soon as an interface was defined to use EIGRP IPv6, processes would start, peers would form, and routes would be exchanged Since all of this could happen before filtering was put into place, we decided it was safer to require the user to explicitly do no shut under the ipv6 router statement to kick off the processing We ve since come to regret this decision since it s so drastically different than IPv4 EIGRP behavior in recent code, we ve changed the default to not require an explicit shutdown; be warned of our inconsistency 178

179 IPv6 and VRF Support EIGRP IPv6 does not support VRFs in classic configuration mode There is a new mode of defining EIGRP you ll be hearing a lot more about in the future that is being use for all advanced features Named mode doesn t include the AS number on the router line and uses address-family commands to create EIGRP processes Also with Named mode configuration, all EIGRP interface commands are performed under the router command rather than on the interfaces themselves This change was made because of the way processes started in the old configuration method and we required more flexibility in when information was provided 179

180 IPv6 and VRF Support Router#sh run sec router eigrp router eigrp foo! address-family ipv6 unicast autonomous-system 1! topology base exit-af-topology exit-address-family address-family ipv4 unicast autonomous-system 2! topology base exit-af-topology network exit-address-family 180

181 IPv6 and VRF Support This slide is a preview of coming attractions in addition to an explanation of how to define EIGRP IPv6 on a VRF. For the last several years, advanced features have been coded to use named mode configuration rather than classic We ran into limitations trying to put many features in classic mode because of the assumptions made when the router command was issued. We decided then to start a new mode for advanced feature but not take away classic EIGRP configuration for the more normal or simple implementations Making things more complicated when you wanted to do the same things you ve been doing for years made no sense to us. Using a new mode for more complicated features seemed more reasonable We hope you agree! 181

182 Troubleshooting Tools

183 EIGRP Troubleshooting Tools Debugs vs. the EIGRP Event Log On a busy, unstable network debugs can be hazardous to your network s health Event log is non-disruptive it s already running 183

184 EIGRP Troubleshooting Tools The two primary weapons at your disposal are debugs and the event log; realize that the output of both debugs and the event log are cryptic and probably not tremendously useful to you (so why am I telling you about them?) There are times when the output of debugs or the event log is enough to lead you in a direction, even if you don t really understand all that it is telling you; don t expect to be an expert at EIGRP through the use of debugs or the event log, but they can help Don t forget, debugs can kill your router don t do a debug if you don t know how heavy the overhead is; I may tell you below about some debugs, but don t consider this approval from Cisco to run them on your production network The event log is non-disruptive, so it is much safer; just display it and see what s been happening lately 184

185 Event Log Always running (unless manually disabled) Defaults to 500 lines (configurable) EIGRP event-log-size <number of lines> Maximum event-log-size is half of available memory Most recent events at top of log by default Read from the bottom to top Later version support displaying the event log in reverse! 185

186 Event Log A separate event log is kept for each AS 500 lines are not very much; on a network where there is significant instability or activity, 500 lines may only be a second or two (or less) you can change the size of the event log (if needed) by the command eigrp event-log-size <number of lines> Recent IOS limits to half of available memory If number of lines set to 0, it disables the log You can clear the event log by typing clear ip eigrp event Most recent events are at the top of the log by default, so time flows from bottom to top 186

187 Event Log New parameters available for showing the event log Rtr2#show ip eigrp event? < > Starting event number errmsg Show Events being logged reverse Show most recent event last sia Show Events being logged type Show Events being logged Output modifiers <cr> 187

188 Event Log Three different event types can be logged EIGRP log-event-type [dual][xmit][transport] Default is dual normally most useful Dual is FSM (decisions in finite state machine) xmit and transport are different aspects of actually sending packets to peers Any combination of the three can be on at the same time Work is in progress to add additional debug information to event log 188

189 Event Log RtrA#show ip eigrp events Event information for AS 1: 1 01:52: NDB delete: / :52: RDB delete: / :52: Metric set: / :52: Poison squashed: /24 lost if 5 01:52: Poison squashed: /24 metric chg 6 01:52: Send reply: / :52: Not active net/1=sh: / :52: FC not sat Dmin/met: :52: Find FS: / :52: Rcv query met/succ met: :52: Rcv query dest/nh: / :52: Change queue emptied, entries: :52: Metric set: /

190 Debugs Remember debugs can be dangerous! Use only in the lab or if advised by the TAC To make a little safer Logging buffered <size> No logging console 190

191 Debugs By enabling logging buffered and shutting off the console log, you improve your odds of not killing your router when you do a debug; still no guarantees We often change the scheduler interval when we do debugs in the TAC, as well; this command is version dependent, so I m not going to give you syntax here 191

192 Debugs Use modifiers to limit the scope of route events or packet debugs Limit debugs to a particular neighbor debug ip eigrp neighbor AS address Limit debugs to a particular prefix debug ip eigrp AS network mask 192

193 Debugs Both packet debugs and route event debugs create so much output that you would have a hard time sorting through it for the pieces you care about; the two modifier commands above allow you to limit what the debug output will include Debug ip eigrp neighbor AS address will limit the output to those entries pertaining to a particular neighbor Debug ip eigrp AS network mask will limit the output to only those entries that pertain to the prefix identified Unfortunately, you have to enable the debug (packet or route events) prior to putting the modifier on, so you could kill your router before you are able to get the limits placed on the output; sorry, but that s the way it is 193

194 Debugs Debug IP EIGRP (Route Events) neighbor filtering RtrA#debug ip eigrp IP-EIGRP Route Events debugging is on RtrA#debug ip eigrp neighbor IP Neighbor target enabled on AS 1 for IP-EIGRP Neighbor Target Events debugging is on RtrA#clear ip eigrp neighbor P-EIGRP: /24 - do advertise out Serial1/2 IP-EIGRP: Int /24 metric IP-EIGRP: /24 - do advertise out Serial1/2 IP-EIGRP: /24 - do advertise out Serial1/2 IP-EIGRP: Int /24 metric IP-EIGRP: Processing incoming UPDATE packet IP-EIGRP: /24 - do advertise out Serial1/1 194

195 Debugs Debug IP EIGRP (Route Events) neighbor filtering In this debug, we are looking at route events recorded when neighbors are cleared (in reality, the debugs produced were far, far more this is only a snapshot of the debug); a modifier was included to limit the output to only the events related to a single EIGRP neighbor, Notice that the debug output doesn t identify which neighbors are involved in any of the events; without knowing the address used in the modifier command, you really can t tell which neighbors you are interacting with in the debug output This output is often useful when trying to determine what EIGRP thinks is happening when there are route changes in the network 195

196 Debugs Debug IP EIGRP (Route Events) prefix filtering RtrA#debug ip eigrp IP-EIGRP Route Events debugging is on RtrA#debug ip eigrp IP Target enabled on AS 1 for /24 IP-EIGRP AS Target Events debugging is on RtrA#clear ip eigrp neighbor IP-EIGRP: /24 - do advertise out Serial1/2 IP-EIGRP: /24 - do advertise out Serial1/1 IP-EIGRP: Int /24 metric IP-EIGRP: /24 - do advertise out Serial1/2 IP-EIGRP: Processing incoming UPDATE packet IP-EIGRP: /24 - do advertise out Serial1/1 196

197 Debugs Debug IP EIGRP (Route Events) prefix filtering Again, this is the output of debugging EIGRP routing events, this time modified to only display output related to a single prefix in the network; this modifier can be very useful when trying to troubleshoot a problem with a single prefix (or representative route) 197

198 Debugs Debug EIGRP Packet RtrA#debug eigrp packet? ack hello ipxsap probe query reply request stub retry terse update verbose EIGRP ack packets EIGRP hello packets EIGRP ipxsap packets EIGRP probe packets EIGRP query packets EIGRP reply packets EIGRP request packets EIGRP stub packets EIGRP retransmissions Display all EIGRP packets except Hellos EIGRP update packets Display all EIGRP packet 198

199 Debugs Debug EIGRP Packet This debug is used in a variety of problems and circumstances; debug eigrp packet hello is used to troubleshoot neighbor establishment/maintenance problems Debug eigrp packet query, reply, update, etc., are also often used to try to determine the process occurring when a problem occurs be careful; I ve crashed/hung more than one router by doing a debug on a router that was too busy Probably the most commonly used debug eigrp packet option is terse, which includes all of the above except hellos; an example follows on the next page 199

200 Debugs Debug EIGRP Packet Terse RtrA#debug eigrp packet terse EIGRP Packets debugging is on (UPDATE, REQUEST, QUERY, REPLY, IPXSAP, PROBE, ACK, STUB) EIGRP: Sending UPDATE on Serial1/0 nbr AS 1, Flags 0x0, Seq 2831/1329 idbq 0/0 iidbq un/rely 0/0 peerq un/rely 0/1 serno EIGRP: Sending UPDATE on Serial1/1 nbr AS 1, Flags 0x0, Seq 2832/1708 idbq 0/0 iidbq un/rely 0/0 peerq un/rely 0/1 serno EIGRP: Sending UPDATE on Serial1/2 nbr AS 1, Flags 0x0, Seq 2833/1680 idbq 0/0 iidbq un/rely 0/0 peerq un/rely 0/1 serno EIGRP: Received ACK on Serial1/0 nbr AS 1, Flags 0x0, Seq 0/2831 idbq 0/0 iidbq un/rly 0/0 peerq un/rely 0/1 EIGRP: Serial1/0 multicast flow blocking cleared EIGRP: Received ACK on Serial1/1 nbr AS 1, Flags 0x0, Seq 0/2832 idbq 0/0 iidbq un/rely 0/0 peerq un/rely 0/1 EIGRP: Serial1/1 multicast flow blocking cleared 200

201 Debugs Debug IP EIGRP Notifications RtrA#debug ip eigrp notifications IP-EIGRP Event notification debugging is on RtrA#clear ip route * RtrA# IP-EIGRP: Callback: reload_iptable IP-EIGRP: iptable_redistribute into eigrp AS 1 IP-EIGRP: Callback: redist frm static AS /24 into: eigrp AS 1 event: 1 IP-EIGRP: Callback: redist frm static AS /24 into: eigrp AS 1 event: 1 201

202 Debugs Debug IP EIGRP Notifications This debug is used to debug problems between EIGRP and the routing/interface infrastructure; for example, if you re having problems with redistribution, the actual place where EIGRP gets the redistributed routes is the RIB/routing table these callbacks are the mechanism to signal changes between the routing infrastructure and EIGRP when routes come and go Callbacks are also used from the interface infrastructure as interfaces are shutdown/no shut, deleted, addresses added, etc. Any problem with EIGRP s interaction with other parts of the system will probably come through this mechanism, thus this debug is your best bet 202

203 Debugs Debug EIGRP FSM (Finite State Machine) RtrA#debug eigrp fsm EIGRP FSM Events/Actions debugging is on RtrA#clear ip route * RtrA# DUAL: Find FS for dest /24. FD is 28160, RD is DUAL: metric 28160/0 found Dmin is DUAL: Find FS for dest /24. FD is , RD is DUAL: metric / found Dmin is DUAL: RT installed /24 via DUAL: Find FS for dest /24. FD is , RD is

204 Debugs Debug EIGRP FSM Debug eigrp fsm is very, very similar to dual event log; since the dual event log is nondisruptive and this debug could certainly cause problems, I rarely use this debug in real life sometimes it s useful in conjunction with another debug to see how the different parts of EIGRP interact (debug ip packet, debug eigrp transport, and debug eigrp fsm to see how all the pieces fit together) FSM stands for Finite State Machine, which describes the behavior of DUAL, the path selection part of EIGRP 204

205 Topology Table The topology table is probably the most critical structure in EIGRP Contains building blocks used by DUAL Used to create updates for neighbors Used to populate the routing table Understanding the topology table contents is extremely important in troubleshooting EIGRP problems 205

206 Topology Table One of the reasons that EIGRP is called an advanced distance vector protocol is that it retains more information than just the best path for each route it receives this means that it can potentially make decisions more quickly when changes occur, because it has a more complete view of the network than RIP, for example; the place this additional information is stored is in the topology table The topology table contains an entry for every route EIGRP is aware of, and includes information about the paths through all neighbors that have reported this route to him when a route is withdrawn by a neighbor, EIGRP will look in the topology table to see if there is a feasible successor, which is another downstream neighbor that is guaranteed to be loop-free; if so, EIGRP will use that neighbor and never have to go looking farther Contrary to popular belief, the topology table also contains routes which are not feasible; these are called possible successors and may be promoted to feasible successors, or even successors if the topology of the network were to change The following slides show a few different ways to look at the topology table and give hints on how to evaluate it 206

207 Topology Table Show IP EIGRP Topology Summary Total number of routes in the local topology table Number of Replies to send from this router Internal data structures used to manage the topology table RtrA#sh ip eigrp topology sum IP-EIGRP Topology Table for AS(200)/ID( ) Head serial 1, next serial routes, 0 pending replies, 0 dummies IP-EIGRP(0) enabled on 12 interfaces, neighbors present on 4 interfaces Quiescent interfaces: Po3 Po6 Po2 Gi8/5 Interfaces with No Outstanding Packets to Be Sent or Acknowledged 207

208 Topology Table Show IP EIGRP Topology Displays a list of successors and feasible successors for all destinations known by EIGRP A B.1 56K K /24 C / E /24 RtrA#show ip eigrp topology IP-EIGRP Topology Table for AS(1)/ID( )..snip.. P /24, 1 successors, FD is via ( / ), Serial1/0 via ( / ), Serial1/1.2 D.1 Feasible distance Successor Feasible successor Computed distance Reported distance 208

209 Topology Table Show IP EIGRP Topology The most common way to look at the topology table is with the generic show ip eigrp topology command; this command displays all of the routes in the EIGRP topology table, along with their successors and feasible successors In the above example, the P on the left side of the topology entry displayed means the route is Passive if it has an A, it means the route is Active; the destination being described by this topology entry is for this route has one successor, and the feasible distance is ; the feasible distance is normally the metric that would appear in the routing table if you did the command show ip route (but not always) Following the information on the destination network, the successors and feasible successors are listed the successors (one or more) are listed first, then the feasible successors are listed; the entry for each next-hop includes the IP address, the computed distance through this neighbor, the reported distance this neighbor told us, and which interface is used to reach him is a feasible successor because his reported distance ( ) is less than our current feasible distance ( ) 209

210 Topology Table Show IP EIGRP Topology All-links Displays a list of all neighbors who are providing EIGRP with an alternative path to each destination A B.1 56K K /24 C / E /24 RtrA#show ip eigrp topology all-links IP-EIGRP Topology Table for AS(1)/ID( )..snip.. P /24, 1 successors, FD is via ( / ), Serial1/0 via ( / ), Serial1/1 via ( / ), Serial1/2.2 D.1 Feasible distance Successor Feasible successor Possible successor Computed distance Reported distance 210

211 Topology Table Show IP EIGRP Topology All-links If you want to display all of the paths which EIGRP contains in its topology table, use the show ip eigrp topology all-links command You ll notice in the above output that not only are the successor ( ) and feasible successor ( ) shown, but another router that doesn t qualify as either is also displayed; the reported distance from ( ) is far worse than the current feasible distance ( ), so it isn t feasible This command is often useful to understand the true complexity of network convergence I ve been on networks with pages of non-feasible alternative paths in the topology table because of a lack of summarization/distribution lists; these large numbers of alternative paths can cause EIGRP to work extremely hard when transitions occur and can actually keep EIGRP from successfully converging 211

212 Topology Table Show IP EIGRP Topology <net><mask> Displays detailed information for all paths received for a particular destination RtrA#show ip eigrp topology /24 IP-EIGRP topology entry for /24 State is Passive, Query origin flag is 1, 1 Successor(s), FD is Routing Descriptor Blocks: (Serial1/0), from , Send flag is 0x0 Composite metric is ( / ), Route is Internal Vector metric: (Serial1/1), from , Send flag is 0x0 Composite metric is ( / ), Route is Internal Vector metric: (Serial1/2), from , Send flag is 0x0 Composite metric is ( / ), Route is Internal Vector metric:... A B.1 56K K /24 C /24.2 D E /24 212

213 Topology Table Show IP EIGRP Topology <network><mask> If you really want to know all of the information EIGRP stores about a particular route, use the command show ip eigrp topology <network><mask> Note that the mask can be supplied in dotted decimal or /xx form In the above display, you ll see that EIGRP not only stores which next-hops have reported a path to the target network, it stores the metric components used to reach the total (composite) metric You also may notice that EIGRP contains a hop count in the vector metrics the hop count isn t actually used in calculating the metric, but instead was included to limit the apparent maximum diameter of the network; in EIGRP s early days, developers wanted to ensure that routes wouldn t loop forever and put this safety net in place in today s EIGRP, it actually isn t necessary any longer, but is retained for compatibility 213

214 Topology Table External Topology Table Entry Showing the topology table entry for an external route shows additional information about the route A B.1 56K K /24 C / E /24 RtrA#show ip eigrp topology /24 IP-EIGRP topology entry for /24 State is Passive, Query origin flag is 1, 1 Successor(s), FD is Routing Descriptor Blocks: (Serial1/2), from , Send flag is 0x0... External data: Originating router is AS number of route is 0 External protocol is Static, external metric is 0 Administrator tag is 0 (0x ).2 D.1 Static Route to /24 is redistributed into EIGRP 214

215 Topology Table External Topology Table Entry If you perform the command show ip eigrp topology <network><mask> for an external route (one redistributed into EIGRP from another protocol), even more information is displayed The initial part of the display is identical to the command output for an internal (native) route the one exception is the identifier of the route as being external; another section is appended to the first part, however, containing external information the most interesting parts of the external data are the originating router and the source of the route The originating router is the router who initially redistributed the route into EIGRP note that the value for the originating router is router-id of the source router, which doesn t necessarily need to belong to an EIGRP-enabled interface; the router-id is selected in the same way OSPF selects router-ids, starting with highest IP address on a loopback interface, if any are defined, or using the highest IP address on the router if there aren t loopback interfaces note that if a router receives an external route and the originating router field is the same as the receiver s router-id, it rejects the route this is noted in the event log as ignored, dup router The originating routing protocol (where it was redistributed from) is also identified in the external data section; this is often useful when unexpected routes are received and you are hunting the source 215

216 Topology Table Show IP EIGRP Topology Zero Zero successor routes are those that fail to get installed in the routing table by EIGRP because there is a route with a better admin distance already installed A B.1 56K K /24 C / E /24 RtrA#show ip eigrp topology zero IP-EIGRP Topology Table for AS(1)/ID( )... P /24, 0 successors, FD is Inaccessible via ( / ), Serial1/0 via ( / ), Serial1/1 via ( / ), Serial1/2.2 RtrA#show ip route Routing entry for /24 Known via "static", distance 1, metric 0 Routing Descriptor Blocks: * Route metric is 0, traffic share count is 1 D.1 216

217 Topology Table Show IP EIGRP Topology Zero And last, the show ip eigrp topology zero command is available to display the topology table entries that are not actually being used by the routing table Typically, zero successor entries are ones that EIGRP attempted to install into the routing table, but found a better alternative there already; in our example above, when EIGRP tried to install its route (with an administrative distance of 90), it found a static route already there (with an administrative distance of one) and thus couldn t install it in case the better route goes away, EIGRP retains the information in the topology table, and will try to install the route again if it is notified that the static (or whatever) route is removed Routes that are active sometimes also show up as zero successor routes, but they are transient and don t remain in that state This command isn t often used or useful 217

218 Other Show Commands Show EIGRP Plugin Detail This command may or may not exist in the version you re running, since it was recently added While this command may not be all that useful to you, the TAC will probably asking you for it to identify exactly the version and capabilities of EIGRP on your router r3#sh eigrp plugin detail EIGRP feature plugins::: eigrp-release : : Portable EIGRP Release : : Source Component Release(Portable EIGRP Release(rel5_1)) igrp2 : : Reliable Transport/Dual Database external-client : : Service Distribution Client Support bfd : : BFD Platform Support ipv4-af : : Routing Protocol Support ipv4-sf : : Service Distribution Support ipv6-af : : Routing Protocol Support ipv6-sf : : Service Distribution Support snmp-agent : : SNMP/SNMPv2 Agent Support r3# 218

219 Other Show Commands Show EIGRP Plugin Detail Show eigrp plugin detail gives information about all of the EIGRP capabilities and what version each is running; EIGRP has gone to a plug-in model for packaging features (similar to Windows DLL, sort of) and a particular platform/branch may have different plugins or different version of those plugins EIGRP is also what s known as a True Component is done in a portable way, so the first line is critical to know which version of the True Component you re running 219

220 Other Show Commands Show EIGRP Tech This gives you the plugins again, with a lot of other internal info; TAC will probably be asking for this one, too r3#show eigrp tech EIGRP feature plugins::: eigrp-release : : Portable EIGRP Release : : Source Component Release(Portable EIGRP Release(rel5_1)) igrp2 : : Reliable Transport/Dual Database external-client : : Service Distribution Client Support bfd : : BFD Platform Support ipv4-af : : Routing Protocol Support ipv4-sf : : Service Distribution Support ipv6-af : : Routing Protocol Support ipv6-sf : : Service Distribution Support snmp-agent : : SNMP/SNMPv2 Agent Support EIGRP Internal Process States procinfoq: 2 deadq: ddbq: 2 220

221 Other Show Commands Show EIGRP Tech EIGRP-IPv4 Protocol for AS(1) {vrid:1 afi:1 as:1 tableid:0 vrfid:0 tid:0 name: } PIDs: Hello: 26 PDM: 25 Router-ID: Threads: procinfo: 0x18DD1A0 ddb: 0x18DD380 workq: iidbq: temp_iidbq: passive_iidbq: peerq: static_peerq: suspendq: networkq: 1 summaryq: Socket Queue: 0/2000/0/0 (current/max/highest/drops) Input Queue: 0/2000/0/0 (current/max/highest/drops) GRS/NSF: enabled hold-timer: 240 Active Timer: 3 min Distance: internal 90 external 170 Max Path: 4 Max Hopcount: 100 Variance: 1 221

222 Other Show Commands Show EIGRP Tech - EIGRP-IPv6 Protocol for AS(1) {vrid:0 afi:2 as:1 tableid:0 vrfid:0 tid:0 name: } PIDs: Hello: (no process) PDM: (no process) Router-ID: Threads: procinfo: 0x185ECA0 ddb: 0x7B45B940 workq: iidbq: temp_iidbq: passive_iidbq: peerq: static_peerq: suspendq: summaryq: Socket Queue: %EIGRP(ERROR): invalid socket Input Queue: 0/2000/0/0 (current/max/highest/drops) Active Timer: 3 min Distance: internal 90 external 170 Max Path: 16 Max Hopcount: 100 Variance: 1 r3# 222

223 Other Show Commands Show EIGRP Tech Show eigrp tech has tons of internal information that will be meaningless to you; there may be nuggets you can glean from it (summaryq entries, etc.), but mainly I wanted to expose you to it so that it won t be completely foreign to you when TAC asks you for it Again, this is a new command and may not exist in the version you re running if it s not available now, it will be available in a later upgrade Note that this is also a show eigrp tech detail with even more undecipherable info 223

224 Other Show Commands Show IP EIGRP commands changing in the future To make commands consistent across address/service families, changing the form of the Show commands EIGRP will become the second argument followed by the address-family Router#show eigrp? address-family EIGRP address-family show commands plugins EIGRP feature plugin installed protocols Show EIGRP protocol info service-family EIGRP service-family show commands tech-support Show EIGRP internal tech support information Router#show eigrp address-family? ipv4 EIGRP IPv4 Address-Family ipv6 EIGRP IPv6 Address-Family Router#show eigrp address-family ipv4? < > Autonomous System accounting Prefix Accounting events Events logged interfaces interfaces multicast Select a multicast instance neighbors Neighbors timers Timers topology Select Topology traffic Traffic Statistics vrf Select a VPN Routing/Forwarding instance 224

225 Other Show Commands Show IP EIGRP command changes We ve had a bit of an issue over the years with inconsistencies in the command structure in EIGRP. Sometimes, the IPv4 commands and the IPv6 commands were slightly different With the addition of service-family command (SAF, not covered in this presentation), we decided to make things consistent. Unfortunately, this also meant that things needed to be different Both the old and new forms of the show commands are currently supported, but in the future as new features are rolled out, some commands may only be available in the new command format You might as well start getting used to them now, if you re running a version that supports the new format! 225

226 Other Show Commands Show IP EIGRP Neighbor Detail The big brother of the show ip eigrp neighbor command shown earlier; some of the additional information available via the detailed version of this command include Number of retransmissions and retries for each neighbor Version of Cisco IOS and EIGRP Stub information (if configured) rtr302-ce1#show ip eigrp neighbor detail IP-EIGRP neighbors for process 1 H Address Interface Hold Uptime SRTT RTO Q Seq Type (sec) (ms) Cnt Num Et1/ :00: Version 12.0/1.2, Retrans: 0, Retries: 0 Stub Peer Advertising ( CONNECTED SUMMARY ) Routes Et0/ :04: Version 12.0/1.2, Retrans: 2, Retries: 0 226

227 Other Show Commands Show IP EIGRP Interface Detail RtrB#show ip eigrp interface detail IP-EIGRP interfaces for process 1 Xmit Queue Mean Pacing Time Multicast Pending Interface Peers Un/Reliable SRTT Un/Reliable Flow Timer Routes Et0/0 1 0/ / Hello interval is 5 sec Next xmit serial <none> Un/reliable mcasts: 0/3 Un/reliable ucasts: 6/3 Mcast exceptions: 0 CR packets: 0 ACKs suppressed: 0 Retransmissions sent: 0 Out-of-sequence rcvd: 0 Authentication mode is not set Et1/0 1 0/ / Hello interval is 5 sec Next xmit serial <none> Un/reliable mcasts: 0/2 Un/reliable ucasts: 5/3 Mcast exceptions: 0 CR packets: 0 ACKs suppressed: 0 Retransmissions sent: 0 Out-of-sequence rcvd: 0 Authentication mode is not set 227

228 Other Show Commands Show IP EIGRP Interface Detail There is also a show ip eigrp interface which contains a subset of this info; You may want to just use that if you don t need all the detail This command supplies a lot of information about how the interfaces are being used and how well they are obeying; some of the interesting information available via this command is Retransmissions sent this shows how many times EIGRP has had to retransmit packets on this interface, indicating that it didn t get an ack for a reliable packet; having retransmits is not terrible, but if this number is a large percentage of packets sent on this interface, something is keeping neighbors from receiving (and acking) reliable packets Out-of-sequence rcvd this shows how often packets are received out of order, which should be a relatively unusual occurrence; again, it s nothing to worry about if you get occasional out-of-order packets since the underlying delivery mechanism is best-effort if the number is a large percentage of packets sent on the interface, however, then you may want to look into what s happening on the interface are there errors? You can also use this command to see if an interface only contains stub neighbors and if authentication is enabled 228

229 Other Show Commands Show IP EIGRP Traffic RtrB#show ip eigrp traffic IP-EIGRP Traffic Statistics for AS 1 Hellos sent/received: 574/558 Updates sent/received: 5/7 Queries sent/received: 2/2 Replies sent/received: 2/2 Acks sent/received: 11/7 Input queue high water mark 2, 0 drops SIA-Queries sent/received: 1/1 SIA-Replies sent/received: 1/1 Hello Process ID: 64 PDM Process ID:

230 Other Show Commands Show IP EIGRP Traffic Show ip eigrp traffic can be very useful to see what kind of activity has been occurring on your network; Some of the most interesting information includes: Input queue high water mark this shows how many packets have been queued inside of the router to be processed when packets are received from the IP layer, EIGRP accepts the packets and queues them up for processing; if the router is so busy that the queue isn t getting serviced, the queue could build up unless there are drops, there is nothing to worry about, but it can give you an indication of how hard EIGRP is working SIA-queries sent/received this is useful to determine how often the router has stayed active for at least one and one-half minutes (as mentioned in the earlier section on stuck-in-active routes; this number should be relatively low if it s not, it s taking a bit of time for replies to be received for queries, and it might be worth exploring why 230

231 Other Show Commands Show IP Protocol RtrA#show ip protocol *** IP Routing is NSF aware *** Routing Protocol is "eigrp 200" Outgoing update filter list for all interfaces is not set Incoming update filter list for all interfaces is not set Default networks flagged in outgoing updates Default networks accepted from incoming updates EIGRP metric weight K1=1, K2=0, K3=1, K4=0, K5=0 EIGRP maximum hopcount 100 EIGRP maximum metric variance 1 Redistributing: eigrp 200 EIGRP NSF-aware route hold timer is 240s EIGRP NSF enabled NSF signal timer is 20s NSF converge timer is 120s 231

232 Other Show Commands Show IP Protocol There are many fields in the show ip protocol which are useful in the troubleshooting process Some of the most interesting include: Outgoing and incoming filter lists Variance setting Redistribution configured (note that if a router is not redistributing other protocols, it still shows that it is redistributing itself) NSF configuration and timers 232

233 Other Show Commands Show IP Protocol Automatic network summarization is not in effect Address Summarization: /8 for Vlan301 Summarizing with metric 1536 Maximum path: 4 Routing for Networks: / Routing Information Sources: Gateway Distance Last Update (this router) 90 00:01: :13: :13: :13: :13:48 Distance: internal 90 external

234 Other Show Commands Show IP Protocol More show ip protocol stuff Summarization defined (both auto and manual) along with the metric associated with each summary Max-path setting Network statements Distance settings Note that the routing information sources section is really useless for EIGRP, since it doesn t use periodic update; we didn t rip it out of the display, but there isn t much useful information for EIGRP in this section 234

235 Summary EIGRP contains many tools and techniques that can be used to keep the EIGRP network running smoothly and efficiently Hopefully this session taught you how to make the best use of these tools and techniques and you re able to translate how these will be useful in your network 235

236 Complete Your Online Session Evaluation Give us your feedback and you could win fabulous prizes. Winners announced daily. Complete your session evaluation through the Cisco Live mobile app or visit one of the interactive kiosks located throughout the convention center. Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online 236

237 Continue Your Education Demos in the Cisco Campus Walk-in Self-Paced Labs Table Topics Meet the Engineer 1:1 meetings 237

238 Recommended Reading for ASIN: ISBN ISBN: Open-EIGRP: draft-savage-eigrp