Highly Available Wide Area Network Design

Transcription

1

2 Highly Available Wide Area Network Design David Prall, Communications Architect CCIE #6508 (R&S/SP/Security) BRKRST-2042

3 Agenda Introduction Cisco IOS and IP Routing Convergence Techniques Design and Deployment Final Wrap Up

4 Goals Efficiently utilize available bandwidth Dynamically respond to all types of disruptions Leverage most effective design techniques that meet the design requirements BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 5

5 Where Can Outages Occur? Link or Device Failure MPLS - SP A C-A-R2 Link or Device Degraded C-A-R1 C-A-R4 C-A-R3 HQ-W1 BR-W1 HQ-W2 MPLS - SP B BR-W2 C-B-R1 C-B-R4 How does outage manifest? How quickly can network detect? How long is bidirectional reconvergence? BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 6

6 Session Scope What methods are used for path selection and packet forwarding How does the network detect outages Focus on network survivability and effective utilization rather than sub-second convergence Does not address zero loss considerations BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 7

7 Agenda Introduction Cisco IOS and IP Routing Multiple Links/Multiple Paths Load Sharing Convergence Techniques Design and Deployment Final Wrap Up

8 Routing Table Basics Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP a - application route + - replicated route, % - next hop override, p - overrides from PfR p /8 is variably subnetted, 14 subnets, 5 masks B p /8 [20/0] via , 00:12:36 B p /16 [20/0] via , 00:12:36 B p /16 [200/0], 00:13:52, Null0 C p /32 is directly connected, Loopback0 D p /24 [90/307200] via , 00:14:32, Ethernet0/0 C p /30 is directly connected, Ethernet0/0 L p /32 is directly connected, Ethernet0/0 B p /16 [20/0] via , 00:12: /8 is variably subnetted, 9 subnets, 2 masks B /24 [20/0] via , 00:13:43 C /24 is directly connected, Ethernet0/2 L /32 is directly connected, Ethernet0/ /16 is variably subnetted, 9 subnets, 2 masks B /31 [20/0] via , 00:12:36 C /31 is directly connected, Ethernet0/1 L /32 is directly connected, Ethernet0/1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 9

9 Administrative Distance The distance command is used to configure a rating of the trustworthiness of a routing information source, such as an individual router or a group of routers Numerically, an administrative distance is a positive integer from 1 to 255. In general, the higher the value, the lower the trust rating An administrative distance of 255 means the routing information source cannot be trusted at all and should be ignored INFORMATIONAL Default Route Source Distance Connected Interface 0 Static Route 1 EIGRP Summary Route 5 BGP External (ebgp) 20 EIGRP Internal 90 OSPF 110 IS-IS 115 RIP 120 EIGRP External 170 BGP Internal (ibgp) 200 Unknown 255 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 10

10 Route Selection How is administrative distance used to determine which route should be installed? Only identical routes are compared Identical prefixes with different prefix lengths are not the same route The route from the protocol with the lower administrative distance is installed OSPF /24 EIGRP OSPF / /25 These Two Routes Are Identical EIGRP Internal = 90 OSPF = 110 EIGRP Internal Installed router#show ip route Routing entry for /24 Known via "eigrp 1", distance 90, metric , type internal Redistributing via eigrp 1 Last update from on Ethernet0/1, 00:01:32 ago Routing Descriptor Blocks: * , from , 00:01:32 ago, via Ethernet0/1 Route metric is , traffic share count is 1 Total delay is 2000 microseconds, minimum bandwidth is Kbit Reliability 255/255, minimum MTU 1500 bytes Loading 1/255, Hops 1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 11

11 Route Selection What about longest prefix comparison? Only identical routes are compared Identical prefixes with different prefix lengths are not the same route The route with the longest prefix is installed OSPF /24 EIGRP OSPF / /25 These Two Routes Are Identical OSPF Installed Longer Prefix router#show ip route longer-prefixes /8 is variably subnetted, 9 subnets, 3 masks D /24 [90/307200] via , 00:01:35, Ethernet0/1 O /25 [110/20] via , 00:00:50, Ethernet0/2 O /25 [110/20] via , 00:00:50, Ethernet0/2 More Specific OSPF Override EIGRP BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 12

12 Agenda Introduction Cisco IOS and IP Routing Multiple Links/Multiple Paths Load Sharing Convergence Techniques Design and Deployment Final Wrap Up

13 Load Sharing Assume the same routing process attempts to install two routes for the same destination in the RIB The routing process may allow the second route to be installed based on its own rules IGP OSPF IS-IS EIGRP Route Cost Must be equal to installed route Must be equal to installed route Must be less than the variance times the lowest cost installed route Maximum Paths Must be fewer than maximum-paths configured under the routing process (default = 4, maximum = 32) Note: BGP default value for maximum-paths = 1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 14

14 CEF Load Sharing Per-Session Default behaviour of IOS Per-flow using source/destination Packets for a given source/ destination session will take the same path More effective as the number of source to destination pairs increase Ensures that traffic for a given session arrives in order Per-Packet Requires ip load-sharing per-packet interface configuration Per-packet using round-robin method Packets for a given source/ destination session may take different paths Ensures traffic is more evenly distributed over multiple paths Potential for packets to arrive out of sequence BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 15

15 Load Sharing router#show ip route Routing entry for /24 Known via "eigrp 100", distance 170, metric , type external Redistributing via eigrp 100 Last update from on Serial0/2/1, 00:18:17 ago Routing Descriptor Blocks: * , from , 00:18:17 ago, via Serial2/0 Route metric is , traffic share count is , from , 00:18:17 ago, via Serial2/1 Route metric is , traffic share count is / = 1 The Traffic Share Count Is Critical to Understanding the Actual Load Sharing of Packets Using These Two Routes BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 16

16 Load Sharing with EIGRP Variance router#show ip route Routing entry for /24 Known via "eigrp 100", distance 170, metric , type external Redistributing via eigrp 100 Last update from on Serial0/2/1, 00:18:17 ago Routing Descriptor Blocks: * , from , 00:18:17 ago, via Serial2/0 Route metric is , traffic share count is , from , 00:18:17 ago, via Serial2/1 Route metric is , traffic share count is 1... If the Lower Metric Is Less than the Second Metric, the Traffic Share Count Will Be Something Other than 1 (EIGRP with Variance Configured) / = / = 2 2x Faster Link Gets 2 Flows vs. 1 Flow BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 17

17 Load Sharing with ebgp dmzlink-bw router#show ip route Routing entry for /24 Known via "bgp 1", distance 20, metric 0 Tag 2, type external Last update from :00:16 ago Routing Descriptor Blocks: , from , 00:00:16 ago Route metric is 0, traffic share count is 1... * , from , 00:00:16 ago Route metric is 0, traffic share count is 2... router#show ip bgp BGP routing table entry for /24, version 9 Paths: (2 available, best #2, table default) Multipath: ebgp from ( ) 2x Faster Link Gets 2 Flows vs. 1 Flow Origin IGP, metric 0, localpref 100, valid, external, multipath(oldest) DMZ-Link Bw 312 kbytes rx pathid: 0, tx pathid: from ( ) Origin IGP, metric 0, localpref 100, valid, external, multipath, best DMZ-Link Bw 625 kbytes rx pathid: 0, tx pathid: 0x0 Only routes learned via ebgp Neighbors BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 18

18 Agenda Introduction Cisco IOS and IP Routing Convergence Techniques First Hop Redundancy Protocols Routing Protocols DDR and Static Routing Performance Routing Design and Deployment Final Wrap Up

19 First Hop Redundancy Protocols (FHRP) Failure Protection for the First Hop IP Router BR-W1 BR-W2 Hot Standby Router Protocol (HSRP) Virtual Router Redundancy Protocol (VRRP) Gateway Load Balancing Protocol (GLBP) BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 20

20 Drivers for FHRPs Provide routing redundancy for access layer How to handle failover when end-hosts have only a single IP default gateway and cached ARP entry Provide routing redundancy for devices that depend on static routing Some firewalls do not support dynamic routing Independent of routing protocols Works with any routing protocol and static routing Capable of providing sub-second failover Provides load sharing capabilities (GLBP) transparent to end host BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 21

21 Hot Standby Routing Protocol (HSRP) interface FastEthernet0/0 ip address standby version 2 standby 4 ip standby 4 priority 110 standby 4 preempt standby 6 ipv6 autoconfig standby 6 priority 110 standby 6 preempt ipv6 address 2001:DB8:5:1::1/64 Active Standby Router Router BR-W1 BR-W2 HSRP (.2) (.3) VIP (.1) Default Gateway: (.1) DG MAC: MAC VIP BR-W1#show standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Fa0/ P Active local Fa0/ P Active local FE80::A8BB:CCFF:FE00:3400 FE80::5:73FF:FEA0 :6 HSRP Global IPv6 Addresses Available for Static Deployments interface FastEthernet0/0 ip address standby version 2 standby 4 ip standby 4 preempt standby 6 ipv6 autoconfig standby 6 preempt ipv6 address 2001:DB8:5:1::2/64 BR-W2#show standby brief Interface Grp Pri P State Active Standby Virtual IP Fa0/ P Standby local Fa0/ P Standby FE80::A8BB:CCFF:FE00:3300 local FE80::5:73FF:FEA0 :6 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 22

22 Hot Standby Routing Protocol (HSRP) Local Failures Active Router BR-W1 HSRP BR-W2 (.2) (.3) (.1) VIP Default Gateway: (.1) DG MAC: MAC VIP BR-W2#show standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Fa0/ P Active local unknown Fa0/ P Active local unknown FE80::5:73FF:FEA0 :6 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 23

23 Hot Standby Routing Protocol (HSRP) Upstream/Remote Failures Active Standby Router Router BR-W1 BR-W2 HSRP (.2) (.3) (.1) VIP Complex Failure Requires Enhanced Object Tracking (EOT) Active Router BR-W1 BR-W2 HSRP (.2) (.3) (.1) VIP BR-W1# track 100 interface serial2/0 line-protocol! interface FastEthernet0/0 standby version 2 standby 4 priority 110 standby 4 track 100 decrement 20 standby 6 priority 110 standby 6 track 100 decrement 20 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 24

24 Gateway Load Balancing Protocol (GLBP) AVG = Active Virtual Gateway SVG = Standby Virtual Gateway AVF = Active Virtual Forwarder BR-W1# interface FastEthernet0/0 ip address glbp 4 ip glbp 4 preempt glbp 4 weighting 110 lower 100 glbp 6 ipv6 autoconfig glbp 6 preempt glbp 6 weighting 110 lower 100 ipv6 address 2001:DB8:5:1::1/64 AVG AVF A AVF B BR-W1 BR-W2 GLBP (.2) (.3) VIP (.1) (.1) VIP Default Gateway: (.1) DG MAC: AVF A SVG Default Gateway: (.1) DG MAC: AVF B BR-W1#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/ Standby local Fa0/ Active 0007.b local - Fa0/ Listen 0007.b Fa0/ Standby FE80::7:B4FF:FE00:600 FE80::A8BB:CCFF:FE00:3400 local Fa0/ Active 0007.b local - Fa0/ Listen 0007.b FE80::A8BB:CCFF:FE00: BR-W2#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/ Active local Fa0/ Listen 0007.b Fa0/ Active 0007.b local - Fa0/ Active FE80::7:B4FF:FE00:600 local F:FE00:3300 Fa0/ Listen 0007.b FE80::A8BB:CCFF:FE00: FE80::A8BB:CCF Fa0/ Active 0007.b Cisco and/or its affiliates. local All rights reserved. Cisco - Public 25

25 Gateway Load Balancing Protocol (GLBP) AVG = Active Virtual Gateway SVG = Standby Virtual Gateway AVF = Active Virtual Forwarder BR-W2# *May 26 19:09:14.260: %GLBP-6-STATECHANGE: FastEth0/0 Grp 4 state Standby -> Act ive *May 26 19:09:15.326: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Liste n -> Active *May 26 19:09:15.826: %GLBP-6-STATECHANGE: FastEth0/0 Grp 6 state Standby -> Act ive *May 26 19:09:16.856: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 6 Fwd 1 state Liste n -> Active Local Failures AVG AVF A BR-W1 BR-W2 GLBP (.2) AVF B (.3) (.1) VIP BR-W2#show glbp brief Interface Grp Fwd Pri State Address Active router Standby router Fa0/ Active local unknown Fa0/ Active 0007.b local - Fa0/ Active 0007.b local - Fa0/ Active FE80::7:B4FF:FE00:600 local unknown Fa0/ Active 0007.b local - Fa0/ Active 0007.b local - Default Gateway: (.1) DG MAC: AVF A Default Gateway: (.1) DG MAC: AVF B BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 26

26 GLBP with Enhanced Object Tracking Upstream/Remote Failures Requires Enhanced Object Tracking AVF A BR-W1 AVG BR-W2 GLBP (.2) AVF B (.3) (.1) VIP AVG = Active Virtual Gateway SVG = Standby Virtual Gateway AVF = Active Virtual Forwarder Complex Failure Requires Enhanced Object Tracking (EOT) BR-W1 AVG AVF A BR-W2 (.2) GLBP (.1) AVF B VIP (.3) Branch BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 27

27 Enhanced Object Tracking Track Options Line-Protocol State of Interface IP-Routing State of Interface IP-Route Reachability Threshold* of IP-Route Metrics Syntax track object-number interface type number line-protocol track 1 interface serial 2/0 line-protocol track object-number interface type number ip routing track 2 interface ethernet 1/0 ip routing track object-number ip route IP-Addr/Prefix-len reachability track 3 ip route /16 reachability track object-number ip route IP-Addr/Prefix-len metric threshold track 4 ip route /16 metric threshold IPv6 Support added in 15.3(3)S 15.4(1)T Router#show track 100 Track 100 Interface Serial2/0 line-protocol Line protocol is Up 1 change, last change 00:00:05 Tracked by: GLBP FastEthernet0/1 1 Router#show track 103 Track 103 IP route reachability Reachability is Up (EIGRP) 1 change, last change 00:02:04 First-hop interface is FastEthernet0/0 Tracked by: GLBP FastEthernet0/1 1 * EIGRP, OSPF, BGP, Static Thresholds Are Scaled to Range of (0 255) BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 28

28 Enhanced Object Tracking IP SLA Track Options IP SLAs Operation Reachability of an IP SLAs Host Syntax track object-number ip sla type number state track 5 ip sla 4 state track object-number ip sla type number reachability track 6 ip sla 4 reachability Types of IP SLA Probes: dhcp dns ethernet frame-relay ftp 1 Available for IPv6 http icmp-echo 1 icmp-jitter mpls path-echo path-jitter tcp-connect 1 udp-echo 1 udp-jitter 1 voip BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 29

29 Enhanced Object Tracking (EOT) Tracking IP SLA Lo Lo IP SLA IP SLA Static Host Route Guarantees probe destination only reachable via desired path ip sla 100 icmp-echo source-ip timeout 100 Null Route Guarantees probe frequency 10 ip sla schedule 100 life forever start-time destination nowisn t reachable when ip sla 200 interface down icmp-echo source-ip timeout 100 frequency 10 ip sla schedule 200 life forever start-time now ip route FastEthernet0/ ip route FastEthernet0/ ip route Null0 2 ip route Null0 2 AVF A AVF B BR-W1 BR-W2 (.2) GLBP (.3) VIP (.1) (.1) VIP BR-W1#show ip sla statistics IPSLA operation id: 100 Latest RTT: 1 milliseconds Latest operation start time: *04:42: UTC Tue Feb Latest operation return code: OK Number of successes: 46 Number of failures: 0 Operation time to live: Forever IPSLA operation id: 200 Latest RTT: 1 milliseconds Latest operation start time: *04:42: UTC Tue Feb Latest operation return code: OK Number of successes: 24 Number of failures: 0 Operation time to live: Forever Cisco and/or its affiliates. All rights reserved. Cisco Public

30 Enhanced Object Tracking Tracking IP SLA BR-W1# track 100 ip sla 100 reachability track 200 ip sla 200 reachability track 1 list boolean or object 100 IP SLA object 200 interface FastEthernet0/0 ip address glbp 4 ip glbp 4 priority 110 glbp 4 preempt glbp 4 weighting 110 lower 100 glbp 4 load-balancing weighted glbp 4 weighting track 1 decrement 20 BR-W1 AVF A IP SLA AVF B BR-W2 GLBP (.2) (.3) VIP (.1) (.1) VIP BR-W1#show glbp FastEthernet0/0 Group 4 State is Active 1 state change, last state change 00:09:59 Virtual IP address is Hello time 3 sec, hold time 10 sec Next hello sent in secs Redirect time 600 sec, forwarder timeout sec Preemption enabled, min delay 0 sec Active is local Standby is , priority 105 (expires in sec) Priority 110 (configured) Weighting 110 (configured 110), thresholds: lower 100, upper 110 Track object 1 state Up decrement 20 Load balancing: weighted Group members: aabb.cc ( ) local aabb.cc ( ) There are 2 forwarders (1 active) Forwarder 1 State is Active <SNIP> Forwarder 2 State is Listen <SNIP> BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 31

31 Enhanced Object Tracking Composite Failure BR-W1 Remains Active Virtual Gateway (AVG) BR-W2 Becomes Active Virtual Forwarder (AVF) for both A and B BR-W1# *Feb 17 05:17:25: %TRACKING-5-STATE: 100 ip sla 100 state Up->Down *Feb 17 05:17:25: %TRACKING-5-STATE: 200 ip sla 200 state Up->Down *Feb 17 05:17:26: %TRACKING-5-STATE: 1 list boolean or Up->Down *Feb 17 05:17:38: %GLBP-6-FWDSTATECHANGE: FastEth0/0 Grp 4 Fwd 1 state Active -> Listen BR-W2#show glbp FastEthernet0/0 Group 4 State is Standby 1 state change, last state change 00:28:16 Virtual IP address is Hello time 3 sec, hold time 10 sec Next hello sent in secs Redirect time 600 sec, forwarder timeout sec Preemption enabled, min delay 0 sec Active is , priority 110 (expires in sec) Standby is local Priority 105 (configured) Weighting 110 (configured 110), thresholds: lower 100, upper 110 Track object 1 state Up decrement 20 Load balancing: weighted Group members: aabb.cc ( ) aabb.cc ( ) local There are 2 forwarders (2 active) Forwarder 1 State is Active <SNIP> Forwarder 2 State is Active <SNIP> BR-W1 AVG IP SLA Unable to Reach Either IP SLA Responder IP SLA AVF A AVF B BR-W2 GLBP (.2) (.3) (.1) VIP BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 32

32 Agenda Introduction Cisco IOS and IP Routing Convergence Techniques First Hop Redundancy Protocols Routing Protocols DDR and Static Routing Performance Routing Design and Deployment Final Wrap Up

33 Routing Protocol Timers INFORMATIONAL Keepalive (B) Holdtime (B,E,I) Hello (E,I,O) Invalid (R) Dead (O) Flush (R) Update (R) Holddown (R) BGP EIGRP (< T1) IS-IS (DIS) OSPF (NBMA) 5 (60) 15 (180) 10 (3.333) 30 (10) 10 (30) 40 (120) RIP/RIPv Note: Cisco Default Values BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 34

34 Routing Protocol Neighbor Behavior INFORMATIONAL R2 R1 R4 R3 Recovery Times by Protocol Link Down Link Up Link Up Link Up Line Protocol Down Loss 100% Neighbor Down Loss ~5% BGP ~ 1 s Never EIGRP (< T1) IS-IS (DIS) OSPF (NBMA) ~ 1s 15 (180) 15 (180) Never ~ 1s 30 (10) 30 (10) Never ~ 1s 40 (120) 40 (120) Never RIP/RIPv2 ~ 1s Never Note: Using Cisco Default Values BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 35

35 Routing Protocol Neighbor Behavior Adjust Hello Timers R2 R1 R4 BR-W1 R3 R4#show ip bgp vpnv4 vrf cisco neighbor BGP neighbor is , vrf cisco, remote AS 65110, external link BGP version 4, remote router ID BGP state = Established, up for 1d10h Last read 00:00:19, hold time is 180, keepalive interval is 60 seconds R4#show ip bgp vpnv4 vrf cisco neighbor BGP neighbor is , vrf cisco, remote AS 65110, external link BGP version 4, remote router ID BGP state = Established, up for 00:01:23 Last read 00:00:03, hold time is 21, keepalive interval is 7 seconds BR-W1# router bgp neighbor timers 7 21 When Configuring the Holdtime Argument for a Value of Less than Twenty Seconds, the Following Warning Is Displayed: % Warning: A Hold Time of Less than 20 Seconds Increases the Chances of Peer Flapping BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 36

36 Bidirectional Forwarding Detection (BFD) Extremely lightweight hello protocol IPv4, IPv6, MPLS, P2MP 10s of milliseconds (technically, microsecond resolution) forwarding plane failure detection mechanism. Single mechanism, common and standardized Multiple modes: Async (echo/non-echo), Demand Independent of Routing Protocols Levels of security, to match conditions and needs Facilitates close alignment with hardware BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 37

37 Drivers for BFD Link-layer detection misses some types of outages e.g. Control Plane failure Control Plane failure detection is very conservative seconds in default configurations Link-layer failure detection is not consistent across media types Less than 50ms on APS- protected SONET A few seconds on Ethernet Several seconds or more on WAN links Provides a measure of consistency across routing protocols Most current failure detection mechanisms are an order of magnitude too long for time-sensitive applications BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 38

38 Routing Protocol Neighbor Behavior Bidirectional Forwarding Detection interface FastEthernet0/1 ip address bfd interval 250 min_rx 250 multiplier 3 router eigrp network bfd all-interfaces R1#show bfd neighbors details IPv4 Sessions NeighAddr LD/RD RH/RS State Int /1 Up Up Fa0/1 Session state is UP and using echo function with 250 ms interval. Session Host: Software OurAddr: Handle: 2 Local Diag: 0, Demand mode: 0, Poll bit: 0 MinTxInt: , MinRxInt: , Multiplier: 3 Received MinRxInt: , Received Multiplier: 3 Holddown (hits): 0(0), Hello (hits): 1000(5796) Rx Count: 5724, Rx Interval (ms) min/max/avg: 1/1008/876 last: 237 ms ago Tx Count: 5800, Tx Interval (ms) min/max/avg: 1/1008/879 last: 624 ms ago Elapsed time watermarks: 0 0 (last: 0) Registered protocols: EIGRP CEF Uptime: 01:23:53 Last packet: Version: 1 - Diagnostic: 0 State bit: Up - Demand bit: 0 Poll bit: 0 - Final bit: 0 C bit: 0 Multiplier: 3 - Length: 24 My Discr.: 1 - Your Discr.: 2 Min tx interval: Min rx interval: Min Echo interval: (Fa0/1) R1 R2 (Fa0/0) interface FastEthernet0/0 ip address bfd interval 333 min_rx 333 multiplier 3 router bgp neighbor fall-over bfd Configured in milliseconds (ms) Displayed in microseconds (µs) BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 39

39 Routing Protocol Neighbor Behavior Bidirectional Forwarding Detection interface FastEthernet0/1 ip address bfd interval 250 min_rx 250 multiplier 3 router eigrp network bfd all-interfaces <show bfd neighbors details cont d> IPv4 Sessions NeighAddr LD/RD RH/RS State Int /2 Up Up Fa0/0 Session state is UP and using echo function with 333 ms interval. Session Host: Software OurAddr: Handle: 1 Local Diag: 0, Demand mode: 0, Poll bit: 0 MinTxInt: , MinRxInt: , Multiplier: 3 Received MinRxInt: , Received Multiplier: 3 Holddown (hits): 0(0), Hello (hits): 1000(5867) Rx Count: 5877, Rx Interval (ms) min/max/avg: 42/1014/881 last: 753 ms ago Tx Count: 5869, Tx Interval (ms) min/max/avg: 42/1007/880 last: 228 ms ago Elapsed time watermarks: 0 0 (last: 0) Registered protocols: BGP CEF Uptime: 01:26:08 Last packet: Version: 1 - Diagnostic: 0 State bit: Up - Demand bit: 0 Poll bit: 0 - Final bit: 0 C bit: 0 Multiplier: 3 - Length: 24 My Discr.: 2 - Your Discr.: 1 Min tx interval: Min rx interval: Min Echo interval: (Fa0/1) R1 R2 (Fa0/0) interface FastEthernet0/0 ip address bfd interval 333 min_rx 333 multiplier 3 router bgp neighbor fall-over bfd Configured in milliseconds (ms) Displayed in microseconds (µs) BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 40

40 Routing Protocol Neighbor Behavior Detecting Unreachable Neighbor (Hello Timers vs. BFD) R1 EIGRP Default: Elapsed Time Between Sec R2 100% Packet Loss (Link Up) R1#show clock *13:08: EST Fri May R1# *May 8 13:08:19.366: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor (Fast Ethernet0/1) is down: holding time expired seconds BFD: Elapsed Time Between ms with 50ms interval (example 250ms) R1#show clock *13:37: EST Fri May R1# *May 8 13:38:00.944: %BFDFSM-6-BFD_SESS_DOWN: BFD-SYSLOG: BFD session ld:4 hand le:2,is going Down Reason: ECHO FAILURE *May 8 13:38:00.944: %BFD-6-BFD_SESS_DESTROYED: BFD-SYSLOG: bfd_session_destroy ed, ld:4 neigh proc:eigrp, handle:2 act *May 8 13:38:00.948: %DUAL-5-NBRCHANGE: EIGRP-IPv4 1: Neighbor (Fast Ethernet0/1) is down: BFD peer down notified *May 8 13:39:01.011: %BFD-6-BFD_SESS_DESTROYED: BFD-SYSLOG: bfd_session_destroy ed, ld:4 neigh proc:cef, handle:2 act seconds BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 41

41 Agenda Introduction Cisco IOS and IP Routing Convergence Techniques Routing Protocols First Hop Redundancy Protocols DDR and Static Routing Performance Routing Design and Deployment Final Wrap Up

42 Other Convergence Techniques Options using Static Routing Floating Static Routes Reliable Static Routing (RSR) using Enhanced Object Tracking (EOT) Dial on Demand Routing (DDR) Backup Interface Dialer Watch EEM Script DMVPN State Tracking For more information: BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 43

43 Reliable Static Routing Tracking IP SLA IP SLA IP SLA (.9) (.9) / /29 BR-W1# Track 400 ip sla 400 reachability! ip sla 400 icmp-echo source-ip timeout 100 frequency 10 ip sla schedule 400 life forever start-time now! ip route ! ip route track 400 ip route BR-W1 BR-W1#show ip route track-table ip route track 400 state is [up] BR-W1#show ip route S /16 [1/0] via BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 44

44 Reliable Static Routing Tracking IP SLA BR-W1# *Mar 12 03:57:37.119: %TRACKING-5-STATE: 400 rtr 400 reachability Up->Down IP SLA Unable to Reach IP SLA Responder (.9) (.9) BR-W1#show ip route track-table / /29 ip route track 400 state is [down] BR-W1#show ip route S /16 [200/0] via BR-W1 Floating Static Installed BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 45

45 EEM Script Example: IPv6 Static Route Event Tracking ipv6 route 2001:DB8::12/ :DB8:B::5 2001:DB8:B::5 WAN RTR IP SLA WAN RTR Unable to Reach IP SLA Responder ip sla 610 icmp-echo 2001:DB8::12 source-interface GigabitEthernet0/1.99 threshold 1000 frequency 10 ip sla schedule 610 life forever start-time now track 601 list threshold percentage <snip additional tracked objects> object 610 threshold percentage down 40 up 60 track 610 ip sla 610 BR RTR event manager applet DISABLE-STATIC-IPv6 event track 601 state down action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "no ipv6 route ::/0 2001:DB8:B::5" action 4 cli command "end" action 99 syslog msg DEFAULT IPv6 ROUTE DISABLED" BR-RTR# 14:22:14: %TRACKING-5-STATE: 610 ip sla 610 state Up->Down 14:22:14: %TRACKING-5-STATE: 601 list threshold percentage Up->Down 14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(eem:disable-static-ipv6) 14:22:14: %HA_EM-6-LOG: DISABLE-STATIC-IPv6: DEFAULT IPv6 ROUTE DISABLED 15.4(1)T added Enhanced Object Tracking BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 46

46 Black Hole Route Detection IPSLA with EEM Lost connection to ISP but DHCP route stays in the routing table ip sla 110 icmp-echo x.x.x.x source-interface GigabitEthernet0/0 vrf INET-PUBLIC1! fvrf configuration threshold 1000 frequency 15 ip sla schedule 110 life forever start-time now ip sla 111 icmp-echo y.y.y.y source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 threshold 1000 frequency 15 ip sla schedule 111 life forever start-time now IP SLA Probes track 60 ip sla 110 reachability track 61 ip sla 111 reachability track 62 list boolean or object 60 object 61 (config)#ip route GigabitEthernet0/0 dhcp 10? <cr> Note: This method is compatible with dual Internet DHCP design. event manager applet DISABLE-STATIC-GIG0-0 event track 62 state down action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "no ip route GigabitEthernet0/0 dhcp 10" action 4 cli command "end" action 99 syslog msg DEFAULT IP ROUTE via GIG0/0 DISABLED" event manager applet ENABLE-STATIC-GIG0-0 event track 62 state up action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "ip route GigabitEthernet0/0 dhcp 10" action 4 cli command "end" action 99 syslog msg DEFAULT IP ROUTE via GIG0/0 ENABLED" BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 47

47 Black Hole Route Detection IPSLA with Recursive Routing Lost connection to ISP but DHCP route stays in the routing table IP SLA Probes ip sla 110 icmp-echo x.x.x.x source-interface GigabitEthernet0/0 vrf INET-PUBLIC1! fvrf configuration threshold 1000 frequency 15 ip sla schedule 110 life forever start-time now ip sla 111 icmp-echo y.y.y.y source-interface GigabitEthernet0/0 vrf INET-PUBLIC1 threshold 1000 frequency 15 ip sla schedule 111 life forever start-time now track 60 ip sla 110 reachability track 61 ip sla 111 reachability track 62 list boolean or object 60 object 61 Note: This method is compatible with dual Internet DHCP design. (config)#ip route GigabitEthernet0/0 dhcp 10? <cr> ip route GigabitEthernet0/0 dhcp 10 ip route GigabitEthernet0/ track 62 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 48

48 EEM Script Example: 3G Backup with Event Tracking WAN RTR 3G-RTR VPN RTR (Ce0/0/0) NAS ip sla 100 icmp-echo source-interface GigabitEthernet0/1.99 threshold 1000 frequency 15 ip sla schedule 100 life forever start-time now track 60 ip sla 100 reachability event manager applet ACTIVATE-3G event track 60 state down action 1 cli command "enable" action 2 cli command "configure terminal" action 3 cli command "interface cellular0/0/0" action 4 cli command "no shutdown" action 5 cli command "end" action 99 syslog msg "Activating 3G interface" 14:22:14: %TRACKING-5-STATE: 60 ip sla 100 reachability Up->Down 14:22:14: %SYS-5-CONFIG_I: Configured from console by on vty0(eem:activate-3g) 14:22:14: %HA_EM-6-LOG: ACTIVATE-3G: Activating 3G interface 14:22:34: %LINK-3-UPDOWN: Interface Cellular0/0/0, changed state to up 14:22:34: %DIALER-6-BIND: Interface Ce0/0/0 bound to profile Di1 14:22:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface Cellular0/0/0, changed state to up 14:22:40: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel11, changed state to up 14:22:40: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON 14:22:42: %DUAL-5-NBRCHANGE: EIGRP-IPv4 201: Neighbor (Tunnel11) is up: new adjacency VPN Remote Site over 3G/4G Technology Design Guide BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 49

49 DMVPN Interface State Control Example: 3G Backup with DMVPN WAN RTR 3G-RTR VPN RTR (Ce0/0/0) NAS track 2 list boolean or object 101 not track 101 interface Tunnel100 line-protocol interface Tunnel200 if-state track 2 tunnel source Cellular0/0/0 end #show track 2 Track 2 List boolean or Boolean OR is Down 7 changes, last change 00:07:55 object 101 not Up Tracked by: IF-State Control 2 17:24:18.682: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel100, changed state to down 17:24:18.682: %TRACK-6-STATE: 101 interface Tu100 line-protocol Up -> Down 17:24:18.744: %TRACK-6-STATE: 2 list boolean or Down -> Up 17:24:28.683: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel200, changed state to up 17:24:29.276: %BGP-5-ADJCHANGE: neighbor Up 17:24:37.505: %BGP-5-ADJCHANGE: neighbor Up #show track 2 Track 2 List boolean or Boolean OR is Up 8 changes, last change 00:00:32 object 101 not Down Tracked by: IF-State Control Cisco and/or its affiliates. All rights reserved. Cisco Public 50

50 Agenda Introduction Cisco IOS and IP Routing Convergence Techniques Routing Protocols First Hop Redundancy Protocols DDR and Static Routing Performance Routing Design and Deployment Final Wrap Up

51 Enterprise WAN Challenge Two Paths, Two Providers WAN Availability Headquarters MPLS ATM Frame Relay Branch Office But, Are the Applications Performing Adequately? Internet VPN Small Office BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 52

52 Performance Routing (PfR) MC ISP A ISP B ISP C Campus BR Alternate Path BR MC/BR Remote Office Si BR MPLS or Primary ISP Bottlenecks Best Metric Path Uses Reachability, Delay, Loss, Jitter, and Load to determine the best path MC/BR SOHO PfR Components BR Border Router (Forwarding Path) MC Master Controller (Decision Maker) BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 53

53 Traditional Topology MPLS - SP A A-R2 A-R1 A-R4 A-R3 HQ-W1 BR-W1 HQ-W2 MPLS - SP B BR-W2 B-R1 B-R2 Headquarters Routing protocol selects path Blackhole reconvergence can take minutes Will not recover from brownouts Branch BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 54

54 PfR Enabled Topology L3 L7 Aware Domain Policy Configured and Deployed by PfR Hub Master Controller MPLS - SP A Live Traffic Monitoring HQ-MC1 Hub MC BR C-A-R1 C-A-R2 C-A-R3 C-A-R4 MC/BR HQ-W1 BR-W1 HQ-IPSLA BR BR HSRP HQ-W2 MPLS - SP B BR-W2 C-B-R1 C-B-R2 Headquarters MC = Master Controller BR = Border Router Branch Smart Probe PfR uses routing protocol and policy to select path Live Traffic Monitoring significantly improves reconvergence due to blackholes and brownouts BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 55

55 Summary of Convergence Techniques R2 R1 R4 R3 Excellent Option SubOptimal Option Bad Option Effectiveness of Various Techniques for Different Outage Types Routing Protocols Link Down Link Up Neighbor Down Link Up Loss ~5% Upstream Blackhole Upstream Brownout BFD N/A 1 N/A EOT RSR using EOT (w/ip SLA) PfR 1 disregarding BFD Multihop support for Static and BGP routes BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public

56 Agenda Introduction Cisco IOS and IP Routing Convergence Techniques Design and Deployment MPLS Dual Carrier MPLS + Internet Final Wrap Up

57 Dual WAN (MPLS Dual Carrier) PE-CE Protocol: BGP / /24 Default behaviour: 1-way load sharing Load is shared from HQ to Branch HQ- CORE1 HQ-W /24 A-R1 MPLS - SP A A-R / /29 BR-W1 HQ-W2 HQ- CORE2 B-R1 MPLS - SP B B-R4 Only one link used Branch to HQ EIGRP ebgp ebgp HQ-CORE1#show ip route D EX /24 [170/258816] via , 02:24:22, Vlan10 [170/258816] via , 02:24:22, Vlan10 BR-W1#show ip route B /16 [20/0] via , 00:34:00 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 58

58 Dual WAN (MPLS Dual Carrier) PE-CE Protocol: BGP EIGRP Routes redistributed from BGP into EIGRP (match & tag) BGP routes are treated as EIGRP external BGP No ibgp required between HQ- W1 & HQ-W2 (CE routers) Routes redistributed from EIGRP into BGP except those tagged as originally sourced from BGP HQ- CORE1 HQ- CORE /16 HQ-W1 HQ-W /24 A-R1 B-R1 MPLS - SP A MPLS - SP B / /29 A-R4 B-R /24 EIGRP ebgp ebgp BR-W1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 59

59 EIGRP ibgp Dual WAN (MPLS Dual Carrier) Mutual Route Redistribution Detail HQ-CORE1 HQ-CORE / /24 BR HQ-W1 HQ-W2 BR ebgp ebgp AS AS Routes into EIGRP / /24 BGP Redistribution to IGP automatically tags routes with neighbor AS Number HQ-W1# router eigrp networkers address-family ipv4 unicast autonomous-system topology base redistribute bgp metric address-family ipv6 unicast autonomous-system topology base redistribute bgp metric Routes into BGP HQ-W1# router bgp address-family ipv4 redistribute eigrp route-map BLOCK-TAGGED-ROUTES address-family ipv6 redistribute eigrp route-map BLOCK-TAGGED-ROUTES! route-map BLOCK-TAGGED-ROUTES deny 10 match tag route-map BLOCK-TAGGED-ROUTES permit 20! BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 60

60 Dual WAN (MPLS Dual Carrier) PE-CE Protocol: BGP Spoke EIGRP No EIGRP required on BR-W1 (collapsed routing) BGP Protect Branch from becoming transit AS HQ- CORE1 HQ- CORE /16 HQ-W1 HQ-W /24 A-R1 B-R1 MPLS - SP A MPLS - SP B BR-W1#show ip bgp Network Next Hop Metric LocPrf Weight Path A-R / /29 B-R /24 EIGRP ebgp ebgp BR-W1 * / ? *> ? router bgp neighbor route-map NO-TRANSIT-AS out neighbor route-map NO-TRANSIT-AS out... ip as-path access-list 1 permit ^$ route-map NO-TRANSIT-AS permit 10 match as-path 1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 61

61 Dual WAN (MPLS Dual Carrier) PE-CE Protocol: BGP Is it possible to load share from Branch to HQ? BGP Multipath Allows installation of multiple BGP paths to same destination Requirements (all must be equal) Neighbor AS or AS-PATH Weight Local Pref AS-PATH length Origin Med HQ- CORE1 HQ- CORE2 BR-W1#show ip bgp /16 HQ-W1 HQ-W /24 A-R1 B-R1 MPLS - SP A MPLS - SP B / /29 Network Next Hop Metric LocPrf Weight Path * / ? *> ? A-R4 B-R /24 EIGRP ebgp ebgp BR-W1#show ip route B /16 [20/0] via , 00:34:00 BR-W1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 62

62 Dual WAN (MPLS Dual Carrier) PE-CE Protocol: BGP Is it possible to load share from Branch to HQ? maximum-paths 2 Requires hidden command: bgp bestpath as-path multipath-relax HQ- CORE1 HQ- CORE /16 HQ-W1 HQ-W /24 router bgp bgp bestpath as-path multipath-relax address-family ipv4 maximum-paths 2 address-family ipv6 maximum-paths 2 BR-W1#show ip route B /16 [20/0] via , 00:03:44 [20/0] via , 00:03:44 A-R1 B-R1 MPLS - SP A MPLS - SP B / /29 A-R4 B-R /24 EIGRP ebgp ebgp BR-W1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 63

63 Performance Routing (PfRv3) Basic Configuration Dedicated MC, BRs HQ-MC# domain default vrf default master hub source-interface Loopback0 password IWAN BR HQ-W1 BR HQ-W1# domain default vrf default border source-interface Loopback0 master password IWAN Interface GigabitEthernet0/1 domain path mplsa MC = Master Controller BR = Border Router PfR = Performance Routing HQ-W2 HQ-MC Hub MC Headquarters HQ-W2# domain default vrf default border source-interface Loopback0 master password IWAN Interface GigabitEthernet0/1 domain path mplsb domain default path mplsb Domain name default is implied Domain Path only configured at Master Hub site BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 64

64 Performance Routing (PfRv3) Dual WAN (MPLS-Dual Carrier) No DMVPN Encapsulation Single Master Hub Site Border Router per Transport/Carrier No Multiple Next Hop support No Multiple Data Center support No Transit Site Support Master Hub loopback must be reachable over the WAN for transport discovery BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 65

65 Performance Routing (PfRv3) Simplification of configuration via MC to MC peering Head-end Configuration Local Prefixes (pre-configured) Performance Traffic-Classes Custom Policy Available Each MC learns and announces its inside prefixes. MC source-interface address is local site-id HQ-MC#show run s domain default domain default vrf default master hub source-interface Loopback0 site-prefixes prefix-list LOCAL_PREFIX password IWAN load-balance class VOICE sequence 10 match dscp ef policy voice Load Balance all other non-performance traffic-classes path-preference mplsa fallback mplsb class VIDEO sequence 20 match dscp af41 policy real-time-video path-preference mplsa fallback mplsb class critical-application sequence 30 match dscp af31 policy low-latency-data path-preference mplsa fallback mplsb Built-In Policy Template! ip prefix-list LOCAL_PREFIX seq 5 permit /16 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 66

66 Performance Routing (PfRv3) Branch Domain Provisioning HQ-MC Loopback 0 from Loopback 0 Can only use a loopback interface for peering BR-W1# domain default vrf default master branch source-interface Loopback0 hub No additional configuration at Branch MC Site Prefix will be discovered via Performance Monitor MC/site-id used for source and destination of smart probes Each MC learns its inside prefixes, announcing them to the Hub MC BR-W1#show domain default master site-prefix Change will be published between 5-60 seconds Next Publish 01:46:09 later Prefix DB Origin: Prefix Flag: S-From SAF; L-Learned; T-Top Level; C-Configured; Site-id Site-prefix Last Updated Flag /32 00:13:52 ago L, /24 00:00:04 ago L, /32 00:30:17 ago S, /32 00:30:14 ago S, /32 00:30:14 ago S, /32 00:30:25 ago S, /32 00:30:38 ago S, /16 00:30:38 ago S,C, * /8 00:30:38 ago S,T, BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 67

67 Performance Routing (PfRv3) Basic Configuration Combined MC and BR domain default vrf default border source-interface Loopback0 master local password IWAN master branch source-interface Loopback0 password IWAN hub ! interface FastEthernet0/0.120 bandwidth 4000 interface FastEthernet0/0.220 bandwidth 4000 Choosing a single path can be desired, PfR will utilize the BGP Table or EIGRP Topology for Parent Route verification Bandwidth affects utilization calculations MC = Master Controller BR = Border Router Branch 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public MC/BR BR-W1#show ip bgp Network Next Hop BGP Configured to Choose Preferred Path Metric LocPrf Weight Path *> / ? * ? *> / ? * ? BR-W1#show ip route B [20/0] via , 03:32:30 B [20/0] via , 03:32:30 68 BR-W1

68 Performance Routing (PfRv3) Simplification of configuration via MC to MC peering: Branch Policy Download BR-W1#show domain default master policy <SNIP> class VOICE sequence 10 path-preference mplsa fallback mplsb class type: Dscp Based Performance Traffic-Class voice Template expanded Custom Policy using delay, loss, and/or jitter can be used Voice Template <SNIP> Threshold Definition priority 1 one-way-delay threshold 150 threshold 150 (msec) priority 2 packet-loss-rate threshold 1 (%) priority 2 byte-loss-rate threshold 1 (%) priority 3 jitter 30 (msec) Real-time-video priority 1 packet-loss-rate threshold 1 (%) priority 1 byte-loss-rate threshold 1 (%) Low-latency-data priority 2 one-way-delay threshold 150 (msec) priority 3 jitter 20 (msec) priority 1 one-way-delay threshold 100 (msec) priority 2 byte-loss-rate threshold 5 (%) priority 2 packet-loss-rate threshold 5 (%) match dscp ef policy voice priority 2 packet-loss-rate threshold 1.0 percent priority 1 one-way-delay threshold 150 msec priority 3 jitter threshold usec priority 2 byte-loss-rate threshold 1.0 percent Number of Traffic classes using this policy: 1 Template Bulk-data Best-effort scavenger Threshold Definition priority 1 one-way-delay threshold 300 (msec) priority 2 byte-loss-rate threshold 5 (%) priority 2 packet-loss-rate threshold 5 (%) priority 1 one-way-delay threshold 500 (msec) priority 2 byte-loss-rate threshold 10 (%) priority 2 packet-loss-rate threshold 10 (%) priority 1 one-way-delay threshold 500 (msec) priority 2 byte-loss-rate threshold 50 (%) priority 2 packet-loss-rate threshold 50 (%) BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 69

69 Performance Routing (PfRv3) Branch Domain Status BR-W1#show domain default master status *** Domain MC Status *** Master VRF: Global Verification of MC Peering Instance Type: Branch Instance id: 0 Configured status: Up Loopback IP Address: Load Balancing: Operational Status: Up Max Calculated Utilization Variance: 51% Last load balance attempt: 00:00:13 ago Last Reason: No channels yet for load balancing Total unbalanced bandwidth: External links: 437 Kbps Internet links: 0 Kpbs Route Control: Enabled Mitigation mode Aggressive: Disabled Policy threshold variance: 20 Minimum Mask Length: 28 Sampling: off Minimum Requirement: Met Borders: IP address: Connection status: CONNECTED (Last Updated 20:17:32 ago ) Interfaces configured: Name: FastEthernet0/0.120 type: external Service Provider: mplsa Status: UP Number of default Channels: 0 Name: FastEthernet0/0.220 type: external Service Provider: mplsb Status: UP Number of default Channels: 0 Tunnel if: Tunnel0 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 70

70 Performance Routing (PfRv3) Branch Domain Provisioning No additional configuration at Branch MC Policy will be learned from Master Hub class default Present if load-balance configured BR-W1#show domain default master policy class VOICE sequence 10 path-preference mplsa fallback mplsb class type: Dscp Based match dscp ef policy voice priority 2 packet-loss-rate threshold 1.0 percent priority 1 one-way-delay threshold 150 msec priority 3 jitter threshold usec priority 2 byte-loss-rate threshold 1.0 percent Number of Traffic classes using this policy: 1 class VIDEO sequence 20 path-preference mplsa fallback mplsb class type: Dscp Based match dscp af41 policy real-time-video priority 1 packet-loss-rate threshold 1.0 percent priority 2 one-way-delay threshold 150 msec priority 3 jitter threshold usec priority 1 byte-loss-rate threshold 1.0 percent class critical-application sequence 30 path-preference mplsa fallback mplsb class type: Dscp Based match dscp af31 policy low-latency-data priority 2 packet-loss-rate threshold 5.0 percent priority 1 one-way-delay threshold 100 msec priority 2 byte-loss-rate threshold 5.0 percent class default match dscp all BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 71

71 Performance Routing (PfRv3) Load Balancing Configuration If Traffic Goes Above 20% utilization, PfR Tries to Move the Traffic from this Exit Link to Another Underutilized Exit Link HQ-MC HQ-MC# domain default vrf default master hub load-balance Not enabled by default This is completely opposite of prior versions MC/BR BR-W1 Performance Traffic Classes (TCs) TCs with performance metrics defined (delay, loss, jitter) Non Performance Traffic Classes Default TCs i.e. TCs that do not match any of the match statements TCs without performance metrics defined Load Balancing: only affects non-performance TCs Branch MC = Master Controller BR = Border Router Variance defaults to 20% BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 72

72 Performance Routing (PfRv3) Load Sharing Example / / /24 Both MPLS carriers are now being utilized More prefixes and flows result in better load sharing BR-W1#show domain default master traffic-classes summary Dst-Site-Pfx Dst-Site-Id APP DSCP TC-ID APP-ID State SP PC/BC BR/EXIT / N/A default 1 N/A CN mplsb 2/ /FastEthernet0/ / N/A ef 4 N/A CN mplsa 7/ /FastEthernet0/ / N/A af41 3 N/A CN mplsa 5/ /FastEthernet0/ / N/A af31 2 N/A CN mplsa 3/NA /FastEthernet0/0.120 HQ- CORE1 HQ- CORE2 Total Traffic Classes: 4 Site: 4 Internet: 0 HQ-W1 HQ-W2 A-R1 B-R1 MPLS - SP A MPLS - SP B A-R / /29 B-R4 EIGRP BGP BGP BR-W1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 73

73 Performance Routing (PfRv3) Load Sharing Example / /24 Both MPLS carriers are now being utilized More prefixes and flows result in better load sharing BR-W1#show domain default master exits BR address: Name: Fast0/0.120 type: external Path: mplsa Egress capacity: 1500 Kbps Egress BW: 292 Kbps Ideal:241 Kbps over: 60 Kbps Egress Utilization: 19 % DSCP: af31[26]-number of Traffic Classes[1] DSCP: af41[34]-number of Traffic Classes[1] DSCP: ef[46]-number of Traffic Classes[1] BR address: Name: Fast0/0.220 type: external Path: mplsb Egress capacity: 2000 Kbps Egress BW: 254 Kbps Ideal:321 Kbps under: 59 Kbps Egress Utilization: 12 % DSCP: default[0]-number of Traffic Classes[1] HQ- CORE1 HQ- CORE /24 HQ-W1 HQ-W2 A-R1 B-R1 MPLS - SP A MPLS - SP B A-R / /29 B-R4 EIGRP BGP BGP BR-W1 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 74

74 Performance Routing (PfRv3) Load Sharing Example Dual Router WAN Edge HSRP facing LAN hosts Loopback Interfaces must be reachable for BR to MC communication. Loopback utilized for Auto Tunnel to redirect traffic, so make sure it is over the LAN interface HQ- CORE1 HQ- CORE / /24 HQ-W1 HQ-W2 A-R1 B-R1 MPLS - SP A MPLS - SP B A-R / /29 B-R /24 EIGRP ebgp ebgp BR-W1 BR-W2 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 75

75 Performance Routing (PfRv3) Multiple Paths Select Best Path by Application Requirements Monitor relevant path characteristics (one-way delay, loss, jitter, ) path A: <5 ms delay, 0% loss path B: < 50 ms delay, 0% loss Live Application Performance with Performance Monitor (TCP and RTP) / / /24 Path without active traffic must be evaluated Smart Probe sent 1/3 Monitor Interval on Active path for validation, default 30 seconds. 10 packets every 500ms on fallback path. HQ- CORE1 HQ- CORE2 HQ-W1 HQ-W2 A-R1 B-R1 MPLS - SP A MPLS - SP B A-R4 B-R4 BR-W1 BR-W2 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 76

76 Performance Routing (PfRv3) Multiple Paths Select Best Path by Application Requirements HQ-MC# domain default vrf default master hub <SNIP> class VOICE sequence 10 match dscp ef policy voice path-preference mplsa fallback mplsb <SNIP> BR-W1#show domain default master policy class VOICE sequence 10 <SNIP> path-preference mplsa fallback mplsb class type: Dscp Based match dscp ef policy voice HQ- CORE1 HQ- CORE2 priority 2 packet-loss-rate threshold 1.0 percent priority 1 one-way-delay threshold 150 msec priority 3 jitter threshold usec /16 HQ-W1 HQ-W /24 priority 2 byte-loss-rate threshold 1.0 percent Number of Traffic classes using this policy: 1 A-R1 B-R1 MPLS - SP A MPLS - SP B A-R4 B-R / Cisco and/or its affiliates. All rights reserved. BRKRST-2042 Cisco Public 77 BR-W1 BR-W2

77 Performance Routing (PfRv3) / /24 Active Prefix Monitoring /24 A-R1 MPLS - SP A A-R4 HQ- CORE1 HQ-W1 BR-W1 HQ-W2 BR-W2 BR-W1# show domain default master traffic-classes dscp ef Dst-Site-Prefix: /16 DSCP: ef [46] Traffic class id:4 TC Learned: 05:55:01 ago Present State: CONTROLLED Current Performance Status: in-policy Current Service Provider: mplsa since 05:54:31 Previous Service Provider: mplsa for 1711 sec BW Used: 8 Kbps Present WAN interface: FastEthernet0/0.120 in Border Present Channel (primary): 7 Backup Channel: 8 Destination Site ID: Class-Sequence in use: 10 Class Name: VOICE using policy voice BW Updated: 00:00:01 ago Reason for Route Change: Uncontrolled to Controlled Transition Total Traffic Classes: 1 Site: 1 Internet: 0 HQ- CORE2 B-R1 MPLS - SP B B-R4 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 78

78 Performance Routing (PfRv3) Smart Probes / /24 BR-W1#show domain default master channels dscp ef <SNIP> Channel Id: 25 Dst Site-Id: Link Name: mplsa DSCP: ef [46] TCs: 1 Channel Created: 03:51:50 ago Provisional State: Initiated and open Operational state: Available Interface Id: 11 Estimated Channel Egress Bandwidth: 8 Kbps Immitigable Events Summary: Total Performance Count: 0, Total BW Count: 0 ODE Stats Bucket Number: 1 Last Updated : 00:05:16 ago Packet Count : 534 Byte Count : One Way Delay : 49 msec* Loss Rate Pkts: 1.47 % Loss Rate Byte: 0.0 % Jitter Mean : 2956 usec Unreachable : FALSE ODE Stats Bucket Number: 2 <SNIP> TCA Statitics: Received:24 ; Processed:24 ; Unreach_rcvd:0 Latest TCA Bucket Last Updated : 00:05:16 ago One Way Delay : NA Loss Rate Pkts: 1.47 % Loss Rate Byte: NA Jitter Mean : NA Unreachability: FALSE HQ- CORE1 HQ- CORE /24 HQ-W1 HQ- W2 A-R1 B-R1 MPLS - SP A MPLS - SP B A-R4 B-R4 Compare against defined policy. BR-W1 BR-W2 If outside policy, a Threshold Crossing Alert (TCA) is sent to the originating site On Demand Export of traffic is sent to the originating site Master Controller. BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 79

79 Performance Routing (PfRv3) Smart Probes Smart Probes are sourced from Border Routers. Probes spoof the local Master Controller (site-id) and routed via external interfaces to the destination Master Controller (site-id). Probes are sent one-way. Destination Border Routers absorb and evaluate the Smart Probes. HQ- CORE1 HQ- CORE / /24 HQ-W1 BR-W1#show domain default border channels site-id Border Smart Probe Stats: Smart probe parameters: Source address used in the Probe: Unreach time: 1000 ms Probe source port: Probe destination port: Interface Discovery: ON Probe freq for channels with traffic :10 secs Discovery Probes: OFF Number of transit probes consumed :0 Number of transit probes re-routed: 0 DSCP's using this: [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] All the other DSCPs use the default interval: 10 secs HQ- W2 A-R1 B-R1 MPLS - SP A MPLS - SP B A-R4 B-R /24 BR-W1 BR-W2 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 80

80 Performance Routing (PfRv3) Path Disruption: Loss /16 2% Packet Loss /24 A-R1 MPLS - SP A A-R4 HQ- CORE1 HQ-W1 BR-W1 HQ-W2 BR-W2 HQ- CORE2 B-R1 MPLS - SP B B-R4 BR-W1#show domain default master traffic-classes dscp ef Dst-Site-Prefix: /16 DSCP: ef [46] Traffic class id:4 TC Learned: 23:00:00 ago Present State: CONTROLLED Current Performance Status: in-policy Current Service Provider: mplsb since 00:00:28 (hold until 61 sec) Previous Service Provider: mplsa for sec (A fallback provider. Primary provider will be re-evaluated 00:02:32 later) BW Used: 4 Kbps Present WAN interface: FastEthernet0/0.220 in Border Present Channel (primary): 8 Backup Channel: 7 Destination Site ID: Class-Sequence in use: 10 Class Name: VOICE using policy voice BW Updated: 00:00:00 ago Reason for Route Change: Loss Total Traffic Classes: 1 Site: 1 Internet: Cisco and/or its affiliates. All rights reserved. BRKRST-2042 Cisco Public 81

81 Performance Routing (PfRv3) Path Disruption: Loss /16 2% Packet Loss A-R1 MPLS - SP A A-R /24 HQ- CORE1 HQ-W1 BR-W1 HQ-W2 BR-W2 HQ- CORE2 B-R1 MPLS - SP B B-R4 BR-W1#show domain default master channels dscp ef <SNIP> Channel Id: 7 Dst Site-Id: Link Name: mplsa DSCP: ef [46] TCs: 0 <SNIP> ODE Stats Bucket Number: 1 Last Updated : 00:00:11 ago Packet Count : 502 Byte Count : One Way Delay : 55 msec* ODE = On-Demand Export Loss Rate Pkts: 2.64 % Loss Rate Byte: 0.0 % TCA = Threshold Crossing Alert Jitter Mean : 4298 usec Unreachable : FALSE <SNIP> Latest TCA Bucket Last Updated : 00:00:11 ago One Way Delay : NA Loss Rate Pkts: 2.64 % <SNIP> 2016 Cisco and/or its affiliates. All rights reserved. BRKRST-2042 Cisco Public 82

82 Agenda Introduction Cisco IOS and IP Routing Convergence Techniques Design and Deployment MPLS Dual Carrier MPLS + Internet Final Wrap Up

83 DUAL WAN (MPLS + Backup) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP Headquarters WAN Edge W1 learns Branch route via ebgp W2 learns Branch route via EIGRP Headquarters Core W1 redistributes ebgp into EIGRP, results in EIGRP external W2 does not require redistribution, results in EIGRP internal Core1, Core2 install Branch route via W2 HQ to Branch Traffic Flows Across Tunnel HQ-CORE1 HQ-CORE2 EIGRP BGP BGP /16 HQ-W1 HQ-W / /29 HQ-W2#show ip route D /24 [90/ ] via , 00:00:04, Tunnel1 A-R1 MPLS - SP A Internet VPN Tunnel EIGRP /29 A-R /24 HQ-W1#show ip route B /24 [20/0] via , 05:24:01 HQ-CORE1#show ip route D /24 [90/ ] via , 00:02:32, Vlan10 BR-W1 BR-W2 EIGRP HSRP BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 84

84 DUAL WAN (MPLS + Backup) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP Branch WAN Edge W1 learns HQ route via ebgp and EIGRP Internal ebgp Administrative Distance preferred EIGRP BGP BGP / / /24 A-R1 MPLS - SP A A-R4 HQ-CORE1 HQ-W /29 BR-W1 Internet HQ-W2 VPN Tunnel HQ-CORE /29 EIGRP Branch to HQ Traffic Flows Across MPLS BR-W1#show ip route B /24 [20/0] via , 04:48:58 B /24 [20/0] via , 03:44:06 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 85

85 DUAL WAN (MPLS + Backup) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP Branch WAN Edge W1 learns HQ route via ebgp W2 learns HQ route via EIGRP No redistribution configured HSRP Primary is on W1 Branch to HQ Traffic Flows Across MPLS HQ-CORE1 HQ-CORE2 EIGRP BGP BGP /16 HQ-W /24 HQ-W /29 A-R1 MPLS - SP A Internet A-R /29 VPN Tunnel EIGRP /24 BR-W1#show ip route B /24 [20/0] via , 04:48:58 B /24 [20/0] via , 03:44:06 BR-W2#show ip route D /24 [90/ ] via , 00:10:56, Tunnel1 D /24 [90/ ] via , 00:10:57, Tunnel1 BR-W1#show standby brief P indicates configured to preempt. Interface Grp Pri P State Active Standby Virtual IP Fa0/ P Active local BR-W1 BR-W2 EIGRP HSRP BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 86

86 DUAL WAN (MPLS + Backup) PE-CE Protocol: BGP, Tunnel Protocol: EIGRP How to force HQ to Branch traffic across MPLS (primary)? Adjust administrative distance of EIGRP routes learned via tunnel Ensure new distance is higher than that of EIGRP external (170) HQ-W2# router eigrp network distance Now: HQ to Branch Traffic Flows Across MPLS HQ-CORE1 HQ-CORE2 EIGRP BGP BGP /16 HQ-W1 HQ-W / /29 HQ-W1#show ip route B /24 [20/0] via , 05:24:01 A-R1 MPLS - SP A HQ-W2#show ip route D EX /24 [170/261120] via , 00:07:25, GigE0/0 Internet VPN Tunnel EIGRP /29 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 87 A-R /24 HQ-CORE1#show ip route D EX /24 [170/258816] via , 00:08:44, Vlan10 BR-W1 BR-W2 EIGRP HSRP

87 DUAL WAN (MPLS + Backup) MPLS Failure Failure within MPLS cloud Worst Case EIGRP BGP BGP / /24 Primary dependency is BGP timers /24 A-R1 MPLS - SP A A-R4 Can result in end to end convergence time as long as BGP Holdtime HQ-CORE1 HQ-W1 Internet /29 BR-W1 EIGRP HSRP Configuration options Would be much lower with BFD Application Restoration as fast as PfR detects After Failure: HQ to Branch Traffic Flows Across Tunnel HQ-CORE2 HQ-W2 HQ Route Tables /29 VPN Tunnel HQ-W2#show ip route D /24 [195/ ] via , 00:06:46, Tunnel1 HQ-CORE1#show ip route D /24 [90/ ] via , 00:09:18, Vlan10 EIGRP BR-W2 BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 88

88 DUAL WAN (MPLS + Backup) MPLS Failure Failure within MPLS cloud Suboptimal routing at Branch EIGRP BGP BGP / / /24 HSRP primary remains unchanged at BR-W1 HQ-CORE1 HQ-W1 A-R1 MPLS - SP A /29 A-R4 BR-W1 Use EOT and move HSRP primary to BR-W2 HQ-CORE2 HQ-W2 Internet VPN Tunnel BR-W2 EIGRP HSRP After Failure: Branch to HQ Traffic Flows Across Tunnel Branch Route Tables /29 BR-W1#show ip route D /24 [90/ ] via , 00:22:42, FastEthernet0/1 D /24 [90/ ] via , 00:22:42, FastEthernet0/1 BR-W2#show ip route D /24 [90/ ] via , 01:08:44, Tunnel1 D /24 [90/ ] via , 01:08:45, Tunnel1 EIGRP BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 89

89 DUAL WAN (MPLS + Internet) Intelligent WAN (IWAN): PE-CE Protocol: Static/unneeded Tunnel Protocol: EIGRP or BGP Transport Independent Design - Any IPv4/IPv6 transport EIGRP / / /24 - ibgp or EIGRP for Routing A-R1 DMVPN MPLS -Tunnel SP A A-R4 Intelligent Path Control HQ-CORE1 HQ-W /29 BR-W1 - PfRv3 for Intelligent Delivery of Application Traffic per Policy - Load Balancing for Full Utilization of Bandwidth HQ-CORE2 HQ-W /29 Internet DMVPN Tunnel EIGRP BR-W2 EIGRP HSRP Application Optimization Secure Connectivity BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 90

90 Intelligent WAN (IWAN) Automated Deployment IWAN App with APIC-EM for a Software Defined WAN Automated deployment of Branch Routers Prime Infrastructure 3.1 IWAN Workflow Customizable Deployment scripts based on CVD Easily customized for deployment Integration with APIC-EM for Certificate Authority and Plug-n-Play Intelligent WAN ONUG Certified Transport Independence Intelligent Path Control SD-WAN Application Optimization Secure Connectivity BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 91

91 Agenda Introduction Cisco IOS and IP Routing Convergence Techniques Design and Deployment Final Wrap Up Key Takeaways

92 Key Takeaways Outages can manifest in many different ways. Network design should be based on application requirements to survive various outages. Cisco IOS has inherent load sharing capabilities. Analyse your network topology and use these to your advantage. End-to-end convergence time is a critical metric. Understand how localized topology changes affect end-to-end resiliency. Multiple links/paths not only increase network reliability but can improve application performance. BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 93

93 Key Takeaways IP SLA based monitoring can detect outage types that are virtually undetectable by traditional hello based techniques. Performance Routing permits path selection based on current real time characteristics. Performance Routing permits full utilization of available bandwidth Most effective network designs incorporate a combination of convergence techniques BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 94

94 Complete Your Online Session Evaluation Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card. Complete your session surveys through the Cisco Live mobile app or from the Session Catalog on CiscoLive.com/us. Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 95

95 Continue Your Education Demos in the Cisco campus Walk-in Self-Paced Labs Lunch & Learn Meet the Engineer 1:1 meetings Related sessions BRKRST Cisco and/or its affiliates. All rights reserved. Cisco Public 96

96 Please join us for the Service Provider Innovation Talk featuring: Yvette Kanouff Senior Vice President and General Manager, SP Business Joe Cozzolino Senior Vice President, Cisco Services Thursday, July 14 th, :30 am - 12:30pm, In the Oceanside A room What to expect from this innovation talk Insights on market trends and forecasts Preview of key technologies and capabilities Innovative demonstrations of the latest and greatest products Better understanding of how Cisco can help you succeed Register to attend the session live now or watch the broadcast on cisco.com