IP NAT Troubleshooting. Solutions. Luke Bibby, CCIE #45527

Similar documents
LAB5: OSPF IPv4. OSPF: Stub. Disclaimer

Routing Information Protocol Version 2.0

EIGRP Lab / lo1. .1 lo / /30

Contents. Introduction. Prerequisites. Requirements

Chapter 7 Lab 7-2, Using the AS_PATH Attribute

Lab 2.8.1: Basic Static Route Configuration

GRE Tunnel with VRF Configuration Example

Step 1: IP Configuration: On R1: On R2: On R3: R1(config)#int s1/2 R1(config-if)#ip addr R1(config-if)#no shu

Implement Static Routes for IPv6 Configuration Example

Configuration and Management of Networks

This document is exclusive property of Cisco Systems, Inc. Permission is granted to print and copy this document for non-commercial distribution and

Chapter 4 Lab 4-2, Redistribution Between EIGRP and OSPF

Easy Virtual Network Configuration Example

Distance vector Routing protocols. 2000, Cisco Systems, Inc. 9-1

Building the Routing Table. Introducing the Routing Table Directly Connected Networks Static Routing Dynamic Routing Routing Table Principles

Chapter 1 Lab 1-1, Basic RIPng and Default Gateway Configuration

AS 100 AS 300. Lab -1 Private Communities - II .1 S1/2. Task 1. On R1: / / /24. Configure the above topology.

RealCiscoLAB.com. Chapter 6 Lab 6-2, Using the AS_PATH Attribute. Topology. Objectives. Background. CCNPv6 ROUTE

This chapter covers the following subjects:

Chapter 7 Lab 7-1, Configuring BGP with Default Routing

Lab: Basic Static Route Configuration

Lab 5-3 Redistribution Between EIGRP and IS-IS

LAB8: OSPF IPv4. OSPF: Virtual Link. Disclaimer

Dynamic Multipoint VPN (DMVPN) Troubleshooting Scenarios

RealCiscoLAB.com. Chapter 2 Lab 2-2, EIGRP Load Balancing. Topology. Objectives. Background. CCNPv6 ROUTE

LAB1: BGP IPv4. BGP: Initial Config. Disclaimer

Lab Configuring IPv4 Static and Default Routes (Solution)

Adapted from the Synchronization example in g/case/studies/icsbgp4.html

Layer3 VPN with OSPF Protocol between CE-PE

Chapter 6 Lab 6-3, Configuring IBGP and EBGP Sessions, Local Preference, and MED

Lab 2-3 Summarization and Default Network Advertisement

Lab Configuring Port Address Translation (PAT) (Instructor Version)

Chapter 4 Lab 4-1, Redistribution Between RIP and OSPF

Lab 3.5.1: Basic Frame Relay

Basic Router Configuration

CIS 83 LAB 3 - EIGRP Rich Simms September 23, Objective. Scenario. Topology

The information in this document is based on Cisco IOS Software Release 15.4 version.

Chapter 4 Lab 4-2, Controlling Routing Updates. Topology. Objectives. CCNPv7 ROUTE

Failover with EIGRP Using VRF Configuration Example

Chapter 6: Network Layer

Antonio Cianfrani. Routing Protocols

Lab - Troubleshooting Connectivity Issues

v5.0 Narbik Kocharians CCSI, CCIE #12410 R&S, Security, SP Physical or Logical

RIPv2. Routing Protocols and Concepts Chapter 7. ITE PC v4.0 Chapter Cisco Systems, Inc. All rights reserved. Cisco Public

STUDENT LAB GUIDE CCNA ( )

LAB8: Named EIGRP IPv4

Chapter 4 Lab 4-1, Redistribution Between EIGRP and OSPF. Topology. Objectives. CCNPv7 ROUTE

LAB11: EIGRP IPv4. EIGRP: Stub. Disclaimer

Lab Configuring Dynamic and Static NAT (Solution)

Lab Configuring Static NAT

Lab Configuring Port Address Translation (PAT) Topology

CCNA Exploration: Routing Protocols and Concepts Chapter 8 Case Study

Chapter 6 Lab 6-4, BGP Route Reflectors and Route Filters

TELECOMMUNICATION MANAGEMENT AND NETWORKS

DMVPN Topology. Page1

CCNP ROUTE LAB MANUAL

Lab Configuring Dynamic and Static NAT (Instructor Version Optional Lab)

LAB15: EIGRP IPv4. LAB 15: Diagram. Disclaimer

RR> RR> RR>en RR# RR# RR# RR# *Oct 2 04:57:03.684: %AMDP2_FE-6-EXCESSCOLL: Ethernet0/2 TDR=0, TRC=0 RR#

Shortcut Switching Enhancements for NHRP in DMVPN Networks

How to Configure a Cisco Router Behind a Non-Cisco Cable Modem

Lab- Troubleshooting Basic EIGRP for 1Pv4

RIP. Table of Contents. General Information. Summary. Specifications

Chapter 8 Lab 8-3, Configuring 6to4 Tunnels

Chapter 5: Maintaining and Troubleshooting Routing Solutions

Lab Configuring Basic RIPv2 (Solution)

Lab Configuring IGRP Instructor Version 2500

Chapter 3 Lab 3-2, Multi-Area OSPFv2 and OSPFv3 with Stub Area

CCNA Exploration: Routing Protocols and Concepts Chapter 11 Case Study

FlexVPN HA Dual Hub Configuration Example

Lab- Configuring Basic Single-Area OSPFv2

EIGRP on SVTI, DVTI, and IKEv2 FlexVPN with the "IP[v6] Unnumbered" Command Configuration Example

MPLS Troubleshooting. Contents. Prerequisites. Document ID: Requirements. Components Used

3 ROUTER SETUP WITH FRAME RELAY CONFIGURATION OF FRAME RELAY ROUTER

2016 Braindump2go Valid Cisco Exam Preparation Materials:

This document is exclusive property of Cisco Systems, Inc. Permission is granted to print and copy this document for non-commercial distribution and

Final exam study Guide

Multihoming with BGP and NAT

Case Study 2: Frame Relay and OSPF Solution

KIM DONNERBORG / RTS. Cisco Lab Øvelse Af Kim Donnerborg / RTS. Side 0 af 8

MPLS for R&S CCIE Candidates

Chapter 2 Lab 2-1, EIGRP Configuration, Bandwidth, and Adjacencies

Lab 1 BGP Automatic-tag, AS-path tag and Table-map

Advanced Troubleshooting CCIE Routing & Switching v5.0

Shooting Trouble with IP

scope scope {global vrf vrf-name} no scope {global vrf vrf-name} Syntax Description

Network Layer Week 5. Module : Computer Networks Lecturer: Lucy White Office : 324

Information About Routing

Draft Manuscript Draft M. Manuscript Draft Ma. t Manuscript Draft Manu. ipt Draft Manuscript Dra. anuscript Draft Manuscri

Routing Overview. Information About Routing CHAPTER

Configuration and Management of Networks

Chapter 4: Manipulating Routing

Configuring FlexVPN Spoke to Spoke

GetCertkey. No help, Full refund!

Cisco 360 CCIE R&S Advanced Workshop 2 Assessment Lab 1

Lab 2.5.1: Basic PPP Configuration Lab

Configuring PPP over Ethernet with NAT

Layer3 VPN with RIP protocol between CE-PE

LAB16: Named EIGRP IPv4

Troubleshooting LSP Failure in MPLS VPN

Transcription:

IP NAT Troubleshooting Solutions Luke Bibby, CCIE #45527

Quick Overview of Scenario Solutions Scenario #1 R2 s E0/0 should be NAT inside not NAT outside ACL 100 is configured incorrectly NAT policy missing the inside keyword Scenario #2 RIP is not enabled on 1.0.0.0 on R1 NAT policy missing the inside keyword NAT policy has the inside local and inside global addresses around the wrong way Scenario #3 R1 s Loopback0 needs to be configured as NAT inside NAT policies are referencing non-existant ACLs; should be referencing route maps The route maps have the interfaces configured incorrectly

Scenario #1 Solutions The first thing we should test is that the connectivity between R1 and R3 is infact actually not working using ping. R1#ping 160.0.23.3 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 160.0.23.3, timeout is 2 seconds:... Success rate is 0 percent (0/5) Do we have NLRI for 160.0.23.3 or a default? R1#show ip route 160.0.23.3 % Network not in table R1#show ip route 0.0.0.0 Routing entry for 0.0.0.0/0, supernet Known via "static", distance 1, metric 0, candidate default path Routing Descriptor Blocks: * 10.0.12.2 Route metric is 0, traffic share count is 1 And is the next-hop reachable? R1#ping 10.0.12.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.0.12.2, timeout is 2 seconds:.!!!! Success rate is 80 percent (4/5), round-trip min/avg/max = 1/1/1 ms

At this point we can deduce that there is no apparent problem between R1 and R2 so let s now focus on R2. Let s make sure the inside and outside interfaces are up according to the diagram. R2#show ip interface brief exclude una down Any interface listed with OK? value "NO" does not have a valid configuration Interface IP-Address OK? Method Status Protocol Ethernet0/0 10.0.12.2 YES manual up up Ethernet0/1 160.0.23.2 YES manual up up Let s eliminate a connectivity problem between R2 and R3 using ping. R2#ping 160.0.23.3 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 160.0.23.3, timeout is 2 seconds:!!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms And lastly let s confirm that R3 has no NLRI for the internal private network of R1-R2. R3#show ip route begin Gateway Gateway of last resort is not set 160.0.0.0/16 is variably subnetted, 2 subnets, 2 masks C L 160.0.23.0/24 is directly connected, Ethernet0/0 160.0.23.3/32 is directly connected, Ethernet0/0 Because we are trying to route traffic from a network with a private address space out towards the provider (R3 in this case) we need NAT to translate the private IP addresses into the public IP address space allocated by the provider, 160.0.23.2 in this case. Obviously from the title of this workbook the problem will be with NAT but in the real lab exam Cisco is not going to tell you what

the problem is so it is always good practice to go through the process to identify where the problem might be even if you are told what the problem actually is. The most basic component of a NAT configuration is specifying which interfaces are the inside interfaces and which ones are the outside interfaces (unless NVI is used in which case you just simply enable NAT under the interface). The following command will confirm which interfaces are enabled for NAT. R2#show ip nat statistics Total active translations: 0 (0 static, 0 dynamic; 0 extended) Peak translations: 0 Outside interfaces: Ethernet0/0, Ethernet0/1 Inside interfaces: Hits: 0 Misses: 0 CEF Translated packets: 0, CEF Punted packets: 0 Expired translations: 0 Dynamic mappings: -- Outside Destination [Id: 1] access-list 100 interface Ethernet0/1 refcount 0 Total doors: 0 Appl doors: 0 Normal doors: 0 Queued Packets: 0 Both interfaces are specified as being NAT outside interfaces? Doesn t it make sense that E0/0 should be an inside interface and E0/1 is the outside interface rather than them both being NAT outside interfaces? Let s hypothesize that this is part of the problem and test whether changing the E0/0 configuration to be a NAT inside interface fixes the problem. R2(config)#int e0/0 R2(config-if)#no ip nat outside

R2(config-if)#ip nat inside Then verify our configuration. R2#show ip nat statistics Total active translations: 0 (0 static, 0 dynamic; 0 extended) Peak translations: 0 Outside interfaces: Ethernet0/1 Inside interfaces: Ethernet0/0 Hits: 0 Misses: 0 CEF Translated packets: 0, CEF Punted packets: 0 Expired translations: 0 Dynamic mappings: -- Outside Destination [Id: 1] access-list 100 interface Ethernet0/1 refcount 0 Total doors: 0 Appl doors: 0 Normal doors: 0 Queued Packets: 0 This looks better, let s test out hypothesis from R1. You may be tempted to look for more problems for each ticket on the real lab but in my opinion you should try to do the minimum amount of changes to get solve the problem. Don t go looking for faults that aren t there. R1#ping 160.0.23.3 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 160.0.23.3, timeout is 2 seconds:... Success rate is 0 percent (0/5)

Still nothing. Let s run a debug on R2 for NAT (note that this command wasn t supported for me on CSR1KV but it works on IOU). R2#debug ip nat IP NAT debugging is on Then try to ping from R1 again. R1#ping 160.0.23.3 repeat 1 Type escape sequence to abort. Sending 1, 100-byte ICMP Echos to 160.0.23.3, timeout is 2 seconds:. Success rate is 0 percent (0/1) However no debug output showed up on R2 meaning that R2 could not identify a translation policy that matched R1 s traffic to R3. Let s look at the policy on R2. R2#show ip nat statistics Total active translations: 0 (0 static, 0 dynamic; 0 extended) Peak translations: 0 Outside interfaces: Ethernet0/1 Inside interfaces: Ethernet0/0 Hits: 0 Misses: 0 CEF Translated packets: 0, CEF Punted packets: 0 Expired translations: 0 Dynamic mappings: -- Outside Destination [Id: 1] access-list 100 interface Ethernet0/1 refcount 0

Total doors: 0 Appl doors: 0 Normal doors: 0 Queued Packets: 0 If you don t look at this output very often you may miss it but the problem has become apparently when we looked at what the configured NAT translation policy is doing. The configured one is for Outside Destination whereas we want to translate the source IP address from inside local to inside global. If you need a refresher on these terms you can look at the following document. http://www.cisco.com/c/en/us/support/docs/ip/network-address-translation-nat/4606-8.html Before we create a new NAT policy (or change the existing policy), let s confirm that the ACL 100 is matching the traffic that we want. R2#show access-list 100 Extended IP access list 100 10 permit ip 10.0.23.0 0.0.0.255 any Isn t the R1-R2 network 10.0.12.0/24 though? This ACL will never match that traffic. Let s create a new ACL that matches the traffic we actually want, which is IP traffic going from 10.0.12.0/24 to any destination. We could parse the config for any more references to this ACL (class maps, packet filtering, etc) to confirm that this ACL is not used by any other IOS functions, or we could simply just create a new ACL. Before creating it, let s confirm that we are not adding to an existing ACL as follows. R2#show access-list 101 Then create the new ACL. R2(config)#access-list 101 permit ip 10.0.12.0 0.0.0.255 any

Seeing as we now believe the problem to be with the NAT policy configuration, we can use a show run to grab the existing the NAT policy, copy it to notepad (incase it is not actually the fix and you need to reapply the configuration), remove it, and add our new configuration. R2#show run s ip nat ip nat inside ip nat outside ip nat source list 100 interface Ethernet0/1 overload Then remove the configuration. R2(config)#no ip nat source list 100 interface Ethernet0/1 overload Then think about what we want to do: we want to translate the source IP address of inside traffic, which is 10.0.12.0/24, going to any destination out of the outside interface, which is E0/1 (we could also use a pool but its extra configuration that is unnecessary to fix this problem. Using these key pieces of information, we can configure the following (given that we have already configured a new ACL). R2(config)#ip nat inside source list 101 interface Ethernet0/1 overload Let s run the debug on R2 again. R2#debug ip nat IP NAT debugging is on Then try to ping from R1 to R3. R1#ping 160.0.23.3 repeat 1 Type escape sequence to abort. Sending 1, 100-byte ICMP Echos to 160.0.23.3, timeout is 2 seconds:

! Success rate is 100 percent (1/1), round-trip min/avg/max = 5/5/5 ms And on R2 we notice the following debug output. *Jan 17 02:08:36.626: NAT*: s=10.0.12.1->160.0.23.2, d=160.0.23.3 [17] *Jan 17 02:08:36.628: NAT*: s=160.0.23.3, d=160.0.23.2->10.0.12.1 [17] This is a good because it is showing us that R2 is translating 10.0.12.1 (inside local) to 160.0.23.2 (inside global), and then back again on the return packet. At this point we can consider this ticket solved!

Scenario #2 Solutions The acceptance criteria of this ticket from the perspective of R3 is that it can connect to 160.0.23.2 on TCP port 80 and have the web server present some HTTP response (in this case, we can simply just use GET for the document root). Let s try to test this acceptance criteria as follows. R3#telnet 160.0.23.2 80 Trying 160.0.23.2, 80... % Connection refused by remote host The remote end (R2 in this case) sent back a RST so we can already confirm that connectivity is working to R2, so we can assume basic things like routing, ARP, etc, is working okay. Let s now confirm on R1 that the HTTP service is running and it has NLRI to the outside network. R1#show ip route Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP + - replicated route, % - next hop override Gateway of last resort is 10.0.12.2 to network 0.0.0.0 R* 0.0.0.0/0 [120/1] via 10.0.12.2, 00:00:24, Ethernet0/0 1.0.0.0/32 is subnetted, 1 subnets C 1.1.1.1 is directly connected, Loopback0 10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks C L 10.0.12.0/24 is directly connected, Ethernet0/0 10.0.12.1/32 is directly connected, Ethernet0/0

And now check that the HTTP service is running. R1#show control-plane host open-ports Active internet connections (servers and established) Prot Local Address Foreign Address Service State tcp *:23 *:0 Telnet LISTEN tcp *:8080 *:0 HTTP CORE LISTEN The above command didn t work on CSR1KV but did on IOU, so an alternative command you can use to verify that the port is listening is as follows. R1#show ip http server status HTTP server status: Enabled HTTP server port: 8080 HTTP server active supplementary listener ports: HTTP server authentication method: enable HTTP server digest algorithm: md5 HTTP server access class: 0 HTTP server base path: HTTP server help root: Maximum number of concurrent server connections allowed: 5 Server idle time-out: 180 seconds Server life time-out: 180 seconds Maximum number of requests allowed on a connection: 1 HTTP server active session modules: ALL HTTP secure server capability: Present HTTP secure server status: Disabled HTTP secure server port: 443 HTTP secure server ciphersuite: 3des-ede-cbc-sha des-cbc-sha rc4-128-md5 rc4-128- sha HTTP secure server client authentication: Disabled HTTP secure server trustpoint:

HTTP secure server active session modules: ALL At this point we know two things: 1. R1 is listening on TCP port 8080 2. R3 is attempting to connect to the web service on TCP port 80 using the IP address 160.0.23.3. From this information we can deduce that R2 needs is configured to translate the destination IP address of 160.0.23.2 and TCP port 8080 to R1 on TCP port 80. This is referred to as port address redirection or port forwarding. As we did with the last scenario, let s confirm that there are NAT inside and outside interfaces, or that NVI is configured. R2#show ip nat statistics Total active translations: 1 (1 static, 0 dynamic; 1 extended) Peak translations: 1, occurred 00:00:06 ago Outside interfaces: Ethernet0/1 Inside interfaces: Ethernet0/0 Hits: 0 Misses: 0 CEF Translated packets: 0, CEF Punted packets: 0 Expired translations: 0 Dynamic mappings: Total doors: 0 Appl doors: 0 Normal doors: 0 Queued Packets: 0

According to the diagram and the original HTTP connectivity test we did at the start, this looks okay. We also notice that a static NAT policy exists so let s see what that looks like as follows. R2#show ip nat translations Pro Inside global Inside local Outside local Outside global tcp --- --- 1.1.1.1:8080 160.0.23.2:80 This is saying that R2 is configured to translate between the outside local address of 1.1.1.1 and TCP port 8080 to the outside global address of 160.0.23.3 and TCP port 80. Referring to the documentation and remembering our NAT terms for inside, outside, local, and global, does this configuration make sense? Aren t the addresses 1.1.1.1 (R1 s loopback) and 160.0.23.3 (R2 s interface connected to the R2-R3 network) both considered inside from the perspective of NAT? At this point we can deduce that there is a mistake in the configuration and I am going to purposely use show run here to show why this is often a bad idea to troubleshoot using that command. R2#show run s ip nat ip nat inside ip nat outside ip nat outside source static tcp 160.0.23.2 80 1.1.1.1 8080 extendable Your instincts might be to assume that the sole problem of this configuration is the outside keyword in the NAT policy and that the fix is to replace that keyword with inside. This is because you what you are translating from is usually listed first in the configuration and what you are translating to is listed second. R2(config)#no ip nat outside source static tcp 160.0.23.2 80 1.1.1.1 8080 extendable R2(config)#ip nat inside source static tcp 160.0.23.2 80 1.1.1.1 8080 extendable Now we verify our configuration. R2#show ip nat translations Pro Inside global Inside local Outside local Outside global tcp 1.1.1.1:8080 160.0.23.2:80 --- ---

But this doesn t make sense because 1.1.1.1 is not an address assigned by a provider that represents one or more private hosts to the outside world 160.0.23.2 is. More specifically, 1.1.1.1 is not an inside global address and 160.0.23.2 is not an inside local address; 1.1.1.1 is the inside local address and 160.0.23.2 is the inside global address. Once again, if you used the configuration solely to isolate this problem then you may miss this. The correct configuration is actually as follows. R2(config)#no ip nat inside source static tcp 160.0.23.2 80 1.1.1.1 8080 extendable R2(config)#ip nat inside source static tcp? A.B.C.D Inside local IP address R2(config)#ip nat inside source static tcp 1.1.1.1 8080? A.B.C.D Inside global IP address interface Specify interface for global address R2(config)#ip nat inside source static tcp 1.1.1.1 8080 160.0.23.2 80 extendable Now verify the configuration. R2#show ip nat translations Pro Inside global Inside local Outside local Outside global tcp 160.0.23.2:80 1.1.1.1:8080 --- --- This is looking much better. Let s run the following debug on R2 before testing it out from R3. R2#debug ip nat IP NAT debugging is on Then test from R3.

R3#telnet 160.0.23.2 80 Trying 160.0.23.2, 80... % Connection timed out; remote host not responding This is different to what we saw earlier; we are timing out the connection request rather being refused. Let s see what the debugs on R2 look like. R2# *Jan 17 03:03:07.213: NAT*: TCP s=48361, d=80->8080 *Jan 17 03:03:07.213: NAT*: s=160.0.23.3, d=160.0.23.2->1.1.1.1 [42624] R2# *Jan 17 03:03:09.213: NAT*: TCP s=48361, d=80->8080 *Jan 17 03:03:09.213: NAT*: s=160.0.23.3, d=160.0.23.2->1.1.1.1 [42624] R2# *Jan 17 03:03:13.213: NAT*: TCP s=48361, d=80->8080 *Jan 17 03:03:13.213: NAT*: s=160.0.23.3, d=160.0.23.2->1.1.1.1 [42624] R2# *Jan 17 03:03:21.213: NAT*: TCP s=48361, d=80->8080 *Jan 17 03:03:21.213: NAT*: s=160.0.23.3, d=160.0.23.2->1.1.1.1 [42624] We already confirmed earlier that R1 should has a default route so reachability back to 160.0.23.3 shouldn t be a problem. Can we reach R1 s loopback from R2? R2#show ip route 1.1.1.1 % Network not in table R2#show ip route 0.0.0.0 % Network not in table We could add a static route to R1 s loopback but the scenario already told us that we couldn t do that. We also noticed earlier that R1 was receiving a default route via RIP from R2 so let s see what R1 itself is advertising via RIP.

R1#show ip protocols *** IP Routing is NSF aware *** Routing Protocol is "rip" Outgoing update filter list for all interfaces is not set Incoming update filter list for all interfaces is not set Sending updates every 30 seconds, next due in 6 seconds Invalid after 180 seconds, hold down 180, flushed after 240 Redistributing: rip Default version control: send version 2, receive version 2 Interface Send Recv Triggered RIP Key-chain Ethernet0/0 2 2 Automatic network summarization is not in effect Maximum path: 4 Routing for Networks: 10.0.0.0 Routing Information Sources: Gateway Distance Last Update 10.0.12.2 120 00:00:03 Distance: (default is 120) As it turns out, RIP is the only IGP we are running and it is not advertising the 1.1.1.1/32 network, therefore R2 has no knowledge of how to get to 1.1.1.1 after it translates the destination IP address from 160.0.23.2 to 1.1.1.1. Therefore, it is important to understand here that the NAT operation works but the routing does not. Let s enable RIP on all subnets in 1.0.0.0/8. R1(config)#router rip R1(config-router)#network 1.0.0.0 Wait for the flash update to occur on R1 and for R2 to process and act on the update and once this occurs we can confirm that R2 has NLRI to 1.1.1.1/32

R2#show ip route 1.1.1.1 Routing entry for 1.1.1.1/32 Known via "rip", distance 120, metric 1 Redistributing via rip Last update from 10.0.12.1 on Ethernet0/0, 00:00:19 ago Routing Descriptor Blocks: * 10.0.12.1, from 10.0.12.1, 00:00:19 ago, via Ethernet0/0 Route metric is 1, traffic share count is 1 Now we can retry the test on R3. R3#telnet 160.0.23.2 80 Trying 160.0.23.2, 80... Open GET / HTTP/1.1 400 Bad Request Date: Sat, 17 Jan 2015 03:16:20 GMT Server: cisco-ios Connection: close Accept-Ranges: none 400 Bad Request [Connection to 160.0.23.2 closed by foreign host] At this point we can consider this ticket solved too!

Scenario #3 Solutions The troubleshooting ticket has already indicated that basic connectivity between R1 to R2, and from R1 to R3 is working correctly. The problem occurs when R1 sources a ping from its loopback to either of these routers and we are not allowed to simply advertise the 10.0.1.0/24 through a routing protocol or use static routing to fix the problem. More specifically, the problem occurs when we are trying to source the ping from a private network to a public address space. We can confirm this as follows. R1#ping 2.2.2.2 source loopback0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds: Packet sent with a source address of 10.0.1.1... Success rate is 0 percent (0/5) R1#ping 3.3.3.3 source loopback0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds: Packet sent with a source address of 10.0.1.1... Success rate is 0 percent (0/5) Just like in the other scenarios, we first verify which interfaces are configured as NAT inside and outside, or whether NVI has been configured. R1#show ip nat statistics Total active translations: 0 (0 static, 0 dynamic; 0 extended) Peak translations: 0 Outside interfaces: Ethernet0/0, Ethernet0/1 Inside interfaces: Hits: 0 Misses: 0

CEF Translated packets: 0, CEF Punted packets: 0 Expired translations: 0 Dynamic mappings: -- Inside Source [Id: 1] access-list ISPR2 pool ISPR2_POOL refcount 0 pool ISPR2_POOL: netmask 255.255.255.0 start 160.0.12.1 end 160.0.12.1 type generic, total addresses 1, allocated 0 (0%), misses 0 [Id: 2] access-list ISPR3 pool ISPR3_POOL refcount 0 pool ISPR3_POOL: netmask 255.255.255.0 start 160.0.13.1 end 160.0.13.1 type generic, total addresses 1, allocated 0 (0%), misses 0 Total doors: 0 Appl doors: 0 Normal doors: 0 Queued Packets: 0 Immediately we see a problem: there are no inside interfaces. We need to first enable R1 s loobpack0 interface as a NAT inside interface. R1(config)#interface loop0 R1(config-if)#ip nat inside Then verify the result of our configuration. R1#show ip nat statistics Total active translations: 0 (0 static, 0 dynamic; 0 extended) Peak translations: 0 Outside interfaces: Ethernet0/0, Ethernet0/1

Inside interfaces: Loopback0 Hits: 0 Misses: 0 CEF Translated packets: 0, CEF Punted packets: 0 Expired translations: 0 Dynamic mappings: -- Inside Source [Id: 1] access-list ISPR2 pool ISPR2_POOL refcount 0 pool ISPR2_POOL: netmask 255.255.255.0 start 160.0.12.1 end 160.0.12.1 type generic, total addresses 1, allocated 0 (0%), misses 0 [Id: 2] access-list ISPR3 pool ISPR3_POOL refcount 0 pool ISPR3_POOL: netmask 255.255.255.0 start 160.0.13.1 end 160.0.13.1 type generic, total addresses 1, allocated 0 (0%), misses 0 Total doors: 0 Appl doors: 0 Normal doors: 0 Queued Packets: 0 Let s see if this has fixed the problem using a general NAT debug and ping. R1#debug ip nat IP NAT debugging is on R1#ping 2.2.2.2 source loopback0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds: Packet sent with a source address of 10.0.1.1... Success rate is 0 percent (0/5)

However we saw nothing in the debug output so this is leaning us towards a problem with the NAT policy configuration. You are probably getting sick of this command already but it is useful for troubleshooting NAT! R1#show ip nat statistics Total active translations: 0 (0 static, 0 dynamic; 0 extended) Peak translations: 0 Outside interfaces: Ethernet0/0, Ethernet0/1 Inside interfaces: Loopback0 Hits: 0 Misses: 0 CEF Translated packets: 0, CEF Punted packets: 0 Expired translations: 0 Dynamic mappings: -- Inside Source [Id: 1] access-list ISPR2 pool ISPR2_POOL refcount 0 pool ISPR2_POOL: netmask 255.255.255.0 start 160.0.12.1 end 160.0.12.1 type generic, total addresses 1, allocated 0 (0%), misses 0 [Id: 2] access-list ISPR3 pool ISPR3_POOL refcount 0 pool ISPR3_POOL: netmask 255.255.255.0 start 160.0.13.1 end 160.0.13.1 type generic, total addresses 1, allocated 0 (0%), misses 0 Total doors: 0 Appl doors: 0 Normal doors: 0 Queued Packets: 0

If we consider the first one isolation (ID of 1), it is referencing an ACL called ISPR2 for matching traffic and is translating it to an IP address in the pool called ISPR2_POOL. We can already see from above that the ISPR2_POOL has the range of 160.0.12.1 to 160.0.12.1, which is the IP address configured on R1 s E0/0 interface. Let s confirm the ACL now. R1#show access-list ISPR2 Noting shows up. Do we have any access-lists at all (i.e. maybe they are configured but are referenced incorrectly)? R1#show access-lists So we have multiple NAT pools but do not have any criteria to select which one to use. The answer to this fix is route maps. The NAT policy configuration should be referencing route maps here, selecting on the ACLs or the outgoing interface. Because we cannot configure any new ACLs, we must use route maps where we can match on the outgoing interface or next-hop of the traffic to select which NAT pool to use. Let s confirm whether any route maps have been configured. R1#show route-map route-map ISPR2, permit, sequence 10 Match clauses: interface Ethernet0/1 Set clauses: Policy routing matches: 0 packets, 0 bytes route-map ISPR3, permit, sequence 10 Match clauses: interface Ethernet0/0 Set clauses: Policy routing matches: 0 packets, 0 bytes

We can see what the network administrator has done here. They have referenced ISPR2 and ISPR3 as an ACL when they should have been referenced as a route map. Let s grab the configuration and change the reference to an ACL to reference a route map. R1#show run s ip nat ip nat inside ip nat outside ip nat outside ip nat pool ISPR2_POOL 160.0.12.1 160.0.12.1 netmask 255.255.255.0 ip nat pool ISPR3_POOL 160.0.13.1 160.0.13.1 netmask 255.255.255.0 ip nat inside source list ISPR2 pool ISPR2_POOL ip nat inside source list ISPR3 pool ISPR3_POOL Then remove the existing policies. R1(config)#no ip nat inside source list ISPR2 pool ISPR2_POOL R1(config)#no ip nat inside source list ISPR3 pool ISPR3_POOL Then we simply use the same commands as above except that we change the list keyword to route-map. R1(config)#ip nat inside source route-map ISPR2 pool ISPR2_POOL R1(config)#ip nat inside source route-map ISPR3 pool ISPR3_POOL Now we run a generalised NAT debug and try to test the original pings again. R1#debug ip nat IP NAT debugging is on R1#ping 2.2.2.2 source loopback0 repeat 1 Type escape sequence to abort.

Sending 1, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds: Packet sent with a source address of 10.0.1.1. Success rate is 0 percent (0/1) *Jan 17 03:58:04.926: NAT: s=10.0.1.1->160.0.13.1, d=2.2.2.2 [20] Hmm, that didn t work. Let s see what happens when we try to ping R3. R1#ping 3.3.3.3 source loopback0 repeat 1 Type escape sequence to abort. Sending 1, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds: Packet sent with a source address of 10.0.1.1. Success rate is 0 percent (0/1) *Jan 17 03:59:34.822: NAT: s=10.0.1.1->160.0.12.1, d=3.3.3.3 [21] Can you see what the problem is? If you aren t used to the above debug output maybe the following will make it clearer on R2. R2#debug ip icmp ICMP packet debugging is on Then run the test again on R1 to R2. R1#ping 2.2.2.2 source loopback0 repeat 1 Type escape sequence to abort. Sending 1, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds: Packet sent with a source address of 10.0.1.1. Success rate is 0 percent (0/1)

And observe the debug output on R2. R2# *Jan 17 04:01:48.323: ICMP: echo reply sent, src 2.2.2.2, dst 160.0.13.1, topology BASE, dscp 0 topoid 0 The source IP address on R1 is being translated from 1.1.1.1 to 160.0.13.1, which is the IP address on the R1-R3 segment, and a similar thing is happening when R1 pings R3 because the source IP address is being translated from 1.1.1.1 to 160.0.12.1 which is the IP address configured on R1 s E0/0 interface on the R1-R2 segment. That is, the route map is configured incorrectly. R1#show route-map route-map ISPR2, permit, sequence 10 Match clauses: interface Ethernet0/1 Set clauses: Policy routing matches: 0 packets, 0 bytes route-map ISPR3, permit, sequence 10 Match clauses: interface Ethernet0/0 Set clauses: Policy routing matches: 0 packets, 0 bytes If we compare this to the IP addresses configured on the interfaces, which are as follows. R1#show ip interface brief exc una down Interface IP-Address OK? Method Status Protocol Ethernet0/0 160.0.12.1 YES manual up up Ethernet0/1 160.0.13.1 YES manual up up Loopback0 10.0.1.1 YES manual up up NVI0 10.0.1.1 YES unset up up

We can see that the route maps have the wrong outgoing interfaces. The ISPR2 route map should be referencing E0/0 and the ISPR3 route map should be referencing E0/1. Let s fix this as follows. R1(config)#route-map ISPR2 permit 10 R1(config-route-map)#no match interface R1(config-route-map)#match interface E0/0 R1(config)#no route-map ISPR3 permit 10 R1(config)#route-map ISPR3 permit 10 R1(config-route-map)#match interface E0/1 Note that there are a few ways you can change the configuration. Let s now verify the configuration on R1 when it tries to ping R2 and R3 sourcing the packet from its loopback interface. R1#ping 2.2.2.2 source loopback0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds: Packet sent with a source address of 10.0.1.1!!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms R1#ping 3.3.3.3 source loopback0 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds: Packet sent with a source address of 10.0.1.1!!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms Then as a final check verify the NAT translation table as follows. R1#show ip nat translations

Pro Inside global Inside local Outside local Outside global icmp 160.0.12.1:11 10.0.1.1:11 2.2.2.2:11 2.2.2.2:11 icmp 160.0.13.1:12 10.0.1.1:12 3.3.3.3:12 3.3.3.3:12 Which looks good!