IPNL: A Simpler and Cheaper Fix for IP

Transcription

1 1 IPNL: A Simpler and Cheaper Fix for IP Paper number: 205 Total number of pages: 14 (Page 14 for appendix) Abstract IP Next Layer (IPNL) is a new internet protocol architecture. IPNL has two advantages over IPv6. First, it solves the site multihoming problem, including issues of scaling, address reassignment, and host address selection. Second, transition from IPv4 is less expensive because IPNL reuses IP in its entirety forever, simply adding a new layer above it to provide a larger address space. IPNL has all of the features required from an internet protocol: mobility, security, multicast and anycast. This paper describes the IPNL architecture, contrasts it with IPv6, and evaluates the performance of an IPNL implementation. I. INTRODUCTION This paper proposes a low-cost solution to the IP address depletion problem. The protocol, called IPNL (for IP Next Layer), is proposed as a cheaper and more flexible alternative to IPv6. IPNL is architected as an extension of IP (Figure 1). The global IP infrastructure, including DNS, stays intact, and is used as is. IPNL adds new infrastructure to the edges, and incorporates existing private IP realms running behind NAT boxes. The IP address space is extended, with the existing global IP addresses serving as the prefix of IPNL addresses. IPNL header is encapsulated in the traditional IP header, and IPNL protocol exists as a shim layer between IP and TCP/UDP layers in a typical protocol stack. Global IP and DNS Global IP and DNS IP Realm IP host IPNL host IPNL router 32 Global IP TCP/UDP IP link 32 Global IP Realm Local IP Extend IP Address TCP/UDP IPNL IP link Add New Protocol Layer autonomous from the rest of the Internet intra-site packets contain no global IP address prefix, and an internal router s forwarding tables contain no global IP address prefixes (even those of the site s own global IP addresses). This means that an ISP can renumber a global IP address without even informing the sites behind that address, and that sites can multihome better, for example. IPNL adds a low-cost, completely-localized, intra-site routing architecture that provides all the traditional benefits of routing, such as scalability, manageability, load-balancing, and failover. Together with its flexible addressing architecture, this routing architecture enables all hosts (not just intra-site hosts) to use the extended address space to robustly communicate with each other end-to-end, even when the addresses they use for a particular connection change for whatever reason, such as when the host s site changes providers, or when a provider renumbers the site s global prefixes for better address aggregation, or when the path used for the connection fails and another path is established, or because of mobility. Additionally, IPNL supports multicast, host auto-configuration, mobility, and anycast. Also, unlike NAT, every host is directly addressable from every other host. IPNL is a low cost solution to the address depletion problem. No IP routers ever need to be upgraded in particular, IPNL does not require an ISP s infrastructure to be altered in any way. There are no changes to any protocols below IP, or to IP itself. Protocol stack changes are limited to end-hosts, NATs, and firewalls only. There is no new global address space (or corresponding new name space for reverse DNS lookups). IPNL simply extends the existing IP address space. New DNS record types are useful but, strictly speaking, not required. IPNL is easy to manage. There is no tunnel or overlay configuration. There is no new host configuration a host needs only its IP address, the address of a DNS server, and, optionally, a domain name if it wishes to be able to accept connections; there is support for host auto-configuration as well. In fact, any two sites on the Internet can directly take advantage of the large address space provided by IPNL without requiring any tunnel configuration and maintenance, and without requiring changes to any infrastructure typically not under their control, such as their ISP s routers, DNS records, wide-area routing protocols, etc. Extend Edges of Infrastructure Fig. 1. IPNL: An Extension of IP IPNL is a complete solution. It extends the IP address space to 10 bytes. IPNL s addressing architecture provides a globally aggregatable address space even while supporting multihoming. It also eliminates site renumbering even when global address prefixes are renumbered. These two important unique benefits result because, in IPNL, a site s internal operation is completely II. IPNL ARCHITECTURE The basic architecture of IPNL is that of IP realms interconnected by IPNL routers (called nl-routers henceforth). IP within each realm is fully self-contained, and operates unchanged, with no awareness of IPNL above it. Nl-routers appear to be normal hosts from the perspective of IP. An important architectural feature of IPNL is that IP addresses in IP headers never cross realm boundaries. The topology of the IPNL Internet is that of a huge global and public middle realm interconnecting much smaller pri-

2 vate realms (Figure 2). The middle realm is a continuation of today s global public IP Internet, and can have directly attached hosts, as well as private realms. Multiple private realms may be reachable through a single middle realm IP address (MRIP); an MRIP address is assigned to an nl-router s network interface that attaches multiple private realms to the middle realm. In Figure 2, we see three private realms using a single MRIP address to reach the middle realm. We also see three separate private realms, and an IPNL host, each using a distinct MRIP address to connect to the middle realm. Network interfaces with MRIP Addresses (Directly attached) Hosts Private realm Realm Group Global Internet (Middle Realm) Private realm IP Realms Private realm (Attached through a private realm) Fig. 2. IPNL Topology Private realm Private realm Private realm This topology looks very much like today s Internet where private realms are separated from the global Internet, and from each other by NAT boxes. IPNL is designed to fit easily into today s operational style, and to solve the problems that come about from trying to operate meshed or cascaded NATs. In addition to the fast disappearing IP address space, post- CIDR IP has a problem with address management because address aggregation occurs above the level of sites. This means that any given site cannot arbitrarily choose its own address prefix it must use address prefixes associated with its Internet access points. If it changes access points, it must get new addresses. If it is multihomed, it must either have multiple address prefixes, or a single prefix that is not aggregatable. There are, fundamentally, two approaches to both these problems. IPv6 s approach is to replace the current 32-bit IP addresses with new 128-bit IPv6 addresses, assign multiple prefixes to all routers and hosts in a site, and renumber them when access points change. With this approach, IPv6 addresses are always carried intact end to end. IPNL s approach, on the other hand, is to incrementally add to the IPv4 address space, and completely isolate local addressing from global address prefixes (i.e., its MRIP addresses). With this approach, there is no site renumbering, but portions of the IPNL address must be translated end to end. In this sense, IPNL has certain similarities with NAT. The benefits of address translation (isolation from global addressing) are well known, and include addressing autonomy, service provider independence, privacy, better multi-homing, and tighter security for subscribers, and better address management and aggregation for providers. The problems associated with doing simple Network Address Translation are equally well known addressing asymmetry, limited scalability, and no resilience to network failures. IPNL provides symmetric addressing, fault-tolerance, mobility, and scalability through its innovative addressing and routing architecture, while still ensuring all the benefits of traditional network address translation. However, if one uses address translation, ultimately, it is imperative that there is a fully-overloaded address carried intact end-to-end, if one wants a flexible symmetric communication facility. By fully overloaded, we mean an address that, like IP and IPv6, serves to both persistently identify and locate an end host. Unlike IP and IPv6, however, this address should allow sites to isolate themselves from global addressing. Towards this end, IPNL uses a second address that has these two characteristics. That is, it is both fully overloaded and independent of a site s Internet access point. This second address is none other than the FQDN (Fully Qualified Domain Name). However, it is too costly to use FQDNs in every IPNL packet header, even if they are to be examined only by edge boxes, due to their inherently variable-length and mis-aligned nature. To deal with this problem, IPNL uses short-term, fixed-length addresses (henceforth called IPNL addresses) exclusively in almost all packets; parts of these IPNL addresses may be translated end-to-end. In the rest of the packets, IPNL uses IPNL addresses together with FQDNs whose purpose is to help automatically learn and maintain these IPNL addresses, and any required translations. Figure 3 shows how the IPNL packet header looks; the Appendix gives the header details IPNL Addresses Optional FQDN Header Fig. 3. Fixed Length IPNL Addresses and Optional Variable Length FQDNs We describe how IPNL uses IPNL addresses for routing and addressing purposes in Section III, and how it uses FQDNs to learn and maintain these addresses in Section IV. III. IPNL ADDRESSING AND ROUTING The IPNL address structure is a direct reflection of the topology. It consists of three fields: 1. the Middle Realm IP Address (abbreviated as MRIP, 32 bits), 2. the Realm Number (abbreviated as RN, 16 bits), and 3. the End Host IP Address (abbreviated as EHIP, 32 bits) The Realm Number is used to aggregate multiple private IP realms; this aggregated set of realms is called a realm group. By definition, each realm within a realm group has a unique realm number. The MRIP address is a globally-unique IP address used by a router on the interface that attaches a private realm group to the middle realm, while the EHIP address is a locallyunique, and realm-specific IP address used by hosts within a

3 realm. The MRIP and EHIP addresses are normal CIDR IP addresses in the context of their respective realms. IPNL provides sufficient addressing and routing flexibility a host can have multiple EHIP addresses, a router can have multiple MRIP addresses, and multiple routers can be used together for multihoming, load-balancing, and failover. Nl-routers connect two IP realms together, and fall into one of three categories, depending on which IP realms they interconnect. A frontdoor IPNL router (called simply a frontdoor henceforth) uses at least one MRIP address to connect a realm group to the middle realm (the global Internet). A backdoor IPNL router (called a backdoor henceforth) interconnects realm groups with potentially overlapping realm numbers. An interior IPNL router (called an interior nl-router henceforth) interconnects multiple private realms within the same realm group. Realm Groups with 1 realm each Backdoor a.com 1 Endhost IP Address (EHIP) 1 b.com Middle Realm IP Address (MRIP) Frontdoors Realm Number (16 bits) cs.xyz.edu Fig. 4. IPNL Address Structure Realm Group with 5 realms 5 ee.xyz.edu 2 math.xyz.edu me.xyz.edu phy.xyz.edu Interior nl-routers This addressing architecture is illustrated in Figure 4. Here, we see three realm groups, one of which has five realms, while the remaining two have only one realm each. In this example, each of the five realms in the realm group houses hosts belonging to one particular DNS zone, where each zone represents a department within a university. These realms are interconnected by interior nl-routers. The entire realm group (a university, in this case) has two connections to the middle realm through two different frontdoors, each with a distinct MRIP address; the realm group could be multi-homed, or both frontdoor routers could be connected via the same ISP. We also see two companies, a.com and b.com, connected privately via a backdoor, in addition to being connected directly to the middle realm via frontdoors. With a three-field hierarchical address structure, nl-routers require only a minimum of routing information. They let IP routing do all the work of carrying packets within a realm, and only need to know how to route packets across realms. They learn this information by running a BGP-like path vector routing protocol within their local realm group to advertise and propagate realm reachability information. Interior nl-routers only need to know how to route packets between realms whose realm numbers are advertised to them. One of these advertised realms is the middle realm, whose reachability information is advertised by the frontdoors; the middle realm can be conceptually thought of as a common realm present in every realm group, and addressed using a globally unique and reserved realm number. If the destination realm number of a packet is known to the interior nl-routers attached to the source realm, the packet is routed directly to the destination realm. Once the packet reaches its private destination realm, the attached interior nl-router simply writes the IPNL destination EHIP field into the destination IP address field of the encapsulating IP header, and forwards it on to the destination. On the other hand, a packet destined for a non-local private realm (a realm for which interior routers of the source realm do not have reachability information) is default routed to the middle realm by the interior nl-routers within the source realm group, using the routes advertised by the frontdoors. The frontdoor that happens to receive this packet, called the egress frontdoor from the packet s perspective, then forwards this packet across the middle realm by writing the IPNL destination MRIP field into the destination IP address field of the encapsulating IP header, and injecting the resulting IP-encapsulated IPNL packet into the middle realm (the global Internet). Conventional IP routing within the middle realm then gets the packet to the frontdoor (called the ingress frontdoor) indicated in the packet s destination IPNL address. The frontdoor examines the IPNL address in the packet header, and routes on the destination realm number field to an interior nl-router router attached to the packet s destination realm, from where it is forwarded to the destination host within that realm. Thus, conceptually, packets are encapsulated and decapsulated within traditional IP packets with each realm crossing. One of the main features of IPNL is that it is designed to perform the entire forwarding scalably, and robustly. Depending on how an nl-router uses the three-field IPNL address, packets can be classified as either globally addressed or locally addressed. Packets are said to be locally addressed if they can be delivered to the destination host without routing on the MRIP field. In other words, the first-hop interior nl-router has a routing table entry for the realm number (RN) of the destination private realm. All other packets those that require the MRIP to route packets on are said to be globally addressed. Thus, locally addressed packets do not have to carry MRIPs at all. In fact, MRIP fields are an optional part of the IPNL header, and are only included in globally addressed packets (the Appendix describes the header format in detail). This is not unlike IPv6 s use of site-local addresses [5]; but see Section VIII for this and other differences with IPv6. Figure 5 illustrates the use of IPNL addresses in the case of locally and globally addressed packets. The source host, with an EHIP address of H2, initiates a global connection to a destination with EHIP H1, and a local connection to another destination with EHIP H3. The global connection uses the tuple (M2,R2,H2) as the source address, and the tuple (M1,R1,H1) as the destination address, so that a packet is uniquely routed from the source to the destination, and vice versa. On the other hand, the local connection only uses (R2,H2), and (R3,H3) as the source and the destination address respectively.

4 IPNL ADDRESS: MRIP RN EHIP S R2 H2 D R3 H3 R3 H3 doors, and go down, without adversely affecting site operations. Thus, the architecture strives to be flexible, manageable, scalable and robust. a.com cs.xyz.edu math.xyz.edu Realm Group S M2 R2 H2 D M1 R1 H1 H1 R1 b.com M1 DNS M2 me.xyz.edu ee.xyz.edu phy.xyz.edu R2 H2 Global Internet (Middle Realm) 1 a.xyz.com 3 2 b.xyz.com c.xyz.com Backdoor NL-router Interior NL-routers H1 Fig. 5. Illustration of global and local routing While, in general, it is true that a locally addressed packet does not cross the middle realm, in cases where realms within a realm group are separated by the middle realm, locally addressed packets do cross the middle realm a frontdoor on the source side of the realm group advertises routes to the interior nl-routers on the source side on behalf of realms on the destination side. Thus, interior nl-routers automatically forward locally addressed packets to the egress frontdoor, which then sends it across the middle realm, just as for globally addressed packets. For flexibility reasons, IPNL includes this possibility, and this is the reason why can not assert that only globally addressed packets have to cross the middle realm, although the converse is true. Note, however, that the routing protocol handles all these cases efficiently, automatically, and robustly. Realm Numbers present in the IPNL header may be translated any time a packet crosses a realm group boundary. This is true for both globally addressed packets, and locally addressed packets traversing realm groups connected together by a backdoor. While the purpose of realm number translation in case of backdoor-connected realm groups is to deal with realm number overlap, the purpose of realm number translation in case of globally addressed packets is to provide further addressing autonomy to sites whose frontdoor is managed by an ISP (there could be multiple frontdoors, each managed by different ISPs as well); thus, a site can renumber its realms, and advertise the new realm numbers to the ISP-maintained frontdoor, which, then, automatically creates a mapping between these actual realm numbers, and a frontdoor-specific realm number used in globally addressed packets. This works because the destination of globally addressed packets does not care what realm the source is in 1, while the destination of locally addressed packets does not generally use a frontdoor. Note that sites with multiple frontdoors do not even have to ensure that the realm numbers used by different frontdoors are consistent a frontdoor can come up, listen to routing advertisements, make up locally consistent realm numbers, forward packets even without knowledge of other front- 1 Only the MRIP is used to get to the frontdoor; the RN is relevant only after the frontdoor injects the packet into the source realm group RN=1 rst.com H2 Fig. 6. Illustration of Realm Number Translation S D S D 3 H1 4 H2 3 H1 1 H2 Figure 6 shows an example of a backdoor communication between two hosts, H1 and H2, that belong to two realm groups. It also shows realm number translation. The source uses H1 as the EHIP, and 3 as the realm number, and communicates with the destination that uses H2 as the EHIP, and appears to use 4 as the realm number, using locally addressed packets. The backdoor translates the source RN field from 1 to 4 as locally addressed packets from H2 flow through it on their way to H1, and the destination RN field from 4 to 1 as packets flow the other way. Note that the backdoor can automatically make up these realm numbers by simply listening to realm reachability routing advertisements, and deducing that realm number 4 is not being used within H1 s realm group. Also, just as in the case of frontdoors, if there were two backdoors serving traffic between H1 and H2, these two do not even have to know about each other at all; they can make up and announce their own realm numbers for reaching H2 s realm. And, if any one of them fails, the routing protocol ensures that the other backdoor automatically takes over. The advantages of aggregating realms into a realm group, and of connecting two realm groups together through a backdoor that provides realm number translation are as follows. Note that each realm within a realm group is identified by its realm number, which is unique within the realm group, but is reused across realm groups. Aggregating realms into a realm group, and addressing them using a realm number allows large sites to operate with plenty of address space, and with only a single MRIP (or a few, if multi-homed). This facility also allows sites to easily accommodate existing private realms, and provides administrative flexibility in managing individual realms, much the same way as DNS provides a sub-domain facility for better autonomy. For example, Figure 5 shows a university network where each de-

5 partment gets its own zone and realm; this way, individual departments can number, renumber, and manage their own realms independently, and the whole realm group still functions efficiently; the only co-ordination required is in assigning unique realm numbers to individual realms. Backdoor nl-routers provide a way for hosts in two realms with overlapping realm numbers to talk to each other without having to go through the global Internet. This is useful in situations when two sites have a private link between them, or when two sites merge with each other, but their networks have not yet been renumbered, etc. The term site used in this paper refers to an independently administered network, such as a corporate or a university network, and does not necessarily conform to any IPNL topological entity. In other words, a site may be a fraction of a single private realm, part of a realm group, an entire realm group, or multiple realm groups. A. Encapsulation IPNL s header format takes advantage of the fact that it always runs over IP. For instance, the IPNL header has no hop count field the hop count of received IP packets is decremented by nl-routers and copied into transmitted IP packets. More importantly, the source RN and EHIP fields are placed within the first 8 bytes of the IPNL header. This is done to take advantage of the ICMP protocol specification [13], which requires a host generating an ICMP packet to include the header and the first 64 bits of the IP packet in the body of the ICMP message. This allows nl-routers to determine the original source of packets that triggered an ICMP error message for both locally and globally addressed packets. For instance, if an ICMP error message is received from the middle realm by an egress frontdoor nl-router, the source IP address of the attached original IP header supplies the MRIP, and the RN and EHIP fields in the first 8 bytes of the original datagram supply the rest of the host address, so that the egress frontdoor can, in turn, return an ICMP message to the source. B. IPNL Address Dynamics In this Section, we elaborate on IPNL s approach to coping with failures, and dealing with scalability and manageability issues, while still providing sufficient addressing flexibility and autonomy. In order to provide address independence, and support robust operation, the IPNL header contains two fields that are novel: 1. A Loc field that carries information about where the packet currently is, and 2. A Used Src MRIP field that indicates which MRIP address was last used to forward packets between two hosts communicating across the middle realm. The Loc field helps with address independence, and the Used Src MRIP field is used for providing scalability, and resilience to network failures. In the following two subsections, we elaborate on each of the two features separately. B.1 The Loc field First, we point out that multiple IPNL addresses may be used to identify and locate a destination host during the course of a connection, for reasons of multihoming, renumbering, anycast, or mobility. By a connection, we mean a unique (possibly asymmetric) communication path between two hosts, as seen by end hosts at the IPNL and IP levels. Note that IPNL is a shim network layer protocol, and the set of all multiplexed TCP/UDP flows between two hosts is what we are referring to here as a connection. IPNL hosts are not statically configured with their own MRIPs or Realm Numbers. Instead, these are dynamically learned, per destination, when data packets are transmitted and received. In other words, an IPNL host learns the high-order parts of its own IPNL address(es) at the same time it learns the IPNL address(es) of the destination host. IPNL hosts require no static configuration information other than what they require today for IP. One important benefit of this approach is that it decouples site addressing from provider addressing, much like what NAT does today. This provides greater addressing autonomy and flexibility to sites, and allows providers to achieve better global address aggregation. Also, such an architecture allows subscribers to multihome better, and change providers easily. In order to attain these objectives without sacrificing scalability and robustness to failures, none of a site s internal nl-routers knows the MRIPs of the frontdoor nl-routers of their realm group. Instead, IPNL includes a field in the packet header that is examined and modified by nl-routers each time they forward a packet across realms. As a result, the only configured information that has to change when a site changes ISPs or a site s ISP renumbers is the MRIP address assigned to the middle realm interface of the frontdoor nl-router. If the ISP itself is managing the frontdoor nl-router, a site literally does not even have to be notified that its global prefix (MRIP) is being changed. Since interior nl-routers must be able to determine, from the packet header alone, whether a globally addressed packet must be default routed to the middle realm (i.e., the packet is still in the source realm group), or routed to a local realm that matches the destination RN field in the packet header (i.e., the packet has reached the destination realm group), IPNL uses an additional field in the packet header, called the Loc field, that gives the location of the packet. We require this field because RNs and EHIPs are not unique, and we want to isolate knowledge of the MRIP addresses used by frontdoors, for reasons described above, even from interior nl-routers serving the same realm group this makes it impossible for an interior nl-router to determine whether a packet is outbound or inbound from the global address fields alone. Instead, it uses the Loc field in the packet header for this purpose. If the Loc field indicates that a globally addressed packet is on its way out (the interior nlrouter attached directly to the source realm would set the field this way), the interior nl-router that receives this packet knows that it must be passed along to an nl-router (either another interior nl-router, or a frontdoor, depending on the routing table information) that can get this packet one-hop closer to the middle realm. If, on the other hand, the Loc field indicates that a globally addressed packet has already reached its destination realm group (the ingress frontdoor would set the field this way), the interior nl-router knows that the packet must be forwarded to the realm indicated in the destination IPNL address.

6 a.com b.com LOC= Route by Realm LOC= Middle Realm math.xyz.edu cs.xyz.edu me.xyz.edu ee.xyz.edu phy.xyz.edu LOC= Default LOC= Route by Realm rated by the middle realm. The Used Src MRIP field carried in the header of every IPNL packet is used for this purpose; thus, there are essentially two source MRIP fields in every packet. The first one contains the value set by the source host, and the second (the Used Src MRIP) contains the MRIP of the egress frontdoor of the source realm group actually used to forward this packet; this field is set by the egress frontdoor itself, and is examined by the destination upon packet reception. If this Used Src MRIP value is different than the MRIP value that the destination has previously used to talk to the source on this connection, the destination deduces that the egress frontdoor of the source realm group has changed, and sets the MRIP field of the source s IPNL address to this new value in future communication, thus rapidly recovering from frontdoor failures. We address security concerns about spoofing in Section IV-B, where we discuss IPNL s security features. Fig. 7. Illustration of the Loc Field Figure 7 shows how the Loc field is used by IPNL to identify a packet s current position. For locally addressed packets, the Loc field of a packet is set by the source s interior nl-router to Route by Realm, and remains set that way. For globally addressed packets, the interior nl-router sets it to Default, and default routes the packet to an egress frontdoor. The frontdoor then sets it to Middle Realm, and forwards the packet across the middle realm to the ingress frontdoor indicated in the destination address. The ingress frontdoor finally sets it to Route by Realm, and forwards it onto the destination. y.b.com H2 M2 M1 M4 M3 D S Used Src MRIP M2,R2, H2 M1,R1,H1 M1 D S Used Src MRIP M1,R1, H1 M2,R2,H2 M2 D S Used Src MRIP M2,R2, H2 M1,R1,H1 M3 x.a.com H1 M1 crashes B.2 The Used Src MRIP field IPNL ensures scalable and robust communication by adhering to the well-tested principle of maintaining as much state as possible at the end hosts, and communicating this state, if required, in the packet header. Since IPNL uses IP addressing and routing to traverse a realm, it enjoys the same benefits provided by IP routing as far as packet forwarding within a realm is concerned. Since the interior nl-routers run a routing protocol within their local realm group to communicate realm reachability information, and since an interior nl-router looks up the IPNL address fields in its routing table for the next-hop nl-router to use for crossing the current private realm, no flow state is kept whatsoever; this ensures a scalable and robust protocol operation within realm groups. In the case of sites with multiple frontdoors, the routing protocol running within a realm group also ensures that these frontdoors are load-balanced. Also, a frontdoor does not store any per-flow state either, and this guarantees the overall scalability of the architecture. Finally, IPNL uses an additional field in the packet header, called the Used Src MRIP field, to cope with frontdoor failures. This additional field is required because an existing connection should switch over to a new frontdoor (if available) in case the current frontdoor becomes non-operational. However, while packets from the host within the same realm group as the frontdoor are automatically routed to the new front-door, packets from the other host must be explicitly directed to use the new frontdoor this is, of course, because the two hosts are sepa- D S Used Src MRIP M1,R1, H1 M2,R2,H2 M2 Fig. 8. Illustration of Frontdoor Failover This failover process is illustrated in Figure 8. A host with FQDN x.a.com and EHIP H1 starts communicating across the middle realm with another host with FQDN y.b.com and EHIP H2. It initially uses M1 as the source MRIP address. However, if the frontdoor associated with M1 becomes unavailable for whatever reason, routing within the source realm group directs the next packet from the source to the second frontdoor, which writes the associated MRIP (M3, in this case) into the Used Src MRIP field of the packet header. y.b.com then sets the source s MRIP field of the IPNL address to this new value in future packets. IV. FQDN NAMING AND ROUTING IN IPNL To make up for the loss of long-term stability in IPNL addresses, IPNL uses FQDNs (fully qualified domain names). In IPNL, FQDNs are given an expanded role. Today, they are used as handles with which to look up IP addresses in the distributed DNS directory system. They have no direct relationship with IP (the protocol). IP was invented before DNS. In IPNL, FQDNs are elevated to the role of fully routable addresses. They are optionally carried in IPNL headers, and are used by IPNL as stable and long-term overloaded addresses for

7 packets. While multiple IPNL addresses may be used for routing and identification during the course of a connection, typically, only one FQDN is used. (The only exception is anycast, where multiple FQDNs come into play; one of the FQDNs, however, is always stable during the course of the connection.) Typically, the first packet transmitted by each end of a connection contains the FQDN, which both routes the packet and resolves the IPNL addresses. Subsequent packets carry only the IPNL addresses, because they can be forwarded more efficiently; such packets are addressed and routed as described previously in Section III. The FQDNs, however, are included in the IPNL pseudo-header as part of the IPNL checksum to help guard against inadvertently mis-routed packets. A. FQDN Routing and Addressing IPNL has the following FQDN-based routing and addressing architecture. Every private realm has one or more DNS zones associated with it. Strictly speaking, it is not necessary for all members of the zone to be attached to that realm, but for scaling purposes, by and large, they should be. The realm for a given zone is known as the zone s home realm. As described previously, internal nl-routers know how to route to local private realms using the RN in the IPNL address. For each such private realm, they also know which zones are homed to that private realm. Once again, such knowledge can be acquired by running a routing protocol within a realm group to distribute FQDN suffix information of the zones homed to realms. (Thus, conceptually, there are two separate localized protocols, one for distributing realm number information, and the other for distributing FQDN information). As such, each internal nl-router knows how to route to every local zone. The term local here is defined as those realms and contained zones an nl-router has routes for. For a given nlrouter, not all realms in a realm group need be local, and realms in other realm groups may be local (by importing routing information). Use DNS to cross middle realm c.b.com a.com b.com D DNS Pre-calculated within a Realm Group Zone cs.xyz.edu ee.xyz.edu Fig. 9. Illustration of Use of FQDNs in IPNL B math.xyz.edu A 1 me.xyz.edu C 3 a.phy.xyz.edu phy.xyz.edu Figure 9 shows how IPNL uses FQDNs to bootstrap connections to locally and globally addressed hosts. The FQDN routes b.math.xyz.edu required by the source host a.phy.xyz.edu to reach the destination local host b.math.xyz.edu are known to C, the source s interior nl-router. On the other hand, such FQDN routing information is not available at C for a global packet addressed to c.b.com; in this case, C forwards such a packet to B, a frontdoor for its realm group. B extracts the destination FQDN information from this packet, and uses DNS to look up the MRIP address of D, the frontdoor for the realm group hosting the b.com zone. DNS supplies this 32-bit MRIP address when asked for the address of c.b.com, just as it currently supplies a 32-bit host address; it is the responsibility of the zone administrator for b.com to make sure that DNS address queries for all hosts within b.com return the MRIP address of D. Once B obtains the MRIP address of D, it encapsulates the IPNL packet in a new IP packet addressed to D, and sends it across the middle realm. When D receives this packet, it determines the realm number of c.b.com, and routes the packet to that realm (in this case, the sole realm housing the b.com zone), and, ultimately, to c.b.com. The zone routing information carried by the interior nl-router C in Figure 9 is shown in Table IV-A. Realm Number Zone Next Hop 1 me.xyz.edu (itself) 2 ee.xyz.edu (itself) 3 phy.xyz.edu (itself) 4 math.xyz.edu. nl-router A 5 cs.xyz.edu nl-router A middle realm. nl-router B TABLE I TYPICAL INTERNAL NL-ROUTER ZONE FORWARDING INFORMATION Thus, DNS does most of the work when routing on FQDNs. A frontdoor simply find paths across the middle realm by using DNS exactly as it is used today, and can be thought of as an nl-router attached to a DNS server. When a packet with an FQDN, but with no destination MRIP, arrives at the frontdoor nl-router, the frontdoor hands the packet off (conceptually) to its DNS server side. If the DNS server side already has a cached A 2 record for the FQDN, the IP address is written into the destination MRIP field, as well as into the encapsulating IP header, and the packet is forwarded across the middle realm. If the DNS server side does not have a cached A record, it queues up the packet, and does a normal DNS lookup. When the answer is received, it is cached and used to forward the packet as described above. Like frontdoor nl-routers, internal nl-routers attached to private realms can be viewed as having a DNS server attached as well. However, unlike frontdoors, which use A records cached from DNS lookups, internal nl-routers constantly maintain the full A record database for all zones attached to the private realm they directly serve. This database may be obtained through a zone transfer from the DNS server for each zone, or the nl-router may itself be configured to be a primary DNS server for each zone. When a packet with an FQDN but no destination EHIP arrives 2 The Address record type

8 at an interior nl-router attached to the destination realm, the nlrouter hands the packet off (conceptually) to its DNS server side. The DNS server side looks up the A record for the FQDN, the IP address is written into the destination EHIP field as well as the encapsulating IP header, and the packet is forwarded to the destination host. Y b.com M2 M3 M1 cs.xyz.edu (M2 learned from DNS lookup over MR) S M1 R1 H1 s.phy.xyz.edu S M1 R1 H1 s.phy.xyz.edu S D M2 R2 H2 y.b.com D M2?? y.b.com D S M2 R2 H2 y.b.com D M1 R1 H1 s.phy.xyz.edu S M1 R1 H1 D M2 R2 H2 S M2 R2 H2 D M1 R1 H1 ee.xyz.edu Fig. 10. Illustration of IPNL Address Learning S math.xyz.edu me.xyz.edu phy.xyz.edu?? H1 s.phy.xyz.edu??? y.b.com This connection setup procedure using FQDNs is illustrated in Figure 10. The Figure shows how the first two packets containing the source and destination FQDNs help determine the ephemeral fixed length IPNL addresses. The question marks in the address fields of a packet indicate that the values are unknown at that point. The Figure shows when each of these fields is filled in, as the packet is routed on the FQDNs. Note that the source S does not have to know its own RN; instead, it learns it from the returned FQDN-attached packet automatically. The same procedure is also used during the lifetime of a connection to track any changes in the IPNL local address fields (the RN, and EHIP fields) of the source and the destination hosts; the MRIP change is handled using the Used Src MRIP field (Section III-B.2). Thus, in Figure 10, if S s realm is renumbered, or if S is assigned a new EHIP, then S starts the 2-way FQDN handshake to propagate its new address to the destination. The security of this approach is discussed in Section IV-B. The primary difference between FQDNs and IPNL addresses, in their role as routable addresses, is that IPNL addresses are topologically aggregatable and have topological dependencies. FQDNs do not have topological dependencies and are not topologically aggregatable. This makes FQDNs more flexible in terms of assignment, but makes them less efficient to route on. It is only by piggy-backing on DNS functionality that FQDNs can be scalably used at all. Conceptually, however, IPNL s use of DNS is little different than its use today. It typically only occurs at the beginning of connections, before IPNL addresses have been resolved. Subsequent packets contain IPNL addresses and, so, are efficiently forwarded. In particular, IPNL does not increase the load on DNS. The only difference from today s operation is that a frontdoor nl-router initiates the DNS lookup instead of an end host. Otherwise, the same number of DNS lookups get initiated. The FQDN routing information kept by nl-routers is summarized in Table II Nl-router type Routing Information Way obtained Internal nl-router All hosts in realm Obtained from DNS, and from (tagged as IP or IPNL) IPNL host registration Internal nl-router Next-hop nl-router to Obtained from realm group all local zones routing protocol Backdoor nl-router Next-hop and RN trans- Obtained from static lation for local zones configuration or from in neighbor realm group realm group routing protocol Frontdoor nl-router Next-hop MRIP and RN Obtained from static configuration translation for local or from realm group zones across middle realm routing information Frontdoor nl-router Next-hop MRIPs for Cached from DNS non-local host FQDNs lookups TABLE II SUMMARY OF FQDN ROUTING INFORMATION B. FQDN-based Security FQDNs form a globally unique and persistent name space, and IPNL leverages them to automatically learn and deal with changes in short-term global and local addresses, as described in Section IV-A. However, since the FQDN address space is not unforgeable, there is still a considerable risk of FQDN spoofing and Denial of Service, whereby malicious hosts send fake packets advertising changes to global and local addresses in order to disrupt or redirect existing connections between IPNL hosts. In this Section, we discuss how IPNL deals with this problem. Whenever an IPNL source detects a change in an end-point address, it verifies the originator of the address change announcement by sending out an FQDN-attached packet (see the Appendix for the exact header format of packets with FQDNs) that includes a nonce 3 before delivering the packet up to higher layers (it queues up the packet until the address is verified). Note that these FQDN-attached packets are originated by higher level protocols, such as TCP and UDP; IPNL layer is passive, and simply includes the nonce in the optional FQDN IPNL header of such packets until the destination address is verified; this way, it does not have to worry about losses, etc. In case of FQDNattached global packets, the frontdoor does a DNS lookup on the destination FQDN, just as it would for all other FQDN-attached packets, determines a valid destination MRIP, and forwards the packet containing the nonce to it. FQDN-based routing within the destination realm group would then deliver the packet to the destination, whose IPNL layer simply reflects the nonce back to the source in an IPNL-ICMP message if the destination did, in fact, change its global address. If the destination did not change its address, then it knows that it is being attacked, and use another IPNL-ICMP message type to inform the source to ignore such address change announcements. A similar procedure is followed for locally addressed packets as well. The important point is that, just as in IPv4, while anyone can create a spoofed packet 4, routing guarantees delivery of packets only to the destination indicated in the packet. Thus, as long as the DNS, source and destination realm group routers are not compromised, IPNL can prevent address spoofing. Of course, 3 A large random number used only once 4 since many ISPs do not implement ingress filtering

9 if source group routers are compromised, then the source would be helpless, just as it would be today; likewise, if the destination realm group routers are compromised, all hosts in that particular destination realm group would be vulnerable (but hosts in other destination realm groups would be unaffected). The point is that IPNL preserves the packet delivery to no host other than the packet s destination security guarantee that IPv4 provides today 5, even as it tracks valid address changes, and leaves it to higher layer protocols, or IPSec (with a minor modification not to consider IPNL header in IP packet as payload), to provide extra security, if required. There is one concern about the overhead of IPNL s FQDN checks during address resynchronization. A malicious host indulging in a DoS attack may succeed in its purpose if it forces IPNL to use FQDNs unnecessarily, because of the overhead of FQDN lookups (even if cached) and complex header parsing. IPNL deals with this problem by providing a socket option for individual connections to explicitly turn off FQDN verification altogether. The idea is that a lot of servers, such as web, proxy, and mail servers, may not care about this feature much, and it is typically these servers that are attacked. Thus, an application can tell IPNL the level of desired security; if we do not want to modify an existing application, we can use heuristics, such as rate limiting address changes (they should not be too often), ignoring address checks on server sockets that have small average connection lifetimes, etc. Note that, in case of TCP connections, there is the usual sequence number field that the host can additionally use to determine if a packet is bogus. When a connection is to be established, each end-point can optionally verify that the other end-point is using an MRIP consistent with its FQDN name by asking the local frontdoor for a list of all MRIPs associated with this FQDN in the DNS database. MRIP changes due to frontdoor crashes are also verified this way. Correctly dealing with mobile destinations that may have changed realm groups is slightly more complex, and is described in Section VI-A. C. Resolving IPNL Addresses With IP today, an initiating host resolves a respondent host s address once, before any packets are sent, and not again for the lifetime of the connection. IPNL expands on this usage in two ways: The initiating host s addresses are resolved along with the respondent host s addresses. A new address resolution can take place any time during a connection. The only advance configuration required by an IPNL host in order to achieve this is its EHIP, FQDN, and a domain name server. In other words, there is no new configuration beyond what is required for IP today, in order to provide this extra feature. Everything else is dynamically learned, including the host s own RNs and MRIPs. Moreover, the RNs and MRIPs are continuously relearned, for both source and destination, literally every time a packet is sent, as described in Section IV-A. This eliminates the need for renumbering, or more accurately, it 5 Destination ARP spoofing on the final shared link is not IP s problem, and can be avoided with switching eliminates the need for a renumbering event whereby all hosts and routers need to be reconfigured at the same time. D. Host Auto-configuration and Nl-router Discovery Before a host can use IPNL, it must, in addition to the advance configuration information described above, know the IP addresses of the nl-routers attached to its private realm. It learns them through a well-known DNS name; in our implementation, an IPNL host learns about interior nl-routers by doing an address lookup on the FQDN attached-nl-router.ipnl.net. 6 A zone housed in a private realm must set the A records for this FQDN to be set of all interior nl-routers attached to that private realm. This way, when an IPNL host boots, or when it determines that its list of attached nl-routers may be out of date, it queries DNS for this FQDN to get the current list. Once the host learns the list of available interior nl-routers, it must register with one of them. The host is then ready to communicate with all hosts (both IPv4-only and IPNL) on the Internet. The purpose of this registration is to let the nl-router know that a host is IPNL capable, so that the host can transparently initiate connections to any IPv4-only hosts present in the same realm an interior nl-router knows whether the destination in an FQDN-attached packet that it receives for forwarding is an IPv4-only host in the same realm as the IPNL source or not from the A records database of all zones in its realm, and from the registration information supplied by IPNL hosts; this way, it instructs the IPNL host to use traditional IP to communicate with the destination, if the destination is found to be IPv4-only. Section V describes all other possible transition cases. Thus, the maximum amount of state required at any nl-router is proportional to the number of IPNL hosts, including mobile hosts from other private realms, currently within the private realms they serve. This is not a major problem because private realms are typically small, and many existing firewalls and NAT boxes are already equipped to handle this state. If an IPNL host is attached directly to the middle realm, DNS lookups (done when it boots up, or when it moves to the middle realm) for the FQDN attached-nl-router.ipnl.net is guaranteed to return a null value. This way, it can deduce its location, and communicate with other hosts on the Internet accordingly 7. If an IPNL host boots up in a foreign realm, it finds out the name of the DNS zone used by the interior nl-routers of the realm when it tries to register with one of the interior nl-routers. This realm name information is used in coping with mobility (Section VI- A). Thus, an IPNL host can boot up, get an IP address via DHCP, find out where it is attached automatically, and learn the list of interior nl-routers, if it is attached to a private realm. FQDN auto-configuration is also possible, and various options can be found in a draft report [16]. V. TRANSITION In this Section, we discuss how a mix of IPNL hosts, and IPv4-only hosts talk to each other. We have four cases to con- 6 The zone ipnl.net itself is registered to us, and, thus, we can guarantee that no realm can accidentally use this zone. 7 An IPNL host on the middle realm must perform some frontdoor functions, such as doing DNS lookups, etc., as well