Scaling the Inter-Domain Routing System

Transcription

1 Scaling the Inter-Domain Routing System Kaustubh Gadkari Computer Science Department Colorado State University Abstract The Internet inter-domain routing system is facing scalability issues due to the increasing sizes of the routing and forwarding tables. Several approaches have been suggested to scaling the sizes of the routing and forwarding tables. In this report, I undertake a survey of the suggested approaches and suggest a taxonomy for classifying the approaches. I then discuss the approaches in the various classes and also discuss the pros and cons of each class of solutions. 1 Introduction The inter-domain routing system is a critical infrastructural piece of the Internet. Sites connecting to the Internet are represented by a set of routing prefixes (for e.g /16 for Colorado State University). These sites utilize the Border Gateway Protocol (BGP) [1] to exchange reachability information. Each BGP speaking router maintains a routing table and a forwarding table. The routing table contains route information, while the forwarding table contains information required to forward packets on the correct router interface. For the past several years, the size of the routing and forwarding tables has been increasing at an alarming rate. Router memory and CPU processing power needs to increase to handle the increasing table sizes, forcing operators into shorter upgrade cycles for their routers. Several approaches have been suggested to manage and reduce the sizes of the routing and forwarding tables. In this report, I undertake a systematic survey of the various approaches that have been proposed and present a taxonomy of these approaches. The remainder of this report is organized as follows. In section 2, I present a brief introduction to BGP. In section 3, I present the routing scaling problem and present the solution space in section 4. In section 5, I present and compare approaches to scale the 1

2 forwarding table, while in section 6 I present and compare approaches to scale the routing table. I conclude in section 7. 2 The Border Gateway Protocol BGP is the dominant inter-domain routing protocol in use in the Internet today. Every site connected to the the Internet, known as an autonomous system (AS), has border routers that exchange reachability information with other sites via BGP. BGP maintains two tables : the Routing Information Base (RIB) and the Forwarding Information Base (FIB) to store the routing information and forwarding information, respectively. 2.1 The Routing Information Base (RIB) A BGP-speaking router maintains a routing information base (RIB) table for each router that it gets reachability information from. Each router also maintains its local RIB that is calculated from the RIB tables maintained for each peer and additional local information available to the router. 2.2 The Forwarding Information Base (FIB) Routers maintain the forwarding information base (FIB) table to store forwarding information required to forward packets to the next hop. The forwarding information consists of the outgoing router interface, next hop and other attributes. The router needs to perform a FIB lookup each time it needs to forward a packet. Therefore, routers store the FIB on fast memory on the linecards. FIB tables are calculated from the RIB tables. 3 Scaling Problems Facing the Inter-Domain Routing System It is widely recognized that the Inter-domain routing system is facing serious scaling problems [17]. This growth has been fueled by various factors, such as multihoming, traffic engineering, de-aggregation of prefixes etc. The increasing sizes of the global RIB and FIB has been a matter of study for several years. In the remainder of this section, I present previous work that studies the table growth and factors contributing to the table growth. 2

3 3.1 IAB Report on Routing and Addressing According to the report from the Internet Architecture Board s (IAB) Workshop on Routing and Addressing [17], routing scalability is the most important problem facing the Internet today. The routing scalability problem includes the size of the RIB and FIB, the implications of the growth of the RIB and FIB on routing convergence times and the cost, power and processing requirements of core router hardware. The report identifies the following factors as the main driving forces behind the growth of the RIB: Multihoming Traffic engineering Non-aggregatable address allocations Business events such as mergers and acquisitions. All of the above can lead to prefix de-aggregation and the introduction of non-aggregated/deaggregatable prefixes into the RIB. Prefix de-aggregation leads to an uncontrolled RIB growth because it takes away the ability to perform topological aggregation, which is the only known practical approach to controlling the growth of the RIB Implications of RIB Growth The size of the RIB affects the size of the FIB, since routers calculate the FIB from the FIB. Thus, the growing RIB size requires more RIB and FIB memory to be available on the routers. Apart from this, the core of the network is exposed to the dynamic nature of the edge i.e., de-aggregation leads to an increased number of BGP update messages into the core (also called update churn). As a result, additional processing is required to maintain state for the prefixes and calculate the FIB Implications of FIB Growth Conventional wisdom is that the FIB technology will scale at rates that surpass the growth rate of routing information handled by core router hardware. According to the IAB, that is not true for the low-volume, customized hardware used in core routing. As a consequence, the hardware may not scale with the increasing size of the FIB. 3

4 3.2 Factors Contributing to BGP Table Growth In [3], the authors study BGP tables from the RouteViews project [18] to determine the factors behind the growth of the routing table. In particular, the authors study the contribution of the following factors to the growth of the routing table. 1. Multihoming 2. Failure to aggregate 3. Load Balancing 4. Address Fragmentation Multihoming Many Internet sites connect to more than one provider for the purpose of fault tolerance. If a site announces a prefix p that is contained in the prefix announced by the one of the multihomed site s providers, then the site s other providers also need to announce p. To quantify the effect of multihoming on the size of the routing table, the authors count the number of multihomed and non-multihomed prefixes in the routing tables. The results show that the number of multihomed prefixes is on the rise, and that multihoming introduces 20-30% more prefixes into the routing table (figure 1). Figure 1: Contribution of multihoming to routing table size [3] 4

5 3.2.2 Failure to Aggregate To quantify the effect of the ISPs failure to aggregate prefixes, the authors aggregate all prefixes that are originated by the same AS and are announced identically. By comparing the sizes of the tables before and after aggregation, the authors find that approximately 15-20% more prefixes can be aggregated, beyond what the ISPs already aggregate, as shown in figure 2. Figure 2: Contributions to routing table growth [3] Load Balancing Often, ASes will de-aggregate otherwise aggregatable prefixes for the sake of load balacing. To quantify the effect of load balancing on the size of the routing table, the authors compute the number of prefixes resulting from aggregating all prefixes originated by the same AS, independent of whether those prefixes are announced independently or not. By comparing the total number of prefixes after aggregation with the number of prefixes excluding those introduced by the failure to aggregate, the authors quantify the effect of load balancing to the table size. They find that load balancing introduces an additional 20-25% more prefixes, as can be seen in figure Address Fragmentation To quantify the effect of address fragmentation, the authors classify prefixes into prefix clusters, in each of which prefixes are originated by the same AS and are announced identically. Then, by comparing the number of prefix clusters with the number of prefixes excluding 5

6 those contributed by the failure to aggregate and load balancing, the authors find that address fragmentation contributes to more than 75% of the routing table size and is the single largest contributor Growth Rate of Contributors Apart from quantifying the effect of various contributors to the table size growth, the authors also calculated how fast each contributor was growing. The results show that load balancing is the fastest growing contributor. Also, load balancing and multihoming contributions grow faster than the growth of the table itself (figure 3). Figure 3: Growth rate of contributors [3] 3.3 Router Requirements for Increasing RIBs In [12], Huston et.al. study BGP update messages to discover trends in demands for router processing power. Based on the trends in the update messages, the authors also make predictions about the size of the routing table on various dates. The results from the paper follow BGP Update Message Trends The authors observed nearly 146 million messages throughout 2005 at their BGP measurement point. The important results from the study are as follows: 1. The size of the routing table grew from 148,000 entries in the beginning of 2005 to 175,400 entries at the end of 2005, which is a growth of 18%. 6

7 2. The number of BGP update messages nearly doubled through the measurement period, from approximately 260K messages per day at the beginning of 2005 to approximately 550K messages per day at the end of The average number of prefixes in each update message decreased from 2.4 at the beginning of 2005 to 2.3 at the end of This is indicative of fine-grained routing policies in place. 4. Some prefixes generate a disproportionately high number of update messages per day Predictions for Routing Table Growth Based on the study of BGP update messages, the authors were able to predict the size of the BGP routing table for various dates. Figure 4 shows the predicted trend in the routing table size. Figure 4: BGP Table Size Projection [12] The projected table size is 370,000 entries by the end of According to [11], the routing table has approximately 343,000 entries today. Thus, the projected table size is within 10% of the actual table size. As the table sizes are projected to grow, the routers will need increasing amounts of memory to store the tables. Also, since the sheer number of updates is increasing along with the granularity of the updates, routers will need more processing power to handle the updates. 3.4 Summary In this section, I have shown previous work that 7

8 Establishes the routing scaling problem Establishes and describes the factors responsible for the routing scaling problem Looked at the contribution of the factors to the routing table growth In the next section, I will present and compare approaches that seek to solve the routing scaling problem. 4 Solutions To The Routing Scaling Problem The proposed solutions to the routing scaling problem fall into two broad categories, according to the table they scale. FIB scaling solutions RIB scaling solutions The proposed solutions can also be classified as: Configuration-only solutions Architectural solutions There are, therefore, four types of solutions to the routing scaling problem: 1. Configuration-only FIB scaling solutions 2. Architectural FIB scaling solutions 3. Configuration-only RIB scaling solutions 4. Architectural RIB scaling solutions 5 FIB Scaling Solutions Scaling the FIB size is arguably the more immediate problem facing the inter-domain routing system today. Routers store FIBs on fast, expensive line card memory. As the IAB report shows [17], this memory may not scale with the increasing size of the FIB. Since routers need to perform lookups when forwarding packets, the increasing FIB size makes lookups more expensive. Also, it forces ISPs into shorter upgrade cycles, increasing the operational cost. FIB scaling solutions must ensure forwarding correctness i.e., they should ensure packet delivery and not change the path that packets take. Forwarding correctness is of two types, strong and weak. 8

9 Strong Forwarding Correctness: The longest-match lookup for any destination address should be the same before and after the scaling algorithm is applied. Weak Forwarding Correctness: For destination addresses which have a next hop (next hop is non-null), the longest-match lookup should be the same before and after the scaling algorithm is applied. 5.1 Configuration-only FIB Scaling Solutions Configuration-only solutions to FIB scaling collapse the FIB without requiring any changes to the architecture or hardware implementation of the FIB. Instead, they rely on configuration updates to router software to implement the scaling algorithms. All the solutions discussed ([22], [15] and [2]) collapse the FIB by aggregating the FIB entries, either by replacing the actual FIB entries by aggregate entries, or by using virtual prefixes to aggregate real FIB entries FIB Aggregation In [22], Zhao et. al. propose a purely local solution: FIB aggregation, to reduce the FIB size. FIB aggregation eliminates and aggregates entries based on next-hop router information. At the same time, it ensures forwarding correctness. The authors propose four FIB aggregation levels. Level 1 Aggregation: This is the simplest form of aggregation. Prefixes that share the same next hop with their immediate ancestor prefixes are eliminated from the FIB, as shown in figure 5. Figure 5: Level 1 Aggregation Since prefix 10.1/16 shares the same next hop A as it s immediate ancestor prefix 10/8, it can be removed from the FIB. Level 1 aggregation satisfies strong forwarding correctness. Results show that this simple aggregation scheme can reduce the FIB size by 30-50%. 9

10 Level 2 Aggregation: In level 2 aggregation, apart from performing level 1 aggregation, sibling prefixes that share the next hop are combined into a single parent prefix, as illustrated in figure 6. This is possible only if the parent prefix does not already exist in the FIB. Figure 6: Level 2 Aggregation In this scheme, prefixes 10.1/16 and 10.2/16 are collapsed into their parent prefix 10/8, since they have the same next hope A and the prefix 10/8 does not exist in the forwarding table. Level 2 aggregation satisfies strong forwarding correctness. Level 3 Aggregation: In level 3 aggregation, apart from performing level 1 and 2 aggregation, a set of non-sibling prefixes that have the same next hop is combined into a super-prefix, as illustrated in figure 7. Figure 7: Level 3 Aggregation The prefixes /24 and /24 are not siblings, since they do not have the same parent. However, since they share the same next hop A, they can be collapsed into a singe super-prefix (the super-prefix cannot already exist in the FIB). However, this newly inserted super prefix covers some previously non-routable space (denoted by the red nodes in figure 7). Thus, some traffic which would previously have been dropped by the router is now forwarded along the same next hop as the new aggregate prefix. This is not always desirable. Thus, level 3 aggregation satisfies weak forwarding correctness, but not strong forwarding correctness. 10

11 Level 4 Aggregation: Level 4 aggregation aggregation is similar to level 3 aggregation, since it too combines non-sibling prefixes into a single aggregate prefix. However, unlike level 3 aggregation, level 4 aggregation permits other prefixes with different next hops between the non-sibling prefixes to be aggregated. Figure 8 illustrates level 4 aggregation. Figure 8: Level 4 Aggregation In figure 8, prefixes /24 and /24 share the same next hop A and are aggregated into the super-prefix 10/8. However, prefix /24 has a different next hop B, which means it cannot be covered by the super-prefix 10/8 and has to announced separately. Thus, B punches a hole in the aggregate prefix 10/8. Like level 3 aggregation, level 4 aggregation also introduces extra routable space that was previously unrouteable. Results show that level 4 aggregation can shrink the FIB by 60-90%. Level 4 aggregation satisfies weak forwarding correctness, but not strong forwarding correctness Incremental FIB Aggregation The level 1-4 aggregation algorithms described above do not handle routing updates efficiently. After each routing update, the FIB needs to be re-aggregated. Therefore, in [15], Liu et. al. investigate aggregation algorithms that are able to handle updates incrementally. They modify the optimal routing table constructor (ORTC) algorithm described in [6] to handle incremental updates. They design two such modifications, one that minimizes the amount of computation time required to process routing messages and update the FIB at each line card (the minimal time scheme), and another one that maintains the optimum aggregated FIB size (the optimal size scheme). The authors then compare the ORTC-based algorithms to the level 2 aggregation described in [22] and discussed above. To compare the performance of the ORTC-based algorithms with the level 2 aggregation scheme, the authors use the following metrics: Aggregation Ratio: The ratio of the size of the aggregated FIB to the size of the unaggregated FIB. A lower aggregation ratio implies better aggregation. 11

12 Computation Time: The time cost of aggregating the initial FIB and updating the aggregated FIB. As can be seen from figure 9, the optimal size scheme performs better than the level 2 aggregation scheme from [22], reducing the FIB size by 10% more. Like level 2 aggregation, the optimal size scheme satisfies strong forwarding correctness. However, this scheme takes almost three times longer to aggregate the FIB than level 2 aggregation. Figure 9: Aggregation Ratio (ORTC vs. Level-2) [15] As figure 10 shows, the minimal time scheme is consistently faster than level 2 aggregation. Like level 2 aggregation, the minimal time scheme also satisfies strong forwarding correctness. This algorithm is over 7 times faster than the ORTC-based optimal size scheme. However, the FIB is about 13% larger. Compared to the level 2 aggregation scheme which takes 200ms, this scheme takes 75ms Virtual Aggregation (ViAggre) Ballani et. al. argue in [2] that it may be not practical to undertake a major architectural change. Instead, it may be more pragmatic to undertake a series of incremental, individually cost-effective changes. They propose Virtual Aggregation (ViAggre) as a FIB scalability technique. ViAggre design has two major goals. 1. No changes to router software and routing protocols should be required. 2. ViAggre adoption should be transparent to external networks. 12

13 Figure 10: Computation Time (ORTC vs. Level-2) [15] The authors present two designs for ViAggre deployment. Design 1 performs only FIB suppression, while design 2 suppresses the RIB as well as the FIB. I discuss design 2 as a configuration-only RIB scaling solution in next section 6.1. An ISP adopting ViAggre can divide the global address space into a set of virtual prefixes such that the virtual prefixes are larger than any aggregatable (real) prefix being advertised today. For example, an ISP could divide the IPv4 address space into 128 parts, with a /7 virtual prefix representing each part. Since these virtual prefixes are not topologically aggregatable, ViAggre aggregates them by organizing virtual networks, one for each virtual prefix. This virtual topology makes the virtual prefixes aggregatable and shrinks the size of the FIB. To create a virtual network, each ISP designates some routers to be within the virtual network. These routers maintain routes for all prefixes in the virtual prefix corresponding to the virtual network and are said to be aggregation points for the virtual prefix. A router can be an aggregation point for multiple virtual prefixes, and is required to only maintain routes for prefixes in the virtual prefixes it is the aggregation point for. When a router receives a packet destined to a prefix it is aggregating, it performs a FIB lookup to determine the route for the packet. Since a router that is not a aggregation point for the virtual prefix may forward the packet back to the aggregation point, causing a loop, such a packet is tunneled from the aggregation point to the next hop (ViAggre uses MPLS Label Switched Paths (LSPs) as the tunneling mechanism). This procedure is illustrated in figure 11. The packets are destine to prefix 4/24 between routers A and E, through an ISP with ViAggre. Router C is an aggregation point for virtual prefix 4/7. The authors applied ViAggre to the FIBs from several ISPs and compared the unaggregated FIB size to the ViAggre FIB size. The results are shown in figure 12. It is clear that even in the worst case, the FIB size is approximately 14-16% of the original FIB size. 13

14 Figure 11: Path of Packets Destined to 4/24 [2] Figure 12: FIB Sizes for Various ISPs Using ViAggre [2] 5.2 Architectural FIB Scaling Solutions FIB lookups are a time consuming step of packet forwarding, and lookups get more expensive as the FIB size increases. In configuration-only solutions, efficient lookups are a consequence of the reduction in FIB size. Architectural solutions, on the other hand, employ caches to reduce the number of FIB lookups that need to be performed. The often-used FIB entries are stored in a fast, small cache on line cards for efficient retrieval, while the full FIB can be stored on slower, larger memories. Thus, architectural solutions require changes to core router hardware Use of Caching to Improve Gateway Performance Route caching has been suggested as a means for improving router performance for a long time. In [10], Feldmeier investigated the use of caches to improve the performance of gateways. His results show that a fully-associative least recently used (LRU) caching scheme that caches source/destination pairs performs the best. As figure 13 shows, the cache size required for a hit rate of 0.9 is only 7. Other results show that it is possible to reduce table lookup-time by about 65%. 14

15 Figure 13: Hit Rate vs. Cache Size Uni-class Caching Feldmeier [10] showed in 1988 that by employing caches, the performance of routers can be improved significantly. He did not, however, suggest the architecture or implementation of such a route cache. Kim et. al. propose a caching architecture in [14]. In this architecture, a line card incorporates a hierarchical, two level memory. The first level is a route cache, which is implemented as a small, but fast memory that stores only a small subset of forwarding entries. The second level is a slow, but large memory that stores all forwarding entries. Once a packet arrives for a destination, the forwarding unit first looks up the destination address in the cache. If it finds a match, it forwards the packet based on the cache lookup. If not, the forwarding unit forwards the packet along a backup route, while it updates the cache by retrieving the route from the full forwarding table. However, this approach cannot be directly applied to normal, variable length prefixes due to the cache hiding problem described below. Suppose that the full forwarding table consists of two entries, prefix 10.1/16 associated with output interface A and prefix /24 associated with output interface B. If a packet arrives for destination address , the router will perform a longest prefix match for the destination address, which is 10.1/16, and install the forwarding entry 10.1/16 -> A in the cache. Now, if a packet arrives for destination , the router will discover the 10.1/16 -> A entry in the cache and forward the packet along interface A. However, this is 15

16 incorrect since the longest prefix match for is /24 and hence the packet should have been sent to interface B. Solutions to this problem involve either a complicated data structure with on-the-fly computation to eliminate inter-dependency between prefixes, or grouping all prefixes containing a destination address as an atomic unit for cache operations. However, the latter approach may lead to cache thrashing, since the size of an atomic group can be of the order of 25,000 prefixes [14]. The authors, therefore, propose to split all prefixes into their constituent /24 prefixes. The cache stores only these /24 prefixes. This model is known as the uni-class model. In the uni-class model, a prefix is considered to be a collection of 256 independent /24 prefixes. While the full forwarding table only contains an entry for the original prefix, the cache stores only the /24 prefix corresponding to the destination address of the packet being forwarded. To de-aggregate the 305K prefixes advertised in the Internet during the time of writing [14], 9.3 million unique /24 prefixes were required. Of these, roughly one tenth (i.e M) prefixes account for nearly 97% of the traffic. The authors evaluated the performance of uni-class caches implemented using the LRU and LFU caching strategies and compared them to the optimal strategy. The results are shown in figure 14. Figure 14: Miss Rate vs. Cache Size It can be seen that the performance of a LRU cache comes closest to the optimal strategy. Also, the size of the required cache is modest (approximately 500K entries) for a miss rate of 1%, compared to the number of /24 prefixes. 16

17 5.3 Comparison of Configuration-only and Architectural FIB Scaling Solutions Previously, I have discussed configuration-only and architectural solutions to FIB scaling. The goal of both approaches is different : whereas configuration-only approaches seek to control the number of entries in the FIB, architectural solutions seek to make lookups efficient by employing caches. Reduction in lookup time in configuration-only approaches is a consequence of reducing the number of entries. The pros (+) and cons (-) of the configuration-only approaches are as follows: (+) Can be implemented as software updates to routers. No hardware updates are required. (+) Can be implemented by an ISP locally, independent of other ISPs. (+) ISPs can implement these approaches immediately. (+) Results indicate that a significant size reduction (50-70%) can be achieved. (-) Level of aggregation depends on the local forwarding table at the router. Some tables might have entries that cannot be aggregated. (-) May not preserve forwarding correctness. The pros (+) and cons (-) of architectural solutions are as follows: (-) Require changes to router hardware. (+) Can be implemented by an ISP locally, independent of other ISPs. (-) Cannot be implemented immediately, as router hardware needs to be updated. (-) Does not reduce size of the FIB. (+) Make lookups more efficient. (+) Reduce the amount of fast line card memory required, thus increasing time between updates. (+) Preserve forwarding correctness. 17

18 6 RIB Scaling Solutions Although FIB scaling is the immediate matter of concern for ISPs, the FIB size ultimately depends on the size of RIB. Since routers calculate the FIB from the RIB and any local knowledge that they have, reducing the number of RIB entries will reduce the number of FIB entries as well. Hence, scaling the RIB is the long-term solution to the routing scaling problem. Like FIB scaling solutions, RIB scaling solutions can also be either configuration-only or architectural. Since configuration-only solutions make incremental changes to the existing architecture, they are also known as dirty-slate designs. On the other hand, architectural solutions seek to change the fundamental architecture of the routing system. These solutions are therefore called clean-slate designs. 6.1 Configuration-only RIB Scaling Solutions ViAggre [2], introduced in section as a configuration-only FIB scaling solution also provides a design to scale the RIB (called design 2). In this design, the task of storing the full RIB is offloaded to devices that are not on the data path. Since many ISPs already employ per-point of presence (PoP) route reflectors for scalable internal route distribution, ViAggre leverages these route reflectors to store the full RIB. The external routers associated with a PoP are made to peer with the PoP s route reflector. The route reflector also peers with other route reflectors and routers in the PoP. By using egress filters on the route reflector s peerings with the PoP s routers, the ISP can ensure that each router only gets routes to prefixes it is aggregating. This shrinks the RIB and FIB on the routers. The packet forwarding mechanism remains the same as previously described in Both ViAggre designs introduce additional stretch for the aggregated prefixes, due to the use of aggregation points i.e. packets in ViAggre can take longer paths than the native paths. Apart from the longer paths, the packets can also incur queuing delays at the extra hops. The extra hops also result in an increase in the load on the ISPs routers and links and a change in the distribution of traffic across them. To overcome this problem, ViAggre leverages the fact that a large majority of the Internet traffic is destined to a very small fraction of the prefixes [7, 9, 20, 19]. For the design described in 5.1.3, each router s FIB is pre-configured with the set of popular prefixes that should not be suppressed. In design 2, the PoP s route reflectors are configured to not filter advertisements for popular prefixes from the PoP s routers. As described in section 5.1.3, the design goals of ViAggre are: 1. No changes to router software and routing protocols should be required. 18

19 2. ViAggre adoption should be transparent to external networks. Design 2 requires reconfiguration of the PoP s external BGP peerings. violates the design goal of not impacting other ISPs. It, therefore, 6.2 Architectural RIB Scaling Solutions As observed in [17], using a single address field for identifying a device and for determing the topological location of the device in the network are conflicting goals. Efficient routing requires topological assignment of addresses, while effective management of devices without renumbering in response to topological changes (such as those caused by mobility events, adding/removing network attachment points), the address must be independent of the topology. The locator/id separation protocol (LISP) [8] replaces the address field (usually an IP address) with two new types of numbers. Routing Locators (RLOCs) are topologically assigned to network attachment points and are therefore easy to aggregate. The RLOCs are used to forward packets through the network. Endpoint Identifiers (EIDs) are used to number devices inside administrative boundaries. EIDs are not routable. Figure 15: LISP Architecture [4] Since EIDs are non-routable, LISP defines a mapping function between the RLOC space and EID space. LISP also defines a function for encapsulating traffic originated by devices 19

20 using non-routable EIDs across a network infrastructure that routes and forwards traffic using RLOCs. Although EIDs and RLOCs are syntactically similar to IP addresses, the usage semantics are different. LISP is representative of a class of RIB scaling solutions called map-n-encap solutions [5]. Other map-n-encap solutions include [16] and [21]. Figure 15 shows a network infrastructure running LISP. Suppose the source S with EID wants to send a packet to the destination D with EID The packet arrives at the ingress tunnel router S2, which does not have the EID-to-RLOC mapping for S2 therefore encapsulates the packet in a LISP header, with the destination as its RLOC ( ). The RLOC forwards the packet using the normal BGP path to the egress tunnel router D2, which decapsulates the packet and forwards it to D. This data packet acts as a mapping request. In response, D2 sends the mapping information to S2, which is cached at S2. When the next packet arrives for D (or any destination within the same EID block), S2 will directly tunnel the packet to D2. The external routers (ITRs and ETRs) need to store only the paths to the RLOCs. Since the number of RLOCs is limited, and RLOCs are easily aggregatable, the size of the routing tables required is reduced significantly Cost of Caching EID-to-RLOC Mappings Since routers cache EID-to-RLOC mappings, it is worth studying the overhead introduced by this cache. In [13], Iannone et. al. emulate a LISP cache to study the cost of caching the EID-to-RLOC mappings. The important results from the study are as follows. 1. The size of the cache can be controlled by using a small timeout value for stale entries. This introduces a non-negligible amount of traffic overhead to perform the mapping lookups; the smaller the cache, the higher the number of lookups and the bandwidth required for the lookups. 2. This bandwidth requirement is similar to existing DNS lookup bandwidth, which implies that the LISP mapping mechanism can scale. 3. The overhead introduced by tunneling packets to and from the ITRs and ETRs is negligible. 6.3 Comparison of Configuration-only and Architectural RIB Scaling Solutions Both configuration-only as well as architectural solutions seek to reduce the number of RIB entries. However, the mechanisms used to achieve this are vastly different. Configurationonly approaches reduce the number of entries in the RIB by making the routers maintain 20

21 routes to only a few virtual prefixes, whereas architectural approaches reduce the number of entries in the RIB by separating the edge of the network from the core. Since the core network routers need to only store routes to other core network routers, the number of routes they need to maintain is small. The pros (+) and cons (-) of the configuration-only approach are follows. (+) Make only incremental changes to the existing routing architecture. (+) Can be independently and incrementally deployed by ISPs. (-) ISPs may need to reconfigure peering sessions with other ISPs. (+) Do not restrict an ISPs routing policies and route selection. (+) Control plane processing at routers is reduced. (-) May introduce stretch into the data path for packets. (-) Additional management overhead regarding virtual prefixes to deploy and aggregation points for each virtual prefix. (-) May influence important aspects of an ISP s day-to-day operation, such as maintenance and debugging. The pros (+) and cons (-) of the architectural approach is as follows. (-) Requires fundamental changes to the exiting routing and addressing infrastructure. (+) Can be independently and incrementally deployed by ISPs. (+) The network core is isolated from edge dynamics. (+) The cost of renumbering devices (if required) is reduced. (+) Provides better support for mobility than the current infrastructure. (-) Separation of the edge from the core requires an additional mapping step. (-) Encapsulation and tunneling packets introduces overhead. 21

22 7 Conclusion The inter-domain routing system of the Internet faces scalability problems due to the increasing sizes of the routing and forwarding tables. Several approaches have been proposed to scale the routing and forwarding tables. As size of the forwarding table is the pressing concern, more work has focused on scaling the forwarding table and making table lookups more efficient. However, although scaling the forwarding table is important in the short term, the size of the forwarding table ultimately depends on the size of the routing table. Hence, scaling the size of the routing table is the long-term solution to the routing-scaling problem. Some approaches scale the tables using small, incremental changes to the routing and forwarding architecture, whereas other other approaches scale the tables by fundamentally changing the routing and forwarding architecture. However, none of the scaling approaches discussed in this survey have been widely adopted. 8 Acknowledgements I would like to thank Mikhail Strizhov, Vamsi Kambhampati and Steve DiBenedetto for their editorial help in writing this report and the oral presentation. References [1] A border gateway protocol 4 (bgp-4). [2] H. Ballani, P. Francis, T. Cao, and J. Wang. Making routers last longer with viaggre. In NSDI 09: Proceedings of the 6th USENIX symposium on Networked systems design and implementation, pages , Berkeley, CA, USA, USENIX Association. [3] T. Bu, L. Gao, and D. Towsley. On characterizing bgp routing table growth. Comput. Netw., 45(1):45 54, [4] Cisco. Lisp data path. ipj/ipj_11-1/111_lisp_fig3_lg.jpg. [5] S. Deering. The map and encap scheme for scalable ipv4 routing with portable site prefixes. [6] R. Draves, C. King, S. Venkatachary, and B. Zill. Constructing optimal ip routing tables. In INFOCOM 99. Eighteenth Annual Joint Conference of the IEEE Computer 22

23 and Communications Societies. Proceedings. IEEE, volume 1, pages vol.1, mar [7] W. Fang and L. Peterson. Inter-as traffic patterns and their implications. In Proc. IEEE GLOBECOM, [8] D. Farinacci, V. Fuller, D. Meyer, and D. Lewis. Locator/ID Separation Protocol (LISP). draft-ietf-lisp-08.txt, [9] A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford, and F. True. Deriving traffic demands for operational ip networks: methodology and experience. SIGCOMM Comput. Commun. Rev., 30(4): , [10] D. Feldmeier. Improving gateway performance with a routing-table cache. pages , mar [11] G. Huston. Bgp analysis reports. [12] G. Huston and G. Armitage. Projecting future ipv4 router requirements from trends in dynamic bgp behaviour. In Australian Telecommunication Networks and Applications Conference (ATNAC), [13] L. Iannone and O. Bonaventure. On the cost of caching locator/id mappings. In CoNEXT 07: Proceedings of the 2007 ACM CoNEXT conference, pages 1 12, New York, NY, USA, ACM. [14] C. Kim, M. Caesar, A. Gerber, and J. Rexford. Revisiting route caching: The world should be flat. In PAM 09: Proceedings of the 10th International Conference on Passive and Active Network Measurement, pages 3 12, Berlin, Heidelberg, Springer- Verlag. [15] Y. Liu, X. Zhao, K. Nam, L. Wang, and B. Zhang. Incremental forwarding table aggregation. In GlobeCom 2010, Dec [16] D. Massey, L. Wang, B. Zhang, and L. Zhang. Proposal for scalable internet routing and addressing. draft-wang-ietf-efit-00.txt, [17] D. Meyer, L. Zhang, and K. Fall. Report from the IAB Workshop on Routing and Addressing, [18] U. of Oregon. The routeviews project. Advanced Network Topology Center and University of Oregon. 23

24 [19] J. Rexford, J. Wang, Z. Xiao, and Y. Zhang. Bgp routing stability of popular destinations. In Proc. Internet Measurement Workshop, [20] N. Taft, S. Bhattacharyya, J. Jetcheva, and C. Diot. Understanding traffic dynamics at a backbone pop. Scalability and Traffic Control in IP Networks, 4526(1): , [21] X. Zhang, P. Francis, J. Wang, and K. Yoshida. Scaling ip routing with the core routerintegrated overlay. In ICNP 06: Proceedings of the Proceedings of the 2006 IEEE International Conference on Network Protocols, pages , Washington, DC, USA, IEEE Computer Society. [22] X. Zhao, Y. Liu, L. Wang, and B. Zhang. On the aggregatability of router forwarding tables. In INFOCOM, 2010 Proceedings IEEE, pages 1 9, Mar