CS269I: Incentives in Computer Science Lecture #8: Incentives in BGP Routing

CS269I: Incentives in Computer Science Lecture #8: Incentives in BGP Routing Tim Roughgaren October 19, 2016 1 Routing in the Internet Last lecture we talke about elay-base (or selfish ) routing, which is common in local area networks. Toay s lecture is about a ifferent type of routing, which is use between ifferent local networks. The Internet is really a network of networks (Figure 1). An autonomous system (AS) is a centrally controlle collection of routers a bunch of routers with a common aministrator. The Internet has aroun 42,000 ASes, an they span the gamut Internet Service Proviers (ISPs), universities, businesses, governments, etc. For example, Stanfor University is an AS, an so is its Internet Service Provier (Cogent), an these two ASes are irectly connecte by physical links. There are of course other ASes that Stanfor is not irectly connecte to, an sening traffic to them involves routing over a path with multiple hops. 1.1 Routing Within an AS Routing within an AS is usually one using shortest-path routing. These routing protocols resemble the shortest-path algorithms that you stuie in unergrauate algorithms (the greey Dijkstra algorithm, an the ynamic programming-base Bellman-For algorithm), with many aitional etails. Shortest-path routing requires a efinition of a shortest path, which in turn requires a efinition of ege length (in this context, calle link weight ). The simplest example is hop-count (where all link weights are 1), in which case breath-first search can be use to compute shortest paths. Alternatively, ifferent eges can have ifferent link weights, for example epening on the recent average elay along the link. Such elay-base routing brings us back to the selfish routing moel we stuie last lecture. c 2016, Tim Roughgaren. Department of Computer Science, Stanfor University, 474 Gates Builing, 353 Serra Mall, Stanfor, CA 94305. Email: tim@cs.stanfor.eu. 1

Stanfor Cogent AT&T MIT Figure 1: The Internet: a network of networks. Note that in shortest path routing, all routers effectively agree on which paths are the best (the shortest ones). These common preferences are ictate by the central aministrator, through the choice of link weights. 1.2 Routing Between ASes Routing between ASes is a ifferent story. There is no central aministrator to whom all ASes report (remember these ASes span the globe). Different ASes have ifferent preferences. For example, an AS may prefer paths with the smallest monetary cost to the AS, which is a function of the AS s business agreements with other ASes, an ifferent ASes have ifferent sets of business agreements. For example, ISP#1 may want to route through ISP#2 only as a last resort (if the former has to the pay the latter a high cost). Link weights are insufficient to moel such general preferences. For example, consier the AS network in Figure 2. Assume that the noe is the estination for all traffic. The ASes 1 an 2 are labele with their favorite path (at the top) an their secon-favorite paths. Note that each AS prefers to route to the other one over routing irectly to. (Maybe charges anyone who sens traffic to it.) These preferences are inconsistent with shortest-path routing (for any choice of link weights). 2 The Borer Gateway Protocol (BGP) In the Internet, routing between ASes is one using the Borer Gateway Protocol (BGP). We next explain the most important aspects of how it works (there are of course many more etails in the actual protocol). Fix a estination AS. BGP runs in parallel for all choices of the estination, an the computations for ifferent estinations are completely unrelate. (So when one AS sens a message to another AS, it inclues a full-blown routing table, with the AS s routing plan for all 42K possible estinations.) 2

12 1 21 2 Figure 2: An example of an AS network where each AS prefers to route to the other one over routing irectly to. Next we escribe the intene behavior for ASes in BGP. Later we ll iscuss whether ASes have an incentive to eviate from this intene behavior (recall the BitTorrent iscussion in Lecture #5). The protocol resembles the Bellman-For shortest-path algorithm, except with the istancebase upate step replace with a more abstract upate step. Every AS v will maintain a path P v to (or the empty path, if a path from v to hasn t been foun yet). Initially, sets P to the empty path ( can reach itself via the empty path), an broacasts this fact to all of its neighbors (ASes to which it has a irect physical connection). For example, in Figure 2, the protocol begins with alerting the other two ASes about its existence. All of the ASes are then suppose to execute the following proceure in parallel, continuously an asynchronously. 1 At an AS u with neighbors N: 1. For each v N: BGP Upate Step (a) Let P v be the last path (from v to ) that v announce to u (if any). 2. Reset P u to u s favorite cycle-free path of the form (u, v) P v (if any, otherwise P u is the empty path). 2 3. If P u changes, announce the new value of P u to all neighbors v N. Note that an AS u has to take care to avoi cycles if P v inclues u in it, then u cannot route using P v (traffic woul then loop forever). For example, let s return to the AS graph in Figure 2. What is the ultimate result of BGP? One possible fixe point of the protocol is shown in Figure 3(a), with 1 routing irectly 1 At the Internet scale, you can t really make any assumptions about the timing of when ifferent events occur. 2 The operator enotes path concatenation. 3

to an 2 routing through 1. AS 2 obviously oesn t want to upate its path, since it s using its favorite path. AS 1 isn t thrille about routing irectly to, but it has no other choice: it can t switch to routing via 2, since this woul create a cycle. This is not the only possible fixe point of BGP in the AS graph in Figure 2; Figure 3(b) shows another (symmetric) one. Which fixe point o we expect BGP to reach? It epens on the timing of the messages whichever of 1, 2 fins out first that the other is using a irect path to can switch paths an win. (a) (b) Figure 3: Two stable routings in the AS graph of Figure 2. 3 Stable Routings We call a fixe point of BGP a stable routing. That is, in a stable routing, no AS wants to change its path to, given the path choices of other ASes an the corresponing options available to u. 3 Thus Figure 3 shows two stable routings in the AS graph of Figure 2. A stable routing must be a tree, irecte into the estination. BGP maintains the invariant that the out-egree of every AS is at most 1. It also explicitly prevents cycles from forming. The only possible sink (other than isolate ASes) is. If the AS graph is not connecte, or if some ASes prefer the empty path to some of its paths to, then the tree nee not span all of the ASes. We ve alreay seen that an AS graph can have more than one stable routing. The next example shows that there might be no stable routing at all. Consier a hypothetical stable routing in the network shown in Figure 4. The resulting tree must inclue at least one ege between an another AS, say AS 1. (Otherwise all ASes are isolate, but every AS prefers the irect path to over the empty path). In any stable routing that inclues the link from 1 to, AS 3 must be using the path 31 (since this path is its favorite). Given this, AS 2 must use the path 2, since its preferre path 23 is not available. But if AS 2 uses the path 2, 3 This shoul remin you of the Nash equilibrium concept mentione in Lectures #5 an #6. We can think of the ASes as the players, the neighboring ASes of an AS u as u s available strategies, an with payoffs inuce by an AS s abstract preferences over paths. 4

then AS 1 wants to switch to the path 12. We conclue that there cannot be any stable routing. 12 1 23 2 3 31 3 Figure 4: An example of an AS network with no stable routing. 4 Dispute Wheels Ultimately, we want to iscuss incentives in BGP o ASes want to follow the intene behavior of the BGP protocol, or are there opportunities to game the system? But the examples in Figures 2 an 4 show that the output of BGP is not even well efine in some cases. In this section we ientify conitions on an AS graph uner which the outcome of BGP makes sense, an then in Section 6 we aress incentive issues. 4.1 No Dispute Wheel Implies BGP Convergence The following theorem ientifies a conition ( no ispute wheel, which we ll efine shortly) uner which BGP has several nice properties. Theorem 4.1 ([3]) Suppose an AS graph has no ispute wheel. Then: (a) There exists a stable routing. (b) This stable routing is unique. (c) BGP is guarantee to converge to the stable routing. Note that part (c) of the theorem is strictly stronger than (a). BGP can only converge to a stable routing, so (c) implies (a). But (a) oes not imply (c) in general: there are AS graphs with a unique stable routing in which BGP can cycle forever. (In light of Theorem 4.1, we know that such examples must have a ispute wheel.) The missing efinition is as follows. A ispute wheel consists of (for some k 2): 1. istinct ASes u 1,..., u k ; 5

2. (cycle-free) paths P 1,..., P k, where P i is a path from u i to ; an 3. (cycle-free) paths Q 1,..., Q k, where Q i is a path from u i to u i+1, such that, for each i = 1, 2,..., k, u i prefers the inirect path Q i P i+1 to the irect path P i. (Where by u k+1 an P k+1 we mean u 1 an P 1.) See Figure 5. The figure is somewhat misleaing, as the P i s an Q i s nee not be internally isjoint. For example, a vertex u i is allowe to appear in the mile of all of the paths P j for j i. Q k u 1 Q 1 u k P 1 P k u 2 P 2 Q 2 P 3 u 3 Figure 5: A ispute wheel. The P i s an Q i s nee not be internally isjoint. This efinition generalizes the two examples we ve seen thus far. The AS graph in Figure 2 is a ispute wheel, as seen by taking P 1 as the path 1, P 2 the path 2, Q 1 the path 12, an Q 2 the path 21. (Recall each AS prefers to route to the other over routing irectly to.) The AS graph in Figure 4 is also a ispute wheel: take P i = i for i = 1, 2, 3, Q 1 = 12, Q 2 = 23, an Q 3 = 31. 4.2 Are There Dispute Wheels in Practice? When someone subjects you to a new efinition, especially a weir one like the one above, you shoul eman two things: interesting consequences of satisfying the efinition, an interesting examples that satisfy the efinition. Theorem 4.1 takes care of the first part AS graphs without a ispute wheel have many esirable properties. But how strong is the no ispute wheel conition? For a sanity check example that is not very interesting in its own right, suppose all ASes prefer shorter paths to longer ones, with respect to some set of nonnegative link weights (with all ASes using the same link weights). Then, it s not har to prove that the corresponing AS graph has no ispute wheel (see Exercise Set #4). But this is not very satisfying the whole point of BGP is to move beyon shortest-path routing. Gao an Rexfor [1] gave a much more satisfying justification of the no ispute wheel conition, an argue that the conition shoul generally hol for realistic AS preferences. 6

Formally, they propose what are now known as the Gao-Rexfor conitions on an AS graph, an prove that these conitions rule out any ispute wheels. Stuying the etails of the Gao-Rexfor conitions woul take us too far afiel, but here s the iea. To first orer, pairs of ASes are in one of two possible relationships. They might be peers, who agree to carry traffic to an from each other. (E.g., all pairs of Tier 1 ISPs are peers.) Otherwise, one is the provier an the other the customer, with the customer paying the provier for connectivity to other parts of the Internet. For example, most Tier 2 ISPs have to pay one or more Tier 1 ISPs to be able to reach the entire Internet. One of the Gao-Rexfor conitions is that every AS always prefers routes through a customer over those through a peer, an those through a peer over those through a provier. This hierarchical structure of ASes prevents ispute wheels. 4 There is anecotal evience that the Gao-Rexfor conitions approximately hol for most ASes [2]. 5 Proof of Theorem 4.1 We now prove part(b) of Theorem 4.1. This will be the only proof of the lecture, an it is chosen to be representative of the types of arguments use in the other proofs. 5 In particular, we ll see why ispute wheels naturally come up in such arguments. We ll prove the contrapositive. Suppose an AS graph has two ifferent stable solutions, S an T. (Both are trees, irecte into.) The plan is to exhibit a ispute wheel. As a thought experiment, imagine oing breath-first search backwar from along eges in both S an T, an let H be the vertices reache. That is, efine H as the subset of vertices that use the same path to in both of the stable solutions. See Figure 6. If nothing else, H inclues the estination AS. We claim that there is an ege (u 1, v 1 ) such that: (i) u 1 / H; (ii) v 1 H; an (iii) (u 1, v 1 ) is in either S or T. Note that such an ege cannot be in both S an T (since v 1 is reachable backwar from by eges of S T, if (u 1, v 1 ) were also in S T, then u 1 woul also be in H, which it isn t). We prove the claim by contraiction. If not, then no eges of S or T cross the bounary of H (i.e., have one enpoint in H an the other outsie H). This means that in both S an T, the vertices that have a path to are precisely H. (If some vertex outsie H has a path to in S or T, then the path woul have to cross the bounary of H at some point.) But S an T are ientical insie H (by efinition), so this woul mean that S = T (a contraiction). So pick an ege (u 1, v 1 ) as in the claim. Say the ege is in S but not T (the case where it s in T but not S is symmetric). Since (u 1, v 1 ) S, the path u 1 uses to reach in S has (u 1, v 1 ) as the first hop, followe by whatever path P v1 uses to reach in S. Denote the path (u 1, v 1 ) P v1 by P 1. 4 Another important conition is that ASes refuse to carry transit traffic (i.e., traffic that both originates from an is estine for other ASes), except from customers. This refusal is implemente by announcing non-empty routes only to your customers. The final conition is uncontroversial: there shoul be no cycle of provier-customer relationships. 5 You will prove part (a) in Exercise Set #4. 7

s t H Figure 6: The set H in the proof of Theorem 4.1. The soli eges belong to both S an T. The ashe eges s an t belong to only S an T, respectively. Now switch to the other stable solution, T. Since S an T are ientical in H, an v 1 H, P v1 is also in T. Thus, the u 1 - path P 1 is an available option to u 1 in T. Since (u 1, v 1 ) / T, u 1 oes not use the path P 1 in T, an instea uses some other path P. P cannot be the empty path: if u 1 preferre the empty path over P 1, it woul have also chosen the empty path in the stable solution S (instea of P 1 ). Similarly, the first hop of P cannot have the form (u 1, w) for some w H; if u 1 preferre (u 1, w) P w over P 1 in T, then it woul o so in S as well (since w H, P w is available in S). So, P must have the form shown in Figure 7: a nonempty path from u 1 to some other vertex u 2 / H, followe by an ege (u 2, v 2 ) crossing the bounary of H, followe by the path P v2 that v 2 uses to get to (in both S an T ). Define Q 1 as the first part of the path (traveling from u 1 to u 2 ), an P 2 as the secon part (from u 2 to ). By efinition, u 1 prefers the path P = Q 1 P 2 to P 1. We now repeat the argument starting from u 2. The ege (u 2, v 2 ) lies in T (because u 2 chooses the path P 2 in T ), an since it crosses the bounary of H, it cannot also lie in S (see the argument above). Since v 2 H an S, T agree insie H, the path P 2 was one of u 2 s available options in S. Since (u 2, v 2 ) / S, in S, u 2 took some other path to, necessarily of the form shown in Figure 7. This path has the form Q 2 P 3, for a path Q 2 from u 2 to some other vertex u 3 (ifferent from u 2, possibly the same as u 1 ), an for a path P 3 from u 3 to, with the first hop of P 3 crossing the bounary of H. We can iterate this argument as many times as we want, to efine u i s, P i s, an Q i s. At some point, a u i will repeat (e.g., say u 1 = u k+1 ). We claim that the vertices u 1,..., u k with the paths P 1,..., P k an Q 1,..., Q k form a ispute wheel. The proof just checks all the conitions: (i) the u i s are istinct, an k 2; (ii) each path P i is from u i to ; (iii) each path Q i is from u i to u i+1 (with path P k from u k to u 1 ); an (iv) each u i prefers Q i P i+1 to P i (by construction). This completes the proof. 8

t 3 t u 1 s 2 1 u 2 u 3 s 1 t 2 s 3 v 1 v 2 v 3 H Figure 7: Illustration for the proof of Theorem 4.1. The soli eges belong to both S an T. The ashe eges s 1, s 2, s 3 an t 1, t 2, t 3 belong to only S an T, respectively. Here, P 1 = s 1 P v1, Q 1 = t 1, P 2 = t 2 P v2, P 2 = t 2 P v2, Q 2 = s 2, P 3 = s 3 P v3, an Q 3 = t 3. 6 Incentive Issues We now focus on incentive issues in AS graphs that have no ispute wheel (where the result of BGP is well efine). Can ASes game BGP? Is there a beneficial unilateral eviation from the intene behavior escribe in Section 2? 6.1 Types of Deviations There are several ways an AS coul eviate from BGP. Here are three types of eviations: 1. Choose your path P u to be something other than your favorite among the available options. 2. Withhol information about your path from (some of) your neighbors. 3. Announce a path to (some of) your neighbors that is ifferent from the one you re actually using, possibly a path that oesn t even exist in the network. It s clear that an AS is in a position to execute the first two types of eviations. What about the thir type? Can an AS really get away with announcing fake paths to its neighbors? It may surprise you that fake path announcements are happening in BGP all the time. One of the more famous examples was in February of 2008. Pakistan Telecom wante to block 9

access from Pakistan to YouTube, because of some offensive vieos. They implemente this by rerouting YouTube traffic to a local server (just a Web page saying access blocke. ). For whatever reason (accientally?), Pakistan Telecom announce this new route to YouTube to some of its neighbors. An ISP in Hong Kong switche its route to YouTube in secons, an much of the Internet quickly followe suit. 6 The results were not goo for anyboy: wie swaths of the Internet coul not reach YouTube, an Pakistan Telecom was burie uner all the requests that were being irecte to it. 7 The vulnerability of BGP to such mistakes an attacks is obviously a problem, an for many years there have been ongoing efforts to roll out a new an more secure version of BGP, such as the propose BGPsec protocol [4]. As you can imagine, with 42,000 ASes an no central authority, it s not easy to eploy major changes to the protocol. 6.2 Examples First we note that the example in Figure 2, where there are two stable routings, alreay shows that withholing information about your path can be beneficial. Recall that each of the 2 ASes wants to route through the other rather than irectly to the estination. If AS 1 never announces any paths an AS 2 follows the BGP protocol, then the protocol converges to the stable routing that AS 1 prefers (with 2 routing irectly to ). But this example is a ispute wheel, an we alreay know that BGP oesn t function well in the presence of ispute wheels. So what if there are no ispute wheels? To see that announcing fake paths can be beneficial, consier the AS graph in Figure 8. It is similar to the AS graph in Figure 4, except there is no irect physical connection between AS 3 an the estination. If the ege 3 were in the network an AS 3 preferre this path less than 31, there woul be a ispute wheel. With the ege missing, however, there is no ispute wheel (as you re invite to check). It may look weir that AS 2 is stating a preference for the non-existent path 23. But remember that there are 42,000 ASes in the Internet, an they are not always very open about who they re connecte to, an the network is always changing. How is AS 2 suppose to know whether or not there is a irect physical link between AS 3 an? An AS has to be reay to express preferences over any paths that it might encounter. Suppose all of the ASes are honest. Since there is no ispute wheel, BGP converges to a unique stable routing, shown in Figure 9(a). AS 3 gets its secon-favorite path. If AS 3 announces the fake path 3 to its neighbors, however, then BGP converges to the routing shown in Figure 9(b). The figure shows the paths that are actually use; AS 2 has been upe into thinking that its traffic is following its preferre path 23, when in reality it s 6 Why i ISPs prefer the new fake route over the trie-an-true real one? Because one commonly use rule is longest prefix matching, which prefers more targete routes to less specific ones. The true path was liste for a block of 1024 IP aresses (i.e., with the first 22 of the 32 bits of the IP aress specifie) while the path avertise by Pakistan Telecom was for a subset of 256 of these IP aresses (i.e., with 2 aitional bits specifie). This trick is sometimes calle prefix hijacking. 7 An earlier example occurre in 2004, when a Turkish ISP basically pretene to be the entire Internet, resulting in lots of traffic getting reirecte to it. 10

12 1 23 2 3 31 312 Figure 8: An example of an AS network in which announcing a fake path can be beneficial. being route also through AS 1. (If AS 2 knew the actual path being use, the routing woul not be stable.) In this new routing, AS 3 gets its favorite path, an we conclue that announcing fake paths can be beneficial even when there is no ispute wheel. 8 3 (a) 3 (b) Figure 9: Stable routings in the AS graph of Figure 8 if (a) all of the ASes are honest; (b) AS 3 announces the fake path 3. 6.3 Incentive-Compatibility with Path Verification We alreay note that fake path announcements really are possible (inee, frequent) in the current BGP protocol. But it is plausible that in the future, faking paths will not be so easy. For example, the propose BGPsec protocol uses cryptographic signatures to verify the existence of announce paths. Do incentive issues persist if fake path announcements are isallowe? Theorem 6.1 ([5]) Assume that: 1. No AS can announce a path that oesn t exist. 8 In effect, the fake route announcement creates a fictitious ispute wheel, which is enough to screw things up. 11

2. The AS graph oes not have a ispute wheel. Then, no AS has an incentive to unilaterally eviate from the intene behavior in BGP (assuming all other ASes follow BGP). That is, no AS can benefit from withholing paths from neighbors, from choosing a path other than its favorite available option, or from falsely announcing a ifferent route that actually exists in the network. Theorem 6.1 is satisfying an surprising, as many network protocols use in practice are at least somewhat vulnerable to gaming by participants, even uner restrictive assumptions (e.g., recall the BitTorrent iscussion in Lecture #5). The theorem can also be extene to protect against coalitions of eviators if a group of ASes eviates in a coorinate way from BGP an at least one of the coalition members becomes strictly better off, then a ifferent member will be worse off [5]. We won t prove Theorem 6.1, but the intuition is similar to that for the proof of uniqueness given in Section 5. Roughly, if an AS ha a profitable eviation, then the stable routings reache before an after the eviation play the same role as the ifferent stable routings in the proof of Theorem 4.1 (leaing to a ispute wheel). This argument isn t quite right, because after the eviation the routing is only stable in the mins of the ASes, who may have been misle about which path their traffic is being route on (recall Figure 9(b)). It turns out that, as long as ASes can only be misle with existing paths, an analogous argument can still be mae. References [1] L. Gao an J. Rexfor. Stable Internet routing without global coorination. IEEE/ACM Transactions on Networking, 9(6):681 692, 2001. [2] P. Gill, M. Schapira, an S. Golberg. A survey of interomain routing policies. Computer Communication Review, 44(1):28 34, 2014. [3] T. Griffin, F. B. Shepher, an G. T. Wilfong. The stable paths problem an interomain routing. IEEE/ACM Transactions on Networking, 10(2):232 243, 2002. [4] Internet Engineering Task Force. BGPsec protocol specification. Version 18. August 18, 2016. [5] H. Levin, M. Schapira, an A. Zohar. Interomain routing an games. SIAM Journal on Computing, 40(6):1892 1912, 2011. 12