Routing Bottlenecks in the Internet: Causes, Exploits, and Countermeasures Min Suk Kang Virgil D. Gligor ECE Department and CyLab, Carnegie Mellon University Nov 4, 2014
2 Route Diversity is Critical to Resiliency of Internet Connectivity link-flooding attack rest of the world geographic area with poor route diversity
3 Fortunately, most countries have enough route diversity # of ISPs with direct international connectivity 40 + 40 10 2 (source: www.renesys.com/2014/02/internetunderfire/) Most countries have 10+ ISPs with international connections => good Internet route diversity Then, do we need to worry about the link-flooding attacks? Unfortunately, YES.
4 Despite high route diversity, Internet connectivity of countries can be degraded Why? routing bottleneck the vast majority of Internet routes to chosen destinations concentrated on a small set of links Paper illustrates 1. pervasive phenomenon of routing bottlenecks 2. causes of routing bottlenecks 3. impact of targeted attacks & countermeasures
5 Mincut and Routing Bottleneck mincut, M(S,D) sources (S) routing bottleneck, B destinations (D) geographic area B M(S,D) e.g. 10 1000 routing bottleneck bandwidth bottleneck
Normalized Link Occurrence 6 Routing Bottlenecks in the current Internet 250 nodes in PlanetLab (in 164 cities in 39 countries) sources (S) M(S,D) B traceroute Link Occurrence 1,000 randomly selected working servers destinations (D) geographic area (ratio) 0.12 0.1 0.08 0.06 0.04 0.02 0 measurement for a country 0.12 0.1 0.08 0.06 0.04 0.02 0 B (0.80) 0 10 20 30 40 50 0 200 400 600 800 1000 1200 high rank low rank Rank of Links in M(S,D)
Normalized link occurrence 7 Routing Bottlenecks 0.4 in 15 Countries 0.2 0.1 link occurrence is accurately modeled by a power-law 0 1 0 500 1000 1500 2000 2500 3000 3500-0.1 Tested Countries (alphabetical) Brazil Egypt France Germany India Iran Israel Italy Japan Romania Russia S. Korea Taiwan Turkey UK 0.1 0.01 0.001 0.0001 0.00001 0.3 Country1 Country2 Country3 Country4 Country5 Country6 Country7 Country8 Country9 Country10 Country11 Country12 Country13 Country14 Country15 Country1 Country2 Country3 Country4 Country5 Country6 Country7 Country8 Country9 Country10 Country11 Country12 Country13 Country14 Country15 Country15 α = 2.36 0.000001 1 10 100 1000 10000 Rank of Link Country1 (β = 7.8) α = 1.31 Zipf-Mandelbrot distribution f(k) = 1 (k + β) α
Normalized link occurrence 8 0.3 Routing Bottlenecks 0.25 in 15 Large Cities 0.1 0.05 link occurrence is accurately modeled by a power-law 0 1-0.05 0 500 1000 1500 2000 2500 3000 3500 Tested Cities (alphabetical) Beijing Berlin Chicago Guangzhou Houston London Los Angeles Moscow New York Paris Philadelphia Rome Shanghai Shenzhen Tianjin 0.1 0.01 0.001 0.0001 0.00001 0.2 0.15 City15 α =2.17 City1 City2 City3 City4 City5 City6 City7 City8 City9 City10 City11 City12 City13 City14 City15 City1 City2 City3 City4 City5 City6 City7 City8 City9 0.000001 1 10 100 1000 10000 Rank of Link City1 (β = 7.8) α = 1.38 Zipf-Mandelbrot distribution f(k) = 1 (k + β) α
9 Causes? An Analogy w/ Word Occurrence Distribution sentence construction: Principle of least effort [Zipf 49, Mandelbrot 53] ==> Z-M distribution of word occurrence Speaker route construction: Internet routers policies word1 word2 wordn conjecture: route-cost minimization ==> Z-M distribution of link occurrence link1 link2 linkn
Norm. Link Occurrence 10 Evidence for Inter-Domain Routing Policy: route-cost minimization BGP favors minimum-cost link => AS-level route concentration Test: policy I: favors min-cost links policy II: distribute routes uniformly AS* $$$? AS AS $ AS AS (*) AS: autonomous system Rank of Inter-AS Links
Norm. Link Occurrence 11 Evidence for Intra-Domain Routing Practice: route-cost minimization hierarchical topology + shortest path routing => route concentration at backbones Test: all possible ingress/egress routes clear Zipf-Mandelbrot distribution AS Rank of Intra-AS Links
12 Link Types of Routing Bottlenecks 3 link locations: intra-as inter-as IXP AS 1 AS 2 IXP (Internet exchange points) AS 3 AS 4 3 AS categories: Tier-1 Tier-2 Tier-2 Tier-3 Tier-3 Tier-3 Tier-3 (Global Transits/ National Backbones) (regional providers) (customers)
Link IXP (N/D: Types not determined) of Routing Bottlenecks < Avg. link types of 50 bottleneck links of 15 countries (percentage) > INTRA-AS (N/D: not determined) N/D N/D inter (N/D) IXP inter (Tier2-Tier3) inter (N/D) (Tier2-Tier2) inter (Tier2-Tier3) (Tier1-Tier3) inter (Tier2-Tier2) (Tier1-Tier2) inter (Tier1-Tier3) (Tier1-Tier1) inter intra (Tier1-Tier2) (Tier2) inter intra Tier-1 (Tier1-Tier1) (Tier1) INTER-AS 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% (N/D: not determined) Tier-2 intra (Tier2) intra (Tier1) (N/D: not determined) intra (Tier1) various link types: intra (30%), inter (30%), and IXP (20%) N/D (N/D: not determined) IXP N/D inter (N/D) IXP inter (Tier2-Tier3) inter N/D (N/D) inter (Tier2-Tier2) inter IXP IXP inter (Tier2-Tier3) (Tier1-Tier3) inter N/D inter (Tier2-Tier2) (N/D) (Tier1-Tier2) inter IXP inter (Tier1-Tier3) (Tier2-Tier3) (Tier1-Tier1) inter N/D intra (Tier2) (Tier1-Tier2) (Tier2-Tier2) (N/D) inter IXP intra (Tier1) (Tier1-Tier1) (Tier1-Tier3) (Tier2-Tier3) intra inter N/D Tier2-Tier2 (Tier2) (Tier1-Tier2) (Tier2-Tier2) (N/D) intra Tier2-Tier3 inter IXP (Tier1) (Tier1-Tier1) (Tier1-Tier3) (Tier2-Tier3) Not intra inter Determined (Tier2) (Tier1-Tier2) (Tier2-Tier2) (N/D) inter (Tier1-Tier1) (Tier1-Tier3) (Tier2-Tier3) intra inter (Tier2) (Tier1-Tier2) (Tier2-Tier2) intra inter (Tier1) (Tier1-Tier1) (Tier1-Tier3) intra inter (Tier2) (Tier1-Tier2) (N/D: not determined) (N/D: not determined) (N/D: not determined) 91% of inter/intra-as links are owned by Tier-1/Tier-2 Not Deter. 13
14 Routing-Bottleneck Exploits Massive Link Flooding e.g., Crossfire attack [IEEE S&P 2013] Link-flooding with indistinguishable attack flows Botnets low-rate attack flows routing-bottleneck link Decoy Servers several hops away (e.g., 40 Gbps = 4 Kbps x 10K bots x 1K decoys) Target Geographic area
Degradation Ratio 15 Connectivity Degradation in 15 Countries 1 0.9 0.8 Country15 α = 2.36 0.7 0.6 0.5 0.4 Country1 α = 1.31 (β = 7.8) 0.3 0.2 0.1 0 Country1 Country2 Country3 Country4 Country5 Country6 Country7 Country8 Country9 Country10 Country11 Country12 Country13 Country14 Country15 0 10 20 30 40 50 Number of Links to Flood
Degradation Ratio 16 Connectivity Degradation in 15 Large Cities 1 0.9 0.8 City15 α = 2.17 0.7 0.6 0.5 0.4 0.3 City1 α = 1.38 (β = 7.8) 0.2 0.1 0 City1 City2 City3 City4 City5 City6 City7 City8 City9 City10 City11 City12 City13 City14 City15 0 10 20 30 40 50 Number of Links to Flood
17 Countermeasures Inter-domain links Load balancing across parallel links between two ASes [ATC 07] Load balancing across links to different ASes [SIGCOMM 06] AS2 AS1 AS2 AS1 AS3 AS4
18 Countermeasures Intra-domain links Equal-cost multipath (ECMP) Needs real-time link-weight re-adjustment MPLS tunnels Needs real-time MPLS traffic enginnering (unknown if recent SDN-based solutions can be applied here) AS 1 1 2 AS
Reduction of degradation ratio (%) 19 Effectiveness of Countermeasures 4 implementation alternatives: Inter-AS links Intra-AS links Tier-1 ASes Tier-1&2 ASes 100 90 80 70 60 50 40 30 20 10 0 one type fits all countermeasures are not very effective countermeasures at large ISPs (Tier-1&2) are most effective
20 Related Work Internet topology studies; e.g., CAIDA, DIMES, etc. Power-law in Internet connectivity; e.g., [SIGCOMM 99, NATURE 00] Link-flooding attacks; e.g., Coremelt [ESORICS 09], Crossfire [S&P 13]
21 Conclusions Notion of the routing bottlenecks they are pervasive (in 15 countries and 15 cities) Causes: route-cost minimization very desirable feature of Internet routing Countermeasures effective when implemented in large ISPs
22 Thank You Min Suk Kang (minsukkang@cmu.edu)