Empirical Analysis of the Effects and the Mitigation of IPv4 Address Exhaustion

Transcription

1 Empirical Analysis of the Effects and the Mitigation of IPv4 Address Exhaustion wissenschaftliche Aussprache 2. August 2017 Philipp Richter

2 Internet Penetration, 2017, ISOC. The Internet connects 3.5 billion people as of (48% of world population) 1

3 users/subscribers [billions] 0 1B 2B 3B 4B Internet users mobile broadband subscriptions fixed broadband subscriptions year 2

4 The Internet Protocol Suite 3

5 The Narrow Waist of the Protocol Stack Original design: One IP address per host IPv4: 32-bit addresses, est ~ 4B unique IPv4 addresses Today: 3.5B users, ~7B connected devices. 3

6 IPv4 Address Exhaustion IPv4 Exhaustion received a lot of attention. But little in terms systematic empirical assessment. 4

7 Systematic Framing of IPv4 Address Exhaustion ACM CCR 15 (Best of CCR)

8 Systematic Framing of IPv4 Address Exhaustion ACM CCR 15 (Best of CCR) IPv4 addresses need to be globally unique We need a management body that distributes them

9 A History of IPv4 Address Block Management 1981 ~1995 ~2011 Early Registration Needs-Based Provision Depletion & Exhaustion Informal Distribution Scarcity minor issue Non-commercial Internet Distribution process Justification of need ISPs don t pay for IPs 4 out of 5 RIRs depleted Address Markets Transfer Policies routable IPv4 addresses (3.7B) allocated IPv4 addresses 5 10

10 A History of IPv4 Address Block Management 1981 ~1995 ~2011 Early Registration Needs-Based Provision Depletion & Exhaustion Informal Distribution Scarcity minor issue Non-commercial Internet Distribution process Justification of need ISPs don t pay for IPs 4 out of 5 RIRs depleted Address Markets Transfer Policies routable IPv4 addresses (3.7B) allocated IPv4 addresses 5 10

11 A History of IPv4 Address Block Management 1981 ~1995 ~2011 Early Registration Needs-Based Provision Depletion & Exhaustion Informal Distribution Scarcity minor issue Non-commercial Internet routable IPv4 addresses (3.7B) Distribution process Justification of need ISPs don t pay for IPs 4 out of 5 RIRs depleted Address Markets Transfer Policies 40% of the space given out by ~1995 LEGACY space allocated IPv4 addresses 5 10

12 A History of IPv4 Address Block Management 1981 ~1995 ~2011 Early Registration Needs-Based Provision Depletion & Exhaustion Informal Distribution Scarcity minor issue Non-commercial Internet Distribution process Justification of need ISPs don t pay for IPs 4 out of 5 RIRs depleted Address Markets Transfer Policies routable IPv4 addresses (3.7B) allocated IPv4 addresses 5 10

13 A History of IPv4 Address Block Management 1981 ~1995 ~2011 Early Registration Needs-Based Provision Depletion & Exhaustion Informal Distribution Scarcity minor issue Non-commercial Internet Distribution process Justification of need ISPs don t pay for IPs 4 out of 5 RIRs depleted Address Markets Transfer Policies routable IPv4 addresses (3.7B) allocated IPv4 addresses 5 10

14 A History of IPv4 Address Block Management 1981 ~1995 ~2011 Early Registration Needs-Based Provision Depletion & Exhaustion Informal Distribution Scarcity minor issue Non-commercial Internet Distribution process Justification of need ISPs don t pay for IPs 4 out of 5 RIRs depleted Address Markets Transfer Policies routable IPv4 addresses (3.7B) allocated IPv4 addresses 5 10

15 A History of IPv4 Address Block Management 1981 ~1995 ~2011 Early Registration Needs-Based Provision Depletion & Exhaustion Informal Distribution Scarcity minor issue Non-commercial Internet Distribution process Justification of need ISPs don t pay for IPs 4 out of 5 RIRs depleted Address Markets Transfer Policies Network operators around the world need to find ways to mitigate their IPv4 scarcity issues. routable IPv4 addresses (3.7B) cumulative yearly allocations 10

16 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6

17 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6

18 Strategy (i): Use IPv4 space more efficiently Hypothesis IPv4 space not fully utilized Underutilized space could be freed up and used/transferred Research Questions What is the potential for utilization increase? Which knobs could be adjusted here? 6

19 Degrees of Address Use Allocation registered to a network? ~99% 7

20 Degrees of Address Use Allocation registered to a network? ~99% Routing advertised in the global routing table? 7

21 IPv4 Address Activity: Global Routing Table 250 total address space limit (256 /8 equivalents) 200 routable address space limit (220.7 /8 equivalents) /8 equivalents allocated address blocks routed address blocks Nov 1997 Jan 2001 Jan 2005 Jan 2009 Jan 2013 Jan

22 IPv4 Address Activity: Global Routing Table 250 total address space limit (256 /8 equivalents) Mostly LEGACY (pre-1995) allocations 200 routable address space limit (220.7 /8 equivalents) /8 equivalents allocated address blocks routed address blocks Nov 1997 Jan 2001 Jan 2005 Jan 2009 Jan 2013 Jan

23 IPv4 Address Activity: Global Routing Table 250 total address space limit (256 /8 equivalents) Mostly LEGACY (pre-1995) allocations 200 routable address space limit (220.7 /8 equivalents) /8 equivalents allocated address blocks routed address blocks Nov 1997 Jan 2001 Jan 2005 Jan 2009 Jan 2013 Jan 2017 Impact of Internet Governance! 8

24 Degrees of Address Use Allocation registered to a network? ~99% Routing advertised in the global routing table? ~75% 9

25 Degrees of Address Use Allocation registered to a network? ~99% Routing advertised in the global routing table? ~75% Activity actively used? 9

26 Measuring IPv4 Activity Passive Measurements Active Measurements How many IPv4 address blocks show activity? 10

27 IPv4 Activity - Counting Active Addresses Our preliminary study 4 passive vantage points 3 active scanning campaigns 33% (4.8M /24s) 31% (4.5M /24s) total active: 36% (5.3M /24s) Overlap, but each vantage point has unique contribution 11

28 IPv4 Activity - Counting Active Addresses Our preliminary study 4 passive vantage points 3 active scanning campaigns 33% (4.8M /24s) 31% (4.5M /24s) total active: 36% (5.3M /24s) Overlap, but each vantage point has unique contribution Related Work (Zander et al.) 7 passive vantage points & 2 active campaigns total active: 41% (5.9M /24s) 11

29 Degrees of Address Use Allocation registered to a network? ~99% Routing advertised in the global routing table? ~75% Activity actively used? lower bound: ~36-41% (/24s) 12

30 Degrees of Address Use Allocation registered to a network? Routing advertised in the global routing table? Activity actively used? ~99% ~75% lower bound: ~36-41% (/24s) Significant potential for increasing the utilization of the IPv4 address space 12

31 IPv4 Address Activity from a CDN The CDN Vantage Point 200,000+ servers in ASes in 120+ countries Web content, mobile content, software updates, etc. 3 trillion requests on a daily basis 13

32 IPv4 Address Activity from a CDN The CDN Vantage Point 200,000+ servers in ASes in 120+ countries Web content, mobile content, software updates, etc. 3 trillion requests on a daily basis CDN Vantage Point: Active IPv4 Addresses 44% active /24 address blocks (6.5M, lower bound raised) 32% active IPv4 addresses (1.2B) 13

33 IPv4 Address Activity Matrix address space days For each day on which an IP address was active (requested content), we draw a red dot 14

34 Address Activity Matrix at Scale ( Bacon Strips ) 20K adjacent IP addresses (in active /24s), University Network addresses time 15

35 Address Activity Matrix at Scale ( Bacon Strips ) Metrics that can capture address activity in space and time Study the effect of addressing mechanisms on Address activity patterns Utilization (seen from the CDN) 15

36 Patterns: Static Address Blocks IP address activity within / IP address activity within / IP address activity within / time [months] time [months] time [months] University Enterprise ISP Residential ISP Most static address blocks show activity gaps 16

37 Patterns: Dynamic Address Blocks IP address activity within / IP address activity within / IP address activity within / time [months] time [months] time [months] DHCP pool US University residential users US ISP residential users DE ISP Activity/utilization depends on pool size and lease time 17

38 Which Knobs could be adjusted to increase Utilization? Addressing mechanisms impact address activity Utilization seen from the CDN: Static address blocks harbor large supply of potentially unused addresses Dynamic address blocks could be adjusted to free up underutilized space

39 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6 Contribution Multi-perspective analysis of address activity, churn, addressing, and utilization. Findings Strong potential for utilization increase. Knobs to adjust: Governance & Addressing mechanisms. Exhaustion effects, stagnation of routed & active addresses. ACM IMC 16 (Best Paper Award) IEEE JSAC 16

40 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6 Contribution Multi-perspective analysis of address activity, churn, addressing, and utilization. Findings Strong potential for utilization increase. Knobs to adjust: Governance & Addressing mechanisms. Exhaustion effects, stagnation of routed & active addresses. ACM IMC 16 (Best Paper Award) IEEE JSAC 16

41 Multiplex IPv4 space with Carrier-Grade NAT (Carrier-Grade) NAT 18

42 Carrier-Grade NAT CGN allows end-user ISPs to ease scarcity issues At the cost of breaking the end-to-end Internet Nobody really talks about it Uncertainty in the community No systematic studies! 19

43 Carrier-Grade NAT CGN allows end-user ISPs to ease scarcity issues At the cost of breaking the end-to-end Internet Nobody really talks about it Uncertainty in the community No systematic studies! Research Questions How can we detect Carrier-Grade NAT? How widespread is Carrier-Grade NAT? What s the effect on the Internet and its users? 19

44 NATs between Subscribers and the Internet Subscriber ISP Internet NAT44 (subscriber-side) internal space e.g., /16 CPE NAT public IPv4 public IPv4 20

45 NATs between Subscribers and the Internet Subscriber ISP Internet NAT44 (subscriber-side) internal space e.g., /16 CPE NAT public IPv4 NAT44 (carrier-side) Carrier-Grade NAT internal space e.g., /8 public IPv4 NAT444 (subscriber-side and carrier-side) internal space e.g., /16 CPE NAT 20

46 NATs between Subscribers and the Internet Subscriber ISP Internet NAT44 (subscriber-side) internal space e.g., /16 CPE NAT public IPv4 NAT44 (carrier-side) Carrier-Grade NAT internal space e.g., /8 public IPv4 NAT444 (subscriber-side and carrier-side) internal space e.g., /16 CPE NAT BitTorrent DHT ICSI Netalyzr 20

47 The BitTorrent DHT : : :6883 tracker give me peers for torrent XYZ give me peers : :6883 Classic BitTorrent Tracker stores peer contact information BitTorrent DHT Peers store each others contact information We can use DHT peers as vantage points 21

48 Crawling the BitTorrent DHT give me peers DHT crawler 22

49 Crawling the BitTorrent DHT i can reach peer 25fc at :6881 peer 492c at :6881 DHT crawler 22

50 Crawling the BitTorrent DHT i can reach peer 25fc at :6881 peer 492c at :6881 DHT crawler NAT i can reach peer id a82d at :6881 a82d 22

51 Crawling the BitTorrent DHT i can reach peer 25fc at :6881 peer 492c at :6881 DHT crawler NAT A i can reach peer id a82d at :6881 a82d B :6881 A a82d :6881 B 23

52 BitTorrent Peer Leakage Graph In this AS: no CGN detected In this AS: CGN detected 24

53 How widespread is Carrier-Grade NAT Deployment? Tested with BitTorrent/Netalyzr: 1,791 Eyeball ASes 25

54 How widespread is Carrier-Grade NAT Deployment? Tested with BitTorrent/Netalyzr: 1,791 Eyeball ASes Eyeball Networks (Non-Cellular) CGN-positive: 17.1% particularly in the European and Asia-Pacific Region AFRINIC APNIC ARIN LACNIC RIPE % eyeball ASes CGN positive

55 How widespread is Carrier-Grade NAT Deployment? Tested with BitTorrent/Netalyzr: 1,791 Eyeball ASes Eyeball Networks (Non-Cellular) CGN-positive: 17.1% particularly in the European and Asia-Pacific Region AFRINIC APNIC ARIN LACNIC RIPE % eyeball ASes CGN positive Cellular Networks CGN-positive: 94% CGN is the norm for cellular AFRINIC APNIC ARIN LACNIC RIPE % cellular ASes CGN positive

56 How widespread is Carrier-Grade NAT Deployment? Tested with BitTorrent/Netalyzr: 1,791 Eyeball ASes Eyeball Networks (Non-Cellular) CGN-positive: 17.1% particularly in the European and Asia-Pacific Region AFRINIC APNIC ARIN LACNIC RIPE % eyeball ASes CGN positive Cellular Networks CGN-positive: 94% CGN is the norm for cellular AFRINIC APNIC ARIN LACNIC RIPE % cellular ASes CGN positive CGN is reality for the majority of Internet Users 25

57 What s the Impact of Carrier-Grade NATs? private IP1 private IP2 private IP3 home NAT public IP Internet 26

58 What s the Impact of Carrier-Grade NATs? private IP1 private IP2 private IP3 home NAT public IP Internet 1) Directionality 26

59 What s the Impact of Carrier-Grade NATs? internal IP ranges external IP ranges Internet Carrier-Grade NAT 1) Directionality 26

60 What s the Impact of Carrier-Grade NATs? internal IP ranges external IP ranges Internet Carrier-Grade NAT IPint, portint IPext, portext 1) Directionality 26

61 What s the Impact of Carrier-Grade NATs? internal IP ranges external IP ranges Internet Carrier-Grade NAT IPint, portint IPext, portext 1) Directionality 2) Limits/Quotas on flows per subscriber 3) Restrictiveness of NAT mappings, timeouts 26

62 What s the Impact of Carrier-Grade NATs? Deployment Issues Exhaustion of internal IPv4 address space Attribution, host reputation 27

63 What s the Impact of Carrier-Grade NATs? Deployment Issues Exhaustion of internal IPv4 address space Attribution, host reputation Impact on End Users Down to 512 ports/subscriber (128 subscribers/ip) CGN mappings often more restrictive than CPE devices Restricts (or rules out) peer-to-peer connectivity 27

64 What s the Impact of Carrier-Grade NATs? Deployment Issues Exhaustion of internal IPv4 address space Attribution, host reputation Impact on End Users Down to 512 ports/subscriber (128 subscribers/ip) CGN mappings often more restrictive than CPE devices Restricts (or rules out) peer-to-peer connectivity CGNs limit how much Internet subscribers receive CGN means very different things for different ISPs 27

65 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6 Contribution Multi-perspective analysis of address activity, churn, addressing, and utilization. Contribution First broad and systematic study of CGN deployment in the Internet and properties. Findings Strong potential for utilization increase. Knobs to adjust: Governance & Addressing mechanisms. Exhaustion effects, stagnation of routed & active addresses. ACM IMC 16 (Best Paper Award) IEEE JSAC 16 Findings CGNs are very broadly deployed (majority of users). CGNs directly limit end-users connectivity and resources. CGN deployment issues (internal space, attribution,..). ACM IMC 16 (IRTF ANRP Award)

66 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6 Contribution Multi-perspective analysis of address activity, churn, addressing, and utilization. Contribution First broad and systematic study of CGN deployment in the Internet and properties. Findings Strong potential for utilization increase. Knobs to adjust: Governance & Addressing mechanisms. Exhaustion effects, stagnation of routed & active addresses. ACM IMC 16 (Best Paper Award) IEEE JSAC 16 Findings CGNs are very broadly deployed (majority of users). CGNs directly limit end-users connectivity and resources. CGN deployment issues (internal space, attribution,..). ACM IMC 16 (IRTF ANRP Award)

67 Strategy (iii): Transition to IPv6 IPv6 (est. 1998) comes with 128-bit IP addresses Long-term solution to the IPv4 scarcity problem Enormous Task: Replacing the central Internet Protocol Home network Dual-stack ISP Service providers IPv4 traffic IPv6 traffic Internet (i) OS (ii) applications (iii) CPE (iv) ISP connectivity (v) service availability 28

68 Strategy (iii): Transition to IPv6 IPv6 (est. 1998) comes with 128-bit IP addresses Long-term solution to the IPv4 scarcity problem Enormous Task: Replacing the central Internet Protocol Home network Dual-stack ISP Service providers IPv4 traffic IPv6 traffic Internet (i) OS (ii) applications (iii) CPE (iv) ISP connectivity (v) service availability As of 2017: A minority of Internet hosts speak IPv6 Majority of Internet traffic carried over IPv4 28

69 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6 Contribution Multi-perspective analysis of address activity, churn, addressing, and utilization. Contribution First broad and systematic study of CGN deployment in the Internet and properties. Contribution Analysis of IPv4/IPv6 connectivity, traffic components, and interplay. Findings Strong potential for utilization increase. Knobs to adjust: Governance & Addressing mechanisms. Exhaustion effects, stagnation of routed & active addresses. ACM IMC 16 (Best Paper Award) IEEE JSAC 16 Findings CGNs are very broadly deployed (majority of users). CGNs directly limit end-users connectivity and resources. CGN deployment issues (internal space, attribution,..). ACM IMC 16 (IRTF ANRP Award) Findings IPv6 connectivity increases, yet lags behind IPv4. Traffic over IPv6 lags behind connectivity. Barriers for IPv6 traffic (devices, software, networks). ACM IMC 14, PAM 15, PAM 17

70 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6 Contribution Multi-perspective analysis of address activity, churn, addressing, and utilization. Contribution First broad and systematic study of CGN deployment in the Internet and properties. Contribution Analysis of IPv4/IPv6 connectivity, traffic components, and interplay. Findings Strong potential for utilization increase. Knobs to adjust: Governance & Addressing mechanisms. Exhaustion effects, stagnation of routed & active addresses. ACM IMC 16 (Best Paper Award) IEEE JSAC 16 Findings CGNs are very broadly deployed (majority of users). CGNs directly limit end-users connectivity and resources. CGN deployment issues (internal space, attribution,..). ACM IMC 16 (IRTF ANRP Award) Findings IPv6 connectivity increases, yet lags behind IPv4. Traffic over IPv6 lags behind connectivity. Barriers for IPv6 traffic (devices, software, networks). ACM IMC 14, PAM 15, PAM 17

71 IPv4 Exhaustion: An Unprecedented Problem IPv4 addresses are truly global virtual resources No central authority, independent decisions 29

72 IPv4 Exhaustion: An Unprecedented Problem IPv4 addresses are truly global virtual resources No central authority, independent decisions Looming IPv4 exhaustion was recognized early (~1990) Yet, IPv4 supplies lasted until ~

73 IPv4 Exhaustion: An Unprecedented Problem IPv4 addresses are truly global virtual resources No central authority, independent decisions Looming IPv4 exhaustion was recognized early (~1990) Yet, IPv4 supplies lasted until ~2011 Today: Economic pressure due to IPv4 scarcity! Growing IPv4 address markets Widespread Carrier-Grade NAT deployment Increasing Dual-Stack IPv4/IPv6 deployment 29

74 Systematic Framing of IPv4 Exhaustion ACM CCR 15 (Best of CCR) mitigation strategies Use IPv4 space more efficiently Multiplex IPv4: Carrier-Grade NAT Transition to IPv6 Contribution Multi-perspective analysis of address activity, churn, addressing, and utilization. Contribution First broad and systematic study of CGN deployment in the Internet and properties. Contribution Analysis of IPv4/IPv6 connectivity, traffic components, and interplay. Findings Strong potential for utilization increase. Knobs to adjust: Governance & Addressing mechanisms. Exhaustion effects, stagnation of routed & active addresses. ACM IMC 16 (Best Paper Award) IEEE JSAC 16 Findings CGNs are very broadly deployed (majority of users). CGNs directly limit end-users connectivity and resources. CGN deployment issues (internal space, attribution,..). ACM IMC 16 (IRTF ANRP Award) Findings IPv6 connectivity increases, yet lags behind IPv4. Traffic over IPv6 lags behind connectivity. Barriers for IPv6 traffic (devices, software, networks). ACM IMC 14, PAM 15, PAM 17