BGP Communities: A measurement study

Transcription

1 BGP Communities: A measurement Amsterdam Florian Streibelt 1, Franziska Lichtblau 1, Robert Beverly 2, Cristel Pelsser 3, Georgios Smaragdakis 4, Randy Bush 5, Anja Feldmann 1 Oct Max Planck Institute for Informatics (MPII), 2 Naval Postgraduate School (NPS), 3 University of Strasbourg, 4 TU Berlin (TUB), 5 Internet Initiative Jaan (IIJ)

2 BGB-Communities: A weaon for the Internet! RIPE 77 / Amsterdam

3 Introduction

4 BGP Community usage is increasing 70k 40k # Unique ASes in Communities # Unique Communities 7B 4B 20k 2B 10k 1B 5k 3k 2k # Absolute Communities # BGP table entries 0.5B 0.3B 0.2B Year Increasing usage warrants a closer look. 2

5 BGP Community usage is increasing 70k 40k # Unique ASes in Communities # Unique Communities 7B 4B 20k 2B 10k 1B 5k 0.5B 3k 2k # Absolute Communities # BGP table entries 0.3B 0.2B Year Increasing usage warrants a closer look. 2

6 BGP Communities Otional Attribute in BGP message (32 bit field) Defined in RFC 1997 By convention written ASN:VALUE ASN can be both sender or intended reciient It s u to the eers to agree uon values used 3

7 BGP Large Communities Defined by RFC 8092 (usage recommendations ins RFC 8195) 12 byte attribute Enable networks with 4-byte ASNs to use communities The first 4 byte contain the ASN of the global administrator 4

8 BGP Large Communities Sorry... as we only found a very small number of occurrences 1 we could not conduct any meaningful measurements, yet individual large communities by 51 global administrators over the whole month of Aril 2018 at all available route collectors at RIPE/RIS, Routeviews, Isolario and PCH 4

9 BGP Communities: Usage Informational Communities (Passive Semantics) Location tagging RTT tagging Action Communities (Active Semantics) Remote triggered blackholing Path reending Local ref/med Selective announcements Without documentation, you can not tell if a community is active or assive! 5

10 What This Talk Is About Given the increasing oularity of BGP communities and the ability to trigger actions as well as relay information, the first question that comes to the mind of an Internet measurement researcher is... 6

11 What This Talk Is About What could ossibly go wrong? 6

12 Proagation behavior 7

13 Proagation behavior 14% of transit roviders roagate received communities (2.2k of 15.5k) Ratio seems small, but AS grah is highly connected RFC 1997: Communities as a transitive otional attribute RFC 7454: Scrub own, forward foreign communities Still many eole do not exect communities to roagate that widely. 7

14 Potential (for) misuse Proagated communities might trigger actions multile AS-hos away No way of knowing if intended or not, e.g., for traffic management But are there also unintended consequences? Our assessment is that there is a high risk for attacks! 8

15 Observations

16 Dataset BGP udates and table dums of Aril 2018 from ublicly available BGP Collector Projects: RIPE RIS, Routeviews, Isolario, PCH. BGP messages bn IPv4 refixes 967,499 IPv6 refixes 84,953 Collectors 194 AS eers 2,133 Communities 63,797 More than 75% of all BGP announcements have at least one BGP community set, 5,659 ASes are using communities. 9

17 BGP Community Proagation Observations Fraction of communities (ECDF) AS ho count 10% of communities have a AS ho count of more than six More than 50% of communities traverse more than four ASes Longest community roagation observed: 11 AS hos 10

18 BGP Community Proagation Observations Fraction of communities (ECDF) AS ho count 10% of communities have a AS ho count of more than six More than 50% of communities traverse more than four ASes Longest community roagation observed: 11 AS hos 10

19 BGP Community Proagation Observations Fraction of communities (ECDF) AS ho count 10% of communities have a AS ho count of more than six More than 50% of communities traverse more than four ASes Longest community roagation observed: 11 AS hos 10

20 BGP Community Proagation Behavior AS3 AS1 AS2 AS4 11

21 BGP Community Proagation Behavior AS3 AS1 AS2 AS4 AS1 announces refix 11

22 BGP Community Proagation Behavior AS3 AS1 3:123 AS2 3:123 3:123 AS4 AS1 announces refix, tagged with 3:123 11

23 BGP Community Proagation Behavior AS3 AS1 3:123 AS2 3:123 3:123 AS4 AS1 announces refix, tagged with 3:123 Community is intended for signaling towards AS3 11

24 BGP Community Proagation Behavior AS3 AS1 3:123 AS2 3:123 3:123 AS4 AS1 announces refix, tagged with 3:123 Community is intended for signaling towards AS3 AS4 also receives this announcement 11

25 BGP Community Proagation Behavior AS1 3:123 AS2 3:123 3:123 AS3 AS4 : 3, 2, 1 3:123 : 4, 2, 1 3:123 AS1 announces refix, tagged with 3:123 Community is intended for signaling towards AS3 AS4 also receives this announcement 11

26 BGP Community Proagation Behavior AS1 3:123 AS2 3:123 3:123 AS3 AS4 "on ath" : 3, 2, 1 3:123 : 4, 2, 1 3:123 AS1 announces refix, tagged with 3:123 Community is intended for signaling towards AS3 AS4 also receives this announcement 11

27 BGP Community Proagation Behavior AS1 3:123 AS2 3:123 3:123 AS3 AS4 "on ath" : 3, 2, 1 3:123 "off ath" : 4, 2, 1 3:123 AS1 announces refix, tagged with 3:123 Community is intended for signaling towards AS3 AS4 also receives this announcement Off-ath: ASN from community is not on the observed AS-ath at AS4. 11

28 On-ath versus off-ath 1.2 % communities observed off-ath on-ath Blackholing communities (e.g., :666) leaking off ath But AS imlementing RTBH SHOULD add NO ADVERTISE or NO EXPORT (RFC7999) Suggests ASes not imlementing RTBH do not filter. 12

29 Exeriments

30 Exerimental setu Exeriments conducted in a lab environment Validated on the Internet Scenarios Remote Triggered Blackholing (RTBH) Traffic redirection attack...for others see our aer. 13

31 RTBH: how it works AS3 AS4 AS2 AS5 AS1 14

32 RTBH: how it works BGP announcements AS3 AS4 AS1 AS2 AS5 14

33 RTBH: how it works Traffic flow BGP announcements AS3 AS4 AS1 AS2 AS5 14

34 RTBH: how it works Traffic flow BGP announcements AS3 AS4 AS announces BH-refix to ustream 2:666 AS2 AS5 AS1 AS1 sends, tagged 2:666 14

35 RTBH: how it works Traffic flow BGP announcements AS3 AS4 AS announces BH-refix to ustream 2:666 AS2 AS5 AS1 AS1 sends, tagged 2:666 AS2 continues announcing 14

36 RTBH: how it works Traffic flow BGP announcements AS3 AS4 AS announces BH-refix to ustream 2:666 AS2 AS5 AS1 AS1 sends, tagged 2:666 AS2 continues announcing 14

37 RTBH: how it works Traffic flow BGP announcements AS3 AS4 AS announces BH-refix to ustream Provider blackholes refix 2:666 AS1 AS2 X AS5 AS1 sends, tagged 2:666 AS2 continues announcing 14

38 RTBH: how it works Traffic flow BGP announcements AS3 AS4 AS announces BH-refix to ustream Provider blackholes refix 2:666 AS1 AS2 X AS5 AS1 sends, tagged 2:666 AS2 continues announcing Traffic to is droed at AS2 14

39 RTBH: how it works Traffic flow BGP announcements AS3 AS4 AS announces BH-refix to ustream Provider blackholes refix 2:666 AS1 AS2 X AS5 AS1 sends, tagged 2:666 AS2 continues announcing Traffic to is droed at AS2 Safeguards: Provider should check customer refix before acceting RTBH Customer may only blackhole own refixes Different olicies for Customers/Peers On receiving RTBH, add NO ADVERTISE or NO EXPORT (RFC7999) 14

40 RTBH: how it should not work AS2 BGP announcements AS3 AS4 AS1 15

41 RTBH: how it should not work AS2 Traffic flow BGP announcements AS3 AS4 AS1 AS1 announces 15

42 RTBH: how it should not work AS2 Attacker Community Target Traffic flow BGP announcements AS3 AS4 AS1 Attackee AS1 announces 15

43 RTBH: how it should not work AS2 Attacker AS3:666 Community Target AS3 Traffic flow BGP announcements AS4 AS1 Attackee AS1 announces AS2 tags with AS3:666 15

44 RTBH: how it should not work AS2 AS1 Attacker AS3:666 Attackee Community Target AS3 X Traffic flow BGP announcements AS4 AS1 announces AS2 tags with AS3:666 Traffic to is droed at AS3 15

45 RTBH: how it should not work AS2 AS1 Attacker AS3:666 Attackee Community Target AS3 X Traffic flow BGP announcements AS4 AS1 announces AS2 tags with AS3:666 Traffic to is droed at AS3 AS on backu ath adds RTBH-community Provider blackholes refix Not only traffic traversing AS2 is droed 15

46 RTBH: how it should not work (with hijack) AS2 AS1 Attacker AS3:666 Attackee Community Target AS3 X Traffic flow BGP announcements AS4 AS1 announces AS2 hijacks, with AS3:666 Traffic to is droed at AS3 Hijacker announces RTBH Prefix filters circumvented due to misconfiguration Provider blackholes refix 16

47 RTBH: Attack confirmed Attack confirmed to work on the Internet, works multi ho and is hard to sot Triggering RTBH is ossible for attackers because, e.g.,: BH refix is more secific, acceted via excetion Providers check BH community before refix filters 2 NO ADVERTISE or NO EXPORT often is ignored / not set Problem: No validation for origin of community 2 we found configuration guides with that bug 17

48 Traffic redirection attack AS4 AS5 AS1 AS2 AS6 AS3 18

49 Traffic redirection attack AS1 AS2 AS4 AS5 AS6 AS3 BGP Announcements 18

50 Traffic redirection attack AS1 AS2 AS4 AS5 AS6 AS Paths at AS6: : 5, 4, 2, 1 : 3, 2, 1 AS3 Trafficflow BGP Announcements 18

51 Traffic redirection attack AS1 AS2 Attackee Attacker Trafficflow BGP Announcements AS4 AS5 AS6 AS3 Community Target AS Paths at AS6: : 5, 4, 2, 1 : 3, 2, 1 18

52 Traffic redirection attack AS4 AS5 AS1 AS2 AS6 Attackee Attacker AS3 AS3:3x Trafficflow Community Target BGP Announcements AS Paths at AS6: : 5, 4, 2, 1 : 3, 3, 3, 2, 1 Attacker AS2 uses community to add ath-reending in AS3 18

53 Traffic redirection attack AS4 AS5 AS1 AS2 AS6 Attackee Attacker AS3 AS3:3x Trafficflow Community Target BGP Announcements AS Paths at AS6: : 5, 4, 2, 1 : 3, 3, 3, 2, 1 Attacker AS2 uses community to add ath-reending in AS3 AS6 routes traffic towards refix via AS5, AS4 18

54 Traffic redirection attack AS4 AS5 AS1 AS2 AS6 Attackee Attacker AS3 AS3:3x Trafficflow Community Target BGP Announcements AS Paths at AS6: : 5, 4, 2, 1 : 3, 3, 3, 2, 1 Attacker AS2 uses community to add ath-reending in AS3 AS6 routes traffic towards refix via AS5, AS4 18

55 Traffic redirection attack AS4 </> AS1 AS2 AS6 Attackee Attacker AS3 AS3:3x Trafficflow Community Target BGP Announcements AS5 AS Paths at AS6: : 5, 4, 2, 1 : 3, 3, 3, 2, 1 Attacker AS2 uses community to add ath-reending in AS3 AS6 routes traffic towards refix via AS5, AS4 Network ta? 18

56 Traffic redirection attack AS4 AS5 AS1 AS2 AS6 Attackee Attacker AS3 AS3:3x Trafficflow Community Target BGP Announcements AS Paths at AS6: : 5, 4, 2, 1 : 3, 3, 3, 2, 1 Attacker AS2 uses community to add ath-reending in AS3 AS6 routes traffic towards refix via AS5, AS4 Network ta? Slow/Congested link?... 18

57 Discussion: What now?

58 BGP Communities Shortcomings Summarized Notation of ASN:value is just convention No defined semantics: values can mean anything Used both for signaling and triggering of actions No crytograhic rotection Attribution is imossible Large Communities have, in rincile, similar limitations 19

59 20

60 BGP Communities: The Problem BGP Communities as they are used are not necessarily broken Secure usage requires good oerational knowledge and diligence 21

61 BGP Communities: The Problem BGP Communities as they are used are not necessarily broken Secure usage requires good oerational knowledge and diligence While eole in this room robably know what they are doing: Based on exerience we do not rely on that globally... Do we need less fragile rotocols and mechanisms? 21

62 Recommendations Filter incoming Informational Communities for your ASN Publish community documentation, to enable others to filter Monitor and log received communities to track abuse Talk to your Downstreams, so they filter Action Communities for your ASN on ingress if neccessary Provide a looking glass (that shows communties!) 22

63 Discussion: Authenticity Communities can be modified, added, removed by every AS No attribution is ossible No crytograhic rotection Still oerators rely on their correctness Large communities artially imrove the situation How can we achieve authenticity, or at least attribution? 23

64 Discussion: Transitivity Communities can hel in debugging Easy, low overhead communication channel Widely in use, but often only 1-2 hos But: High risk of being abused! Are fully transitive communities still worth the clear risk? 24

65 Discussion: Monitoring There is no global state in BGP Route collectors only see the end-result Inferring modifications between origin-as and collector: almost imossible The meaning of a articular community can not be known No universal way for attribution of changes Monitoring communities to detect abuse is extremely difficult. 25

66 Discussion: Standards There are limited standardized communities Many AS do not imlement these Is the lack of standardized communities a roblem? Are standards doing harm, by heling attackers? Security by obscurity never works Standardization is necessary. 26

67 Discussion: Documentation There is no easy way to find meaning of a community: Some ASes document in the whois Some ASes document on their website Some ASes rovide documentation only to customers Some ASes do not rovide any documentation Documentation is limited and fragmented. 27

68 Summary But: Communities are widely in use Foundation of many olicies Relies heavily on mutual trust in caabilities No authenticity/security in lace Attribution is imossible Hard to detect attacks While our refix hijacks were reorted, no one reorted our community attacks It s unknown if there are other unnoticed attacks. 28

69 Get the rerint at: htts://eole.mi-inf.mg.de/~fstreibelt/rerint/ communities-imc2018.df Published at ACM IMC 2018 htts://conferences.sigcomm.org/imc/2018/ 29

70 30

71 Contact: Florian Streibelt Images: Unicorn illustrations: Telegram stickers by Darya Ogneva: htts://tlgrm.eu/stickers/borntobeaunicorn The Sanish Inquisition: by Miki Montllo htt://miquelmontllo.blogsot.com/2013/10/ the-sanish-inquisition-wallaer.html 31