BGP Fredrik Söderquist Michael Silvin 1
Table of Contents Background...3 A quick look at the mechanics...3 Message Types...3 BGP Message Header...3 OPEN Message (Type 1 RFC 1771)...4 UPDATE Message (Type 2 RFC 1771)...4 KEEPALIVE Message (Type 3 RFC 1771)...5 NOTIFICATION Message (Type 4 RFC 1771)...5 ROUTE-REFRESH Message (Type 5 RFC 2918)...5 ASN...6 Community...6 ibgp...6 ibgp scalability issues...7 Route reflection (RR)...7 Hierarchical RR...7 Confederation...8 Comparision...8 BGP Security...9 TCP MD5 signatures...9 Graded Route Flap Dampening...9 Public Peering...9 Pointing default...9 Third-Party Next Hop...10 Distributed Denial-of-Service Dynamic Black Hole Routing...10 Interconnecting with other networks...10 Transit...10 Peering...10 ISP Tier...10 Dual-homing / Multi-homing...10 Requesting IP address space and AS number...11 RIR, LIR (revisited)...11 PA/PI...11 Route filtering...12 Inbound...12 Outbound...12 Policy routing...12 IRR...12 Routing Policy Specification Language (RPSL)...12 The Maintainer Object...12 Autonomous System Object...13 References...13 2
Background BGP first became an Internet standard in 1989 and was originally defined in RFC 1105. It was then adopted as the EGP of choice for inter-domain routing. The current version, BGP-4, was adopted in 1995 and is defined in RFC 1771. BGP-4 supports Classless Inter Domain Routing (CIDR) and is the routing protocol that people use in today to route between autonomous systems. A quick look at the mechanics BGP uses TCP to establish a reliable connection between two BGP speakers on port 179. Exactly one TCP session is established between each peer for each BGP session We call BGP a path vector protocol, because it stores routing information as a combination of a destination and attributes of the path to that destination. The protocol uses a deterministic route selection process to select the best route from multiple feasible routes, using the path attributes as criteria. Characteristics such as delay, link utilization or router hops are not considered in this process. BGP runs in two modes: ebgp and ibgp. ebgp (External BGP) is run between different autonomous systems, and ibgp (Internal BGP) is run between BGP routers in the same autonomous system. Message Types Five message types are used by BGP to negotiate parameters, exchange routing information and indicate errors. Each message can be between 19 and 4096 bytes long, and relies on TCP/IP for delivery, sequencing and fragmentation. BGP Message Header The BGP message header is used in all messages, and contains the following fields. Marker (16 bytes): contains all 1s (0xFF) and is used for synchronization when multiple messages are in a TCP segment Length (2 bytes): total message length Type (1 bytes): message type Illustration 1BGP Message Header Data (variable): depending on the message type, data may or may not be present 3
OPEN Message (Type 1 RFC 1771) Illustration 2OPEN Message The first BGP message that is sent after the TCP connection has been established is the OPEN message. It is used to exchange configuration information and to negotiate common parameters for the peering session. It contains the following fields. Version (1 byte): BGP version, defaults to BGP-4 and cannot be changed My Autonomous System (2 bytes): AS number of the BGP speaker Hold Time (2 bytes): number of seconds that can elapse without receipt of an UPDATE or KEEPALIVE message before the peer is assumed to be down (Riverstone s default is 180 seconds) BGP Identifier (4 bytes): sender s BGP ID, equal to the router ID Optional Parameter Length (1 byte): length is set to 0 if none are present Optional Parameters (variable): Header password authentication (RFC 1771), defined in the RFC but not implemented by any vendor Capabilities advertisement (RFC 2842), provides a mechanism to negotiate the capability to use various BGP features 4
UPDATE Message (Type 2 RFC 1771) Illustration 3UPDATE Message UPDATE messages are used to distribute the routing information in BGP, and are only sent after the session is established. An UPDATE message can be used to withdraw existing routes, advertise new routes, or both. In this message, 2-tuples consisting of a mask length and prefix are used to represent IP prefixes. These prefixes are used to communicate which routes are to be withdrawn, or which routes are to be advertised. When routes are advertised, the prefixes are called Network Layer Reachability Information (NLRI). Often, several path attributes are shared among the same prefixes, for example if they all are originated by the same BGP speaker. In this case, BGP will send the path attributes and the associated prefixes as NLRI in the same message. This makes sending and storing the routing information much more efficient. Similarly, multiple IP prefixes can be withdrawn at the same time. The following fields comprise the UPDATE message. Withdrawn Routes Length (2 bytes): length of withdrawn routes field, a length of 0 means there are no withdrawn routes Withdrawn Routes (variable): 2-tuple IP prefix(es) of unfeasible route(s) to be withdrawn Total Path Attribute Length (2 bytes): total length of the path attribute information, a length of 0 means there is no NLRI Path Attributes (variable): path attributes represented as variable length TLVsNLRI (variable): 2- tuple IP prefix(es) of reachability information KEEPALIVE Message (Type 3 RFC 1771) KEEPALIVE messages are sent periodically at 1/3 the Hold Time to indicate that a peer is still operating normally to keep the BGP session alive (Riverstone s default is 60 seconds). This message only contains the BGP header and no data. 5
NOTIFICATION Message (Type 4 RFC 1771) The NOTIFICATION message is sent when BGP detects an error condition, after which the peering session is terminated and the TCP is connection is closed. The cause of the error condition is sent to the peer for debugging and troubleshooting. Error codes are defined in RFC 1771 and indicate exactly what the problem is through an error code and subcode. A NOTIFICATION message contains the following fields. Error Code (1 byte): type of error Illustration 4NOTIFICATION Message Error Subcode (1 byte): indicates more details about the error Data (variable): optional error data ROUTE-REFRESH Message (Type 5 RFC 2918) Illustration 5ROUTE-REFRESH Message The ROUTE-REFRESH message is not defined in RFC 1771, but as a BGP capability in RFC 2918. However, it has been so widely implemented that is only makes sense to discuss this message in this section with the other BGP messages. Route refresh is used to request a complete retransmission of a peer s routing information without tearing down and reestablishing the BGP session. Using this feature, routing policy changes can be made without storing an unmodified copy of the peer s routes on the local router, which in turn saves RAM and resources. The ROUTE-REFRESH message was designed to be protocol independent. Thus you can request a retransmission of a peer s IPv4 unicast routes only, but none of it s IPv6 routes, for example. ROUTE-REFRESH messages use the following fields. AFI (2 bytes): Address Family Identifier, for example IPv4 or Ipv6 Reserved (1 byte): always set to 0 SAFI (1 byte): Subsequent Address Family Identifier, for example unicast or multicast ASN AS Number; A 16-bit number identifying an AS. The numbers in the range 64512 to 65535 has been reserved for private use by IANA. Private ASN:s should not be advertised to the general Internet. Community A path attribute. A group of prefixes that share a common property. A prefix may reside in several communities. Two types of communites exist: Well-known communities (Communities with predefined meanings, RFC 1997) 6
Private communities (Communities defined by the administrators) Some examples of well-known communities are: NO_EXPORT (not to advertised to ebgp) NO_ADVERTISE (do not advertise to any peer) INTERNET (advertise to the Internet community no restrictions) ibgp Internal BGP; ibgp is an intra AS version of BGP. When a router advertised prefixes via ibgp, it does not add its own AS number to the AS_PATH. Due to this fact, it is possible for a routing information loop to form within the ibgp domain.this is avoided by disallowing advertisement of prefixes learned via ibgp to other ibgp peers. For this to work, the ibgp speakers have to be fully meshed (every router should be connected to all other routers). The use of ibgp doesn't outrule the usage of an IGP one is often required to provide infrastructure reachability inside the AS. To sum up, you could say that an IGP provides information on how to traverse an AS, while ibgp provides information about a location in an AS without information about how to reach it. ibgp scalability issues Due to the requirement of full mesh connectivity by ibgp, an AS with a large number of BGP speakers would require a lot of sessions between all the speakers. Having a lot of sessions lead to high resource consumption in each of the routers. In this section we will have a look at some different solutions for making a BGP routing architecture more scalable. Route reflection (RR) RR is a technique where a special group of routers called Route Reflectors (RRs) are created. The loop prevention rules of ibgp are relaxed for these routers. RRs are allowed to readvertise routes from one ibgp speaker to another ibgp speaker only under certain restrictions. Using RR, speakers are classified as: Route Reflectors (RRs) Route reflector clients (clients) Regular ibgp speakers (nonclients) A client can of course be both a Route Reflector and a client, since client and RR are relative attributes, i.e. a client of a Route Reflector can be a Route Reflector of another client. A Route Reflector can reflect routes from regular client to nonclients and vice versa, and also between one client and another. This gives that full mesh connectivity is required only between RRs and between RRs and nonclients. In an example with 5 routers, a non-rr solution would require 10 sessions, whereas a solution with one router acting as a Route Reflector, two routers being clients and the rest being nonclients, would require only 5 sessions. When using RR, only the best path for each prefix is reflected. When a Route Reflector receives multiple paths for the same destination, it first determines which of these paths are the best, and then 7
reflects the selected path. There is a downside to this approach, and that is that routing information can be lost, or routing loops might happen. To avoid creating routing information loops, some rules applied for prefix advertisement involving Route Reflectors: 1. A Route Reflector advertises or reflects only its best path. 2. A Route Reflector always advertises to ebgp peers. 3. A client follows the regular ibgp loop-prevention rules when advertising prefixes. 4. When advertising to ibgp peers, the rules are dependent on where the prefix is learned (see rules below). 5. If a Route Reflector learns a prefix from an external peer, it advertises to all clients and nonclients. 6. If the prefix comes to the Route Reflector through a nonclient, the route is reflected to all clients. 7. If the prefix comes to the Route Reflector from a client, the route is reflected to all other clients and nonclients. Hierarchical RR Illustration 6A RR example A method to further reduce the number of sessions in a routing domain, is to introduce RR hierarchicies. With hierarchical RR there are several levels of Route Reflectors, with lower level Route Reflectors 8
acting as clients for higher level ones. There are no limit on the maximum number of levels, but seldom more than two or three levels are used. Route Reflection is described in RFC 2796. Illustration 7Example of Hierarchical RR Confederation Confederation solves the full mesh issue by splitting a AS into a number of smaller ASs. These smaller AS:es are called member autonomous systems (member-as) or subautonomous systems (sub-as). ebgp sessions are used among the member-as:es, and therefore a full mesh is not required. It is is still a requirement to have full mesh connectivity within a member-as. An ebgp session (intraconfederation ebgp session) within a confederation is different from a normal ebgp session. The intraconfederation ebgp session follows ibgp rules for prefix advertisment in some cases, and ebgp rules in others. For example the AS_PATH attribute is modified when sending updates. To an external peer, these changes are not visible, since the AS_PATH is modified again before the updates reach outside the confederation. It was mentioned above that full mesh connectivity was required in member-as:es. RR, as described above, can also be used inside of the member-as:es. A member-as does not even need to use BGP as their IGP. A confederation has three different types of peerings: 1. External peerings (from confederation to external) 9
Illustration 8An example Confederation 2. Confederation external peerings (between member-as:es) 3. Internal peerings (between peers in a member-as) With confederation two new AS_PATH segment types where introduced AS_CONFED_SEQUENCE and AS_CONFED_SET. These two has principally the same meaning as their non-confederation version. The AS_PATH is changed differently depending on the type of session. For the above mentioned types of peering, the following apply: 1. Member-AS numbers are removed from the AS_PATH, and the confederation number is prepended. 2. Member-AS number is prepended to the AS_CONFED_{SEQUENCE,SET}. 3. AS_PATH is not modified. The member-as numbers used within the confederation is never visible from the outside of the confederation. This allows for usage of any range of ASN:s to identify member-as:es. However a private AS number is typically used. Confederation is described in RFC 3065. Comparision Here we will take a quick look at how the above two techniques differ. Parameter RR Confederation Multilevel hierarchy Yes Yes (RR in member-as) Migration complexity Minor Major changes to configuration and architecture 10
Parameter RR Confederation Capabilities support Only Route Reflectors required to have support for RR All routers need to understand the confederation AS_PATH attribute IGP expansion Single IGP inside AS Single or separate IGPs BGP Security Protecting the BGP infrastructure (sessions), and protecting against malicious advertisements. TCP MD5 signatures An attacker can reset a session by sending TCP reset packets to one of the peers in session. To overcome this, or atleast make this harder, a session can be configured to use a MD5 signature (as a TCP option, see RFC 2385) that is based on the data in the packet, and a password. Graded Route Flap Dampening A route flap is a routing change that causes a change in the BGP tables. This could for example be caused by links going up and down. If we have a large number of flaps in a short period of time, there will be high load on the router because it will calculate new paths thru the network to reach certain destinations. A solution to this problem is called Route Flap Dampening (RFD). RFD is described in RFC 2439, and aims at the following: Reduce router processing load caused by unstable routes. Prevent route oscillation. Increase route stability without sacrificing convergence time for well-behaved routes. The RFD process works by having the router keep track of the of the flapping history for every prefix. Several parameters are also kept for the prefixes, such as state, penalty, suppress limit, half life, reuse limit and maximum suppress limit. When a route flaps, it gets a penalty increase, and when it has reached a certain level (the suppress limit) the route is dampened, i.e. the prefix is no longer advertised. Based on the half life time, the penalty is decreased. When the penalty falls below the reuse limit, the prefix is advertised again. The maximum suppress limit parameter allows for an upper bound on the penalty. This is helpful if a route is very instable for short period of time, and after that stabilizes. Having the upper bound in this case leads to a short suppression time for this particular route. In the standard case, all prefixes are equal with respect to Route Flap Dampening. This is not true in reality a /8 represents so many more hosts than for example a /24, so the suppression time for /8 would naturally be shorter than for the /24. For essential prefixes, such as those for the DNS root servers. In these cases, no graded dampening should be performed. Public Peering In the case of public peering, it is possible to redirect traffic over other providers networks, or build tunnel to offload traffic from one ISP to a peer of that ISP. Pointing default Originating a default route into ISP network thru a NAP router. Traffic is then sent to the NAP router of 11
the ISP. BGP To solve this, full BGP routes should not be carried by the NAP router. If the NAP router carries only customer routes, malicious traffic will be black holed. Because transit services are not provided on the NAP router, it does not need to know full routing tables. The only inbound traffic should be to the customers of the ISP. Third-Party Next Hop This method achives the opposite from the above method. The ISP attempts to redirect inbound traffic elsewhere (a peering link) to reduce the load on its backbone/transit links. The solution to this problem is the same as in the Pointing default case. Distributed Denial-of-Service Dynamic Black Hole Routing The basic idea of Dynamic Black Hole Routing (DBHR) is to advertise a BGP prefix and set the nexthop to a address that is a null route. The victim of the DDoS will have its prefix/address advertised with its next-hop set to the null route. This prefix is then advertised (using ibgp) to the edge routers. This results in the DDoS stopping at the edge of the network. A sink router can also be used in case you would like to analyse the DDoS traffic. In this case the prefix advertised should point at this sink router instead of at the null route. Interconnecting with other networks Transit Transit connectivity essential means buying full connectivity to the Internet from another ISP. An ISP sells transit service to its end customers. A transit connection means a that the upstream provider lets the customer transit its network to reach any available destination on the Internet. Peering Peering between two ISPs means that the reachability to the ISP (and its direct customers) is provided over that connection. The two ISP can reach each other, but not a third ISP (if it is not a customer of either of the ISPs). A peering link offloads traffic passing between the ISPs customers from their transit links. There a two types of peering public and private. Public peering occurs at public peering points such as Internet Exchange Points. Private peering means that two ISPs make an agreement and establish a connection between themselves. ISP Tier There are three types of tiers: Tier 1 Tier level Nationwide backbone Comments Does not purchase any transit Relies completely on peering 12
Tier 2 Tier 3 Tier level BGP Comments Nationwide backbone Combination of peering and transit Regional or local network Relies almost completely on transit Might have some peering, but often does not Dual-homing / Multi-homing This is the primary reason to use BGP, when you want to connect your network to two or more ISPs, so called multi-homing. By announcing reachability information for your network to two ISPs, you can make sure everybody still knows how to reach you if one of those ISPs has an outage. Requesting IP address space and AS number The Internet Assigned Numbers Authority (IANA) is responsible for assigning the protocol numbers used on the Internet. This includes IP addresses and AS numbers. RIR, LIR (revisited) But IANA has delegated these activities to a few Regional Internet Registries (RIRs), at the moment there are four: APNIC (Asia Pacific Network Information Centre) - Asia/Pacific Region ARIN (American Registry for Internet Numbers) - North America and Sub-Sahara Africa LACNIC (Regional Latin-American and Caribbean IP Address Registry) Latin America and some Caribbean Islands (since nov 2002) RIPE NCC (Réseaux IP Européens) - Europe, the Middle East, Central Asia, and African countries located north of the equator There is also work currently underway to establish an African RIR (AfriNIC). The RIRs in their turn delegate responsibility for assigning IP address space to Local Internet Registries (LIRs) or to the ISPs directly (which are considered LIRs by RIPE). PA/PI All IP-addresses aren t created equal. There are two types of addresses. First there is the Provider Aggregatable (PA). This is the usual type and PA-blocks is given to ISPs so that they in turn can give their customers addresses from this block. The smallest PA-blocks allocated to ISP today are /20 s, this is of some importance as you will see later. The second type is Provider Independent (PI). These are given directly to network customers. These are fairly rare and to be able to get a PI-block you need to be multihomed. With this in mind there are a few ways to go about when requesting address-space. Announcing a provider independent prefix 13
This is the simplest solution but has some problems when being subject to route filtering. You are almost guaranteed to be filtered if announcing a network smaller than /24 and many networks filter prefixes longer than the smallest PA-block allocated (/20). If you announce a network smaller than that you can have reachability problems. Shooting holes in ISP PA block This is when you use part of one of your ISPs PA block and announce this as a PI block over the other ISP. With no filtering this provides the same benefits as the first case with the exception that if you leave your primary ISP you have to renumber. The upside is that if your longer prefix is filtered you can still be reached from those networks of the shorter prefix that your primary ISP announces but you will appear as singlehomed to these networks so if your primary ISP goes down you will not be reached. It is also important that your primary ISP accepts the announced prefix from the secondary ISP, if this isn t the case there will be major problems if the link between you and your primary ISP goes down. Request your own PA-block These are the highest quality addresses and this is the best way to avoid filtering. Since you aren t expected to use the entire block yourself you can get a larger block than you would be able to get with PI space under similar circumstances. Route filtering Two types of route filtering, inbound filtering and outbound filtering. Inbound RFC1918 addresses, Intended for private networks, should never be advertised globally. System local addresses, 127.0.0.0/8 is reserved for use internal to a system. End node autoconfiguration block, 169.254.0.0/16, intended for automatic address assignment when a DHCP server is unavailable 0.0.0.0/8, sometimes used internally. Is not assigned and should not be used. Test network addressing, 192.0.2.0/24, is reserved for test networks. Intended for use in documentation and sample code. Class D and E space. Class D is 224.0.0.0/4 and reserved for multicast group and are not advertised by unicast routing protocols. Class E 240.0.0.0/4 is reserved and not in use. Bogons Outbound To protect you from misconfiguration, for example you don t want to announce the default route to you peers and in that way provide transit for them. If you don t have enough bandwidth things will go terribly wrong. 14
Policy routing IRR The Internet Routing Registry (IRR) is a repository of routing policies. There are a few around the globe which exchange its data among each other so you should only register with one of them, in Europe preferably the RIPE registry. Registering with more than one will probably lead to inconsistent data. Routing Policy Specification Language (RPSL) Based on RIPE-181 (RIPE-81++). This document describes the original database formats used by the RIPE NCC for the storage of routing policy in its database. The key purpose of RPSL is to allow you to specify you routing configuration so that you and others can check your policies and announcements for consistency. Moreover you can base your policies and router configuration on other peoples policies. The Maintainer Object The maintainer object is used to introduce some kind of authorization for registrations. Lists various contact persons and describes security mechanisms that will be applied when updating objects in the IRR. First step in creating policies for an AS. mntner: MAINT-AS3701 descr: Network for Research and Engineering in Oregon remark: Internal Backbone admin-c: DMM65 tech-c: DMM65 upd-to: noc@nero.net auth: CRYPT-PW 949WK1mirBy6c auth: MAIL-FROM.*@nero.net notify: noc@nero.net mnt-by: MAINT-AS3701 changed: meyer@antc.uoregon.edu 970318 source: RADB Autonomous System Object The peering policies of an AS are registered in an aut-num object which looks something like this: aut-num: AS2 as-name: CAT-NET descr: Catatonic State University import: from AS1 accept ANY import: from AS3 accept <^AS3+$> export: to AS3 announce ANY export: to AS1 announce AS2 AS3 admin-c: AO36-RIPE tech-c: CO19-RIPE mnt-by: OPS4-RIPE 15
changed: source: orange@ripe.net RIPE Typical for a small service provider providing connectivity for a customer AS3 and using AS1 for transit. It accepts all routes from AS1, accepts routes from AS3 that only contains AS3 in the AS-path. whois h whois.ripe.net AS1653 (sunet) References Iljitch van Beijnum, BGP (2002), O'Reilly Rekhter Y. and Li T., RFC 1771 - A Border Gateway Protocol 4 (BGP-4) (1995) Rekhter Y. and Gross P., RFC 1772 - Application of the Border Gateway Protocol in the Internet (1995) Zhang R. and Bartell M., BGP Design and Implementation (2003), Cisco Press 16