BGP Border Gateway Protocol A short introduction Karst Koymans Informatics Institute University of Amsterdam (version 18.3, 2018/12/03 13:53:22) Tuesday, December 4, 2018 General ideas behind BGP Background Providers, Customers and Peers External and Internal BGP BGP information bases The BGP protocol BGP attributes BGP messages Traffic Engineering Outbound Traffic Engineering Inbound Traffic Engineering IBGP scaling BGP version 4 Autonomous system (AS) Border Gateway Protocol version 4 (BGP4) Specified in RFC 4271 The inter-as routing protocol Monopolises the Internet Based on path vector routing which is in-between distance vector and link state routing Uses (often non-coordinated) routing policies which can be problematic for convergence Definition (AS Autonomous System) A connected group of networks and routers representing some assigned set of IP prefixes having a single, consistent routing policy both internally and externally
Autonomous system illustration Providers and Customers Internet IP IP Provider IP Customer Internet The IP means IP data packet flow The means money transfer Peers Providers, Customers and Peers Provider 1 Provider 2 Provider 3 IP IP Customer 1 Customer 2 Customer 3 G1 IP R1 G2 No packets No packets here means that packet flow between Customer 1 and Customer 3 usually will not flow through Provider 2. C1 C2 P1 P2 IP C3 C4 What route will traffic between C1 and C2 take?
The AS abstraction Different AS types Source: RIPE NCC BGP course Providers, Customers and Peers routing preferences Providers, Customers and Peers: Export filtering The order of preference for incoming route advertisements is Customers have highest preference Peers have the next highest preference Providers have the lowest preference Transit relationships are enforced by export filtering Outgoing route advertisements attract incoming traffic Do not advertise provider or peer routes to other providers or peers Do advertise all routes to customers Do advertise customer routes to providers and peers
External and Internal BGP (1) External and Internal BGP (2) EBGP (External BGP) Used for BGP neighbors between different ASs Exchanging prefixes Implementing policies IBGP (Internal BGP) Used for BGP neighbors within one and the same AS Distributing Internet prefixes across the backbone in order to create a consistent view among all entry/exit points and in the whole routing domain Inserting locally originated prefixes, for instance for customers that do not speak BGP Routes imported from one IBGP peer are not distributed to another IBGP peer This prevents possible routing loops Loop detection is based on duplicates in AS paths EBGP detects this between different ASs IBGP cannot detect this inside one and the same AS Requires IBGP peers to be configured as a full mesh 1 1 I have seen authors write this as full mess Routing Information Bases (RIBs) BGP route processing Adj 2 -RIB-In (one per peer) Unprocessed 3 routes received from peer Loc-RIB (only one globally) Routes after input filtering and best path selection Every AS needs an input policy Path selection is a fixed and specified algorithm Adj-RIB-Out (one per peer) Routes after output filtering Every AS needs an output policy Receive BGP update Install route in Adj-RIB-In Apply import policy: filter routes and tweak attributes Apply best route selection algorithm Install selected route in Loc-RIB Influence IP forwarding table Apply export policy: filter routes and tweak attributes Possibly apply route aggregation Install exported routes in Adj-RIB-Out Transmit BGP update Send Adj-RIBs-Out to peers 2 Adj stands for Adjacent, although the RFC doesn t say so 3 According to the RFC, but implementations may tweak?
BGP protocol Some important BGP attributes Uses TCP over port 179 EBGP usually connects to a directly connected neighbor on layer 2, except when using multi-hop EBGP (not common) IBGP uses routes from an IGP to reach its peers Exchanges Network Layer Reachability Information (NLRI) Prefixes that can or can no longer be reached through the router Accompanied by BGP attributes used by the best route selection algorithm In order of path selection importance LOCAL_PREF (Local Preference) AS_PATH ORIGIN (Historical) MULTI_EXIT_DISC (MED; Multi-exit discriminator) Unrelated to path selection NEXT_HOP Must be reachable directly or via IGP, except in the case of multi-hop BGP, otherwise route is not used in path selection For instance used by route servers in IXPs Next Hop in EBGP and IBGP BGP attribute types Unless router BC uses next-hop-self, in which case the NEXT_HOP in the second BGP UPDATE would contain C instead of A. Well-known, mandatory AS_PATH, NEXT_HOP, ORIGIN Well-known, discretionary LOCAL_PREF, ATOMIC_AGGREGATE Optional, transitive COMMUNITIES, AGGREGATOR Optional, non-transitive MULTI_EXIT_DISC
LOCAL_PREF (Local Preference) AS_PATH Advertised within a single AS (via IBGP) Used to implement local policies Can depend on any locally available information This might be learned outside of BGP Default value is 100 Highest value wins Sequence of ASs An AS can also be generalized to a set of ASs Used for loop detection The sequence length defines the metric (distance) Shortest path wins Prepend your own AS in EBGP updates Possibly multiple times, enabling traffic engineering Leave unchanged in IBGP updates AS_PATH example (origin AS 1) Another AS_PATH example (origin D) Source: RIPE NCC BGP course
AS_PATH length can be deceptive Traffic often follows AS_PATH (in reverse) Sometimes traffic does not follow AS_PATH ORIGIN (Historic attribute) The ORIGIN attribute tells where the route (NLRI) originated Interior to the originating AS: ORIGIN = 0 Via the EGP protocol (historic): ORIGIN = 1 Via some other means (incomplete): ORIGIN = 2 A lower ORIGIN wins In practice the ORIGIN is always 0
MULTI_EXIT_DISC (Multi-Exit Discriminator or MED) Best route selection criteria The MED (or metric, formerly INTER_AS_METRIC) is meant to be advertised between neighboring ASs (via EBGP) Some implementations carry MED on by IBGP Hot potato versus cold potato The MED is non-transitive (is not transferred into a third AS) A lower MED wins The default MED is 0 (lowest possible value) Some implementations choose the highest possible value Definition (Route preference order) 1. (Drop if own AS in AS_PATH) 2. (Weight; Cisco specific) 3. Highest Local Preference 4. Shortest AS Path 5. (Lowest Origin; hardly used; historic) 6. Lowest MED (to the same neighbor) 7. Prefer ebgp over ibgp Best route selection tiebreakers BGP message header 0 15 16 23 24 31 Definition (Route preference tiebreakers) 1. Lowest IGP cost to BGP egress (Next Hop) 2. Already existing received path 3. Lowest Router ID 4. Lowest neighbor IP address Length Marker Type We use the term message and not packet, because BGP packets are in fact part of one single TCP-stream.
BGP header fields BGP OPEN message BGP header fields Marker 128 bits of 1 (compatibility) Length Total length (min 19, max 4096) No padding 4, Including header Type 1: OPEN 2: UPDATE 3: NOTIFICATION 4: KEEPALIVE 5: Route-REFRESH 0 7 8 15 16 31 Version My Autonomous System Opt Parm Len Hold Time These bits are absent BGP Identifier as well as these Optional Parameters and these and these (variable) 4 No superfluous bytes are allowed inside the TCP stream OPEN message fields BGP KEEPALIVE message OPEN message fields Version 4 My Autonomous System Sender s AS Hold Time Liveness detection BGP Identifier Sender s identifying IP address Opt Parm Length Length of parameter field Optional Parameters TLV-encoded options This page intentionally left blank. http://www.this-page-intentionally-left-blank.org/ 5 One interesting parameter is the Capabilities Optional Parameter, which defines (among others) the Route Refresh, 4-Octet AS number, Multiprotocol and BGPsec capabilities. 5 One of those Internet memes like http://endoftheinternet.com/
KEEPALIVE message fields BGP NOTIFICATION message KEEPALIVE message fields :) 0 7 8 15 16 31 Error code Error subcode Data Usually the keepalive timer is something like 60 seconds and the hold time three times as long. (variable) NOTIFICATION message fields BGP Route-REFRESH message NOTIFICATION message fields 0 15 16 23 24 31 AFI Reserved SAFI Error code Error subcode Data 1: Message Header Error 2: OPEN Error 3: UPDATE Error 4: Hold Timer Expired Depends on error code Depends on error code and subcode The AFI (Address Family Identifier) is used for Multiprotocol extensions to BGP (MBGP). This AFI is not compatible with the one used in RIP. A SAFI (Subsequent Address Family Identifier) is also defined for a further subdivision, for instance to distinguish unicast from multicast. Route refresh is a better solution than the soft-reconfiguration solution, which kept all received updates unmodified in memory (Adj-RIB-In).
Route-REFRESH message fields BGP UPDATE message Route-REFRESH message fields AFI Address Family Identifier 6 Reserved 0 SAFI Subsequent Address Family Identifier 7 0 15 16 31 Unfeasible Routes Length Total Path Attribute Length Withdrawn Routes (variable length) Path Attributes (variable length) Absent bits Absent bits Network Layer Reachability Information (variable length) 6 https://www.iana.org/assignments/address-family-numbers/address-family-numbers.xhtml 7 https://www.iana.org/assignments/safi-namespace/safi-namespace.xhtml UPDATE message fields Tweaking your policies UPDATE message fields Unfeasible Routes Length Length of Withdrawn Routes Withdrawn Routes List of prefixes 8 Total Path Attribute Length Length of Path Attributes Path Attributes TLV-encoded attributes Network Layer Reachability Information List of NLRI prefixes Outbound traffic Influenced by inbound routes and filters Tweak attributes to influence best route selection You are in control yourself Inbound traffic Influenced by outbound routes and filters Tweak attributes trying to influence your peers best route selection You are dependent on your peers policies 8 A prefix is specified by its length in bits and just enough bytes of the network IP address to cover this length
Outbound Traffic Engineering Choice between provider, peer or customer Outbound TE works by manipulating incoming routes Changing local preference Extending inbound AS paths Manipulating the metric (MED), for instance by using inbound communities It is relatively simple Based on your own policy You are in control yourself Manipulating local preference Prefer customer over peer over provider Multihomed setup
Singlehomed primary and backup links Inbound Traffic Engineering Inbound TE works by manipulating outgoing routes Extending outbound AS_PATHs is a traditional hack Manipulating the metric (MED) is the official way Setting outbound communities is a more modern approach Agreements with your neighbors are necessary (common policy) Inbound is more complex than outbound Inbound depends (also) on neighbor s policy You are not in control all by yourself Announcing more specific routes Method of last resort, but often a bad idea Advertising a longer AS_PATH Your providers may overrule your effort
But you can make an agreement by using a community Hot potato routing Burnt by the hot potato Cold potato routing by honoring MEDs
Communities Use of communities An optional transitive attribute A community can be used to communicate preferred treatment of a route Communities can be used with both inbound as well as outbound Communities can also be used with IBGP, for instance tagging customer, peer and provider routes Some communities have a well-known semantics NO_EXPORT: don t export beyond current AS (or confederation) NO_ADVERTISE: don t export at all Inbound from your upstream Learn where your upstream imported this route You can base policy decisions on that Outbound to your upstream Request specific upstream treatment Setting of local preference Announcements or not to specific ASs AS_PATH prepending for certain peerings Your upstream promises to implement the requested policy Route reflectors Full mesh IBGP Specified in RFC 4456 A route reflector is a kind of super IBGP peer A route reflector has clients with which it peers via IBGP and for which it reflects (transitively) routes A route reflector is part of a full mesh of other route reflectors and non-clients
Route reflector mesh Confederations Specified in RFC 5065 Use multiple private ASs inside your main AS Talk to the outside world with your main AS This hides the private ASs Talk to the inside world as if using EBGP and IBGP Using the different private ASs This needs special AS_PATH segment types Confederation with SubAS s BGP in data centers Modern (Clos) fabrics in data centers can Eliminate layer 2 Use IP only on point-to-point links Use EBGP internally with private ASs as if it were an IGP