BGP Border Gateway Protocol (an introduction) Karst Koymans Informatics Institute University of Amsterdam (version 1.9, 2012/03/14 10:21:22) Monday, March 12, 2012 General ideas behind BGP Background Providers, Customers and Peers External and Internal BGP BGP information bases The BGP protocol BGP attributes BGP packets Traffic Engineering Outbound Traffic Engineering Inbound Traffic Engineering IBGP scaling BGP version 4 Autonomous system (AS) Border Gateway Protocol version 4 (BGP4) Specified in RFC 4271 The inter-as routing protocol Monopolises the Internet Based on path vector routing which is inbetween distance vector and link state Uses (often non-coordinated) routing policies which can be problematic for convergence Definition (AS Autonomous System) A connected group of networks and routers Representing some assigned set of IP prefixes Having a single, consistent routing policy Both internally and externally
Autonomous system illustration Providers and Customers Customers and Providers Autonomous Systems AS192 AS2503 IP traffic AS29077 Customer pays for access to the Internet 3 Slide courtesy Iljitsch van Beijnum Peers Providers, Customers and Peers The Peering Relationship Peering Provides Shortcuts Peers provide transit between their respective s Peers do not provide transit between s traffic allowed traffic NOT allowed Peers (often) do not exchange $$$ Peering also allows connectivity between the s of Tier 1 s.
The AS abstraction Providers, Customers and Peers routing preferences AS Graph!= Internet Topology BGP was designed to throw away information! The order of preference for a route is Customers have highest preference Peers have the next highest preference Providers have the lowest preference Transit relationships are enforced by export filtering Do not advertise or routes to other s or s Do advertise all routes to s Do advertise routes to s and s The AS graph may look like this. Reality may be closer to this Providers, Customers and Peers: Import Import Routes Providers, Customers and Peers: Export Export Routes route route route ISP route route route route ISP route From From To From From From To To From From To To filters block
External and Internal BGP (1) External and Internal BGP (2) External BGP (EBGP) Used for BGP neighbors between different AS s Exchanging prefixes Implementing policies Internal BGP (IBGP) Used for BGP neighbors within one and the same AS Distributing Internet prefixes across the backbone in order to create a consistent view among all entry/exit points Inserting locally originated prefixes for instance for s that do not speak BGP Routes imported from one IBGP are not distributed to another IBGP This prevents possible routing loops Loop detection is based on duplicates in AS paths EBGP detects this between different AS s IBGP cannot detect this inside one the same AS Requires IBGP s to be configured as a full mesh Routing Information Bases (RIB s) BGP route processing BGP Route Processing Open ended programming. Constrained only by vendor configuration language Adj-RIB-In (one per ) Routes after input filtering Loc-RIB (one globally) Routes after best path selection Adj-RIB-Out (one per ) Routes after output filtering Receive BGP Updates Apply Policy = filter routes & tweak attributes Apply Import Policies Based on Attribute Values Best Route Selection Best Routes Best Route Table Apply Policy = filter routes & tweak attributes Install forwarding Entries for best Routes. Apply Export Policies Transmit BGP Updates IP Forwarding Table 52
BGP protocol Some important BGP attributes Uses TCP over port 179 Usually with a directed connected neighbor on layer 2 Exchanges NLRI Network Layer Reachability Information Prefixes that can or can no longer be reached through the router Accompanied by BGP attributes In order of path selection importance LOCAL_PREF (Local Preference) AS_PATH ORIGIN (Historical) MULTI_EXIT_DISC (MED; Multi-exit discriminator) And unrelated to path selection NEXT_HOP Must be reachable (directly or via IGP) except in the case of multi-hop BGP Next Hop in EBGP and IBGP BGP Next Hop Attribute Interaction between BGP and IGP Join EGP with IGP For Connectivity 12.125.133.90 AS 6431 AT&T Research AS 7018 AT&T Next Hop = 12.125.133.90 12.127.0.121 2654 RIPE NCC RIS project Next Hop = 12.127.0.121 Every time a route announcement crosses an AS boundary, the Next Hop attribute is changed to the IP address of the border router that announced the route. 53 10.10.10.10 Forwarding Table destination next hop 192.0.2.0/30 + EGP destination 10.10.10.10 next hop 192.0.2.1 Next Hop = 192.0.2.1 192.0.2.1 192.0.2.0/30 Forwarding Table destination next hop 10.10.10.10 192.0.2.0/30 10.10.10.10
BGP attribute types LOCAL_PREF (Local Preference) Well-known mandatory ORIGIN, AS_PATH, NEXT_HOP Well-known discretionary LOCAL_PREF, ATOMIC_AGGREGATE Optional transitive COMMUNITIES, AGGREGATOR Optional non-transitive MULTI_EXIT_DISC Advertised within a single AS (via IBGP) Used to implement local policies Can depend on any locally available information This might be learned outside of BGP Default value is 100 Highest value wins AS_PATH Sequence of AS s An AS can also be generalized to a set of AS s Used for loop detection The sequence length defines the distance metric Shortest path wins Prepend your own AS in EBGP updates Possibly multiple times for traffic engineering Leave unchanged in IBGP updates AS_PATH example AS Path = 1239 7018 6341 239 Sprint AS Path = 6341 AS 6341 AT&T Research Prefix Originated ASPATH Attribute AS Path = 1755 1239 7018 6341 755 Ebone AS Path = 7018 6341 AS7018 AT&T AS Path = 7018 6341 129 Global Access AS Path = 1129 1755 1239 7018 6341 2654 RIPE NCC RIS project AS Path = 3549 7018 6341 AS 3549 Global Crossing 64
AS_PATH length can be deceptive Shorter Doesn t Always Mean Shorter AS_PATH for loop prevention Interdomain Loop Prevention In fairness: could you do this right and still scale? Exporting internal state would dramatically increase global instability and amount of routing state AS 3 Mr. BGP says that path 4 1 is better than path 3 2 1 Duh! AS 4 BGP at AS YYY will never accept a route with ASPATH containing YYY. AS 7018 Don t Accept! 12.22.0.0/16 ASPATH = 1 333 7018 877 66 Traffic often follows AS_PATH Traffic Often Follows ASPATH Sometimes traffic does not follow AS_PATH But It Might Not AS 3 ASPATH = 3 2 1 AS 4 ASPATH = 1 135.207.44.0/25 ASPATH = 5 filters all subnets with masks longer than /24 AS 3 ASPATH = 3 2 1 AS 4 IP Packet Dest = 135.207.44.66 AS 5 135.207.44.0/25 IP Packet Dest = 135.207.44.66 From AS 4, it may look like this packet will take path 3 2 1, but it actually takes path 3 2 5
ORIGIN MULTI_EXIT_DISC (Multi-Exit Discriminator or MED) The ORIGIN attribute tells where the route (NLRI) originated Interior to the originating AS: ORIGIN = 0 Via the EGP protocol (historic): ORIGIN = 1 Via some other means: ORIGIN = 2 A lower ORIGIN wins The MED (or metric, formerly INTER_AS_METRIC) is meant to be advertised between neighboring AS s (via EBGP) Some implementations carry MED on by IBGP Hot potato versus cold potato The MED is non-transitive (is not transferred into a third AS) A lower MED wins The default MED is 0 (lowest possible value) Some implementations choose the highest possible value Best route selection BGP packet header Definition (Route selection preference) 0 15 16 23 24 31 1. (Weight; Cisco specific) 2. Highest Local Preference 3. Shortest AS Path 4. (Lowest Origin; hardly used) 5. Lowest MED 6. Prefer EBGP over IBGP 7. Lowest IGP cost to BGP egress 8. Lowest Router ID Length Marker Type Remember that BGP packets are in fact part of a TCP-stream
BGP header fields BGP OPEN message BGP header fields Marker Length Type 128 bits of 1 (compatibility) Total length No padding, Including header 1: OPEN 2: UPDATE 3: NOTIFICATION 4: KEEPALIVE 5: Route-REFRESH 0 7 8 15 16 31 Version My Autonomous System Opt Parm Len Hold Time BGP Identifier Optional Parameters (variable) OPEN message fields BGP KEEPALIVE message OPEN message fields Version 4 My Autonomous System Sender s AS Hold Time Liveness detection BGP Identifier Sender s identifying IP address Opt Parm Length Length of parameter field Optional Parameters TLV-encoded options This page intentionally left blank. http://www.this-page-intentionally-left-blank.org/ One interesting parameter is the Capabilities Optional Parameter, which defines (among others) the Route Refresh Capability.
KEEPALIVE message fields BGP NOTIFICATION message KEEPALIVE message fields :) 0 7 8 15 16 31 Error code Error subcode Data (variable) NOTIFICATION message fields BGP Route-REFRESH message NOTIFICATION message fields Error code Error subcode Data 1: Message Header Error 2: OPEN Error 3: UPDATE Error 4: Hold Timer Expired... Depends on error code Depends on error code and subcode 0 15 16 23 24 31 AFI Reserved SAFI
Route-REFRESH message fields BGP UPDATE message 0 15 16 31 Unfeasible Routes Length Route-REFRESH message fields AFI Address Family Identifier Reserved 0 SAFI Subsequent Address Family Identifier Total Path Attribute Length Withdrawn Routes (variable length) Path Attributes (variable length) Network Layer Reachability Information (variable length) UPDATE message fields Tweaking your policies Tweak Tweak Tweak UPDATE message fields Unfeasible Routes Length Length of Withdrawn Routes Withdrawn Routes List of prefixes 1 Total Path Attribute Length Length of Path Attributes Path Attributes TLV-encoded attributes Network Layer Reachability Information List of NLRI prefixes For inbound traffic Filter outbound routes Tweak attributes on outbound routes in the hope of influencing your neighbor s best route selection For outbound traffic Filter inbound routes Tweak attributes on inbound routes to influence best route selection inbound traffic outbound traffic outbound routes inbound routes In general, an AS has more control over outbound traffic 1 A prefix is specified by its length and just enough bytes of the network IP address to cover this length
Outbound Traffic Engineering Choice between, or So Many Choices This works by manipulating incoming routes Changing local preference Extending inbound AS paths Manipulating the metric (MED), for instance by using inbound communities It is relatively simple Based on your own policy You are in control yourself Frank s Internet Barn AS 4 AS 3 Which route should Frank pick to 13.13.0.0./16? 13.13.0.0/16 60 Manipulating local preference Prefer over over LOCAL PREFERENCE Primary and backup links Implementing Backup Links with Local Preference (Outbound Traffic) Local preference used ONLY in ibgp AS 4 local pref = 80 local pref = 90 AS 3 primary link backup link local pref = 100 Set Local Pref = 100 for all routes from AS 65000 Set Local Pref = 50 for all routes from Higher Local preference values are more preferred 13.13.0.0/16 61 Forces outbound traffic to take primary link, unless link is down. We ll talk about inbound traffic soon 70
Multihomed primary and backup links Multihomed Backups (Outbound Traffic) primary link Set Local Pref = 100 for all routes from AS 3 backup link Set Local Pref = 50 for all routes from AS 3 Inbound Traffic Engineering This works by manipulating outgoing routes Extending outbound AS_PATH s is a traditional hack Manipulating the metric (MED) is the traditional way Setting outbound communities is the more modern approach Agreements with your neighbors are necessary Inbound is more complex than outbound Inbound depends on neighbor s policy You are not in control yourself Announcing more specific routes Method of last resort, but often a bad idea Forces outbound traffic to take primary link, unless link is down. 71 Traffic engineering a longer AS_PATH Shedding Inbound Traffic with ASPATH Padding. Yes, this is a Glorious Hack Your might overrule your effort But Padding Does Not Always Work ASPATH = 2 AS 3 ASPATH = 2 2 2 2 2 2 2 2 2 2 2 2 2 2 ASPATH = 2 primary backup ASPATH = 2 2 2 Padding will (usually) force inbound traffic from to take primary link 72 primary backup AS 3 will send traffic on backup link because it prefers routes and local preference is considered before ASPATH length! Padding in this way is often used as a form of load 73 balancing
But you can make an agreement by using a community COMMUNITY Attribute to the Rescue! Hot potato routing Hot Potato Routing: Go for the Closest Egress Point AS 3 AS 3: normal local pref is 100, local pref is 90 192.44.78.0/24 ASPATH = 2 primary backup ASPATH = 2 COMMUNITY = 3:70 Customer import policy at AS 3: If 3:90 in COMMUNITY then set local preference to 90 If 3:80 in COMMUNITY then set local preference to 80 If 3:70 in COMMUNITY then set local preference to 70 74 egress 1 egress 2 15 56 IGP distances This Router has two BGP routes to 192.44.78.0/24. Hot potato: get traffic off of your network as Soon as possible. Go for egress 1! 75 Burnt by the hot potato Getting Burned by the Hot Potato Cold potato routing by honoring MED s Cold Potato Routing with MEDs (Multi-Exit Discriminator Attribute) High bandwidth Provider backbone 2865 17 Heavy Content Web Farm Prefer lower MED values 2865 17 Heavy Content Web Farm SFF NYC 192.44.78.0/24 MED = 15 192.44.78.0/24 MED = 56 Low bandwidth backbone 15 56 15 56 San Diego Many s want their to carry the bits! tiny http request huge http reply 76 192.44.78.0/24 This means that MEDs must be considered BEFORE IGP distance! Note1 : some s will not listen to MEDs Note2 : MEDs need not be tied to IGP distance 77
Communities Use of communities An optional transitive attribute A community can be used to communicate preferred treatment of a route Communities can be used inbound and outbound Some communities have a well-known semantics NO_EXPORT: don t export beyond current AS (or confederation) NO_ADVERTISE: don t export at all Inbound from your upstream Learn where your upstream imported this route You can base policy decisions on that Outbound to your upstream Request specific upstream treatment Setting of local preference Announcements or not to specific AS s AS_PATH prepending for certain ings Your upstream promises to implement the requested policy Structure and semantics of communities How Can Routes be Colored? BGP Communities! A community value is 32 bits By convention, first 16 bits is ASN indicating who is giving it an interpretation community number Used for signally within and between ASes Very powerful BECAUSE it has no (predefined) meaning Community Attribute = a list of community values. (So one route can belong to multiple communities) Route Reflectors Specified in RFC 4456 A route reflector is a kind of super IBGP A route reflector has clients with which it s via IBGP and for which it reflects (transitively) routes A route reflector is part of a full mesh of other route reflectors and non-clients RFC 1997 (August 1996) Two reserved communities no_export = 0xFFFFFF01: don t export out of AS no_advertise 0xFFFFFF02: don t pass to BGP neighbors 58
Route reflectors illustration Route reflectors illustration Full Mesh Route Reflection Slide courtesy Iljitsch van Beijnum 39 Slide courtesy Iljitsch van Beijnum 40 Confederations Confederations illustration Specified in RFC 5065 Use multiple private AS s inside your main AS Talk to the outside world with your main AS This hides the private AS s Talk to the inside world as if using EBGP and IBGP Using the different private AS s This needs special AS_PATH segment types Confederations Slide courtesy Iljitsch van Beijnum 41