The Access Tier ISP, considering upstream connectivity

Transcription

1 College of Engineering, Design, Art and Technology Makerere University The Access Tier, considering upstream connectivity / Diarmuid O Briain GameCORE / netlabs!ug 30 January 2018 MikroTik MUM,

2 The Access, connection to the Internet Access s connectivity to the Internet falls into one of these general categories: with minimal L3 ress from upstream provider with L3 ress from upstream provider Full, with ASN, IPv4 and IPv6 blocks Full with IXP Connection Virtualised MikroTik MUM,

3 with minimal L3 ress from upstream provider Internet Access customers connected to upstream provider using Network Address Translation (NAT) Easy to setup but limits size of network Gateway NAT router high workload Dependence from upstream provider/ Renumbering headaches When scaling up When changing providers No ability to multi-home Dependency on NAT limits services to customers MikroTik MUM, Border Tier 2/3 ASN 5333 Border Gateway NAT Tier 3

4 with L3 ress from upstream provider Internet Dependence from upstream provider/ Renumbering headaches if changing providers Difficult to change providers Cannot multi-home between different upstream s Services to end customers limited by services of upstream provider Routing policies dictated by upstream provider MikroTik MUM, Border Tier 2/3 ASN 5333 Border Gateway Access Tier 3

5 Full, with ASN, IPv4 and IPv6 blocks Internet Operational Advantage Independence from upstream provider/ no renumbering headaches when changing providers Ability to multi-home No dependency on NAT, can offer enhanced services to end customers Simpler to implement policies that suit the company and services Border Tier 2 ASN 5333 Border Border Access Tier 3 ASN 5111 REF: MikroTik MUM,

6 Full with IXP Connection Internet All of the previous advantages Plus; Local traffic kept local Reduced transit costs Reduced latency to local content and content caches Improved service to customers Route Server BIRD Border MikroTik MUM, Tier 2 ASN 5333 Border IXP Switch Access IXP ASN 5999 Border Tier 3 ASN 5222 Border Access Tier 3 ASN 5111

7 Virtualused Internet Virtualised over the top Access rollout someone else s problem Common with brand s like Sky and Tesco Usually offer itional services Route Server BIRD IXP Switch Tier 3 ASN 5444 Access Tier 2 ASN 5333 Border Border Border Tunnel Terminator IXP ASN 5999 Border Tier 3 ASN 5222 Border Access Tier 3 ASN 5111 Services MikroTik MUM,

8 College of Engineering, Design, Art and Technology Makerere University Inter Autonomous System What exactly is an Autonomous System and how is carried out in one? MikroTik MUM,

9 The Autonomous System & BGP RFC BGPv1 RFC bit 2-Octet AS65526 RFC BGPv4 RFC4893/ bit 4-Octet AS ebgp TCP/179 OSPF IS-IS AS ibgp TCP179 IS-IS IS-IS OSPF ibgp TCP179 ebgp TCP/179 AS OSPF OSPF ORIGIN: defines the origin of the path information AS PATH: sequence of AS path segments NEXT HOP: defines the IP ress of the next hop router MikroTik MUM,

10 BGP Finite State Machine (FSM) TCP Failed TCP Failed Active TCP Failed BGP Start TCP Established OpenSend Idle Error Error Open Receive correct KEEPALIVE OpenConfirm KEEPALIVE timer expires bgp peer print Flags: X - disabled, E - established # INSTANCE REMOTE-ADDRESS 0 E ASN E ASN5111 2a99:5:5::33 Connect MikroTik MUM, Error Established Receive correct KEEPALIVE REMOTE-AS

11 Basic BGP Message flow IXP - AS AS5111 lo: /32 IXP lo: / / /24 1 OPEN Message OPEN Message, KEEPALIVE Message KEEPALIVE Message UPDATE Message Path Attribute - AS_PATH: 5999 Path Attribute - NEXT_HOP: Network Layer Reachability Information (NLRI): /24 KEEPALIVE Message, KEEPALIVE Message UPDATE Message Path Attribute - AS_PATH: 5111 Path Attribute - NEXT_HOP: Network Layer Reachability Information (NLRI): /24 KEEPALIVE Message, KEEPALIVE Message MikroTik MUM,

12 BGP Update IPv4 IXP AS /24 2a99:9:9::1/ /24 2a99:9:9::11/64 1 AS5111 IPv4, Src: , Dst: TCP, Src Port: 40932, Dst Port: 179, Seq: 65, Ack: 65, Len: 52 BGP - UPDATE Message Marker: ffffffffffffffffffffffffffffffff Length: 52 Type: UPDATE Message (2) Withdrawn Routes Length: 0 Total Path Attribute Length: 25 Path attributes Path Attribute - ORIGIN: IGP Path Attribute - AS_PATH: 5999 Path Attribute - NEXT_HOP: Path Attribute - MULTI_EXIT_DISCRIMINATOR: 0 Network Layer Reachability Information /24 MikroTik MUM,

13 BGP Update IPv6 IXP AS /24 2a99:9:9::1/ /24 2a99:9:9::11/64 1 AS5111 IPv6, Src: 2a99:9:9::1, Dst: 2a99:9:9::11 TCP, Src Port: 40932, Dst Port: 179, Seq: 65, Ack: 65, Len: 52 BGP - UPDATE Message Marker: ffffffffffffffffffffffffffffffff Length: 90 Type: UPDATE Message (2) Withdrawn Routes Length: 0 Total Path Attribute Length: 67 Path attributes Path Attribute - ORIGIN: IGP Path Attribute - AS_PATH: 5999 Path Attribute - MULTI_EXIT_DISCRIMINATOR: 0 Path Attribute MP_REACH_NLRI Next hop network ress (32 bytes) Next Hop: 2a99:9:9::1 Next Hop: fe80::216:c7ff:fe9c:4860 Network Layer Reachability Information 2a99:9:9::/64 MikroTik MUM,

14 College of Engineering, Design, Art and Technology Makerere University Internet exchange Points How can an IXP help me? Are they not just for the big boys? MikroTik MUM,

15 The make-up of the Internet Content providers International Tier 1 Tier 2 Tier 3 National Local International National Local SME LE Consumer MikroTik MUM,

16 Tromboning Content providers International Tier 1 Tier 2 Tier 3 National Local International National Latency Cost Growth of ecosystem Local SME LE Consumer MikroTik MUM,

17 IXP Keeping local traffic local Content providers International Tier 1 Tier 2 National International Reduced latency Reduced costs Increased autonomy National IXP Tier 3 Local Content Provider Cache Local SME LE Consumer MikroTik MUM,

18 Route server Route reflector IXP AS AS 2434 AS birdc> show route /23 via /22 via /24 via /24 via on on on on 3 eth0 eth0 eth0 eth0 AS 4565 [1 [2 [3 [4 n AS ] ] ] ] * * * * (100) (100) (100) (100) [AS2434i] [AS34565i] [AS4565i] [AS38675i] birdc> show route export /24 via on eth0 [ ] * (100) [AS38675i] Type: BGP unicast univ BGP.origin: IGP BGP.as_path: BGP.next_hop: BGP.med: 0 BGP.local_pref: 100 MikroTik MUM,

19 College of Engineering, Design, Art and Technology Makerere University BGP v4 BGP Peering Configuration MikroTik MUM,

20 Demonstration testbed T2-3 Server T2-3 ASN ASN Client MikroTik MUM,

21 Demonstration testbed T2-3 Server ether2 3 ether1 ether5 1 ether /24 2a99:3:3::100/ /24 2a99:3:3::1/64 T2-3 ASN 5333 lo: / /24 2a99:5:5::33/ /24 2a99:5:5::11/64 lo: / /24 2a99:1:1::1/ /24 2a99:1:1::100/64 1 ASN Client MikroTik MUM,

22 1 The preliminaries Enable IPv6 system package print Flags: X - disabled # NAME 0 routeros-mipsbe 1 system 2 X ipv6 3 wireless 4 hotspot 5 dhcp 6 mpls 7 8 ppp 9 security 10 advanced-tools VERSION SCHEDULED system package enable 2 Reset configuration system reset-configuration no-defaults=yes System identity system identity set name=1 MikroTik MUM,

23 1 Initial IP ressing for transit Add IP Addresses to the interfaces interface bridge name=loopback0 ip ress ress= /32 interface=loopback0 ip ress ress= /24 interface=ether2 ipv6 ress ress=2a99:1:1::1/64 interface=ether2 ip ress ress= /24 interface=ether5 ipv6 ress ress=2a99:5:5::11/64 interface=ether5 ip ress print Flags: X - disabled, I - invalid, D - dynamic # ADDRESS NETWORK INTERFACE / loopback / ether / ether5 ipv6 ress print Flags: X - disabled, I - invalid, D - dynamic, G # ADDRESS FROM-POOL 0 DL fe80::20c:42ff:fec2:117c/64 1 DL fe80::20c:42ff:fec2:117e/64 2 DL fe80::20c:42ff:fec2:117f/64 3 IG 2a99:9:9::11/64 4 G 2a99:5:5::11/64 MikroTik MUM, - global, L - link-local INTERFACE ADVERTISE ether2 no ether4 no ether5 no ether1 yes

24 1 BGP configuration Create a BGP instance and networks to be routed bgp instance name=asn5111 as=5111 router-id= bgp network network= /24 bgp network network=2a99:1:1::/64 bgp instance print from=asn5111 Flags: * - default, X - disabled 0 name="asn5111" as=5111 router-id= redistribute-connected=no redistribute-static=no redistribute-rip=no redistribute-ospf=no redistribute-other-bgp=no out-="" client-to-client-reflection=yes ignore-as-path-len=no -table="" bgp network print Flags: X - disabled # NETWORK SYNCHRONIZE /24 yes 1 2a99:1:1::/64 yes MikroTik MUM,

25 BGP ingress Filters Ingress s are used to advertisements into the network Best practice for ingress s for all peers are: Discard receiving ones own prefix Discard receiving a default route as we are doing full Discard special purpose ress registry entries stated at RFC 6980 RFC Special-Purpose IP Address Registries IETF reserved blocks that should not be received in update messages MikroTik MUM,

26 Create BGP ingress Filters for transit Create a BGP ingress for IPv4 Discard own network Discard Default route Jump the to the RFC-6980 IPv4 chain=in-3-ipv4 prefix= /24 action=discard chain=in-3-ipv4 prefix= /0 action=discard chain=in-3-ipv4 action=jump jump-target=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 chain=in-rfc-6980-ipv4 prefix= /8 action=discard prefix= /8 action=discard prefix= /10 action=discard prefix= /8 action=discard prefix= /16 action=discard prefix= /12 action=discard prefix= /24 action=discard prefix= /24 action=discard prefix= /16 action=discard prefix= /15 action=discard prefix= /24 action=discard prefix= /24 action=discard prefix= /4 action=discard prefix= /32 action=discard MikroTik MUM,

27 Create BGP ingress Filters for transit Create a BGP ingress for IPv6 Discard own network Discard Default route Jump the to the RFC-6980 IPv6 chain=in-3-ipv6 prefix=2a99:1:1::/64 action=discard chain=in-3-ipv6 prefix=::/0 action=discard chain=in-3-ipv6 action=jump jump-target=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 chain=in-rfc-6980-ipv6 prefix=::1/128 action=discard prefix=::/128 action=discard prefix=64:ff9b::/96 action=discard prefix=::ffff:0:0/96 action=discard prefix=100::/64 action=discard prefix=2001::/23 action=discard prefix=2001::/32 action=discard prefix=2001:2::/48 action=discard prefix=2001:db8::/32 action=discard prefix=2001:10::/28 action=discard prefix=2002::/16 action=discard prefix=fc00::/7 action=discard prefix=fe80::/10 action=discard MikroTik MUM,

28 Create BGP egress Filters for transit Egress s are used to advertisements from the network, limiting it to only advertise the specific local networks invert-match=yes statement, this instructs the to discard all but the prefix specified chain=out-ipv4 prefix= /24 invert-match=yes action=discard chain=out-ipv6 prefix=2a99:1:1::/64 invert-match=yes action=discard MikroTik MUM,

29 Add BGP Peers for transit Create the peer link to the other BGP peer at IXP3 for IPv4 and IPv6 Note that IPv4 family is the default and for IPv6 the family must be specified bgp peer name=isp3 instance=asn5111 remote-as=5333 remote-ress= in-=in-3-ipv4 out-=out-ipv4 bgp peer name=isp3 instance=asn5111 remote-as=5333 remote-ress=2a99:5:5::33 ress-families=ipv6 in-=in-3-ipv6 out-=out-ipv6 MikroTik MUM,

30 Confirm with transit 3 Confirm that the routes have been learnt from 3 bgp peer print Flags: X - disabled, E - established # INSTANCE REMOTE-ADDRESS 0 E ASN E ASN5111 2a99:5:5::33 REMOTE-AS ip route print Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC GATEWAY DISTANCE 0 ADC / ether2 0 1 ADb / ADC / ether5 0 3 ADC / loopback0 0 MikroTik MUM,

31 BGP ingress Filters Team Cymru A bogon prefix should never appear in the Internet table. bogon route-servers 65333:888 fullbogon route-servers 65332:888 MikroTik MUM,

32 College of Engineering, Design, Art and Technology Makerere University Internet exchange Point (IXP) Configuration MikroTik MUM,

33 Demonstration testbed, IXP ed T2-3 Server T2-3 ASN 5333 BIRD LXD Host 3 IXP Switch 1 IXP ASN ether2 1 ASN Client 2 Client MikroTik MUM, 2 ASN 5222

34 Demonstration testbed, IXP ed T2-3 Server ether2 T2-3 ASN /24 2a99:9:9::1/64 BIRD /24 2a99:3:3::100/ /24 2a99:9:9::100/ /24 2a99:3:3::1/64 3 ether1 IXP Switch lo: / /24 2a99:5:5::33/ /24 2a99:5:5::11/64 ether5 1 ether2 IXP ASN /24 2a99:9:9::11/64 ether1 lo: / /24 2a99:9:9::22/64 ether /24 2a99:1:1::1/64 1 ASN Client lo: /32 ether /24 2a99:2:2::1/ /24 2a99:1:1::100/64 LXD Host /24 2a99:2:2::100/64 2 Client MikroTik MUM, 2 ASN 5222

35 1 IP Address on IXP interface Add IP Addresses to the interface facing the IXP ip ress ress= /24 interface=ether1 ipv6 ress ress=2a99:9:9::11/64 interface=ether1 ip ress print Flags: X - disabled, I - invalid, D - dynamic # ADDRESS NETWORK INTERFACE / loopback / ether / ether / ether1 ipv6 ress print Flags: X - disabled, I - invalid, D - dynamic, G # ADDRESS FROM-POOL 0 DL fe80::20c:42ff:fec2:117c/64 1 DL fe80::20c:42ff:fec2:117e/64 2 DL fe80::20c:42ff:fec2:117f/64 3 G 2a99:5:5::11/64 4 DL fe80::20c:42ff:fec2:117d/64 5 G 2a99:1:1::1/64 6 DL fe80::20c:42ff:fec2:117b/64 7 G 2a99:9:9::11/64 MikroTik MUM, global, L - link-local INTERFACE ADVERTISE ether2 no ether4 no ether5 no ether5 yes ether3 no ether2 yes ether1 no ether1 yes

36 1 Routing s for IXP Chains IN-IXP-IPv4 and IN-IXP-IPv6 are input s that discards receiving ones own prefix or a default route as full is taking place IPv4 chain=in-ixp-ipv4 prefix= /24 action=discard chain=in-ixp-ipv4 prefix= /0 action=discard chain=in-ixp-ipv4 action=jump jump-target=in-rfc-6980-ipv4 IPv6 chain=in-ixp-ipv6 prefix=2a99:1:1::/64 action=discard chain=in-ixp-ipv6 prefix=::/0 action=discard chain=in-ixp-ipv6 action=jump jump-target=in-rfc-6980-ipv6 MikroTik MUM,

37 Add IXP Route Server as a BGP Peer Create the peer link to the IXP for IPv4 and IPv6 Note that IPv4 family is the default and for IPv6 the family must be specified IPv4 bgp peer name=ixp instance=asn5111 remote-as=5999 remote-ress= in-=in-ixp-ipv4 out-=out-ipv4 IPv6 bgp peer name=ixp instance=asn5111 remote-as=5999 remote-ress=2a99:9:9::1 ress-families=ipv6 in-=in-ixp-ipv6 out-=out-ipv6 MikroTik MUM,

38 Confirm peering with IXP Confirm that the routes have been learnt from IXP bgp peer print Flags: X - disabled, E - established # INSTANCE REMOTE-ADDRESS 0 E ASN E ASN5111 2a99:5:5::33 2 E ASN E ASN5111 2a99:9:9::1 REMOTE-AS ip route print Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC GATEWAY DISTANCE 0 ADC / ether2 0 1 ADb / ADb / ADC / ether5 0 4 ADC / ether1 0 5 ADC / loopback0 0 MikroTik MUM,

39 College of Engineering, Design, Art and Technology Makerere University Thank you for your attention Complete configurations can be found at: itcarlow.ie netlabsug.org MikroTik MUM,