Multihoming Case Study

Transcription

1 Multihoming Case Study ISP Workshops Last updated 10 October 2007

2 Multihoming Case Study p Set of slides based on work assisting an ISP with their multihoming needs between 2000 and 2002 n Should be taken as an indicative example only

3 Case Study First Visit (2000)

4 Case Study Requirements (1) p ISP needs to multihome: n To AS5400 in Europe n To AS2516 in Japan n /19 allocated by APNIC n AS assigned by APNIC n 1Mbps circuits to both upstreams

5 Case Study Requirements (2) p ISP wants: n Symmetric routing and equal link utilisation in and out (as close as possible) p international circuits are expensive n Has two Cisco 2600 border routers with 64Mbytes memory p Cannot afford to upgrade memory or hardware on border routers or internal routers p Philip, make it work, please

6 Case Study Upstream ISP AS5400 Upstream ISP AS2516 A B AS ISP Core Allocated /19 from APNIC Circuit to AS5400 is 1Mbps, circuit to AS2516 is 1Mbps

7 Case Study p Both providers stated that routers with 128Mbytes memory required for AS17660 to multihome n Those myths again L n Full routing table is rarely required or desired p Solution: n Accept default from one upstream n Accept partial prefixes from the other

8 Case Study Inbound Loadsharing p First cut: Went to a few US Looking Glasses n Checked the AS path to AS5400 n Checked the AS path to AS2516 n AS2516 was one hop closer n Sent AS-PATH prepend of one AS on AS2516 peering

9 Case Study Inbound Loadsharing p Refinement n Did not need any n First cut worked, seeing on average 600kbps inbound on each circuit n Does vary according to time of day, but this is as balanced as it can get, given customer profile n J

10 Case Study Outbound Loadsharing p First cut: n Requested default from AS2516 n Requested full routes from AS5400 p Then looked at my Routing Report n Picked the top 5 ASNs and created a filter-list p If 701, 1, 7018, 1239 or 7046 are in AS-PATH, prefixes are discarded p Allowed prefixes originated by AS5400 and up to two AS hops away n Resulted in prefixes being accepted in AS17660

11 Case Study Outbound Loadsharing p Refinement n prefixes quite a lot, seeing more outbound traffic on the AS5400 path n Traffic was very asymmetric p out through AS5400, in through AS2516 n Added the next 3 ASNs from the Top 20 list p 209, 2914 and 3549 n Now seeing prefixes n Traffic is now evenly loadshared outbound p Around 200kbps on average p Mostly symmetric

12 Case Study MRTG Graphs Router A to AS5400 Router B to AS2516

13 Case Study Configuration Router A router ospf 100 log-adjacency-changes passive-interface default no passive-interface Ethernet0/0 default-information originate metric 20! router bgp no synchronization no bgp fast-external-fallover bgp log-neighbor-changes bgp deterministic-med...next slide

14 Case Study Configuration Router A neighbor remote-as 5400 neighbor descr ebgp multihop to AS5400 neighbor ebgp-multihop 5 neighbor update-source Loopback0 neighbor prefix-list in-filter in neighbor prefix-list out-filter out neighbor filter-list 1 in neighbor filter-list 3 out! prefix-list in-filter deny rfc1918etc in prefix-list out-filter permit /19! ip route serial 0/ next slide

15 Case Study Configuration Router A ip as-path access-list 1 deny _701_ ip as-path access-list 1 deny _1_ ip as-path access-list 1 deny _7018_ ip as-path access-list 1 deny _1239_ ip as-path access-list 1 deny _7046_ ip as-path access-list 1 deny _209_ ip as-path access-list 1 deny _2914_ ip as-path access-list 1 deny _3549_ ip as-path access-list 1 permit _5400$ ip as-path access-list 1 permit _5400_[0-9]+$ ip as-path access-list 1 permit _5400_[0-9]+_[0-9]+$ ip as-path access-list 1 deny.* ip as-path access-list 3 permit ^$!

16 Case Study Configuration Router B router ospf 100 log-adjacency-changes passive-interface default no passive-interface Ethernet0/0 default-information originate! router bgp no synchronization no auto-summary no bgp fast-external-fallover...next slide

17 Case Study Configuration Router B bgp log-neighbor-changes bgp deterministic-med neighbor remote-as 2516 neighbor description ebgp peering neighbor soft-reconfiguration inbound neighbor prefix-list default-route in neighbor prefix-list out-filter out neighbor route-map as2516-out out neighbor maximum-prefix 100 neighbor filter-list 2 in neighbor filter-list 3 out!...next slide

18 Case Study Configuration Router B! prefix-list default-route permit /0 prefix-list out-filter permit /19! ip as-path access-list 2 permit _2516$ ip as-path access-list 2 deny.* ip as-path access-list 3 permit ^$! route-map as2516-out permit 10 set as-path prepend 17660!

19 Configuration Summary p Router A n Hears full routing table throws away most of it n AS5400 BGP options are all or nothing n Static default pointing to serial interface if link goes down, OSPF default removed p Router B n Hears default from AS2516 n If default disappears (BGP goes down or link goes down), OSPF default is removed

20 Case Study Summary p Multihoming is not hard, really! n Needs a bit of thought, a bit of planning n Use this case study as an example strategy n Does not require sophisticated equipment, big memory, fast CPUs

21 Case Study Second Visit (2002)

22 Case Study Current Status p ISP currently multihomes: n To AS5400 in the UK n To AS2516 in Japan n /19 allocated by APNIC n AS assigned by APNIC n 1Mbps circuits to both upstreams

23 Case Study Requirements p ISP wants: n To add a new satellite connection, a 640K link to AS22351 in Germany to support the AS5400 link to UK n Still want symmetric routing and equal link utilisation in and out (as close as possible) p international circuits are expensive n Has upgraded to two Cisco 3725 border routers with plenty of memory p Despite the working previous configuration with sparse routing table, wanted full prefixes p Talked them out of that, and here is how

24 Case Study Upstream ISP AS5400 1Mbps Upstream ISP AS2516 Upstream ISP AS kbps A B AS Mbps ISP Core Allocated /19 from APNIC

25 Case Study Inbound Loadsharing p First cut: Went to a few US Looking Glasses n Checked the AS path to AS5400 n Checked the AS path to AS2516 n Checked the AS path to AS22351 n AS2516 was one hop closer than the other two n Sent AS-PATH prepend of one AS on AS2516 peering n this is unchanged from two years ago

26 Case Study Inbound Loadsharing p Refinement n Needed some AS5400 seemed to be always preferred over AS22351 n AS5400 now supports RFC1998 style communities for customer use p see whois h whois.ripe.net AS5400 n Sent AS5400 some communities to insert prepends towards specific peers p Now saw some traffic on AS22351 link but not much n Sent a /23 announcement out AS22351 link p Now saw more traffic on AS22351 link

27 Case Study Inbound Loadsharing p Results: n Around 600kbps on the AS5400 link n Around 750kbps on the AS2516 link n Around 300kbps on the AS22351 link n Inbound traffic fluctuates quite substantially based on time of day p Status: n Situation left pending monitoring by the ISP s NOC

28 Case Study Outbound Loadsharing p First cut: n Already receiving default from AS2516 n Receiving full routes from AS5400 n Requested full routes from AS22351 the only option p Retained the AS5400 configuration n Discard prefixes which had top 5 ASNs in the path p AS22351 configuration uses similar ideas to AS5400 configuration n But only accepted prefixes originated from AS22351 or their immediate peers

29 Case Study Outbound Loadsharing p Results: n Around prefixes from AS5400 n Around 2000 prefixes from AS22351 n Around 200kbps on both the AS5400 and AS2516 links n Around 50kbps on the AS22351 link n Outbound traffic fluctuates quite substantially based on time of day p Status: n Situation left pending monitoring by the ISP s NOC

30 Case Study MRTG Graphs Router A to AS5400 Router A to AS22351 Router B to AS2516

31 Case Study Configuration Router A router bgp no synchronization no bgp fast-external-fallover bgp log-neighbor-changes bgp deterministic-med neighbor remote-as neighbor description ebgp peer to AS22351 neighbor send-community neighbor prefix-list in-filter in neighbor prefix-list out-filter-as22351 out neighbor route-map as22351-out out neighbor maximum-prefix warning-only neighbor filter-list 3 in neighbor filter-list 5 out...next slide

32 Case Study Configuration Router A neighbor remote-as 5400 neighbor description ebgp multihop to AS5400 neighbor ebgp-multihop 5 neighbor update-source Loopback0 neighbor send-community neighbor prefix-list in-filter in neighbor prefix-list out-filter out neighbor route-map as5400-out out neighbor filter-list 1 in neighbor filter-list 5 out! ip prefix-list in-filter deny rfc1918 prefixes etc ip prefix-list out-filter permit /19 ip prefix-list out-filter-as22351 permit /19 ip prefix-list out-filter-as22351 permit /23...next slide

33 Case Study Configuration Router A ip as-path access-list 1 deny _701_ ip as-path access-list 1 deny _1_ ip as-path access-list 1 deny _7018_ ip as-path access-list 1 deny _1239_ ip as-path access-list 1 deny _7046_ ip as-path access-list 1 permit _5400$ ip as-path access-list 1 permit _5400_[0-9]+$ ip as-path access-list 1 permit _5400_[0-9]+_[0-9]+$ ip as-path access-list 1 deny.* ip as-path access-list 3 permit _22351$ ip as-path access-list 3 permit _22351_[0-9]+$ ip as-path access-list 3 deny.* ip as-path access-list 5 permit ^$! route-map as5400-out permit 10 set community 5400: : : : :2128 route-map as22351-out permit 10

34 Case Study Configuration Router B router bgp no synchronization no auto-summary no bgp fast-external-fallover bgp log-neighbor-changes bgp deterministic-med neighbor remote-as 2516 neighbor descr ebgp Peering with AS2516 neighbor send-community neighbor prefix-list default-route in neighbor prefix-list out-filter out neighbor route-map as2516-out out neighbor maximum-prefix 100 neighbor filter-list 2 in neighbor filter-list 5 out...next slide

35 Case Study Configuration Router B! prefix-list default-route permit /0 prefix-list out-filter permit /19! ip as-path access-list 2 permit _2516$ ip as-path access-list 2 deny.* ip as-path access-list 5 permit ^$! route-map as2516-out permit 10 set as-path prepend 17660!

36 Interesting Aside p Prior to installation of the new 640kbps link, ISP was complaining that both 1Mbps links were saturated inbound n Hence the requirement for the new 640kbps circuit p Research using NetFlow, cflowd and FlowScan showed that Kazaa was to blame! n Kazaa is a peer to peer file sharing utility n Consumes all available bandwidth n Found that many customers were using Kazaa for file sharing, saturating the links inbound

37 Interesting Aside p Solution n A time of day filter which blocked Kazaa during working hours, 8am to 8pm n Inbound and outbound ACLs on border routers had tcp/ 1214 filters added access-list 100 deny tcp any any eq 1214 time-range OfficeHrs access-list 101 deny tcp any any eq 1214 time-range OfficeHrs! time-range OfficeHrs periodic weekdays 8:00 to 20:00 n The result: inbound traffic on external links dropped by 50% n And complaints about the net being slow have reduced

38 Interesting Aside Typical FlowScan graph no longer showing the effects of Kazaa

39 Summary p Multihoming solution with three links of different bandwidths works well n Fluctuates significantly during the day time, maybe reflecting users browsing habits? n NOC is monitoring the situation n NOTE: Full routing table is not required J

40 Multihoming Case Study ISP Workshops