The Value of Peerig ISP/IXP Workshops These materials are licesed uder the Creative Commos Attributio-NoCommercial 4.0 Iteratioal licese (http://creativecommos.org/liceses/by-c/4.0/) Last updated 25 th August 2017 1
Ackowledgemets p This material origiated from the Cisco ISP/IXP Workshop Programme developed by Philip Smith & Barry Greee p Use of these materials is ecouraged as log as the source is fully ackowledged ad this otice remais i place p Bug fixes ad improvemets are welcomed Please email workshop (at) bgp4all.com Philip Smith 2
The Iteret p Iteret is made up of ISPs of all shapes ad sizes Some have local coverage (access providers) Others ca provide regioal or per coutry coverage Ad others are global i scale p These ISPs itercoect their busiesses They do t itercoect with every other ISP (over 58100 distict autoomous etworks) wo t scale They itercoect accordig to practical ad busiess eeds p Some ISPs provide trasit to others They itercoect other ISP etworks Just over 7800 autoomous etworks provide trasit 3
Categorisig ISPs Tier 1 ISP Tier 1 ISP $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ Regioal ISP Access ISP Tier 1 ISP Regioal ISP IXP Access ISP Access ISP Tier 1 ISP Regioal ISP IXP Access ISP Access ISP Regioal ISP Access ISP 4
Peerig ad Trasit p Trasit Carryig traffic across a etwork Usually for a fee Example: Access provider coects to a regioal provider p Peerig Exchagig routig iformatio ad traffic Usually for o fee Sometimes called settlemet free peerig Example: Regioal provider coects to aother regioal provider 5
Private Itercoect p Two ISPs coect their etworks over a private lik Private Network Itercoect (PNI) Ca be peerig arragemet Private Peerig p No charge for traffic p Share cost of the lik Ca be trasit arragemet p Oe ISP charges the other for traffic p Oe ISP (the customer) pays for the lik ISP 1 ISP 2 6
Public Itercoect p Several ISPs meetig i a commo eutral locatio ad itercoect their etworks Usually is a peerig arragemet betwee their etworks ISP 1 ISP 2 ISP 6 IXP ISP 3 ISP 5 ISP 4 7
Types of Peerig (1) p Private Peerig Where two etwork operators agree to itercoect their etworks, ad exchage their respective routes, for the purpose of esurig their customers ca reach each other directly over the peerig lik p Settlemet Free Peerig No traffic charges The most commo form of peerig p Paid Peerig Where two operators agree to exchage traffic charges for a peerig relatioship 8
Types of Peerig (2) p Bi-lateral Peerig Very similar to Private Peerig, but usually takes place at a public peerig poit (IXP) p Multilateral Peerig Takes place at Iteret Exchage Poits, where operators all peer with each other via a Route Server p Madatory Multilateral Peerig Where operators are forced to peer with each other as coditio of IXP membership Strogly discouraged: Has o record of success 9
Types of Peerig (3) p Ope Peerig Where a ISP publicly states that they will peer with all parties who approach them for peerig Commoly foud at IXPs where ISP participates via the Route Server p Selective Peerig Where a ISP s peerig policy depeds o the ature of the operator who requests peerig with them At IXPs, operator will ot peer with RS but will oly peer bilaterally p Restrictive Peerig Where a ISP decides who its peerig parters are, ad is geerally ot approachable to cosiderig peerig opportuities 10
Types of Peerig (4) p The Peerig Database documets ISPs peerig policies https://www.peerigdb.com p All operators of ASNs should register i the PeerigDB All operators who are cosiderig peerig or are peerig must be i the PeerigDB to ehace their peerig opportuities p Participatio i peerig fora is ecouraged too Global Peerig Forum (GPF) Regioal Peerig Fora (Europea, Middle Easter, Asia, Caribbea, Lati America) May coutries ow have their ow Peerig Fora 11
12
13
ISP Goals p Miimise the cost of operatig the busiess p Trasit ISP has to pay for circuit (iteratioal or domestic) ISP has to pay for data (usually per Mbps) Repeat for each trasit provider Sigificat cost of beig a service provider p Peerig ISP shares circuit cost with peer (private) or rus circuit to public peerig poit (oe off cost) No eed to pay for data Reduces trasit data volume, therefore reducig cost 14
Trasit How it works p Small access provider provides Iteret access for a city s populatio Mixture of dial up, wireless ad fixed broadbad Possibly some busiess customers Possibly also some Iteret cafes p How do their customers get access to the rest of the Iteret? p ISP buys access from oe, two or more larger ISPs who already have visibility of the rest of the Iteret This is trasit they pay for the physical coectio to the upstream ad for the traffic volume o the lik 15
Peerig How it works p If two ISPs are of equivalet sizes, they have: Equivalet etwork ifrastructure coverage Equivalet customer size Similar cotet volumes to be shared with the Iteret Potetially similar traffic flows to each other s etworks p This makes them good peerig parters p If they do t peer They both have to pay a upstream provider for access to each other s etwork/customers/cotet Upstream beefits from this arragemet, the two ISPs both have to fud the trasit costs 16
The IXP s role p Private peerig makes sese whe there are very few equivalet players Coectig to oe other ISP costs X Coectig to two other ISPs costs 2 times X Coectig to three other ISPs costs 3 times X Etc (where X is half the circuit cost plus a port cost) p The more private peers, the greater the cost p IXP is a more scalable solutio to this problem 17
The IXP s role p Coectig to a IXP ISP costs: oe router port, oe circuit, ad oe router to locate at the IXP p Some IXPs charge aual maiteace fees The maiteace fee has potetial to sigificatly ifluece the cost balace for a ISP p Geerally coectig to a IXP ad peerig there becomes cost effective whe there are at least three other peers The real $ amout varies from regio to regio, IXP to IXP 18
Who peers at a IXP? p Access Providers Do t have to pay their regioal provider trasit fees for local traffic Keeps latecy ad costs for local traffic low Ulimited badwidth through the IXP (compared with costly ad limited badwidth through trasit provider) p Regioal Providers Do t have to pay their global provider trasit fees for local ad regioal traffic Keeps latecy ad costs for local ad regioal traffic low Ulimited badwidth through the IXP (compared with costly ad limited badwidth through global provider) 19
Who peers at a IXP? p Cotet Providers & Cotet Distributio Services Do t have to pay their regioal provider trasit fees for local traffic Keeps latecy ad costs for local traffic low Ulimited badwidth through the IXP (compared with costly ad limited badwidth through trasit provider) p Root, cctld ad gtld operators Adds to the resiliecy of the global DNS system Keeps latecy ad respose time for local resolver traffic very low 20
The IXP s role p Global Providers ca be located close to IXPs Attracted by the potetial trasit busiess available p Advatageous for access & regioal providers They ca peer with other similar providers at the IXP Ad i the same facility pay for trasit to their regioal or global provider (Not across the IXP fabric, but a separate coectio) IXP Access Trasit 21
Coectivity Decisios p Trasit Almost every ISP eeds trasit to reach rest of Iteret Oe provider = o redudacy Two providers: ideal for traffic egieerig as well as redudacy Three providers = better redudacy, traffic egieerig gets harder More the three = dimiishig returs, rapidly escalatig costs ad complexity p Peerig Meas low (or zero) cost access to aother etwork Private or Public Peerig (or both) 22
Trasit Goals 1. Miimise umber of trasit providers But maitai redudacy 2 is ideal, 4 or more is hard 2. Aggregate capacity to trasit providers More aggregated capacity meas better value p Lower cost per Mbps 4x STM-1/OC3 liks to 4 differet ISPs will almost always cost more tha 2x STM-4/OC12 liks to 2 differet ISPs p Yet badwidth of latter (1.2Gbps) is greater tha that of former (620Mbps) ad is much easier to operate 23
Peerig or Trasit? p How to choose? p Or do both? p It comes dow to cost of goig to a IXP Free peerig Payig for trasit from a ISP co-located i same facility, or perhaps close by p Or ot goig to a IXP ad payig for the cost of trasit directly to a upstream provider There is o right or wrog aswer, someoe has to do the arithmetic 24
Private or Public Peerig p Private peerig Scalig issue, with costs, umber of providers, ad ifrastructure provisioig p Public peerig Makes sese the more potetial peers there are (more is usually greater tha two ) p Which public peerig poit? Local Iteret Exchage Poit: great for local traffic ad local peers Regioal Iteret Exchage Poit: great for meetig peers outside the locality, might be cheaper tha payig trasit to reach the same cosumer base 25
Local Iteret Exchage Poit p Defied as a public peerig poit servig the local Iteret idustry p Local meas where it becomes cheaper to itercoect with other ISPs at a commo locatio tha it is to pay trasit to aother ISP to reach the same cosumer base Local ca mea differet thigs i differet regios! 26
Regioal Iteret Exchage Poit p These are also local Iteret Exchage Poits p But also attract regioal ISPs ad ISPs from outside the locality Regioal ISPs peer with each other Ad show up at several of these Regioal IXPs p Local ISPs peer with ISPs from outside the locality They do t compete i each other s markets Local ISPs do t have to pay trasit costs ISPs from outside the locality do t have to pay trasit costs Quite ofte ISPs of disparate sizes ad iflueces will happily peer to defray trasit costs 27
Which IXP? p How may routes are available? What is traffic to & from these destiatios, ad by how much will it reduce cost of trasit? p What is the cost of co-lo space? If prohibitive or space ot available, poitless choosig this IXP p What is the cost of ruig a circuit to the locatio? If prohibitive or competitive with trasit costs, poitless choosig this IXP p What is the cost of remote hads/assistace? If o remote hads, doig maiteace is challegig ad potetially costly with a serious outage 28
Example: South Asia ISP @ LINX p Time: May 2013 p Data: Route Server plus bilateral peerig offers 70k prefixes IXP traffic averages 247Mbps/45Mbps Trasit traffic averages 44Mbps/4Mbps p Aalysis: 85% of iboud traffic comes from 70k prefixes available by peerig 15% of iboud traffic comes from remaiig 380k prefixes from trasit provider 29
Example: South Asia ISP @ HKIX p Time: May 2013 p Data: Route Server plus bilateral peerig offers 67k prefixes IXP traffic is 159Mbps/20Mbps Trasit traffic is 108Mbps/50Mbps p Aalysis: 60% of iboud traffic comes from 67k prefixes available by peerig 40% of iboud traffic comes from remaiig 383k prefixes from trasit provider 30
Example: South Asia ISP p Summary: Traffic by Peerig: 406Mbps/65Mbps Traffic by Trasit: 152Mbps/54Mbps 73% of icomig traffic is by peerig 55% of outboud traffic is by peerig 31
Example: South Asia ISP p Router at remote co-lo Beefits: ca select peers, easy to swap trasit providers Costs: co-lo space ad remote hads p Servers at remote co-lo Beefits: mail filterig, cotet cachig, etc Costs: co-lo space ad remote hads p Overall advatage: Ca cotrol what goes o the expesive coectivity back to home 32
Value propositios p Peerig at a local IXP Reduces latecy & trasit costs for local traffic Improves Iteret quality perceptio p Participatig at a Regioal IXP A meas of offsettig trasit costs p Maagig coectio back to home etwork p Improvig Iteret Quality perceptio for customers 33
Summary p Beefits of peerig Private Iteret Exchage Poits p Local versus Regioal IXPs Local services local traffic Regioal helps defray trasit costs 34
The Value of Peerig ISP/IXP Workshops 35