ISP Systems Design. ISP Workshops

Transcription

1 ISP Systems Desig ISP Workshops These materials are licesed uder the Creative Commos Attributio-NoCommercial 4.0 Iteratioal licese ( Last updated 24 th April

2 Ackowledgemets p This material origiated from the Cisco ISP/IXP Workshop Programme developed by Philip Smith & Barry Greee p Use of these materials is ecouraged as log as the source is fully ackowledged ad this otice remais i place p Bug fixes ad improvemets are welcomed Please workshop (at) bgp4all.com Philip Smith 2

3 Ageda p DNS Server placemet p Mail Server placemet p News Server placemet p Services etwork desig p Services Network Security 3

4 ISP Services p Most ISP services such as DNS, Mail, etc are easily deliverable o low budget hardware platforms Sigle Rack Uit i height (1RU) Dual processor is default ow RAM is very cheap (may as well use 2Gbytes or more) Hard drives are very cheap (SCSI more reliable) Uix like operatig systems (FreeBSD, Debia, Ubutu, CetOS) are very commo p I additio to commercial operatig systems such as Solaris, RedHat Eterprise Liux &c Miimal overhead, miimal OS istall, plus the service required 4

5 DNS p Domai Name System Provides ame ad address resolutio Servers eed to be differetiated, properly located ad specified p Primary ameserver p Secodary ameserver p Cachig ameserver resolver 5

6 DNS p Primary ameserver Holds ISP zoe files p Forward zoe (list of ame to address mappigs) for all ISP s ad ay customer zoes p Reverse zoe (list of address to ame mappigs) for all ISP s address space Hardware & OS: easily satisfied by simple specificatio Located i secure part of et, e.g. NOC LAN Usually ru as hidde master secodary ameservers are the official listed ameservers 6

7 DNS p Secodary ameserver Holds copies of ISP zoe files At least two are required, more is better Hardware & OS: easily satisfied by simple specificatio Strogly recommeded to be geographically separate from each other ad the primary DNS p At differet PoPs p O a differet cotiet e.g. via services offered by ISC, PCH ad others p At aother ISP 7

8 Secodary DNS Example $ dig apic.et s ;; ANSWER SECTION: apic.et NS s1.apic.et. apic.et NS s3.apic.et. apic.et NS s4.apic.et. apic.et NS s5.apic.com. apic.et NS cumi.apic.et. apic.et NS s-sec.ripe.et. apic.et NS tiie.ari.et. apic.et NS tiie.apic.et. ;; ADDITIONAL SECTION: s1.apic.et A s3.apic.et A s4.apic.et A s5.apic.com A cumi.apic.et A tiie.apic.et A s-sec.ripe.et A tiie.ari.et A Brisbae Tokyo Hog Kog Washigto Brisbae Amsterdam Washigto 8

9 Secodary DNS Example p apic.et zoe Primary DNS i Brisbae (s1.apic.et) Secodary DNS ru all over the world by APNIC: p Brisbae p Hog Kog p Tokyo p Washigto Zoe secodaried by p RIPE NCC i Amsterdam p ARIN i Washigto Geographical ad service provider redudacy this is the perfect example! 9

10 DNS p Cachig ameserver This is the resolver it is the DNS cache Your customers use this as resolver, NEVER your primary or secodary DNS Provides very fast lookups Does NOT secodary ay zoes Oe, or preferably two per PoP (redudacy) Hardware & OS: easily satisfied by simple specificatio 10

11 Cachig Nameserver Web Cache DIAL etwork To Core Routers Switch redudacy Router redudacy DNS Cache redudacy Radius proxy DNS Cache DNS Cache p DIAL users automatically give the IP addresses of DNS caches whe they dial i 11

12 Aycastig the Cachig Nameserver p Oe trick of the trade Assig two uique IP addresses to be used for the two DNS resolver systems Use these two IP addresses i every PoP Route the two /32s across your backboe Eve if the two resolver systems i the local PoP are dow, the IGP will esure that the ext earest resolvers will be reachable Kow as IP Aycast 12

13 DNS p Efficiet ad resiliet desig Primary DNS keep it secure Secodary DNS geographical ad provider redudacy p Do t ever put them o the same LAN, switched or otherwise p Do t put them i the same PoP Cachig DNS oe or two per PoP p Reduces DNS traffic across backboe p More efficiet, spreads the load 13

14 DNS p Software Make sure that the BIND distributio o the Uix system is up to date p The vedor s distributio is rarely curret Pay attetio to bug reports, security issues Reboot the DNS cache o a regular (e.g. mothly) basis p Clears out the cache p Releases ay lost RAM p Accepted good practice by system admiistrators 14

15 DNS p Implemetatio Put all your hosts, poit-to-poit liks ad loopbacks ito the DNS p Uder your ISP s domai ame p Use sesible/meaigful ames Put all your hosts, poit-to-poit liks ad loopbacks ito the REVERSE DNS also p Do t forget about i-addr.arpa ad ip6.arpa may ISPs do p Some systems demad forward/reverse DNS mappig before allowig access 15

16 Mail p Must have at least two mail hosts (MX records) for all supported domais Geographical separatio helps p Dedicated POP3 server Cosumers/mobile users get mail from here p SMTP gateway dedicated to that fuctio Cosumers/mobile users sed mail via here p Mail relay ope to CUSTOMERS oly! Do t let outside world use your mail relay p Block port 25 outboud for all customers Isist that outboud goes through SMTP relay SMTP relay does virus (ClamAV) ad spam (Spamassassi) filterig 16

17 Mail Cofiguratio Mail out to the Iteret Icomig mail from Iteret SpamAssassi ClamAV mail.isp.et ISP Mail Gateway smtp.isp.et Customer mail relay SpamAssassi ClamAV pop3.isp.et Customer POP3/IMAP server Icomig mail from customer Mail pulled by customer cliet 17

18 Mail Example p cisco.com mail (MX records) primary MX are 6 systems i Sa Jose Three backup MXes i RTP, Amsterdam ad Sydey backup MX oly used if primary uavailable $ dig cisco.com mx ;; ANSWER SECTION: cisco.com MX 10 sj-iboud-a.cisco.com. cisco.com MX 10 sj-iboud-b.cisco.com. cisco.com MX 10 sj-iboud-c.cisco.com. cisco.com MX 10 sj-iboud-d.cisco.com. cisco.com MX 10 sj-iboud-e.cisco.com. cisco.com MX 10 sj-iboud-f.cisco.com. cisco.com MX 15 rtp-mx-01.cisco.com. cisco.com MX 20 ams-iboud-a.cisco.com. cisco.com MX 25 syd-iboud-a.cisco.com. 18

19 Mail p Software Make sure that the MAIL ad POP3 distributios o the Uix system are up to date p The vedor distributios are rarely curret Pay attetio to bug reports, security issues, usolicited juk mail complaits IMPORTANT: Do NOT allow o-customers to use your mail system as a relay 19

20 News p News servers provide a Useet ews feed to customers p Distributed desig required Icomig ewsfeed to oe large server Distributed to feed servers i each PoP Feed servers provide ews feed to customers Outgoig ews goes to aother server Separate readig ews system Separate postig ews system 20

21 News System Placemet News Feeder Customer coectios POP Two Customer coectios Customer coectios POP Three POP Oe News Feeder News Feeder Exteral coectios News Collector News Distributor Exteral coectios 21

22 News System Placemet News Feeder Customer coectios POP Two Customer coectios Customer coectios POP Three POP Oe News Feeder News Feeder Exteral coectios News Collector News Distributor Exteral coectios 22

23 News p Software Make sure that the Iteret News distributio o the Uix system is up to date p The vedor distributios are rarely curret Pay attetio to bug reports, security issues, usolicited juk postig complaits IMPORTANT: Do NOT allow o-customers to use your ews system for postig messages 23

24 Services Security 24

25 Security p ISP Ifrastructure security p ISP Services security p Security is ot optioal! p ISPs eed to: Protect themselves Help protect their customers from the Iteret Protect the Iteret from their customers p The followig slides are geeral recommedatios Do more research o security before deployig ay etwork 25

26 ISP Ifrastructure Security p ISP server security Userames, passwords, TCP wrappers, IPTABLES Protect all servers usig routers with strog filters applied p Hosted services security Protect etwork from hosted servers usig routers with strog filters Protect hosted servers from Iteret usig routers with strog filters 26

27 ISP Ifrastructure Security ISP Server Protectio Access-list examples: To core routers Allow tcp/established to all servers ICMP DNS 2ary: udp/53 ad tcp/53 POP3: tcp/110 Mail Relay: tcp/25 ad ISP address rage oly News: tcp/119 ad ISP address rage oly DNS Cache: udp/53 Web server: tcp/80 Service Network Gateway Routers Other ecessary filters: DNS secodary POP3 All servers: SSH (tcp/22) from NOC LAN oly Mail Relay NEWS DNS cache Web server 27

28 ISP Ifrastructure Security Hosted Server Protectio Access-list examples: Iboud Allow tcp/established to all servers ICMP Web server: tcp/80 SSH for customer access Ay other ports for services sold to customers To core routers Service Network Gateway Routers Outboud ICMP Allow DNS udp/53 ad tcp/53 Block all access to ISP address rage Server1 Server2 Server3 Server4 Server5 Server6 28

29 ISP Ifrastructure Security p Premises security Locks electroic/card key preferred Secure access 24x7 security arragemets Eviromet cotrol good airco p Staff resposibility Password policy, stragers, temp staff Employee exit procedures p RFC2196 (Site Security Hadbook) p RFC3871 (Operatioal Security Requiremets for Large ISP IP Network Ifrastructure ) 29

30 ISP Network Security Secure exteral access p How to provide staff access from outside Set up ssh gateway (Uix system with ssh daemo ad othig else cofigured) Provide ssh cliet o all staff laptops ssh available o Uix ad Widows ssh is Secure Shell ecrypted lik p How ot to provide access from outside telet, rsh, rlogi these are all isecure Ope host isecure, ca be compromised 30

31 ISP Systems Desig Summary 31

32 ISP Desig Summary p KEEP IT SIMPLE & STUPID! (KISS) p Simple is elegat is scalable p Use Redudacy, Security, ad Techology to make life easier for yourself p Above all, esure quality of service for your customers 32

33 ISP Systems Desig ISP Workshops 33