In practice that means at least 200,000,000 computers on the net! How many people use the Internet? As of November 2000:

Transcription

1 Core World Wide Web Technologies: Ethernet, TCP/IP, DNS&Bind, HTTP Henry S. Thompson HCRC Language Technology Group Division of Informatics University of Edinburgh First, some terminology The Web is not the Internet The Internet is everything that happens using a packet-switched network of computers The World Wide Web is one particular family of protocols and applications which use the Internet How big is the Internet? In practice that means at least 200,000,000 computers on the net! How many people use the Internet? As of November 2000: Africa 3.11 million Asia/Pacific million Europe million UK 20 million Middle East 2.40 million Canada & USA million

2 G+F Latin America million World Total million By May 2001, UK usage was up to 22 million! How big is the World Wide Web Nobody knows One index of the WWW covers over 10 9 pages: "Search 2,073,418,204 web pages" There are probably at least twice as many as that Some popular pages get millions of 'hits' a day So how does it work? We've already seen the answer, in a sense There are four layers of technology which make the Web work: Packets Low- and high-level Protocols Applications Packets: the Ethernet The premier local area network technology A basic mechanism for asynchronous communication between many devices Without requiring point-to-point connections Invented by Bob Metcalfe at Xerox PARC in the early 1970s Now an IEEE standard, (various minor versions) The medium (the wire) Access control (contention management) Packets (what goes on the wire) Access control Democracy rules All machines on a wire have equal access Three-part algorithm Listen Transmit Detect collision and retransmit Randomised to minimise contention Packets Three-part bit sequence Two 48-bit addresses, destination and source Every ethernet device has a unique wired-in address Usually known as a MAC address This machine's internal interface MAC address is F3-06 Every device reads any packet which has its MAC address as its destination FF-FF-FF-FF-FF-FF is reserved as the broadcast address Everybody reads broadcast packets

3 H'I Variable length payload Error checksum Packets are one-way, single-payload, and intact delivery is not guaranteed Low-level protocols: TCP/IP Problems with raw packets Unstructured numerical addresses Single networks Not reliable Maximum length The combination of TCP and IP solve these problems IP deals with addressing and internet issues TCP deals with reliability and streaming Beyond the LAN: IP IP adds two things to raw packets Structured addresses Interconnected networks Or internets With two hundred million hosts on the Internet, a flat address space is unmanagable IP address are hierarchical 4 octets instead of 6 Locality gets tighter from left to right IP continued The payload of an IP packet contains destination and source IP addresses IP addresses usually presented as 4 decimal numbers is this machine's IP address Within a single LAN, ARP (Address Resolution Protocol) provides a simple distributed lookup mechanism Send out a broadcast packet asking for the MAC address for an IP address The owner of that IP address should reply Internets: Gateways and routing A gateway connects one LAN to another If the destination IP address is not on the sender's LAN, it gets sent to a gateway (via its MAC address) first A routing table tells you which gateway to use, if there are several There's usually a fallback, based on the hierarchy That's how packets get from here to Philadelphia (and back),

4 J9 without being connected directly to anything in Philadelphia Reliability and Streaming: TCP For most purposes, a reliable connection with unlimited length is required TCP provides this by Handshaking (getting an acknowledgement for each packet sent) Breaking data down into as many packets as required, and reassembling them Even if they arrive out of order TCP also provides for several distinct connections to the same destination Packets may come from and go to different ports The nested packet picture So any given TCP/IP packet looks roughly like this Packets and the protocol stack Each layer in the preceding picture has a way of indicating who should deal with the contents The ethernet type code IP in this case The IP protocol field TCP in this case The TCP destination port e.g. port 80 for HTTP

5 IKH This gives us the concept of the protocol stack IP or ARP on top of ethernet TCP or UDP on top of IP HTTP or FTP or SMTP on top of TCP Beyond numeric addresses Even though they have some structure, IP addresses are clearly insufficient The Domain Name system provides a mapping between names and addresses The hierarchical nature of names enables distributed responsibility for managing the lookup database Ownership of names is temporary, managed by an NGO called IANA (Internet Assigned Numbers Authority) and its operating arm ICANN (Internet Corporation for Assigned Names and Numbers) The structure of names The top-level of the hierarchy is shamelessly American-centric There were 7 'global' top-level domains.com, edu,.gov,.int,.mil,.net,.org Only.com,.net and.org are really global and open The rest are closed, and, except for.int, for USA only Recently 3 more were added.biz,.info,.name The rest of the world get two-letter cctld names.uk,.fr,.de etc. Within which the 'global' domains, slightly renamed, are nested.co.uk,.ac.be,.ac.uk,.net.uk Or not: companies and universities are directly under.fr and.nl Names continued You can apply to rent (in units of 1 year) the association between a name and an IP address two-step names ending in one of the global TLDs (e.g. mit.edu Two or three-step names ending in one of the cctlds (e.g. ed.ac.uk, uu.nl) The going rate is around 15/year Name structure within a domain is managed by the domain owner The university has a two-level structure of its own calvin.cogsci.ed.ac.uk Domain name servers and BIND The association between domain names and IP addresses is maintained by a world-wide network of domain name servers

6 ILH (DNS) A protocol called BIND is used to manage the propagation of associations from the authoritative name server for a domain and the rest The name lookup system has coped remarkably well with the exponential growth in the number of domains High-level protocols What arrives at a computer in TCP/IP streams has to be interpreted Protocols are conventions for interpreting packets The Web is based on a protocol called HTTP HTTP messages look like GET HTTP/1.1 ACCEPT text/html This is a request for an HTML web-page The bit after the ACCEPT is a MIME type Applications A Web application is one which works with messages using the HTTP protocol The two most important ones are Servers They respond to GET messages by sending web pages Browsers They send GET messages for users and show them the results HTML was what made the first major web applications go an agreement about The appearance of web pages Connections between them Standards and standards bodies So where does agreement about all the levels involved come from? A surprising range of more-or-less official bodies The ethernet protocol itself is the responsibility of the IEEE (Institute of Electrical and Electronic Engineers), an international membership organisation of individuals The IP, TCP, HTTP and BIND protocols and the MIME system are the responsibility of the IETF (Internet Engineering Task Force), an ad-hoc cooperative Standards bodies, cont'd HTML is the responsibility of the W3C (World Wide Web

7 HI Consortium), an international membership organisation of companies and non-profits What's astonishing about the above is that in fact none of what makes the World Wide Web work is an official national or international standard None of ISO, BSI, ANSI, etc. own any of this Although ISO has ratified HTML post-hoc Next lecture How web servers and browsers work HTML basics Pictures for the web Forms, security and other sources of complexity