Announcements. The World Wide Web. Retrieving From the Server. Goals of Today s Lecture. The World Wide Web. POP3 Protocol

Transcription

1 Announcements The World Wide Web Project #2 out Checkpoint due Weds Oct 18 Full project due Thurs Oct 26 EE 122: Intro to Communication Networks Fall 2006 (MW 4-5:30 in Donner 155) Vern Paxson TAs: Dilip Antony Joseph and Sukun Kim Materials with thanks to Jennifer Rexford, Ion Stoica, and colleagues at Princeton and UC Berkeley 1 2 Goals of Today s Lecture (Finish - retrieving mail from the server) Main ingredients of the Web URIs, HTML, HTTP Key properties of HTTP Request-response, stateless, and resource meta-data Performance of HTTP Parallel connections, persistent connections, pipelining Web components, proxies, and servers Caching vs. replication 3 Retrieving From the stores incoming by mailbox Based on the From field in the message Users need to retrieve Variety of ways to do this: Directly by same-machine access to the mailbox Via Interactive Mail Access Protocol (IMAP) Supports concurrent access by multiple clients, server-side searchers, partial MIME fetches, multiple mailboxes Via HTTP (Web) E.g., GMail Via Post Office Protocol (POP) 4 POP3 Protocol Authorization phase Client commands: user: declare username pass: password responses +OK -ERR Transaction phase, client: list: list message numbers retr: retrieve message by number dele: delete quit S: +OK POP3 server ready C: user bob S: +OK C: pass hungry S: +OK user successfully logged on C: list S: S: S:. C: retr 1 S: <message 1 contents> S:. C: dele 1 C: retr 2 S: <message 1 contents> S:. C: dele 2 C: quit S: +OK POP3 server signing off 5 The World Wide Web 6 1

2 Main Components: URIs Uniform Resource Identifier (URI) Denotes a resource Could be its name Could be its location Which is better? Uniform Resource Name (URN) Names a resource independent of how to get it E.g., urn:ietf:rfc:2396 is a standard URN for RFC 2396 Uniform Resource Locator (URL) Specifies how to access a resource E.g., ftp://ftp.rfc-editor.org/in-notes/rfc2396.txt 7 URL Syntax protocol://hostname[:port]/directorypath/resource Protocol might be http, ftp, https, smtp, rtsp, In practice, hostname can instead be an IP address What does your browser (maybe) show for Port defaults to the standard port associated w/ protocol E.g., 80/tcp for http, 443/tcp for https Directory path is hierarchical, often reflecting file system Can extend resource to program executions as well MsgId=2604_ _29699_1123_1261_0_28917_3552_ &Search=&Nhead=f&YY=31454&order=down&sort=date&pos= 0&view=a&head=b 8 Main Components: HTML HyperText Markup Language (HTML) Representation of hypertext documents in ASCII format Format text, reference images, embed hyperlinks (HREF) Interpreted by Web browsers when rendering a page Straight-forward to learn Can basically start with a plain text file Easy to add formatting, references, bullets, etc. Automatically generated by authoring programs Tools to aid users in creating HTML files Your browser likely can show a page s raw HTML Web page Base HTML file referenced objects (e.g., images) Each object has its own URL 9 Main Components: HTTP HyperText Transfer Protocol (HTTP) Client-server protocol for transferring resources Important properties of HTTP Request-response protocol Reliance on a global URI namespace Resource metadata Stateless ASCII format % telnet 80 GET /vern/ HTTP/1.0 <blank line, i.e., CRLF> 10 HTTP Request Message Request message sent by a client Request line: method, resource, and protocol version Request headers: provide information or modify request Body: optional data (e.g., to POST data to the server) request line (GET, POST, HEAD commands) Carriage return, line feed indicates end of message header lines GET /somedir/page.html HTTP/1.1 Host: User-agent: Mozilla/4.0 Connection: close Accept-language:fr blank line Not optional 11 HTTP Response Message Response message sent by a server Status line: protocol version, status code, status phrase Response headers: provide information Body: optional data status line (protocol status code status phrase) data, e.g., requested HTML file header lines HTTP/ OK Connection close Date: Thu, 06 Aug :00:15 GMT : Apache/1.3.0 (Unix) Last-Modified: Mon, 22 Jun Content-Length: 6821 Content-Type: text/html blank line data data data data data

3 Request Methods and Response Codes Request methods include GET: return current value of resource, run program, HEAD: return the meta-data associated with a resource POST: update a resource, provide input to a program, Response code classes 1xx: informational (e.g., 100 Continue ) 2xx: success (e.g., 200 OK ) 3xx: redirection (e.g., 304 Not Modified ) 4xx: client error (e.g., 404 Not Found ) 5xx: server error (e.g., 503 Service Unavailable ) (Similar to other ASCII app. protocols like SMTP, FTP) 13 HTTP Resource Meta-Data Meta-data Information relating to a resource but not part of the resource itself Examples of meta-data Size of a resource Type of the content Last modification time Typing of content borrowed from Multipurpose Internet Mail Extensions (MIME) Data format classification (e.g., Content-Type: text/html) Enables browsers to automatically launch a viewer 14 Example: Conditional GET Request Fetch resource only if it has changed at the server GET /~ee122/fa06/ HTTP/1.1 Host: inst.eecs.berkeley.edu User-Agent: Mozilla/4.03 If-Modified-Since: Sun, 27 Aug :25:50 GMT <CRLF> avoids wasting resources to send again inspects the last modified time of the resource and compares to the if-modified-since time Returns 304 Not Modified if resource has not changed. or a 200 OK with the latest version otherwise 15 Stateless Operation Stateless protocol Each request-response exchange treated independently and servers not required to retain state Statelessness improves scalability Avoid need for server to retain info across requests Enable server to handle a higher rate of requests However, some applications need persistent state To uniquely identify the user or store temporary info E.g., personalize a Web page, compute profiles or access statistics by user, track a shopping cart. Done using cookies 16 Cookies Cookie Small(?) state stored by client on behalf of server Included in future requests to the server Request Response Set-Cookie: XYZ Request Cookie: XYZ 17 Web Components Send requests and receive responses Browsers, spiders, and agents s Receive requests and send responses Store or generate the responses Proxies Act as a server for the client, and a client to the server Perform extra functions such as caching, anonymization, logging, transcoding, filtering access. Can be explicit or transparent ( interception ) 18 3

4 Web Browser Generating HTTP requests: User types URL, clicks a hyperlink, or selects bookmark User clicks reload, or submit on a Web page Automatic downloading of embedded images Fetches content Via one or multiple HTTP connections Layout of response Parsing HTML and rendering the Web page Invoking helper applications (e.g., Acrobat, RealPlayer) Maintains cache Storing recently-viewed objects Checking that cached objects are fresh 19 Web Web site vs. Web server Web site: collections of Web pages associated with a particular host name Web server: program that satisfies client requests for Web resources Handling a client request: Accept a TCP connection Read and parse the HTTP request message Translate the URL to a filename Determine whether the request is authorized Generate and transmit the response 20 Web : Generating a Response Returning a file URL corresponds to a file (e.g., /www/index.html) the server returns the file as the response along with the HTTP response header Returning meta-data with no body Example: client requests object If-modified-since checks if the object has been modified and simply returns HTTP/ Not Modified Dynamically-generated responses URL corresponds to a program the server needs to run runs the program and sends the output to client 21 5 Minute Break Questions Before We Proceed? 22 HTTP Performance Fetch HTTP Items: Stop & Wait Most Web pages have multiple objects ( items ) E.g., HTML file and a bunch of embedded images Obvious way to fetch these: one item at a time Client Start fetching page Request item 1 Transfer item 1 Request item 2 Time What is that directly analogous to? Transfer item 2 Request item 3 Finish; display page Transfer item

5 HTTP Performance Pipelined Requests/Responses Most Web pages have multiple objects ( items ) E.g., HTML file and a bunch of embedded images Obvious way to fetch these: one item at a time What is that directly analogous to? What are the performance implications? Where does the analogy possibly break down? 25 Batch requests and responses to reduce the number of packets Multiple requests can be contained in one TCP segment Small items (common) can also share segments Note: maintains order of responses Item 1 always arrives before item 2 HTTP 1.1 feature (not in 1.0) Client Request 1 Request 2 Request 3 Transfer 1 Transfer 2 Transfer 3 26 Concurrent Requests/Responses Persistent Connections Use multiple connections to issue requests and responses in parallel Does not necessarily maintain order of responses Raises question of fairness Set of N parallel connections grabs bandwidth N times more aggressively than just one What s a reasonable/fair limit as traffic competes with that of other users? Client Request 1 Request 2 Request 3 Transfer 2 Transfer 3 Transfer 1 Handle multiple transfers per connection Including transfers subsequent to imaging current page Maintain TCP connection across multiple requests Either client or server can tear down the connection Performance advantages Avoid overhead of connection set-up and tear-down Allow TCP to learn more accurate RTT estimate Allow the TCP congestion window to increase I.e, leverage previously discovered bandwidth Performance - Caching Exploit locality of reference Modifier to GET requests: If-modified-since returns not modified if resource not modified since specified time Response header: Expires specifies for how long it s safe to cache the resource Or: No-cache ignore all caches; always get resource directly from server Features best for use by HTTP proxies Locality of reference increases if many clients share a proxy 29 HTTP Without Caching Many clients transfer same information Generates redundant server and network load experience unnecessary latency 30 5

6 Reverse Proxies Cache documents close to server decrease server load Typically done by content providers Only works for static content Forward Proxies Cache documents close to clients reduce network traffic and decrease latency Typically done by ISPs or corporate LANs Reverse proxies Reverse proxies Forward proxies Content Distribution Networks (CDNs) CDNs (cont d) Integrate forward and reverse caching functionality into one overlay network (usually) administrated by one entity Example: Akamai Documents are cached both as a result of clients requests (pull) due to being pushed in expectation of high access rate Forward proxies CDN Besides caching, do processing, e.g., Handle dynamic web pages Transcoding Example: Akamai Example: Akamai CDN Akamai creates new domain names for each client content provider. e.g., i.a.cnn.net = CNAME custom.i.cnn.net.edgesuite.net custom.i.cnn.net.edgesuite.net = CNAME a1921.aol.akamai.net Customer content provider modifies content so that embedded URLs reference new domains: CNN page s HREF s refer to i.a.cnn.net Customer pushes content out to Akamai as it changes Or: Akamai pulls it on demand w/ usual caching mech. Both CNN & Akamai have control over load distribution 35 DNS server for cnn.com GET cnn.com Akamaizes its content. lookup a1921.aol.akamai.net local DNS server akamai.net DNS servers Akamaized response object has inline URLs for secondary content at (after resolving CNAMEs) a1921.aol.akamai.net and other Akamai-managed DNS names. a Akamai servers store/cache secondary content for Akamaized services. b c 36 6

7 Hosting: Multiple Sites Per Machine Multiple Web sites on a single machine Hosting company runs the Web server on behalf of multiple sites (e.g., and Problem: GET index.html Is it for or Solution #1: multiple servers on the same machine Run multiple Web servers on the machine Have a separate IP address (or port) for each server Solution #2: include site name in the HTTP request Run a single Web server with a single IP address and include Host header (e.g., Host: ) Required header with HTTP Hosting: Multiple Machines Per Site Replicating a popular Web site Running on multiple machines to handle the load and to place content closer to the clients particularly if content isn t cacheable by proxies/cdns Problem: directing client to a particular replica To balance load across the server replicas To pair clients with nearby servers Solution #1: manual selection by clients Each replica has its own site name A Web page lists the replicas (e.g., by name, location) and asks clients to click on a hyperlink to pick 38 Hosting: Multiple Machines Per Site Solution #2: single IP address, multiple machines Run multiple machines behind a single IP address Hosting: Multiple Machines Per Site Solution #3: multiple addresses, multiple machines Same name but different addresses for all of the replicas Configure DNS server to return different addresses Load Balancer Internet Ensure all packets from a single TCP connection go to the same replica Caching vs. Replication Motivations for moving content close to users Reduce latency for the user Reduce load on the network and the server Caching Replicating the content on demand after a request Storing the response message locally for future use May need to verify if the response has changed and some responses are not cacheable Replication Planned replication of the content in multiple locations Updating of resources handled outside of HTTP Conclusions Key ideas underlying the Web Uniform Resource Identifier (URI), Locator (URL) HyperText Markup Language (HTML) HyperText Transfer Protocol (HTTP) Browser helper applications based on content type Performance implications Concurrent connections, pipelining, persistent conns. Main Web components, servers, proxies, CDNs Next lecture: drilling down to the link layer P&D , 2.6, 2.8 Can replicate scripts that create dynamic responses