COSC4377. Chapter 2: Outline

Transcription

1 Lecture 5 Chapter 2: Outline 2.1 principles of network applications app architectures app requirements 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail SMTP, POP3, IMAP 2.5 DNS 2.6 P2P applications 2.7 socket programming with UDP and TCP 2 1

2 Question: What is the Web? The World Wide Web, or simply Web, is a way of accessing information over the medium of the Internet. It is an informationsharing model that is built on top of the Internet. The Web uses the HTTP protocol, only one of the languages spoken over the Internet, to transmit data. Web services, which use HTTP to allow applications to communicate in order to exchange business logic, use the Web to share information. The Web also utilizes browsers, such as Internet Explorer or Firefox, to access Web documents called Web pages that are linked to each other via hyperlinks. Web documents also contain graphics, sounds, text and video. The Internet is a massive network of networks, a networking infrastructure. It connects millions of computers together globally, forming a network in which any computer can communicate with any other computer as long as they are both connected to the Internet. Information that travels over the Internet does so via a variety of languages known as protocols. 3 Web and HTTP First, a review web page consists of objects object can be HTML file, JPEG image, Java applet, audio file, web page consists of base HTML file which includes several referenced objects each object is addressable by a URL, e.g., host name HTTP = HyperText Transport Prototcol HTML = HyperText Markup Language path name JPEG = Joint Photographic Experts Group 4 2

3 WWW, HTTP, HTML WorldWideWeb proposed by Tim Berners Lee in March 1989 and implemented a first version in November 1989 while working at CERN. The first version of the protocol had only one method, namely GET, which would request a page from a server. The response from the server was always an HTML page. The first documented version of HTTP was HTTP V0.9 (1991). Lee 5 HTTP overview HTTP: hypertext transfer protocol Web s application layer protocol client/server model client: browser that requests, receives, (using HTTP protocol) and displays Web objects server: Web server sends (using HTTP protocol) objects in response to requests PC running Firefox browser iphone running Safari browser server running Apache Web server 6 3

4 HTTP overview (continued) uses TCP: client initiates TCP connection (creates socket) to server, port 80 server accepts TCP connection from client HTTP messages (applicationlayer protocol messages) exchanged between browser (HTTP client) and Web server (HTTP server) TCP connection closed HTTP is stateless server maintains no information about past client requests aside protocols that maintain state are complex! past history (state) must be maintained if server/client crashes, their views of state may be inconsistent, must be reconciled 7 Port numbers Service names and port numbers are used to distinguish between different services that run over transport protocols such as TCP, UDP, DCCP, and SCTP. Service names are assigned on a first come, firstserved process, as documented in [RFC6335]. Port numbers are assigned in various ways, based on three ranges: System Ports (0 1023), User Ports ( ), and the Dynamic and/or Private Ports ( ); the difference uses of these ranges is described in [RFC6335]. System Ports are assigned by IETF process for standards track protocols, as per [RFC6335]. User Ports are assigned by IANA using the "IETF Review" process, the "IESG Approval" process, or the "Expert Review" process, as per [RFC6335]. Dynamic Ports are not assigned. names port numbers/service names port numbers.xhtml 8 4

5 Port numbers examples Service Port Description ftp data 20 File transfer, data ftp 21 File transfer, control ssh 22 Secure Shell smtp 25 Simple Mail Transfer Prototcol name 42 Host Name Server nicname 43 Who is Whoispp/whois++ 63 http/www 80 Hostname 101 NIC Host Name Server pop3 110 Post Office Protocol v3 snmp 161 Simple Network Management Protocol bgp 179 Border Gateway Protocol https 443 http over TLS/SSL names port numbers/service names port numbers.xhtml 9 HTTP connections non persistent HTTP at most one object sent over TCP connection connection then closed downloading multiple objects required multiple connections persistent HTTP multiple objects can be sent over single TCP connection between client, server 10 5

6 suppose user enters URL: Non persistent HTTP (contains text, references to 10 jpeg images) time 1a. HTTP client initiates TCP connection to HTTP server (process) at on port HTTP client sends HTTP request message (containing URL) into TCP connection socket. Message indicates that client wants object somedepartment/home.index 1b. HTTP server at host waiting for TCP connection at port 80. accepts connection, notifying client 3. HTTP server receives request message, forms response message containing requested object, and sends message into its socket 11 Non persistent HTTP (cont.) 4. HTTP server closes TCP connection. 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects time 6. Steps 1-5 repeated for each of 10 jpeg objects 12 6

7 Non persistent HTTP: response time RTT (definition): time for a small packet to travel from client to server and back HTTP response time: one RTT to initiate TCP connection one RTT for HTTP request and first few bytes of HTTP response to return file transmission time non persistent HTTP response time = 2RTT+ file transmission time initiate TCP connection RTT request file RTT file received time time time to transmit file 13 Persistent HTTP non persistent HTTP issues: requires 2 RTTs per object OS overhead for each TCP connection browsers often open parallel TCP connections to fetch referenced objects persistent HTTP: server leaves connection open after sending response subsequent HTTP messages between same client/server sent over open connection client sends requests as soon as it encounters a referenced object as little as one RTT for all the referenced objects 14 7

8 HTTP request message two types of HTTP messages: request, response HTTP request message: ASCII (human readable format) request line (GET, POST, HEAD commands) header lines carriage return, line feed at start of line indicates end of header lines carriage return character line-feed character GET /index.html HTTP/1.1\r\n Host: www-net.cs.umass.edu\r\n User-Agent: Firefox/3.6.10\r\n Accept: text/html,application/xhtml+xml\r\n Accept-Language: en-us,en;q=0.5\r\n Accept-Encoding: gzip,deflate\r\n Accept-Charset: ISO ,utf-8;q=0.7\r\n Keep-Alive: 115\r\n Connection: keep-alive\r\n \r\n 15 HTTP request message: general format method header field name sp URL sp version cr lf value ~ ~ cr lf request line header lines header field name value cr lf cr lf entity body ~ ~ body For specification see RFC

9 Sample Header Fields Field name Description Example Accept Content Types that are acceptable for the response Accept: text/plain Accept Charset Character sets that are acceptable Accept Charset: utf 8 Accept Datetime Acceptable version in time Accept Datetime: Thu, 31 May :35:00 GMT Accept Encoding List of acceptable encodings. See HTTP compression. Accept Encoding: gzip, deflate Accept Language List of acceptable human languages for response Accept Language: en US Authorization Authentication credentials for HTTP authentication Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ== Cache Control Used to specify directives that MUST be obeyed by all caching mechanisms along the request/response chain Cache Control: no cache Connection What type of connection the user agent would prefer Connection: keep alive Content Length The length of the request body in octets (8 bit bytes) Content Length: 348 Content Type The MIME type of the body of the request (used with POST and PUT requests) Content Type: application/x www form urlencoded Cookie an HTTP cookie previously sent by the server with Set Cookie (below) Cookie: $Version=1; Skin=new; Date The date and time that the message was sent Date: Tue, 15 Nov :12:31 GMT From The address of the user making the request From: user@example.com Host The domain name of the server (for virtual hosting), and the TCP port number on which the server is listening. The port number may be omitted if the port is the standard port for the service requested. [8] Mandatory since HTTP/1.1. Although domain name are specified as case insensitive, [9][10] it is not specified whether the contents of the Host: en.wikipedia.org:80 Host: en.wikipedia.org Host field should be interpreted in a case insensitive manner [11] and in practice some implementations of virtual hosting interpret the contents of the Host field in a case sensitive manner. [citation needed Uploading form input POST method: web page often includes form input input is uploaded to server in entity body e.g. for Annotation of existing resources; Posting a message to a bulletin board, newsgroup, mailing list, or similar group of articles; Providing a block of data, such as the result of submitting a form, to a data handling process; Extending a database through an append operation. URL method uses GET method input is uploaded in URL field of request line:

10 Method types HTTP/1.0 (1995, RFC 1945): GET POST HEAD asks server to leave requested object out of response HTTP/1.1 (1999, RFC2616): GET, POST, HEAD PUT uploads file in entity body to path specified in URL field DELETE deletes file specified in the URL field Derived from Kurose & Ross, 6 th Ed 19 HTTP response message status line (protocol status code status phrase) header lines data, e.g., requested HTML file HTTP/ OK\r\n Date: Sun, 26 Sep :09:20 GMT\r\n Server: Apache/ (CentOS)\r\n Last-Modified: Tue, 30 Oct :00:02 GMT\r\n ETag: "17dc6-a5c-bf716880"\r\n Accept-Ranges: bytes\r\n Content-Length: 2652\r\n Keep-Alive: timeout=10, max=100\r\n Connection: Keep-Alive\r\n Content-Type: text/html; charset=iso \r\n \r\n data data data data data

11 HTTP response status codes status code appears in 1st line in server to client response message. some sample codes: 200 OK request succeeded, requested object later in this msg 301 Moved Permanently requested object moved, new location specified later in this msg (Location:) 400 Bad Request request msg not understood by server 404 Not Found requested document not found on this server 505 HTTP Version Not Supported 21 HTTP Status Codes 1XX This class of status code indicates a provisional response 2XX This class of status code indicates that the client's request was successfully received, understood, and accepted. 3XX This class of status code indicates that further action needs to be taken by the user agent in order to fulfill the request. 4XX The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications. 5XX This class of status codes indicate that the server is aware that it has erred or is incapable of performing the request. sec10.html 22 11

12 URLs RFC 3986, August 1998, specifies a standard for inclusion of attributes/objects in URLs 23 Trying out HTTP (client side) for yourself 1. Telnet to your favorite Web server: telnet cis.poly.edu 80 opens TCP connection to port 80 (default HTTP server port) at cis.poly.edu. anything typed in sent to port 80 at cis.poly.edu 2. type in a GET HTTP request: GET /~ross/ HTTP/1.1 Host: cis.poly.edu by typing this in (hit carriage return twice), you send this minimal (but complete) GET request to HTTP server 3. look at response message sent by HTTP server! (or use Wireshark to look at captured HTTP request/response) 24 12

13 User server state: cookies many Web sites use cookies four components: 1) cookie header line of HTTP response message 2) cookie header line in next HTTP request message 3) cookie file kept on user s host, managed by user s browser 4) back end database at Web site example: Susan always access Internet from PC visits specific e commerce site for first time when initial HTTP requests arrives at site, site creates: unique ID entry in backend database for ID RFC 2109, February 1997, provides a standard for cookies 25 Cookies: keeping state (cont.) client server ebay 8734 cookie file ebay 8734 amazon 1678 one week later: ebay 8734 amazon 1678 usual http request msg usual http response set-cookie: 1678 usual http request msg cookie: 1678 usual http response msg usual http request msg cookie: 1678 usual http response msg Amazon server creates ID 1678 for user create entry cookiespecific action cookiespecific action access access backend database 26 13

14 Cookies (continued) what cookies can be used for: authorization shopping carts recommendations user session state (Web e mail) aside cookies and privacy: cookies permit sites to learn a lot about you you may supply name and to sites how to keep state : protocol endpoints: maintain state at sender/receiver over multiple transactions cookies: http messages carry state 27 Set Cookie (RFC 2109) set cookie = "Set Cookie:" cookies cookies = 1#cookie cookie = NAME "=" VALUE *(";" cookie av) NAME = attr VALUE = value cookie av = "Comment" "=" value "Domain" "=" value "Max Age" "=" value "Path" "=" value "Secure" "Version" "=" 1*DIGIT Informally, the Set Cookie response header comprises the token Set Cookie:, followed by a comma separated list of one or more cookies. Each cookie begins with a NAME=VALUE pair, followed by zero or more semi colonseparated attribute value pairs. The NAME=VALUE attribute value pair must come first in each cookie. The others, if present, can occur in any order. If an attribute appears more than once in a cookie, the behavior is undefined

15 Set Cookie (RFC2109) NAME=VALUE Required. The name of the state information ("cookie") is NAME, and its value is VALUE. NAMEs that begin with $ are reserved for other uses and must not be used by applications. Domain=domain. Optional. The Domain attribute specifies the domain for which the cookie is valid. An explicitly specified domain must always start with a dot. Max Age=delta seconds. Optional. The Max Age attribute defines the lifetime of the cookie, in seconds. The delta seconds value is a decimal nonnegative integer. After delta seconds seconds elapse, the client should discard the cookie. A value of zero means the cookie should be discarded immediately. Path=path. Optional. The Path attribute specifies the subset of URLs to which this cookie applies. Secure Optional. The Secure attribute (with no value) directs the user agent to use only (unspecified) secure means to contact the origin server whenever it sends back this cookie. The user agent (possibly under the user's control) may determine what level of security it considers appropriate for "secure cookies. The Secure attribute should be considered security advice from the server to the user agent, indicating that it is in the session's interest to protect the cookie contents. Version=version. Required. The Version attribute, a decimal integer, identifies to which version of the state management specification the cookie conforms. For this specification, Version=1 applies. 29 Web caches (proxy server) Goal: satisfy client request without involving origin server user sets browser: Web accesses via cache browser sends all HTTP requests to cache object in cache: cache returns object else cache requests object from origin server, then returns object to client client client proxy server origin server origin server 30 15

16 More about Web caching cache acts as both client and server server for original requesting client client to origin server typically cache is installed by ISP (university, company, residential ISP) why Web caching? reduce response time for client request reduce traffic on an institution s access link Internet dense with caches: enables poor content providers to effectively deliver content (so too does P2P file sharing) 31 Caching example: assumptions: avg object size: 100K bits avg request rate from browsers to origin servers:15/sec avg data rate to browsers: 1.50 Mbps RTT from institutional router to any origin server: 2 sec access link rate: 1.54 Mbps consequences: LAN utilization: 0.15% access link utilization = 99% total delay = Internet delay + access delay + LAN delay = 2 sec + minutes + usecs institutional network public Internet 1.54 Mbps access link origin servers 1 Gbps LAN Due to link load close to capacity. Problem! Recall traffic intensity = La/R 32 16

17 Caching example: fatter access link assumptions: avg object size: 100K bits avg request rate from browsers to origin servers:15/sec avg data rate to browsers: 1.50 Mbps RTT from institutional router to any origin server: 2 sec access link rate: 1.54 Mbps 15.4 Mbps consequences: LAN utilization: 0.15% access link utilization = 99% 9.9% total delay = Internet delay + access delay + LAN delay = 2 sec + minutes + usecs institutional network public Internet 1.54 Mbps access link origin servers 1 Gbps LAN 15.4 Mbps msecs Cost: increased access link speed (not cheap!) 33 Caching example: install local cache assumptions: avg object size: 100K bits avg request rate from browsers to origin servers:15/sec avg data rate to browsers: 1.50 Mbps RTT from institutional router to any origin server: 2 sec access link rate: 1.54 Mbps consequences: LAN utilization: 0.15%, except link to cache access link utilization = 100%? total delay = Internet? delay + access delay + LAN delay = 2 sec + minutes + usecs How to compute link utilization, delay? Cost: web cache (cheap!) institutional network public Internet 1.54 Mbps access link origin servers 1 Gbps LAN local web cache 34 17

18 Caching example: install local cache Calculating access link utilization, delay with cache: suppose cache hit rate is % requests satisfied at cache, 60% requests satisfied at origin access link utilization: 60% of requests use access link data rate to browsers over access link = 0.6*1.50 Mbps =.9 Mbps utilization = 0.9/1.54 =.58 institutional network total delay = 0.6 * (delay from origin servers) * (delay when satisfied at cache) = 0.6 (2 sec) (~msecs) = ~ 1.2 secs less than with 15.4 Mbps link (and cheaper too!) public Internet 1.54 Mbps access link origin servers 1 Gbps LAN local web cache 35 Goal: don t send object if cache has up to date cached version no object transmission delay lower link utilization Conditional GET cache: specify date of cached copy in HTTP request If-modified-since: <date> server: response contains no object if cached copy is up to date: HTTP/ Not Modified local web cache HTTP request msg If-modified-since: <date> HTTP response HTTP/ Not Modified server object not modified after <date> cache: specify date of cached copy in HTTP request If-modified-since: <date> server: response contains object if cached copy is up to date: HTTP/ OK <data> HTTP request msg If-modified-since: <date> HTTP response HTTP/ OK <data> object modified after <date> 36 18