Web Engineering. Basic Technologies: Protocols and Web Servers. Husni

Size: px

Start display at page:

Download "Web Engineering. Basic Technologies: Protocols and Web Servers. Husni"

Rosanna Smith
6 years ago
Views:

1 Web Engineering Basic Technologies: Protocols and Web Servers Husni

2 Basic Web Technologies HTTP and HTML Web Servers Proxy Servers Content Delivery Networks

3 Where we will be later in the course...

4 Where we will be later in the course... Supporting a range of client devices

5 World-Wide Web Series of Protocols URL/URI unique identification of resources URI examples mailto: ftp://ftp.inf.ethz.ch/ed/report.txt tel: URL is a URI that provides information about how to locate a resource HTTP Hypertext Transfer Protocol HTML Hypertext Mark-Up Language Web Browsers Internet Explorer, Mozilla Firefox..

6 HTML <html> <head> <title>michael's Personal Home Page</title> </head> <body bgcolor="#ffffff" text="#000000"> <h1> Michael </h1> <img src="michael.bmp" align="right"/> <h2>work</h2> Michael works at <a href=" ETH Zurich </a> <h2>personal</h2> <address> CNB E106 <br/> Zurich <br/> Switzerland </address> </body> </html>

7 HTML Based on Hypertext Style of Navigation Simple and Easy to Publish on Web Structure, Content or Presentation? wide use of table elements for formatting layout address elements describe the content Problems of link maintenance document interpretation Flexible unknown tags ignored by browsers => easy to extend with customised tags

8 HTML... Document meta data can be included in header <head> <title>michael's Personal Home Page</title> <meta name="keywords" content="web, databases, java"> <meta name= "authors" content="michael"> <meta http-equiv="expires" content="25 Mar 2006"> <meta http-equiv="refresh" content="15"> </head> Keywords used by search engines

9 HTML5 : The Next Generation of HTML New standard for HTML, XHTML and HTML DOM Work in progress, most browsers now have some support Cooperation between W3C and Web Hypertext Application Technology Working Group (WHATWG) One goal was to have a clearer separation of content and presentation HTML5 - content CSS3 - layout as well as look and feel Second goal to make it easier to process documents and their content Third goal to reduce the need for plug-ins

10 Key Features of HTML5 Tags to support a stronger document model to make it easier to identify logical elements of documents section, article, aside, details, header, footer Support for other media types video, audio Take over some of the things normally handled by JavaScript such as form field validation form field input types such as , url, dates, numbers. far richer set of event attributes Support for client-side storage replacing cookies

11 HTTP 1.0 hypertext transfer protocol one object transferred per connection

12 HTTP request GET /www/globis.html HTTP/1.0 Accept: www/source Accept: text/html Accept: image/gif User-Agent: Lynx/2.4 libwww/2.14

13 HTTP result HTTP/ OK Date: Thursday, 23-April-98 09:00:05 GMT Server: NCSA/1.4.2 MIME-version: 1.0 Content-type: text/html Content-length: 3500 <html> </html> note blank line between header and body of message

14 HTML Form GET /cgi-bin/globis.pl?user=moira&pass=fred

15 HTML Form <html>... <form action="/cgi-bin/globis.pl" method="get"> Name: <input type="text" name="user" size=10> <br/> Password: <input type="password" name="pass" size=6> <br/> <input type="submit" value="enter"> <input type="reset" value="clear"> </form> </html>

16 Introducing Dynamic Content need to introduce some mechanism to execute programs on the server side and dynamically generate HTML documents

17 CGI Programming Common Gateway Interface Executes Programs on Server Side

18 CGI Result

19 CGI Programs Can be written in any language Desirable Features ease of text manipulation ability to access environment variables ability to interface with other services Commonly Used Languages Perl, C/C++, Tcl, Java

20 Accessing Form Data #!/usr/local/bin/perl print "Content-type: text/html", "\n\n"; $query_string=$env {'QUERY_STRING'};.. ($field_name, $param) = split (/=/, $query_string);.. if ($user eq "moira" ) { print "Location: /globis.html", "\n\n"; } else

21 Unix Environment Variables SERVER_NAME REMOTE_HOST REMOTE_ADDR REMOTE_USER REQUEST_METHOD QUERY_STRING...

22 GET and POST Two methods for sending Form Data GET appends form data to url GET /cgi-bin/globis.pl?name=globis HTTP/1.0 POST form data read from standard input POST /cgi-bin/globis.pl HTTP/ user =moira pass=fred

23 Server Side Includes Directives included in HTML Documents execute programs output data such as environment variables

24 Server Side Includes... <html> <head><title>globis</title></head> <body> <h1>welcome to </h1>... <address>moira()</address> </body> </html>

25 Server Side Includes... Configure Server to say documents which should be parsed AddType text/x-server-parsed-html.shtml AddType text/x-server-parsed-html.html directives supported Includes - display environment variables etc. Exec - execute External Programs

26 Where to Cache? Caching can occur at many different levels and locations in web architectures Four fundamental ways for implementing a caching mechanism browser caching proxy caching reverse proxy caching/server accelerators content delivery networks (CDN) We will go on to look at each of these in turn

27 Browser Caching Every browser contains cache of HTML docs & multimedia files Browser cache is a directory in user s hard disk Advantages simple universal Disadvantages applies only to static resources can be by-passed by content provider who can add suitable HTTP headers to response or directives to HTML page forcing browser not to cache

28 Proxy Caching A proxy cache lies between a community of users (e.g. D-INFK, ETHZ) and the public internet Works on same basic principles as browser cache, but on much larger scale (may be hundreds or thousands of users) Proxy caches sometimes implemented together with firewalls which control flow of requests/responses between intranet and internet Client requests have to somehow be routed to proxy server can be done through browser s proxy setting interception proxies have requests redirected to them by underlying network

30 proxy server: cache miss http_proxy=

31 proxy server: cache hit

32 HTTP/1.0 GET URL HEAD URL HTTP/ OK Date: Wed, 10 May :33:08 GMT Server: Apache/ (Win32) Last-Modified: Mon, 01 May :37:40 GMT Content-type: text/html Content-length: 907 GET URL If-Modified-Since: Sunday, 05 Mar :00:00 GMT HEAD similar to GET but only asks for response header rather than content

33 Browser and proxy caches All caches have a set of rules used to determine what can be cached and when to use cached resources some rules set in protocols some set by cache software (e.g. browser) some set by cache administrator Many of these rules based on information in the HTTP request/response header added by server/browser explicitly generated by content provider may be based on type of request or type of content example: HTTP header containing Cache-Control: no-cache

34 General caching rules If response s header says not to keep it, it won t be cached If no validator (e.g. a Last-Modified header) is present on a response, it will be considered uncacheable If request is authenticated or secure, it won t be cached A cached object is considered fresh (i.e. able to be sent to client without checking with origin server) if it has an expiry time or other age-controlling header set and is still fresh if object already seen and browser cache set to check once a session if proxy cache has seen object recently and long time since modified If a representation is stale, the server will be asked to validate it

35 HTTP header information for caching Example

36 HTTP validators and validation Validation used by servers and caches to communicate when an object has changed Most common validator is Last-Modified time if cache has object with last-modified time t, generate If-Modified-Since t request to server to check if object still current HTTP 1.1 introduced ETags as another kind of validator every time object changes, server generates a unique identifier ETag which is included in HTTP response header of object request server controls how ETags generated Most modern web servers generate both ETags and Last-Modified validators automatically for static content

37 HTTP cache-control max-age=[seconds] max amount of time page considered fresh; relative to time of request s-maxage=con [seds] public no-cache no-store must-revalidate similar to max-age, except only applies to shared caches (e.g. proxies) marks authenticated responses as cacheable; normally if HTTP authentication required, responses uncacheable forces cache to submit request to original server every time for validation before releasing cached copy instructs cache not to keep a copy under any circumstances instructs cache to obey any freshness information given about an object; counteracts some conditions in which cache may serve stale representations proxy-revalidate similar to must-revalidate, but only applies to proxy caches

38 What doesn t work HTML metatags example <meta http-equiv="expires" content="thu, 26 May :50:00 GMT"> <meta http-equiv="pragma" content="no-cache"> easy to use, but are not very effective HTML not usually read by proxy servers few browsers honour such specifications Pragma HTTP headers can include in HTTP header Pragma: no-cache HTTP specification does not specify how these should be handled and many browsers ignore it

39 Problems of proxy servers Connections to servers still required Still a high server load Servers lose control over their documents authorisation billing access statistics

40 Prefetching caches Request objects from the server without an explicit request Based on access patterns object analysis (HTML documents,...) explicit subscriptions Reduces latency If level of prefetching too high then may pay severe penalties in terms of increased network traffic server load

41 Proxy servers advantages reduce latency, network bandwidth and server load opportunity to analyse an organisation's usage patterns transparent to clients and servers disadvantages additional resources single point of failure chance that users get stale data from the cache

42 Reverse proxy caching Reverse proxy caches are also intermediaries, but instead of being deployed by network administrators are deployed by the webmasters themselves (i.e. server side) Improve web site s performance reliability scalability Typically some form of load balancer used to make one or more gateway caches look like the origin server to clients Sometimes known as gateway caches, surrogate caches, or server accelerators

43 Content Delivery Networks (CDNs) A content delivery network distributes gateway caches throughout the Internet (or part of it) and sells caching to interested web sites Akamai ( Original idea: when a client requests a page to the origin server, the server returns a page with rewritten links that point to a node of the CDN so that further client requests are handled by the CDN CDN serves requests using multiple cache nodes, selecting the optimal copy of the page given the geographical location of the user and the real-time network traffic conditions

44 Content Delivery Networks (CDNs)... CDNS now perform dynamic request routing using the Internet's Domain Name System (DNS) DNS is a distributed directory mapping fully qualified domain names (FQDN) to IP addresses To determine an FQDN's address, a DNS client sends a request to its local DNS server which then queries a set of authoritative servers When local DNS receives a response, it sends address to DNS client and saves it in cache Each DNS record has a time-to-live (TTL) field that tells DNS server how long it may cache result Normally the association of FQDN to IP address is static

45 Content Delivery Networks (CDNs)... CDNs use modified DNS servers for CDN server selection Results of a DNS query to one of these servers may vary depending on source of request and network condition To enable fast reaction to dynamic resource changes, the answer returned by the CDN's DNS server has a small TTL This approach is largely transparent to client and works for any web content Issues usually assumed client close to their local DNS servers single request from a local DNS server can represent differing number of web clients (hidden load factor)

Caching. Caching Overview

Caching. Caching Overview Overview Responses to specific URLs cached in intermediate stores: Motivation: improve performance by reducing response time and network bandwidth. Ideally, subsequent request for the same URL should be