Web Technologies Claudio Fornaro ver. 1.2 1 Web Standards At its core, the Web is made up of three standards: the Uniform Resource Identifier (URI), which is a universal system for referencing resources on the Web, such as Web pages; the HyperText Transfer Protocol (HTTP), which specifies how the browser and server communicate with each other; the HyperText Markup Language (HTML), used to define the structure and content of hypertext documents. 2 Web Standards URI and URL The World Wide Web Consortium (W3C, headed by Tim Berners-Lee) develops and maintains these and other standards that enable computers on the Web to effectively store and communicate different forms of information. A Uniform Resource Identifier (URI) is a compact string of characters used to identify or name a resource. The main purpose of this identification is to enable interaction with representations of the resource over a network, typically the World Wide Web, using specific protocols. 3 4
URI and URL http://user:pass@example.com:992/ animal/mammal/aquatic?species=whale#tail protocol login host port path query anchor 5 URI and URL A Uniform Resource Locator (URL) is a URI that, in addition to identifying a resource, provides a means of locating the resource by describing its primary access mechanism (e.g., its network location). In non technical language the term "URL" is used when referring to URI 6 URI and URL A Web browser will usually dereference URI http://example.org/ by performing an HTTP request to the host example.org, at the default HTTP port. Dereferencing URI mailto:bob@example.com will usually open a "Compose e-mail" window with the address bob@example.com in the "To" field. 7 URI and URL A Web browser will dereference URL http://example.org/animals/index.html by retrieving the identified resource (page index.html) located at the host example.org (default HTTP port). 8
HyperText Transfer Protocol Hypertext Transfer Protocol (HTTP) is a method used to transfer or convey information on the World Wide Web. Its original purpose was to provide a way to publish and retrieve HTML pages. Development of HTTP was coordinated by the World Wide Web Consortium (W 3 C) and the Internet Engineering HyperText Transfer Protocol HTTP is a request/response protocol between clients and servers. The originating client, such as a web browser or other end-user tool, is referred to as the user agent. The destination server, which stores or creates resources such as HTML files and images, is called the origin server. Task Force (IETF). 9 10 HTTP Versions HTTP has evolved into multiple, mostly backwards-compatible protocol versions. The client tells in the beginning of the request the version it uses, and the server uses the same or earlier version in the response. HTTP Versions Version 0.9 Deprecated. Only supports one command: GET, and does not support headers. Since this version does not support command POST, the client cannot pass much information to the server. 11 12
HTTP Versions Version 1.0 (May 1996) This is the first protocol revision to specify its version in communications. Still in wide use, especially by proxy servers. Allows persistent connections (alias keepalive connections: more than one requestresponse exchange use the same TCP/IP connection before it is closed) when HTTP Versions Version 1.1 (June 1999) Persistent connections enabled by default. Negotiation of contents (e.g. language): Accept-Language: en. Support of request pipelining: multiple requests are allowed to be sent at the same time (in the same packet) so that the server can prepare for the workload (better perormance). explicitly negotiated. 13 14 HTTP Request An HTTP client initiates a request by establishing a TCP connection to a particular port on a remote host (port 80 by default). An HTTP server listening on that port waits for the client to send a request message. 15 HTTP Request The request message consists of the following: a request line, such as GET /images/logo.gif HTTP/1.1 which requests the file logo.gif from the /images directory headers, such as Accept-Language: en an empty line an optional message body (data to be sent to the server) 16
HTTP Response Upon receiving the request, the server sends back a message composed of: a status line, such as HTTP/1.1 200 OK, headers, such as Content-Length: 438 an empty line an optional message body that could be the requested file, or an error message, or some other information. 17 HTTP Sample Connection Client Request GET /index.html HTTP/1.1 Accept-Language: en 18 HTTP Sample Connection Server Response HTTP Request Methods HTTP/1.1 200 OK Date: Mon, 23 May 2005 22:38:34 GMT Server: Apache/1.3.27 (Suse/Linux) Last-Modified: 08 Jan 2003 23:11:55GMT Content-Length: 438 Connection: close Content-Type: text/html;charset=utf-8 19 HTTP defines eight methods indicating the desired action to be performed on the identified resource: HEAD Asks for the response identical to the one that would correspond to a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content. 20
HTTP Request Methods continuation: GET Requests a copy of the specified resource. By far the most common method used on the Web today. POST Submits data to be processed (e.g. from an HTML form) to the identified resource. The data is included in the body of the request. 21 HTTP Request Methods continuation: PUT Uploads a copy of the specified resource. DELETE Deletes the specified resource. TRACE Echoes back the received request, so that a client can see what intermediate servers are adding or changing in the request. 22 HTTP Request Methods continuation: OPTIONS Returns the HTTP methods that the server supports. This can be used to check the functionality of a web server. CONNECT For use with a proxy that can change to being an SSL tunnel. HTTP servers are supposed to implement HTTP Session State HTTP is a stateless protocol: servers do not retain information about users between requests. State retaining is required (for example): to realize a virtual shopping basket, allowing for the content of the shopping cart to depend on the user's actions, at least: GET, HEAD, possibly OPTIONS. 23 24
HTTP Session State State retaining is required (cont.): to allow the webserver to know that the user is already authenticated (after a successful login), and therefore is allowed to access services or perform operations that are restricted to logged-in users, to allow websites for personalization based on users' preferences. HTTP Session State Methods for solving the state problem are based on establishing a session. A session is a set of related requests to the same server coming from the same browser within a limited amount of time. 25 26 HTTP Session State A session can be: anonymous, when the user need not to be identified, nominal, when the user is identified (e.g. by means of a username and password). A session is established by using: external applications, session identification numbers. HTTP Session State External applications are Java applets or ActiveX controls that directly connect to the remote server and manage the session. This solution is complex and nongeneral. 27 28
HTTP Session State A session identification number (SessionID) is an unique number associated to a session by the server. SessionID can be used with: hidden variables values, URL rewriting techniques, cookies. 29 HTTP Session State Hidden Variables Applicable only when using HTML forms. Forms submitted to a Web server have hidden fields (variables) containing the SessionID, to be passed to the server. This method is useful only when more forms are to be filled in sequence. Pages sent back by the server must be rewritten to have a hidden field set with the SessionID (to be passed back). 30 HTTP Session State URL Rewriting Each URL in the HTML page sent back to that user is previously rewritten by the server to include user s SessionID (usually in the query string part). When the user follows a link, the browser returns the attached query string to the server. E.g. resource page.php becomes page.php?userid=2f3bc2d. 31 HTTP Session State Cookies Cookies are parcels of text (not programs) sent by a webserver to a web browser and then sent back unchanged by the browser each time it accesses that server. By returning a cookie to a web server, the browser provides the server a means of connecting the current page view with prior page views. 32
HTTP Session State Cookies HTTP Session State Cookies Cookie setting can be requested directly by a web server response, or through a page embedded script. The cookie setter can specify a deletion (expiration) date. If the cookie setter does not specify a date, the cookie is removed once the user quits his/her browser. Specifying a date makes a cookie Cookies have been of concern for Internet privacy, they can be used to track users across multiple sites to create an anonymous profile of the user (third-party coockies). This allows the advertising company to select the banner to show to a user based on the user's profile. survive across sessions (persistent). 33 34 Cookies can also be stolen and forged. HTTP Session State Cookie Setting by a Webserver Proxy Servers Client first request: GET /index.html HTTP/1.1 Server first response: HTTP/1.1 200 OK Content-type: text/html Set-Cookie: name=abcd123; expires=31-dec-2010 23:59:59 GMT; path=/; domain=.example.net Client next request(s): GET /spec.html HTTP/1.1 A proxy server offers a service to allow clients to make indirect network connections to other network services (e.g. a Web site). A client connects to the proxy server, then requests a connection, file, or other resource available on a different server. The proxy provides the resource either by connecting to the specified server or by serving it from a cache. Cookie: name=abcd123 35 36
Proxy Servers Proxy Servers In some cases, the proxy may modify the client's request or the server's response for various purposes. Web proxies may be used to control web content surfing and intercept hostile content served by remote sites. Other web proxies reformat web pages for a specific purpose or audience (e.g. Some proxy servers are used to circumvent restrictions. A famous example is 'elgoog', which allowed users in China to use Google after it had been blocked there. Anonymous proxies, as opposed to regular transparent proxies, do not pass the information about the real user to for cell phones and PDAs). the remote web server. 37 38 MIME Encoding MIME Encoding HTTP requires that data be transmitted in the form of e-mail-like messages. The basic Internet e-mail transmission protocol (SMTP) supports only 7-bit ASCII characters. This effectively limits Internet e-mail to text messages which, when transmitted, include only the characters sufficient for writing a small number of Multipurpose Internet Mail Extensions (MIME) extends the format of e-mail to support: text in character sets other than US-ASCII, non-text attachments (binary content such as files containing images, sounds, movies, and programs.), multi-part message bodies, header information in non-ascii character languages, primarily English. 39 40 sets.
MIME Encoding Mapping messages into and out of MIME format is typically done automatically by clients and servers. MIME full encoding may raise the data size up to 33% more. The "Content Types" defined by MIME standards are also of importance outside of e-mail and HTTP. HyperText Markup Language HTML is the predominant markup language for the creation of web pages. It provides a means to describe the structure of text-based information in a document and to supplement that text with interactive forms, embedded images, and other objects. 41 42 HyperText Markup Language Hypertext is ordinary text that has been dressed up with extra features, such as formatting, images, multimedia, and links to other documents. Markup is the process of taking ordinary text and adding extra symbols. Each of the symbols used for markup in HTML is a command that tells a browser how to display the text. 43 Versions of HTML The last HTML specification published by the W3C is the HTML 4.01. The W3C's HTML Working Group (WG) has increasingly focused on the development of XHTML, an XML-based counterpart to HTML. In Nov. 2006, the HTML WG published a new charter indicating its intent to resume development of HTML in a way that unifies HTML 4 and XHTML 1. 44
XHTML XHTML applies the more rigorous, less ambiguous syntax requirements of XML to HTML to make it easier to process and extend. As support for XHTML has increased in browsers and tools, it has been embraced by many web standards advocates in preference to HTML. XHTML XHTML documents need to be wellformed (syntactically correct), this allows for automated processing to be performed using a standard XML library (unlike HTML, which requires a relatively complex, lenient, and generally custom parser) 45 46 The Modern Approach Each page can be thought as composed by three separated parts: Structure Presentation Behavior The Modern Approach Structure deals with the organization of the information in headings, paragraphs, lists, etc. The (X)HTML language is used to define the structure. Non structural tags and properties (e.g. font, color) are never used nor needed. 47 48
The Modern Approach Presentation languages format the page layout, controlling fonts, displacement, colors, etc. The most common language is Cascading Style Sheets (CSS) 49 The Modern Approach Presentation and structure can be changed without affecting each other: the page content can be modified without affecting the layout, each user can have different style sheets without changing the content: a layout optimized for small portable device displays, a large text layout for visuallyimpaired users (accessibility), the layout for an entire website can be held in one place, and updated easily. 50 The Modern Approach Browsers are Different Sofisticated effects (behavior) working on most platforms and browsers are obtained by scripting languages such as JavaScript applied to page objects. Page object organization is described by its DOM (more on this later). Despite the claimed high (but never complete) adherence to Web standards, each browser renders the web page in a slightly different way. Sometimes not so slightly... Old browsers do not support any standard and should be avoided. 51 52
Client-side Execution Some particular programs are executed by the user's web browser instead of the web server. This type of programs enables web pages to have different and changing content depending on user input, environmental conditions (such as the time of day), or other variables. Client-side Execution Scripts are small interpreted programs, often embedded within an HTML document, or contained in a separate file requested by the HTML document. The user's web browser itself executes the script, then displays the whole document including any visible output from the script. 53 54 Client-side Execution Scripts may also contain instructions for the browser to follow if the user interacts with the document in a certain way (e.g. a click on a certain button starts a script that checks input correctness). These instructions can be followed without further communication with the server, though they may require such communication. 55 Client-side Execution JavaScript is a scripting language developed to be embedded in Web pages. The standardized version is called ECMAScript. While its name is similar to Java, JavaScript was developed by Netscape and it has almost nothing to do with Java. 56
Client-side Execution JavaScript can modify the Document Object Model (DOM) properties of a page to change document content, structure, and style. Changes are applied immediatly, so Web pages can simulate the behavior of standalone programs. The manipulation of a page's DOM after the page is delivered to the client is Client-side Execution Ajax ("Asynchronous JavaScript And XML") is a JavaScript-based technology that provides a method whereby large or small parts within a Web page may be updated, using new information obtained over the network in response to user actions. called Dynamic HTML (DHTML). 57 58 Client-side Execution Ajax partial update capability allows the page to be much more responsive, interactive and interesting, without the user having to wait for whole-page reloads. Examples of Ajax techniques currently in use are Gmail, Google Maps, etc. 59 Client-side Execution Java applets are small pre-compiled programs (bytecode) downloaded upon request by the hosting HTML page. A Java Virtual Machine (JVM) must be present on the user s computer in order to execute the applet bytecode. Any graphical output of the program is displayed in the HTML page and can change independently. 60
Client-side Execution Client-side Execution Java never gained the popularity that Sun had hoped for, for a variety of reasons including: the lack of integration with other content (applets were confined to small boxes within the rendered page) computers at the time were supplied to end users without an installed Java Virtual Machine, this required a download by the ActiveX controls are small compiled programs downloaded by the hosting HTML page and used by the Internet Explorer web browser to incorporate applet-like functionality into web pages. ActiveX controls are unsafe for users of IE who turn on the browser's ability to automatically download and activate ActiveX controls within a non-trusted web page. user before applets would appear. 61 62 Client-side Execution Adobe Flash now performs many of the functions that were originally envisioned for Java applets including: playing of video content, animation (and games), some rich User Interface features. 63 Client-side Execution Flash files are called "Flash movies" or "Flash games", have a.swf file extension, and are executed by the Flash Player application or plug-in. Flash Player is freely available for the most common web browsers. Flash Player features support for a scripting language and bi-directional streaming of audio and video. 64
Server-side Execution To highly customize the response based on: the user's requirements or access rights, queries into databases or other data stores, some programs are executed on the server computer itself: by the O.S. as external programs (CGI) within the Web server process 65 Server-side Execution The Common Gateway Interface (CGI) is a standard protocol for interfacing external application software with a web server. This allows the server to pass requests from a web browser to the external application (running on the server). The web server can then return the output from the application to the web browser. 66 Server-side Execution Server-side Execution CGIs can be: compiled programs run by the Operating System (O.S.), tipically written in C, interpreted programs run by an interpreter, e.g. Perl and shell scripts. Compiled programs must be started by the O.S. for each CGI request. Scripts require that an interpreter be executed for each CGI request. Repeatedly starting a program can significantly reduce web server performance. Today, scripts are preferred and interpreters are integrated directly into Web servers as modules (extensions). This avoids the overhead of repeatedly starting a program or a language interpreter. Usually speed reduction of interpretation is hardly noticeable. Compiled programs run faster. 67 68
Server-side Execution When program code (usually a script) is embedded into the HTML page stored in the web server, when a page is requested: the Web server executes the embedded server-side code, assembles the result in an HTML page, sends back the page to the user. Server-side Execution Technologies using this approach are: PHP ASP (Active Server Pages - Microsoft) ColdFusion (Adobe) Server-side JavaScript 69 70 Server-side Execution Server-side Execution Other technologies pre-load (as a module) a JVM or similar runtime environments to execute pre-compiled programs. Technologies using this approach are: Servlet (Java) Perl Pyton Other technologies use a hybrid approach, when a page is requested: the code embedded in the HTML page is compiled, the resulting program is executed by the virtual machine, and it is kept for further use. Technologies using this approach are: Java Server Pages (JSP), that produce servlet Ruby 71 72
Web Services Web services are application components that can be used by other applications. Web services communicate over the Internet using open protocols. Web services are self-contained and self-describing. XML is the basis for Web services. Web Services Unlike traditional Web client-server 2/3- tier model (Web browser and Web server/database), Web services do not provide the user with a GUI. Web services allow different applications from different locations to communicate with each other without custom coding (program code is ready to be used). 73 74 Web Services All communication is in XML, thus Web services are not tied to any one operating system or programming language. Web services do not require the use of browsers or HTML. Web services are sometimes called application services. Web Services Web services allow organizations to communicate data without deep knowledge of each other's IT systems behind a firewall. 75 76
Web Services The open standards used are: The XML standard, that describes how to tag the data. The SOAP standard, that describes how to transfer the data. The WSDL standard, that describes how to use the available services. The UDDI standard, that describes how to list (publish) the available services. 77 Web Services An UDDI application is accessed to get the list of available services. An XML document is used to request an available service. The XML document is tranferred using the SOAP standard. The application is used as described in the WSDL document. 78 Web Services Simple Object Access Protocol SOAP is a communication protocol between applications. SOAP is a format for sending messages via Internet. SOAP is platform and language independent. SOAP is based on XML, simple and extensible. Web Services Web Services Description Language WSDL is used to describe Web services and also to locate Web services. WSDL is written in XML. WSDL itself is described by an XML document. SOAP allows to get around firewalls. 79 80
Web Services Universal Description, Discovery and Integration Web Services UDDI is a directory for storing information about web services. UDDI is a directory of web service interfaces described by WSDL. UDDI communicates via SOAP. 81 82 Secure HTTP Securing a session means preventing eavesdropping, tampering, and message forgery. Endpoint authentication and communications privacy over the Internet is obtained by using cryptography. Secure HTTP Typically, only the server is authenticated (i.e., its identity is ensured) while the client remains unauthenticated. This means that the end user (be that a person, or an application such as a web browser), can be sure of whom they are "talking" to. 83 84
Secure HTTP The next level of security - both ends of the "conversation" being sure of who they are "talking" to - is known as mutual authentication. Mutual authentication requires Public Key Infrastructure (PKI) deployment to clients. 85 Secure HTTP There are currently two methods of establishing a secure HTTP connection: the https URI scheme, the HTTP 1.1 Upgrade header. The https URI scheme has been deprecated; however, as browser support for the Upgrade header is nearly non-existent, the https URI scheme is still the dominant method. 86 Secure HTTP https URI Scheme https: is a URI scheme syntactically identical to the http: scheme used for normal HTTP connections. It tells the browser to use an added encryption layer of SSL/TLS to protect the traffic. SSL/TLS is especially suited for HTTP since it can provide some protection even if only one side (the server) of the Secure HTTP HTTP 1.1 Upgrade Header HTTP 1.1 introduced support for the Upgrade header. In the exchange, the client begins by making a clear-text request, which is later followed by a server demand to upgrade the connection to TLS. The server returns a code to the plaintext request that alerts legacy clients that the failure was client-related. communication is authenticated. 87 88
Secure HTTP HTTP 1.1 Upgrade Header The benefits of using this method for establishing a secure connection are the following: it removes messy and problematic redirection and URL rewriting on the server side, it reduces user confusion by providing a single way to access a particular resource. 89 SSL/TLS Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL), are cryptographic protocols which provide secure communications and authentication on the Internet for web browsing, e-mail, and other data transfers. There are slight differences between SSL 3.0 and TLS 1.1, but the protocol remains substantially the same. 90