Make the transition from server-side to (partially) client-side web applications

Size: px

Start display at page:

Download "Make the transition from server-side to (partially) client-side web applications"

Irma Collins
6 years ago
Views:

1 Goal of this lecture Make the transition from server-side to (partially) client-side web applications Learn that Ajax is more than a cleaning product Become friends with JSON, the «cooler» cousin of XML Learn how to dance the «OAuth dance» Give it a REST! Learn about a way to offer your (web) API functionality to non-programmers Make them integrate with you instead of the other way round norrie@inf.ethz.ch 1

2 Web Frameworks AJAX Client-Side Frameworks

3 What can JavaScript do? (Repetition) Embed snippets of code in HTML pages React to events Include dynamic pieces of text in HTML pages Access and write HTML elements Validate form data Detect client browser Store and retrieve cookies

4 AJAX Asynchronous JavaScript And XML combines Asynchronous processing of HTTP requests JavaScript And...? XML data serialization format Rather fuzzy term that incorporates a lot of different technologies: HTML/CSS for the presentation DOM manipulation to integrate received data into web page XML to exchange data XMLHttpRequest as a means to perform asynchronous communication JavaScript to implement the logic and handle the requests Send/receive data from a server asynchronously, no need to refresh whole page Has evolved over time Data format not constrained to XML anymore, JSON common alternative norrie@inf.ethz.ch 4

5 Request AJAX Architecture Traditional Architecture AJAX Architecture Web Server Web Server Request Response Response Web Browser Web Browser AJAX Engine

6 XMLHttpRequest API JavaScript API to send XML messages over HTTP Important methods: open(http method, url, async, ) setrequestheader(header, value) send(data) Important properties: onreadystatechange readystate status opens a connection sets value of an HTTP request header sends a request (data can be omitted) event listener to register callback state of the request (int enum) HTTP status code of the response Callback function Handles the response of the server asynchronously norrie@inf.ethz.ch 6

getelementbyid("mydiv").innerhtml = xmlhttp.responsetext; } } xmlhttp.open("get", "ajax_info.txt", true); xmlhttp.

7 AJAX Example function loadxmldoc() { var xmlhttp; xmlhttp = new XMLHttpRequest(); xmlhttp.onreadystatechange = function() { if (xmlhttp.readystate === 4 && xmlhttp.status === 200) { document.getelementbyid("mydiv").innerhtml = xmlhttp.responsetext; } } xmlhttp.open("get", "ajax_info.txt", true); xmlhttp.send(); } <div id="mydiv"><h2>let AJAX change this text</h2></div> <button type="button" onclick="loadxmldoc()">change Content</button> norrie@inf.ethz.ch 7

8 Common AJAX Operation Flow 1. Event triggers execution of handler function 2. Handler function processes event 3. Handler function instantiates an XMLHttpRequest object and registers a callback function 4. Handler function sends an HTTP request to remote server 5. Server processes request and sends XML response back to client 6. Callback function is invoked to process the server response asynchronously norrie@inf.ethz.ch 8

types Primitive types: null value, booleans, numbers and arbitrary strings Structured types: Arrays and objects A JSON object is an

9 JSON JavaScript Object Notation (JSON) is a lightweight, text-based, language-independent data interchange format. Used to represent data structures and associative arrays in a simple and human-readable way Four primitive types and two structured types Primitive types: null value, booleans, numbers and arbitrary strings Structured types: Arrays and objects A JSON object is an unordered collection (a set) of arbitrary name/value pairs Name: Any String Value: Any primitive or structured type norrie@inf.ethz.ch 9

JSON Format { Name and value separated by colon anullvalue : null, astring : Chuchichäschtli, anumber : 342.

10 JSON Format { Name and value separated by colon anullvalue : null, astring : Chuchichäschtli, anumber : e10, aboolean : false, Explicit nullable type Double-quoted Unicode String Scientific format, no octal or hexadecimal Two possible literals: true and false myarray : [ Anything, 42, true], } nestedobject : { } Enclosing Curly Braces Ordered, comma-separated, values can have different type key1 : value1, key2 : value2 Name/Value pairs separated by comma norrie@inf.ethz.ch 10

JSON (Data Exchange Format) Became a very popular alternative to XML Almost every large Web API supports JSON, often the primary data format JSON Schema provides validation (similar to XML

11 JSON (Data Exchange Format) Became a very popular alternative to XML Almost every large Web API supports JSON, often the primary data format JSON Schema provides validation (similar to XML Schema) but still in development Advantages: Human-readable Compact, simple syntax Disadvantages: Less efficient than binary protocols No (official) validation yet Does not support object references norrie@inf.ethz.ch 11 { } "id": 1, "name": "TV", "price": 2300, "tags": [ "LED", "QHD" ], "stock": { "warehouse": 300, "retail": 20 } { "name": "Product", "properties": { "id": { "type": "number", "description": "Product identifier", "required": true }, "name": { "type": "string", "description": "Name of the product", "required": true },... } JSON JSON Schema

JavaScript Web Frameworks Usually not «full

Event-Handling REST Abstractions Often promote

12 JavaScript Web Frameworks Usually not «full stack» web frameworks, but typical features include: Declarative Bindings Template Language Filter / Transformations Event-Handling REST Abstractions Often promote particular design pattern: MVC, MVP, MVVM norrie@inf.ethz.ch 13

13 AngularJS DemoLive Demo Exemplary JavaScript Web framework for client-side applications

14 TodoMVC Challenge: How to pick the right JavaScript web framework? How to compare different design patterns? Idea: Implement the same simple Todo application with each framework

15 Web APIs From Screen Scraping to Web Integration

Web Scraping Idea: Extract information from websites, often transforming unstructured data (HTML) into structured records Extracted data can be stored in databases and integrated with other data or

16 Web Scraping Idea: Extract information from websites, often transforming unstructured data (HTML) into structured records Extracted data can be stored in databases and integrated with other data or analyzed offline Web Scraper: Software that mimicks a user (human being) browsing the web, but processes HTML documents directly (no rendering or user interaction involved) Web scraping techniques Parse DOM tree XPath and XQuery Regular Expressions jquery / CSS Selectors norrie@inf.ethz.ch 17

17 Example: Scrapy Python library to crawl websites and extract structured data class DmozSpider(BaseSpider): name = "dmoz" allowed_domains = ["dmoz.org"] start_urls = [ " ] def parse(self, response): hxs = HtmlXPathSelector(response) sites = hxs.select('//ul/li') for site in sites: title = site.select('a/text()').extract() link = site.select('a/@href').extract() desc = site.select('text()').extract() print title, link, desc norrie@inf.ethz.ch 18

18 Mashups In web development, a mashup is a Web application that combines data or functionality from two or more sources into a single integrated application. The term mashup implies easy, fast integration, frequently done by access to open APIs and data sources to produce results that were not the original reason for producing the raw source data. -- Wikipedia.org Types of Mashups Data Aggregation: (Similar) data objects from different sources Computation: Currency converter, media type converter Data Visualisation: Mapping Data connectors to fetch data from various sources Facebook, Twitter, Flickr, etc. Data converters and manipulation methods Visual data rendering controls Text effects, image slideshows, etc norrie@inf.ethz.ch 19

19 Example: Yahoo! Pipes

20 Programmable Web APIs to access platforms and services Heterogeneity Protocols Data formats Programming languages Services

21 Source: Growth of the ProgrammableWeb API Directory ( 23

22 Web APIs Application Programming Interfaces to access services over the web Call remote (service) methods or manipulate «web resources» directly Popular techniques: RESTful Web Services Web Services Description Language (highly standardized) XML RPC Standardized message/serialization formats JSON XML SOAP (Simple Object Access Protocol), XML Information Set

23 REST An architectural style for distributed hypermedia systems Introduced by Roy Fielding in 2000 Worked on HTTP 1.0/1.1 Developed REST in parallel to HTTP/1.1 as part of his doctoral dissertation REST describes an architecture, not a protocol The World Wide Web based on HTTP is the largest implementation of that architecture style norrie@inf.ethz.ch 27

24 REST Style Architecture Deriving REST by applying several constraints: Client-server: Separation of concerns, improves portability of user interface on the client side, improves scalability on the server side Stateless: Client-server interaction is stateless, no context, each client request contains all necessary information Cache: Mechanism to indicate whether server reponses are cacheable or non-cacheable Uniform Interface: Main feature of REST, four interface constraints Layered System: Proxies, Gateways, Caches Code-On-Demand (optional): Extend clients by client-side scripts RESTful interfaces and architectures comply with those constraints Fielding, Roy Thomas. Architectural styles and the design of network-based software architectures. Diss. University of California,

25 REST Style Architecture: Uniform Interface Identification of Resources Resources have to be uniquely identifiable URI/URL identifiers predominant in most systems Manipulation of Resources through representations Resources can have several representations but each needs to include all the necessary information to manipulate the resource Clients (i.e. the web browser) work with representations only Self-descriptive messages Hypermedia as the engine of application state Interaction with application is determined by the structure embedded in the resources, such as links to related resources Such a (idealized) Uniform Interface is rarely achieved in practice norrie@inf.ethz.ch 29

(Web) Resources Any information that can be uniquely identified, i.e. has a name, within an information system (such as the Web) Unifom Resource Identifiers are used to identify (web) resources, URIs

maps to zero or more different representations: HTML document, an image, RSS item, RDF, JSON, XML Type of representation specified by Internet media type Examples http://www.flickr.

26 (Web) Resources Any information that can be uniquely identified, i.e. has a name, within an information system (such as the Web) Unifom Resource Identifiers are used to identify (web) resources, URIs are standardized in RFC 3986 Can refer to anything that has an identity A person, a document, a file, abstract concepts such as mathematical operations or feelings Representations A single resource maps to zero or more different representations: HTML document, an image, RSS item, RDF, JSON, XML Type of representation specified by Internet media type Examples RFCs (Request for Comment) are publications by the Internet Engineering Task Force, the main standards body for Internet standards norrie@inf.ethz.ch 30

27 RESTful Web Services Adhere to the principle of REST Implementations based on HTTP/1.1 (RFC 2616), use HTTP/1.1 methods to manipulate resources Assume certain semantics (but this cannot be enforced technically) Safe: Retrieval only, no side-effects Idempotent: Calling method several times yields same result, no additional effect HTTP VERB Semantics Description GET POST Safe, Idempotent Retrieves information about an entity identified by Request-URI Submit an (new) entity as a subordinate of resource identified by Request-URI PUT Idempotent Replaces existing entity identified by Request-URI with new one DELETE Idempotent Deletes resource identified by Request-URI HTTP verbs (methods) used in REST style APIs norrie@inf.ethz.ch 31

28 RESTful Web Services (contd.) For a given Web Resource, HTTP methods translate to actions on the resource GET Retrieve the representation of «Michael» PUT Replace or modify existing friend POST Create a new friend (usually operates on «parent collection»: ) DELETE Delete friend resource Error Handling API errors often indicated with HTTP status codes and (non-standardized) response messages No official standard Everyone seems to have a REST API Provides no means to describe the Web API in a standardized way (as opposed to WSDL) norrie@inf.ethz.ch 32

29 RESTful Web Services: Partial Updates Problem: PUT semantics require the entire representation of the resource to be sent to the server for every update HTTP verb «PATCH» is new kid on the block Neither safe nor idempotent Allows partial updates to resources Propagated in Rails 4.0 Specifications RFC 5789 specifies general PATCH semantics, but no «diff» format RFC 6902 specifies partial updates for JSON Old Solutions POST updates POST /friends/alfonso Content-Type: application/x-wwwform-urlencoded age=24 => How to remove (optional) attributes? POST urls with query parameters that follow APIspecific conventions POST /friends/alfonso?age=24 => No generic semantics! Define attributes as (sub-)resources PUT /friends/alfonso/age => Cumbersome!

30 Example: Dropbox Core API

RESTful Web Service Example Atom Publishing Protocol with WordPress Retrieve available collections (posts, categories ) GET http://yourblog.com/wp-app.

31 RESTful Web Service Example Atom Publishing Protocol with WordPress Retrieve available collections (posts, categories ) GET Retrieve all posts GET Retrieve specific post GET Add a new blog entry POST Update a blog entry PUT Delete a blog entry DELETE norrie@inf.ethz.ch 35

REST Application Programming Interfaces REST style not constrained to web services/applications Many NoSQL databases offer REST style APIs Example: CouchDB Open source document-oriented database

32 REST Application Programming Interfaces REST style not constrained to web services/applications Many NoSQL databases offer REST style APIs Example: CouchDB Open source document-oriented database written in Erlang Manages and stores a collection of JSON documents A single JSON document is exposed as a resource and has its own, unique URI Provides a RESTful API to access the database via HTTP requests Uses JSON exclusively for data interchange norrie@inf.ethz.ch 36

33 CouchDB Live Demo Demonstrates use of REST style architecture and JSON

34 Create your own REST API Most web frameworks can be used to create REST APIs Main difference: Serve JSON or XML instead of HTML Many platform-specific solutions JAX-RS: Java API for RESTful Web Services Version 1.1: Official API for Java Enterprise Edition 6 Version 2.0: Adds Client API, asynchronous processing, message filters etc. Reached release stage in May 2013 WCF Data Services (.NET) Exposes Entity Data Model objects via REST-style interface Mostly driven by relational databases Route-based solutions Map URL routes (HTTP method + URL matching pattern) to actions/responses norrie@inf.ethz.ch 38

35 Example: JAX-RS 1.1 Jersey Java Library: Reference Implementation of JAX-RS Based on Annotations of Model Classes import javax.ws.rs.get; import javax.ws.rs.produces; import javax.ws.rs.path; // The Java class will be hosted at the URI path public class HelloWorldResource { // The Java method will process HTTP GET // The Java method will produce content identified by the // MIME Media type "text/plain«@produces("text/plain") public String getclichedmessage() { // Return some cliched textual content return "Hello World"; } } Example taken from Oracle Documentation: norrie@inf.ethz.ch 39

36 Example: Sinatra (Ruby) Ruby-based, domain-specific language to create web applications get '/' do.. show something.. end post '/' do.. create something.. end get '/hello/:name' do # matches "GET /hello/foo" and "GET /hello/bar«# params[:name] is 'foo' or 'bar "Hello #{params[:name]}!«end get '/foo', :agent => /Songbird (\d\.\d)[\d\/]*?/ do "You're using Songbird version #{params[:agent][0]}«end Example from Sinatra Documentation: norrie@inf.ethz.ch 40

Libraries/Wrappers for Web APIs Challenge: Each service has its own, site-specific Web API Generic, universal REST libraries and JSON parsers exist, but require manual adaptation and mapping Current

37 Libraries/Wrappers for Web APIs Challenge: Each service has its own, site-specific Web API Generic, universal REST libraries and JSON parsers exist, but require manual adaptation and mapping Current solution: Different wrappers for different services for different programming languages Twitter Java: Twitter4j, java-twitter, JTwitter, TweetStream4J.Net: LINQ to Twitter, TweetSharp, Twitterizer Standardisation? Flickr Java: FlickrJ, Jickr.Net: Flickr.NET Ruby: flickraw, flickr.rb, rflickr

38 API Standardisation and Aggregation? My Web Application OpenSocial API OAuth OpenID

Open Social Common set of APIs to for web-based social network applications that span accross many websites Uses JavaScript APIs and REST/RPC protocols Based on Google s gadget architecture

39 Open Social Common set of APIs to for web-based social network applications that span accross many websites Uses JavaScript APIs and REST/RPC protocols Based on Google s gadget architecture Applications are XML documents with HTML/JavaScript embedded Originally conceived by Google, MySpace and other social networks, now mainly supported by IBM, Jive Software and SugarCRM for enterprise social platforms Illustrations created by Google, released under Creative Commons norrie@inf.ethz.ch 43

outside of social networks Communicate via REST/RPC APIs 2-legged OAuth authorization flow

40 Open Social: Architecture Social Mashup Runs inside of a social network Scales well Limited in terms of data storage and data processing Social Website / Social Mobile Application Runs outside of social networks Communicate via REST/RPC APIs 2-legged OAuth authorization flow Illustrations created by Google, released under Creative Commons norrie@inf.ethz.ch 44

41 Open Social: Architecture Social Application Runs inside of a social network, but uses an external server for processing and rendering of data Advanced functionality Scaling might be an issue 3-legged OAuth authorization flow norrie@inf.ethz.ch 45

very fine-grained access control (resource/time based) Credentials never leave the original service provider Implemented by Twitter (v1.

42 Open Standards for Authorization and Authentication OAuth 1.0a/2.0 (Authorization) Allows sharing of private resources across web sites To access the data, tokens are used instead of credentials Tokens allow very fine-grained access control (resource/time based) Credentials never leave the original service provider Implemented by Twitter (v1.0a), Facebook (v2.0), Google (v2.0), Microsoft OpenID (Authentication) Identify users accross web sites No central authority, based on mutual trust Several providers: Google, Yahoo!, LiveJournal, MySpace, Steam

43 OAuth Core 1.0a (RFC 5849) Illustration by courtesy of Idan Gazit, Oauth Documentation

44 OAuth 2.0 (RFC 6749) - Let s try again In RFC 6749, OAuth 2.0 defines four roles: Resource Owner: An entity capable of granting access to a protected resource When the resource owner is a person, it is referred to as an end-user Resource server: Server hosting the protected resources, capable of accepting and responding to protected resource requests using access tokens Client: An application making protected resource requests on behalf of the resource owner and with its authorization (e.g. a web browser) Authorization server: The server issuing access tokens to the client after successfully authenticating the resource owner and obtaining authorization norrie@inf.ethz.ch 48

45 OAuth 2.0 Protocol Flow (A)- Authorization Request -> Resource Owner <-(B)-- Authorization Grant (C)-- Authorization Grant --> Authorization Client Server <-(D)----- Access Token (E)----- Access Token > Resource Server <-(F)--- Protected Resource (A) The authorization request can also be made (preferably) indirectly via the authorization server as an intermediary Illustration by Dick Hardt, RFC

46 API-enabled web components Idea: Encapsulate web API functualities in HTML user interface components Buttons, Input Forms, Links Different names, same principle Social Plugins (Facebook), Social Gadgets (Google), Embedded Tweets (Twitter),... Advantages Can be embedded in any website Allow non-technical persons to «access» the web API

47 Example: Google+ Button Load JavaScript API library <script type="text/javascript«src=" </script> Place HTML markup to render button <g:plusone size = "tall"></g:plusone> Alternative: Load JavaScript asynchronous <script type="text/javascript"> (function() { var po = document.createelement('script'); po.type = 'text/javascript'; po.async = true; po.src = ' var s = document.getelementsbytagname('script')[0]; s.parentnode.insertbefore(po, s); })(); </script> norrie@inf.ethz.ch 51

Example: Facebook Comments Box Load JavaScript API library <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getelementsbytagname(s)[0]; if (d.getelementbyid(id)) return; js = d.

48 Example: Facebook Comments Box Load JavaScript API library <div id="fb-root"></div> <script>(function(d, s, id) { var js, fjs = d.getelementsbytagname(s)[0]; if (d.getelementbyid(id)) return; js = d.createelement(s); js.id = id; js.src = "//connect.facebook.net/en_us/all.js#xfbml=1&appid= "; fjs.parentnode.insertbefore(js, fjs); } (document, 'script', 'facebook-jssdk')); </script> Place HTML 5 markup to render button <div class="fb-comments data-href= data-num-posts="2" data-width="470"> </div> norrie@inf.ethz.ch 52

49 Open Graph Protocol Idea: Annotate web sites with meta data to turn them into «rich objects in a social graph» Allows third-party developers to integrate their Web resources into Facebook s Open Graph and perform «actions» on them (i.e. like) Developed by Facebook but specification is open Web pages that include Open Graph tags can be «consumed» by any provider (e.g. Google) Protocol Usage Implemented with <meta> tags embedded into the HTML code Adds structured information about a single web resource/page norrie@inf.ethz.ch 53

50 Open Graph Protocol Specification Four required properties (Open Graph Core): og:title The (visible) title of the web resource og:type Any type that is supported (i.e. actor, city, product etc.) og:image An image URL that represents the resource in the graph og:url Canonical URL and permanent ID of the resource Facebook-specific meta data (mandatory for Facebook Open Graph integration): fb:app_id App ID of the Facebook application representing the site fb:admins Facebook user IDs of administrators (deprecated) Additional attributes available to specify Location, contact information and video/audio attachments Type-specific attributes (example for og:type=music.song) music:album The album this song is from

Open Graph Example <html xmlns:og="http://ogp.me/ns#" xmlns:fb="http://www.facebook.

51 Open Graph Example <html xmlns:og=" xmlns:fb=" > <head> <title>the Rock (1996)</title> <meta property="og:title" content="the Rock" /> <meta property="og:type" content= video.movie" /> <meta property="og:url" content=" /> <meta property="og:image" content=" /> <meta property="og:site_name" content="imdb" /> <meta property="fb:app_id" content= APP_ID" />... </head>... </html>

52 OpenGraph Live Demo How to integrate your web page with Facebook s Open Graph norrie@inf.ethz.ch 56

53 Conclusion: Web APIs There is no single Web API standard REST style is a widely used architecture but does not specify anything in terms of implementation Several open standards emerged from company driven efforts OAuth by Blaine Cook (Twitter) OpenID by Brad Fitzpatrick (LiveJournal) OpenSocial developed by Google Open Graph Protocol initated by Facebook Integrating different web services and applications is a challenge! norrie@inf.ethz.ch 57

54 References REST Roy Thomas Fielding Architectural Styles and the Design of Network-based Software Architectures Dissertation, University of California, Irvine, 2000 Uniform Resource Identifier JSON Specification Open Web Standards OAuth 1.0 Protocol Open Graph Protocol OpenSocial Foundation

ReST 2000 Roy Fielding W3C

ReST 2000 Roy Fielding W3C Outline What is ReST? Constraints in ReST REST Architecture Components Features of ReST applications Example of requests in REST & SOAP Complex REST request REST Server response Real REST examples REST