COMP 150-IDS: Internet Scale Distributed Systems (Spring 2017) URIs and RFC 3986

Size: px

Start display at page:

Download "COMP 150-IDS: Internet Scale Distributed Systems (Spring 2017) URIs and RFC 3986"

Daniella Black
6 years ago
Views:

1 COMP 150-IDS: Internet Scale Distributed Systems (Spring 2017) URIs and RFC 3986 Noah Mendelsohn Tufts University Web: Copyright: 2012, 2013, 2015, 2016 & 2017 Noah Mendelsohn

2 Goals What is named when we use the Web Learn the detailed design of URIs See how the naming principles we ve explored are reflected in Web architecture and URIs Learn to read RFCs and to study the art of writing specifications Understand why grammars are important 2

3 Review: Naming Questions 3

4 Some characteristics of names Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too few names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

5 Review Web Architecture Basics

6 Architecting a universal Web Identification: URIs Interaction: HTTP Data formats: HTML, JPEG, etc.

7 What Happens When We Browse a Web Page? Consider: What are all the things that are named in the interaction between browser and Web Server?

8 The user clicks on a link URI is

9 The http scheme tells client to send HTTP GET msg URI is HTTP GET

10 The client sends an HTTP GET URI is HTTP GET demo1/test.html Host: webarch.noahdemo.com GET /demo1/test.html HTTP/1.0 Host: webarch.noahdemo.com User-Agent: Noahs Demo HttpClient v1.0 Accept: */* Accept-language: en-us

11 HTTP/ OK Date: Tue, 28 Aug :49:33 GMT Server: Apache Transfer-Encoding: HTTP GET chunked Content-Type: text/html The server sends an HTTP Response HTTP Status Code 200 Means Success! <html> <head> <title>demo #1</title> </head> <body> <h1>a very simple Web page</h1> </body> </html> Host: webarch.noahdemo.com demo1/test.html HTTP RESPONSE

12 HTTP/ OK Date: Tue, 28 Aug :49:33 GMT Server: Apache Transfer-Encoding: HTTP GET chunked Content-Type: text/html The server sends an HTTP Response <html> <head> <title>demo #1</title> </head> <body> <h1>a very simple Web page</h1> </body> </html> Host: webarch.noahdemo.com demo1/test.html HTTP RESPONSE The representation returned is an HTML document

13 Architecting a universal Web Identification: URIs Interaction: HTTP Data formats: HTML, JPEG, etc.

14 Assign URIs for all Resources A resource is something that has information (e.g. a Web page) If a resource doesn t have a URI, you can t link to it it s not part of the Web.

15 The Structure of URIs 16

16 A simple URI

17 A simple URI

18 A simple URI Scheme

19 Schemes Scheme Schemes let us name different kinds of things, accessed in different ways.

20 A simple URI Authority Authority: who controls allocation of this name?

21 A simple URI // Fixed in grammar to indicate authority follows

22 A simple URI // Fixed in grammar to indicate authority follows Tim BL was asked if he had any regrets about the Web Mr. Berners-Lee smiled and admitted he might make one change a small one. He would get rid of the double slash // after the http: in Web addresses. The double slash, though a programming convention at the time, turned out to not be really necessary, Mr. Berners-Lee explained. Look at all the paper and trees, he said, that could have been saved if people had not had to write or type out those slashes on paper over the years not to mention the human labor and time spent typing those two keystrokes countless millions of times in browser address boxes. (Today s browsers, of course, automatically fill in the preamble when a user types a Web address.) From:

23 A simple URI Path Path: provides for hierarchical naming also supports../xxx relative syntax

24 A simple URI Path Path: provides for hierarchical naming maps well to hierarchical information systems

25 A more complex URI

26 A more complex URI Query component The query is part of the URI... However, in many cases, all URIs with a common path are processed by the same server-side code Also HTML forms are useful for filling in the query components

27 Fragments Fragments identify parts of documents Fragment interpretation depends on the media type of the returned representation (text/html) this is useful but tricky and causes a variety of problems.

28 Characteristics of URIs 29

29 Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

30 Some characteristics of URIs Both supported Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

31 Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Depends on scheme Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

32 Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Yes URIs on the side of a bus is an important goal Opaque vs. data-carrying? but some URIs are complex Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

33 Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Allowed but not required Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

34 Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) With most schemes, absolute URIs are global Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

35 Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) FILE: scheme is not global! Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

36 Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) NO!! Status code 404 is key to Web scalability Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

37 Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Some aliases required e.g.: http vs. HTTP worst cases depend on users Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

38 Depends on scheme and user see Metadata in URI finding Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

39 URIs are the structuring mechanism for the Web as a whole Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

40 Designed to allow mappings to hierarchical systems Some characteristics of URIs Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

41 Some characteristics of URIs Decentralized allocation except: Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

42 Some characteristics of URIs scheme names centrally registered with IANA Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

43 Some characteristics of URIs For http and mailto schemes: central Domain Name (DNS) registration Absolute vs. relative required for authority Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

44 Some characteristics of URIs Yes. E.g.: ASCII-only, spaces and some punctuation must be %encoded Absolute vs. relative Address (locator)? Human readable? Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

45 Some characteristics of URIs Absolute vs. relative URIs are silent on this but HTTP Address (locator)? redirection provides for indirect Human readable? identification Short/convenient? Global (context independent)? Ensures referent exists? Aliases? (too many names) Opaque vs. data-carrying? Reflect structure of system? Supports navigation: e.g...? Who can generate them? Constraints from environment E.g. no - in C/C++ variable names Indirect identification allowed?

46 Grammars 47

47 What are formal grammars? Grammars are formal languages for specifying other languages A grammar allows you to: Always: determine whether a given string is in the specified language Often: associate structures in the grammar with parts of the string The Chomsky hierarchy: Different grammars have different expressive power Regular expressions recognize regular languages (ab*) a, ab, abb, abbb Context-free grammars are more powerful: typically used for programming languages The ABNF used in RFC s is a context-free grammar Context-free grammars can be recognized (parsed) by a finite-state pushdown automaton Tutorial at:

48 Why use formal grammars for specifying languages? Precise and rigorous Less ambiguous than an explanation in English Membership of a string in a language can be checked automatically Tools to process the language can often be constructed automatically from the grammar

49 ABNF: the grammar for IETF RFCs ABNF example from RFC 3986: URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ] ABNF is itself specified in RFC 2234 ABNF is convenient to use in fixed-font specification documents like RFCs

50 Summary 51

51 Summary The structure and interpretation of URIs is set out in RFC 3986 URIs embody many of the principles we have studied Formal grammars are powerful tools for specifying names The design decisions embodied in URIs are keys to the success of the Web!!

Naming System Design Tradeoffs

COMP 117: Internet Scale Distributed Systems (Spring 2018) Naming System Design Tradeoffs Noah Mendelsohn Tufts University Email: noah@cs.tufts.edu Web: http://www.cs.tufts.edu/~noah Copyright: 2012, 2013,