Ghislain Fourny. Big Data 7. Syntax

Size: px

Start display at page:

Download "Ghislain Fourny. Big Data 7. Syntax"

Bartholomew Woods
6 years ago
Views:

1 Ghislain Fourny Big Data 7. Syntax

2 Introduction 2

3 The stack: Syntax Text CSV XML JSON RDF/XML Turtle XBRL Syntax 3

Data Shapes Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam vel erat nec dui aliquet vulputate sed quis nulla. Donec eget ultricies magna, eu dignissim elit.

Aenean eu efficitur orci. Aenean ac posuere tellus. Ut id commodo turpis. Praesent nec libero metus. Praesent at turpis placerat, congue ipsum eget, scelerisque justo.

Vestibulum pharetra justo massa, ac venenatis mi condimentum nec. Proin viverra tortor non orci suscipit rutrum. Phasellus sit amet euismod diam.

4 Data Shapes Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam vel erat nec dui aliquet vulputate sed quis nulla. Donec eget ultricies magna, eu dignissim elit. Nullam sed urna nec nisl rhoncus ullamcorper placerat et enim. Integer varius ornare libero quis consequat. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean eu efficitur orci. Aenean ac posuere tellus. Ut id commodo turpis. Praesent nec libero metus. Praesent at turpis placerat, congue ipsum eget, scelerisque justo. Ut volutpat, massa ac lacinia cursus, nisl dui volutpat arcu, quis interdum sapien turpis in tellus. Suspendisse potenti. Vestibulum pharetra justo massa, ac venenatis mi condimentum nec. Proin viverra tortor non orci suscipit rutrum. Phasellus sit amet euismod diam. Nullam convallis nunc sit amet diam suscipit dapibus. Integer porta hendrerit nunc. Quisque pharetra congue porta. Suspendisse vestibulum sed mi in euismod. Etiam a purus suscipit, accumsan nibh vel, posuere ipsum. Nulla nec tempor nibh, id venenatis lectus. Duis lobortis id urna eget tincidunt. 4

5 Trees... 5

6 ... and Graphs 6

7 2000s: The NoSQL Era foo Triple stores bar foobar Key-value stores Column stores Document stores 7

8 2000s: The NoSQL Era foo Triple stores bar foobar Key-value stores Column stores Document stores 8

9 Semi-Structured Documents gajus / 123RF Stock Photo 9

10 Semi-Structured Documents Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Structured Unstructured 10 10

11 Semi-Structured Documents e:f d This is a c b. text Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Structured Semi-structured Unstructured 11 11

12 Standards 12 12

13 For whom? 13 13

14 Syntax 14

15 Well-formedness 15 15

16 Well-formedness One syntax = one language 16 16

17 Well-formedness One syntax = one language D L 17 17

18 Well-formedness One syntax = one language D L D is well-formed D is not well-formed 18 18

19 XML 19 <?xml&version="1.0"?>& <country&code="ch">& &&<name>switzerland</name>& &&<population> </population>& &&<currency&code="chf">swiss&franc</currency>& &&<cities>& &&&&<city>zurich</city>& &&&&<city>geneva></city>& &&&&<city>bern&<!ii&the&federal&city&ii></city>& &&</cities>& &&<description>& &&&&We&produce&<b>very</b>&good&chocolate.& &&</description>& </country>& & 19

20 JSON { } "code": "CH", "name": "Switzerland", "population": , "currency": { "name": "Swiss Franc", "code": "CHF" }, "confederation": true, "president" : "Johann Schneider-Amman", "capital": null, "cities": [ "Zurich", "Geneva", "Bern" ], "description": "We produce very good chocolate." 20 20

21 HTML 21 <!DOCTYPE*html>* <html>* **<head>* ****<title>country</title>* **</head>* **<body>* ****<h1*class="title">switzerland</h1>* ****<div>population:* <br>* ******Currency:*Swiss*Franc*(CHF)</div>* ****<h2>cities</h2>* ****<ul>* ******<li>zurich</li>* ******<li>geneva></li>* ******<li>bern <!SS*the*Federal*City*SS></li>* ****</ul>* **</body>* </html>*! 21

22 XML Robert Eastman / 123 Stock Photo 22

23 XML: Element <foo>[more XML]</foo> 23

24 XML: Element <foo>[more XML]</foo> <bar/> = <bar></bar> 24

25 XML: Element <foo>[more XML]</foo> opening tag closing tag <bar/> = <bar></bar> 25 empty tag

26 XML: Attribute <a attr="value"/> 26 26

27 XML: Text <a>this is text</a> 27 27

28 XML: Comment  28 28

29 XML: Processing Instruction <?myapp do whatever?> <?xml version="1.0"?> Charles Goldfarb "In a perfect world, processing instructions would not be necessary. However, as you might have noticed, 29 the world is not perfect." 29

30 (XML: Text declaration) <?xml version="1.0" encoding="utf-8"?> 30 30

31 What Appears Where? Top-Level Between Element Tags Inside Opening Element Tag Elements once Attributes Text Comments Processing Instructions 31 31

32 XML: Well-formedness <a foo="bar" foo="bar2"/> 32 32

33 XML: Well-formedness <a foo="bar" foo="bar2"/> <a foo="bar" bar="foo"/> 33 33

34 XML: Well-formedness <a><b></a></b> 34 34

35 XML: Well-formedness <a><b></a></b> <a><b></b></a> 35 35

36 XML: Well-formedness <a>1 < 2</a> 36 36

37 XML: Well-formedness <a>1 < 2</a> <a>1 < 2</a> 37 37

38 XML: Entity References <?xml version "1.0"?> <document> 2 < 3 </document> 38

39 XML: Entity References < < 39

40 XML: Entity References > > 40

41 XML: Entity References ' ' 41

42 XML: Entity References " " 42

43 XML: Entity References & & 43

44 XML: Double escaping <?xml version "1.0"?> <document> 2 < 3 </document> 44

45 XML: Character References (hex) <?xml version "1.0"?> <document> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do π eiusmod tempor incididunt ut labore et dolore magna aliqua. </document> π 45

46 XML: Character References (dec) <?xml version "1.0"?> <document> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do π eiusmod tempor incididunt ut labore et dolore magna aliqua. </document> π 46

47 XML: Well-formedness <a attr="a "quote""/> 47 47

48 XML: Well-formedness <a attr="a "quote""/> <a attr="a "quote""/> 48 48

49 XML: Well-formedness  49 49

50 XML: CDATA sections <?xml version "1.0"?> <document> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do <![CDATA[ &<<>>"' ]]> eiusmod tempor incididunt ut labore et dolore magna aliqua. </document> 50

51 XML: CDATA sections -- wellformedness <?xml version "1.0"?> <document> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do <![CDATA[ &<<>>"' ]]> ]]> eiusmod tempor incididunt ut labore et dolore magna aliqua. </document> 51

52 XML: Document Type <?xml version="1.0"?> <!DOCTYPE document> <document> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. </document> 52 52

53 XML: Document Type <?xml version "1.0"?> <!DOCTYPE document [ (internal subset) ]> <document> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. </document> 53 53

54 XML: Entity Declarations <?xml version "1.0"?> <!DOCTYPE document [ <!ENTITY myownentity "foobar"> ]> <document> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do &myownentity; 54 foobar eiusmod tempor incididunt ut labore et dolore magna aliqua. </document> 54

55 XML Names <1234/> <a<b/> <xml/> 55 55

56 XML Names <1234/> <a<b/> <xml/> <foo1234/> <_bar/> 56 56

57 Characters allowed ":" [A-Z] "_" [a-z] [#xc0-#xd6] [#xd8-#xf6] [#xf8-#x2ff] [#x370-#x37d] [#x37f-#x1fff] [#x200c-#x200d] [#x2070-#x218f] [#x2c00-#x2fef] [#x3001-#xd7ff] [#xf900-#xfdcf] [#xfdf0-#xfffd] [#x10000-#xeffff] Anywhere "-" "." [0-9] #xb7 [#x0300-#x036f] [#x203f-#x2040] Not starting 57 57

58 XML: Namespaces XML with Namespaces XML 58 58

59 XML Names Namespace + Local name entity Expanded name {

60 XML Names Namespace + Local name entity Expanded name {

61 Life without QNames (Clark Notation) <{ <{ <{ <{ x </{ <{ <{ <{ 2 </{ </{ </{ </{

62 Life with Prefixes and QNames <m:math xmlns:m=" <m:apply> <m:eq/> <m:ci> x </m:ci> <m:apply> <m:root/> <m:cn> 2 </m:cn> </m:apply> </m:apply> </m:math> 62 62

63 Life with Prefixes and QNames <m:math xmlns:m=" <m:apply> <m:eq/> <m:ci> x </m:ci> <m:apply> <m:root/> The namespace is represented by a prefix. <m:cn> 2 </m:cn> </m:apply> </m:apply> </m:math> 63 63

64 Life with Prefixes and QNames <m:math xmlns:m=" <m:apply> <m:eq/> <m:ci> x </m:ci> <m:apply> <m:root/> <m:cn> 2 </m:cn> </m:apply> </m:apply> </m:math> Prefix m is bound to a namespace using an xmlns:m attribute

Life with Prefixes and QNames <m:math xmlns:m="http://www.w3.

65 Life with Prefixes and QNames <m:math xmlns:m=" <m:apply> <m:eq/> <m:ci> x </m:ci> <m:apply> Prefix: m <m:root/> Namespace: <m:cn> Local name: apply 2 </m:cn> </m:apply> </m:apply> </m:math> QName 65 65

66 Default Namespace <math xmlns =" <apply> <eq/> <ci> x </ci> <apply> <root/> <cn> 2 </cn> </apply> </apply> </math> 66 66

67 Default Namespace <math xmlns =" <apply> <eq/> <ci> x </ci> <apply> <root/> <cn> No Prefix: Default Namespace 2 </cn> </apply> </apply> </math> 67 67

Binding scopes 68 <m:math xmlns:m="http://www.w3.

68 Binding scopes 68 <m:math xmlns:m=" <m:apply> <m:eq/> <m:ci> x </m:ci> <m:apply> <m:root/> <m:cn> 2 </m:cn> </m:apply> </m:apply> </m:math> Lifetime of the prefix binding 68

69 (Not) Well-Formed XML <?xml version="1.0" encoding="utf-16"?> <movies> <movie id= > <title>love Story</title> <title></title> <year>1980</year> <_director name='coppola'></_director> <comment text= Five start text= Average /> <xml>introduce XML content</xml> <newcomment text="an <important> text">oscar</newcomment> <comment lang=de> 1980 Warner Bros.</comment>  </Movie> </movies> 69 69

70 (Not) Well-Formed XML <?xml version="1.0" encoding="utf-16"?> <movies> <movie id= > <title>love Story</title> <title></title> <year>1980</year> <_director name='coppola'></_director> <comment text= Five start text= Average /> <xml>introduce XML content</xml> <newcomment text="an <important> text">oscar</newcomment> <comment lang=de> 1980 Warner Bros.</comment>  </Movie> </movies> 70 70

71 Well-Formedness: How To Tell? An editor (oxygen,...) will tell you

72 Well Formed XML <?xml version="1.0" encoding="utf-16"?> <!DOCTYPE movies [ <!ENTITY copy " "> ]> <movies> <Movie id="56225"> <title>love Story</title> <title></title> <year>1980</year> <_director name='coppola'></_director> <comment text="five start"/> <comment text="average"/> <newcomment text="an <important> text">oscar</newcomment> <comment lang="de"> 1980 Warner Bros.</comment>  </Movie> </movies>! 72 72

73 Which QNames are in which Namespaces? 73 <?xml version="1.0"?> <!DOCTYPE eth> <eth xmlns=" xmlns:xmldb=" date=" " xmldb:date=" "> <date> </date> <president number="1">empty</president> <Rektor>Name 2</Rektor> </eth>! 73

74 Which QNames are in which Namespaces? 74 <?xml version="1.0"?> <!DOCTYPE eth> <eth xmlns=" xmlns:xmldb=" date=" " xmldb:date=" "> <date> </date> <president number="1">empty</president> <Rektor>Name 2</Rektor> </eth>! 74

75 XML: Not covered Notations Unparsed entities Parameter entities 75 75

76 JSON 76

77 JSON: String "foo" "foo\nbar\u005f" 77

78 JSON: Number E+5 78

79 JSON: Boolean true false 79

80 JSON: Null null 80

81 JSON: Array [ , true, "This is a string", { "foo" : false }, null 81 ]

82 JSON: Object { foo: , bar: true, str: "This is a string", obj: { "school" : "ETH"}, Q: null } 82

83 JSON: Well-formedness { "foo" : "bar", "foo" : "bar2" } 83 83

84 JSON: Well-formedness { "foo" : "bar", "foo" : "bar2" } { "foo" : "bar", "bar" : "foo" } (SHOULD) 84 84

85 JSON: Well-formedness { [ 1 ] : "bar", 2 : "bar2" } 85 85

86 JSON: Well-formedness { [ 1 ] : "bar", 2 : "bar2" } { "1" : "bar", "2" : "foo" } 86 86

87 JSON: Well-formedness { foo: "bar", bar: "bar2" } 87 87

88 JSON: Well-formedness { foo: "bar", bar: "bar2" } { "foo" : "bar", "bar" : "foo" } 88 88

89 HTML 89

90 XHTML syntax HTML syntax 90 90

91 HTML Syntax <!DOCTYPE html> <html> <head> <title>untitled</title> </head> <body> Dear jane <br> <p>you are invited at the weekly meeting</p> <p>yours sincerely, <br> John</p> </body> </html>! 91 91

92 HTML Elements: Void <br> 92 92

93 HTML Elements: Raw text <style> body { color: black; background: white; } em { font-style: normal; color: red; } </style> 93 93

94 HTML Elements: Escapable raw text <title> This is a "title" </title> 94 94

95 HTML Elements: Foreign 95 <math xmlns=" <apply> <eq/> <ci> x </ci> <apply> <root/> <cn> 2 </cn> </apply> </apply> </math> 95

96 HTML Elements: Normal <ol> <li>ett</li> <li>två</li> <li>tre</li> </ol> 96 96

97 XHTML Syntax <?xml version "1.0"?> <!DOCTYPE html> <html xmlns=" <head> <title>untitled</title> </head> <body> Dear jane <br/> <p>you are invited at the weekly meeting</p> <p>yours sincerely, <br/> John</p> </body> </html>!

98 YAML 98

99 YAML 99 %YAML&1.2& ***& Country:& &&code:&'ch'& &&name:&'switzerland'& &&population:& & &&currency:& &&&&name:&'swiss&franc'& &&&&code:&'chf'& &&confederation:&true& &&president&:&'didier&burkhalter'& &&capital:&null& &&cities:& &&&&*&'Zurich'& &&&&*&'Geneva'& &&&&*&'Bern'& &&description:&'we&produce&very&good&chocolate.'&! 99

Ghislain Fourny. Big Data 2. Lessons learnt from the past

Ghislain Fourny. Big Data 2. Lessons learnt from the past Ghislain Fourny Big Data 2. Lessons learnt from the past Mr. Databases: Edgar Codd Wikipedia Data Independence (Edgar Codd) Logical data model Lorem Ipsum Dolor sit amet Physical storage Consectetur Adipiscing