Big Data 14. Graph Databases

Size: px

Start display at page:

Download "Big Data 14. Graph Databases"

Darren Williams
6 years ago
Views:

1 Ghislain Fourny ig Data 14. Graph Databases pinkyone / 123RF Stock Photo tovovan / 123RF Stock Photo 1

2 Why graph databases? 2

3 The NoSQL paradigms foo Triple stores bar foobar Key-value stores Column stores Document stores 3

4 Relational databases... 4

5 Relational databases... Entity Entity Relationship 5

6 Relational databases... have expensive Entity Entity joins! Relationship 6

7 Relational databases are not that efficient at relationships! 7

8 We already know how to partly solve this though 3NF 0NF 8

9 We already know how to partly solve this though 3NF... but it has its limits, 0NF too! 9

10 Traversals... 10

11 Traversals translate into multiple joins! 11

12 Reverse traversals... 12

13 Reverse traversals need even more indices! 13

14 Traversals... what if links would be more... "direct"? translate into multiple joins! 14

15 Index-free adjacency 15

16 Graphs 16

17 Graphs: ingredients Nodes Edges 17

18 Graphs: nodes 18

19 Graphs: edges 19

20 Graphs: directed graph 20

21 Graphs: undirected graph 21

22 Graph representation: adjacency list A C Node Edges A [ ] [ A, C ] C [ A ] 22

23 Graph representation: adjacency matrix A A C A C C

24 Graph representation: incidence matrix A Edges C Nodes A C

25 Labeled property graphs: ingredients Nodes Edges Properties Labels 25

26 Property graph 26

27 Properties Name: Einstein First name: Albert Profession: Physicist 27

28 Labeled graph 28

29 Labels on nodes 29

30 Names on relationships A A A A A A A 30

31 Labeled property graph 31

32 Node with properties and label Name: Einstein First name: Albert Profession: Physicist Person In Switzerland 32

33 Graph database 33

34 Graph databases: families Property Graph Triple stores (RDF) 34

35 Graph databases: native or not Source Target Name Alice ob knows Eve ob eavesdrop Eve Alice eavesdrop Native Graph Database Graph stored as RDMS, document store,... 35

36 RDF 36

37 Triple-based graph 37

38 RDF: one triple ETH Zürich Is located in Switzerland Subject Property Object 38

39 IRI

40 Literal Foo includes XML Schema types! 40

41 lank Node ETH Zürich Is built on ground Is subset of Switzerland 41

42 What can appear where? IRI Literal lank node Subject Property Object 42

43 Generalized Graphs IRI Literal lank node Subject Property Object 43

44 Syntax 44

45 RDF Formats RDF/XML Turtle JSON-LD RDFa N-Triples 45

46 RDF/XML <rdf:rdf xmlns:rdf=" xmlns:geo=" <rdf:description rdf:about=" <geo:islocatedin rdf:resource=" <geo:population> </geo:population> </rdf:description> </rdf:rdf> 46

47 RDF/XML: Subject <rdf:rdf xmlns:rdf=" xmlns:geo=" <rdf:description rdf:about=" <geo:islocatedin rdf:resource=" <geo:population> </geo:population> </rdf:description> </rdf:rdf> 47

RDF/XML: Property <rdf:rdf xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:geo="http://www.example.

48 RDF/XML: Property <rdf:rdf xmlns:rdf=" xmlns:geo=" <rdf:description rdf:about=" <geo:islocatedin rdf:resource=" <geo:population> </geo:population> </rdf:description> </rdf:rdf> 48

49 RDF/XML: Object <rdf:rdf xmlns:rdf=" xmlns:geo=" <rdf:description rdf:about=" <geo:islocatedin rdf:resource=" <geo:population> </geo:population> </rdf:description> </rdf:rdf> 49

50 RDF/XML <rdf:rdf xmlns:rdf=" xmlns:geo=" <rdf:description rdf:about=" <geo:islocatedin rdf:resource=" <geo:population> </geo:population> </rdf:description> </rdf:rdf> 50

51 RDF/XML <rdf:rdf xmlns:rdf=" xmlns:geo=" <rdf:description rdf:about=" <geo:islocatedin rdf:resource=" <rdf:type rdf:resource=" <geo:population> </geo:population> </rdf:description> </rdf:rdf> 51

52 RDF/XML <rdf:rdf xmlns:rdf=" xmlns:geo=" <rdf:description rdf:about=" <geo:islocatedin rdf:resource=" <rdf:type rdf:resource=" <geo:population> </geo:population> </rdf:description> </rdf:rdf> 52

53 RDF/XML <rdf:rdf xmlns:rdf=" xmlns:geo=" <geo:school rdf:about=" <geo:islocatedin rdf:resource=" <geo:population> </geo:population> </geo:school> </rdf:rdf> 53

54 JSON-LD { } "@context": { "rdf": " "geo": " }, "@id" : " "rdf:type": "geo:school", "geo:islocatedin": " "geo:population" :

55 geo: countries: eth: < eth:self geo:islocated countries:switzerland. eth:self geo:population

56 geo: countries: eth: < eth:self geo:islocated countries:switzerland ; eth:self geo:population

57 geo: countries: eth: < eth:self geo:islocated countries:switzerland, eth:self geo:islocated countries:europe ; eth:self geo:population

58 Querying 58

59 Querying paradigms? Classical declarative querying Query by example 59

60 Two languages Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Cypher SPARQL 60

61 Two languages Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Cypher SPARQL 61

62 Querying labeled property graphs by example 62

63 Querying labeled property graphs by example 63

64 Querying labeled property graphs by example 64

65 Querying labeled property graphs by example 65

66 Querying labeled property graphs by example 66

67 Querying labeled property graphs by example A A A A A 67

68 Cypher pattern A A gamma A A beta alpha (alpha)-[:a]->(beta)-[:]->(gamma) 68

69 Cypher pattern: anchoring a label A A gamma A yellowbeta A alpha (alpha) -[:A]->(beta:yellow) -[:]->(gamma) 69

70 Cypher pattern: filtering a property A A gamma A A beta alpha name: Einstein (alpha {name: 'Einstein' }) -[:A]->(beta) -[:]->(gamma) 70

71 Cypher pattern: anchoring and filtering name: ETH A A blue gamma A A beta alpha (alpha) -[:A]->(beta) -[:]->(gamma: blue {name: 'ETH'}) 71

72 Cypher pattern: right to left A gamma A delta A A beta alpha (alpha) -[:A]->(beta) -[:]->(gamma) <-[:]-(delta) 72

73 Cypher pattern: variable repetition A gamma A delta A A beta alpha (alpha) -[:A]->(beta) -[:]->(gamma) <-[:]-(delta) -[:]->(alpha) 73

74 Cypher pattern: variable length path A A A alpha (alpha) -[*1..4]->(beta) A beta 74

75 Cypher pattern: MATCH clause MATCH (alpha {name: 'Einstein' })-[:A]->(beta)-[:]->(gamma) 75

76 Cypher pattern: MATCH clause MATCH (alpha {name: 'Einstein' })-[:A]->(beta)-[:]->(gamma) RETURN gamma 76

77 Cypher pattern: WHERE clause MATCH (alpha {name: 'Einstein' })-[:A]->(beta)-[:]->(gamma) RETURN gamma MATCH (alpha)-[:a]->(beta)-[:]->(gamma) WHERE alpha.name = 'Einstein' RETURN gamma 77

78 Cypher pattern: CREATE clause CREATE (einstein:scientist {name: 'Einstein', first: 'Albert' }), (eth:university {name: 'ETH Zurich' }), (einstein)-[:visited]->(eth) 78

79 Other clauses WITH MERGE START SET DELETE UNION FOREACH MERGE 79

80 Two languages Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Switzerland Is located in ETH Zürich Cypher SPARQL 80

81 Querying RDF: SPARQL PREFIX geo: < PREFIX countries: < SELECT?s WHERE {?s geo:islocatedin countries:switzerland } 81

82 SPARQL PREFIX geo: < PREFIX countries: < SELECT?s WHERE {?s geo:islocatedin countries:switzerland } 82

83 SPARQL PREFIX geo: < PREFIX countries: < SELECT?s WHERE {?s geo:islocatedin countries:switzerland } 83

84 SPARQL PREFIX geo: < PREFIX countries: < SELECT?s WHERE {?s geo:islocatedin countries:switzerland.?s :deliversdiplom :bachelor. } 84

85 SPARQL PREFIX geo: < PREFIX countries: < SELECT?s WHERE {?s geo:islocatedin?c.?c geo:isincontinent geo:america. } 85

86 SPARQL PREFIX geo: < PREFIX countries: < SELECT?s WHERE {?s geo:islocatedin countries:switzerland.?s :deliversdiplom :bachelor } LIMIT 10 86

87 SPARQL PREFIX geo: < PREFIX countries: < SELECT?s?name WHERE {?s geo:islocatedin countries:switzerland.?s :deliversdiplom :bachelor.?s :hasname?name. } ORDER Y?name LIMIT 10 87

88 Architecture (Neo4j) 88

89 No sharding 89

90 Document stores don't like joins Graph databases don't like shards 90

91 Why? Fast traversal 91

92 Master-slave architecture Master Slave Slave Slave Slave Slave Slave 92

93 Data replication Master Slave Slave Slave 93

94 Data replication Master Synchronization Slave Slave Slave 94

95 Data replication (full) Master Synchronization Slave Slave Slave 95

96 Read scale-up Slave 96

97 Writes or Write to the master Write to a slave 97

98 Caching and pages Index-free adjacency Fixed-size records 98

99 Label storage Person Jedi Geek Person Jedi Geek 99

100 Properties storage name: Einstein first-name: Albert name Einstein first-name Albert 100

101 Relationship storage A A 101

102 Relationship storage A A 102

103 Relationship storage A A 103

104 Relationship storage A 104

105 Relationship storage A 105

106 Relationship storage A 106

107 Relationship storage A 107

108 Relationship storage A 108

109 Relationship storage A 109

110 Relationship storage A 110

111 Relationship storage A 111

112 Relationship storage A 112

113 Relationship storage A A 113

114 Relationship storage A A 114

115 Relationship storage A A 115

116 Relationship storage A s-previous s-next A Source t-previous t-next Target 116

117 Typical sizes Node: Relationship: 9 bytes 33 bytes Relationship name: 5 bytes Property: 33 bytes 117

118 Semantics 118

119 RDF has no semantics Schwiiz Chuchichäschtli Schoggi 119

120 RDF Schema Class Property 120

121 Classes rdfs:resource rdfs:class rdf:property rdfs:literal rdfs:datatype rdf:html rdf:xmlliteral 121

122 Properties On any resources rdf:type rdfs:label rdfs:comment On properties On classes rdfs:range rdfs:domain rdfs:subpropertyof rdfs:subclassof 122

123 Self-awareness rdfs:resource rdf:type rdfs:resource 123

124 Self-awareness rdfs:class rdf:subclassof rdfs:resource 124

125 Self-awareness rdf:type rdf:range rdfs:class 125

126 Self-awareness rdf:subclassof rdf:type rdfs:property 126

127 Simple Entailment (RDF semantics) E I(E)=true I 127

128 OWL 128

129 OWL (In principle) standalone (Much) More powerful than RDF(S) 129

130 OWL <xml/> 130

131 OWL and description logic / AI 131

132 Entailment (and Syllogisms) Major Minor All men are mortal. Socrates is a man. Conclusion Therefore, Socrates is mortal. 132

133 Trees

134 ... and Graphs 134

Semantic Web In Depth: Resource Description Framework. Dr Nicholas Gibbins 32/4037

Semantic Web In Depth: Resource Description Framework Dr Nicholas Gibbins 32/4037 nmg@ecs.soton.ac.uk RDF syntax(es) RDF/XML is the standard syntax Supported by almost all tools RDF/N3 (Notation3) is also