Graph Database. Relation

Transcription

1 Graph Distribution

2 Graph Database SRC Relation DEST

3 Graph Database Use cases: Fraud detection Recommendation engine Social networks...

4 RedisGraph Property graph Labeled entities Schema less Cypher query language Aggregations, Arithmetic expressions, Sort... Tabular resultset

5 Structure

6 Tables Person Visit Country Name Age Height SRC DEST Name Population Roi Israel 8.5M Hila Japan 127M Shany Italy 60M Amit

7 Documents ID: 1, Name: Roi, Age: 33, ID: 6, Name: Japan, Population: 127M Height: 187, Visited: [6]

8 Graph structure 101

9 Adjacency list

10 Adjacency matrix Node i is connected to node j If A[i,j] = 1

11 Hexastore S Subject SPO OSP SOP PSO OPS POS P Predicate O Object 6

12 Graph structure Hexastore Triplets SPO:Michael:Boss:Jim SOP:Michael:Jim:Boss OPS:Jim:Boss:Michael OSP:Jim:Michael:Boss PSO:Boss:Michael:Jim POS:Boss:Jim:Michael Michael S Boss P Jim O

13 Node property set Entities - Key value store. Person node with attributes: { name : Bruce Buffer, age : 60, gender : male }

14 Problem 2 billion users 338 average friends for user 676 billion edges 152 terabytes ~= 1024*32 bytes per user + 64 * 2 bytes per edge

15

16 Partitioning

17 Entities distribution Property set 1 Property set 2 Graph index

18 Query Find friends of mine who ve visited places I ve been to and are older than me. Match (ME:person)-[friend]->(F:person)-[visited]->(C:country)<-[visit]-(ME) WHERE ME.ID = 33 AND F.age > ME.age RETURN F.name, C.name

19 Graph traversal (ME:person) ME.ID = 33 Graph index

20 Graph traversal (ME:person)-[friend]->(F:person) Graph index

21 Graph traversal (F:person)-[visited]->(C:country) Graph index

22 Graph traversal (C:country)<-[visit]-(ME) Graph index

23 Resultset Friend ID Friend name Country ID Country name 70? 25? 92? 55? 56? 4?

24 Query WHERE F.age > ME.age RETURN F.name, C.name NETWORK! Fetch age for ID 33 Index Entities

25 Query example continued WHERE F.age > ME.age RETURN F.name, C.name NETWORK! Fetch name of every entity in (IDs) Entity s age > 29 Index Entities

26 Resultset Friend ID Friend name Country ID Country name 70 Noam 25 Japan

27 Index distribution Friend relation Visit relation Graph index

28 Query Find all posts liked by friends of friends of mine, written by author X. MATCH (ME:person)-[friend]->(:person)-[friend]->(F:person)-[like]->(post)<-[author]-(A:author) WHERE ME.ID=46 AND A.ID=71070 RETURN A.name, F.name

29 Query 1. Node X contains FRIEND relations. 2. Seek to my ID in Node X (1 RPC). Retrieve a list of friend uids. 3. Do multiple seeks for each of the friend uids, to generate a list of friends of friends uids. result set 1 Friend Index (ME:person)-[friend]->(:person)-[friend]->(F:person) Query executor

30 Resultset 1 Friends of friends Friend ID Friend name 70? 92? 56?

31 Query 1. Node Y contains posting list for predicate LIKE. 2. Ship result set 1 to Node Y (1 RPC), and do seeks to generate a list of all posts liked by result set 1. result set 2 Like Index (F:person)-[like]->(post) Resultset 1 Query executor

32 Resultset 2 Liked posts Friend ID Friend name Post ID 70? ? ? ? 12 56? 5356

33 Query Node Z contains relations for predicate AUTHOR. Ship result set 2 to Node Z (1 RPC). Seek to author X, and generate a list of posts authored by X. result set 3 Author Index (post)<-[author]-(a:author) Resultset 2 Query executor

34 Resultset 4 Intersected resultset 2 and 3 Friend ID Friend name Post ID Author ID Author name 70? ? 92? ?

35 Query Node N contains names for all uids, ship result set 4 to Node N (1 RPC), and convert uids to names by doing multiple seeks. Author Index RETURN A.name, F.name Resultset 4 Query executor

36 Resultset 4 Intersected resultset 2 and 3 Friend ID Friend name Post ID Author ID Author name 70 Ailon Omri 92 Boaz Omri

37 RedisGraph Not distributed, Yet, Work in progress: Compact distributed index Concurrent fast independent traversals

38 @roilipman (you)-[ask]->(question)

39 Solutions JanusGraph successor of Titan Relays on a storage backend e.g. Casandar. Provides a graph interface on top of a table. Delegates storing, replicating, distributing and persisting a graph to the underline storage backend. Takes a mature application from a similar domain and introduce a new data type API on top of existing data structure. (not optimal)

40 Solutions DGraph Uses the concept of RDF NQuad to represents connections and badger as its key value store. Both the graph index and the entities are distributed.

41 Solutions Arangodb From my understanding this multi model database uses documents to represent all three data types: Documents, key value store and graph. Not sure about how it distributes its data but it s using RAFT to ensure consistency It is ACID.