Data Modeling with Neo4j. Stefan Armbruster, Neo Technology (slides from Michael Hunger)

Transcription

1 Data Modeling with Neo4j Stefan Armbruster, Neo Technology (slides from Michael Hunger) 1 1

2 33

3 is a 44

4 NOSQL 55

5 Graph Database 66

6 A graph database... NO: not for charts & diagrams, or vector artwork YES: for storing data that is structured as a graph remember linked lists, trees? graphs are the general-purpose data structure A relational database may tell you the average age of everyone in this place, but a graph database will tell you who is most likely to buy you a beer. 77

7 You know relational foo foo_bar bar 88

8 now consider relationships... 99

9 We're talking about a Property Graph Properties (each a key+value) + Indexes (for easy look-ups) 11 00

10 Aggregate vs. Connected Data-Model

11 NOSQL Databases 11 22

12 Aggregate Oriented Model There is a significant downside - the whole approach works really well when data access is aligned with the aggregates, but what if you want to look at the data in a different way? Order entry naturally stores orders as aggregates, but analyzing product sales cuts across the aggregate structure. The advantage of not using an aggregate structure in the database is that it allows you to slice and dice your data different ways for different audiences. This is why aggregate-oriented stores talk so much about map-reduce. Martin Fowler

13 Connected Data Model The connected data model is based on fine grained elements that are richly connected, the emphasis is on extracting many dimensions and attributes as elements. Connections are cheap and can be used not only for the domain-level relationships but also for additional structures that allow efficient access for different use-cases. The fine grained model requires a external scope for mutating operations that ensures Atomicity, Consistency, Isolation and Durability - ACID also known as Transactions. Michael Hunger

14 Data Modeling

15 Why Data Modeling What is modeling? Aren t we schema free? How does it work in a graph? Where should modeling happen? DB or Application 11 66

16 Data Models

17 Model mis-match Real World Model

18 Model mis-match Application Model Database Model

19 Trinity of models

20 Whiteboard --> Data Andre as knows knows Peter knows Emil knows Alliso n // Cypher query - friend of a friend start n=node(0) match (n)--()--(foaf) return foaf 22 11

21 You traverse the graph // then lookup traverse starting topoint find results in an index n=node:people(name == Andreas ) START me=node:people(name Andreas MATCH (me)-[:friend]-(friend)-[:friend]-(friend2) RETURN friend

22 START user = node(1) MATCH user -[user_skill]-> skill RETURN skill, user_skill SELECT skills.*, user_skill.* FROM users JOIN user_skill ON users.id = user_skill.user_id JOIN skills ON user_skill.skill_id = skill.id WHERE users.id =

23 An Example

24 What language do they speak here? Language Country

25 What language do they speak here? Language Country

26 What language do they speak here? Language Country

27 Tables Language Country language_code country_code language_nam e country_name word_count flag_uri

28 Need to model the relationship Language Country language_code country_code language_nam e country_name word_count language_cod e flag_uri

29 What if the cardinality changes? Language Country language_code country_code language_nam e country_name word_count country_code flag_uri

30 Or we go many-to-many? Language language_code language_nam e word_count LanguageCountr y Country language_cod e country_code country_code flag_uri country_name

31 Or we want to qualify the relationship? Language LanguageCountry Country language_code language_code country_code language_nam e country_code country_name primary flag_uri word_count

32 Start talking about Graphs

33 Explicit Relationship Language Country name name word_count IS_SPOKEN_IN flag_uri

34 Relationship Properties Language Country name name word_count IS_SPOKEN_IN as_primary flag_uri

35 What s different? Language IS_SPOKEN_IN LanguageCountr y Country language_code language_code country_code language_nam e country_code country_name primary flag_uri word_count

36 What s different? Implementation of maintaining relationships is left up to the database Artificial keys disappear or are unnecessary Relationships get an explicit name can be navigated in both directions

37 Relationship specialisation Language Country name name word_count IS_SPOKEN_IN as_primary flag_uri

38 Bidirectional relationships Language name word_count Country IS_SPOKEN_IN PRIMARY_LANGUAGE name flag_uri

39 Weighted relationships Language Country name name word_count POPULATION_SPEAKS population_fraction flag_uri

40 Keep on adding relationships Language Country name name word_count POPULATION_SPEAKS flag_uri population_fraction SIMILAR_TO ADJACENT_TO

41 EMBRACE the paradigm

42 Use the building blocks Nodes Relationships Properties RELATIONSHIP_NAME name: value

43 Anti-pattern: rich properties name: Canada languages_spoken: [ English, French ]

44 Normalize Nodes

45 Anti-Pattern: Node represents multiple concepts Country name flag_uri language_name number_of_words yes_in_language no_in_language currency_code currency_name

46 Split up in separate concepts Country Country name flag_uri name SPEAKS number_of_words currency_code yes currency_name no Currency currency_code currency_name USES_CURRENCY

47 Challenge: Property or Relationship? Can every property be replaced by a relationship? Should every entities with the same property values be connected?

48 Object Mapping Similar to how you would map objects to a relational database, using an ORM such as Hibernate Generally simpler and easier to reason about Examples Java: Spring Data Graph Ruby: Active Model Why Map? Do you use mapping because you are scared of SQL? Following DDD, could you write your repositories directly against the graph API?

49 CONNECT for fast access In-Graph Indices

50 Relationships for querying like in other databases same structure for different use-cases (OLTP and OLAP) doesn t work graph allows: add more structures Relationships should the primary means to access nodes in the database Traversing relationships is cheap that s the whole design goal of a graph database Use lookups only to find starting nodes for a query Data Modeling examples in Manual

51 Anti-pattern: unconnected graph name: Jones name: Jones name: Jones name: Jones name: Jones name: Jones name: Jones name: Jones name: Jones name: Jones name: Jones

52 Pattern: Linked List 55 33

53 Pattern: Multiple Relationships 55 44

54 Pattern-Trees:Tags and Categories 55 55

55 Pattern-Tree: Multi-Level-Tree 55 66

56 Pattern-Trees: R-Tree (spatial) 55 77

57 Example: Activity Stream 55 88

58 Graph Evolution

59 Evolution: Relationship to Node SENT_ ... G A T _FROM EM D E G _TO C C _ L I A see Hyperedges 66 00

60 Combine multiple Domains in a Graph you start with a single domain add more connected domains as your system evolves more domains allow to ask different queries one domain indexes the other Example Facebook Graph Search social graph location graph activity graph favorite graph...

61 Notes on the Graph Data Model Schema free, but constraints Model your graph with a whiteboard and a wise man Nodes as main entities but useless without connections Relationships are first level citizens in the model and database Normalize more than in a relational database use meaningful relationship-types, not generic ones like IS_ use in-graph structures to allow different access paths evolve your graph to your needs, incremental growth 66 22

62 How to get started? Documentation neo4j.org docs.neo4j.org - tutorials+reference Data Modeling Examples Neo4j in Action Good Relationships Worldwide one-day Neo4j Trainings Get Neo4j Participate

63 Really, once you start thinking in graphs it's hard to stop What will you build? Recommendations MDM Business intelligence Geospatial catalogs Systems access control Social computing Management your brain Biotechnology routing genealogy linguistics Making Sense of all that compensation data market vectors 66 99

64 Thank You! Questions?