millions of SQL Server users worldwide, this feature broadens enormously concepts behind the model; how relationships are handled and what are the

Size: px

Start display at page:

Download "millions of SQL Server users worldwide, this feature broadens enormously concepts behind the model; how relationships are handled and what are the"

Cornelius Gervais Hamilton
5 years ago
Views:

2 SQL Server 2017 introduced the extension for graph databases. As there are millions of SQL Server users worldwide, this feature broadens enormously the audience of potential users. But, what to expect exactly from a graph database? How to query it? Is SQL Server fully featured compared to other products? In this session we answer these questions. We start illustrating the concepts behind the model; how relationships are handled and what are the common patterns and issues for a graph. What are the data connections a graph can easily represent. Then we compare the semantic model with SQL Server to discover how to apply it to real world. We analyze some case study: pattern matching, path finding, aggregation, ranking For each of them we show how to use standard T-SQL and how to rewrite the query using graph objects. What is the benefit of reformulate our queries in terms of clearness and performances, what is already available in order to consider SQL Server a valuable player.

3 BIG Thanks to SQL Sat Denmark sponsors GOLD SILVER BRONZE

4 Raffle and goodbye Beer Remember to visit the sponsors, stay for the raffle and goodbye beers Join our sponsors for a lunch break session in : cust 0.01 and cust 1.06 We hope you ll all have a great Saturday. Regis, Kenneth

5 Speaker Info First name: Andrea. Last name: Martorana Tusa. Microsoft MVP Data Platform Title: BI Specialist (database, datawarehouse, cubes, reporting,, big data,.) Company: Widex. A Danish manufacturer of hearing aids. Speaker for many community driven events: SQL Saturday (all Europe), Power BI Summit (Ireland), Power BI World Tour (Denmark), SQL Konferenz (Germany), Data Minds Connects (Belgium), SQL Nexus (Denmark), Intelligent Cloud (Denmark), SQL Days (Poland), Author for sqlservercentral.com, sqlshack.com, UGISS (User Group Italiano SQL Server). andrea.martoranatusa@gmail.com

6 Agenda Introducing graphs databases Graphs and SQL Server 2017 Query sintax and T-SQL graph exentions in SQL Server 2017 Rethink your code: how to query a graph database

7 Introducing graphs

8 What a graph is? A graph is a collection of Nodes and Edges Nodes = Entities (customers, products, territories, ) Edges = Relationships between entities Properties = Node or Edge attributes

9 What a graph is? Node Represents an entity: Product, Customer, Supplier, Nodes can contain some properties Nodes can be labeled with one or more labels Stored as physical table in the database

10 What a graph is? Edges Relationships between entities Relationships are named and directed and always have a start and end node Relationship can also contain properties Stored as physical table in the database

11 What a graph is? A graph database adds to graph features (Nodes and Edges) the standard CRUD methods available for every DBMS (Create, Read, Update, Delete)

12 What a graph is? The strength of graph databases lies in the concepts of relationships and connection between elements In a graph database connected data is stored as connected data. In a relational database data is stored in different tables linked through joins. Furthermore the flexibility of a graph model allows to add new nodes and new edges without compromising the existing structure.

13 What a graph is? Edges Nodes

14 What a graph is?

15 What a graph is? Twitter relationships and users messages Robinson, Webber, Eifrem Graph Databases O Reilly

16 Why using a graph? Typical use cases for a graph database: Real-Time Recommendation Engines correlate product, customer, inventory, supplier, logistics and even social sentiment data. Instantly capture any new interests shown in the customer s current visit Master Data Management unify your master data, including customer, product, supplier and logistics information to power the next generation of ecommerce, fraud detection, supply chain and logistics applications Social networks people and interactions, how they are related over a social media. Who follows who on Twitter, who is friend of who on Facebook. Network & IoT interconnections of devices, machines, sending signals to a receiver Identity & Access Management Managing multiple changing roles, groups, products and authorizations seamlessly track all identity and access authorizations and inheritances

17 Graphs and SQL Server 2017

18 SQL graph database architecture Source:

19 Writing queries in a graph database There s nothing you can do in a graph database that cannot be done in a relational database A graph database can handle some relationships easier than a relationship database. Some queries can be written in a more linear way A graph database is optimized for higly connected data

20 SQL graph database architecture Users can create one graph per database Two new tables types Node or edge tables can be created under any schema in the database Since nodes and edges are stored in tables, most of the operations supported on regular tables are supported on node or edge tables Users can model many-to-many relationships using edge tables. A single edge type can connect multiple type of nodes with each other, in contrast to foreign keys in relational tables.

21 Node tables Every time a node table is created, along with the user-defined columns, an implicit $node_id column is created, which uniquely identifies a given node in the database The values in $node_id are automatically generated and are a combination of object_id of that node table and an internally generated bigint value When the $node_id column is selected, a computed value in the form of a JSON string is displayed

22 Edge tables Edges are always directed and connect two nodes. An edge table enables users to model many-to-many relationships in the graph Every time an edge table is created, along with the user-defined attributes, three implicit columns are created in the edge table: 1. $edge_id: Uniquely identifies a given edge in the database 2. $from_id: Stores the $node_id of the node, from where the edge originates 3. $to_id: Stores the $node_id of the node, at which the edge terminates

23 Node and edge tables in SQL Server

24 Query sintax and T-SQL graph exentions in SQL Server 2017

25 T-SQL Extensions CREATE TABLE AS NODE / AS EDGE CREATE EDGE CONSTRAINTS: enforce specific semantics and maintain data integrity MATCH: built-in function to support pattern matching and traversal through the graph

26 T-SQL Extensions MATCH Specifies a search condition for a graph. MATCH can be used only with graph node and edge tables, in the SELECT statement as part of WHERE clause. Syntax: node-(edge)->node or node<-(edge) node From one node to another via an edge Edge names inside brackets Easier than a relational JOIN

27 T-SQL Extensions Cypher START a=node:user(name='michael') MATCH (a)-[:knows]->(b)-[:knows]->(c), (a)-[:knows]->(c) RETURN b, c T-SQL SELECT Person2.name AS FriendName FROM Person person1, friend, Person person2 WHERE MATCH(Person1-(friend)->Person2) AND Person1.name = 'Michael' START: starting point in the graph MATCH: relationship pattern RETURN: what data return to query engine SELECT FROM WHERE MATCH

28 T-SQL Extensions New system functions to extract information from the generated columns. OBJECT_ID_FROM_NODE_ID GRAPH_ID_FROM_NODE_ID NODE_ID_FROM_PARTS OBJECT_ID_FROM_EDGE_ID GRAPH_ID_FROM_EDGE_ID EDGE_ID_FROM_PARTS

29 Rethink your code: how to query a graph database

30 Demo: social relationship Relational DB 1: LivesIn PersonID 1: Person PersonID 1: CityID PersonName City CityID CityName FriendOf PersonID1 PersonID2 Likes PersonID RestaurantID Rating LocatedIn 1: 1: CityID RestaurantID Restaurant RestaurantID RestaurantName 1:

31 Demo: social relationship Graph DB Nodes Person City Restaurant Edges LivesIn Likes LocatedIn Friends

32 Demo: social relationship Search for all the restaurants that John s friend like Relational Person1 John 1: FriendOf John. Likes Person2 :1 1: Mary :1 Mary Pizza Hut Alice 9 Restaurant Pizza Hut Graph FriendOf Likes Person Restaurant

33 Demo: social relationship Search for all the restaurants that John s friend like Relational Person1 FriendsOf Person2 Likes Restaurant 1: :1 1: :1 Graph Person1 Person2 Restaurant FriendOF Likes

34 Hierarchies Implement hierarchies in a database All columns in one table Different related tables Self-Join table Graph DB Nodes are related via Edges No redundancy Handle many-to-many relationships Handle multiple hierarchies Easy to write and read

35 Hierarchies Is Manager Alice Ken Peter Is Report Is Manager Is Report Employee

36 Hierarchies - Limitations Currently complex traverses not supported via MATCH. Cannot browse up to the first level of the hierarchy Must wrap the query into a loop, a function, (not a recursive CTE) to perform transitive closure on your hierarchy

37 Graph Reporting in Power BI Use Force Directed Graph custom visual in Power BI to visualize connections between data

38 Merge relational and graph queries Graph database features are integrated with the database engine. You can work with graph and relational data side-byside, for instance merging graph tables into an existing relational structure. Nodes Stores SalesReps Products Vendors Edges Purchases Sells Supplies

39 Route calcolation and shortest path Global Post parcel network Robinson, Webber, Eifrem Graph Databases O Reilly

40 Route calcolation and shortest path Graph database model associated to the network Two types of connections: CONNECTED_TO DELIVERY_ROUTE Robinson, Webber, Eifrem Graph Databases O Reilly

41 Route calcolation and shortest path Route calculations involve finding the cheapest route between two locations. Requirements is that calculated route must go via at least one parcel center in the part of the graph. Robinson, Webber, Eifrem Graph Databases O Reilly

42 Route calcolation and shortest path START s=node:location(name={startlocation}), e=node:location(name={endlocation}) MATCH upleg = (s)<-[:delivery_route*1..2]-(db1) WHERE all(r in relationships(upleg) WHERE r.start_date <= {intervalstart} AND r.end_date >= {intervalend}) WITH e, upleg, db1 MATCH downleg = (db2)-[:delivery_route*1..2]->(e) WHERE all(r in relationships(downleg) WHERE r.start_date <= {intervalstart} AND r.end_date >= {intervalend}) WITH db1, db2, upleg, downleg MATCH toproute = (db1)<-[:connected_to]-()-[:connected_to*1..3]-(db2) WHERE all(r in relationships(toproute) WHERE r.start_date <= {intervalstart} AND r.end_date >= {intervalend}) WITH upleg, downleg, toproute, reduce(weight=0, r in relationships(toproute) : weight+r.cost) AS score ORDER BY score ASC LIMIT 1 RETURN (nodes(upleg) + tail(nodes(toproute)) + tail(nodes(downleg))) AS n Cypher query to calculate parcel route Robinson, Webber, Eifrem Graph Databases O Reilly

43 Route calcolation and shortest path A traversal-based implementation of the route calculation engine must solve two problems: finding shortest paths, and filtering paths based on time period. The real solution for calculating this routes has been implemented using a traversal framewok engine available into a native graph database (Neo4J). This kind of calculation is not replicable in the current version of SQL Server. The database engine does not allow traversal functions and some clauses are missing. Robinson, Webber, Eifrem Graph Databases O Reilly

44 Summary What is ready Get rid of redundancy Write simple queries with MATCH pattern Handle higly connected data Handle many-to-many rel Handle hierarchical data with multiple parents What is missing Using nodes as derived table in complex queries No Update for edges Edges constraints OR and NOT operators in MATCH pattern Traversal functions Routing Shortest path

45 Summary What is coming (SQL Server 2019) Edge constraints Check on node types Check on existing nodes Force referential integrity

46 Raffle and goodbye Beer Remember to visit the sponsors, stay for the raffle and goodbye beers Join our sponsors for a lunch break session in : cust 0.01 and cust 1.06 We hope you ll all have a great Saturday. Regis, Kenneth

47 BIG Thanks to SQL Sat Denmark sponsors GOLD SILVER BRONZE

48 References / introduction/

Overview Architecture Sample

Table of Contents Overview Architecture Sample Graph processing with SQL Server and Azure SQL Database 1/17/2018 2 min to read Edit Online THIS TOPIC APPLIES TO: SQL Server (starting with 2017) Azure SQL