Introduction to Graph Processing Chapter 02: Graph Processing on SAP HANA

Size: px

Start display at page:

Download "Introduction to Graph Processing Chapter 02: Graph Processing on SAP HANA"

Deborah Booker
5 years ago
Views:

3 Graph Engine in SAP HANA Graph engine was introduced in SAP HANA SPS07 (2013) and has been tested since then by SAP and Ramp-Up Customers With SAP HANA SPS12, the engine was fully released The graph engine is directly built into the SAP HANA system It re-uses as many existing technologies as possible Offers many features directly from the beginning, e.g. backup, security Easy to integrate into applications (e.g. SAP Business Suite) 2016 SAP UCC Magdeburg. All rights reserved. 3

Relational In-Memory Stores Row Store Row Engine Execution Engine Column Store Text Engine Spatial Engine Graph

4 Technical basis for graph processing Business Applications Connection and Session Management SQL SQL Script MDX Planning PAL AFL Authorization Manager Calculation Engine SQL Optimizer and Plan Generator Metadata Manager Relational In-Memory Stores Row Store Row Engine Execution Engine Column Store Text Engine Spatial Engine Graph Engine Authorization Manager Persistence Layer Logging and Recovery Page Management 2016 SAP UCC Magdeburg. All rights reserved. 4

Technical basis for graph processing SQL Console Modeler Graph Viewer SQL / SQLScript and JDBC SQL Optimizer and Plan Generator Calculation Engine

5 Technical basis for graph processing SQL Console Modeler Graph Viewer SQL / SQLScript and JDBC SQL Optimizer and Plan Generator Calculation Engine Graph Algorihms SQL Console Pattern Matching Primary Graph Store Secondary Graph Store Attribute Engine 2016 SAP UCC Magdeburg. All rights reserved. 5

6 The SAP HANA Graph model SAP HANA uses a property graph Graphs in SAP HANA: A Graph is a set of vertices and a set of edges. Each edge connects two vertices (source and target) Edges are always directed There can be two or more edges connecting the same two vertices Vertices and edges can have an arbitrary number of attributes An attribute consists of a name that is associated with a data type and a value SAP UCC Magdeburg. All rights reserved. 6

7 Building graphs in SAP HANA The primary storage of a graph are two relational objects - The vertex table and edge table One of the attributes must uniquely identify vertices/edges. This attribute is also referred to as a vertex/edge key. The edge table contains two additional columns referencing the key column of the vertex table. One of them identifies the source vertex and the other identifies the target vertex of an edge. By using regular tables as basis for the graph, all features of the SAP HANA system apply as well security, backup, export/import etc SAP UCC Magdeburg. All rights reserved. 7

Building graphs in SAP HANA Based on a vertex

defined A graph workspaces contains one graph

schema in which they reside in and their name,

schema can hold many graph workspaces Sources:

8 Building graphs in SAP HANA Based on a vertex and an edge table, a Graph Workspace can be defined A graph workspaces contains one graph Graph workspaces can be distinguished by the schema in which they reside in and their name, which must be unique within one schema One schema can hold many graph workspaces Sources: [11] 2016 SAP UCC Magdeburg. All rights reserved. 8

on the pre-built algorithms that can be used to create specific A graph-specific language for SAP

9 Handling graphs in SAP HANA Currently, there are some extensions to SQL that can be used to handle the graph, but there is no possibility to traverse the graph directly Currently, the main focus is on the pre-built algorithms that can be used to create specific A graph-specific language for SAP HANA is planned for one of the next releases Sources: [12] 2016 SAP UCC Magdeburg. All rights reserved. 9

10 SAP HANA Graph Algorithms SAP HANA s Graph Engine brings with it pre-defined algorithms that can be configured The algorithms are configured by creating so-called Calculation Scenarios The SAP HANA Graph Viewer implements an interface to create calculation scenarios by entering only the parameters required for the analysis The calculation scenarios created with the SAP HANA Graph Viewer can be reused 2016 SAP UCC Magdeburg. All rights reserved. 10

11 SAP HANA Graph Algorithms The way the graph algorithms are used is like prototyping: A template that can be configured according to your requirements The configured template can be saved and re-used as often as required Example: The template is a blueprint for a car engine You can configure how many cylinders you would like, if the engine will be super-/turbocharged etc. After you ve configured your engine, it can be build automatically and re-used as often as you like This concept allows a great flexibility without the need to re-implement standard algorithms over and over again - and all without any programming knowledge 2016 SAP UCC Magdeburg. All rights reserved. 11

certain distance, the neighborhood, by traversing

12 Graph Algorithms: Neighborhood Search Find all adjacent vertices of a specified vertex within a certain distance, the neighborhood, by traversing the graph 2016 SAP UCC Magdeburg. All rights reserved. 12

13 Graph Algorithms: Neighborhood Search The algorithm requires a start vertex, a direction (only follow incoming / outgoing edges), a start / end level (distance) as well as filters for vertices and edges As output, the algorithm delivers a list of vertices with their depth ( distance from the start vertex) Use Cases Social Network Marketing: o Find all friends (depth: 0), friends-of-friends (depth:1), friends-of-friends-of-friends (depth:2) and so on o Find users with a big network which should consist of users that also have big networks, then target these users for marketing campaigns 2016 SAP UCC Magdeburg. All rights reserved. 13

number of vertices between source and target or over an optional edge weight (attribute) Can also

14 Graph Algorithms: Single Source Shortest Path The algorithm finds the shortest path between one vertex and all other vertices of the graph Distance between two vertices is calculated over the number of vertices between source and target or over an optional edge weight (attribute) Can also be restricted to find the distance between two vertices 2016 SAP UCC Magdeburg. All rights reserved. 14

15 Graph Algorithms: Single Source Shortest Path Takes a start vertex, an optional end vertex and an optional edge weight attribute as input parameters As output, the algorithm returns the calculated weight ( distance) and the path for each target vertex Use Cases Trivial: Navigation Systems o Navigation system can use all of the attributes that are stored in the relationshipts in order to better perform (e.g. if two nodes are connected by a street without tarmak, the algorithm should avoid this street) Logistics: Calculating the best route for a delivery o The weights between nodes can represent different dimensions: distance, cost, time etc. o Depending on the requirements, one or many of the different dimensions can be used to calculate the best route 2016 SAP UCC Magdeburg. All rights reserved. 15

Graph Algorithms: Strongly Connected Components A directed graph is strongly connected if there is a path from each vertex to every other vertex All vertices can be reached from all vertices: Graph

16 Graph Algorithms: Strongly Connected Components A directed graph is strongly connected if there is a path from each vertex to every other vertex All vertices can be reached from all vertices: Graph is strongly connected No vertex can be reached from vertex 4: Graph is not strongly connected A graph can have one of more subgraphs that are strongly connected 2016 SAP UCC Magdeburg. All rights reserved. 16

17 Graph Algorithms: Strongly Connected Components The algorithm takes a graph and finds all subgraphs ( components ) that are strongly connected As output, the algorithm produces a list of all vertices and their respective components 2016 SAP UCC Magdeburg. All rights reserved. 17

18 Graph Algorithms: Strongly Connected Components Use Cases Social Media Analytics: Groups of people (friends, students) are usually strongly connected and share similar interests Process Mining: Find cycles in process graphs which may prevent the deterministic execution Production Process Analysis: Find dependencies in the production graph of a product 2016 SAP UCC Magdeburg. All rights reserved. 18

19 Graph Algorithms: Pattern Matching Create an abstract graph (the pattern) that you want to find (as a subgraph) in a graph The algorithm extracts all subgraphs that match the patttern Example: Find all persons over 35 who have bought a red product age > 35 bought color == red 2016 SAP UCC Magdeburg. All rights reserved. 19

21 Graph Algorithms: Pattern Matching Also called the Isomorphic Subgraph Problem Use Cases Marketing: o Find customers with certain properties, e.g. which customers like to buy certain products in certain shops? o Find connected customer groups that match certain properties, e.g. which groups of friends where only a small percentage has yet bought the latest smartphone? 2016 SAP UCC Magdeburg. All rights reserved. 21

22 Future of Graph Processing in SAP HANA Current feature set is limited but will be extended in the future Planned innovations include: More algorithms Custom algorithms / own language Optimizier optimizations Tighter integration into other SAP HANA Features, e.g. Geospatial Processing, Text Analysis, Predictive Analytics etc SAP UCC Magdeburg. All rights reserved. 22

24 Sources [1] Jaehyun Park - Basic Graph Algorithms, CS 97SI Stanford University, June 29, 2015 [2] Rudolf et al. - SynopSys: Large Graph Analytics in the SAP HANA Database Through Summarization [3] SAP Startup Focus - SAP HANA Graph Engine Graph-based Data Processing for Flexible Information Management, Exploration, and Analysis [4] Rudolf et al. - The Graph Story of the SAP HANA Database [5] May P. Chen SAP HANA SPS12 What s New? SAP HANA Graph Part I, May 2016 [6] May P. Chen SAP HANA SPS12 What s New? SAP HANA Graph Part II, May 2016 [7] SAP HANA Graph Reference - SAP HANA Platform SPS 12, Document Version: [8] Ross McNeely - Data Modeling with Graph Databases [9] SAP HANA Graph Reference - SAP HANA Platform SPS 12, Document Version: [10] Marcus Stuber - Eine Einführung in das Graphdatenbankmodell, March SAP UCC Magdeburg. All rights reserved. 24

SAP HANA ADMINISTRATION

IT HUNTER SOLUTIONS Contact No - +1 9099998808 Email ID ithuntersolutions@gmail.com SAP HANA ADMINISTRATION SAP HANA Technology Overview Introduction to SAP HANA SAP In-Memory Strategy HANA compare to