Hierarchical Data in RDBMS

Transcription

1 Hierarchical Data in RDBMS

2 Introduction There are times when we need to store "tree" or "hierarchical" data for various modelling problems: Categories, sub-categories and sub-sub-categories in a manufacturing company Navigation menus of complex sitemaps Relationship of members of a multilevel marketing system Data organization and indexing should provide the following: Display of results (how do you build a navigation tree?) Report on statistics (how many people are in a multilevel marketing branch?) Extract a specific segment of the tree desired (you want to show the products with categories within a certain subcategory)

3 Introduction The tables of a relational database are not hierarchical (except perhaps XML), they are simply a flat list. Hierarchical data is defined as a set of data items that are related to each other by hierarchical relationships. Hierarchical relationships exist where one item of data is the parent of another item. Hierarchical data relationship is not naturally suitable to be represented in a relational database table. In what follows, most of the time we consider hierarchical data to be a collection of data where each item has a single parent and zero or more children (with the exception of the root item, which has no parent).

4 Data Modelling The main role of a database is to store data into it for searching purposes. As a consequence, the DBMS should supply means to retrieve various relationship between nodes (such as "subtree", "parent node", "child nodes", "ancestor nodes", "descendant nodes"). The role of a model is to facilitate finding related nodes of any node efficiently. There are some DBMSs that offer support for some hierarchical representations or hierarchical queries

5 Modelling Options Adjacency List Nested Set (Modified Preorder Tree Traversal) Nested Intervals Bridge Table (Closure Table) Path Enumeration (Materialized Path, Lineage Column) Flat Table Multiple lineage columns Fertile Forest Model

6 Model Comparison In order to compare the models in the presentation, we need to address the same problems for each of them in particular: storage complexity and size CRUD operations complexity ease of displaying full tree finding the root node finding all leaf nodes find all descendants of a given node find all ancestors of a given node find all descendants of a given node up to a certain depth Not all operations are covered with examples

7 Adjacency List In the adjacency list model, each item in the table contains a pointer to its parent. The topmost element has a NULL value for its parent. The adjacency list model has the advantage of being quite simple. Adding a new record to the system only requires to know the parent, with no other indexing. While the adjacency list model can be dealt with fairly easily in client-side code with ADTs, working with the model can be more problematic in pure SQL (set based).

8 Adjacency List Suppose that we intend to store the following hierarchy: [A]-+-[B]-+-[D] +-[E] +-[C]-+-[F] +-[G]-+-[H] +-[I]

9 Adjacency List CREATE TABLE adjacency_list_models ( id integer NOT NULL UNIQUE, name varchar(225) DEFAULT NULL, parent_id integer DEFAULT NULL, PRIMARY KEY (id) ); INSERT INTO adjacency_list_models (id, name, parent_id) VALUES (1, 'A', NULL), (2, 'B', 1), (3, 'C', 1), (4, 'D', 2), (5, 'E', 2), (6, 'F', 3), (7, 'G', 3), (8, 'H', 7), (9, 'I', 7);

10 Adjacency List In order to retrieve the full tree: SELECT t1.name AS lev1, t2.name as lev2, t3.name as lev3, t4.name as lev4 FROM adjacency_list_models AS t1 LEFT JOIN adjacency_list_models AS t2 ON t2.parent_id = t1.id LEFT JOIN adjacency_list_models AS t3 ON t3.parent_id = t2.id LEFT JOIN adjacency_list_models AS t4 ON t4.parent_id = t3.id WHERE t1.name = 'A'; The query depends on the number of levels of the tree, unmanageable for larger trees

11 Adjacency List In order to retrieve the full tree, some DBMSs have features to build the result independent of the number of levels For example, in PostgreSQL: CREATE RECURSIVE VIEW tree (id, ancestors) AS ( SELECT id, '{}'::integer[] FROM adjacency_list_models WHERE parent_id IS NULL UNION ALL SELECT n.id, t.ancestors n.parent_id FROM adjacency_list_models n, tree t WHERE n.parent_id = t.id); select * from tree; SELECT name, id FROM adjacency_list_models INNER JOIN tree USING (id) WHERE 7 = ANY(ancestors);

12 Adjacency List Finding all leaf nodes: SELECT t1.name FROM adjacency_list_models AS t1 LEFT JOIN adjacency_list_models as t2 ON t1.id = t2.parent_id WHERE t2.id IS NULL; Find the root of the tree SELECT name FROM adjacency_list_models WHERE parent_id is NULL;

13 Adjacency List Pros: Lightweight storage of data (only one field parent_id required to build an entire affiliate tree) Adding a new row doesn t alter any other row Moving a subtree to a different location only requires a change to one row in the database Not possible to obtain referential integrity errors, the database prevents deleting a parent row when still having children Cons: Very intensive overhead to report For each find level in the tree (how deep you are) requires a new query Hard to find all descendants/ancestors for a given node

14 Nested Sets The nested set model is to number the nodes according to a tree traversal, which visits each node twice, assigning numbers in the order of visiting, and at both visits. This leaves two numbers for each node, which are stored as two attributes. Querying becomes inexpensive: hierarchy membership can be tested by comparing these numbers. Updating requires renumbering and is therefore expensive. Refinements that use rational numbers instead of integers can avoid renumbering, and so are faster to update, although much more complicated. When considering each node as a circle, Nested Set Model represents the parent-child relationship by nested circles.

15 Nested Sets Consider the same tree structure as for previous model In order to create the traversal numerotation of each node, the following algorithm has to be applied, starting with root node and value 1: If current node has no left number, give a number in left of current node. If the current node has a child node that has no left number, move to the left-most child node. Return to first step. If the current node has no child node which has no left number, assign to current node a right number and move to parent node. Returns to first step. Exit if the current node has no parent node.

16 Nested Sets

17 Nested Sets From the picture can be seen that the left and right numbers of each node are within the range of the left and right of its parent node. For example, the left and right number of [G] is (11, 16). And the left and right number of [H] and [I] is within the range of [G]. Therefore, can determine [H] and [I] to be a descendant node of [G]. Each set of nodes having a common ancestor is nested within the node of this ancestor. That's why this model is called "nested sets".

18 Nested Sets CREATE TABLE nested_set_models ( id integer NOT NULL UNIQUE, name varchar(225) DEFAULT NULL, lft integer NOT NULL, rgt integer NOT NULL, PRIMARY KEY (id) ); INSERT INTO nested_set_models (id, name, lft, rgt) VALUES (1, 'A', 1, 18), (2, 'B', 2, 7), (3, 'C', 8, 17), (4, 'D', 3, 4), (5, 'E', 5, 6), (6, 'F', 9, 10), (7, 'G', 11, 16), (8, 'H', 12, 13), (9, 'I', 14, 15); SELECT * FROM nested_set_models;

19 Nested Sets Retrieving the full tree SELECT node.name FROM nested_set_models AS node, nested_set_models AS parent WHERE node.lft BETWEEN parent.lft AND parent.rgt AND parent.name = 'A' ORDER BY node.lft;

20 Nested Sets Find all leaf nodes: SELECT name FROM nested_set_models WHERE rgt = lft + 1; Find root node: SELECT name FROM nested_set_models WHERE lft = 1;

21 Nested Sets Find children of a node: SELECT DISTINCT child.name, child.lft, child.rgt FROM nested_set_models as child, nested_set_models as parent WHERE parent.lft < child.lft AND parent.rgt > child.rgt GROUP BY child.name, child.lft, child.rgt HAVING max(parent.lft) = 1 ; or SELECT Child.name, Child.lft, Child.rgt FROM nested_set_models as Parent, nested_set_models as Child WHERE Child.lft BETWEEN Parent.lft+1 AND Parent.rgt-1 AND NOT EXISTS ( SELECT * FROM nested_set_models as Mid WHERE Mid.lft BETWEEN Parent.lft AND Parent.rgt AND Child.lft BETWEEN Mid.lft AND Mid.rgt AND Mid.name NOT IN (Parent.name, Child.name) ) AND Parent.lft = 1 ;

22 Nested Sets Pros Reporting is easy and lightweight Quick selection of downline sets Queries using nested sets can be expected to be faster than queries using a stored procedure to traverse an adjacency list, and so are the faster option for databases which lack native recursive query constructs Cons Nested sets are very slow for inserts because it requires updating left and right domain values for all records in the table after the insert Report details (such as finding the number of levels in the child-set of records) can require complex queries

23 Closure Table This model uses two tables for managing hierarchical data. First table stores information of each node, and second table stores the relationships between nodes. Second table is referred as "closure table". The closure table has two columns for modelling hierarchical relationship. The columns contain the relationship between the ancestor nodes and each node. The closure table stores all paths from one node in the tree to another. This solution allows for fast node traversals of the tree. The tradeoff is obviously space. Consider the same tree structure as for previous model

24 Closure Table CREATE TABLE closure_table_models ( id integer NOT NULL UNIQUE, parent_id integer, -- NOT NEEDED name varchar(225) DEFAULT NULL, PRIMARY KEY (id) ); CREATE TABLE closure_table_relations ( ancestor_id integer NOT NULL, descendant_id integer NOT NULL, depth integer NOT NULL DEFAULT 0, PRIMARY KEY (ancestor_id, descendant_id) );

25 Closure Table INSERT INTO closure_table_models (id, parent_id,name) VALUES (1, 0, 'A'),(2, 1, 'B'),(3, 1, 'C'),(4, 2, 'D'), (5, 2, 'E'), (6, 3, 'F'),(7, 3, 'G'),(8, 7, 'H'),(9, 7, 'I'); INSERT INTO closure_table_relations VALUES (1, 1, 0), (2, 1, 1), (2, 2, 0), (3, 1, 1), (3, 3, 0), (4, 1, 2), (4, 2, 1), (4, 4, 0), (5, 1, 2), (5, 2, 1), (5, 5, 0), (6, 1, 2), (6, 3, 1), (6, 6, 0), (7, 1, 2), (7, 3, 1), (7, 7, 0), (8, 1, 3), (8, 3, 2), (8, 7, 1), (8, 8, 0), (9, 1, 3), (9, 3, 2), (9, 7, 1), (9, 9, 0);

26 Closure Table Retrieve full tree: SELECT c.name FROM closure_table_relations AS cp JOIN closure_table_models AS c ON c.id = cp.ancestor_id WHERE cp.descendant_id = 1; Retrieve children of a node: SELECT c.name FROM closure_table_relations AS cp JOIN closure_table_models AS c ON c.id = cp.ancestor_id WHERE cp.descendant_id = 1 AND cp.depth=1;

27 Closure Table Pros Inserts to the tree do not trigger updates on other nodes. They instead, populate the paths to the inserted node in the closure table. Moving a node only triggers updates to the node's own children. Using a "depth" or "path_length" column makes it easy to represent the whole tree structure. You can also easily query any specific level in the tree. Cons Storing the hierarchy in a closure table requires an additional table with a large number of rows. The closure table also won t work if you need to sort items by hierarchy, and re-parenting items is slow and costly.

28 Materialized Path "Materialized Path Model" is the model designed to be easier to find the relational nodes (such as sub-tree, ancestor nodes, etc.). The table of model has a column to save "path from the root node" for each node. The column manages a parent-child relationship in hierarchical data comprehensive. Also known as "Path enumeration model". In this approach each record stores the whole path to the root. First, create a unique key, which distinguishes the node's children. Then, list all the ancestor unique keys as the node's encoding. This list encoded as string is referred to as materialized path. Modern databases allow representing a list of nodes as a single value, but since materialized path has been invented long before then, the convention stuck to plain character string of nodes concatenated with some separator; most often '.' or '/'.

29 Materialized Path CREATE TABLE path_enumeration_models ( id integer NOT NULL UNIQUE, name varchar(225) DEFAULT NULL, tree_path varchar(225) DEFAULT NULL, PRIMARY KEY (id)); INSERT INTO path_enumeration_models (id, name, tree_path) VALUES (1, 'A', '1/' ), (2, 'B', '1/2/' ), (3, 'C', '1/3/' ), (4, 'D', '1/2/4/'), (5, 'E', '1/2/5/' ), (6, 'F', '1/3/6/' ), (7, 'G', '1/3/7/'), (8, 'H', '1/3/7/8/'), (9, 'I', '1/3/7/9/'); SELECT * FROM path_enumeration_models;

30 Materialized Path Retrieve full tree: select t1.name from path_enumeration_models t1, path_enumeration_models t2 where t1.tree_path like t2.tree_path '%' and t2.name = 'A' Ancestors of a given node: select t1.name from path_enumeration_models t1, path_enumeration_models t2 where t1.tree_path like t2.tree_path '%' and t2.name = 'I'

31 Materialized Path Ancestors of a given node (better performance): select t1.name from path_enumeration_models t1, path_enumeration_models t2 where t2.tree_path between t1.tree_path and t1.tree_path chr(255) and t2.name = 'I' Another way to determine the ancestors is to write a function that parses the path of the given node

32 Materialized Path Pros Performs well numerous inserts and deletes. Selects performs well but has typical gotchas LIKE '%.%'. (LTREE improves this with gist support) Using SQL92, easily returns all levels of the tree per statement and can limit to immediate levels Cons Analyzing tree structure is complicated

33 Summary Adjacency List: Columns: ID, ParentID Easy to implement. Cheap node moves, inserts, and deletes. Expensive to find level (can store as a computed column), ancestry & descendants Use Common Table Expressions in those databases that support them to traverse. Nested Set: Columns: Left, Right Cheap level, ancestry, descendants Requires a specific sort order (e.g. created). Sorting all descendants in a different order requires additional work.

34 Summary Nested Intervals Like nested set, but with real/float/decimal so that the encoding isn't volatile (inexpensive move/insert/delete) Have to deal with real/float/decimal representation issues Bridge Table (Closure Table) Columns: ancestor, descendant Stands apart from table it describes. Use triggers for maintaining this approach. Cheap ancestry and descendants For complete knowledge of a hierarchy needs to be combined with another option.

35 Summary Flat Table A modification of the Adjacency List that adds a Level and Rank (e.g. ordering) column to each record. Expensive move and delete Cheap ancestry and descendants Good Use: threaded discussion - forums / blog comments Lineage Column (Materialized Path, Path Enumeration) Column: lineage (e.g. /parent/child/grandchild/etc...) Limit to how deep the hierarchy can be. Descendants cheap Ancestry tricky (database specific queries)

36 Summary Multiple lineage columns Columns: one for each lineage level, refers to all the parents up to the root, levels down from the items level are set to NULL Limit to how deep the hierarchy can be Cheap ancestors, descendants, level Cheap insert, delete, move of the leaves Expensive insert, delete, move of the internal nodes