Relational Database Systems 2 4. Trees & Advanced Indexes

Size: px

Start display at page:

Download "Relational Database Systems 2 4. Trees & Advanced Indexes"

Poppy Johns
5 years ago
Views:

1 Relational Database Systems 2 4. Trees & Advanced Indexes Wolf-Tilo Balke Benjamin Köhncke Institut für Informationssysteme Technische Universität Braunschweig

2 4 Trees & Advanced Indexes 4.1 Introduction 4.2 Binary Search Trees 4.3 Self Balancing Binary Search Trees 4.4 B-Trees Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 2

3 4.1 Introduction Indexes need a suitable data structure For efficient index look-ups search keys need to be ordered Remember: All indexes should be stored in a separate database file, not together with data A suitable number of DB blocks (adjacent on disk) is reserved at index creation time If the space is not sufficient, another file is created and linked to the original index file Search Key 1 Block Address 1 Search Key 2 Block Address 2 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 3

4.1 Introduction Search within an index Bisection search possible: log 2 n ; O(log n) 4 6 7 16 18 21 24 33 39 47 68 72 89 92 99 But usually indexes span several DB blocks If index is in n

4 4.1 Introduction Search within an index Bisection search possible: log 2 n ; O(log n) But usually indexes span several DB blocks If index is in n blocks, O(n) blocks need to be read from disk Example: search for Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 4

5 4.1 Introduction Maintenance of index is also difficult Insert a new search key with value 5! In worst case, all cells need to be shifted and all blocks need to be accessed Similar problem occurs when deleting a value Often: do not shift values, but mark key as deleted Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 5

4.1 Introduction In this lecture, we discuss more efficient multilevel data structures B-trees Prevalent in database systems Better access performance Much better update

6 4.1 Introduction In this lecture, we discuss more efficient multilevel data structures B-trees Prevalent in database systems Better access performance Much better update performance To understand B-trees better, we start by examining binary search trees Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 6

7 4.2 Binary Trees Binary trees are Rooted and directed trees Each node has none, one or two children Each node (except root) has exactly one parent 0/1 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 7

8 4.2 Binary Trees Some naming conventions Nodes without children are called leaf nodes The depth of node N is the path length from the root The tree height is the maximum node depth If there is a path from node N1 to node N2, N1 is an ancestor of N2 and N2 is a descendant of N1 The size of a node N is the number of all descendants of N including itself A subtree of a node N is formed of all descendant nodes including N and the respective links root subtree red red Leaf nodes tree height = 3 red node: size = 3 depth = 1 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 8

4.2 Binary Trees Properties of binary trees Full binary tree (or proper) Each node has either zero or two children Perfect binary tree All leaf nodes have the same depth With height h, contains 2 h

9 4.2 Binary Trees Properties of binary trees Full binary tree (or proper) Each node has either zero or two children Perfect binary tree All leaf nodes have the same depth With height h, contains 2 h nodes Height-balanced binary tree Depth of all leaf nodes differ by at most 1 With height h, contains between 2 h-1 and 2 h nodes Degenerated binary tree Each node has either zero or one child Behaves like a linked list: search in O(n) Full and Balanced Full and Perfect Degenerated Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 9

10 4.2 Binary Search Trees Binary search trees are binary trees with Each node has a unique value assigned There is a total order on all values Left subtree of a node contains only values less than node value Right subtree of a node contains only values larger than the node value Aiming for O(log n) search complexity Structurally resembles bisection search / Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 10

11 4.2 Binary Search Trees Constructing and inserting into binary search trees Values are inserted incrementally First value is root Additional values sink into tree Sink to left subtree if value smaller Sink to right subtree if value larger Attach to last node as left/right child, if subtree is empty Insert order of values does highly influence resulting and intermediate tree properties Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 11

4.2 Binary Search Trees Suppose insert order 57, 33, 42, 85, 17, 61, 99 57 33 42 85 17 61 33 57 42 85 17 61 99 Insert 57 99 Insert 33, 42 Degenerated 57 33 85 61 99 57 33

12 4.2 Binary Search Trees Suppose insert order 57, 33, 42, 85, 17, 61, Insert Insert 33, 42 Degenerated Insert 85, 17 Full and Balanced Insert 61, 99 Perfect and Full Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 12

13 4.2 Binary Search Trees Suppose insert order 99, 85, 61, 57, 42, 33, Insert complexity is thus O(n) worst case O(log n) average case Degenerated Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 13

14 4.2 Binary Search Tree Search Key Start with root Recursive Procedure If node value = v Return node If node is leaf Value not found if v < node value Descend to left subtree Else Descend to right subtree Complexity: Average case: O(log n) Worst case: O(n) Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 14

right subtree Visit node 17-35-49-42-33-61-99-85-57 57 33 85 17 42 61 99 35 49 Pre-Order: 57-33-17-42-35-49-85-61-99 57 33 85 17 42

15 4.2 Binary Search Tree Tree Traversal Accesses all nodes of the tree Pre-Order Visit node Traverse left subtree Traverse right subtree In-Order (sorted access) Traverse left subtree Visit node Traverse right subtree Post-Order Traverse left subtree Traverse right subtree Visit node Pre-Order: In-Order: Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 15

4.2 Binary Search Tree Deleting Nodes has complexity O(n) worst case, O(log n) average case Locate the node to delete by tree traversal If node is leaf, just delete it If node has one child, delete

16 4.2 Binary Search Tree Deleting Nodes has complexity O(n) worst case, O(log n) average case Locate the node to delete by tree traversal If node is leaf, just delete it If node has one child, delete node and attach child to parent If node has two children Replace either by a) in-order successor (the left-most child of the right subtree) b) in-order predecessor (the right-most child of the left subtree) Example: delete search key with value 57 a) b) Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 16

17 4.2 Binary Search Trees Summary Very simple, dynamic data structure Quite efficient on average O(log n) for all operations Can be very inefficient for degenerated cases O(n) for all operations 0/1 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 17

4.3 Self-Balancing Binary Search Trees Observation: Binary Search Trees are very efficient when perfect or balanced Idea: Continuously optimize tree structure to keep tree

18 4.3 Self-Balancing Binary Search Trees Observation: Binary Search Trees are very efficient when perfect or balanced Idea: Continuously optimize tree structure to keep tree balanced Popular Implementations AVL-Tree (classic example) Red-Black-Tree Splay-Tree Scapegoat-Tree Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 18

19 4.3 Self-Balancing Binary Search Trees Basic Concepts for Deletion: Global Rebuild (Lazy Deletion) Start with balanced tree Don t delete a node, just mark it as deleted Search algorithm scans deleted nodes, but does not return them If Rebuild Condition is met, rebuild the whole tree without the deleted nodes Rebuild as soon as half of the nodes are marked as deleted Complete rebuild can be performed in O(n) Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 19

20 4.3 Self-Balancing Binary Search Trees Global Rebuild (cont.) Search Efficiency n number of unmarked nodes Tree is balanced, contains max 2n nodes overall Number of accesses during search usually just increases by 1 O(log n) Delete Efficiency Global rebuild is in O(n) But only necessary after n deletions Amortized additional costs per deletion is O(1) Overall complexity Average: O(log n) Worst Case: O(n), if actual rebuild is performed Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 20

21 4.3 Self-Balancing Binary Search Trees Global Rebuild (cont.) Direct Deletion with Rebuild Similar complexity as with lazy deletion Increased per delete effort Reduced per search effort until rebuild Delete nodes as in normal binary trees Increment deletion counter c d Rebuild tree as soon as c d = n, reset c d Delete 57,33,42,61 Rebuild Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 21

22 4.3 Self-Balancing Binary Search Trees Basic Concepts for Insertion and Deletion: Local Balancing (Subtree Balancing) Start with balanced tree Insert/delete nodes normally If a subtree becomes too unbalanced, locally balance subtree to regain global balance To detect unbalanced subtrees, each node n needs to know the size v and the height h(v) of it s subtree Unbalanced Condition: (Height Balancing) Subtree is too unbalanced when h(left(v))-h(right(v)) > α α is a constant which can be adjusted (for AVL, α=1) Alternative Unbalanced Condition: Subtree is too unbalanced when h(v) > α * log 2 v α is a constant which can be adjusted Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 22

4.3 Self-Balancing Binary Search Trees Local Balancing (cont.) After inserting a node, walk back the tree and update stored subtree statistics h(v) and v.

23 4.3 Self-Balancing Binary Search Trees Local Balancing (cont.) After inserting a node, walk back the tree and update stored subtree statistics h(v) and v. If node a node v is too imbalanced, balance subtree of v 57 2, , 6 Height Imbalanced for α=1 2-0 = 2 > , , , , , , 1 key 17 1, , 1 h(v) v 5 0, 1 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 23

24 4.3 Self-Balancing Binary Search Trees Local Balancing can be achived by Rebuilding the subtree O( v ) = O(n) the worst case However, O(log n) in average This operation is expensive. But in the context of DBMS, it may pay off as it can also consolidate and optimize physical storage locations Especially suited for disk based trees Rotating Only pointers are moved very efficient O(1) Does not change physical storage of nodes Especially suited for main memory based trees Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 24

4.3 Self-Balancing Binary Search Trees Local Balancing Rotating Simple Rotation (left, right) Pivot y right x x 3 1 y 1 2 left 2 3 Double Rotation (left-left, right-right,

25 4.3 Self-Balancing Binary Search Trees Local Balancing Rotating Simple Rotation (left, right) Pivot y right x x 3 1 y 1 2 left 2 3 Double Rotation (left-left, right-right, Rollercoaster) z right-right y right-right x y 4 x z 1 y x z 1 2 left-left left-left 3 4 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 25

26 4.3 Self-Balancing Binary Search Trees Local Balancing Rotating Double Rotation (left-right, Zig-Zag) z left z right x y 4 x 4 y z 1 x y Double Rotation (right-left, Zig-Zag) Analogous to left-right Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 26

27 Self-Balancing Binary Search Trees The presented concepts can be combined in different ways to implement self-balancing trees AVL-Tree (classic example) Red-Black-Tree Splay-Tree Scapegoat-Tree Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 27

Self-Balancing Binary Search Trees Implementation: AVL-Trees Invented 1962 by Adelson-Velsky and Landis Uses Local Rebalancing with Rotations for Insertion and Deletion Unbalanced criterion:

28 Self-Balancing Binary Search Trees Implementation: AVL-Trees Invented 1962 by Adelson-Velsky and Landis Uses Local Rebalancing with Rotations for Insertion and Deletion Unbalanced criterion: h(left(v)) - h(right(v)) > 1 Height difference of left and right subtree of v is 2 or more Height information is stored explicitly within nodes Update backtracking after each insert and delete Storage overhead of O(n) Guaranteed maximum height of 1.44 log 2 n Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 28

Self-Balancing Binary Search Trees Implementation: Scapegoat-Trees Invented

Balancing with Rotations for Insertions Unbalanced criterion: h(v) > log 1/α

5 α 1 Node statistics (height, size) determined dynamically during

29 Self-Balancing Binary Search Trees Implementation: Scapegoat-Trees Invented 1993 by Galperin and Rivest Uses Global Rebuilding for Deletions Local Balancing with Rotations for Insertions Unbalanced criterion: h(v) > log 1/α v + 1; 0.5 α 1 Node statistics (height, size) determined dynamically during backtracking Only global statistics are stored Storage overhead of O(1) Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 29

4.4 Problems with Binary Search Trees Are binary trees

Yes and No Binary Trees are great data-structures for usage

when stored on external storage (i.e. hard disks) 0/1 & =

30 4.4 Problems with Binary Search Trees Are binary trees really suitable for disk based databases? Yes and No Binary Trees are great data-structures for usage in internal memory But they have a very bad performance when stored on external storage (i.e. hard disks) 0/1 & = Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 30

31 4.4 Problems with Binary Search Trees Binary tree nodes have to be stored within hard disk blocks in linear fashion When tree is large, nodes are scattered among the blocks In worst case, a new block must be read from disk for every node accessed during search or traversal Every linearization scheme for binary trees has that problem Reading a block from disk is very expensive Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 31

32 4.4 Problems with Binary Search Trees Sample linearization Search for 42 In worst case needs to fetch 3 blocks from disk for just 4 nodes Problem is even worse for full tree traversal Tree: Disk/DB Blocks: Block 1 Block 2 Block 3 Block 4 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 32

4.4 B-Trees B-Trees adapt concepts and techniques learned for binary trees and optimize them for harddisk storage Basic Ideas: Searching within a DB/disk block is very efficient Take advantage of

33 4.4 B-Trees B-Trees adapt concepts and techniques learned for binary trees and optimize them for harddisk storage Basic Ideas: Searching within a DB/disk block is very efficient Take advantage of static nature within a block Search can be performed in memory with bisection search Treat entire blocks as tree nodes Reading blocks from the disk is expensive Most data resides in the leaf nodes Thus minimize the height of the tree Dramatically increase fan-out factor Tree becomes bushy Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 33

4.4 Block Search Trees First Improvement: Block Search Tree Nodes are complete DB blocks Each node can store up to q pointers p i and q-1 unique and ordered key entries k i : <p 1, k 1,, k q-1, p q >

34 4.4 Block Search Trees First Improvement: Block Search Tree Nodes are complete DB blocks Each node can store up to q pointers p i and q-1 unique and ordered key entries k i : <p 1, k 1,, k q-1, p q > k i < k i+1 Pointers p i link to subtrees (or are empty). All keys in subtree of p i are less than k i and greater as k i-1 Node Pointers Key Value Node EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 34

35 4.4 Block Search Trees Locate a key k Recursive Procedure: Start with root node Use bisection search within the current node If key found Return it If key not found If there is a p i with k i-1 < k < k i» Follow p i and repeat algorithm with link node Else» Key not in tree Example: Locate EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 35

36 4.4 Block Search Trees Insert a key k Recursive Procedure: Start with root node Use bisection search within the current node If key found Key cannot inserted twice, abort If key not found If there is a p i with k i-1 < k < k i» Follow p i and repeat algorithm with link node Else» If there is space left in the node Insert key and restore sort order» Else Create new, empty node Insert k into new node Link new node to p i in current node such that with k i-1 < k < k i EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 36

37 4.4 Block Search Trees Insert a key : EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 37

38 4.4 Block Search Trees Delete a key k Start with root node Locate k If k is in leaf node, delete k from node and restore order If leaf node is now empty, delete the node If k is in internal node If no or only one directly adjacent pointer of k are used» Delete k and restore order If k is a separator between two used pointers,» If space in both subnodes is sufficient Union both nodes into one Delete k and restore order» Else Replace k with new separator key Either largest key in left node or smallest key in right node Any completely empty node is deleted as in binary search trees EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 38

39 4.4 Block Search Trees Delete a key : EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 39

40 4.4 Block Search Trees Delete a key : EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 40

41 4.4 Block Search Trees Delete a key : EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 41

42 4.4 Block Search Trees Block Search Trees have similar properties to Binary Search Trees Can be perfect, balanced or degenerated Assume height h=3; fan-out-factor q=2048; and total number of keys n Block Search Tree One node can store up to 2047 keys and 2048 links Perfect : n = 8581M Balanced : 4M n 8581M Degenerated : n = 6141 Binary Search Tree One node can store 1 key and up to 2 links Perfect : n = 7 Balanced : 3 < n 7 Degenerated : n = 3 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 42

43 4.4 Block Search Trees Assume n = 1,000,000,000; fan-out-factor q=2048; and height h Block Search Tree Balanced : h = 3 Degenerated : h = 488,520 Binary Search Tree Balanced : h = 30 Degenerated : h = 1,000,000,000 During search, there is one disk access per tree height in worst case In this example, block search tree are already 10 times more efficient when balanced Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 43

to log 2 n Accessing a node is expensive on disks, huge improvement Can be very inefficient for degenerated cases O(n) for all operations

44 4.4 Block Search Trees Summary Data structure optimized for disk storage Is very efficient in average case O(log n) for all operations Even better Average node-accesses to locate a key is log fan-out n» Fan-out usually in the order of several thousands» Binary tree averages only to log 2 n Accessing a node is expensive on disks, huge improvement Can be very inefficient for degenerated cases O(n) for all operations Performs almost as bad as binary trees in degenerated case Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 44

self-balancing No degenerated cases anymore EN 14.3.

45 4.4 B-Trees B-Tree are specialized Block Search Trees for Indexing Invented by Rudolf Bayer in 1971 Keys may be non-unique Tree is self-balancing No degenerated cases anymore EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 45

46 4.4 B-Trees Basic structure of a B-tree node Nodes contain key values and respective data (block) pointers to the actual data records Additionally, there are node pointers for the left, resp. right interval around a key value Key Value Data Pointer Tree Node Node Pointers Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 46

47 1, Adams, $ 887,00 2, Bertram, $19,99 3, Behaim, $ 167,00 4, Cesar, $ 1866,00 5, Miller, $179,99 6, Naders, $ 682,56 7, Ruth, $ 8642,78 8, Smith, $675,99 9, Tarrens, $ 99, B-Trees B-Trees as Primary Index EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 47

48 4.4 B-Trees All base operations similar to Block Search Tree with small changes Guaranteed fill degree Self-Balancing Each node contains between L(ower) and U(pper) links Usually 2* L = U Nodes are split during insertion as soon as they contain more than U-2 keys Nodes are unioned during deletion as soon as they contain less than L keys If complete node is created or deleted, use local rebalancing to re-balance tree Local rebuilding for disk-based storage, rotations for memory based storage EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 48

49 4.4 B-Trees All insertions happen at the leaf nodes Search the tree to find leaf node where new element should be added If the leaf node contains fewer than the maximum legal number of elements ( leaf node < U) Insert the new element in the node and restore order Otherwise the leaf node is split into two nodes (node split) The median is chosen from among the leaf's elements and the new element Values less than the median are put in the new left node and values greater than the median are put in the new right node, with the median acting as a separation value That separation value is added to the node's parent, which may cause it to be split, and so on If the splitting goes all the way up to the root, it creates a new root with a single separator value and two children Remember: the lower bound on the size of internal nodes does not apply to the root Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 49

50 4.4 B-Trees Deleting nodes is problematic, because nodes sizes can decrease under the minimum number of elements Deleting an element may put it under the minimum number of elements and children ( node size < L) Deleting an element in an internal node may be a separator for its child nodes Deletion from a leaf node Search for the value to delete If the value is in a leaf node, it can simply be deleted from the node, perhaps leaving the node with too few elements; in that case the tree has to be rebalanced Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 50

51 4.4 B-Trees Rebalancing after deletion If some leaf node is under the minimum size, some elements must be redistributed from its siblings to bring all children nodes again up to the minimum (stealing) If all siblings have only minimum size the parent node is affected and has to hand over an element If the parent then falls under the minimum degree, the redistribution must be applied iteratively up the tree Since the minimum element count does not apply to the root, making the root the only deficient node is not a problem Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 51

52 4.4 B-Trees The rebalancing strategy is to find a sibling of the deficient node which has more than the minimum number of elements Choose a new separator, move it to the parent node and redistribute the values in both original nodes to the new left and right children If the sibling node immediately to the right of the deficient node has only the minimum number of elements, examine the sibling node immediately to the left Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 52

53 4.4 B-Trees If both immediate siblings have only the minimum number of elements, create a new node with all the elements from the deficient node, all the elements from one of its siblings, and the separator in the parent between the two combined sibling nodes Remove the separator from the parent, and replace the two children it separated with the combined node. If that brings the number of elements in the parent under the minimum, repeat these steps with that deficient node, unless it is the root, since the root may be deficient Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 53

54 4.4 B-Trees Example: Steal Keys from Siblings Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 54

55 4.4 B-Trees Example: Join Child Nodes Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 55

56 4.4 B-Trees Each element in an internal node acts as a separation value for two subtrees. When such an element is deleted, there are two cases: Both of the two child nodes to the left and right of the deleted element have the minimum number of elements (L-1) and then can then be joined into a legal single node with (2L-2) elements One of the two child nodes contains more than the minimum number of elements. Then a new separator for those subtrees must be found. There are two possible choices: The largest element in the left subtree is the largest element which is still less than the separator The smallest element in the right subtree is the smallest element which is still greater than the separator Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 56

57 4.4 B-Trees Deletion from an internal node If the value is in an internal node, choose a new separator, remove it from the leaf node it is in, and replace the element to be deleted with the new separator This has deleted an element from child node so the deletion has been passed down the tree iteratively If the child is a leaf node the leaf node deletion procedure applies Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 57

58 4.4 B-Trees Example: Build a B-Tree Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 58

4.4 B-Trees Summary Very efficient data structure for disk storage O(log n) for all operations Even better Guaranteed maximum node-accesses to locate a key is Balanced binary tree guarantees only log

59 4.4 B-Trees Summary Very efficient data structure for disk storage O(log n) for all operations Even better Guaranteed maximum node-accesses to locate a key is Balanced binary tree guarantees only log 2 n) log fan out ( n ) Accessing a node is expensive on disks huge improvement No degenerated cases Self-Balancing rarely necessary as most updates affect just one node Wasted space decreased due to guaranteed minimal fill factor EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 59

60 4.4 B*Trees The B*Tree is a constrained B-Tree All non-root nodes need to be filled to 2/3 Implemented in various file systems HFS Raiser 4 Used to be quite popular, but lost its importance EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 60

61 4.4 B + Trees The B + Tree is an optimization of the B-Tree Improved traversal performance Increased search efficiency Increased memory efficiency B + Tree uses different nodes for leaf nodes and internal nodes Internal Nodes: Only unique keys and node links No data pointers! Leaf Nodes: Replicated keys with data pointer Data pointers only here Node Pointer Key Value Key Value Data Pointer EN Node Internal Node Leaf Node Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 61

62 4.4 B + Trees Internal Nodes are used for search guidance A block can contain more keys fan-out higher Leafs just contain data links All leafs are linked to each other in-order for increased traversal performance Internal Search Nodes Data Nodes EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 62

Increased search efficiency Increased memory efficiency EN 14.3.

63 4.4 B + Trees Summary B + Tree is THE super index structure for disk-based databases Improved over B-Tree Improved traversal performance Increased search efficiency Increased memory efficiency EN Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 63

IMDB s Observation Loading data from the hard disk is a major bottleneck Available main memory still doubles every 18 month Moore s Law Idea Store all data in fast main memory!

64 IMDB s Observation Loading data from the hard disk is a major bottleneck Available main memory still doubles every 18 month Moore s Law Idea Store all data in fast main memory! Solutions Use traditional DBMS with huge buffer pool (block cache) DBMS are usually optimized for sequential disk access Design special In-Memory Databases Systems Or MMDB (Main Memory Database) Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 64

65 IMDB s Why do we need in-memory databases? Embedded Systems Mobile Phones PDA s Sensors Diskless Computing Devices Ultra-High-Performance (Real Time) Scenarios Network Applications Telecommunication Applications High-Volume Trading Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 65

66 IMDB s Why should IMDB s be different? Traditional DBMS do also work in-memory but they waste potential Random access has nearly no penalty compared to sequential access Optimizing for linear storage and block read/write unnecessary Type Media Size Random Acc. Speed Transfer Speed Characteristics Price Price/GB Pri DDR3-Ram (Corsair 1600C7DHX) 2 GiB ms 8000 MB/sec Vol, Dyn, Ra, OL Sec Harddrive Magnetic (Seagate ST As) 1000 GB 12 ms 80 MB/sec Stat, RA, OL Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 66

IMDB s Storing a DB in main memory also has problems Main memory usually smaller and more expensive Main memory is not persistent What happens in case of power failure?

67 IMDB s Storing a DB in main memory also has problems Main memory usually smaller and more expensive Main memory is not persistent What happens in case of power failure? How to ensure the durability requirement of DBs? Severe problem for transaction management Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 67

68 IMDB s IMDB Index Structures B-Trees are great, but they are shallow and bushy which is unnecessary in main memory Can save some performance there Hash Indexes are very suitable in main memory for unsorted data Especially bucket chained hashing is very efficient For sorted data: Use the T-Tree instead of B-Tree Specialized tree for main memory databases Blend between AVL-Tree and B-Tree Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 68

69 The T-Tree T-Tree design considerations I/O access is cheap in main memory Expensive resources are computation time and memory space Properties T-Tree is a self-balancing binary tree (AVL algorithm) T-Tree nodes contain only links Each node links to m data records (d 1 d m ) Data entries are ordered, smallest left, biggest right All nodes contain a maximum of c max entries Each internal node contains c min to c max entries (usually c max -c min 2) Each node has a link to it s parent Each node has at most a left and a right subtree Left subtree contains only entries smaller than the minimal node entry Right subtree contains only entries bigger than maximal node entry d 1 p d 2 d m-1 d m l r Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 69

70 The T-Tree Search a key Search similar to Binary Search Tree, but If smaller than node min, go left If bigger than node max, go right Else do bisection search within node Example: Locate 44 Naming: Internal Nodes: 2 Children; Half-Leafs: 1 Child; Leafs: 0 Children p p p 5 12 l r 13 l r l r p 55 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 70 l r

71 The T-Tree Insert a key 1. Locate responsible node for key (node min key node max ) 2. If node contains free space 1. Insert key, restore order 3. Else 1. Replace node min with key, store node min as new insert key m 2. Scan left subtree for node with biggest node min key m 3. If there is such a node 4. Else 1. Insert key m and recursively push down it s smallest element if full 1. Create new node and insert key m 2. Rebalance tree with local rebalancing (as with AVL) if necessary Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 71

72 The T-Tree Insert a key: 16 p p p 5 12 l r 13 l r l r p 55 l r p p p 5 12 l r 13 l r l r p 55 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 72 l r

73 The T-Tree Insert a key: 22 p p p 5 12 l r 13 l r l r p 55 p l r p p l r l r p l r p l r l r Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 73

74 The T-Tree Delete a key 1. Locate responsible node for key (node min key node max ) 2. If node contains key, delete it; else stop 3. If node is internal node and contains less than c min entries 1. Replace it with greatest node min from a sub-leaf/sub-half-leaf. That node is the new working node from now on. 4. If node is a half-leaf and can be merged with a leaf 1. Merge the nodes, delete empty leaf. Goto If node (a leaf) is not empty, stop. Else delete it. 6. Locally Rebalance tree similar to AVL tree. Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 74

75 IMDB Indexes How do main memory index structures compare? Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 75

76 IMDB Indexes How do main memory index structures compare? Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 76

77 IMDB Indexes How do main memory index structures compare? Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 77

78 IMDB Indexes Why not always use Chained Bucked Hashing? No range queries Storage overhead Suboptimal if amount of data is unknown during initialization Why do T-Tree and AVL-Tree perform better than B- Tree and ordered array for search? Bisection search within a B-tree node/array needs to compute position of next comparison AVL and T-Tree do only need 2 comparisons in each node Why does T-Tree perform better than AVL for updates? Due to larger nodes, many updates do not require a rebalancing Why does ordered array suck for updates? Reordering of all elements necessary for each update Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 78

79 References and Timeline AVL-Tree G. Adelson-Velskii, E. M. Landis: An algorithm for the organization of information. Proceedings of the USSR Academy of Sciences 146: (Russian), English translation by M. J. Ricci in Soviet Math. Doklady, 3: , 1962 B-Trees R. Bayer, E. M. McCreight: Organization and Maintenance of Large Ordered Indexes. Acta Informatica 1, , 1972 T-Trees T. J. Lehman, M. J. Carey: A Study of Index Structures for Main Memory Database Management Systems, Int. Conf. On Very Large 12th Database, Kyoto, August 1986 Scapegoat Trees I. Galperin, R. L. Rivest: Scapegoat trees, ACM-SIAM Symposium on Discrete Algorithms, Austin, Texas, US, 1993 Datenbanksysteme 2 Wolf-Tilo Balke Institut für Informationssysteme TU Braunschweig 79

Relational Database Systems 2 4. Trees & Advanced Indexes

Relational Database Systems 2 4. Trees & Advanced Indexes Silke Eckstein Benjamin Köhncke Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 3 Indexing Buffer