Trees. Reading: Weiss, Chapter 4. Cpt S 223, Fall 2007 Copyright: Washington State University

Transcription

1 Trees Reading: Weiss, Chapter 4 1

2 Generic Rooted Trees 2

3 Terms Node, Edge Internal node Root Leaf Child Sibling Descendant Ancestor 3

4 Tree Representations n-ary trees Each internal node can have at most n children Representation #1 Each node stores pointers to all of its children O(n 2 ) space complexity How? Representation #2 Each node stores only two pointers: Leftmost child pointer Right sibling pointer O(n) space How? 4

5 Child-Sibling Representation Space is O(n) What about access run-times? Accessing a child from a parent is not constant time anymore 5

6 Traversals Decide on the tree representation based on how you plan to access/traverse it Access patterns Depth-first Breadth-first Top-down Bottom-up Traversals Pre-order Post-order In-order Eulerian 6

7 Pre-order Traversal (node, left, right) Visit parent first And then recursively traverse each child from left to right Example: Pseudocode for a binary tree: Pre-order(T: root of an input binary tree) BEGIN Print T->contents; IF T is a leaf RETURN; Pre-order (T->left); Pre-order (T->right); END 7

8 Unix Directory - Preorder listing 8

9 Post-order Traversal (left, right, node) First recursively traverse each child from left to right Visit Parent last Example: Pseudocode for a binary tree: Post-order(T: root of an input binary tree) BEGIN IF T is an internal node THEN Post-order (T->left); Post-order (T->right); END IF Print T->Contents; END 9

10 Unix Directory Size - Post-order Listing 10

11 In-order Traversal (left, node, right) Recursively traverse each child from left to right Between each consecutive children s traversal, visit the parent Example: Pseudocode for a binary tree: In-order(T: root of an input binary tree) BEGIN IF T is an internal node THEN Post-order (T->left); Print T->Contents; Post-order (T->right); ELSE Print T->Contents; END IF END 11

12 An Example a b g c f h d Fill up the blanks: e i Pre-order: _ Post-order: _ In-order: _ 12

13 Example: Evaluating Post-fix Expressions a b + c d e + * * ==> In-fix expression: (a + b) * (c * (d + e)) * a + b * c + Q) Does a post-order traversal, in general, always lead to a unique binary tree? d e 13

14 Tree Retrieval from Traversals Example Pre-order: a b c d e In-order: b a d c e What is the tree? (if one exists) 14

15 Tree Retrieval from Traversals Example Post-order: b e d c a In-order: b a e d c What is the tree? (if one exists) 15

16 Tree Retrieval from Traversals Example Pre-order: a b c d Post-order: c d b a What is the tree? (if one exists) 16

17 More Traversal Patterns Depth-first Eg., Eulerian Tour Breadth-first Access Patterns Top-down Bottom-up Eg., Tree accumulations 17

18 Binary Search Tree 18

19 BST: Definition Binary search tree is a binary tree such that: For every node u, the values of the nodes in the left (alternatively, right) subtree of u are strictly less (alternatively, greater) than the value at u = < > 19

20 Examples BST T 1 T 2 Not a BST because of 20

21 21

22 22

23 23

24 BST: Insert() Insert (X: new node, T: Tree) BEGIN IF T==NULL THEN T=X; ELSE IF X < T->element THEN Insert(X,T->left); ELSE IF X > T->element THEN Insert (X,T->right); ELSE Report Duplicate Error; END 24

25 Insertion Example T: before Insert(5, T) T: after 25

26 BST: Deletion 26

27 Deletion of node 4 T: before T: after Removed edge& node 27

28 Deletion of node 2 T: before T: after Removed edge& node 28

29 Run-time Analysis Find (x) Insert (x) Delete (x) Expected O(lg n) O(lg n) O(lg n) Worst-case O(n) O(n) O(n) 29

30 Average Case Analysis of BST Assumption: Every possible input sequence is equally likely Define: height of a node u is the number of levels away from the root Let D(n) = sum of heights of all nodes in a n- node tree Average height = D avg (n)/n ==> this will be the average number of hops required for an access Goal: Compute D avg (n) for an average input 30

31 Average height calculation D(n) = D(i) + D(n-i+1) + n-1 On average: D(n) root D(i)=D(n-i+1)=1/n j=0 n-1 D(j) Therefore: D(n)=2/n j=0 n-1 D(j) + n-1 D(i) D(n-i+1) O(n lg n) Average height = O(log n) i nodes n-i-1 nodes 31

32 A bad binary tree 32

33 Randomly Generated BST 33

34 AVL Trees Height-Balanced Binary Search Tree 34

35 AVL Trees AVL: Adelson-Velskii and Landis How to maintain the tree heightbalanced? Keep both sides of the root roughly balanced 35

36 AVL Trees: Definition An AVL tree is also a binary search tree but with the following additional property: For every node in the tree, the heights of the left and right subtrees can differ at most by 1 Every subtree in an AVL tree is also an AVL tree h l -h r <=1 h l h r 36

37 AVL Not an AVL Both are binary search trees 37

38 An AVL tree with 143 nodes Height=9 38

39 Height Bound of an AVL Tree The height of an n-node AVL tree can be at most 1.44 log (n+2) (not proven here) O(log n) Main idea: Let S(h) be the minimum number of nodes in an AVL tree of height h For the root S(h) = S(h-1) + S(h-2) +1 Similar to Fibonacci numbers 39

40 Operations Insert(x) into an AVL tree Condition: Inserting a new element should not disturb the height property If it does, then restructure the tree until the property is satisfied More involved insert procedure Same for remove/delete 40

41 Run-time Analysis Find (x) Insert (x) Delete (x) Expected O(lg n) O(lg n) O(lg n) Worst-case O(lg n) O(lg n) O(lg n) 41

42 Insert () into an AVL Tree root path The first stage of the algorithm is same as insertion into a binary search tree Rebalancing: x If the tree becomes unbalanced because of the new insertion, then it needs to be fixed Only the nodes that are on the path from the newly inserted node to the root may have their height altered 42

43 Rebalancing through Rotation Height of any of the green nodes could have been altered Take the bottommost such green node, u Rebalance u The rest will be automatically balanced as a consequence root x u 43

44 Rebalancing u Four cases for insertion under u: 1. X is inserted into the left subtree of the left child of u 2. X is inserted into the right subtree of the left child of u 3. X is inserted into the left subtree of the right child of u 4. X is inserted into the right subtree of the right child of u Case 1 & 4 are mirrored equivalents Case 2 & 3 are mirrored equivalents 44

45 Case 1: Single Rotation node to be rebalanced New node Inserted here PS: Case 4 is handled in a mirror-equivalent manner 45

46 Example: Single Rotation for Case 1 Problem node 46

47 Case 4: Single Rotation Problem node 47

48 In-class Exercise Insertion sequence: 3, 2, 1, 4, 6, 7 Incremental build procedure: Start with an empty tree and insert elements, one at a time 48

49 Case 2 Cases 2 & 3: Single Rotation Does NOT work Problem node Problem node Case 3 Reason: We are not lifting subtree Y up! 49

50 Case 2: Double Rotation Case 2 Problem node Y Logic: Lift up subtrees B and C one level without affecting subtrees A and D 50

51 Case 2: Why is it Double Rotation? Problem node k 3 k 3 k 2 k 1 k 2 D Rotate k 2 k 1 D Rotate k 2 k 3 k 3 k 1 A k 2 k 1 C A B C D B C A B 1. Rotate problem node s grandchild with its child 2. Rotate problem node with its new child 51

52 Case 3: Double Rotation Case 3 Problem node Y 52

53 In-class Exercise Insert sequence: 3, 2, 1, 4, 6, 7, 16, 15, 14, 13, 12, 11, 10, 8, 9 53

54 54

55 55

56 56

57 57

58 58

59 B-Trees A Data Structure for Disks 59

60 Data Structures for Disks Hardware Storage capacity Data persistence Data access speeds Primary Storage RAM (main memory), cache ~ >100 MB to 2-3GB Transient (erased after process terminates) ~ a few clock cycles (ie., x 10-9 seconds) Secondary Storage Disk (ie., I/O) Giga (10 9 ) to Terabytes (10 12 ) Persistent (permanently stored) milliseconds (10-3 sec) = Data seek time + read time could be million times slower than main memory read 60

61 Need for Different Disk-based Why? Data Structures BST, AVL trees at best have heights O(lg n) N=10 6 lg 10 6 is roughly disk seeks for each level would be too much! So reduce the height! How? Increase the log base beyond 2 Eg., log is < 9 Instead of binary (2-ary) trees, use m-ary search trees s.t. m>2 61

62 5-ary tree of 31 nodes has only 3 levels Index to the Data Real Data Items stored at leaves as disk blocks 62

63 B+ trees: Definition A B+ tree of order M is an M-ary tree with the following properties: 1. Leaves store the real data items 2. Internal nodes store up to M-1 keys s.t., key i is the smallest key in subtree i+1 3. Root can have between 2 to M children 4. Each internal node (except root) has between ceil(m/2) to M children 5. All leaves are at the same depth 6. All leaves have between ceil(l/2) and L data items, for some L 63

64 B+ tree of order 5 Root Internal nodes Leaves M=5 (order of the B+ tree) L=5 (#data items bound for leaves) 64

65 B+ tree of order 5 Index to the Data Each internal node = 1 disk block Data items stored at leaves Each leaf = 1 disk block 65

66 Node Data Structures Root & internal nodes M child pointers 4 x M bytes Leaf node L data items in the worst-case L x D bytes M-1 key entries (M-1) x K bytes D denotes the size of each data item K denotes the size of a key (ie., K <= D) 66

67 How to choose M and L? Choose M and L based on the size of each data item Example: Let 1 disk block = 8,192 bytes We will denote this by the symbol B So, B = 8 KB Let each data item be an integer So, D = 4 bytes Also, K = 4 bytes 67

68 Example K=4 bytes, B=8 KB How to calculate M? Each internal node needs 4 x M + (M-1) x K bytes Each internal node has to fit inside a disk block So we have B bytes Solving the above: M = floor[ (B+K) / (4+K) ] For K=4, B=8 KB: M = 1,024 Alternatively, if K=32 M=228 68

69 Example D=4 bytes, B=8 KB How to calculate L for leaves? L = floor[ B / D ] For D=4, B = 8 KB: L = 2,048 ie., each leaf has to store 1,024 to 2,048 data items Alternatively, if D=32: L = 256 data items / leaf 69

70 B+ trees: Other Counters Let N be the total number of data items Number of leaves = ceil [ N / L ] to ceil [ 2N / L] Height = O ( log M #leaves) 70

71 Example: Find (81)? - O(log M #leaves) disk block reads - Within the leaf: O(L) or - or even better, O(log L) if data items are kept sorted - we can use binary search within each leaf node - 71

72 B+ tree: Insertion Have to maintain all leaves at the same level before and after insertion Note that this could mean increasing the height of the tree Height vs. Level 72

73 Example: Insert (57) before Insert here, there is space! 73

74 Example: Insert (57) after Next: Insert(55) Empty now So, split the previous leaf into 2 parts 74

75 Example.. Insert (55) after Split parent node There is one empty room here Next: Insert (40) Hmm.. Leaf already full, and no empty neighbors! 75

76 Example.. Insert (40) after Note: Splitting the root itself would mean we are increasing the height by 1 76

77 Example.. Delete (99) before Too few (<3 data items) after delete Will be left with too few keys (<2) after move Borrow leaf from left neighbor 77

78 Example.. Delete (99) after 78