Data Structures and Algorithms(12)

Transcription

1 Ming Zhang "Data s and Algorithms" Data s and Algorithms(12) Instructor: Ming Zhang Textbook Authors: Ming Zhang, Tengjiao Wang and Haiyan Zhao Higher Education Press, 28.6 (the "Eleventh Five-Year" national planning textbook)

2 Chapter 12 Advanced data structure 12.1 Multidimensional Array 12.2 Generalized Lists 12.3 Storage management 12.4 Trie 2 Ming Zhang Data s and Algorithms"

3 12.3 Trie Ideal situation: The average time of insertion, deletion, and search is O(logN) Input 9, 4, 2, 6, 7, 15, 12, 21 2 Output 2, 4, 6, 7, 9, 12, 15, Ming Zhang Data s and Algorithms"

4 12.4 Trie of Trie Space division of key trie comes from retrieval Application Information retrieval Large scale of English dictionary 26-branch Trie Binary Trie Letters (numbers) represented as binary coding Coding includes just and 1 4 Ming Zhang Data s and Algorithms"

5 12.4 Trie Tree of English words: 26-branch Trie Store words and, ant, bad, bee Subtree an contains set {and, ant} that every word from the set has the same prefix an. n a a b e d t d e and ant bad bee A subtree contains the words with the same prefix 5 Ming Zhang Data s and Algorithms"

6 12.4 Trie Store words an, and ant, bad, bee a n * t d an * * and 6 b a e * * bad bee ant Ming Zhang Data s and Algorithms"

7 12.4 Trie Compact the Single Paths close to the leaf Store words an and ant bad bee a b n a e * an t d and ant bad bee 7 Ming Zhang Data s and Algorithms"

8 12.4 Trie Binary Trie Elements are (<32) 1 (>32) (<16) 1 (>16) 1 (>48) (<8) 1 (>8) (<48) 17 1 (>4) 63 (<4) 1 (>4) 9 1 (>44) (<44) Ming Zhang Data s and Algorithms"

9 12.4 Trie PATRICIA xxxxx 1xxxxx 1 1 xxxx 1xxxx 1xxxx 11xxxx Compression xxx 1xxx 11xxx xx 111xx xx 1xx Code: 2:1 5:11 9:11 17:11 41:111 45: : Ming Zhang Data s and Algorithms"

10 12.4 Trie Characteristics of PATRICIA Tree The compressed PATRICIA tree is a full binary tree Every internal node represents a 1-bit comparison Always at least two children are generated The number of comparisons will not exceed the length of the key 1 Ming Zhang Data s and Algorithms"

11 ab b a Chapter Trie explicit states a b b c b a c c c b Suffix Trees implicit states ab b ab b abc c abc T = ababc c c c Ascending Order ababc abc abc bc c c abc c bc abc bc ababc babc Suffix Trie 11 ababc babc Suffix Tree Ming Zhang Data s and Algorithms"

12 5 ALAM$ 1 ALAYALAM$ 7 AM$ 3 AYALAM$ 6 LAM$ 2 LAYALAM$ MALAYALAM$ 8 M$ 4 YALAM$ 9 $ 12.4 Trie M A L A Y A L A M $ Suffix Array Suffix Array The longest common prefix array Suffix 5 and Suffix 1 share ALA Suffix 1 and Suffix 7 share A LCP always adjacent 12 Ming Zhang Data s and Algorithms"

13 12.4 Trie 5 ALAM$ 1 ALAYALAM$ 7 AM$ AYALAM$ 6 LAM$ 2 LAYALAM$ MALAYALAM$ 8 M$ 4 YALAM$ 9 $ D = 3 SA lcp 13 D = LA D = 2 D = Ming Zhang Data s and Algorithms"

14 12.4 Trie Discussions Can Trie handle Chinese characters? What about PATRICIA Trie structure? Learn related document about Suffix Array and Suffix Tree. And think about their applications. 14 Ming Zhang Data s and Algorithms"

15 Chapter 12 Advanced data structure 12.1 Multidimensional Array 12.2 Generalized Lists 12.3 Storage management 12.4 Trie Balanced binary search tree Concept and inserting operation of AVL tree Deleting operation and efficiency analysis of AVL tree Splay Tree 15 Ming Zhang Data s and Algorithms"

16 AVL The performance of BST operations are affected by the input sequence Best O(log n); Worst O(n) Adelson-Velskii and Landis AVL tree, a balanced binary search tree Always O(log n) Ming Zhang Data s and Algorithms"

17 Single Rotation AVL Swap the node with its father, while keeping the property of BST y zig x x C A y A B B C zag 17 Ming Zhang Data s and Algorithms"

18 AVL Single Rotation and Double Rotation: Keep the BST property Ming Zhang Data s and Algorithms"

19 AVL Equivalent rotation: Keep the BST property Ming Zhang Data s and Algorithms"

20 AVL Empty tree is allowed The height of AVL tree with n nodes is O(log n) If T is an AVL tree Then the left and right subtree of T: T L, T R are also AVL trees And h L -h R 1 h L, h R are the heights of its left and right subtree. 2 Ming Zhang Data s and Algorithms"

21 Examples of AVL tree T1 T4 T2 T3 Ti-1 Ti-2 Ti 21 Ming Zhang Data s and Algorithms"

22 Balance Factor Balance Factor,bf (x): bf (x) = height x rchild height(x lchild ) Balance Factor might be, 1 and Ming Zhang Data s and Algorithms"

23 Insertion in an AVL tree Just like BST: insert the current node as a leaf node Situations during adjustment The current node was balanced. Now its left or right subtree becomes heavier. Modify the balance factor of the current node The current node had a balance factor of ±1. Now the current node becomes balanced. Height stays the same. Do not modify. The current node had a balance factor of ±1. Now the heavier side becomes heavier Unbalanced dangerous node 23 Ming Zhang Data s and Algorithms"

24 Rebalance Become unbalanced after inserting 17 Adjustment 24 Ming Zhang Data s and Algorithms"

25 Unbalanced situation occurs after insertion Insert the current node as leaf node in BST Assume a is the most close node to the current node. And its absolute value of balance factor is not zero. The current node s with key is in its left subtree or its right subtree. Assume that it is inserted into the right subtree. The original balance factor: (1) bf(a) = 1 (2) bf(a) = (3) bf(a) = Ming Zhang Data s and Algorithms"

26 Assume a is the most close node to the current node s. And its absolute value of balance factor is not zero. S is in a s left subtree or right subtree. Assume S is in the right subtree. Because balance factors of nodes in paths from s to a change from to +1. So as for node a: 1. bf(a) = 1, then bf(a) =, and the height of node a s subtree stays the same. 2. bf(a) =, then bf(a) = +1, and the height of node a s subtree stays the same. Because of the definition of a (bf(a) ), we can know that node a is the root. 3. bf(a) = +1, then bf(a) = +2, and adjustment is needed. 26 Ming Zhang Data s and Algorithms"

27 Unbalanced Cases The balance factors of any nodes must be, 1, 1 a s left subtree was heavier, bf(a) = 1, and bf(a) become -2 after insertion. LL: insert into the left subtree of a s left child. Left heavier+left heavier, bf(a) become 2 LR: insert into the right subtree of a s left child. Left heavier+right heavier, bf(a) become 2 Likewise, bf(a) = 1, and bf(a) become 2 after insertion RR: the node that causes unbalanced is in the right subtree of a s right child. RL: the node that causes unbalanced is in the left subtree of a s right child. 27 Ming Zhang Data s and Algorithms"

28 Unbalanced Cases a -2 a 2 b -1 h h 1 b h+1 h h h+1 LL RR 28 Ming Zhang Data s and Algorithms"

29 Summary of unbalanced cases LL is symmetry with RR, and LR is symmetry with RL. Unbalanced nodes happen on the path from inserted node to the root. Its balance factor must be 2 or 2 If 2, the balance factor before insertion is 1 If 2, the balance factor before insertion is 1 29 Ming Zhang Data s and Algorithms"

30 LL single rotation a -2 b -1 h h+1 h T 3 T1 T 2 3 Ming Zhang Data s and Algorithms"

31 Insight of Rotations Take RR for instance, there are 7 parts Three nodes: a, b, c Four subtrees T, T 1, T 2, T 3 The structure will not change after making c s subtree heavier. T 2, c, T 3 could be regarded as b s right subtree. Goal: construct a new AVL structure Balanced Keep the BST property T a T 1 b T 2 c T 3 31 Ming Zhang Data s and Algorithms"

32 Double Rotation RL or LR needs double rotation. They are symmetry with each other We discuss about RL only LR is the same. 32 Ming Zhang Data s and Algorithms"

33 First step of RL double rotation h T 2 a First step -1 c 1 Or -1 b h Height of a s subtree is h+2 before inserting Height of a s subtree is h+3 after inserting h-1 /h T 1 h/ h-1 T 2 T 3 33 Ming Zhang Data s and Algorithms"

34 Second step of RL double rotation Balance factor is meaningless in the middle status T h a 2 Balance factor of a is -1 or Balance factor of b is or 1 h-1 /h T 1 RL 第一步 RL 第二步 1 c h/ h-1 T 2 1 b h T 3 34 Ming Zhang Data s and Algorithms"

35 Insight of Rotations Doing any rotations (RR, RL, LL, LR) New tree keeps the BST property Few pointers need to be modified during rotations. Height of the new subtree is h+2, and heights of subtrees before insertion stay same Rest parts upon node a (if not empty) are always balanced 35 Ming Zhang Data s and Algorithms"

36 AVL tree after inserting word:cup, cop, copy, hit, hi, his and hia cup -1-2 cop 1 copy Unbalanced after inserting copy LR double rotation 36 Ming Zhang Data s and Algorithms"

37 AVL tree after inserting word:cup, cop, copy, hit, hi, his and hia copy 2 1 cop cup 12 hit -1 hi 37 Ming Zhang Data s and Algorithms"

38 AVL tree after inserting word:cup, cop, copy, hit, hi, his and hia copy RL double rotation cop cup hit hi 38 Ming Zhang Data s and Algorithms"

39 AVL tree after inserting word:cup, cop, copy, hit, hi, his and hia copy 12 cop hi 1 cup hit -1 his 39 Ming Zhang Data s and Algorithms"

40 AVL tree after inserting word:cup, cop, copy, hit, hi, his and hia copy RR single rotation cop hi cup hit his 4 Ming Zhang Data s and Algorithms"

41 AVL tree after inserting word:cup, cop, copy, hit, hi, his and hia hi 1 copy hit -2-1 cop cup his -1 hia 41 Ming Zhang Data s and Algorithms"

42 AVL tree after inserting word:cup, cop, copy, hit, hi, his and hia LL single rotation hi copy hit cop cup his hia 42 Ming Zhang Data s and Algorithms"

43 Discussions Can we modify the definition of balance factor of AVL tree? For example, allow the height difference as big as 2. Insert 1, 2, 3,..., 2 k -1 into an empty AVL tree consecutively. Try to prove the result is a complete binary tree with height k. 43 Ming Zhang Data s and Algorithms"

44 Chapter 12 Advanced data structure 12.1 Multidimensional Array 12.2 Generalized Lists 12.3 Storage management 12.4 Trie Balanced binary search tree Concept and insertion operation of AVL tree Deletion operation and efficiency analysis of AVL tree Splay Tree 44 Ming Zhang Data s and Algorithms"

45 Deletion in an AVL Tree Deletion is the reverse operation of insertion. Deletion in an AVL tree is almost the same with the deletion of BST. Deletion in an AVL tree is a little bit complicated. 45 Ming Zhang Data s and Algorithms"

46 Adjustment after Deletion Conditions of adjustment Height and balance factor has changed. Adjust these changes from the current node to the root Balance factor needs to be modified Modify it If modification is not needed then stop the adjustment Use bool variable modified to mark, initialize it with TRUE when modified = FALSE, stop the adjustment. 46 Ming Zhang Data s and Algorithms"

47 Case 1 Balance factor of the current node a is Its left or right subtree is cut, balance factor is 1 or -1 modified = FALSE Modification will not influent the nodes above it. So the adjustment is over. 1 delete h h h-1 h 47 Ming Zhang Data s and Algorithms"

48 Case 2 Balance factor of the current node is not, but the taller subtree is cut. Modify its balance factor to modified = TRUE Keep modifying upward -1 h+1 h Height reduced by1 h h delete 48 Ming Zhang Data s and Algorithms"

49 Case 3.1 Balance factor of the current node a is not zero. And the shorter subtree is cut. So node a isn t balanced. Assume the root of the taller subtree is b. There are three cases. case 3.1:balance factor of b is zero Single rotation modified = FALSE a b 1-1 delete h h b h zag h-1 a 1 h h 49 Ming Zhang Data s and Algorithms"

50 Case 3.2 Case 3.2: balance factor of b is equal to that of a. Single rotation Balance factors of a and b change to zero modified =TRUE delete h 1 a b zag 1 h-1 h h-1 a b h-1 h 5 Ming Zhang Data s and Algorithms"

51 Case 3.3 Case 3.3: balance factor of b is opposite to that of a Double rotation, rotate b and c, then rotate c and a Because of balance factor of the new root is Other nodes need adjustment Besides modified = TRUE delete h h-2 /h-1 1 c a -1 h-1 /h-2 b h-1 Double rotation Height reduced by one c a b h-1 h-1 h-1 or h-2 51 Ming Zhang Data s and Algorithms"

52 Series of adjustments after deletion Series of adjustments Ancestor nodes may be unbalanced after the adjustment Keep adjusting. These adjustments might pass to the root. Find the grandfather of the node deleted just now. Start single rotation or double rotation Rotation times is O(log n) 52 Ming Zhang Data s and Algorithms"

53 Examples of deletion d -1 1 b g -1 k n i m -1 1 d -1 l b g k n -1-1 i -1 a c f h j l e -1 a c f h j (a) Delete node m, replace m with node I (case 1) e 53 Ming Zhang Data s and Algorithms"

54 Examples of deletion d b g -1 k -1 n i l d -1 1 b g k -1-1 i -2 l a c f h j e (b) delete n (Case 3.2) -1 a c f h j Ming Zhang Data s and Algorithms" e (c) Regard l as the root, doing LL single rotation (Case 3.2)

55 Examples of deletion a b d c 1 g -1-1 f 55-2 h i k e (d) After LL single rotation, then adjust its father node I, doing LR double rotation (Case 3.3) j l b d Ming Zhang Data s and Algorithms" f -1 g h 1 i a c e j k (e) Adjustment is done, AVL tree become balanced l

56 Height of AVL tree Height of AVL tree with n nodes must be O(log n) The maximum height of AVL tree with n nodes is no more than K log 2 n K is a small factor AVL tree that is the most close to unbalanced Construct a series of AVL tree T 1, T 2, T 3, 56 Ming Zhang Data s and Algorithms"

57 Ti s height is i T1 T2 T3 Any other AVL trees with height I have more nodes than T i Ti Ti-1 Ti-2 T4 T i is the most unbalanced AVL tree. Deleting any node will cause it to be unbalanced. 57 Ming Zhang Data s and Algorithms"

58 Proof of height We can see that: t(1) = 2 t(2) = 4 t(i) = t(i-1) + t(i-2) + 1 (t(i)+1) = (t(i-1)+1) + (t(i-2)+1) As for i>2 these relations are very similar to Fibonacci numbers: F() = F(1) = 1 F(i) = F(i-1) + F(i-2) 58 Ming Zhang Data s and Algorithms"

59 Proof of height As for i>l, we have t (i) = F (i+3) - 1 Fibonacci number has this formula F( i) 1 5 i So the approximate formula is,here t(i) i Ming Zhang Data s and Algorithms"

60 Proof of height (result) Relation between height I and the number of nodes t(i) i 3 5(t(i) 1) Use formula log φ X = log 2 X / log 2 φ and log 2 φ.694, calculate the approximate upper bound. t (i) = n i 3 log 5 log (t(i) 1) So height of AVL tree with n nodes must be O (log n) 3 i log 2 (n 1) Ming Zhang Data s and Algorithms"

61 Efficiency of AVL tree Time cost of searching, inserting and deleting is O(1og 2 n) Height of AVL tree with n nodes must be O(log n) AVL tree is fit to data of small scale in memory For large scale data stored in external memory B tree or B+ tree 61 Ming Zhang Data s and Algorithms"

62 Discussion Compare AVL tree with black-red tree, which is better? Height of tree in the worst case Efficiency of operation in statistics Which one is easier to implement? 62 Ming Zhang Data s and Algorithms"

63 Chapter 12 Advanced data structure 12.1 Multidimensional Array 12.2 Generalized Lists 12.3 Storage management 12.4 Trie Balanced binary search tree Splay tree 63 Ming Zhang Data s and Algorithms"

64 Splay Tree A self-organized data structure Adjust data during search Dictionary of Chinese input method Splay tree is not a new data structure. It just adds some rules to BST to improve efficiency. Make sure the total cost of search is not high and achieve a satisfactory performance. Heights of tree are not balanced. 64 Ming Zhang Data s and Algorithms"

65 Splay Tree Single Rotation Swap the current node and its father. Keep the BST property y zig x x C A y A B zag B C 65 Ming Zhang Data s and Algorithms"

66 Splay Tree Double Rotation: LL and RR. Keep the BST property. z y D x C A B zig zag x A y B z C D 66 Ming Zhang Data s and Algorithms"

67 Splay Tree A Double Rotation: LL and RR. Keep the BST property. zig-zag zag-zig z y D x B C x y z A B C D A B x y z C D 67 Ming Zhang Data s and Algorithms"

68 Splaying Visit a node (e.g., node x). Then do an operation called splaying. After inserting or visiting node x, splay (move) node x to the root. After deleting node x. Splay (move) the father of node x to the root. Like AVL tree. The operation of splaying x consists of a series of rotations. Adjust x, father of x and grandfather of x. Move x toward the root. 68 Ming Zhang Data s and Algorithms"

69 Single rotation x is the child of the root Swap x and the father of x Keep the BST property X, y is marks of inner node not value. A, B, C represent subtree. y x C A B 69 Ming Zhang Data s and Algorithms"

70 Double rotation Double rotation involves: Node x The father of node x (called y) The grandfather of node x (called z) Rise node x by two (height). zigzig rotation Also called homogeneous configuration zigzag rotation Also called heterogeneous configuration 7 Ming Zhang Data s and Algorithms"

71 ZigZig y z D Node x is the left child of y Node y is the left child of z x C A B 71 Ming Zhang Data s and Algorithms"

72 ZigZig y x z A B C D 72 Ming Zhang Data s and Algorithms"

73 z ZigZag y D Node x is the right child of y Node y is the left child of z A B x C 73 Ming Zhang Data s and Algorithms"

74 ZigZag z x D y C A B 74 Ming Zhang Data s and Algorithms"

75 ZigZag x y z A B C D 75 Ming Zhang Data s and Algorithms"

76 Different results of ZigZig and ZigZag ZigZag Move the current node toward the root The height of its subtree decreased by one Make the tree more balanced ZigZig The height of the subtree will not change usually Move the current node toward the root 76 Ming Zhang Data s and Algorithms"

77 Adjustment of Splay Tree A series of rotations Until node x become root or the child of root. If the node x is the child of root Single rotation will make x become root This process makes tree more balanced Make nodes, which are frequently visited, closer to the root Reduce the cost of search 77 Ming Zhang Data s and Algorithms"

78 Adjustment of Splay Tree h A f g e H I (a-b-c) ZagZag B d c G (a, c)zag C b (a, b)zag D a E F 78 Ming Zhang Data s and Algorithms"

79 Adjustment of Splay Tree h g I A f d e H G a-d-e ZagZig B a b F c E C D 79 Ming Zhang Data s and Algorithms"

80 Adjustment of Splay Tree (a, g) Zig f g h H I (a-f-g) ZagZig A a (a, f) Zag d e B b F G c E C D 8 Ming Zhang Data s and Algorithms"

81 Single rotation Adjustment of Splay Tree (a, h) Zag a h I f g A d e H B b F G c E C D 81 Ming Zhang Data s and Algorithms"

82 Adjustment of Splay Tree a f h A d g I B b e H c E F G C D 82 Ming Zhang Data s and Algorithms"

83 Splay Tree struct TreeNode { int key; ELEM value; TreeNode *father, * left, *right; }; Splay(TreeNode *x, TreeNode *f);// Splay x to the child of f Splay(x, NULL); // Splay x to the root Find(int k) ; Insert(int k); Delete(TreeNode *x) ; // delete node x DeleteTree(TreeNode *x); // delete subtree x 83 Ming Zhang Data s and Algorithms"

84 void Splay (TreeNode *x, TreeNode *f) { while (x->parent!= f) { TreeNode *y = x-> parent, *z = y-> parent; } if (z!= NULL) { } } else { if (z->lchild == y) { if (y->lchild == x) { Zig(y); Zig(x); } // ZigZig else { Zag(x); Zig(x); } } else { if (y->lchild == x) // grandfather is null // Zag x then Zig x { Zig(x); Zag(x); } // Zig x then Zag x else { Zag(y); Zag(x); } // ZagZag if (y->lchild == x) Zig(x); else Zag(x); } } if (x->parent == NULL) Root = x; // Zig // Zag 84 Ming Zhang Data s and Algorithms"

85 Delete all nodes range from u to v Splay u to the root Splay v to the right child of u Delete the left subtree of v void DeleteUV(TreeNode* rt, TreeNode* u, TreeNode* v ) { Splay(u, NULL); Splay(v, u); DeleteTree(v->lchild); v->lchild = NULL; } Ming Zhang Data s and Algorithms"

86 Efficiency of Splay tree The number of nodes is N m operations (insertion, deletion and search). when m n the total time cost is O(m logn) Single operation may not be efficient The average time cost is O(log n) 86 Ming Zhang Data s and Algorithms"

87 ZigZig A * Semi-Splay Tree z y D x C B ==> * A x y B z C D Semi- ZigZig A * x y B z C D ==> * x A B y z C D Next rotation start from y not x 87 Ming Zhang Data s and Algorithms"

88 Discussion What are the applications of splay tree? Comparisons among red-black tree, AVL tree and Splay tree Visiting frequency? Relationship between tree structure and input data? Which is better from the statistics? Which is easier to implement? 88 Ming Zhang Data s and Algorithms"

89 Ming Zhang Data s and Algorithms Data s and Algorithms Thanks the National Elaborate Course (Only available for IPs in China) Ming Zhang, Tengjiao Wang and Haiyan Zhao Higher Education Press, 28.6 (awarded as the "Eleventh Five-Year" national planning textbook)