Algorithms. 演算法 Data Structures (focus on Trees)

Transcription

1 演算法 Data Structures (focus on Trees) Professor Chien-Mo James Li 李建模 Graduate Institute of Electronics Engineering National Taiwan University 1 Dynamic Set Dynamic set is a set of elements that can grow or shrink over time Each element is represented by an object of two attributes key: a unique number/character to identify the object satellite data: carried around but not used in set operation Query operations SEARCH MINIMUM MAXIMUM SUCCESSOR PREDECESSOR Modifying operations INSERT DELETE 2

2 Outline Elementary Data Structures, CH10 Hash Table CH 11 * (not in exam) Binary Search Trees, CH12 Red Black Trees, CH13 3 Stack Last In First Out (LIFO) underflow: pop an empty stack overflow: push a full stack S.top S.top S.top 4

3 Stack Operations What are their time complexities? STACK-EMPTY(S) 1 if S.top == 0 2 return TRUE 3 else return FALSE PUSH(S, x) 1 S.top = S.top S[S.top] = x POP(S) 1 if STACK-EMPTY(S) 2 error underflow 3 else S.top = S.top 1 4 return S[S.top + 1] 5 First In First Out (FIFO) Queue leaves from head and enters from tail Fig (b) enqueue 17,3,5 (c) dequeue 15 Q.head Q.tail Q.tail Q.head Q.tail Q.head 6

4 Queue Operations What are their time complexities? ENQUEUE(Q, x) 1 Q[Q.tail] = x 2 if Q.tail == Q.length 3 Q.tail = 1 4 else Q.tail = Q.tail + 1 DEQUEUE(Q) 1 x = Q[Q.head] 2 if Q.head == Q.length 3 Q.head = 1 4 else Q.head = Q.head return x 7 Doubly Linked List An object has three attributes key, prev (pointer) and next (pointer) Fig 10.3 L.head L.head L.head 8

5 List Operations What are their time complexities? LIST-SEARCH(L, k) 1 x = L.head 2 while x NIL and x.key k 3 x = x.next 4 return x LIST-INSERT(L, x) 1 x.next = L.head 2 if L.head NIL 3 L.head.prev = x 4 L.head = x 5 x.prev = NIL LIST-DELETE(L, x) 1 if x.prev NIL 2 x.prev.next = x.next 3 else L.head = x.next 4 if x.next NIL 5 x.next.prev = x.prev 9 Linked List with Sentinels Sentinel is a dummy object for boundary condition simplification sentinel has no key, but has pointers L.nil points to the sentinel Fig 10.4 L.nil L.nil L.nil L.nil 10

6 List Operation w/ Sentinel Sentinel helps to make code easier Does sentinel help to improve complexity? LIST-SEARCH (L, k) 1 x = L.nil.next 2 while x L.nil and x.key k 3 x = x.next 4 return x LIST-INSERT (L, x) 1 x.next = L.nil.next 2 L.nil.next.prev = x 3 L.nil.next = x 4 x.prev = L.nil LIST-DELETE (L, x) 1 x.prev.next = x.next 2 x.next.prev = x.prev 11 Other Lists Singly linked list no previous pointer Sorted list elements are sorted by their keys Unsorted list elements are not sorted Circular list previous of head points to the tail next of tail points to the head 12

7 Outline Elementary Data Structures, CH10 Hash Table CH 11 (NOT in exam) Binary Search Trees, CH12 Red Black Trees, CH13 13 Hash Table Use hashing when universe of keys is very large Hash function: map key k is to slot h(k) Collision: two keys hashed to the same h(k) 14

8 Collision Resolution by Chaining Chaining: place all elements hashed to the same slot in a linked list 15 Outline Elementary Data Structures, CH10 Hash Table CH 11 Binary Search Trees, CH12 Red Black Trees, CH13 16

9 Representation of a Tree Each node x has 4 attributes x.key (and possibly other satellite data) x.left: points to x s left child. x.right: points to x s right child. x.p: points to x s parent. T.root points to the root of tree T ; T.root.p = NIL A node with no children is a leaf (or external node) T.root leaf 17 Binary Search Tree (BST) Binary Search Tree Property If y is in left subtree of x, then y.key x.key If y is in right subtree of x, then y.key x.key Height of node # of edges on longest downward path from the node to a leaf Height of tree = height of root Worst running time to search a BST is proportional to its height Example: (a) height = 2 (b) height =4, (b) is less efficient to search 18

10 Visit nodes in this order left-subtree, root, Right-subtree ITW sorts keys increasingly Example: 2, 3, 5, 5, 7, 8 Inorder Tree Walk Q1: are the results of (a) and (b) the same? Q2: what is time complexity? (see 12.1) 19 Preorder tree walk root, subtree Other Tree Walks Postorder tree walk subtree, root Q1: What is the postorder of the following trees? Q2: What is the order of keys? 20

11 Min. and Max. Always follow the left child minimum=2 Complexity = Θ(h) Always follow the right child maximum=20 Complexity = Θ(h) 21 Path to find 13: Complexity = Θ(h) Searching x.key 22

12 FFT Which version is better, recursive or iterative? x.key x.key x.key = x.left = x.right 23 Successor Assume all keys are distinct, successor of x is y if y.key is the smallest key x.key Successor of x = next key of x in inorder tree walk if x is the largest key, successor is NIL Example successor of 15 is 17 successor of 17 is 18 successor of 4 is 6 successor of 13 is? 24

13 Finding Successor case 1. If node x has non-empty right subtree, x s successor (y) is minimum in x s right subtree example: successor of 15 is 17 case 2. If node x has empty right subtree, x s successor (y) is the lowest ancestor of x y s left child is x s ancestor Example: successor of 4 is 6 // case 1 // case 2 // x==y.left 25 Predecessor Assume all keys are distinct, predecessor of x is y if y.key is the largest key < x.key Predecessor of x = previous key of x in inorder tree walk if x is the smallest key, predecessor is NIL Example predecessor of 15 is 13 predecessor of 7 is 6 predecessor of 17 is? 26

14 Exercises Q1: show successor time complexity = O(h) Hint: Theorem 12.2 Q2: please write predecessor algorithm hint: symmetric to successor 27 Insertion Insert z into BST Beginning at root, trace downward x: traces the downward path looking for NIL to insert z y is trailing pointer, parent of x. When z.key < x.key trace left subtree otherwise, trace right subtree When x is NIL, it is the position for z Compare z s value with y s value, z.key < y.key: insert z at y s left z.key > y.key: insert z at y s right // empty tree Time complexity = Θ(h) 28

15 Insertion Example Insert 14 Insert Deletion Examples z is deleted: (a) z has no child, (b) z has 1 child, (c) z has 2 children 30

16 Deletion (case 1, 2) case 1. If z has no children, just remove z case 2. If z has just one child, child take z s position, dragging the child s subtree along. Fig 12.4 (a) Fig 12.4 (b) 31 Deletion (case 3) case 3. If z has two children, find z s successor y 3a: If y is z s right child, replace z by y (leave y s right child alone) Fig 12.4 (c) 3b. Otherwise, replace y by x (y s right child), replace z by y Fig 12.4 (d) 32

17 Tree-Delete* *this algorithm is different from 2nd ed. case 1+2 case 3 case 3b: replace y by x case 3a: replace z by y 33 Transplant Replace the subtree rooted at u by the subtree rooted at v Time complexity = Θ(1) 34

18 Exercise Case 1: Delete A Case 2: Delete G Case 3a: Delete K Case 3b: Delete B 35 TRANSPLANT = Θ(1) TREE-MINIMUM = Θ(h) other lines = Θ(1) Complexity Analysis over all = Θ(h) 36

19 Why does y not have left child? FFT 37 Summary BST supports all dynamic set operations in Θ (h) SEARCH INSERT DELETE MINIMUM MAXIMUM SUCCESSOR PREDECESSOR CH 12.4 shows the expected tree height is O(lg n) but what about the worst case? is there a smart algorithm that guarantee h=lg(n) in worst case? 38

20 Outline Elementary Data Structures, CH10 Hash Table CH 11 Binary Search Trees, CH12 Red Black Trees, CH13 39 Red-black Tree A red-black tree is a binary search tree plus an attribute color red-black tree is approximately balanced Red-black tree must satisfy five properties: 1. Every node is either red or black 2. Root is black 3. Every leaf (T.nil) is black 4. If a node is red, then both its children are black Hence, no two consecutive reds on a path from root to a leaf 5. All paths from a node to descendant leaves contain the same number of black nodes i.e. black height of a node is unique 40

21 Height of node x, h(x): Black Height number of edges in a longest path from node x to a leaf Black height of node x, bh(x): number of black nodes on the path from node x to leaf, including T.nil not counting x itself T.root 41 Fig Red-Black Tree Example T.nil 42

22 FFT Why R&B tree is (approx. ) balanced? 43 What is T.nil? Sentinel T.nil is a dummy object that represents NIL T.nil has five attributes p, left, right, and key color is black Only one T.nil for the whole tree to save space Use T.nil helps to make boundary condition easier will see later T.nil is usually not plotted in the tree 44

23 Two Claims Claim 1: For any node x, bh(x) h(x)/2 By property 4, fewer than h(x)/2 nodes are red Hence, more than bh(x)/2 are black Claim2: Subtree rooted at node x contains more than 2 bh(x) -1 nodes math induction: 1. x is leaf, bh(x) =0, subtree contains zero nodes 2. Any child of x has black-height either bh(x) (if the child is red) or bh(x)-1 (if the child is black) By hypothesis, each child contains 2 bh(x)-1-1 internal nodes Thus, the subtree rooted at x contains 2(2 bh(x)-1-1)+1 = 2 bh(x) -1internal nodes +1 is x itself 45 Lemma 13.1 A red-black tree with n internal nodes has height h 2 lg (n +1) according to above two claims, at root n h 2 bh( root) 1 2lg( n + 1) 2 h( root) / 2 1 = 2 h / 2 1 Hence, red-black tree is approximately balanced height is at most twice minimum SEARCH, MIN, MAX, PRECEDESSOR, SUCCESSOR all query operations are O(h)= O(lg (n)) how about insertion and deletion? must maintain the red-black tree properties 46

24 Rotation Rotation is a local operation for binary search trees left/right rotations. they are inverse of each other preserve BST property does not change order Example: Fig inorder walk: α, x, β, y, γ unchanged after left/right rotation 47 Left-Rotate // β assume y T.nil y.p x.p time complexity = O(1) 48

25 Left-Rotate Example inorder tree walk unchanged 49 Exercise Please write pseudo code for right-rotate hint: exchange left and right 50

26 Insertion insert node z four differences 51 Keep RB Tree Properties Assign red color to inserted node, z.color = red, any violation? 1. Every node is either red or black OK 2. The root is black OK, unless z is root 3. Every leaf (T.nil) is black OK 4. If a node is red, then both its children are black if z.p is red, two reds can be in a row, violation! how to fixup? check uncle s color 5. For each node, black height is unique OK, why? Color fixup: move the extra red from z up the tree until it reaches the root simply paint the root black suitable rotation and recoloring is performed to ensure all properties are preserved below z 52

27 Case 1: If uncle y is red Fixup: Case 1 paint z s parent (A) and uncle (D) black paint z s gradparent (C) red move new z up two levels keep checking upwards Fig Fixup: Case 2, 3 Case 2: If uncle y is black and z is a right child left rotate around z.p (A) turn into case 3 Case 3: If uncle y is black and z is a left child paint parent z.p (B) black and grandparent z.p.p (C) red right rotate on z.p.p (C) no more iterations Fig

28 Exercise Try to insert Too Many Cases? How can I remember so many cases? 55 Fixup Algorithm // y is uncle 56

29 Loop Invariant At the start of each iteration of the while loop, z is red There is at most one red-black violation: Property 2: z is a red root, or Property 4: z and z.p are both red. Initialization: We already seen why the loop invariant holds initially. Termination: The loop terminates because z.p is black. Hence, property 4 is OK. Only property 2 might be violated, the last line fixes it. Maintenance: When we start the loop body, the only violation is property 4 need to consider 6 cases 3 cases are symmetric 57 Complexity of RB-Insert RB-Insert-Fixup O(lg n) no more than two rotations RB-Insert O(lg n) 58

30 RB-Insert-Fixup Example Fig Exercise Please finish the other 3 cases of RB-INSERT-FIXUP algorithm 60

31 Tree-Delete (Review) case 1 (z has no child): remove z case 2 (z has one child): replace z by its own successor case 3 (z has two child): y = z s successor 3a: If y is z s right child, replace z by y 3b. Otherwise, replace y by x then replace z by y 61 RB-Delete case1,2 z has < 2 child y=z x=y s child *different from 2nd ed. y x x x y x Because y(=z) is removed, y s original color is gone color fixup from node x 62

32 RB-Delete case 3 z has 2 children y = z s successor x = y s successor replace y by x y inherits z s color, y s original color is gone color fixup from node x replace z by y y inherits z s color 63 Delete Fixup If y originally was red, no violation why? If y originally was black, push the black onto its only child x what if x is already black? it becomes doubly black violation! Color fixup: move the doubly black from x up the tree until 1. x points to a red node, simply color it black 2. x points to the root, simply remove doubly black suitable rotation and recoloring is performed to ensure all properties are preserved below x 64

33 Case 1: x s sibling w is red w must have black children. Fixup Case 1 Make w (D) black and x.p (B) red. Left rotate on x.p (B). x s new sibling (C) is black new w is now C Go immediately to case 2, 3, or 4 Fig Fixup Case 2 Case 2: w is black and both w s children are black Take one black off x (doubly black singly black) and also take one black off w (black red). Move that one black up to x.p (B) new x is now B If B is red, color it black and terminates otherwise, keep going up and do the next iteration 66

34 Fixup Case 3 Case 3: w is black, w s left child (C) is red and right child (E) is black Make w (D) red and w s left child (C) black Then right rotate on w (D). New sibling of x (C) is black with a red right child (D) new w is now C becomes case 4 67 Fixup Case 4 Case 4: w is black, w s right child (E) is red, w s left child (C) is don t care Make w (D) be x.p s (B) color (D.color = B.color) Make x.p (B) black and w s right child (E) black Left rotate on x.p (B) Remove the extra black on x Setting x to root causes the loop to terminate 68

35 Too Many Cases? How can I remember so many cases? Any real life example? 69 70

36 Complexity of RB-Delete O(lg n) time to get through RB-DELETE up to RB-DELETE-FIXUP. Within RB-DELETE-FIXUP: Case 2 is the only case in which more iterations occur. x moves up 1 level, O(lg n) iterations. Each of cases 1, 3, and 4 has 1 rotation less than 3 rotations in all. Totally,O(lg n) time. 71 FFT in case 3a, since x = y.right x.p=y is redundant? redundant??? 72

37 RB-Transplant Make two differences for RB tree RB-TRANSPLANT references T.nil v.p = u.p, even if v is the sentinel T.nil can trace from T.nil up to its parent correctly fixup exploits this feature 73 Other Self-balancing BST The idea of balancing a BST is due to Adel son Verl skii and Landis AVL tree AA-tree Treaps Splay tree 74

38 Reading CH10 (optional) CH 12, CH 13 Appendix B.5 RB-tree 的 Demo 程式 可以一步一步 demo 如何加入及删除 好玩! 請試試看 75