Round 3: Trees. Tommi Junttila. Aalto University School of Science Department of Computer Science. CS-A1140 Data Structures and Algorithms Autumn 2017

Transcription

1 Round 3: Trees Tommi Junttila Aalto University School of Science Department of Computer Science CS-A1140 Data Structures and Algorithms Autumn 2017 Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

2 Contents Trees in general Mathematical definitions Traversals (pre, post, in, level) Heaps and priority queues Disjoint-sets forests and union-find Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

3 Material in Introduction to Algorithms, 3rd ed. (online via Aalto lib): Binary trees: Appendix B5 Representing trees: Section 10.4 Heaps: Chapter 6 Union-find: Chapter 21 Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

4 Similar material elsewhere: Union-find: Section 1.5 in Algorithms, 4th ed. Heaps in the OpenDSA book Union-find in the OpenDSA book External links: MIT OCW video on heaps and heap sort Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

5 Trees Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

6 We have already seen trees many times: Recursion trees in Programming 2 and in this course Parse trees of formulas in the form of object diagrams in the Expressions part of the recursion round of Programming 2... A call tree of quicksort input partition(0,7) quicksort(0,4) quicksort(6,7) partition(0,4) partition(6,7) quicksort(2,4) partition(2,4) quicksort(2,3) 17 7 partition(2,3) 7 17 An object diagram for a parse tree of the expression (2.0x + y) Num value = 2.0 Multiply left right Negate expr Add left right Var name = "x" Var name = "y" Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

7 Trees are a very common concept and data structure in computer science In this round, we ll take a closer look at trees 1 mathematically, and 2 as an underlying data structure for other data structures and algorithms For the second point above, we ll see how to implement priority queues with heaps, which are trees having some special properties, and handle disjoint sets with union operations by using trees Trees are further discussed and used in following rounds as well Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

8 Basic definitions Trees are a special subclass of graphs 1 An undirected graph is a pair (V,E), where V is a (usually finite) set of vertices (also called nodes, and E is a set of edges between the vertices, i.e. a set of pairs of form {u,v} such that u,v V and u v 2 Example Consider the graph (V,E) with the vertices V = {a,b,c,d,e,f,g,h}, and the edges E = {{a,b},{a,c},{a,e},{a,f}, {b,f},{e,f},{c,d},{c,g},{d,g}}. It is illustrated graphically on the right with vertices drawn as circles and edges as lines between the vertices e f a c b g d 1 We ll consider graphs and graph algorithms in more detail later in the course 2 Self-loops, i.e. edges between vertex and itself, are forbidden in undirected graphs Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

9 A path of length k from a vertex v 0 to a vertex v k is a sequence (v 0,v 1,...,v k ) of k + 1 vertices such that {v i,v i+1 } E for each i = 0,1,...k 1 A vertex v is reachable from a vertex u if there is a path from u to v. Note that each vertex is reachable from itself as there is a path of length 0 from the vertex to itself A path is simple if all the vertices in it are distinct The graph is connected if every vertex in it is reachable from all the other vertices Example The graph on the right has a (non-simple) path (a,c,d,g,c,d) from a to d e c g has a simple path (a,c,d) of length 2 from a to d a d is connected but deleting the edge {a,c} would make it disconnected f b Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

10 A cycle is a path (v 0,v 1,...,v k ) with k 3 and v 0 = v k Example A cycle (v 0,v 1,...,v k ) is simple if v 1,v 2,...,v k are distinct A graph is acyclic if it does not have any simple cycles The graph on the left is not acyclic as it contains, among many others, a simple cycle (a,e,f,a) The graph on the right is acyclic e c g e c g a d a d f b f b Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

11 Trees and forests A free tree is a connected, acyclic, and undirected graph We usually omit free and just speak of trees An undirected graph that is acyclic but not necessarily connected is called a forest Example The graph on the left is a free tree The graph on the right is a forest of two trees e c g e c g a d a d f b f b Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

12 Theorem (B.2 in Introduction to Algorithms, 3rd ed. (online via Aalto lib)) If G = (V,E) is an undirected graph, then the following statements are equivalent: G is a free tree Any two vertices in G are connected by a single simple path G is connected but removing any edge from it makes it disconnected G is connected and E = V 1 G is acyclic and E = V 1 G is acyclic but adding any edge to it would make it to contain a cycle Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

13 Rooted trees A rooted tree is a free tree where one vertex is a special one called the root of the tree Example The figure below shows a rooted version of our earlier free tree when the root node is c. When we draw rooted trees, the root is always drawn in at the same place, usually at top. c d g a b e f Again, we speak only of trees instead of rooted trees when it is clear from the context that the trees are rooted Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

14 Take a rooted tree with the root node r Any node y on the unique simple path from r to a node x is an ancestor of x If y is an ancestor of x, then x is a descendant of y If y is an ancestor of x and x y, then y is a proper ancestor of x and x is a proper descendant of y If y is a ancestor of x and {y,x} is an edge, then y is the unique parent of x and x is a child of y If two nodes have the same parent, they are siblings The root has no parent and a node with no children is a leaf The leaf nodes are also called external nodes and nonleaf ones internal nodes Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

15 Example On the rooted tree on the right, the ancestors of e are e, a and c the descendants of a are a, b, e and f the parent of a is c and its children are b and e the leaves are d, g, b and f c d g a b e f Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

16 The subtree rooted at a node x is the tree induced by the descendants of x The length of the simple path from the root to a node x is the depth of x A level consists of all nodes the the same depth The height of a node x is the length of the longest simple path from it to any leaf node The height of the tree is the height of its root node The degree of a node in the tree is the number of children it has Example On the rooted tree on the right, the subtree rooted at a contains the nodes a, b, e and f the depth of b is 2 the nodes at the level 1 are d, g and a the height of a is 2 and the degree is 2 the height of the tree is 3 c d g a b e f Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

17 An ordered tree is a rooted tree where the children of each node are ordered That is, if a node has k children, then one of them is the first one, another is the second one and so on Example The rooted trees below are the same if we interpret them as unordered but different if we treat them as ordered trees c c d g a g a d b e e b f f On the tree on the left, the first child of c is d, the second is g and the third is a Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

18 Binary trees A special class of ordered trees where each node has at most two children, left and right, and either of them or both can be missing This different from 2-ary ordered tree (an ordered tree with maximum degree of at most 2) because if a node has one child, it makes a difference whether it is the left or the right child Example The trees below are binary trees (left child is drawn on the left from the parent, right on the right) They are different binary trees but would be the same if considered as ordered trees c c d a d a b e b e f Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62 f

19 A binary tree is full if each node in it is either a leaf, or an internal node with exactly two children. A binary tree is complete if all the leaves have the same depth Example The tree on the left is neither full or complete The tree in the middle is full but not complete The tree on the right is complete c c c d a d a d a b e b e i m b e f g f k l n o h j g f Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

20 A complete binary tree of height h has 2 h leaves, and 2 h 1 internal nodes Exercise: prove the above claim by using induction on the height of the tree A complete binary tree with n leaves has height log 2 n Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

21 Data structures for trees Different tree types can be represented in different way Different algorithms on trees need different information, e.g. some need to traverse trees downwards and thus need to find the children of a node but some only need the parent information Thus no single best data structure for trees in general Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

22 One generic way to represent binary trees is Example to store the nodes as objects each object having a reference to the parent node (if the node is root, either null or the node itself) the left and right children (null is missing) some key/value/etc as we usually use trees to store/reference other data key parent left child right child Using the notation for nodes, the object diagram for representing the tree on the left is shown on the right. Here we are not interested in node names but show the keys/values associated to the nodes inside them also in the tree on the left Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

23 As a second example, one way to represent trees in which nodes can have unbounded amount of children is to have each node reference to Example the first child, and the next sibling key parent first child next sibling Using the representation on the left is shown on the right. 12 for nodes, the object diagram for the tree Random access to ith child of a node with n children requires Θ(n) time in the worst case but iterating over all children is easy Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

24 Traversing trees In many situations it is required to traverse all the nodes in a rooted tree This can be done in many ways, depending in which order the nodes are visited: preorder first visits the current node and then (recursively) its children one-by-one inorder traversal, especially on binary trees, first (recursively) visits the left child, then the node itself and then (recursively) the right child postorder traversal first visits (recursively) all the children one-by-one and then the node itself level order traversal, which is not so common, visits the nodes level-by-level left-to-right starting from the root node traverse-preorder(tree T ): def traverse(n): visit(n) for each child c of n: traverse(c) traverse(t.root) traverse-inorder(tree T ): def traverse(n): traverse(n.leftchild) visit(n) traverse(n.rightchild) traverse(t.root) Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

25 traverse-postorder(tree T ): def traverse(n): for each child c of n: traverse(c) visit(n) traverse(t.root) traverse-levelorder(tree T ): level new Queue level.enqueue(t.root) while level is non-empty: nextlevel new Queue while level is non-empty: n level.dequeue() visit(n) for each child c of n: nextlevel.enqueue(c) level nextlevel Example Visiting order for the nodes in the tree on left: c Preorder: c,d,e,g,f,a,b,h Inorder: g,e,f,d,c,b,a,h Postorder: g,f,e,d,b,h,a,c e d b a h Level order: c,d,a,e,b,h,g,f g f Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

26 Practical traversal code may mix orders Printing arithmetic expressions in prefix, infix, and postfix notations abstract class Expr { def p r e f i x S t r : S t r i n g def i n f i x S t r : S t r i n g def p o s t f i x S t r : S t r i n g } case class Negate ( expr : Expr ) extends Expr { def p r e f i x S t r : S t r i n g = Negate ( +expr. p r e f i x S t r + ) def i n f i x S t r : S t r i n g = +expr. i n f i x S t r def p o s t f i x S t r : S t r i n g = expr. p o s t f i x S t r + } case class Add ( l e f t : Expr, r i g h t : Expr ) extends Expr { def p r e f i x S t r : S t r i n g = Add ( + l e f t. p r e f i x S t r +, + r i g h t. p r e f i x S t r + ) def i n f i x S t r : S t r i n g = ( + l e f t. i n f i x S t r r i g h t. i n f i x S t r + ) def p o s t f i x S t r : S t r i n g = l e f t. p o s t f i x S t r + + r i g h t. p o s t f i x S t r + + }... case class Num( value : Double ) extends Expr { def p r e f i x S t r : S t r i n g = Num( +value. t o S t r i n g + ) def i n f i x S t r : S t r i n g = value. t o S t r i n g def p o s t f i x S t r : S t r i n g = value. t o S t r i n g } Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

27 The tree corresponding to the expression (2.0x + y) is obtained with val e = Negate(Add(Multiply(Num(2.0),Var( x )),Var( y ))) e.prefixstr produces Negate(Add(Multiply(Num(2.0),Var( x )),Var( y ))) e.infixstr produces -((2.0*x)+y) e.postfixstr gives 2.0 x * y + - The postfix notation does not need parentheses and it is easy to read and evaluate from the textual representation by using a stack (see e.g. this article) Num Negate expr Add left right Multiply Var left name = "y" right Var value = 2.0 name = "x" Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

28 Heaps Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

29 A heap is a tree data structure commonly used for implementing priority queues, and sorting (heap sort) Abstractly, a priority queue is a container data structure that allows inserting an element with some priority key, finding an element with the highest key, and removing an element with the highest key. (Equivalently, smallest priority could have been used.) Multiple elements can have the same key Thus an abstract data type for priority queues could have an API with INSERT(x) for inserting the element x with the key x.key MAX() for getting an element with the largest key REMOVE-MAX() for removing an element with the maximum key INCREASE-KEY(x, v) for increasing the key of x to v DECREASE-KEY(x, v) for decreasing the key of x to v Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

30 Priority queus can be used, e.g., for prioritizing network traffic based on the type or previous usage, ordering pending orders according to the customer type, building an event simulator, where events must be simulated in an order of the associated start times, and selecting the best next option in other algorithms (Dijkstra s algorithm later in this course etc) Note that elements are inserted and removed from the queue in an online fashion all the time We could use resizable arrays and then apply O(n log 2 n)-time sorting before each find/remove operation but we want to do better so that each insert and remove takes O(log 2 n) time if we currently have n elements in the queue Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

31 Heaps and heap property A heap is a nearly-complete binary tree such that all levels are full, except possibly the last one that is filled from left to right up to some point the nodes are associated with some elements having comparable keys (e.g., integers), and the nodes satisfy a heap property: the key of each node is at least as large as that of any of its children (in which case the heap is a max-heap), or the key of each node is at least as small as that of any of its children (in which case the heap is a min-heap) As a direct consequence, A heap with n nodes has height log 2 n In a non-empty max-heap, an element with the largest key is in the root node In a non-empty min-heap, an element with the smallest key is in the root node Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

32 Example Three nearly-complete binary trees, the keys of the elements associated with the nodes are shown inside the nodes A max-heap A min-heap A tree that is not a heap Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

33 Representing heaps One does not usually represent heaps as node objects with pointers to children etc Instead, as heaps are nearly-complete binary trees, they can be represented very compactly with arrays The idea: A heap with n nodes is represented with an array a of n elements The nodes are numbered from 0 to n 1 level-wise, left to right The element of the node i is stored in a[i] The left child of the i:th node is the node 2i + 1 If 2i + 1 n, then the i:th node does not have a left child The right child of the i:th node is the node 2i + 2 If 2i + 2 n, then the i:th node does not have a right child The parent of a non-root node i is the node parent(i) = i 1 2 Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

34 Example The max- and min-heaps of the previous example as trees with the node numbers shown besides the nodes, and as arrays Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

35 Building heaps In the following, we will only consider max-heaps min-heaps can be handled in a similar, dual way In the heap operations, such as inserting an element, we 1 start with a valid max-heap, 2 make a small modification, such as increasing the key of a node, and 3 if the max-heap property is violated after the previous step, restore the max-heap property Before describing the heap operations themselves, we first show the two auxiliary methods that are used to restore the max-heap property (the third step above) Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

36 upheap aka swim Called Heap-Increase-Key in Introduction to Algorithms, 3rd ed. (online via Aalto lib) Suppose that we increase the key of the node i The max-heap property can now be violated if the key becomes larger than that of the parent node parent(i) If this is the case, swap the elements in the nodes i and parent(i) The max-heap property now holds in the sub-tree rooted at i as key of parent(i) was larger than that of i before the increase But the max-heap property may now be violed if the key of the node parent(i) is larger than that of its parent repeat the process until the max-heap property is preserved At each step, a constant amount of steps is taken The process repeats at most log 2 n times as it must stop when the root node is reached Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

37 Example Increasing the key of a node and applying upheap to restore the max-heap property Start Increase the key Swap the elements Swap the elements of the node 4 in the nodes 1 and 4 in the nodes 0 and 1 Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

38 downheap aka sink Max-Heapify in Introduction to Algorithms, 3rd ed. (online via Aalto lib) Suppose that we decrease the key of the node i The max-heap property becomes violated if the new key is smaller than that of any of its children In such a case, swap the elements of i and the child j with the larger key the key of i is now at least as large as the key of the other child the key of i is smaller or equal to that of parent(i) because the original key of i was at least as large as that of j The max-heap property may now be violated if the new smaller key of j is smaller than that of any of its children repeat the process until the max-heap property is preserved At each step, a constant amount of steps is taken The process repeats at most log 2 n times as it must stop when a leaf node is reached Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

39 Example Decreasing the key of a node and applying down-heap to restore the max-heap property Start Decrease the key Swap the elements Swap the elements of the node 0 in the nodes 1 and 2 in the nodes 2 and 5 Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

40 Operations on max-heaps: inserting an element With the upheap process, inserting elements is easy Example 1 Increase the size of the array by one and add the new element in the new node at the end of the array 2 Apply upheap to the new node so that the max-heap property is restored Inserting an element with the key 42 in the heap on the left: Start Insert the key Upheap Upheap in the last index Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

41 Operations on max-heaps: getting and removing an element with the maximum key Getting an element with maximum key is easy as it is the first element in the array Removing it can be done with downheap 1 Move the last node in the array to be the first node (its key is at most as large as the max-heap held) 2 Decrease the size of the array by one 3 Apply downheap to the first node so that the max-heap property is restored Example Removing the element with the largest key from the heap on the left: Start Move the last key Downheap Downheap to be the first one Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

42 Implementations in some libraries java.util.priorityqueue implements priority queues with min-heaps Inserting elements and removing a smallest one in O(log 2 n) time Searching and removing an arbitrary element in O(n) time PriorityQueue in Scala uses max-heaps Inserting elements and removing a largest one in O(log 2 n) time Searching and removing an arbitrary element as well as many other methods in O(n) time C++ priority queue uses max-heaps Inserting elements and removing a largest one in O(log 2 n) time No methods for searching arbitrary elements etc Interestingly, none of the above implement increase key or decrease key operations Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

43 Disjoint-sets or union-find data structure Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

44 An another example of using trees as a data structure is representing disjoint sets Such a disjoint sets data structure, or union-find data structure, maintains a partitioning of a set S of elements into mutually disjoint subsets S 1,...,S k, i.e. S 1... S k = S ja S i S j = /0 for all 1 i < j k. In the beginning each element a is in its own singleton set {a} Given two elements x and y, one can merge (make union) the sets S x and S y holding them into one new set S x S y A representative element is defined for each set and for each element in the set one can find this representative one can check whether two elements are in the same set by testing whether their representative elements are the same Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

45 The abstract data type The abstract data type for disjoint sets has the following methods: MAKE-SET(x) introduces a new element x and puts it in its own singleton set {x} FIND-SET(x) finds the current representative of the set that contains x. Two elements x and y are in the same set if and only if their representatives are the same, i.e. when FIND-SET(x) = FIND-SET(y) UNION(x,y) merges the sets of x and y to one set. This may change the representative of the elements in the resulting set. Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

46 Example: Testing whether an undirected graph is connected Consider the graph on right. Initialize the disjoint-sets data structure by calling MAKE-SET(x) for each x {a,b,c,d,e,f,g} The disjoint sets are now {a},{b},{c},{d},{e},{f},{g} e a c g d Next call UNION(x,y) for each edge {x,y} in the graph f b After each call, two vertices x and y are in the same set if it is possible to reach x and y from each other by using only the edges considered so far At the end, the sets are {a,b,e,f} and {c,d,g} As there is more than one set, the graph is not connected Two vertices x and y belong to the same connected component of the graph if FIND-SET(x) = FIND-SET(y) Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

47 Union-find with disjoint-sets forests Forests or rooted trees provide a simple way to represent disjoint sets Each set is represented as a tree, the nodes in a tree containing the elements in the set The root of the tree is the representative element of the set We will only traverse the trees upwards, from nodes towards the root we only need the parent pointer for each node Example Suppose that we have introduced the elements a, b,..., f with MAKE-SET() and then made unions so that the current sets are {a,c,d,e} and {b,f}. On the right is one possible representation for the current situation The representatives of the sets are e and f, respectively a e d c f b Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

48 Making sets Introducing a new element is easy, just introduce a new node that has no parent and is not a parent of any other node As pseudocode // Insert the element x as a singleton set if not already in any set MAKE-SET(x): if x is not yet in any set: insert x in some data structure recording the elements in the sets x.parent NIL // as a field or with a map As mentioned earlier, we ll only traverse the trees upwards and thus child references are not needed For recording which elements appear in any of the sets (line 2 above) and associating elements to parents etc, one can use some standard set/map data structure (Scala HashSet or HashMap, for instance) whose implementations are discussed in the next rounds Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

49 Finding the representative To find the representative for an element x, we just find its node, and traverse the parent pointers upwards to the root of the tree As pseudocode // Find the representative of the set containing x FIND-SET(x): raise an error if x is not in any set while x.parent NIL: x x.parent return x Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

50 Union Merging the sets of two elements to their union is easy as well First, find the roots of the trees containing the elements If they are not the same (i.e., the elements are not yet in the same set), make the other tree to be a subtree of the another by making the parent pointer of its root to point to the root of the another tree As pseudocode UNION(x, y): Check that y and y are in some set x FIND-SET(x) // Find the root of the tree with x y FIND-SET(y) // Find the root of the tree with y if x y : // Not yet in the same set? x.parent = y // Merge the trees by making x a subtree of y Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

51 Example: Inserting elements and making unions b c d e f b d e f a b c d e f after calling MAKE-SET() after UNION(a, d) after UNION(c, a) for the elements a,...,f a a c f e f b e d e f d b d a c b a c a c after UNION(b, f) after UNION(c, e) after UNION(d, b) Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

52 As you may expect, the basic versions of the operations given above do not guarantee the desired efficiency In the worst case, the trees degenerate to linear trees that resemble linked lists and the find and union operations take linear time Example: Worst-case behaviour The result of applying UNION(a, b), UNION(a, c), UNION(a, d), UNION(a, e), and UNION(a, f) is a linear tree. f e d c b a b c d e f (a) initially a (b) finally Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

53 Balancing The problem with the tree in the previous example is that it is not balanced If we can force the trees to be (at least roughly) balanced trees so that non-leaves have at least two children and the subtrees rooted at the children are roughly of the same size, a tree with n elements would have height O(log 2 n) and the find and union operations would be logarithmic-time ones Fortunately, there is a relatively easy way to do this Associate each node with a rank which is the height of the subtree rooted at the node When making an union, make the tree with smaller rank to be a child of the other one (if the ranks are the same, choose arbitrarily) Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

54 The updated pseudocodes: // Insert the element x as a singleton set if not already in any set MAKE-SET(x): if x is not yet in any set: insert x in some data structure recording the elements in the sets x.parent NIL // as a field or with a map x.rank 0 // as a field or with a map UNION(x, y): Check that y and y are in some set x FIND-SET(x) // Find the root of the tree with x y FIND-SET(y) // Find the root of the tree with y if x y : // Not yet in the same set? if x.rank < y.rank: // Subtree at x has less height? x.parent y // Merge the trees by making x a subtree of y else: y.parent x // Merge the trees by making y a subtree of x if x.rank = y.rank: x.rank x.rank + 1 Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

55 Example Replay of our earlier example but now with ranks (shown besides the nodes): a 0 b 0 c 0 d 0 e 0 f 0 b 0 c 0 d 1 e 0 f 0 b 0 d 1 e 0 f 0 a 0 initially after UNION(a, d) after UNION(c, a) a 0 c 0 f 2 d 1 e 0 f 1 d 1 f 1 b 0 d 1 a 0 c 0 b 0 a 0 c 0 e 0 b 0 a 0 c 0 e 0 after UNION(b, f) after UNION(c, e) after UNION(d, b) Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

56 Example Replay of our earlier worst-case example but now with ranks. The result of applying UNION(a, b), UNION(a, c), UNION(a, d), UNION(a, e), and UNION(a, f) is no longer a linear tree but a very shallow one. a 1 a 0 b 0 c 0 d 0 e 0 f 0 (a) initially b 0 c 0 d 0 e 0 f 0 (b) finally Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

57 Theorem With ranks, the height of each tree in the forest is at most log 2 s, where s is the number of nodes in the tree. Proof By induction on the performed operations. Induction base: the claim holds after 0 operations as there no trees in the forest. Induction hypothesis: assume that the claim holds after m operations. Induction step: If we next perform a find operation, the trees do not change and thus the claim holds for m + 1 operations. If we insert a new element, we insert a new isolated node (a tree with 1 node and of height 0 = log 2 1 ) and the other trees stay the same. As any other tree was of height log 2 s or less by the induction hypothesis, the tree is of height log 2 (s + 1) or less after the operation. Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

58 If we make union operation on x and y, we have two cases. 1 If the elements are in the same set, the trees stays unmodified. 2 If the elements are in different sets of sizes s x and s y, then the heights of the corresponding trees are, by induction hypothesis, h x log 2 s x and h y log 2 s y. If h x < h y, then the resulting tree has s x + s y nodes and height h y. As h y log 2 s y, h y log 2 (s x + s y ). The case h y < h x is symmetric to the previous one. If h x = h y, then the resulting tree has s = s x + s y nodes and height h = h x + 1 = h y + 1. If s x s y, then s 2s x and log 2 (s ) log 2 (2s x ) = log 2 (2) + log 2 s x = 1 + log 2 s x = h. Similarly, if s x s y, then s 2s y and log 2 (s ) log 2 (2s y ) = log 2 (2) + log 2 s y = 1 + log 2 s y = h. Therefore, the claim holds after m + 1 operations and by the induction principle, the claim holds for all values m. Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

59 Let n be the number of elements in the current sets Assume that we can 1 find the node of each element in O(log 2 n) time, and 2 access and modify the parent and rank values in constant time Corollary With ranks, the make-set, find-set and union operations take O(log 2 n) time, Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

60 Path compression An another idea to make the trees even shallower is to compress the paths from the nodes to the roots during the find operations We perform a recursive search to find the root, and after each recursive call update the parent of the current node to be the root As pseudocode: // Find the representative of the set containing x FIND-SET(x): raise an error if x is not in any set def FIND-AND-COMPRESS(y): if y.parent = NIL: // In the root? return y else r FIND-AND-COMPRESS(y.PARENT) y.parent r return r return FIND-AND-COMPRESS(x) Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

61 Example Assume that we have the disjoint-sets forest on the left below. When we call FIND-SET(c), the forest is modified to that on the right below. e f e f d b c d b a c a Observe that the find operation is also called during the union operation, and thus compression may occur there as well Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62

62 Theorem With union-by-rank and path compression, making m disjoint-set operations on n elements takes O(m α(m)) time, where α is a very slowly growing function with α(m) 4 for all m Proof See Section 21.4 in Introduction to Algorithms, 3rd ed. (online via Aalto lib), not required in the course. Tommi Junttila (Aalto University) Round 3 CS-A1140 / Autumn / 62