PART IV. Given 2 sorted arrays, What is the time complexity of merging them together?

Transcription

1 General Questions: PART IV Given 2 sorted arrays, What is the time complexity of merging them together? Array 1: Array 2: Sorted Array: Pointer to 1 st element of the 2 sorted arrays Pointer to the 1 st element of the resultant merge-sorted array To merge the 2 sorted arrays, we essentially compare the elements pointed to by the pointers of the 2 sorted arrays at any given time, and choose the element with the lower value and place it in the final output array. This step is repeated until one array is empty at which time the other array is just appended to the output array. Computationally, each basic step of a comparison occurs in constant time i.e. O(1). But we repeat this process at most n times. Hence the time complexity is Θ(n). Problem 8-3 (pg 179 in text) Sorting words in lexical order The best way to sort such data is to use radix sort. First as each string varies in length, we normalize the length of all strings by appending the letter a, the lowest value letter in the alphabet to all strings. The no. of characters appended to each string is determined based on the longest string available in the given data set. i.e.

2 If the given set of words were: a, Jack, ab, Victoria as Victoria is the longest string with length 8, all strings are normalized to length 8. The data set on which radix sort will now be called looks as below: aaaaaaaa, Jackaaaa, abaaaaaa, Victoria. Now radix sort sorts the strings on each column as usual, to finally sort the data. B Tree: Binary Search works because of the under-lying data structure is a tree. A tree is a data structure accessed beginning at the root node. Each node is either a leaf or an internal node. An internal node has one or more child nodes and is called the parent of its child nodes. All children of the same node are siblings. Contrary to a physical tree, the root is usually depicted at the top of the structure, and the leaves are depicted at the bottom.

3 Trees can either be balanced or unbalanced. Unbalanced trees can cause a lot of problems when working on the data structure. Balanced trees are the solution to most of these problems. A balanced tree is one where no leaf is much farther away from the root than any other leaf. Different balancing schemes allow different definitions of "much farther" and different amounts of work to keep them balanced. B-Tree is one such balanced tree structure. A Red-Black tree is a similar data structure but a B- Tree has been designed to work well on magnetic disks, thus minimizing required I/O operations. In B-trees, internal nodes can have a variable number of child nodes within some pre-defined range. When data is inserted or removed from a node, its number of child nodes changes. In order to maintain the pre-defined range, internal nodes may be joined or split. Because a range of child nodes is permitted, B-trees do not need re-balancing as frequently as other self-balancing search trees, but may waste some space, since nodes are not entirely full. The lower and upper bounds on the number of child nodes are typically fixed for a particular implementation. Internal nodes in a B-tree nodes which are not leaf nodes are usually represented as an ordered set of elements and child pointers. Every internal node contains a maximum of U children and other than the root a minimum of L children. For all internal nodes other than the root, the number of elements is one less than the number of child pointers; the number of elements is between L-1 and U-1. The number U must be either 2L or 2L-1; thus each internal node is at least half full. This relationship between U and L implies that two half-full nodes can be joined to make a legal node, and one full node can be split into two legal nodes (if there is room to push one element up into the parent). These properties make it possible to delete and insert new values into a B-tree and adjust the tree to preserve the B-tree properties. Leaf nodes have the same restriction on the number of elements, but have no children, and no child pointers. The root node still has the upper limit on the number of children, but has no lower limit. For example, when there are fewer than L-1 elements in the entire tree, the root will be the only node in the tree, and it will have no children at all.

4 Sample B Tree Searching for an element: Search is performed in the typical manner, analogous to that in a binary search tree. Starting at the root, the tree is traversed top to bottom, choosing the child pointer whose separation values are on either side of the value that is being searched. Insertion of an element into a b-tree: All insertions happen at the leaf nodes. 1. By searching the tree, find the leaf node where the new element should be added. 2. If the leaf node contains fewer than the maximum legal number of elements, there is room for one more. Insert the new element in the node, keeping the node's elements ordered. 3. Otherwise the leaf node is split into two nodes. 1. A single median is chosen from among the leaf's elements and the new element. 2. Values less than the median are put in the new left node and values greater than the median are put in the new right node, with the median acting as a separation value. 3. That separation value is added to the node's parent, which may cause it to be split, and so on. If the splitting goes all the way up to the root, it creates a new root with a single separator value and two children, which is why the lower bound on the size of internal nodes does not apply to the root. The maximum number of elements per node is U-1. When a node is split, one element moves to the parent, but one element is added. So, it must be possible to divide the maximum number U-1 of elements into two legal nodes. If this number is odd, then U=2L and one of the new nodes contains (U-2)/2 = L-1 elements, and hence is a legal node, and the other contains one more element, and hence it too is legal. If U-1 is even, then U=2L-1, so there are 2L-2 elements in the node. Half of this number is L-1, which is the minimum number of elements allowed per node.

5 Sample Insertion with each iteration Elements numbered 1-7 being inserted in order, resulting in the B-Tree as shown. Another example: Given numbers 1, 12, 8, 2, 25,5 show an order 5 B-Tree cannot be inserted as max no. elements is 4, hence 8 is promoted as it is the median of the 5 elements, and the remaining elements are split into nodes as shown below Any further addition of elements would cause the elements being inserted into the leaf nodes, appropriately. If the nodes fill up, then the node is split as before and median is promoted. If on promotion, the parent node is filled, then one element is promoted from the node. If this occurs at

6 the root, then there would be a new root, with the previous root being split into 2 nodes, both being children of the new root. Deletion of elements from a B-Tree: There are two popular strategies for deletion from a B-Tree. or locate and delete the item, then restructure the tree to regain its invariants Do a single pass down the tree, but before entering (visiting) a node, restructure the tree so that once the key to be deleted is encountered; it can be deleted without triggering the need for any further restructuring. The method described below employs the former strategy. The Text Book covers the 2 nd strategy. There are two special cases to consider when deleting an element: 1. the element in an internal node may be a separator for its child nodes 2. Deleting an element may put it under the minimum number of elements and children. Each of these cases will be dealt with in order. Deletion from a leaf node Search for the value to delete. If the value is in a leaf node, it can simply be deleted from the node, perhaps leaving the node with too few elements; so some additional changes to the tree will be required. Deletion from an internal node Each element in an internal node acts as a separation value for two subtrees, and when such an element is deleted, two cases arise. In the first case, both of the two child nodes to the left and right of the deleted element have the minimum number of elements, namely L-1. They can then be joined into a single node with 2L-2 elements, a number which does not exceed U-1 and so is a legal node. Unless it is known that this particular B-tree does not contain duplicate data, we must then also (recursively) delete the element in question from the new node. In the second case, one of the two child nodes contains more than the minimum number of elements. Then a new separator for those subtrees must be found. Note that the largest element in the left subtree is the largest element which is still less than the separator. Likewise, the smallest element in the right subtree is the smallest element which is still greater than the separator. Both of those elements are in leaf nodes, and either can be the new separator for the two subtrees.

7 If the value is in an internal node, choose a new separator (either the largest element in the left subtree or the smallest element in the right subtree), remove it from the leaf node it is in, and replace the element to be deleted with the new separator. This has deleted an element from a leaf node, and so is now equivalent to the previous case. Rebalancing after deletion If deleting an element from a leaf node has brought it under the minimum size, some elements must be redistributed to bring all nodes up to the minimum. In some cases the rearrangement will move the deficiency to the parent, and the redistribution must be applied iteratively up the tree, perhaps even to the root. Since the minimum element count doesn't apply to the root, making the root be the only deficient node is not a problem. The strategy is to find a sibling of the deficient node which has more than the minimum number of elements and of the separator and the values in both nodes as the new separator and put that in the parent. o Redistribute the remaining elements to the right and left children. o Redistribute the subtrees of the two nodes to parallel the redistribution of the elements. The subtrees themselves are transplanted entirely, and are not altered if moved to a different parent node, and this can be done as the elements are redistributed. If the sibling node immediately to the right of the deficient node has only the minimum number of elements, examine the sibling node immediately to the left. If both immediate siblings have only the minimum number of elements, create a new node with all the elements from the deficient node, all the elements from one of its siblings, and the separator in the parent between the two combined sibling nodes. o Remove the separator from the parent, and replace the two children it separated with the combined node. o If that brings the number of elements in the parent under the minimum, repeat these steps with that deficient node, unless it is the root, since the root may be deficient.

8 Some Important Concepts: In Place Sorting: This is a sorting method where no auxiliary array is needed. i.e. Sorting is done within the initial input array. No additional memory is expended in the sort process. Examples: Counting Sort is not an In Place sorting technique as it uses additional array(memory) to perform the sort. QuickSort is an In Place sorting algorithm as the Partitioning occurs on the input array and no additional memory is used. Stability Sort: Here the elements with the same value, i.e. identical keys appear in the sorted output array in the same order as they appeared in the initial input array. Examples: Counting and Radix Sort

9 Dynamic Programming Shortest Path on a graph: To find the shortest path on a graph from node A to node B, we can list out all possible paths and then calculate the shortest path. However this approach, while easy to understand is a very tedious task, which takes too long to compute. We can use certain properties to minimize the calculation needed to achieve the same result. A good method is to find a subset of the final solution as it must be part of the overall problem solution. i.e. for a shortest path between 2 nodes, the path must also satisfy the shortest path conditions for intermediate nodes. Dijsktra s algorithm computes the shortest path iteratively by moving from source node to the destination node by adding intermediate nodes to a subset S of nodes from the subset T. This process takes longer, hence we compute the shortest path by using a technique known as Dynamic Programming. Initial Graph:

10 Graph showing shortest path: Now if the path shown above was not chosen as the shortest path, rather, the figure below with the dashed path was chosen to be the shortest path, we can see that the route is not the shortest path from the start node to any intermediate nodes. Therefore there must be an alternative path which must be the shortest path such that even distances from start node to intermediate nodes result in shortest paths

11 Longest Common Subsequence: Consider 2 strings as below: String 1 -BDCABA String 2- ABCBDAB A subsequence of some sequence is a new sequence which is formed from the original sequence by deleting some of the elements without disturbing the relative positions of the remaining elements. That is BCAA is a subsequence of String 1 formed using indices 0,2,3,5. The longest subsequence possible which is common to both Strings 1 and 2 is known as the longest common subsequence (LCS). The LCS may be found for a set of sequences, where the set has more than 2 sequences. Thus in Strings 1 & 2 the LCS would be: B D C A B A A B - C - B D A B BCBA Thus BCBA is a sequence with indices 0,2,4,5 in String 1 and indices 1,2,3,5 in String 2 i.e the sequences can be matched up by including null spaces in between the string itself, to find a longer matching sequence. The method to find the longest sequence is as follows: Let i and j be 2 counters used to indicate the index value currently being read in Strings 1 and 2 where string 1 (s1) = x1, x2, x3.xi String 2(s2) = y1, y2, y3 yj Then C (i, j) = Length of LCS for Strings 1 and 2 C (i, j) = 0 if i = 0 or j = 0

12 Now if s1 and s2 are such that they both end with the same character i.e s1= x1x2x3 A s2=y1y2y3.a Then as xi = yj; C(i,j) = C(i-1,j-1) +1 Now if s1 = x1x2x3.a s2 = y1y2y3.b Then xi yj; however A may appear in y1,y2.y(j-1) OR B may appear in x1,x2,..x(i-1) Then C(i,j) = max { C(i-1,j), C(i,j-1) } as xi yj; Obviously this means that we must know values of C(i-1,j-1), C(i-1,j) and C(i,j-1). However we know that if i or j =0; C( i, j) =0. Thus using a matrix of size (m+1) x (n+1), where m and n are the lengths of the 2 sequences, we can store previous C values. We can then back trace to determine the actual solution/ path taken to determine which of the 3 possible cases was used to arrive at the end. Let X be "XMJYAUZ" and Y be "MZJAWXU". The longest common subsequence between X and Y is "MJAU". The table C shown below, which is generated by the function LCSlength, shows the lengths of the longest common subsequences between prefixes of X and Y. The ith row and jth column shows the length of the LCS between X 1..i and Y 1..j M Z J A W X U X M J Y A U Z

13 The underlined numbers show the path the function backtrack would follow from the bottom right to the top left corner, when reading out an LCS. If the current symbols in X and Y are equal, they are part of the LCS, and we go both up and left. If not, we go up or left, depending on which cell has a higher number. This corresponds to either taking the LCS between X 1..i 1 and Y 1..j, or X 1..i and Y 1..j 1 The function below takes as input sequences X[1..m] and Y[1..n] computes the LCS between X[1..i] and Y[1..j] for all 1 i m and 1 j n, and stores it in C[i,j]. C[m,n] will contain the length of the LCS of X and Y. function LCSLength(X[1..m], Y[1..n]) C = array(0..m, 0..n) for i := 0..m C[i,0] = 0 for j := 1..n C[0,j] = 0 for i := 1..m for j := 1..n if X[i] = Y[j] C[i,j] := C[i-1,j-1] + 1 else: C[i,j] := max(c[i,j-1], C[i-1,j]) return C[m,n] The following function backtracks the choices taken when computing the C table. If the last characters in the prefixes are equal, they must be in an LCS. If not, check what gave the largest LCS of keeping x i and y j, and make the same choice. Just choose one if they were equally long. Call the function with i=m and j=n. function backtrack(c[0..m,0..n], X[1..m], Y[1..n], i, j) if i = 0 or j = 0 return "" else if X[i] = Y[j] return backtrack(c, X, Y, i-1, j-1) + X[i] else if C[i,j-1] > C[i-1,j] return backtrack(c, X, Y, i, j-1) else return backtrack(c, X, Y, i-1, j)