Unit 3 Chapter 10 ABSTRACT ALGORITHMS 2GREEDY METHODS

Transcription

1 DESIGN AND ANALYSIS OF ALGORITHMS Unit 3 Chapter 10 ABSTRACT ALGORITHMS 2GREEDY METHODS WHAT IS GREEDY METHOD? There are n inputs of some values and an objective function. The method gives an optimal solution to the problem by considering the inputs one at a time, checking to see if it can be included in the set of values which give an optimal solution and then check if it is the feasible solution. All of n inputs may not be included, only those needed to form the optimal solution will be included. Each input may consume some resource, which is generally available in limited quantity. We may say that the feasible solution represents the basic or the essential requirements of the problem and the optimal solution denotes the more specific and desirable requirements of the problem. Thus the flow of data is given in Figure. 1

2 The strategy adopted for achieving the optimal solution is called Greedy method. The word greedy refers to allocating the maximum possible value of some limited resource to the first element which enters the optimal solution. The feasibility of the solution is expressed in terms of obeying the constraints of the resource. Thus Greedy method depends upon local (short range) maximum. The abstract algorithm can be represented as following algorithm: The three functions SELECT (), FEASIBLE (), and UNION () do the detailed work of this abstract algorithm. Their details will obviously vary from one problem to another: SELEC() select an element from a[] such that it has a potential for satisfying the optimality criterion or selection policy. FEASIBLE() checks if the selected element x satisfies the feasibility criterion. UNION() integrates the element x in the solution. 2

3 KNAPSACK PROBLEM There are n objects numbered i, 0 <= i <= n-1, having weights wi and contribution to profit Pi. There is a sack having a capacity of M. If a fraction xi of the object i is put in the sack, then it increases its weight by wixi and contributes pixi to the profits. Find the filling that maximizes the profits. It is obvious from the problem statement that We can assume that p and w are positive numbers. To understand the problem better, consider a numerical example for this problem: n = 3, M = 20, P = {25, 24, 15}, W = {18, 15, 10}. Some of the feasible solutions are shown in following Table 3

4 These solutions were obtained by considering different greedy strategies. 1) 2) 3) 4) The first solution is just a random solution. The selection policy we adopt is based on the approach that an object with the maximum profit should be included. We try to fit the largest Xi possible under feasibility constraints. Thus at each stage of the algorithm, the largest increase in profit is obtained. This policy leads to the solution no. 2. We try another selection policy: include an object i with the lowest Wi first. This will minimize the weight that is added to the sack at each stage and the sack will fill up slowly only. This leads to the solution no. 3. We find that both the above policies do not lead to an optimum solution and we should actually select the objects at each stage having maximum plw ratio. It can be proven that this selection policy leads to the optimal solution. This is solution no. 4 4

5 PROOF SKETCH OF GREEDY ALGORITHM Consider the most valuable object weighing w. If we remove weight w of item i from the sack, then the remaining load must contain the optimal selection of the other (n - 1) items and the weight wi - w of item i. If not, then the original load was not the most valuable. First, we show that as much as possible of the highest profit/ weight item must be included in the optimum solution. Let (wh, ph) be the weight and the profit of the item with the highest profit/weight ratio (item h). Let L(i) be the weight of the item i contained in the sack L. If some part of item h is left and wi 0 for some j h, then replacing j with h will yield a higher value. L(j)p(j)/w(j) <= L(j)p(h)/w(h) and p(j)/w(j) p(h)/w(h) are true by definition of h. There is a variation on this problem, known as the 0/1 knapsack problem. In this case, Xi can be either 1 or 0, that is, either wholly present or not (completely absent, that is, the object cannot be broken). Again, we want to maximize the total profit: 5

6 JOB SEQUENCING WITH DEADLINES Given a set of n jobs, each having a "deadline" (an integer) d[i] and a profit p[i] associated with it. For a job i the profit is earned if and only if the job is completed within its deadline. Each job takes one unit of time on a machine (processor) and only one machine is available. We want to maximize the profit. The jobs are arranged in decreasing order of profit in an array. A feasible solution to this problem is a subset J of the n jobs, such that each of them can be completed within its deadline. The value of the feasible solution is Σp[i] for all i Є J. An optimal solution is a feasible solution with maximum value. For example, let n=4, profit vector P={30,35,20,25}, deadline vector D = {2,1, 2, 1}. Assumption: Execution time of each job is one time unit. The feasible solutions and their values are: It is seen that solution no. 3 is optimal, where only jobs 1 and 2 are processed and the profit is maximum. Note that the jobs must be done in the order shown, first 2 and then 1, in order to obey the deadlines. In order to use the greedy strategy, we have to decide the selection criterion. As a first attempt we can use ΣPi as our optimisation measure. Then we select the next job which i Є J increases this measure the most, provided that the resulting J is a feasible solution. 6

7 Thus we should consider the jobs in non-increasing order of profit p. This was the reason that in the above example, the jobs are given in such an order. Considering the jobs in order of decreasing p values, the 1st can be in J because J = {1} is a feasible solution. Then the 2nd can be in J, because J = {1, 2} is a feasible solution, since the deadlines can be met. Now job 3 cannot be in J as {1, 2, 3} is not a feasible solution. The largest value of the deadline is 2 and as each included job would take 1 time unit each, we can have at most 2 jobs in J. So we have J = {1, 2}. What should be the order of execution of the jobs? They should be executed earliest deadline first, that is, in the non-decreasing order of the deadline d. Thus the jobs {1,2} should be done in the order 2, 1. The greedy method given above always obtains an optimal solution to the job sequencing problem. We give a high level algorithm for this method as follows: 7

8 We can keep the jobs in the set J in an array j[] sorted by non-decreasing order of deadlines, which will be easier to perform the feasibility check. On the other hand, we assume that job numbers are given in non-increasing order of profit p, thus their deadlines are stored in d[ ] in that order. In the C language implementation, a fictitious job no. 0 is used with d[0] = 0 and j[0] = 0 to simplify the insertion of a job i in J in the deadline order. The jobs in J with subscript larger than the new job i are to be moved. Only the jobs after the insertion point for i, say k, need be checked for their deadlines. It is to be noted that the actual values of profit do not enter into consideration in this algorithm. EXAMPLE-MINIMUM SPANNING TREES Let G=<V,E> be an undirected, connected graph, having vertices V and edges E. A sub-graph T = <V', E'> of G is a spanning tree of G if T is a tree. An undirected graph with V = n, may have as many as n(n - 1)/2 edges, but a spanning tree T will have exactly (n - 1) edges. For example if the nodes or vertices represent cities and the edges represent the roads between them, then each city may be connected by an independent road to the others. This would lead to n(n - 1)/2 such roads. That would give us a graph (see figure). But we may be able to go from any city to any other if at the minimum there are certain roads, (n - 1), such that each city is approachable. That would give us a spanning tree. There could be different possibilities here. 8

9 In a practical situation, the edges will have a weight or cost associated with it. For example, the weight of an edge in the case of the roads between cities is the length of the road. Then suppose we are planning roads between cities, we would be interested in the roads with a minimum total length. We are interested in minimizing this length. That would give us a minimum spanning tree for the problem graph. Figure shows a weighted graph and one of its minimum spanning trees. A greedy method to obtain the minimum spanning tree would construct the tree edge by edge, where each next edge is chosen according to some optimization criterion. An obvious criterion would be to choose an edge which adds a minimum weight to the total weight of the edges so far selected. There are two ways in which this criterion can be achieved: 9

10 1) 2) The set of edges selected so far always forms a tree; the next edge to be added is such that not only it adds a minimum weight, but also forms a tree with the previous edges; it can be shown that this algorithm results in a minimum cost tree; this algorithm is called Prim's algorithm. The edges are considered in non-decreasing order of weights; the set T of edges at each stage is such that it is possible to complete T into a tree; thus T may not be a tree at all stages of the algorithm; this also results in a minimum cost tree; this algorithm is called Kruskal's algorithm. PRIM'S ALGORITHM This algorithm starts with a tree that has only one edge, the minimum weight edge. Then edges (j, q) will be added one by one, such that: node j is already included, node q is NOT included and weight WT(j, q) is the minimum amongst all the edges (x, y) for which x is in the tree and y is not. See above Fig. In order to execute this algorithm efficiently, we have a node index NEAR(j) associated with each node j not yet included in the tree. NEAR(j) = 0 if a node is included in the tree. NEAR(j) is a node in the tree such that WT(j, NEAR(j)) is the minimum amongst all possible choices for NEAR( j). 10

11 The time complexity of Prim's algorithm can be seen to be O(n2). 11

12 KRUSKAL'S ALGORITHM This algorithm starts with a list of edges sorted in non-decreasing order of weights, the order in which they are considered for processing. The algorithm goes on including edges in the partial Tree till it finds an edge which would result in a cycle. It discards this edge, considers the next edge and repeats this operation till all the edges are exhausted. In the top-level algorithm KRUSKAL( ), Algorithm (in next slide), E is the set of edges already sorted in ascending order of weights. T is the resultant minimum spanning tree. 12

13 The algorithm has a worst case time complexity, excluding the time to sort the edges, of the order O(e.log.e), where e is the number of edges in the graph. This algorithm is implemented in two versions, differing in the way in which the data is stored and method of sorting the edges according to their weights. 1ST VERSION OF KRUSKAL The three operations marked by bold letters in the KRUSKAL algorithm have to be implemented as separate algorithms. In the implementation the edges are stored in the form of a linked list edges, and are sorted by Insertion sort on arrival of each node, by the function sinsert(). Thus traversing the linked list from beginning will be equivalent to selecting the lowest weight edge at each stage. Checking that a particular edge does not result in a cycle is ensured by the function nocycle (). An edge is added to the tree by the function addtree (). The data file whose name is fixed as prim. dat, is read in by read _in (). The data structures used are: See Figs. 1 and 2. EDGE <start node no.><end node no.><weight><next edge ptr.> EDGEL <edge ptr.><next edge-list elem.> NODE <number><ptr to edge-list><marker flag> 13

14 The nodes are stored in an array (the maximum size presently defined is 100, which can be changed). The user is required to use node numbers 1, 2, 3,..., n - 1. See Fig. A. The tree is defined by the union of the edges linked to the elp pointer in the NODE structure. Initially this tree is empty. As the edges are added, edge-list elements are added to corresponding nodes, see Fig. B We add an edge pointer to both the starting and ending nodes of an edge, thus at any stage, the number of edge-pointers in the tree is exactly double the number of edges added till now. The most interesting function is nocycle(). In order to find if an edge will result in a cycle, it proceeds as follows: start with the s node of the test edge. If there is no edge connected to that node, then return nocycle = 1. Otherwise, mark all the nodes connected to this node. Then check each node sequentially, if it is already marked, then mark all the nodes connected to it. If we mark any node in the process, we start from the first node again. Soon we shall reach a stage where no further nodes are marked. Then we check if the e node of the test edge is marked. If it is, then the addition of this edge will result in a cycle, and we return 0. 14

15 Figure A Figure B 15

16 These results are much better than the previous version, though not comparable to the Prim's algorithm. The culprit is the cycle testing function nocycle (). If it can be made more efficient, the Kruskal's algorithm can compete with Prim's. The theory of Union-Find data-structure, presently the most efficient known approach to resolve the problem of cycle testing, while implementing the Kruskal's algorithm. The constant column in the above table refers to the constants c1, c2 and k in the corresponding time complexity formula c1 O(n2), c O(e), and ko(e log e) respectively. Thus we see that Prim's algorithm is good when there are more edges per node, while the Kruskal's algorithm is good when the graph is less dense, that is, the graph is characterised by a few edges per node. 16

17 UNION-FIND DATA-STRUCTURE A tree is a acyclic data-structure, so obviously while building the MST, we would have to check for cycles. At each step of the Kruskal's algorithm, the composite partial solution (V, A) is a forest, where V is the set of nodes and A is the set of edges which constitute the partial solution. We note that: If nodes u and v are in the same tree, then adding the edge (u, v) to A creates a cycle. 2) If nodes u and v are not in the same tree, then adding the edge (u, v) to A does not create a cycle. 1) The obvious problem that arises is how to test whether u and v are in the same tree. This question can be answered at two levels of implementation: Higher level: we can use a Disjoint-set data structure. Vertices in a tree are considered to be in the same set. Then the test is FindSet(u) = FindSet(v). Lower level: Obviously we require to implement FindSet(). The Union-Find data structure implements this. We need the Disjoint-set data structure to implement Union-Find. A disjoint-set data structure represents a collection of sets that are disjoint-that is, no element is found in more than one set. The collection of disjoint sets is called a partition, because the elements are partitioned among the sets. Moreover, we work with a universe of elements. The universe is made up of all the elements that can be a member of a set. Every element is a member of exactly one set. For example, suppose the elements in our universe are corporations that still exist today or were acquired by other corporations. Our sets are corporations that still exist under their own name. For instance, Tata, TCS and VSNL are all members of the "Tata" set. 17

18 We will limit ourselves to two operations. The first is called a union operation, in which we merge two sets into one. The second is called a find query, in which we ask a question similar to, "What corporation does TCS belong to today?" More generally, a find query takes an element and tells us which set it is in. We will not support operations that break up a set into two or more sets (not quickly, anyway). Data structures designed to support these operations are called partition or union/find data structures. Union/find data structures start off by having with every element in a separate set, see Fig. 1. The query "find(tatasteel)" returns "TataSteel". Suppose we take the union of TataMotors and TataSteel and called the resulting corporation TataMotors. Similarly, we unite Tata with VSNL-the result being called VSNL-and TataElexi with TCS-the result being called TataElexi, see Fig

19 The query "find(tatasteel)" now returns "TataElexi". When Tata takes over TataElexi, everything will be in one set and no further mergers will be possible. The obvious data structure for disjoint sets would have the following structure: 1) 2) Each set references a list of the elements in that set. Each element references the set that contains it. With this data structure, find operations take O(1) time. Hence, we say that list-based disjoint sets use the quickfind algorithm. However, the union operations are slow, because when the two sets are united, we must walk through one set and relabel all the elements so that they reference the other set. Space limitations prevent us from analyzing this algorithm in detail (see Goodrich and Tamassia). Instead, let's move on to the less obvious but far superior quick-union algorithm. TREE-BASED DISJOINT SETS AND THE QUICKUNION ALGORITHM In tree-based disjoint sets, union operations take O(1) time at the expense of slower find operations. However, for any sequence of union and find operations, the quick-union algorithm is faster overall than the quick-find algorithm. To support fast unions, each set is maintained as a general tree. The quick-union data structure comprises a forest, in which each element is initially the root of its own tree; then trees are merged by union operations. The quick-union data structure is simpler than the general tree structures you may have studied so far, because there are no child or sibling references. Every node knows only its parent, and you can only walk up the tree. Fortunately, the true identity of each set is recorded at its root. Union is a simple O(1) time operation: we simply make the root of one set a child of the root of the other set. For example, when we form the union of TataElexi and TataMotors, as shown in Fig. 1. TataElexi becomes a set containing four members. 19

20 However, finding the set to which a given element belongs, is not a constant-time operation. The find operation is performed by following the chain of parent references from an element to the root of its tree. For example, find(tatasteel) will follow the trail of references until it reaches TataElexi. The cost of this operation is proportional to the element's depth in the tree. To prevent elements from getting too deep, we unite sets intelligently. At each root, we record the size of its tree. When we unite two trees, we make the smaller one a subtree of the larger one (breaking ties arbitrarily). This strategy is called union-by-size. 20

21 IMPLEMENTING QUICK-UNION WITH AN ARRAY Suppose the elements are non-negative integers, indexed from zero. We'll use an array to record the parent of each element. If an element has no parent, we shall record the size of the corresponding tree. To distinguish it from a parent reference, we shall record the size s as the negative number -s Initially, every element is the root of its own tree, so we set every array element to -1. See Fig. See Fig. below The forest illustrated on the left is represented by the array on the right. See Fig. below This is not a good way to implement treebased disjoint sets, but it is fast (in terms of the constant in the asymptotic notation). 21

22 Let root1 and root2 be two elements that are roots of their respective trees. The algorithm for union is given as following Algorithm The find() method is equally simple, but we need one more trick to obtain the best possible speed. Suppose a sequence of union operations create a tall tree, and we perform find() repeatedly on its deepest leaf. Each time we perform find(), we walk up the tree from leaf to root, perhaps at considerable expense. When we perform find() the first time, why not move the leaf up the tree so that it becomes a child of the root? In doing so, the next time we execute find() on the same leaf, it will run much more quickly. Furthermore, why not do the same for every node we encounter as we walk up to the root? In the example given here, Figure, find(7) walks up the tree from 7, discovers that 0 is the root, and then makes 0 the parent of 4 and 7, so that future find operations on 4, 7, or their children will be faster. This technique is called path compression. 22

23 Let x be an element whose tree we wish to identify. The algorithm for find(), presented as next Algorithm, returns the identity of the element at the root of the tree. Recall that elements are numbered starting from zero. 23

24 What if we wanted some control over the names of the sets? The solution is to maintain a separate table that maps root elements to set names, and perhaps vice-versa (depending on the application's needs). For example, the table might map 0 to Tata. The union() method must be modified so that when it unites two sets, it assigns the union an appropriate name. Often, however, we don't care about the name of a set at all; we only want to know if two elements x and y are in the same set. We need only to execute find(x) and find(y), and check if the two roots are the same. COMPLEXITY ANALYSIS OF QUICK-UNION Union operations obviously take Θ(1) time. A single find operation can take time as large as Θ(log u), where u is the number of union operations that took place prior to the find. However, the worst-case average running time of union/find operations is better because of path compression. The average running time of find and union operations in the quick-union data structure is so close to a constant that it is hardly worth mentioning, though, in a rigorous asymptotic sense, it is slightly slower. A sequence of f find and u union operations (in any order and possibly interleaved) takes Θ(u +fα(f + u, u)) time in the worst case. Here, α is an extremely slowgrowing function known as the inverse Ackermann function. The value of this function is never larger than 4 for any values of f and u we could ever use. Hence, for all practical purposes, we should think of quick-union as having find and union operations that run, on the average, in constant time. 24

25 USING UNION-FIND IN KRUSKAL ALGORITHM The UNION-FIND Data Structure supports three operations on collections of disjoint sets: Let n be the size of the universe. Create-Set(u): time complexity O(1). Create a set containing the single element u. Find-Set(u): time complexity O(log n). Find the set containing the element u. Union(u, v): time complexity O(log n). Merge the sets respectively containing u and v into a common set. From now onwards we shall treat UNION-FIND as a black box. Now we shall see how the data structures discussed above are applied to achieve a more efficient Kruskal's Algorithm, see Algorithm 25

26 EXAMPLE [SHORTEST PATH] The problem of the roads between cities that we considered while introducing the Minimum Spanning Tree above can be modified as follows. The weights of the edges can be considered to represent the lengths of the roads or cost of the travel on the roads. A traveler will be interested to know the following: 1) 2) Is there a path from city A to city B? If there are more than one paths from A to B, which is the shortest or least-cost path? When we compare this particular problem to the minimum spanning tree problem, we realize that, here we are interested in the length of a particular path and not the total length. In general, we shall have to deal with a directed graph. There is a graph G = <V, E>, a weighing function c(e) for the edges in E and a source node vo. The problem is to determine the shortest path from vo to all the remaining nodes of G. Assume that all the weights are positive. In passing one might note that we are considering the cost of travel and not the distance. Cost may not be directly proportional to the distance. For example, consider a graph denoted by Table. It can be immediately seen that as there is no edge with node 6 as destination, there is no path available to that node from any other node. A greedy strategy for this problem requires a multi-stage solution with an optimization measure. The path can be built up edge by edge. The sum of the length of the path built until now can be used as the optimisation measure. This means that each individual path should be of a minimum length. A greedy way to generate these paths would be by non-decreasing edge length. For example, for the graph given, (see Fig.) starting from node 1, the shortest path is to node 3 (c(s) = 10), thus the path {1, 3} will be the first one to be generated. The next node selected will be node 4, giving us a path 1, 3, 4 of length = 25, and so on. 26

27 The critical point to be noted is this: when a node enters the set of nodes already included in the path, it is possible that a new path shorter than the previous one may be found. For example, while finding path from node 4 to node 5, the direct path length is 35, but when node 2 enters the set of nodes being considered, the length reduces to = 30. Algorithm by E. W. Dijkstra solves this problem efficiently. Write the complete example from the video given. Video link also mention at end of the slides. 27

28 DIJKSTRA'S SHORTEST PATH ALGORITHM The algorithm described in next page is given by E.W. Dijkstra. It is well-known and a popular algorithm for solving shortest path problems. In fact, it is implemented in highspeed computer network routers and switches. The original algorithm calculates only the path lengths, though the actual paths can be obtained by simple changes to the algorithm. Informal Proof of correctness: A close examination of the inner loop of the algorithm, the one that examines edge (v, w) and updates dist, will reveal that it examines each edge in the graph exactly once, so at any stage in the algorithm, we can talk about the subgraph G' = (V,E') examined so far. It consists of all the nodes and edges that have been processed. Each pass through the inner loop adds one edge to E'. We shall show that the following property dist[ ] is an invariant of the inner loop of the algorithm: dist[w] is the minimum distance from s to w in G', and if vk is the kth vertex marked, then v1 through vk are the k closest vertices to s in G, and the algorithm has found the shortest paths to them. 28

29 We shall prove this by induction on the number of edges E' in E'. At the very beginning, before examining any edges, E =0 and E' = 0, so the correct minimum distances are dist[s]=0 and dist[w]=, as initialized in the algorithm. Now s is marked first, with the distance to it equal to 0 as desired. Now consider the next edge (v, w) to get E equal to E' U (v, w). Adding this edge means there is a new way to get to w from s in E: from s to v to w. The shortest way to do this is dist[v] + weight (v, w) by induction. Also by induction, the shortest path to get to w not using edge (v, w) is dist[w]. The algorithm then replaces dist[w] by the minimum of its old value and possible new value. Thus this w is still the shortest distance to w in E. See Fig. We still have to show that all the other dist[u] values are correct. For this we need to use the heap property, which guarantees that (v,w) is an edge coming out of the node v, in the heap, closest to the source s. To complete the induction we have to show that adding the edge (v, w) to the graph G' does not change the values of the distance of any other vertex u from s. dist[u] could only change, had the shortest path from s to u been previously through w. This is impossible, since we just decided that v was closer to s than w, since the shortest path to w is via v, so the heap would not have marked wand examined edges out of it. This algorithm has a time complexity O(n2), which can be easily derived from the consideration of the number of times each loop is executed. 29

30 OPTIMAL MERGE PATTERNS One of the usual methods of creating large sorted files is to prepare a number of smaller files, possibly from different sources, sort them and then merge them into a single large file. In fact this procedure can be executed recursively. A question arises regarding the optimum order in which the files should be merged, so that the total time to get the final file is minimized. If we have two files f1 and f2 with sizes s1 and s2 respectively, then the merging time is O(s1 + s2). Given n files, f1,f2,f3,...,fn what's the minimum time needed to merge all the n files and how to achieve it? If we could merge all the n files together in a single operation, the time taken is expected to be O(s1 + s2... sn), but multi-way merging does add its own overhead and does take more time than O(s1 + s2... sn). It is usual to merge two files at a time. EXAMPLE It may not be very obvious that the total merging time depends upon the order in which the merging is done, assuming that at a time two files are merged. For example, let us say we have 5 files (f1,f2,f3,f4,f5), with lengths (20, 30, 10, 5, 30). If we merge them in some arbitrary order, say, in the order in which the files are given to us, we get the following result: 30

31 On the other hand, if we do optimal merging, using Greedy strategy, the total time may reduce. We take the following steps: 1) 2) Sort the array giving the file lengths. Thus for our example, the sorted length array is (5, 10, 20, 30, 30). Sort the file numbers also in the same order, i.e. (f4,f3,f1,f2,f5). Now do the two-way merging as before: Just by changing the order of merging we did get a time reduction. Note that in the second case, we took up the files in an ascending sorted order keyed on the file lengths. That gives us our first observation: Observation I: We should have a fast sorting method available. Though we did get a merging speed advantage in our rather naive approach of re-arranging the files, we did not get any where near the theoretical minimum time of = 95. Actually, we missed one important point-not only should we use the files in their original length order, but the ascending order should be dynamically decided. More specifically, after we obtained m2 = m1 +f1 with length 35, we should have considered merging f2 +f5 to get m3 with length 60. Let us try this and see what happens. 31

32 We did get some reduction in time. This gives us our second observation: Observation 2: we should have a fast sorting method, which works dynamically while the merging is in progress. The file length array to be sorted goes on changing while the merging steps are being done. It is not correct to sort only once at the beginning of the merge, we have to sort the length array after each merge step and then merge the smallest two files. It will give us our required Optimal Merge Pattern. The most convenient method of achieving the required dynamic sort is to use a minheap (a heap with the minimum element at the root) or a priority queue. We need to mainly use initialize (), deletemin () and insert () functions to respectively insert the initial file length values in the heap, get the current minimal two file lengths and re-insert their sum back in the heap. 32

33 We obtained the following results: We now formalize the problem a bit and show that we have indeed followed the Greedy approach. 33

34 GREEDY OPTIMAL MERGE PATTERNS Input: n sorted arrays of length L[1], L[2],..., L[n] Problem: To merge the arrays pairwise as fast as possible. The problem is to determine which pair to merge every time. Method (the Greedy method): The selection policy, which chooses the best pair of arrays to merge next, is to select the two shortest remaining arrays. Implementation: Need a data structure to store the lengths of the arrays, to find the shortest 2 arrays at any time, to delete those lengths, and insert in a new length (for the newly merged array). The data structure has to support deletemin and insert. Clearly, a minheap is ideal. The time complexity of the algorithm: The algorithm iterates (n-1) times. At every iteration two deletemins and one insert is performed. The 3 operations take O(log n) time in each iteration. Thus the total time is O(n log n) for the while loop plus O(n) for initial heap construction. That is, the total time is O(n log n). 34

35 THE GREEDY ALGORITHM 1) 2) Store the files names in a minheap, keyed by their lengths. Repeat the following until there is only one file: a) b) Extract the two smallest elements fi and fj. Merge fi and fj and insert this new file with its length in the minheap. To show that we have followed the Greedy method, we first note its characteristics: Optimal substructure: An optimal solution contains within it optimal solutions to subproblems. Greedy choice property: A globally optimal solution can be arrived at by making a locally optimal choice. Greedy algorithm: Makes the choice that looks the best at each particular point in the algorithm execution. Then we note the problem solution design 1) 2) 3) Cast the problem as one where you make choices and each choice results in a smaller subproblem to solve. Prove that the greedy choice property holds. Demonstrate that the solution to the subproblem can be combined with the greedy choice to get an optimal solution for the original problem. 35

36 Please see previous figure for a merge tree for our example. Total merging cost for a merge tree T=sum of the internal nodes. Thus Optimal Substructure If we look at any subtree T' in the merging tree T, it represents the cost of merging the files that are at its leaves. The cost of the optimal merge tree includes the cost of this subtree C(T') plus the cost of merging the files in T' with the rest of the files in T. The merging tree T' must be optimal in the optimal solution or else we would be able to construct an even better solution. GREEDY CHOICE PROPERTY Claim: There exists an optimal merging pattern such that the two shortest files are merged together. If this is true, we can therefore greedily pick the two shortest files and merge them together and get an optimal solution. Proof: Suppose the two shortest files fi and fj are not merged together in some optimal merging pattern. Find the two deepest nodes that are merged together and swap them with fi and fj. The cost of the resulting merge pattern is at most that of the merging pattern we had before since the cost is We have now constructed an optimal merging pattern that merges the two smallest files. Greedy Algorithm is already given above. 36

37 REFERENCE VIDEOS Websites for algorithm learning Permutation using recursion Knapsack problem ithms/design_and_analysis_of_algorithms_fractional_knapsa ck.htm Shortest Path(Dijkstra Algorithm) THANK YOU Dr. Milan Vachhani Assistant Professor, Sunshine Group of Institutions, Rajkot 37