1 DESIGN AND ANALYSIS OF ALGORITHMS UNIT II Objectives GREEDY METHOD Explain and detail about greedy method Explain the concept of knapsack problem and solve the problems in knapsack Discuss the applications of minimum spanning tree Compare prim s and kruskal algorithm Explain the single source shortest path problem Compare Floyd s and warshall s algorithm 2. Greedy method This is another approach that is often used to design algorithms for solving. In contrast to dynamic programming, however, Greedy algorithms do not always yield a genuinely optimal solution. In such cases the greedy method is frequently the basis of a heuristic approach. Even for problems which can be solved exactly by a greedy algorithm, establishing the correctness of the method may be a non-trivial process. The greedy method has, Most straightforward design technique Most problems have n inputs Solution contains a subset of inputs that satisfies a given constraint Feasible solution: Any subset that satisfies the constraint Need to find a feasible solution that maximizes or minimizes a given objective function optimal solution Used to determine a feasible solution that may or may not be optimal At every point, make a decision that is locally optimal; and hope that it leads to a globally optimal solution _ The OPT algorithm for process scheduling, and its variant SRTN, in operating systems May not guarantee the best solution Ultimate goal is to find a feasible solution that minimizes [or maximizes] an objective function; this solution is known as an optimal solution. 2. Optimal Storage on tapes There are 'n' programs that are to be stored on a computer tape of length 'l'.
2 Associated with each other i is the length li, 1<=i<=n. All the programs can only be written on the tape if the sum of all the lengths of the program is at most l. Assumption: that whenever a program is to be retrieved from the tape, the tape is positioned at the front. Now if the programs are stored in the order as I=i1,i2,i3,i4...,in, then the time tj needed to retrieve program (i,j) is proportional to 1 k j lik. So the problem is that we have to store them in the tape in such an order that the M.R.T (mean retrieval time) is minimum. A greedy approach to build such an algorithm requires such a permutation that chooses the next program on the basis of some optimization measure. In this we take the optimization measure to be 'd' (which is the current length of the tape that is written with the program) So now every time when we write onto the disk, we keep in mind that the increment in the length of 'd' is minimum. So this greedy method requires us to just sort the lengths of the programs or to assign them in a non-decreasing order, so that they can be directly written into disk on their length basis. So it in turns takes the complexity equal to O(n logn) just to sort the length of the programs. Here's a sample pseudo code for the program if we have multiple disks on which we have to write the data Pseudo tapes(n,m) // here n is the numbers of programs and m is the numbers of tapes { j=0; for(i=1 to n do) { write ("append", i "to the permutation of the tape", j) j=(j+1) mod m } } If we want to store files of lengths (in MB) {12,34,56,73,24,11,34,56,78,91,34,91,45} on three tapes. First sort the files in increasing order of length. For this we can use heapsort, meregesort or quicksort algorithms. Now distribute the files: First element 11
3 Finally we get, 3. Knapsack problem Knapsack problem is one of the classical optimization problems. Description: given a set of items each with a cost and a value, determine the number of each item to include in a collection so that the total cost does not exceed some given cost and the total value is as large as possible. Different variations: 1. Unbounded knapsack problem: no limits on the number of each item 2. 0-1 knapsack problem: number of each item is either 0 or 1. 3. Fractional knapsack problem: number of each item can be a fractional. 4. Subset problem: Decision problem 0-1 problem For each item cost is equal to value The goal of Knapsack problem is, Choose only those objects that give maximum profit. The total weight of selected objects should be <=W Example: Consider that there are three items. Weight and profit value of each item is as given below. I Wi Pi
4 1 18 30 2 15 21 3 10 18 And weight W=20. Solution: X1 X2 X3 ½ 1/3 ¼ 1 2/15 0 0 2/3 1 0 1 ½ Let us compute WiXi 1. ½*18+1/3*15+1/4*10=16.5 2. 1*18+2/15*15+0*8=20 3. 0*18+2/3*15+10=20 4. 0*18+1*15+1/2*10=20 Let us compute PiXi 1. ½*30+1/3*21+1/4*18=26.5 2. 1*30+2/15*21+0*18=32.8 3. 0*30+2/3*21+18=32 4. 0*30+1*21+1/2*18=30 To summarize this, we get Solution WiXi PiXi 1 16.5 26.5 2 20 32.8 3 20 32 4 20 30 Algorithm Knapsack_greedy(W,n) For i:=1 to n do If(w[i]<W) then X[i]=1.0 W=W-w[i] If(i<=n) then X[i]=W/w[i]; 4. Minimum spanning trees A tree is defined to be an undirected, acyclic and connected graph (or more simply, a graph in which there is only one path connecting each pair of vertices). A spanning tree is a subgraph of G, is a tree, and contains all the vertices of G. A minimum spanning tree is a spanning tree, but has weights or lengths associated with the edges, and the total weight
5 of the tree (the sum of the weights of its edges) is at a minimum. Here are some examples: A weighted graph G: Three (of the many possible) spanning trees from graph G: The minimum spanning tree from weighted graph G: Types of Minimum spanning trees: 1. Prim s algorithm 2. Kruskal s algorithm 4.1. Prim s algorithm Prim's algorithm is based on a generic MST algorithm. The main idea of Prim's algorithm is similar to that of Dijkstra's algorithm for finding shortest path in a given graph. Prim's algorithm has the property that the edges in the set A always form a single tree. We begin with some vertex v in a given graph G =(V, E), defining the initial set of vertices A. Then, in each iteration, we choose a minimum-weight edge (u, v), connecting a vertex v in the set A to the vertex u outside of set A. Then vertex u is brought in to A. This process is repeated until a spanning tree is formed. Like Kruskal's algorithm, here too, the important fact about MSTs is we always choose the smallest-weight edge joining a vertex inside set A to the one outside the set A. The implication of this fact is that it adds only
6 edges that are safe for A; therefore when the algorithm terminates, the edges in set A form a MST. Analysis The performance of Prim's algorithm depends of how we choose to implement the priority queue Q. Definitions Sparse graphs are those for which E is much less than V 2 i.e., E << V 2 we preferred the adjacency-list representation of the graph in this case. On the other hand, dense graphs are those for which E is graphs are those for which E is close to V 2. In this case, we like to represent graph with adjacency-matrix representation.
DESIGN AND ANALYSIS OF ALGORITHM BY MR.T.GOBINATH ME., 7 4.2. Kruskal s algorithm The Kruskal Algorithm starts with a forest which consists of n trees. Each and everyone tree, consists only by one node and nothing else. In every step of the algorithm, two different trees of this forest are connected to a bigger tree. Therefore, we keep having less and bigger trees in our forest until we end up in a tree which is the minimum genetic tree (m.g.t.).in every step we choose the side with the least cost, which means that we are still under greedy policy. If the chosen side connects nodes which belong in the same tree the side is rejected, and not examined again because it could produce a circle which will destroy our tree. Either this side or the next one in order of least cost will connect nodes of different trees, and this we insert connecting two small trees into a bigger one.
DESIGN AND ANALYSIS OF ALGORITHM BY MR.T.GOBINATH ME., 8 Kruskal Algorithm looks easier than Prim s but is harder to implement (checking for cycles!) less efficient Θ(m log m)
9 5. Single source shortest path method Dijkstra s algorithm is used to find the shortest paths from a single source vertex to all other vertices in a weighted, directed graph. All weights must be nonnegative. Dijkstra's algorithm keeps two sets of vertices: S V- the set of vertices whose shortest paths from the source have already been determined and the remaining vertices S Assumptions: The graph is connected the edges are undirected the edge weights are nonnegative The basic mode of operation is: 1. Initialize d and pi, 2. Set S to empty, 3. While there are still vertices in V-S, i. Sort the vertices in V-S according to the current best estimate of their distance from the source, ii. Add u, the closest vertex in V-S, to S, iii. Relax all the vertices still in V-S connected to u
10 and edges is The worst-case running time for the Dijkstra algorithm on a graph with nodes because it allows for directed cycles. It even finds the shortest paths from a source node to all other nodes in the graph. This is basically for node selection and for distance updates. While is the best possible complexity for dense graphs, the complexity can be improved significantly for sparse graphs. Summary Greedy algorithms do not always yield a genuinely optimal solution. In such cases the greedy method is frequently the basis of a heuristic approach. A greedy approach to build such an algorithm requires such a permutation that chooses the next program on the basis of some optimization measure. Knapsack problem is one of the classical optimization problems. Knapsack problem - given a set of items each with a cost and a value, determine the number of each item to include in a collection so that the total cost does not exceed some given cost and the total value is as large as possible. A spanning tree is a subgraph of G, is a tree, and contains all the vertices of G.
11 The worst-case running time for the Dijkstra algorithm on a graph with nodes And edges is because it allows for directed cycles. Key Terms >> greedy method >> knapsack >> minimum spanning tree >> prims >> kruskal >> dijkstra Key Term Quiz 1. Greedy method is frequently the basis of a ---------------- approach. 2. ---------------- algorithm requires such a permutation that chooses the next program on the basis of some optimization measure. 3. ------------------- given a set of items each with a cost and a value, determine the number of each item to include in a collection so that the total cost does not exceed some given cost and the total value is as large as possible. 4. ------------------ algorithm is used to find the shortest paths from a single source vertex to all other vertices in a weighted, directed graph. All weights must be nonnegative. 5. The worst-case running time for the Dijkstra algorithm on a graph with nodes and edges is ------------------- because it allows for directed cycles. 6. ------------------ algorithm is one of the single source shortest path problem. 7. Prim s algorithm is used to find ----------------------- Objective Type Questions 1. Greedy technique is (a)is the process of choosing the best ones among the choices (b)is the process of bounding the unwanted solutions. (c)is the method that uses the state space tree. (d)none of the above 2. Knapsack problem can be solved by using (a)greedy technique (b)dynamic programming (c)backtracking (d)all the above 3. Which one of the following algorithm is used to find the minimum spanning tree? (a) prim s algorithm
12 (b) dijkstra s algorithm (c) knapsack (d) none of the above 4. Which one of the following algorithm is used to find the single source shortest path problem? (a) prim s algorithm (b) dijkstra s algorithm (c) knapsack (d) none of the above 5. What is the minimum spanning tree of the following graph? 6. Which one of the following is true? (a) Kruskal Algorithm looks easier than Prim s (b) Kruskal Algorithm is harder to implement than prim s ( c) Kruskal Algorithm is less efficient than prim s (d) All the above 7. The running time of Kruskals algorithm is (a) Θ (log n) (b) Θ (n log m) (c) Θ (n2) (d) Θ(m log m) Review Questions Two Mark Questions 1. What is a minimum cost spanning tree? 2. How efficient is prim s algorithm? 3. Compare Prim s and Kruskal s algorithm. 4. What is the running time of Kruskals a algorithm. 5. Write short notes on Kruskal s algortithm
13 6. Define dijkstra s algorithm 7. What is the use of knapsack s problem? 8. What are the three different ways to implement knapsack problem? 9. Define greedy method. 10. Define spanning tree. 11. Define DAG. 12. Define Graph. Big Questions 1. Apply Prim's Algorithm and Kruskal algorithm to the graph to obtain minimum spanning tree. 2. Solve the following instance of the single-source shortest paths problem with vertex a as the source. 3. Explain how the knapsack problem is solved using greedy method. 4. Write short notes on optimal storage on tapes. ---------- END OF SECOND UNIT -----------