Greedy Algorithms. This is such a simple approach that it is what one usually tries first.

Transcription

1 Greedy Algorithms A greedy algorithm tries to solve an optimisation problem by making a sequence of choices. At each decision point, the alternative that seems best at that moment is chosen. This is such a simple approach that it is what one usually tries first. How does one choose the best alternative? By using a priority queue. Recall that in Prim s algorithm to find a minimal spanning tree for a weighted connected undirected graph, a priority queue is used to make the choices. Each iteration of the while loop begins with the extraction of a vertex from the priority queue. The vertex with the smallest (best) priority is chosen. Similarly, in Dijkstra s algorithm, each iteration of the while loop also begins by extracting the vertex with the smallest priority value from the priority queue. COSC242 Lecture 21 / Slide 1

2 Knapsack problems Consider now a totally different kind of optimisation problem. Suppose we are given a set S = {s 1, s 2,..., s n } where each item s i has a positive benefit (or value) v i and has a weight (or cost) w i. Take v i and w i to be integers. Suppose we want to choose a maximum-benefit subset that doesnt exceed a given weight w max (like a thief who wants to load up his knapsack with valuables but can t carry more than W kilograms, and wants to know which items to pack in). If we are restricted to entirely accepting or rejecting each item, we have the 0-1 Knapsack Problem. (Think of the thief loading up gold bars of various weights.) If we are allowed to take fractions of items, we have the Fractional Knapsack Problem. (Think of bags of gold dust instead of solid bars of gold.) Let us try to use a greedy strategy to solve first the Fractional and then the 0-1 Knapsack Problem. COSC242 Lecture 21 / Slide 2

3 Fractional knapsack problem 1: procedure FRACKNAP(S, V, W, w max ) 2: Initiallise priority queue Q 3: for each s i S do 4: p i = v i w i 5: Q.enqueue(s i ) using p i as priority. 6: end for 7: current weight 0 8: set knapsack to empty 9: while current weight < w max do 10: s k Q.dequeue() 11: x k min(w k, w max current weight) 12: current weight current weight +x k 13: add x k w of s k to knapsack k 14: end while 15: end procedure p i is normalised value COSC242 Lecture 21 / Slide 3

4 0-1 Knapsack problem Let s try the greedy algorithm on the 0-1 knapsack problem. In this version we have to put either all of an item, or none of an item in the knapsack. What is the greedy solution for the above problem? What is the optimal solution? COSC242 Lecture 21 / Slide 4

5 Solving the 0-1 knapsack problem Given: a set S = {s 1, s 2,..., s n } where each item s i has a positive benefit (or value) v i and has a weight (or cost) w i. Take v i and w i to be integers. A maximum total weight, w max. Required: to choose a subset of S such that the total weight does not exceed w max and the sum of the values v i is maximal. Let V be a 2D array storing the maximum value possible using the first k items in S (call those k items, S k ), and maximum total weight w. If we remove the k th item from S k (call it S k 1 ), then the resulting set must be the optimum for the problem with a maximum total weight of w w k. Why? COSC242 Lecture 21 / Slide 5

6 Recursive solution We are left with the following observations: If there are no items in our set (S 0 ), then the maximum value is 0. If there is no space in our knapsack, then the maximum value is 0 If the k th item can t fit in the knapsack, then the maximum is the same as the maximum for k 1 items. Otherwise, the maximum is either: the maximum without the k th item in the optimal set, in which case we have a new problem with k 1 items and maximum weight w. the maximum with the k th item in the optimal set, in which case we have a new problem with k 1 items and maximum weight w w k. So we can define our optimum V [k, w] recursively as: V [0, w] = 0 V [k, 0] = 0 V [k, w] = V [k 1, w] if w k > w V [k, w] = max(v [k 1, w], v k + V [k 1, w w k ]) COSC242 Lecture 21 / Slide 6

7 The Algorithm 1: function RECURSIVEKNAPSACK(k, W, V, wmax) 2: if k==0 or wmax 0 then 3: return 0, φ 4: end if 5: if W[k]>wmax then Can t fit k into knapsack 6: return RecursiveKnapsack(k-1,W,V,wmax) 7: end if 8: Check the maximum value without k 9: v1, items not RecursiveKnapsack(k-1,W,V,wmax) 10: Check the maximum value with k 11: v2, items do RecursiveKnapsack(k-1,W,V,wmax-W[k]) 12: v2 v2 + V[k] add the value of item k 13: items do.add(k) add item k to the list 14: if v2>v1 then 15: return v2, items do do use k 16: else 17: return v1, items not don t use k 18: end if 19: end function What is the complexity of this function? COSC242 Lecture 21 / Slide 7

8 Can we be more efficient? In RecursiveKnapsack, W and V don t change, so the only things that change in different recursive calls are k and wmax. k can be any integer from 1 to n and wmax can be any integer from 1 to w max. So we should be able to produce a solution in O(nw max ). The reason why RecursiveKnapsack is so expensive is because it recomputes the same values over and over again. If we could store those values when they re computed and then retrieve them when needed, we could save ourselves a lot of computation. This technique is called memoisation. Alternatively, we could write an iterative version of the algorithm. COSC242 Lecture 21 / Slide 8

9 Memoised Version 1: initiallise global memo[n,w max ] as 2D array, set all to -1 2: function KNAPMEMO(k, W, V, wmax) 3: if k==0 or wmax 0 then return 0, φ 4: end if 5: if memo[k,wmax] 1 then return memo[k,wmax] 6: end if 7: if W[k]>wmax then Can t fit k into knapsack 8: memo[k,wmax] KnapMemo(k-1,W,V,wmax) 9: else 10: v1, items not KnapMemo(k-1,W,V,wmax) 11: v2, items do KnapMemo(k-1,W,V,wmax-W[k]) 12: v2 v2 + V[k] add the value of item k 13: items do.add(k) add item k to the list 14: if v2>v1 then 15: memo[k,wmax] v2, items do do use k 16: else 17: memo[k,wmax] v1, items not don t use k 18: end if 19: end if 20: return memo[k,wmax] 21: end function COSC242 Lecture 21 / Slide 9

10 Iterative Version 1: function KNAPITER(n, W, V, wmax) 2: initiallise bestval[n,wmax] as 2D array, set all to 0 3: initiallise bestset[n,wmax] as 2D array, set all to φ 4: for k 1 to n do 5: for w 1 to wmax do 6: if W[k]>w then 7: bestval[k,w] bestval[k-1,w] 8: bestset[k,w] bestset[k-1,w] 9: else 10: withkval V[k] + bestval[k-1,max(w-w[k],0)] 11: if bestval[k-1,w] > withkval then 12: bestval[k,w] bestval[k-1,w] 13: bestset[k,w] bestset[k-1,w] 14: else 15: bestval[k,w] withkval 16: bestset[k,w] bestset[k-1,max(w-w[k],0)]+k 17: end if 18: end if 19: end for 20: end for 21: return bestval, bestset 22: end function COSC242 Lecture 21 / Slide 10

11 Exercises With these algorithms, it is really worth running through them on paper. That will give you a much better understanding of what is going on if you re not sure. 1. Run through each of the algorithms RecursiveKnapsack, KnapMemo, and KnapIter on paper with the following (easy) problem: W = {20, 30, 10}, V = {100, 120, 60}, w max = 50. Because there are 3 items, n = 3. COSC242 Lecture 21 / Slide 11