Problem Strategies. 320 Greedy Strategies 6

Transcription

1 Problem Strategies Weighted interval scheduling: 2 subproblems (include the interval or don t) Have to check out all the possibilities in either case, so lots of subproblem overlap dynamic programming: 1 D array for subproblem answers, backtracking for overall problem answer Merge sort 2 subproblems (roughly half of the items to sort in each) No subproblem overlap divide and conquer: create final answer as you solve subproblems recursively 320 Greedy Strategies 6

2 Problem Strategies Interval scheduling How many subproblems? Strategy? Subset sums (subset with largest sum of weights) How many subproblems? Strategy? Rod cutting How many subproblems? Strategy? Inversion count merge sort for rank analysishow many subproblems? Strategy? Matrix chain multiply How many subproblems? Strategy? 320 Greedy Strategies 7

3 Greedy Strategies CS 320, Fall 2017 Dr. Geri Georg, Instructor 320 Greedy Strategies 8

4 Greedy Algorithms Always make the choice that looks best at the moment a locally optimal choice. Assumes that the choice will lead to a globally optimal solution to the overall problem. Optimization problems solved by dynamic programming can often be improved using greedy algorithms that don t explore every choice. Not all problems have a greedy solution. 320 Greedy Strategies 9

5 Greedy in the Scheme of Strategies Divide and Conquer Roughly equal size DISJOINT subproblems that can be solved recursively (top down) that can be combined Dynamic Programming Optimal solution to the overall problem constructed from optimal solutions to subproblems that OVERLAP so storing their results saves time. Usually bottom up subproblem (can be top down memorization) algos. Greedy Strategy Don t need to explore ALL the choices explored in Dynamic Programming, so don t need to save results to subproblems. 320 Greedy Strategies 10

6 Greedy Strategy 1. Determine optimal problem substructure 2. Develop recursive solution 3. Show that if we make the greedy choice only 1 subproblem remains 4. Prove it is always safe to make the greedy choice 5. Develop a recursive algorithm for the greedy strategy 6. Convert the recursive algorithm into an iterative algorithm 320 Greedy Strategies 11

7 Key Elements Optimal substructure (#1): an optimal solution to the problem contains within it optimal solutions to subproblems Greedy choice property (#4): we can assemble a globally optimal solution by making locally optimal choices Choices may depend on choices made so far, but cannot depend on any future choices or on solutions to future subproblems: therefore a top down algorithm making a choice and reducing the problem instance to a smaller one 320 Greedy Strategies 12

8 Interval Scheduling We used a greedy strategy let s see why we can: Problem: Find the largest set of compatible intervals that can scheduled using a single resource. Sort in increasing order of finish time: 320 Greedy Strategies 13

9 Greedy Strategy 1. Determine optimal problem substructure S ij : set of intervals that start after a i finishes and finish before a j starts E.g., S 1,11 = {4,6,7,8,9} A ij : max set of mutually compatible intervals in S ij, assume it includes a k. This interval splits the problem into 2 subproblems: finding max set of compatible intervals in S ij that start after a i finishes and before a k starts, and the set that start after a k finishes and before a j starts. A ij = A ik {a k } A kj Max set size is A ij = A ik + A kj Greedy Strategies 14

10 Proof of optimal substructure The optimal solution A ij must contain optimal solutions to the 2 subproblems for S ik and S kj. Max set size is A ij = A ik + A kj + 1 Assume we found a better set for one of the subproblems: A kj > A kj, then we could use it in a solution to the subproblem for S kj : A ik + A kj + 1 > A ik + A kj + 1 A ik + A kj + 1 > A ij, but we said that A ij was an optimal solution so this is a contradiction. 320 Greedy Strategies 15

11 Greedy Strategy 3. Show that if we make the greedy choice only 1 subproblem remains Greedy choice: Choose the activity in S with the earliest finish time (a k ) since that will leave the most time to schedule other activities. How many remaining subproblems? Just 1: find all the activities that start after the one we chose finishes (a k ), and choose the one with the earliest finish time. S k = {a i S : s i f k } 320 Greedy Strategies 17

12 Greedy Choice Property Proof Examine a globally optimal solution to some subproblem Modify the solution to substitute the greedy choice for some other choice that results in a similar subproblem 320 Greedy Strategies 18

13 Greedy Strategy 4. Prove it is always safe to make the greedy choice S k = {a i S : s i f k } Theorem: S k, a m S k with the earliest finish time. Then a m is included in some max size subset of mutually compatible activities of S k, called A k. Proof: Assume a j A k with the earliest finish time. If a j a m, create A k by substituting a m for a j. Now a j is the first activity to finish in A k, and f m f j. Since A k = A k, then A k is a max subset and it includes a m 320 Greedy Strategies 19

14 Optimal Substructure Required for both Dynamic Programming and Greedy strategies. How can you tell if Greedy is appropriate? Consider Knapsack problem extension to Subset Sums problem we looked at for Dynamic Programming. Set of objects with weights and a maximum weight, but now the objects have value. So the problem is finding the set with the largest value that doesn t exceed the weight limit. 320 Greedy Strategies 20

15 Knapsack 0/1 version: include the object or don t Fractional version: include part of the object Fractional version: Figure out value/unit weight of each item, then take the most possible from the most value/wt item. If you run out, then go to the next most value/wt, etc. The choice of what to include does not depend on solutions to subproblems 320 Greedy Strategies 21

16 0/1 version: Decision on what to include requires looking at subproblems what if we included the item versus what if we didn t include it We have to have the solutions to these subproblems before we can make the decision, so a greedy strategy won t work. 320 Greedy Strategies 22

17 Huffman Codes The problem is to compress data; we ll think about the data as being a sequence of characters. Assume we just collected 2K bytes of raw data from an observation using the solar telescope on Mauna Kea, and this data is in the form of characters. We ll be making 200 different observations during our session. When the session is over we ll need to ship all the observation data over the net to the Cray computer here at CSU. 320 Greedy Strategies 23

18 Huffman Codes Each 2K bytes consists of 16 bit UTF characters, which means 2 bytes/character, so we have 1K characters in our 2K of raw data. At the end of the session we ll end up with 400K of raw data, consisting of 200K characters. We decide to compress it. Huffman invented a greedy algorithm to do this. 320 Greedy Strategies 24

19 Huffman s Algorithm Greedy algorithm Uses a table of character frequency to create an optimal code to represent each character as a binary string Uses a prefix code no code is a prefix of some other code Uses a variable length code with higherfrequency characters having shorter codes 320 Greedy Strategies 25

20 Example Our data has 10 unique characters with the following frequencies (all frequencies x 1K): b d k c e g f a h j If we coded these characters with a fixedlength code it would take 4 bits/character x 200K characters is 800K bits = 100K bytes which is better than 400K bytes where we started. With variable length codes we can do even better. 320 Greedy Strategies 26

21 Build a tree from the leaves Start with leaves sorted by frequency: j:3 h:5 a:12 f:17 g:18 e:19 c:20 k:22 d:30 b:54 Take 2 smallest and create a tree with a root summing their frequencies and insert it into the proper sorted location, label lines 0, j:3 h:5 a:12 f:17 g:18 e:19 c:20 k:22 d:30 b: Greedy Strategies 27

22 Repeat Start with leaves sorted by frequency: f:17 g:18 e:19 c:20 20 k:22 d:30 b: j:3 h:5 a: Greedy Strategies 28

23 Image Credits clustercomparison: GLay: community structure analysis of biological networks, Gang Su, Allan Kuchinsky, John H. Morris, David J. States, and Fan Meng, BIOINFORMATICS APPLICATIONS NOTE,Vol. 26 no , 2 Comparison between clusters produced by MCODE with default parameters left and 320 Greedy Strategies 30