Analysis of Algorithms Prof. Karen Daniels

Transcription

1 UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 2 Tuesday, 2/2/10 Design Patterns for Optimization Problems Greedy Algorithms

2 Algorithmic Paradigm Context Divide & Conquer Dynamic Programming Greedy Algorithm View problem as collection of subproblems Recursive nature Independent subproblems overlapping typically sequential dependence Number of subproblems depends on partitioning factors typically small Preprocessing typically sort Characteristic running time typically log function of n often dominated by nlogn sort Primarily for optimization problems Optimal substructure: optimal solution to problem contains within it optimal solutions to subproblems Greedy choice property: locally optimal produces globally optimal Heuristic version useful for bounding optimal value Subproblem solution order depends on number and difficulty of subproblems Solve subproblem(s), then make choice Make choice, then solve subproblem(s)

3 Greedy Algorithms

4 What is a Greedy Algorithm? Solves an optimization problem Optimal Substructure: optimal solution contains in it optimal solutions to subproblems Greedy Strategy: At each decision point, do what looks best locally Choice does not depend on evaluating potential future choices or presolving overlapping subproblems Top-down algorithmic structure With each step, reduce problem to a smaller problem Greedy Choice Property: locally best = globally best

5 A Greedy Strategy Approach source: textbook Cormen,, et al. 1. Determine the optimal substructure of the problem. 2. Develop a recursive solution. 3. Prove that, at any stage of the recursion, one of the optimal choices is the greedy choice. 4. Show that all but one of the subproblems caused by making the greedy choice are empty. 5. Develop a recursive greedy algorithm. 6. Convert it to an iterative algorithm. With experience, it is also possible to directly devise a (possibly iterative) greedy strategy.

6 Examples of Greedy Algorithms Activity Selection Minimum Spanning Tree Dijkstra Shortest Path Huffman Codes Fractional Knapsack

7 Activity Selection

8 Activity Selection Problem Instance: Optimization Problem Set S = {a{ 1, a 2,..., a n } of n activities Each activity a i has: start time: s i finish time: f i si f i Activities i, j are compatible iff non-overlapping: overlapping: [ si fi ) and [ s j f j ) and si f j or s j fi Objective: and ( ) select a maximum-sized set of mutually compatible activities source: textbook Cormen,, et al.

9 Activity Selection Activity Time Duration Activity Number

10 Algorithmic Progression Brute-Force Check all possible solutions Exponential number of subproblems! Dynamic Programming Quadratic number of subproblems Greedy Algorithm Linear number of subproblems

11 Activity Selection: Dynamic Programming Formulation S ij = { ak S : fi sk < fk s j} Solution to S ij including a k produces 2 subproblems: 1) S ik (start after a i finishes; finish before a k starts) 2) S kj (start after a k finishes; finish before a j starts) c[i,j]=size of maximum-size subset of mutually compatible activities in S ij. c[ i, j] = max a k 0 if Sij = 0/ + + / S { c[ i, k] c[ k, j] 1} if Sij 0 ij source: textbook Cormen,, et al.

12 Activity Selection: Recursive Greedy Algorithm S k = { ai S : si fk} Only 1 subproblem to solve at each level. High-level call: REC-ACTIVITY ACTIVITY-SELECTOR( SELECTOR(s,f,0,,0,n) Returns an optimal solution for S source: textbook Cormen,, et al. (3 rd edition)

13 source: web site accompanying textbook Cormen,, et al.

14 Algorithm: Activity Selection: Iterative Greedy source: textbook Cormen,, et al. Presort activities in S by non-decreasing finish time and renumber. Running time? source: textbook Cormen,, et al. (3 rd edition)

15 Streamlined Greedy Strategy Approach 1. View optimization problem as one in which making choice leaves one subproblem to solve. 2. Prove there always exists an optimal solution that makes the greedy choice. 3. Show that greedy choice + optimal solution to subproblem optimal solution to problem. Greedy Choice Property: locally best = globally best source: textbook Cormen,, et al.

16 Minimum Spanning Tree

17 Minimum Spanning Tree Time: O( E lg E ) ) given fast FIND-SET, UNION Time: O( E lg V ) ) = O( E lg E ) ) slightly faster with fast priority queue Invariant: Minimum weight spanning forest Becomes single tree at end Where are the greedy choices made? Invariant: Minimum weight tree Spans all vertices at end Produces minimum weight tree of edges that includes every vertex. A 2 B 4 3 G E F 4 D 2 C for Undirected, Connected, Weighted Graph G=(V,E) source: textbook Cormen et al. (Ch 23)

18 Dijkstra Shortest Path

19 Single Source Shortest Paths: Dijkstra s Algorithm for (nonnegative) weighted, directed graph G=(V,E) A 2 B 4 3 G E F D 4 C 2 We will revisit this algorithm when we study shortest paths. source: textbook Cormen et al. (Ch 24)

20 Huffman Codes

21 source: web site accompanying textbook Cormen,, et al. Huffman Code Motivation Prefix Code: : No code is a prefix of any other.

22 Huffman Code Tree Example Goal: : Minimize number of bits required to encode a file. ( c. freq)( d T ( c ) B ( T ) = ) c C length of codeword for character c source: textbook Cormen,, et al.

23 source: web site accompanying textbook Cormen,, et al. Code Tree Comparison Fixed-length code: not optimal Huffman prefix code: optimal

24 Example of Huffman Code Steps source: web site accompanying textbook Cormen,, et al.

25 source: web site accompanying textbook Cormen,, et al. Huffman Code Greedy Algorithm Running Time?

26 source: web site accompanying textbook Cormen,, et al. Huffman Correctness: A Key Idea Lemma 16.2: : Let C be an alphabet in which each character c in C has frequency c.freq.. Let x and y be 2 characters in C with lowest frequencies. Then there exists an optimal prefix code for C in which codewords x and y have the same length and differ only in the last bit. (T represents arbitrary optimal prefix code.) (Claim: T also has optimal cost.)

27 Fractional Knapsack

28 Value: $60 $100 $120 Knapsack Each item has value and weight. Goal: : maximize total value of items chosen, subject to weight limit. fractional: can take part of an item 0-1: take all or none of an item item1 item2 item3 knapsack

29 source: web site accompanying textbook Cormen,, et al.

30 Greedy Heuristic If optimization problem does not have greedy choice property,, greedy approach may still be useful as a heuristic in bounding the optimal solution Example: minimization problem Solution Values Upper Bound (heuristic) Optimal (unknown value) Lower Bound