Page 1. CSCE 310J Data Structures & Algorithms. CSCE 310J Data Structures & Algorithms. Greedy Algorithms and Graph Optimization Problems

Transcription

1 CSCE J Data Strctres & Algorithms Greedy Algorithms and Graph Optimization Problems Dr. Steve Goddard goddard@cse.nl.ed CSCE J Data Strctres & Algorithms Giving credit where credit is de:» Most of slides for this lectre are based on slides created by Dr. David Lebke, University of Virginia.» Some examples and slides are based on lectre notes created by Dr. Chck Csack, UNL, Dr. Jim Cohoon, University of Virginia, and Dr. Ben Choi, Loisiana Technical University.» I have modified them and added new slides Greedy Algorithms Main Concept: Make the best or greedy choice at any given step.» This is what yo did before yo learned to not to think ahead and plan Choices are made in seqence sch that» Each individal choice is best according to some limited shortterm criterion, that is not too expensive to evalate» Once a choice is made, it cannot be ndone! Even if it becomes evident later that it was a poor choice Sometimes life is like that The goal is to make progress by choosing an action that» Incrs the minimm short-term cost,» With the hope that a lot of small short-term costs add p to small overall cost. When to be Greedy Greedy algorithms apply to problems with» The greedy choice property: an optimal soltion can be obtained by making the greedy choice at each step.» Optimal sbstrctres: optimal soltions contain optimal sbsoltions.» Many optimal soltions, bt we only need one sch soltion. Unlike dynamic programming, we do not need to know the soltions to the sb-problems to make choices.» Hence, greedy algorithms are often more efficient than dynamic programming. Possible drawback:» Actions with a small short-term cost may lead to a sitation, where frther large costs are navoidable. 4 The Fractional Knapsack Problem A thief breaks into a cookie store. She has a bag that can hold p to C ponds of cookies. There are n cookies in the store. The i th cookie weighs w i ponds and is worth v i dollars. She can break the cookies and sell fractions of them. The thief wants to maximize the vale of the cookies she steals, of corse. How mch of each cookie shold she steal? Some Greedy Observations The i th cookie is worth p i = v i /w i dollars per pond. The item with the largest p i has the most bang for the bck, so the thief shold take as mch as she can of this item. If the thief takes x ponds of a cookie with p j < p i instead of cookie i, her profit will be smaller for the same weight. 6 Page

2 Fractional Knapsack Problem Problem» Given a set of n objects where object i has vale v i per cookie and weight w i and a knapsack capacity C, determine the fractional amont f i of each object i to be inclded in the knapsack sch that the profit is maximized while the weight of the inclded objects does not exceed the knapsack capacity n maximize v i f i sch that i= n i= w f C i i Strategy: Pick by Density Strategy» Sort objects in non-increasing order of profit p i = v i /w i That is, p =v /w p =v /w... p n =v n /w n» Consider objects by increasing order of sbscript.» When an object is considered choose a maximal amont sch that the knapsack capacity is not violated. where 0 f i 7 A Greedy Soltion fractionalknapsack(v, W, capacity, n, KnapSack) sortbydescendingprofit(v,w,n) KnapSack = 0; capacityleft = C; for (i = ; (i <= n) && (capacityleft > 0); ++i) if (W[i] < capacityleft) KnapSack[i] = ; capacityleft -= W[i]; else KnapSack[i] = capacityleft/w[i]; capacityleft = 0; What is the complexity of this algorithm? Knapsack Example The thief s knapsack holds ponds The cookie inventory has the following properties: i v i 4 w i p i Same properties sorted by profit i v i 4 w i 4 4 p i Knapsack Example Soltion The thief takes pond of cookie, 4 ponds of cookie, 4 ponds of cookie 6, ponds of cookie, and ponds of cookie 7: i v i 4 w i 4 4 / p i Ths, the profit is = $.6 Soltion property for non-trivial instances» knapsack = (,,..., f j, 0, 0,... 0) with 0 < f j Is it Optimal? 0/ Knapsack f i is restricted to be either 0 or Does pick by vale work? Does pick by weight work? Does pick by density work? Page

3 Minimm Spanning Tree (MST) A spanning tree for a connected, ndirected graph, G=(V,E) is. a connected sbgraph of G that forms an. ndirected tree incident with each vertex. In a weighted graph G=(V,E,W),» the weight of a sbgraph is the sm of the weights of the edges in the sbgraph. A minimm spanning tree (MST) for a weighted graph is» a spanning tree with the minimm weight. Minimm Spanning Tree Problem: given a connected, ndirected, weighted graph: find a spanning tree sing edges that minimize the total weight Minimm Spanning Tree Which edges form the minimm spanning tree (MST) of the below graph? Minimm Spanning Tree Answer: A H B C G E D F A H B C G E D F 6 Another Example Given a weighted graph G=(V, E,W), find a MST of G 7 MST Uniqeness Is a MST niqe? What if the weights are (or are not) niqe? Prove or disprove the niqeness of a MST for a given graph G. 7 Page

4 Minimm Spanning Tree MSTs satisfy the optimal sbstrctre property: an optimal tree is composed of optimal sbtrees» Let T be an MST of G with an edge (,v) in the middle» Removing (,v) partitions T into two trees T and T» Claim: T is an MST of G = (V,E ), and T is an MST of G = (V,E ) (Do V and V share vertices? Why?)» Proof: w(t) = w(,v) + w(t ) + w(t ) (There can t be a better tree than T or T, or T wold be sboptimal) Minimm Spanning Tree Thm:» Let T be MST of G, and let A T be sbtree of T» Let (,v) be min-weight edge connecting A to V-A» Then (,v) T Proof: left as an exercise 0 Finding a MST Principal greedy methods: algorithms by Prim and Krskal Prim» Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree Intermediary soltion is a sbtree Krskal» Grow a tree by repeatedly adding the least cost edge that does not introdce a cycle among the edges inclded so far Intermediary soltion is a spanning forest Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree Intermediary soltion is a sbtree Rn on example graph Rn on example graph 4 Page 4

5 r 0 Pick a start vertex r 0 Red vertices have been removed from Q 6 0 Red arrows indicate parent pointers Page

6 Page 6

7 Page 7

8 Review: What is the hidden cost in this code? Review: DecreaseKey(v, w(,v)); 4 44 Review: How often is ExtractMin() called? How often is DecreaseKey() called? DecreaseKey(v, w(,v)); 4 Review: What will be the rnning time? There are n= V ExtractMin calls and m= E DecreaseKey calls. Ths, the worst case is O(n +m). The priority Q implementation has a large impact on performance. E.g., O((n+m)lg n) = O(m lg n) sing binary heap for Q Can achieve O(n lg n + m) with Two-pass pairing or Fibonacci heaps 46 Grow a tree by repeatedly adding the least cost edge that does not introdce a cycle among the edges inclded so far Intermediary soltion is a spanning forest T = T,v; 7 T = T,v; 47 4 Page

9 7 T = T,v; 7 T = T,v; T = T,v;? 7 T = T,v;? 7 T = T,v; 7 T = T,v; 4 Page

10 7? T = T,v; 7 T = T,v; 6 7? T = T,v; 7 T = T,v; 7 7 T = T,v;? 7 T = T,v; 60 Page

11 7? T = T,v; 7 T = T,v; 6 6? 7 T = T,v; 7 T = T,v; ? T = T,v;? 7 T = T,v; 6 66 Page

12 7? T = T,v; 7? T = T,v; T = T,v; 7 T = T,v; 6 70 Correctness Of Krskal s Algorithm Sketch of a proof that this algorithm prodces an MST for T:» Assme algorithm is wrong: reslt is not an MST» Then algorithm adds a wrong edge at some point» If it adds a wrong edge, there mst be a lower weight edge (ct and paste argment)» Bt algorithm chooses lowest weight edge at each step. Contradiction Again, important to be comfortable with ct and paste argments What will affect the rnning time? T = T U,v; 7 7 Page

13 What will affect the rnning time? Let n= V and m= E Sort O(n) MakeSet() calls O(m) FindSet() calls O(n) Union() calls T = T U,v; : Rnning Time To smmarize:» Sort edges: O(m lg m)» O(n) MakeSet() s» O(m) FindSet() s» O(n) Union() s Upshot:» Best disjoint-set nion algorithm makes above operations take O(m α(m,n)), α almost constant» Overall ths O(m lg m), almost linear w/o sorting 7 74 Single-Sorce Shortest Path Problem: given a weighted directed graph G, find the minimm-weight path from a given sorce vertex s to another vertex v» Shortest-path = minimm weight» Weight of path is sm of edges» E.g., a road map: what is the shortest path from Minneapolis to Lincoln? Shortest Path Properties Again, we have optimal sbstrctre: the shortest path consists of shortest sbpaths:» Proof: sppose some sbpath is not a shortest path There mst then exist a shorter sbpath Cold sbstitte the shorter sbpath for a shorter path Bt then overall path is not shortest path. Contradiction 7 76 Shortest Path Properties Define δ(,v) to be the weight of the shortest path from to v Shortest paths satisfy the triangle ineqality: δ(,v) δ(,x) + δ(x,v) Proof : x Shortest Path Properties In graphs with negative weight cycles, some shortest paths will not exist (Why?): < 0 v This path is no longer than any other path 77 7 Page

14 Relaxation d[] A key techniqe in shortest path algorithms is relaxation» Idea: for all v, maintain pper bond d[v] on δ(s,v) Relax(,v,w) if (d[v] > d[]+w) then d[v]=d[]+w; Estimated d[v] v v 6 Dijkstra s Algorithm If no negative edge weights, we can beat BFS Similar to breadth-first search» Grow a tree gradally, advancing from vertices taken from a qee Also similar to Prim s algorithm for MST» Use a priority qee keyed on d[v] Relax v 7 New d[v] Relax v Dijkstra s Algorithm Dijkstra(G,s) B d[v] = ; A 4 D d[s] = 0; S = ; Q = V; C while (Q ) Ex: rn the algorithm S = S ; if (d[v] > d[]+w(,v)) Note: this d[v] = d[]+w(,v); is really a call to Q->DecreaseKey() Relaxation Step Dijkstra s Algorithm Dijkstra(G,s) d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) S = S ; if (d[v] > d[]+w(,v)) d[v] = d[]+w(,v); How many times is ExtractMin() called? How many times is DecraseKey() called? What will be the total rnning time? Analysis of Dijkstra s Algorithm Dijkstra s Algorithm Dijkstra(G,s) d[v] = ; There are n= V ExtractMin calls and m= E DecreaseKey calls. Ths, the worst case is O(n +m). d[s] = 0; S = ; Q = V; while (Q ) The priority Q implementation has a large impact on S = S ; performance. if (d[v] > d[]+w(,v)) d[v] = d[]+w(,v); E.g., O((n+m)lg n) = O(m lg n) sing binary heap for Q Can achieve O(n lg n + m) with Fibonacci heaps Dijkstra(G,s) d[v] = ; d[s] = 0; S = ; Q = V; while (Q ) S = S ; if (d[v] > d[]+w(,v)) d[v] = d[]+w(,v); Correctness: we mst show that when is removed from Q, it has already converged 4 Page

15 Correctness Of Dijkstra's Algorithm Correctness Of Dijkstra's Algorithm s p x y p s p x y p Note that d[v] δ(s,v) v Let be first vertex picked s.t. shorter path than d[] d[] > δ(s,) Let y be first vertex V-S on actal shortest path from s d[y] = δ(s,y)» Becase d[x] is set correctly for y's predecessor x S on the shortest path, and» When we pt x into S, we relaxed (x,y), giving d[y] the correct vale Note that d[v] δ(s,v) v Let be first vertex picked s.t. shorter path than d[] d[] > δ(s,) Let y be first vertex V-S on actal shortest path from s d[y] = δ(s,y) d[] > δ(s,) = δ(s,y) + δ(y,) (Why?) = d[y] + δ(y,) d[y] Bt if d[] > d[y], woldn't have chosen. Contradiction. 6 Page