Greedy Algorithms Spanning Trees

Transcription

1 Greedy Algoritms Spnning Trees Cpter 1, Wt mkes greedy lgoritm? Fesible Hs to stisfy te problem s constrints Loclly Optiml Te greedy prt Hs to mke te best locl coice mong ll fesible coices vilble on tt step If tis locl coice results in globl optimum ten te problem s optiml substructure Irrevocble Once coice is mde it cn t be un-done on subsequent steps of te lgoritm Simple exmples: Plying cess by mking best move witout looked Giving fewest number of coins s cnge Simple nd ppeling, but don t lwys give te best solution 1

2 Activity Selection Problem Problem: Scedule n exclusive resource in competition wit oter entities. For exmple, sceduling te use of room (only one entity cn use it t time) wen severl groups wnt to use it. Or, renting out some piece of equipment to different people. Definition: Set S={1,, n} of ctivities. Ec ctivity s strt time s i nd finis time f j, were s i <f j. Activities i nd j re comptible if tey do not overlp. Te ctivity selection problem is to select mximum-size set of mutully comptible ctivities. Greedy Activity Selection Just mrc troug ec ctivity by finis time nd scedule it if possible:

3 Activity Selection Exmple Scedule job 1, ten try rest: (end up wit 1,, ): T=1 T= T= T= T= T= T=7 T= T= T= T=11 T=1 T= Runtime? Greedy vs. Dynmic? Greedy lgoritms nd dynmic progrmming re similr; bot generlly work under te sme circumstnces ltoug dynmic progrmming solves subproblems first. Often bot my be used to solve problem ltoug tis is not lwys te cse. Consider te 0-1 knpsck problem. A tief is robbing store tt s items 1..n. Ec item is wort v[i] dollrs nd weigs w[i] pounds. Te tief wnts to tke te most mount of loot but is knpsck cn only old weigt W. Wt items sould e tke? Greedy lgoritm: Tke s muc of te most vluble item first. Does not necessrily give optiml vlue!

4 Frctionl Knpsck Problem Consider te frctionl knpsck problem. Tis time te tief cn tke ny frction of te objects. For exmple, te gold my be gold dust insted of gold brs. In tis cse, it will beoove te tief to tke s muc of te most vluble item per weigt (vlue/weigt) e cn crry, ten s muc of te next vluble item, until e cn crry no more weigt. Morl Greedy lgoritm sometimes gives te optiml solution, sometimes not, depending on te problem. Dynmic progrmming, wen pplicble, will typiclly give optiml solutions, but re usully trickier to come up wit nd sometimes trickier to implement. Definition Spnning Tree A spnning tree of grp G is tree (cyclic) tt connects ll te vertices of G once i.e. te tree spns every vertex in G A Minimum Spnning Tree (MST) is spnning tree on weigted grp tt s te minimum totl weigt w( T ) = w( u, v) suc tt w(t) is minimum u, v T Were migt tis be useful? Cn lso be used to pproximte some NP-Complete problems

5 Smple MST Wic links to mke tis MST? 1 Optiml substructure: A subtree of te MST must in turn be MST of te nodes tt it spns. MST Clim Clim: Sy tt M is MST If we remove ny edge (u,v) from M ten tis results in two trees, T1 nd T. T1 is MST of its subgrp wile T is MST of its subgrp. Ten te MST of te entire grp is T1 + T + te smllest edge between T1 nd T If some oter edge ws used, we wouldn t ve te minimum spnning tree overll

6 Greedy Algoritm We cn use greedy lgoritm to find te MST. Two common lgoritms Kruskl Prim Kruskl s MST Algoritm Ide: Greedily construct te MST Go troug te list of edges nd mke forest tt is MST At ec vertex, sort te edges Edges wit smllest weigts exmined nd possibly dded to MST before edges wit iger weigts Edges dded must be sfe edges tt do not ruin te tree property.

7 Kruskl s Algoritm Kruskl(G,w) ; Grp G, wit weigts w A {} ; Our MST strts empty for ec vertex v V [ G] do Mke-Set(v) ; Mke ec vertex set Sort edges of E by incresing weigt for ec edge ( u, v) E in order ; Find-Set returns representtive (first vertex) in te set do if Find-Set(u) Find-Set(v) ten A A {( u, v )} Union(u,v) ; Combines two trees return A Kruskl s Exmple 1 A={ }, Mke ec element its own set. {} {b} {c} {d} {e} {f} {g} {} Sort edges. Look t smllest edge first: {c} nd {f} not in sme set, dd it to A, union togeter. Now get {} {b} {c f} {d} {e} {g} {} 7

8 Kruskl Exmple Keep going, cecking next smllest edge. Hd: {} {b} {c f} {d} {e} {g} {} {e} {}, dd edge. 1 Now get {} {b} {c f} {d} {e } {g} Keep going, cecking next smllest edge. Hd: {} {b} {c f} {d} {e } {g} {} {c f}, dd edge. Kruskl Exmple 1 Now get {b} { c f} {d} {e } {g}

9 Kruskl s Exmple Keep going, cecking next smllest edge. Hd {b} { c f} {d} {e } {g} {b} { c f}, dd edge. 1 Now get { b c f} {d} {e } {g} Kruskl s Exmple Keep going, cecking next smllest edge. Hd { b c f} {d} {e } {g} { b c f} = { b c f}, dont dd it! 1

10 Kruskl s Exmple Keep going, cecking next smllest edge. Hd { b c f} {d} {e } {g} { b c f} = {e }, dd it. 1 Now get { b c f e } {d}{g} Kruskl s Exmple Keep going, cecking next smllest edge. Hd { b c f e } {d}{g} {d} { b c e f }, dd it. 1 Now get { e f } {g}

11 Kruskl s Exmple Keep going, ceck next two smllest edges. Hd { e f } {g} { e f } = { e f }, don t dd it. 1 Kruskl s Exmple Do dd te lst one: Hd { e f } {g} 1 11

12 Runtime of Kruskl s Algo Runtime depends upon time to union set, find set, mke set Simple set implementtion: number ec vertex nd use n rry Use n rry member[] : member[i] is number j suc tt te it vertex is member of te jt set. Exmple member[1,,1,,] indictes te sets S1={1,}, S={,} nd S={}; i.e. position in te rry gives te set number. Ide similr to counting sort, up to number of edge members. Given te Member rry Set Opertions 1 Mke-Set(v) member[v] = v member = [1,,] ; {1} {} {} Mke-Set runs in constnt running time for single set. Find-Set(v) Return member[v] Find-Set runs in constnt time. Union(u,v) for i=1 to n do if member[i] = u ten member[i]=v find-set() = Union(,) member = [1,,] ; {1} { } Scn troug te member rry nd updte old members to be te new set. Running time O(n), lengt of member rry. 1

13 Overll Runtime Kruskl(G,w) ; Grp G, wit weigts w O(V) A {} ; Our MST strts empty for ec vertex v V [ G] do Mke-Set(v) ; Mke ec vertex set Sort edges of E by incresing weigt O(ElgE) using epsort for ec edge ( u, v) E in order O(E) ; Find-Set returns representtive (first vertex) in te set do if Find-Set(u) Find-Set(v) O(1) ten A A {( u, v )} Union(u,v) ; Combines two trees return A O(V) Totl runtime: O(V)+O(ElgE)+O(E*(1+V)) = O(E*V) Book describes version using disjoint sets tt runs in O(E*lgE) time Prim s MST Algoritm Also greedy, like Kruskl s Will find MST but my differ from Prim s if multiple MST s re possible MST-Prim(G,w,r) ; Grp G, weigts w, root r Q V[G] for ec vertex u Q do key[u] ; inite distnce key[r] 0 P[r] NIL wile Q<>NIL do u Extrct-Min(Q) ; remove closest node ; Updte cildren of u so tey ve prent nd min key vl ; te key is te weigt between node nd prent for ec v Adj[u] do if v Q & w(u,v)<key[v] ten P[v] u key[v] w(u,v) 1

14 Prim s Exmple Exmple: Grp given erlier. Q={ (e,0) (, ) (b, ) (c, ) (d, ) (f, ) (g, ) (, ) } 1 0/nil Extrct min, vertex e. Updte neigbor if in Q nd weigt < key. Prim s Exmple 1/e 1 0/nil /e Q={ (, ) (b,1) (c, ) (d, ) (f, ) (g, ) (,) } Extrct min, vertex. Updte neigbor if in Q nd weigt < key 1

15 Prim s Algoritm / 1 0/nil / /e Q={ (, ) (b,) (c, ) (d, ) (f,) (g, ) } Extrct min, vertex f. Updte neigbor if in Q nd weigt < key Prim s Algoritm / /f 1 /f 0/nil / /e Q={ (, ) (b,) (c, ) (d, ) (g,) } Extrct min, vertex c. Updte neigbor if in Q nd weigt < key

16 Prim s Algoritm /c /c /f /c 1 /f 0/nil / /e Q={ (,) (b,) (d,) (g,) } Extrct min, vertex. No keys re smller tn edges from (> on edge c, > on edge b) so noting done. Q={ (b,) (d,) (g,) } Extrct min, vertex b. Sme cse, no keys re smller tn edges, so noting is done. Sme for extrcting d nd g, nd we re done. Prim s Algoritm Get spnning tree by connecting nodes wit teir prents: /c /c /f /c 1 /f 0/nil / /e 1

17 O(V) Runtime for Prim s Algoritm MST-Prim(G,w,r) ; Grp G, weigts w, root r Q V[G] for ec vertex u Q do key[u] ; inite distnce key[r] 0 P[r] NIL wile Q<>NIL do u Extrct-Min(Q) ; remove closest node ; Updte cildren of u so tey ve prent nd min key vl ; te key is te weigt between node nd prent for ec v Adj[u] do if v Q & w(u,v)<key[v] ten P[v] u key[v] w(u,v) O(V) if using ep O(lgV) if using ep O(E) over entire wile(q<>nil) loop O(lgV) to updte if using ep! Te inner loop tkes O(E lg V) for te ep updte inside te O(E) loop. Tis is over ll executions, so it is not multiplied by O(V) for te wile loop (tis is included in te O(E) runtime troug ll edges. Te Extrct-Min requires O(V lg V) time. O(lg V) for te Extrct-Min nd O(V) for te wile loop. Totl runtime is ten O(V lg V) + O(E lg V) wic is O(E lg V) in connected grp ( connected grp will lwys ve t lest V-1 edges). Prim s Algoritm Liner Arry for Q Wt if we use simple liner rry for te queue insted of ep? Use te index s te vertex number Contents of rry s te distnce vlue E.g. Vl[ ] Pr[ 7 ] Sys tt vertex 1 s key =, vertex s key =, etc. Use specil vlue for inity or if vertex removed from te queue Sys tt vertex 1 s prent node, vertex s prent node, etc. Building Queue: O(n) time to crete rrys Extrct min: O(n) time to scn troug te rry Updte key: O(1) time 17

18 O(V) Runtime for Prim s Algoritm wit Queue s Arry MST-Prim(G,w,r) ; Grp G, weigts w, root r Q V[G] for ec vertex u Q do key[u] ; inite distnce key[r] 0 P[r] NIL wile Q<>NIL do u Extrct-Min(Q) ; remove closest node ; Updte cildren of u so tey ve prent nd min key vl ; te key is te weigt between node nd prent for ec v Adj[u] do if v Q & w(u,v)<key[v] ten P[v] u key[v] w(u,v) Te inner loop tkes O(E ) over ll itertions of te outer loop. It is not multiplied by O(V) for te wile loop. Te Extrct-Min requires O(V ) time. Tis is O(V ) over te wile loop. Totl runtime is ten O(V ) + O(E) wic is O(V ) Using ep our runtime ws O(E lg V). Wic is worse? Wic is worse for fully connected grp? O(V) to initilize rry O(V) to serc rry O(E) over entire wile(q<>nil) loop O(1) direct ccess vi rry index Approximtions for Hrd Problems Greedy lgoritms re commonly used to find pproximtions for NP-Complete problems Use euristic to drive te greedy selection Heuristic: A common-sense rule tt pproximtes moves towrd te optiml solution If our problem is to minimize function f were f(s*) is te vlue of te exct solution; globl minimum f(s ) is te vlue of our pproximte solution Ten we wnt to minimize te rtio: f(s ) / f(s*) suc tt tis pproces 1 Opposite if mximizing function 1

19 Exmple: Trveling Slesmn Problem Cep greedy solution to te TSP: Coose n rbitrry city s te strt Visit te nerest unvisited city; repet until ll cities ve been visited Return to te strting city Exmple grp: 1 b Strting t : ->b->c->d-> Totl = Optiml: ->b->d->c-> d 1 c r(s ) = / = 1. Is tis good pproc? Wt if ->d =? Greedy TSP Our greedy pproc is not so bd if te grp deres to Eucliden geometry Tringle inequlity d[i,j] d[i,k] + d[k,j] for ny triple cities i,j,k Symmetry d[i,j] = d[j,i] for ny pir of cities i,j In our previous exmple, we couldn t ve one-wy edge to city of were ll te oter edges re smller (if city is fr wy, forced to visit it some wy) It s been proven for Eucliden instnces te nerest neigbor lgoritm: f(s ) / f(s*) (lg n + 1) / n cities 1

20 Minimum Spnning Tree Approximtion We cn use MST to get better pproximtion to te TSP problem Tis is clled twice-round-te-tree lgoritm We construct MST nd fix it up so tt it mkes vlid tour Construct MST of te grp corresponding to te TSP problem Strting t n rbitrry vertex, perform DFS wlk round te MST recording te vertices pssed by Scn te list of vertices from te previous step nd eliminte ll repet occurrences except te strting one. Te vertices remining will form Hmiltonin circuit tt is te output of te lgoritm. MST Approximtion to TSP Exmple grp: 1 e 7 e 7 b d b d 1 c 1 c MST: b, bc, bd, de Wlk:, b, c, b, d, e, d, b,, b, c, d, e, 0

21 MST Approximtion Runtime: polynomil (Kruskl/Prim) Clim: f(s ) < f(s*) Lengt of te pproximtion solutuion t most twice te lengt of te optiml Since removing ny edge from s* yields spnning tree T of weigt w(t) tt must be w(t*), te weigt of te grp s MST, we ve: f(s*) > w(t) w(t*) f(s*) > w(t*) Te wlk of te MST tree we used to generte te pproximte solution trversed te MST t most twice, so: w(t*) > f(s ) Giving: f(s*) > w(t*) > f(s ) f(s*) > f(s ) 1