Greedy Algorithms. Interval Scheduling. Greedy Algorithms. Interval scheduling. Greedy Algorithms. Interval Scheduling

Transcription

1 Greedy Algorithms Greedy Algorithms Witer Paul Beame Hard to defie exactly but ca give geeral properties Solutio is built i small steps Decisios o how to build the solutio are made to maximize some criterio without lookig to the future Wat the best curret partial solutio as if the curret step were the last step May be more tha oe greedy algorithm usig differet criteria to solve a give problem Greedy Algorithms Iterval Schedulig Greedy algorithms Easy to produce Fast ruig times Work oly o certai classes of problems Two methods for provig that greedy algorithms do work Greedy algorithm stays ahead At each step ay other algorithm will have a worse value for the criterio Exchage Argumet Ca trasform ay other solutio to the greedy solutio at o loss i quality Iterval Schedulig Sigle resource Reservatio requests Of form Ca I reserve it from start time s to fiish time f? s < f Fid: maximum umber of requests that ca be scheduled so that o two reservatios have the resource at the same time Iterval schedulig Greedy Algorithms for Iterval Schedulig Formally Requests,,, request i has start time s i ad fiish time f i > s i Requests i ad j are compatible iff either request i is for a time etirely before request j What criterio should we try? Earliest start time s i f i s j or, request j is for a time etirely before request i Shortest request time f i -s i f j s i Set A of requests is compatible iff every pair of requests i,j A, i j is compatible Goal: Fid maximum size subset A of compatible requests Earliest fiish fime f i

2 Greedy Algorithms for Iterval Schedulig What criterio should we try? Earliest start time s i Does t work Shortest request time f i -s i Does t work Eve fewest coflicts does t work Earliest fiish fime f i Works Greedy Algorithm for Iterval Schedulig R set of all requests A While R do Choose request i R with smallest fiishig time f i Add request i to A Delete all requests i R that are ot compatible with request i Retur A Greedy Algorithm for Iterval Schedulig Claim: A is a compatible set of requests ad these are added to A i order of fiish time Whe we add a request to A we delete all icompatible oes from R Claim: For ay other set O R of compatible requests the if we order requests i A ad O by fiish time the for each k: If O cotais a k th request the so does A ad the fiish time of the k th request i A, is the fiish time of the k th request i O, i.e. a k o k where a k ad o k are the respective fiish times Iductive Proof of Claim: a k o k Base Case: This is true for the first request i A sice that is the oe with the smallest fiish time Iductive Step: Suppose a k o k By defiitio of compatibility If O cotais a k+ st request r the the start time of that request must be after o k ad thus after a k Thus r is compatible with the first k requests i A Therefore A has at least k+ requests sice a compatible oe is available after the first k are chose r was amog those cosidered by the greedy algorithm for that k+ st request i A Therefore by the greedy choice the fiish time of r which is o k+ is at least the fiish time of that k+ st request i A which is a k+ Implemetig the Greedy Algorithm Exchage argumets Sort the requests by fiish time O(log ) time Maitai curret latest fiish time scheduled Keep array of start times idexed by request umber Oly elimiate icompatible requests as eeded Walk alog array of requests sorted by fiish times skippig those whose start time is before curret latest fiish time scheduled O() additioal time for greedy algorithm Schedulig to miimize lateess Sigle resource as i iterval schedulig but istead of start ad fiish times request i has Time requiremet t i which must be scheduled i a cotiguous block Target deadlie d i by which time the request would like to be fiished Overall start time s Requests are scheduled by the algorithm ito time itervals [s i,f i ] such that t i =f i -s i Lateess of schedule for request i is If d i <f i the request i is late by L i = f i -d i otherwise its lateess L i = Maximum lateess L=max i L i Goal: Fid a schedule for all requests (values of s i ad f i for each request i) to miimize the maximum lateess, L

3 Greedy Algorithm: Earliest Deadlie First Order requests i icreasig order of deadlies Schedule the request with the earliest deadlie as soo as the resource becomes available Greedy Algorithm: Earliest Deadlie First Sort deadlies i icreasig order (assume wlog that d d d ) f s for i to to s i f f i s i +t i f f i ed for Why does this give optimal value of maximum lateess? Easy observatios This schedule has o idle time The overall schedule starts at time s ad fiishes as soo as possible at time s+t +t + + t There is a optimal schedule with o idle time Shiftig the requests earlier by the amout of the idle time ca oly decrease maximum lateess (It might ot improve the maximum lateess but it certaily ca t hurt to do this.) There are oly a fiite # of schedules with o idle time so oe of them must be best Exchage Argumet We will show that if there is aother schedule O (thik optimal schedule) the we ca gradually chage O so that at each step the maximum lateess i O ever gets worse it evetually becomes the same as A Iversios Optimal schedules ad iversios Iversio d j < d i but i is scheduled before j Earliest deadlie first has o iversios Claim: All schedules with o iversios ad o idle time have the same maximum lateess Proof Schedules ca differ oly i how they order requests with equal deadlies Cosider all requests havig some commo deadlie d Maximum lateess of these jobs is based oly o the fiish time of the last of these jobs but the set of these requests occupy the same time segmet i both schedules Last of these requests fiishes at the same time i ay such schedule. Claim: There is a optimal schedule with o idle time ad o iversios Proof: By previous argumet there is a optimal schedule O with o idle time (i) If O has a iversio the it has a cosecutive pair of requests i its schedule that are iverted i.e. d j < d i but i is scheduled immediately before j (Simple trasitivity of <, otherwise cosecutive pairs would always be scheduled i icreasig order of deadlies)

4 Optimal schedules ad iversios Optimal schedules ad iversios (ii) If d j < d i but i is scheduled i O immediately before j the swappig requests i ad j to get schedule O does ot icrease the maximum lateess Lateess L j L j sice j is scheduled earlier i O tha i O Requests i ad j together occupy the same total time slot i both schedules All other requests k i have L k =L k (iii) Evetually these swaps will produce a optimal schedule with o iversios Each swap decreases the umber of iversios by There are at most (-)/ iversios QED f i =f j so L i = f j -d i < f j -d j =L j Maximum lateess has ot icreased! Earliest Deadlie First is optimal Optimal Cachig/Pagig We kow that There is a optimal schedule with o idle time or iversios All schedules with o idle time or iversios have the same maximum lateess EDF produces a schedule with o idle time or iversios Therefore EDF produces a optimal schedule Memory systems may levels of storage with differet access times smaller storage has shorter access time to access a item it must be brought to the lowest level of the memory system Cosider the maagemet problem betwee adjacet levels Mai memory with data items from a set U Cache ca hold k< items Simplest versio with o direct-mappig or other restrictios about where items ca be Suppose cache is full iitially Holds k data items to start with Optimal Cachig/Pagig Give a memory request d from U If d is stored i the cache we ca access it quickly If ot the we call it a cache miss ad (sice the cache is full) we must brig it ito cache ad evict some other data item from the cache which oe to evict? Give a sequece D=d,d,,d m of elemets from U correspodig to memory requests Fid a sequece of evictios (a evictio schedule) that has as few cache misses as possible Cachig Example =, k=, U={a,b,c} Cache iitially cotais {a,b} D= a b c b c a b S= a c C= a b a b c b This is optimal

5 A Note o Optimal Cachig I real operatig coditios oe typically eeds a o-lie algorithm make the evictio decisios as each memory request arrives However to desig ad aalyze these algorithms it is also importat to uderstad how the best possible decisios ca be made if oe did kow the future Field of o-lie algorithms compares the quality of o-lie decisios to that of the optimal schedule What does a optimal schedule look like? Belady s Greedy Algorithm: Farthest-I-Future Give sequece D=d,d,,d m Whe d i eeds to be brought ito the cache evict the item that is eeded farthest i the future Let NextAccess i (d)=mi{ j i : d j =d} be the ext poit i D that item d will be requested Evict d such that NextAccess i (d) is largest Other Algorithms Reduced Schedule Ofte there is flexibility, e.g. k=, C={a,b,c} D= a b c d a d e a d b c S FIF = c b e d S = b c d e Why are t other algorithms better? Least-Frequety-Used-I-Future? Exchage Argumet We ca swap choices to covert other schedules to Farthest-I-Future without losig quality We seemed to assume that we oly brought i ew items (ad evicted curret items) at cache misses Would actig i advace would help? Call a evictio schedule reduced if its oly evictios are of sigle items at cache misses Farthest-I-Future produces a reduced schedule Give a evictio schedule S defie schedule reduced(s) as follows: I step i if S brought i item d that is ot accessed util step j > i ad evicted item e from cache the preted to do this evictio leavig d i mai memory util step j Claim: reduced(s) yields at most as may cache misses as S does Optimal Cachig Fix some sequece D=d,d,,d m of requests Let S FIF be the schedule produced o this sequece by Farthest-I-Future Claim: If S is ay reduced schedule (for all of D) that agrees with S FIF for (at least) the first j steps the there is a reduced schedule S that agrees with S FIF for (at least) the first j+ steps such that #misses(s ) #misses(s) Applyig this claim m times we could start with ay optimal reduced schedule S (which agrees with S FIF for at least steps) ad produce S FIF without icreasig the umber of cache misses Provig the Claim Suppose S is a reduced schedule that agrees with S FIF for the first j steps At the time of the j+ st request d j+, S ad S FIF yield the same cache cotets Case : d j+ is i the cache Sice both S ad S FIF are reduced either has a miss so they agree for j+ steps ad we ca set S =S Case : d j+ is ot i the cache Say that S evicts f ad S FIF evicts e If e=f the we ca take S =S sice S ad S FIF agree for j+ steps What if e f?

6 Buildig S NextAccess j+ (e)>nextaccess j+ (f) Defie schedule S that agrees with S except that At step j+ it evicts e istead of f The first time after step j+ that oe of the followig occurs it does somethig differet that will result i S havig the same cache as S so that the rest of the simulatio ca take place Evet A: There is a request to a item g other tha e or f s.t. S evicts e Evet B: There is a request to f Note: S will have trouble hadlig ay access to e while it remais i the cache uder S but we wo t have to deal with it because Evet B will happe before ay access to e Buildig S Evet A: There is a request to a item g other tha e or f s.t. S evicts e I this case the cache uder S also does t cotai g (sice the caches uder S ad S ca at most disagree o e ad f) So we have S evict f Caches ow agree ad S is exactly the same as S for the rest #misses(s )=#misses(s). Buildig S Sigle-source shortest paths Evet B: There is a request to f Now f is i the cache uder S but ot S Say S evicts item e If e =e the after this step the caches uder S ad S are the same so we let S match S for the rest #misses(s )<#misses(s) If e e the have S evict e also ad brig i e istead Now cache uder S ad S agree ad S ca match S #misses(s )=#misses(s) But S is ot reduced so we replace S by reduce(s ) Reduce(S ) still agrees with S FIF for j+ steps Give a (u)directed graph G=(V,E) with each edge e havig a o-egative weight w(e) ad a vertex v Fid legth of shortest paths from v to each vertex i G A greedy algorithm Dijsktra s Algorithm : Maitai a set S of vertices whose shortest paths are kow iitially S={s} Maitaiig curret best legths of paths that oly go through S to each of the vertices i G path-legths to elemets of S will be right, to V-S they might ot be right Repeatedly add vertex v to S that has the shortest path-legth of ay vertex i V-S update path legths based o ew paths through v Dijkstra(G,w,s) S {s} d[s] while S V do of all edges e=(u,v) s.t. v S ad u S select* oe with the miimum value of d[u]+w(e) S S {v} d[v] d[u]+w(e) pred[v] u *For each v S maitai d [v]=miimum value of d[u]+w(e) over all vertices u S s.t. e=(u,v) is i of G

7 Update distaces Update distaces

8 Update distaces Update distaces Update distaces

9 Update distaces Update distaces Update distaces

10 Update distaces Update distaces Update distaces

11 Correctess Suppose all distaces to vertices i S are correct ad u has smallest curret value i V-S distace value of vertex i V-S=legth of shortest path from s with oly last edge leavig S s S x v Suppose some other path to v ad x= first vertex o this path ot i S d (v) d (x) x-v path legth other path is loger Therefore addig v to S keeps correct distaces Algorithm also produces a tree of shortest paths to v followig pred liks From w follow its acestors i the tree back to v If all you care about is the shortest path from v to w simply stop the algorithm whe w is added to S Implemetig Data Structure Review Need to keep curret distace values for odes i V-S fid miimum curret distace value reduce distaces whe vertex moved to S Priority Queue: Elemets each with a associated key Operatios Isert Fid-mi Retur the elemet with the smallest key Delete-mi Retur the elemet with the smallest key ad delete it from the data structure Decrease-key Decrease the key value of some elemet Implemetatios Arrays: O() time fid/delete-mi, O() time isert/ decrease-key Heaps: O(log ) time isert/decrease-key/delete-mi, O() time fid-mi with Priority Queues For each vertex u ot i tree maitai cost of curret cheapest path through tree to u Store u i priority queue with key = legth of this path Operatios: - isertios (each vertex added oce) - delete-mis (each vertex deleted oce) pick the vertex of smallest key, remove it from the priority queue ad add its edge to the graph <m decrease-keys (each edge updates oe vertex) Dijskstra s Algorithm with Priority Queues Priority queue implemetatios Array isert O(), delete-mi O(), decrease-key O() total O(+ +m)=o( ) Heap isert, delete-mi, decrease-key all O(log ) total O(m log ) d-heap (d=m/) isert, decrease-key O(log m/ ) delete-mi O((m/) log m/ ) total O(m log m/ )

12 Miimum Spaig Trees (Forests) Weighted Udirected Graph Give a udirected graph G=(V,E) with each edge e havig a weight w(e) Fid a subgraph T of G of miimum total weight s.t. every pair of vertices coected i G are also coected i T if G is coected the T is a tree otherwise it is a forest - First Greedy Algorithm Prim s Algorithm: start at a vertex s add the cheapest edge adjacet to s repeatedly add the cheapest edge that jois the vertices explored so far to the rest of the graph Exactly like Dijsktra s Algorithm but with a differet metric Dijsktra s Algorithm Dijkstra(G,w,s) S {s} d[s] while S V do of all edges e=(u,v) s.t. v S ad u S select* oe with the miimum value of d[u]+w(e) S S {v} d[v] d[u]+w(e) pred[v] u *For each v S maitai d [v]=miimum value of d[u]+w(e) over all vertices u S s.t. e=(u,v) is i of G Prim s Algorithm Prim(G,w,s) S {s} while S V do of all edges e=(u,v) s.t. v S ad u S select* oe with the miimum value of w(e) S S {v} pred[v] u *For each v S maitai small[v]=miimum value of w(e) over all vertices u S s.t. e=(u,v) is i of G Secod Greedy Algorithm Kruskal s Algorithm Start with the vertices ad o edges Repeatedly add the cheapest edge that jois two differet compoets. i.e. that does t create a cycle

13 Why greed is good Cuts ad Spaig Trees Defiitio: Give a graph G=(V,E), a cut of G is a partitio of V ito two o-empty pieces, S ad V-S Lemma: For every cut (S,V-S) of G, there is a miimum spaig tree (or forest) cotaiig ay cheapest edge crossig the cut, i.e. coectig some ode i S with some ode i V-S. call such a edge safe - The greedy algorithms always choose safe edges Prim s Algorithm Prim s Algorithm Always chooses cheapest edge from curret tree to rest of the graph This is cheapest edge across a cut which has the vertices of that tree o oe side. - The greedy algorithms always choose safe edges Kruskal s Algorithm Kruskal s Algorithm Always chooses cheapest edge coectig two pieces of the graph that are t yet coected This is the cheapest edge across ay cut which has those two pieces o differet sides ad does t split ay curret pieces. -

14 Kruskal s Algorithm Proof of Lemma: A Exchage Argumet - Suppose you have a MST ot usig cheapest edge e u e v Edpoits of e, u ad v must be coected i T Proof of Lemma Proof of Lemma Suppose you have a MST ot usig cheapest edge e Suppose you have a MST ot usig cheapest edge e u e v u e h v Edpoits of e, u ad v must be coected i T Edpoits of e, u ad v must be coected i T Proof of Lemma Proof of Lemma Suppose you have a MST ot usig cheapest edge e Replacig h by e does ot icrease weight of T w(e) w(h) w(e) w(h) u e h v u e h v Edpoits of e, u ad v must be coected i T All the same poits are coected by the ew tree

15 Kruskal s Algorithm Implemetatio & Aalysis First sort the edges by weight O(m log m) Go through edges from smallest to largest if edpoits of edge e are curretly i differet compoets the add to the graph else skip Uio-fid data structure hadles last part Total cost of last part: O(m α()) where α()<< log m Overall O(m log ) Uio-fid disjoit sets data structure Maitaiig compoets start with differet compoets oe per vertex fid compoets of the two edpoits of e m fids uio two compoets whe edge coectig them is added - uios Prim s Algorithm with Priority Queues For each vertex u ot i tree maitai curret cheapest edge from tree to u Store u i priority queue with key = weight of this edge Operatios: - isertios (each vertex added oce) - delete-mis (each vertex deleted oce) pick the vertex of smallest key, remove it from the p.q. ad add its edge to the graph <m decrease-keys (each edge updates oe vertex) Prim s Algorithm with Priority Queues Priority queue implemetatios Array isert O(), delete-mi O(), decrease-key O() total O(+ +m)=o( ) Heap isert, delete-mi, decrease-key all O(log ) total O(m log ) d-heap (d=m/) isert, decrease-key O(log m/ ) delete-mi O((m/) log m/ ) total O(m log m/ ) Boruvka s Algorithm () May other miimum spaig tree algorithms, most of them greedy A bit like Kruskal s Algorithm Start with compoets cosistig of a sigle vertex each At each step, each compoet chooses its cheapest outgoig edge to add to the spaig forest Two compoets may choose to add the same edge Useful for parallel algorithms sice compoets may be processed (almost) idepedetly Cherito & Tarja O(m loglog ) time usig a queue of compoets Chazelle O(m α(m) log α(m)) time Icredibly hairy algorithm Karger, Klei & Tarja O(m+) time radomized algorithm that works most of the time

16 Applicatios of Miimum Spaig Tree Algorithms Applicatios of Miimum Spaig Tree Algorithms Miimum cost etwork desig: Build a etwork to coect all locatios {v,,v } Cost of coectig v i to v j is w(v i,v j )> Choose a collectio of liks to create that will be as cheap as possible Ay miimum cost solutio is a MST If there is a solutio cotaiig a cycle the we ca remove ay edge ad get a cheaper solutio Maximum Spacig Clusterig Give a collectio U of objects {p,,p } Distace measure d(p i,p j ) satisfyig d(p i,p i )= d(p i,p j )> for i j d(p i,p j )=d(p j,p i ) Positive iteger k Fid a k-clusterig, i.e. partitio of U ito k clusters C,,C k, such that the spacig betwee the clusters is as large possible where spacig = mi{d(p i,p j ): p i ad p j i differet clusters} Greedy Algorithm Proof that this works Start with clusters each cosistig of a sigle poit Repeatedly fid the closest pair of poits i differet clusters uder distace d ad merge their clusters util oly k clusters remai Removig the k- most expesive edges from a MST yields k compoets C,,C k ad the spacig for them is precisely the cost d* of the k- st most expesive edge i the tree Gets the same compoets as Kruskal s Algorithm does! The sequece of closest pairs is exactly the MST Alteratively we could ru Kruskal s algorithm oce ad for ay k we could get the maximum spacig k-clusterig by deletig the k- most expesive edges Cosider ay other k-clusterig C,,C k Sice they are differet ad cover the same set of poits there is some pair of poits p i,p j such that p i,p j are i some cluster C r but p i, p j are i differet clusters C s ad C t Sice p i,p j C r, p i ad p j have a path betwee them all of whose edges have distace at most d* This path must cross betwee clusters i the C clusterig so the spacig i C is at most d*