Introduction. Example

Transcription

1 OMS0 Introution isjoint sets n minimum spnning trees In this leture we will strt by isussing t struture use for mintining isjoint subsets of some bigger set. This hs number of pplitions, inluing to mintining onnete omponents of grph, n to fining minimum spnning trees in unirete grphs. eprtment of omputer Siene, University of ristol ristol, UK November 0 We will then isuss two lgorithms for fining minimum spnning trees: n lgorithm by Kruskl bse on isjoint-set strutures, n n lgorithm by Prim whih is similr to ijkstr s lgorithm. In both ses, we will see tht effiient implementtions of t strutures give us effiient lgorithms. OMS0: isjoint sets n MSTs Slie /48 OMS0: isjoint sets n MSTs Slie /48 isjoint-set t struture isjoint-set t struture mintins olletion S = {S,..., S k } of isjoint subsets of some lrger universe U. The t struture supports the following opertions:. MkeSet(x): rete new set whose only member is x. s the sets re isjoint, we require tht x is not ontine in ny of the other sets.. Union(x, y): ombine the sets ontining x n y (ll these S x, S y ) to reple them with new set S x S y.. inset(x): returns the ientity of the unique set ontining x. Opertion Returns S (strt) (empty) MkeSet() {} MkeSet(b) {}, {b} inset(b) b {}, {b} Union(, b) {, b} inset(b) {, b} inset() {, b} MkeSet() {, b}, {} The ientity of set is just some unique ientifier for tht set for exmple, the ientity of one of the elements in the set. OMS0: isjoint sets n MSTs Slie /48 OMS0: isjoint sets n MSTs Slie 4/48

2 Implementtion simple wy to implement isjoint-set t struture is s n rry of linke lists. We hve linke list for eh isjoint set. h element elem in the list stores pointer elem.next to the next element in the list, n the set element itself, elem.t. We lso hve n rry orresponing to the universe, with eh entry in the rry ontining pointer to the linke list orresponing to the set in whih it ours. Then to implement: MkeSet(x), we rete new list n set x s pointer to tht list. inset(x), we return the first element in the list to whih x points. Union(x, y), we ppen y s list to x s list n upte the pointers of everything in y s list to point to to x s list. Implementtion In more etil: MkeSet(x). [x] new linke list. elem new list element. elem.t x 4. [x].he elem 5. [x].til elem inset(x). return [x].he.t OMS0: isjoint sets n MSTs Slie 5/48 OMS0: isjoint sets n MSTs Slie 6/48 Implementtion Imgine we hve universe U = {, b,, }. The initil onfigurtion of the rry (orresponing to S = ) is Union(x, y). [x].til.next [y].he. [x].til [y].til. elem [y].he 4. while elem nil 5. [elem.t] [x] 6. elem elem.next Then the following sequene of uptes ours: MkeSet() b b he til OMS0: isjoint sets n MSTs Slie 7/48 OMS0: isjoint sets n MSTs Slie 8/48

3 MkeSet() b he til he til Union(, ) b he til OMS0: isjoint sets n MSTs Slie 9/48 OMS0: isjoint sets n MSTs Slie 0/48 MkeSet() b he til he til Union(, ) b he til OMS0: isjoint sets n MSTs Slie /48 OMS0: isjoint sets n MSTs Slie /48

4 Improvement: the weighte-union heuristi MkeSet n inset tke time O() but Union might tke time Θ(n) for universe of size n. Union(x, y) nees to upte til pointers in lists (onstnt time) but lso the informtion of every element in y s list. So the Union opertion is slow when y s list is long n x s is short. Heuristi: lwys ppen the shorter list to the longer list. Might still tke time Θ(n) in the worst se (if both lists hve the sme size), but we hve the following mortise nlysis: lim Using the linke-list representtion n the bove heuristi, sequene of m MkeSet, inset n Union opertions, n of whih re MkeSet opertions, uses time O(m + n log n). Improvement: the weighte-union heuristi lim Using the linke-list representtion n the bove heuristi, sequene of m MkeSet, inset n Union opertions, n of whih re MkeSet opertions, uses time O(m + n log n). Proof MkeSet n inset tke time O() eh, n there n be t most n non-trivil Union opertions. t eh Union opertion, n element s informtion is only upte when it ws in the smller set of the two sets. So, the first time it is upte, the resulting set must hve size t lest. The seon time, size t lest 4. The k th time, size t lest k. So eh element s informtion is only upte t most O(log n) times. So O(n log n) uptes re me in totl. ll other opertions use time O(), so the totl runtime is O(m + n log n). OMS0: isjoint sets n MSTs Slie /48 OMS0: isjoint sets n MSTs Slie 4/48 Improvements pplition: omputing onnete omponents nother wy to implement isjoint-set struture is vi isjoint-set forest (LRS.). This struture is bse on repling the linke lists with trees. simple pplition of the isjoint-set t struture is omputing onnete omponents of n unirete grph. One n show tht using isjoint-set forest, long with some optimistions, sequene of m opertions with n MkeSet opertions runs in time O(m α(n)), where α(n) is n extremely slowly growing funtion whih stisfies α(n) 4 for ny n or exmple: isjoint-set forests were introue in 964 by Gller n isher but this boun ws not proven until 975 by Trjn. G mzingly, it is known tht this runtime boun nnot be reple with boun O(m). OMS0: isjoint sets n MSTs Slie 5/48 OMS0: isjoint sets n MSTs Slie 6/48

5 pplition: omputing onnete omponents onneteomponents(g). for eh vertex v G: MkeSet(v). for eh ege u v in rbitrry orer. if inset(u) inset(v) 4. Union(u, v) Time omplexity: O( + V log V ) if implemente using linke lists, O( α(v )) if implemente using n optimise isjoint-set forest. fter onneteomponents ompletes, inset n be use to etermine whether two verties re in the sme omponent, in time O(). This tsk oul lso be hieve using breth-first serh, but using isjoint sets llows serhing n ing verties to be rrie out more effiiently in future. Minimum spnning trees Given onnete, unirete weighte grph G, subgrph T is spnning tree if: T is tree (i.e. oes not ontin ny yles) very vertex in G ppers in T. T is minimum spnning tree (MST) if the sum of the weights of eges of T is miniml mong ll spnning trees of G. spnning tree n minimum spnning tree of the sme grph. OMS0: isjoint sets n MSTs Slie 7/48 OMS0: isjoint sets n MSTs Slie 8/48 MSTs: pplitions generi pproh to MSTs Teleommunitions n utilities luster nlysis Txonomy Hnwriting reognition Mze genertion... The two lgorithms we will isuss for fining MSTs re both bse on the following bsi ie:. Mintin forest (i.e. olletion of trees) whih is subset of some minimum spnning tree.. t eh step, new ege to, mintining the bove property.. Repet until is minimum spnning tree. This pproh of mking lolly optiml hoie of n ege t eh step mkes them greey lgorithms. We will isuss: Kruskl s lgorithm, whih is bse on isjoint-set t struture. Prim s lgorithm, whih is bse on priority queue. Pis: ntionlgri.om, onnetiutvlleybiologil.om, Wikipei The lgorithms mke ifferent hoies for whih new ege to t eh step. OMS0: isjoint sets n MSTs Slie 9/48 OMS0: isjoint sets n MSTs Slie 0/48

6 How to hoose new eges? ut property Let X be subset of some MST T. Let S be subset of the verties of G suh tht X oes not ontin ny eges with extly one enpoint in S. Let e be lightest ege in G tht hs extly one enpoint in S. Then X {e} is subset of n MST. Proof or exmple: If e T, the lim is obviously true, so ssume e / T. s T is spnning tree, there must exist pth p in T between the enpoints of e, where p ontins n ege e with one enpoint in S.... How to hoose new eges? ut property Let X be subset of some MST T. Let S be subset of the verties of G suh tht X oes not ontin ny eges with extly one enpoint in S. Let e be lightest ege in G tht hs extly one enpoint in S. Then X {e} is subset of n MST. Proof xerise: or ny ege e on the pth p, if we reple e with e in T, the resulting set T is still spnning tree. urther, the totl weight of T is weight(t ) = weight(t ) + w(e) w(e ). s e is the lightest ege with one enpoint in S, w(e) w(e ). Hene weight(t ) weight(t ), so T is lso n MST. OMS0: isjoint sets n MSTs Slie /48 OMS0: isjoint sets n MSTs Slie /48 Kruskl s lgorithm The lgorithm hs similr flow to the lgorithm for omputing onnete omponents. It mintins forest, initilly onsisting of unonnete iniviul verties, n isjoint-set t struture. We use Kruskl s lgorithm to fin n MST in the following grph. Kruskl(G). for eh vertex v G: MkeSet(v). sort the eges of G into non-eresing orer by weight. for eh ege u v in orer 4. if inset(u) inset(v) 5. {u v} 6. Union(u, v) Informlly: the lightest ege between two omponents of. OMS0: isjoint sets n MSTs Slie /48 OMS0: isjoint sets n MSTs Slie 4/48

7 irst n rbitrry ege with weight is pike: Then ny other ege with weight : OMS0: isjoint sets n MSTs Slie 5/48 OMS0: isjoint sets n MSTs Slie 6/48 Then ny other ege with weight : The finl ege with weight nnot be pike beuse n re in the sme omponent, so one of the eges with weight is hosen: OMS0: isjoint sets n MSTs Slie 7/48 OMS0: isjoint sets n MSTs Slie 8/48

8 inlly, one of the other eges with weight is hosen n the MST is omplete. Proof of orretness Kruskl(G). for eh vertex v G: MkeSet(v). sort the eges of G into non-eresing orer by weight. for eh ege u v in orer 4. if inset(u) inset(v) 5. {u v} 6. Union(u, v) Proof of orretness Whenever inset(u) inset(v), the ege u v onnets two trees T, T. Set S = T in the ut property. This ege is lightest ege with one enpoint in S. So, by the ut property, {u v} is subset of n MST. OMS0: isjoint sets n MSTs Slie 9/48 OMS0: isjoint sets n MSTs Slie 0/48 omplexity nlysis of Kruskl s lgorithm Kruskl(G). for eh vertex v G: MkeSet(v). sort the eges of G into non-eresing orer by weight. for eh ege u v in orer 4. if inset(u) inset(v) 5. {u v} 6. Union(u, v) Prim s lgorithm Kruskl s lgorithm mintins forest n uses the rule: the lightest ege between two omponents of t eh step. ifferent pproh is use by Prim s lgorithm: mintin onnete tree T n exten T with the lightest possible ege. Prim s lgorithm is bse on the use of priority queue Q. V MkeSet opertions Time O( log ) to sort eges O() inset n Union opertions The flow of the lgorithm is lmost extly the sme s ijkstr s lgorithm; the only ifferene is the hoie of key for the queue. So, using isjoint-set struture implemente using n rry of linke lists, we get n overll runtime of O( log ). If the eges re lrey sorte, n we use n optimise isjoint-set forest, we n hieve O( α(v )). OMS0: isjoint sets n MSTs Slie /48 or eh vertex v, v.key is the weight of the lightest ege onneting v to T. OMS0: isjoint sets n MSTs Slie /48

9 Prim s lgorithm Prim(G). for eh vertex v G: v.key, v.π nil. hoose n rbitrry vertex r. r.key 0 4. every vertex in G to Q 5. while Q not empty 6. u xtrtmin(q) 7. for eh vertex v suh tht u v 8. if v Q n w(u, v) < v.key 9. v.π u 0. eresekey(v, w(u, v)) The lgorithm n be seen s mintining growing tree, efine by the preeessor informtion v.π, to whih eh vertex extrte from the queue is e. OMS0: isjoint sets n MSTs Slie /48 We use Prim s lgorithm to fin n MST in the following grph. OMS0: isjoint sets n MSTs Slie 4/48 The stte t the strt of the lgorithm: irst the lgorithm piks n rbitrry strting vertex r n uptes its key vlue to 0. In the bove igrm, the re text is the key vlues of the verties (i.e. v.key) n the green text is the preeessor vertex (i.e. v.π). Here we rbitrrily hoose s our strting vertex. OMS0: isjoint sets n MSTs Slie 5/48 OMS0: isjoint sets n MSTs Slie 6/48

10 Then is extrte from the queue, n the keys of its neighbours re upte: Then either or is extrte from the queue (here, we pik ):,,,,,, Vertex olours: lue: urrent vertex, green: verties e to tree. The re line shows the growing tree. OMS0: isjoint sets n MSTs Slie 7/48 OMS0: isjoint sets n MSTs Slie 8/48 Then is extrte from the queue: Then either or is extrte from the queue (here, we pik ):,,,,,,,,, OMS0: isjoint sets n MSTs Slie 9/48 OMS0: isjoint sets n MSTs Slie 40/48

11 Then is extrte from the queue: inlly is extrte from the queue n the lgorithm is omplete:,,,,,,,,,, OMS0: isjoint sets n MSTs Slie 4/48 OMS0: isjoint sets n MSTs Slie 4/48 orretness n omplexity Proof of orretness Prim s lgorithm mintins single, growing tree T strting with r, n to whih eh vertex remove from Q is ppene. h vertex e to T is vertex onnete to T by lightest ege. The ut property is therefore stisfie (tking S = T ), so when the lgorithm ompletes, T is n MST. The preeessor informtion v.π n be use to output T. omplexity nlysis: The omplexity is symptotilly the sme s ijkstr s lgorithm. If the priority queue is implemente using binry hep, we get n overll boun of O( log V ); if it is implemente using iboni hep, we get O( + V log V ). omprison of MST lgorithms To summrise the two MST lgorithms isusse: lgorithm Unerlying struture Runtime Kruskl isjoint-set O( log ) (linke lists) O( α(v )) (isjoint-set forest, eges lrey sorte) Prim Priority queue So whih lgorithm to use? O( log V ) (binry hep) O( +V log V ) (iboni hep) If the eges re not lrey sorte, n nnot be sorte in liner time, the most effiient lgorithm in theory is Prim with iboni hep (but in prtie, either Kruskl with isjoint-set forest or Prim with binry hep is likely to be quiker). If the eges re lrey sorte, or n be sorte in time O(), then Kruskl with n optimise isjoint-set forest is quikest. OMS0: isjoint sets n MSTs Slie 4/48 OMS0: isjoint sets n MSTs Slie 44/48

12 Summry isjoint-set struture provies n effiient wy to store olletion of isjoint subsets of some universe, n n be implemente using n rry of linke lists. isjoint-set strutures n be use to mintin set of onnete omponents of grph, n lso to fin minimum spnning trees using Kruskl s lgorithm. n lterntive wy of fining minimum spnning trees is Prim s lgorithm, whih is bse on the use of priority queue n is similr to ijkstr s lgorithm. oth lgorithms re greey lgorithms whih rely on the optiml substruture property of minimum spnning trees. urther Reing Introution to lgorithms T. H. ormen,.. Leiserson, R. L. Rivest n. Stein. MIT Press/MGrw-Hill, ISN: hpter t Strutures for isjoint Sets (N: presente slightly ifferently to leture) hpter Minimum Spnning Trees lgorithms S. sgupt,. H. Ppimitriou n U. V. Vzirni hpter 5 Greey lgorithms lgorithms leture notes, University of Illinois Jeff rikson Leture 8 Minimum spnning trees OMS0: isjoint sets n MSTs Slie 45/48 OMS0: isjoint sets n MSTs Slie 46/48 iogrphil notes iogrphil notes Joseph. Kruskl, Jr. (98 00) Robert. Prim III (9 ) Kruskl ws n merin mthemtiin n omputer sientist who i importnt work in sttistis n ombintoris, s well s omputer siene. His lgorithm ws isovere in 956 while t Prineton University; he spent most of his lter reer t ell Lbs. His two brothers Willim n Mrtin were lso fmous mthemtiins. Pi: ms.org Prim is n merin mthemtiin n omputer sientist, who evelope his lgorithm while working t ell Lbs in 957, where he ws lter iretor of mthemtis reserh. Prim s lgorithm ws originlly n inepenently isovere in 90 by Jrník. It ws lter reisovere gin by sger ijkstr in 959. Pi: ms.org OMS0: isjoint sets n MSTs Slie 47/48 OMS0: isjoint sets n MSTs Slie 48/48