Massachusetts Institute of Technology Lecture : Theory of Parallel Systems Feb. 25, Lecture 6: List contraction, tree contraction, and

Transcription

1 Massachusetts Istitute of Techology Lecture.89: Theory of Parallel Systems Feb. 5, 997 Professor Charles E. Leiserso Scribe: Guag-Ie Cheg Lecture : List cotractio, tree cotractio, ad symmetry breakig Work-eciet list prex I the previous lecture, we saw a elegat \poiter jumpig" algorithm for computig a prex operatio o a liked list, with a good critical path of (lg ) o a -elemet list. Ufortuately, the poiter jumpig approach requires ( lg ) work for a job that ca easily be doe i liear time o a serial machie. This sectio presets a work-eciet radomized algorithm for prex computatio o a list, with a expected critical path of O(lg ) ad expected work of O(). This algorithm also gives us a work-eciet method for computig ode depths of a biary tree, sice the Euler-tour method see i class reduces the computatio of depths to a list prex.. List cotractio algorithm The key idea is \list cotractio." We are able to reduce the size of the prex calculatio ad the recursively compute the \cotracted" list. Specically, give a associative biary operator ad a list of elemets, umbered from to ad with values a ; a ; : : : ; a, we follow the followig procedure, which we call Radomized-List-Prefix:. Choose victims. We radomly choose a set of elemets for elimiatio from the list, by rst idepedetly (i the probablity sese) labelig each elemet either head or tail, ad the pickig ay head-labeled elemet which is ot followed immediately by a head-labeled elemet. (I this scheme, the last elemet of the list is always chose if it is a head.) The victims form a idepedet set (that is, a subset of the graph i which o two odes are adjacet), sice if a elemet is a victim, its successor is a tail ad its predecessor is followed by a head (the victim itself) ad so either eighbor ca also be a victim.. Splice out victims. The victims are the spliced out of the list by jumpig the poiters of their predecessors over them. As each victim i is removed from the list, its value a i is pushed ito its successor by settig the successor's value to be a i a i+. (Idepedece of victims guaratees that the successor is ot also a victim.) This computatio maitais the followig ivariat: if elemet i has ot bee spliced out of the list, the the product (with respect to, ad i order) of all the values stored i all elemets before i still i the list is equal to [; i], which is the desired al value for elemet i. (Recall that [j; k] where j k is shorthad otatio for a j a j+ : : : a k.)

2 Lecture : List cotractio, tree cotractio, ad symmetry breakig. Recursively compute. We ow recursively call Radomized-List-Prefix o the cotracted list (which i fact may be the same list, if by chace o elemets were elimiated). Due to the ivariat maitaied above, the prex computatio o the cotracted list gives the right aswer for the origial list for o-elimiated elemets. (The base case of recursio is a sigle elemet.). Splice i. Now we wat to splice i the elimiated elemets ito the cotracted list (which ow has the property that elemet i cotais the value [; i]) by restorig the poiters (which, of course, eeded to have bee saved somewhere) ad settig the value of each spliced-i elemet i to a i? a i = [; i? ] [i; i] = [; i]. At this poit, the prex computatio is doe. Figure shows how Radomized-List-Prefix rus o a list of elemets.. Aalysis of work Ituitively, the reaso that list cotractio does less work tha poiter jumpig is that the elimiated elemets icur o further work oce they are spliced i ad out, but i poiter jumpig, every elemet cotiues to icur work at every step (util the base case is reached). We ca show that the expected legth of the list with which Radomized-List-Prefix is called is reduced by a costat fractio at every cotractio. Let k be the legth of the list after k cotractios. Give a list of elemets, for ay elemet i, Pr fi is elimiatedg = ( Pr fi is headg Pr fi's successor is tailg = if i < ; Pr fi is headg = if i = ; ad thus, after cotractio, E [ k ] k?. Substitutig recursively, ad usig the fact that E [E [ k ]] = E [ k ], we have that E [ k ] = k 0 k : This directly leads to a geometric series that upper bouds the amout of work doe. We get a work which is liear i the legth of the list: X k= E [ k ] = () : Therefore the list cotractio algorithm is work-eciet.

3 Lecture : List cotractio, tree cotractio, ad symmetry breakig (a) [, ] [, ] [, ] [, ] [5, 5] [, ] choose victims (b) TAIL HEAD HEAD TAIL HEAD TAIL [, ] [, ] [, ] [, ] [5, 5] [, ] splice out (c) [, ] [, ] [, ] [, ] [5, 5] [5, ] recursively compute (d) [, ] [, ] [, ] [, ] [5, 5] [, ] splice i (e) [, ] [, ] [, ] [, ] [, 5] [, ] Figure : Radomized-List-Prefix ru o a list of elemets. This sequece shows a sigle cotractio. (a) The iput list. (b) Elemets are assiged head or tail ad the victims are chose accordigly. (c) Victims are spliced out, with their values beig pushed ito successors. (d) Prex of the cotracted list is recursively computed. (e) Victims are spliced back i.

4 Lecture : List cotractio, tree cotractio, ad symmetry breakig. Aalysis of critical path The critical path is liear i the umber of cotractios eeded, i.e. the umber of cotractios eeded to reach the base case of a oe-elemet list. Cetral to our probabalistic aalysis is Markov's iquality: for ay oegative radom variable v ad costat t, k. This boud is so Pr fv tg E [v] =t. First we observe that Pr f k g E [ k ] weak that we eed k log = just to beat the trivial boud Pr fv tg. Nevertheless, if we set k = r log = for some costat r, we have o h Pr r log= E r log= r log= = =?r : i Now we aalyze the expected umber of cotractios k. Sice for a oegative itegral radom variable v, E [v] = P i=0 Pr fv = ig i, we have the followig upper boud o k, by groupig the elemets of the expectatio summatio: o E [k] Pr 0 log = < k log = = X X r=0 r=0 X r=0 + Pr log = < k log = + Pr log = < k log = + : : : + log = Pr r log = < k (r + ) log = o Pr k r log = (r + ) log = o Pr k r log = (r + ) log = X r= o o log = log = o Pr k r log = (r + ) log = log = + log = + = log = + O() = (lg ) : X r= o?r (r + ) log = (r + ) log = Thus the critical path of Radomized-List-Prefix is O(lg ).

5 Lecture : List cotractio, tree cotractio, ad symmetry breakig 5 u Figure : Example of treex computatios with the additio operator. The umber iside each ode is the value of the elemet stored there, the umber to the left of each ode is the result of the leax operatio, ad the umber to the right is the result of the rootx operatio. Cosider ode u. Its leax result is the i-order sum of all elemets stores i the tree rooted at u ( = 0), ad its rootx result is the prex sca from the root ( + 0 = ). Treex computatio Suppose we have a biary tree of elemets ad a associative biary operator. I this sectio, we cosider a class of \treex" computatios: a \rootx" operatio computes the -sca of every simple path from root to leaf, ad a \leax" operatio computes, for every ode u, the result of the i-order -product described by the tree rooted at u. Figure shows a example of rootx ad leax with the additio operator.. Rootx computatio with tree cotractio To perform rootx with the additio operator, we ca use the Euler-tour method: for each ode with value v, set the icomig A value to be v, the middle B value to be 0, ad the outgoig C value to be?v. Ufortuately, this method does ot geeralize to ay associative biary operator, sice it requires a iverse, so that the values computed o the way dow (the A values) ca be udoe o the way up (the C values). Maximum, for istace, could ot be computed with a Euler-tour. For a geeral rootx computatio, we use a method iveted by Miller ad Reif [] called \tree cotractio," which is aalogous to list cotractio. Though a determiistic algorithm with the same bouds exists, we cosider a radomized algorithm that is both simpler to aalyze ad geerally faster i practice. This algorithm, Radomized-Rootfix, is aalogous to Radomized-List-Prefix ad works as follows:. Choose victims. To choose odes to be elimiated, we radomly choose pairs of adjacet odes i the biary tree so that all pairs are idepedet (i.e. o pairs share odes), ad select the child of each pair to be a victim. To do this, we make each ode idepedetly choose oe of its eighbors (either paret or childre) based o a

6 Lecture : List cotractio, tree cotractio, ad symmetry breakig Figure : The probability distributio of choosig a eighbor depeds o the umber of childre. (a) Leaves always choose parets. (b) Oe-child odes choose betwee eighbors uiformyly. (c) Nodes with two childre ever choose their paret, guarateeig that the child i ay pair ever has two childre, which guaratees that a biary tree remais biary after pairs are combied. probability distributio that depeds o how may childre the ode has. If two odes choose each other, they are paired. If a ode has o childre, i.e. it is a leaf, the it always chooses its paret. If a ode has oe child, it radomly chooses its paret or child uiformly. (I the special case of a root with oe child, we ca make it always choose its child.) If a ode has two childre, it radomly choose oe of its childre uiformly, ever choosig its paret. Figure shows the three types of odes with the various probabilities.. Splice out. After victims are selected, we cotract the tree by splicig them out, settig the paret's child poiter (whichever oe poited to the victim) to the victim's ow child, if oe exists. We kow that o victim will have two childre, sice odes with two childre ever choose their paret. This property guaratees that a biary tree remais biary after cotractio. As a ode is spliced out, its value is pushed ito its child, if oe exists; specically if u is a victim ad it has a child v, the v's value is set to u v. This maitais the ivariat that the -product of the sequece of odes from the root to ay o-elimiated ode is the correct al value for that ode.. Recursively compute. We recursively compute the roox of the cotracted tree by callig Radomized-Rootfix, with a sigle ode as base case. The ivariat maitaied above isures that the values computed by the rootx of the cotracted tree are the correct values for the origial tree.. Splice i. Now we splice i the previously elimiated odes. As a ode is spliced i, its proper rootx value is calculated from its paret's value (which has bee recursively computed). Specically, if a ode u is the paret of a ode v which is beig spliced i, the value of v is set to u v, which will be the correct al value for v. Figure shows how this method works with the additio operator.

7 Lecture : List cotractio, tree cotractio, ad symmetry breakig 7 (a) 0 7 (e) (b) 0 choose victims v splice i 7 (d) splice out (c) u recursively compute Figure : Example of rootx with the additio operator. (a) The iitial biary tree. (b) Each ode chooses a eighbor, idicated by outgoig arrows, ad the childre of pairs (odes which choose each other) become victims, which are shaded. (c) Victims are spliced out, with their values beig pushed to their childre. For example, ode u becomes + = 8. (d) The rootx of the cotracted tree is computed recursively. Notice these are the correct al values for the odes i the cotracted tree. (e) Victims are spliced back i, with their values beig computed from parets. For example, ode v becomes + = 7.

8 8 Lecture : List cotractio, tree cotractio, ad symmetry breakig. Aalysis of work ad critical path Let k be the umber of odes i the tree after k cotractios. We wat to show that the expected umber of odes i the tree decreases by a costat fractio at each cotractio, ad the the aalysis is the same as that of Radomized-List-Prefix. Cosider a biary tree with odes. Let c i be the umber of odes i the tree with i childre, so = c 0 + c + c. Now we cosider that expected umber of each type of ode that is elimiated durig a cotractio. Sice a ode is elimiated if it chooses its paret ad its paret also chooses it ad these choices are idepedet, the probability that a give ode is elimiated is the probability that it chooses its paret times the probability that its paret choose it. For leaves, this product is =, ad for odes with oe child, it is =. Thus, the expected umber of leaves elimiated is c 0 = ad the expected umber of oe-child odes elimiated is c =. Now recall that a biary tree has the property that c 0 = c +, ad thus c 0 = = c =+=. The the expected total umber of elimiated odes is at least c 0 + c = c 0 + c 0 + c = c 0 + c + (c 0 + c + c ) = : + c Sice the expected umber of odes i the tree decreases by at least a fourth durig each cotractio, we kow by the aalysis see for Radomized-List-Prefix that the expected work of Radomized-Rootfix is O() ad its expected critical path is O(lg ). (I fact, prex computatio o a list is a special case of rootx, ad Radomized-List-Prefix performs a special case of Radomized-Rootfix's fuctioality, with the head ad tail labels beig equivalet to a strig of oe-child odes choosig betwee paret ad child.). Leax computatio with tree cotractio To compute the leax o a tree, we ca use a tree cotractio algorithm which uses the same basic approach as Radomized-Rootfix ad achieve bouds of O() expected work ad O(lg ) expected critical path. However, we have to be more clever, sice the i a leax This is easily proved by iductio: give a biary tree, remove a leaf, ad the c 0 = c + holds by assumptio for the ew tree. Restore the removed leaf: if it is added to a leaf, the c 0 is uchaged sice the ew leaf replaces the leaf it is added to, ad c is also uchaged, ad if it is added to a ode with oe child, the the ew leaf icreases c 0 by oe ad the ode it was added to chages ito a ode with two leaves, also icreasig c by oe.

9 Lecture : List cotractio, tree cotractio, ad symmetry breakig 9 computatio the al value depeds o a etire subtree of elemets as opposed to a list of acestors. Whereas i the rootx calculatio we directly modied the elemet values as odes were spliced i ad out, i our leax calculatio we have each ode keep track of a triple of values ad the modify these triples accordig to some ivariat. Specically, our algorithm Radomized-Leaffix keeps track, at each ode, a triple of costats ha; b; ci, which represets the followig computatio: a l b r c, where l ad r are variables for the leax values of the left ad right childre, or, if a child does ot exist, the idetity I of. For each ode, we iitialize the triple by settig a to be the elemet value ad b ad c to be I. Radomized-Leaffix maitais the ivariat that the al leax value of a ode ca be obtaied by recursively evaluatig the computatio represeted by that ode's triple (i.e. dig the values for l ad r based o the subtrees ad the computig a l b r c). For coveiece, we call this the \triple ivariat." After triples are iitialized, Radomized-Leaffix works as follows:. Choose victims. Radomly choose ode for elimiatio as i Radomized-Rootfix.. Splice out. If u is the paret of a victim v, we push v's triple up ito u's triple as v gets spliced out. Relyig o the fact that v has at most oe child, we adjust the values of v's triple so that the triple ivariat is maitaied. Specically, rememberig that the values i the triple represet a computatio, we eectively substitute the computatio for v ito the computatio for u, ad the combie adjacet costats with. This will result i a computatio that ca be represeted by a triple, sice v has at most oe child. For example, if u's triple is ha u ; b u ; c u i, v's triple is ha v ; b v ; c v i, v is u's right child ad v itself has a left child, the u's triple becomes ha u ; b u a v ; b v I c v c u i. If istead v has a right child, for example, the u's triple becomes ha u ; b u a v I b v ; c v c u i. The key observatios are that this adjustmet of u's triple takes costat time because of the costat size of the triples, there is a costat umber of ways the paret's triple ca be adjusted ad that the triple ivariat is maitaied.. Recursively compute. We recursively compute the leax operatio o the cotracted tree with Radomized-Leaffix, without eedig to iitialize the triples as i the top-level call. The base case is whe a sigle ode, i which case the al leax value is simply a I b I c where ha; b; ci is the ode's triple. After the recursive call, the values computed for the odes i the cotracted tree will be the correct leax values for those odes, due to the triple ivariat.. Splice i. As a ode is spliced back ito the tree, its leax value is computed i costat time by evaluatig the computatio represeted by its triple: hal br ci. This is possible because the value of the ode's child (if oe exists) will have bee recursively computed, so both l ad r are kow (at least oe will be I). At this poit, every ode cotais the leax value ad the algorithm termiates. This algorithm requires to have a idetity, which is ot a additioal costrait o sice a toke idetity ca always be created ad the operator modied to recogize it, if a atural idetity does ot exist.

10 0 Lecture : List cotractio, tree cotractio, ad symmetry breakig Because we have esured that splicig i ad out take costat time, the work ad critical path aalysis for Radomized-Leaffix is the same as that for Radomized-Rootfix, ad thus we have expected work of O() ad expected critical path of O(lg ).. Geeral expressio evaluatio with tree cotractio Radomized-Leaffix works by eectively keepig track of a expressio at each ode (represeted by a triple) which depeds o the expressios of that ode's childre. The key poits are that a victim's expressio ca always be pushed up ito its paret's expressio i costat time durig splicig out, that this pushig up results i a expressio i the same form (so recursio works), that a victim's expressio ca be evaluated i costat time i the base case ad durig splicig i, ad that the triple ivariat is maitaied. This poweful approach is i fact quite geeral ad ca be applied to may biary tree problems. For example, cosider a biary tree that describes a expressio with the + ad operators, where iteral odes each cotai + or ad each leaf cotais a umber. The value of the expressio rooted at ay give ode ca always be formulated as (a l r) + (b l) + (c r) + d (ad thus stored as a -tuple), where a; b; c; d are costats ad l ad r are variables that represet the values of the expressios rooted at the left ad right subtrees. Substitutig a victim's -tuple (which represets a expressio with at most oe o-idetity-valued variable) ito it's paret's -tuple results i a expressio that ca be represeted by the same kid of -tuple. (This is easily see by checkig the possible cases.) The ivariat that a ode's -tuple represets the value of the expressio rooted at that ode is maitaied, ad base case ad evaluatio durig splicig i are straightforward, ad so we are doig essetially the same thig as Radomized-Leaffix with a dieret expressio form at each ode. For other applicatios of this approach, see []. Determiistic symmetry breakig Give a rig of elemets (a circular liked list), how ca we d a large idepedet set of elemets, where by \large" we mea a costat fractio? I Radomized-List-Prefix, we have already see a O()-critical path method for choosig a idepedet set with expected size of =, where is the size of the list. I this sectio we preset a algorithm for doig so determiistically with a critical path of O(lg ). This is called symmetry breakig because if adjacet elemets share a resource (the symmetry), choosig a idepedet set prevets competitio (the breakig). The algorithm Choose-Idepedet-Set works i two steps:. Fid -colorig. First, we d a -colorig of the rig elemets. This is doe by startig with a -colorig, where each elemet is assiged a distict color 0; ; : : : ;? Recall that lg is the miimum umber of repeated applicatios of lg eeded to reduce to at most. It is a very slow fuctio (lg 55 = 5) ad i practice ca be cosidered a small costat.

11 Lecture : List cotractio, tree cotractio, ad symmetry breakig based o its memory locatio (or processor idex, if we have oe processor per elemet), ad iteratively reducig the umber of colors i the colorig (wihtout ever havig same-colored eighbors) i the followig way. Suppose the color idices of the rig are k bits log. Let ha k?; a k?; : : : ; a 0 i be the color bits of a give elemet a, ad hb k?; b k?; : : : ; b 0 i be the colors bits of that elemet's sucessor b. Sice these two colors are dieret, we have some i where a i = b i. The umber i, sice it is less tha k, ca be writte i dlg ke bits: hi dlg ke?; i dlg ke?; : : : ; i 0 i. Assig hi dlg ke?; i dlg ke?; : : : ; i 0 ; a i i to be a's ew color, ad assig ew colors to all elemets i the same way. Now we have a colorig with idices of lg k + bits, sice if the rst dlg ke bits of a elemet's ew color are the same as the successor's rst dlg ke bits, the by costructio the last bit will dier (a i = b i ). This iterative re-colorig will reduce the umber of colors bits per elemet for all k, sice dlg ke + < k for k. Whe k =, the a elemet's color ca dier with its successor's bit i locatio 0,, or, ad so the ew colorig will be oe of the three -bit strigs h0; 0i, h0; i, or h; i cocateated with either 0 or, ad thus the ew colorig will be bits represetig oe of six values. Hece the iterative reductio of colors termiates at six colors, ad sice the umber of bits per color is reduced by a lg at each iteratio, O(lg ) iteratios, each with costat critical path, are eeded.. Choose elemets. To determie the idepedet set of elemets from after - colorig, we start will all elemets marked as live ad the iterate through each color c = 0; ; : : : ; 5, killig ay live eighbors of elemets with color c as we go. There are at least = live elemets left at the ed of this process, because it is impossible to have three cosecutive dead elemets: either the middle elemet kills its eighbors ad thus stays alive, havig o oe left to kill it, or it is killed by oe of its eighbors, ad thus the eighbor stays alive. These remaiig live elemets comprise a idepedet set, sice it is impossible to have two adjacet elemets remai alive: oe would have killed the other. Thus this O()-critical path iteratio gives us a idepedet set of size =. Overall, Choose-Idepedet-Set has O(lg ) critical path. Refereces [] C. E. Leiserso ad B. M. Maggs. Commuicatio-eciet parallel graph algorithms. Algorithmica, pages 5{77, 988. [] G. L. Miller ad J. H. Reif. Parallel tree cotractio ad its applicatios. Proceedigs of the th Aual Symposium o Foudatios i Computer Sciece, pages 78{89, 985. This shows that the set chose is a maximal idepedet set, meaig that we caot revive ay of the dead elemets ad cotiue to have a idepedet set. Note the distictio from havig a maximum idepedet set, the idepedet set with the most elemets; dig the maximum idepedet set is i fact NP-hard.