Parallel scan on linked lists

Transcription

1 Parallel scan on linked lists prof. Ing. Pavel Tvrdík CSc. Katedra počítačových systémů Fakulta informačních technologií České vysoké učení technické v Praze c Pavel Tvrdík, 00 Pokročilé paralelní algoritmy (PI-PPA) LS 00/, Seminář 4 Evropský sociální fond. Praha & EU: Investujeme do vaší budoucnosti prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 /

2 Linked lists An n-element single-linked list L is represented by an array S of successors. Each element has a unique identification from {,..., n}. L: S(ucc): prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 /

3 Conversion of a single-linked list to a double-linked list P [i] = j S[j] = i Fully parallel (i.e., linearly scalable) EREW PRAM(n, p) algorithm: T (n, p) = O(n/p). Algorithm Algorithm EREW PRAM SingleDouble (in: S[,..., n]; out: P [,..., n]) for all i :=,..., n do in parallel {P [i] := i; if (S[i] i) then P [S[i]] := i} prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 /

4 List ranking: sequential vs. parallel algorithms The rank = the distance (= the # of pointers) from the end of the list. Sequential algorithm: trivial by traversing pointers backwards. List ranking by pointer jumping (PJ) on CREW PRAM(n, n) Algorithm Algorithm CREW PRAM ListRanking (in,out:s[,..., n], out: R[,..., n]) for all i :=,..., n do in parallel { if (S[i] = i) then R[i] := 0 else R[i] := ; repeat log n times { R[i] := R[i] + R[S[i]]; S[i] := S[S[i]]}} ( pointer jumping ) prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 4 /

5 Parallel suffix sum Parallel suffix sum on linked lists on CREW PRAM(n, n) Algorithm Alg. CREW PRAM Par Suffix Sum (in,out: S[,..., n], out: V [,..., n]) for all i :=,..., n do in parallel { if (S[i] = i) then V [i] := 0 else V [i] := v i ; repeat log n times { V [i] := V [i] V [S[i]]; S[i] := S[S[i]] }; ( pointer jumping ) if (original V [last] 0) then V [i] := V [i] V [last]} prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 5 /

6 An example of CREW PRAM(6, 6) pointer jumping L: S(ucc): R(ank): prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 6 /

7 The importance of list ranking The ranking allows to transform (permute) a linked list into an array: each element is placed into the location equal to its rank. Since then, all prefix computations can be performed by previous PPSs on this array. prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 7 /

8 Scalability issues of pointer jumping The CREW PRAM(n, n) PJ is not cost optimal: C(n, n) = O(n log n). The PJ is oblivious: it keeps jumping even over pointers to the last element. Hence, W (n, n) = O(n log n), too. However, a cost optimization cannot be based on reducing p and assigning n/p elements of S to each processor, since Θ(n/) elements require Θ(log n) jumps to get to the end. The number of elements which are done after step i is i + for i (see the yellow nodes x on the previous figure). CREW PRAM(n, p) PJ algorithm can be made work-optimal only if each processor keeps a list of and jumps only over active (=not done yet) elements. Even this optimized non-oblivious approach does not improve the cost, since one processor can have n/p active elements in all log n steps and hence T (n, p) = O((n/p) log n) and C(n, p) = O(n log n). prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 8 /

9 Observation Consider a linked list of n = n/ log n elements and apply pointer jumping using p = n/ log n processors T (n, p) = O(log n) and C(n, p) = O(n). The idea of a scalable list ranking Let p = Θ(n/ log n) and L = n elements. Using p processors, shrink L to L of size n = O(n/ log n) with C(n, p) = O(n), i.e., in T (n, p) = O(log n). Apply pointer jumping to L. Then T (n, p) = O(log n ) = O(log n) and C(n, p) = O(n). 4 Restore L from L and finish ranking computations for elements in L L with the same complexity as in step. prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 9 /

10 Symmetry breaking: Independent sets Definition 4 A subset I L is an independent set of a linked list L if i I; if S[i] i then S[i] I. Lemma 5 I L is an independent set i I can be deleted from L in parallel. Proof. If I L is an independent set, then i I; P [i] I. L: L : prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 0 /

11 Local minima of a linked list coloring Lemma 6 The set of local minima of a k-coloring c of an n-element list L is an independent set of size Ω(n/k) and a work-optimal parallel algorithm to determine the local minima. Proof. Let u, v = local minima of c with no other local minima in between. Then u and v cannot be adjacent (If u = S[v], then c[u] < c[v] and c[v] < c[u].) Colors of elements between u and v must form a bitonic sequence of at most k colors I n/(k ) = Ω(n/k). Determining local minima on EREW PRAM is inherently parallel: u in parallel: compare c[s[u]], c[u], and c[p [u]]. prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 /

12 Lemma 7 The best possible coloring is a -coloring. Proof. Two colors do not allow parallel coloring by processors working independently on disjoint sublists of L. Lemma 8 After Θ(log log n) removals of local minima of -colorings of an n-element list L L reduces to L with L n/ log n. Proof. Let m = the number of iterations needed to reduce L to L. Let L k = L after k reductions and I k = the set of local minima of a -coloring of L k. Then I k L k /4 and L k+ = L k I k (/4) L k. By recursion, L k ( k 4) n. L m n/ log n m.4 log log n. prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 /

13 Symmetry breaking and deterministic coin tossing (DCT) Definition 9 Symmetry breaking = an initial n-coloring. If t log i, let bin t (i) = i t... i 0 = the binary representation of i on t bits. DiffPos(t, i, j) = the least bit number in which bin t (i) and bin t (j) differ. Example, DiffPos(5,, 9) = (since bin 5 () = 0 and bin 5 (9) = 00). prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 /

14 Reduction of an n-coloring to a 6-coloring by DCT for i :=,..., n do in parallel c[i] := bin t (i), where t = log n repeat log n times { for i :=,..., n do in parallel { π[i] := DiffPos(t, c[i], c[s[i]]); c [i] := bin t (π[i] + c[i] π[i] ), where t = log t + c[i] := c [i]; }; t := t } prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 4 /

15 Proof of correctness Lemma 0 If c is a valid coloring of L, then c is a coloring of L, too. Proof. c is a coloring c[i] c[s[i]] π[i] is well defined. Assume c is not a valid coloring. Then i such that c [i] = c [S[i]]. c[i]π[i] π[i] = π[s[i]] and c[i] π[i] = c[s[i]] π[s[i]]. But then c[i]π[i] = c[s[i]] π[i] : contradiction. Lemma DCT reduces a t-bit coloring c to t = ( log t + )-bit coloring c. Proof. If c is a t-bit coloring, then for any i, 0 π[i] t the greatest color number in c can be (t ) + = t to encode colors from 0 to t, we need log(t) bits. prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 5 /

16 Corollary DCT must be applied O(log n) times, where the log-star function is defined as log (n) = min{i log (i) (x) }. For any realistic value of n, the number of iterations of DCT is at most 5. prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 6 /

17 n log n < n < n < n 4 4 < n < n Lemma Iterations of DCT can reduce the number of colors of a coloring only to 6. Proof. Assume t = log n =. Then t = log(t) = and 0 c [i] t = 5. prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 7 /

18 6-coloring -coloring for i :=,..., n do in parallel if c[i] = 5 then c[i] {0,, } {c[s[i]], c[p [i]]}; for i :=,..., n do in parallel if c[i] = 4 then c[i] {0,, } {c[s[i]], c[p [i]]}; for i :=,..., n do in parallel if c[i] = then c[i] {0,, } {c[s[i]], c[p [i]]}; Theorem 4 Using DCT, a -coloring on p processors takes T (n, p) = O((n log n)/p) and C(n, p) = O(n log n). prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 8 /

19 Input: L = {n, S[,..., n], P [,..., n], R[,..., n]}; Output: R[,..., n]; ( final values of ranks of elements ) Aux: F [,..., n], N[,..., n], c[,..., n];. k := 0; L k := L; n k := L k ;. while n k > n/ log n do {.. apply -coloring c to L k ;.. ( identify I k = the set of local minima of c in L k ); for i :=,..., n k do in parallel if (c[i] < min(c[p [i]], c[s[i]])) then F [i] := N[i] := else F [i] := N[i] := 0;.. ( remove I k from L k );... apply parallel scan to N[,..., n k ];... for i :=,..., n k do in parallel if F [i] = then {U[N[i]] := {i, S[i], P [i], R[i], k}; R[P [i]] := R[P [i]] + R[i]; S[P [i]] := S[i]; P [S[i]] := P [i] };.4. ( compact L k+ = L k I k into consecutive mem. locations );.4.. for i :=,..., n k do in parallel N[i] := F [i];.4.. apply parallel scan to N[,..., n k ];.4.. for i :=,..., n k do in parallel if F [i] = 0 then { S[N[i]] := N[S[i]]; P [N[i]] := N[P [i]]; R[N[i]] := R[i];}.5. n k+ := n k I k ; k := k + ; };. apply pointer jumping to compute ranking in L k ; 4. restore L from L k and rank all removed nodes by reversing steps. and.4; prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 9 /

20 A more scalable list ranking - performance Theorem 5 The previous list ranking algorithm takes on EREW PRAM with p = Θ(n/ log n) T (n, p) = O(log n log log n), C(n, p) = O(n log log n), and W (n, p) = O(n), supposing that we use approximation log n = O(). prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 0 /

21 A more scalable list ranking - performance Proof..: T (n k, p k ) = O(log n k ) if p k = Ω(n k / log n k ).,..,.4.,.4.: T (n k, p k ) = O(n k /p k ) = O(log n k ) if p k = Ω(n k / log n k ) T..,.4.: T (n k, p k ) = O(log n k ) if p k = Ω(n k / log n k ), 4: T (n, p) = O(log n log log n) : T (n/ log n, p) = O(log n) C, 4: C(n, p) = O(n log log n) : C(n/ log n, p) = O(n).: W (n k, p k ) = O(n k log n k ) = O(n k ).,..,.4.,.4.: W (n k, p k ) = O(n k ) W..,.4.: W (n k, p k ) = O(n k + p k log p k ) = O(n k ), 4: W (n, p) = O( log log n k= n k ) = O( log log n k= (/4) k n) = O(n) : W (n/ log n, p) = O(n) Note that p = n/ log n > n k / log n k for all k =,..., log log n. prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 /

22 Example S: P: R: F: N: U: i 5 6 S[i] 7 8 P[i] 4 R[i] R: L: R: L: F: N: S: P: R: R: L: R: L: prof. Pavel Tvrdík (FIT ČVUT) Linked List ParScan PI-PPA, 0, Seminář 4 /