CSL 730: Parallel Programming. Algorithms

Transcription

1 CSL 73: Parallel Programming Algorithms

2 First 1 problem Input: n-bit vector Output: minimum index of a 1-bit

3 First 1 problem Input: n-bit vector Output: minimum index of a 1-bit Algorithm: Divide into n blocks of n bits each For block i, B[i] = 1 if there is a 1 in the block Arbitrary CRCW, O(n) work, O(1) time Find the first j such that B[j] = 1 O(n) work, O(1) time Find first 1 in the j th block O(n) work, O(1) time

4 All Nearest Smaller Value For each element A[i] in a vector find the largest j<i, such that A[j] < A[i] 3

5 All Nearest Smaller Value For each element A[i] in a vector find the largest j<i, such that A[j] < A[i] Algorithm: Find left match: For each i For each j < i, B[i,j] = (A[j] < A[i]) O(n ) work For each B[i] find largest j<i with B[i,j] = 1 3

6 Prefix Minima For each element A[i] in a vector find the minimum of all A[j], j < i 4

7 Prefix Minima For each element A[i] in a vector find the minimum of all A[j], j < i m+ m+ m+ m+ m a 4

8 Prefix Minima For each element A[i] in a vector find the minimum of all A[j], j < i m+ m+ m+ m+ m b a 4

9 Prefix Minima For each element A[i] in a vector find the minimum of all A[j], j < i m+ m+ m+ m+ m b a 4

10 Prefix Minima For each element A[i] in a vector find the minimum of all A[j], j < i Algorithm: Find ANSV For each A[i] For each A[j], j<i PrefixMin[i] = A[j] where j is the largest index such that A[j] has no smaller value on its left m+ m+ m+ m+ m b a 4

11 Prefix Minima For each element A[i] in a vector find the minimum of all A[j], j < i Algorithm: Find ANSV For each A[i] For each A[j], j<i O(n ) work PrefixMin[i] = A[j] where j is the largest index such that A[j] has no smaller value on its left m+ m+ m+ m+ m b a 4

12 Prefix Minima 1.Partition into n blocks of n each.find prefix min for each block j recursively 3.Consider B = m 1.. m n where m j is the minima of block j Find prefix min of B in constant time 4.Minimum of A[1]..A[i], A[i] in block j is the minimum of m 1..m j-1 and A[i] s block prefix minima (computed in step ) 5

13 Prefix Minima 1.Partition into n blocks of n each.find prefix min for each block j recursively 3.Consider B = m 1.. m n where m j is the minima of block j Find prefix min of B in constant time 4.Minimum of A[1]..A[i], A[i] in block j is the minimum of m 1..m j-1 and A[i] s block prefix minima (computed in step ) O(n) work 5

14 Prefix Minima 1.Partition into n blocks of n each.find prefix min for each block j recursively 3.Consider B = m 1.. m n where m j is the minima of block j Find prefix min of B in constant time 4.Minimum of A[1]..A[i], A[i] in block j is the minimum of m 1..m j-1 and A[i] s block prefix minima (computed in step ) O(n) work O(n log log n) work, O(log log n) time 5

15 General Algorithmic Techniques 6

16 General Algorithmic Techniques Pipelining 6

17 General Algorithmic Techniques Pipelining Balanced binary tree 6

18 General Algorithmic Techniques Pipelining Balanced binary tree Divide and conquer 6

19 General Algorithmic Techniques Pipelining Balanced binary tree Divide and conquer Partitioning 6

20 General Algorithmic Techniques Pipelining Balanced binary tree Divide and conquer Partitioning Accelerated Cascading 6

21 General Algorithmic Techniques Pipelining Balanced binary tree Divide and conquer Partitioning Accelerated Cascading Pointer doubling 6

22 General Algorithmic Techniques Pipelining Balanced binary tree Divide and conquer Partitioning Accelerated Cascading Pointer doubling Symmetry breaking 6

23 PARALLEL ALGORITHM TECHNIQUES: PATH DOUBLING / POINTER JUMPING

24 Find Roots in a Forest

25 Find Roots in a Forest p(i): parent of node i

26 Find Roots in a Forest p(i): parent of node i 1 8 Do in parallel p(i) = p(p(i))

27 Find Roots in a Forest p(i): parent of node i Do in parallel p(i) = p(p(i)) Stop if p(i) = i

28 Find Roots in a Forest p(i): parent of node i Do in parallel p(i) = p(p(i)) Stop if p(i) = i Time: log(height) 9 7 9

29 Find Roots in a Forest p(i): parent of node i Do in parallel p(i) = p(p(i)) Stop if p(i) = i Time: log(height) Work: n log(height) 9 7 9

30 Find Roots in a Forest p(i): parent of node i Do in parallel p(i) = p(p(i)) Stop if p(i) = i Time: log(height) Work: n log(height) 9 4 7

31 Find Roots in a Forest p(i): parent of node i Do in parallel p(i) = p(p(i)) Stop if p(i) = i Time: log(height) Work: n log(height) 9 1

32 Pointer Jumping Progressively push computation to all elements at a given distance Doubling the distance at each step After k steps the computation has been performed for elements within a distance of k Applies to array, list, tree

33 List Ranking Compute the number of nodes before node i in a list

34 List Ranking Compute the number of nodes before node i in a list Parallel for all i do: if next[i]=null then d[i] else d[i] 1 Parallel for all i do: if next[i] null d[i] d[i] + d[next[i]] next[i] next[next[i]] un@l next[i] == null for all i d(i) = distance of i from the end of the list

35

36

37

38

39 O(n log n) work, O(log n) time 5 13

40 PARALLEL ALGORITHM TECHNIQUES: SYMMETRY BREAKING

41 Cycle Coloring c[i]!= c[j] if e[i] = j e[i] = The next vertex from i O(n) Sequen@al algorithm Parallel: Make local decision But all ver@ces look the same locally Break symmetry Put ver@ces in iden@fiable classes

42 3-Coloring Use vertex index to break symmetry But too many indices Algorithm: Initialize c[i] = i Iterate, reducing #colors each time parallel for all i do Let k = Least significant bit in which c[i] and c[e[i]] differ Set c[i] = k + bit k of c[i] 16

43 3-Coloring k = ; nd bit = => New color = * + = 4

44 3-Coloring k = ; nd bit = => New color = * + = k = ; th bit = 1=> New color = * + 1 = 1

45 3-Coloring k = ; nd bit = => New color = * + = k = ; th bit = 1=> New color = * + 1 = 1 Why does this produce a coloring?

46 3-Coloring #new colors = O(log (#old colors)) Time O(log*n) Work O(n log*n) But can t reduce the number of colors below 6 t bits for i => log t + 1 bits for iteration i+1, t > 3 One more steps each to remove three extra colors parallel for all i If(c[i] = extra-color) c[i] = {,1,} and NOT (c[[e[i]], c[predecessor[i]) 18