Programmazione di sistemi multicore

Transcription

1 Programmazione di itemi multicore A.A LECTURE 9 IRENE FINOCCHI

2 More complex parallel pattern PARALLEL PREFIX PARALLEL SORTING PARALLEL MERGESORT PARALLEL QUICKSORT

3 Outline 2 Done: Simple way to ue parallelim or counting, umming, inding Analyi o running time and implication o Amdahl Law Next: Clever way to parallelize more than i intuitively poible Parallel preix: A key trick that underlying ome urpriing parallelization Enable other thing like pack (ome time called iltering) Parallel orting: quickort (not in place) and mergeort Eay to get a little parallelim With cleverne can get a lot

4 The preix-um problem Given int[] input, produce int[] output where output[i] = input[0] + input[1] + + input[i] Sequential code i a very eay CS1 exam problem: 3 any binary aociative operator int[] preix_um(int[] input){ int[] output = new int[input.length]; output[0] = input[0]; or (int i=1; i < input.length; i++) output[i] = output[i-1]+input[i]; return output; } Doe not eem parallelizable Execution DAG i a chain: why? Data dependencie Work O(n), pan O(n) Thi algorithm i inherently equential, but a dierent algorithm can achieve pan O(log n)

5 Hilli & Steele olution [1986] 4

6 Invariant property 5 At the beginning o i-th iteration, or i 0, any array item A[x] contain the um o the at mot 2 i preceding element (including input[x]) Proo True at the beginning: i=0 Single item Induction tep: property true up to iteration i implie property true up to iteration i+1 During iteration i+1, we et A[x] = A[x] + A[x-2 i ] By inductive hypothei: A[x] = input[x] + input[x-1] + + input[x-2 i -1] By inductive hypothei: A[x-2 i ] = input[(x-2 i )] + input[(x-2 i )-1] + + input[(x-2 i )-2 i -1] Hence, ater iteration i+1, A[x] will contain the um o the (at mot) 2 i +2 i =2 i+1 immediately preceding item

7 Analyi 6 log n iteration, hence pan = O(log n) Θ(n) work per iteration, hence total work = Θ (n log n) Work higher than equential algorithm, but much more parallelim! Can we do any better?

8 Parallel preix-um via balanced tree 7 The work-eicient parallel-preix algorithm doe two pae Each pa ha O(n) work and O(log n) pan So, in total, there i O(n) work and O(log n) pan Like with array umming, parallelim i n/log n An exponential peedup Firt pa build a tree bottom-up: the up pa Second pa travere the tree top-down: the down pa Hitorical note: Original algorithm due to R. Ladner and M. Ficher at the Univerity o Wahington in 1977

9 The algorithm: up pa 8 1. Build a binary tree where Root ha um o the range [0,n) I a node ha um o [lo,hi) and hi>lo, Let child ha um o [lo,middle) Right child ha um o [middle,hi) A lea ha um o [i,i+1), i.e., input[i

10 Up pa range 0,8 um 76 romlet range 0,4 range 4,8 um 36 um 40 romlet romlet range 0,2 range 2,4 range 4,6 range 6,8 um 10 um 26 um 30 um 10 romlet romlet romlet romlet r 0,1 r 1,2 r 2,3 r 3,4 r 4,5 r 5,6 r 6,7 r 7, input output

11 The algorithm: up pa 1. Build a binary tree where Root ha um o the range [0,n) I a node ha um o [lo,hi) and hi>lo, Let child ha um o [lo,middle) Right child ha um o [middle,hi) A lea ha um o [i,i+1), i.e., input[i] Thi i an eay ork-join computation: combine reult by actually building a binary tree with all the range-um 10 Tree built bottom-up in parallel Clever implementation with an array like with binary heap Analyi: O(n) work, O(log n) pan

12 The algorithm: down pa Pa down a value romlet Root given a romlet o 0 Node take it romlet value and Pae it let child the ame romlet Pae it right child it romlet + it let child um (a tored in the up pa) At the lea or array poition i, output[i]=romlet +input[i]

13 Down pa range 0,8 um 76 romlet 0 range 0,4 range 4,8 um 36 um 40 romlet 0 romlet 36 range 0,2 range 2,4 range 4,6 range 6,8 um 10 um 26 um 30 um 10 romlet 0 romlet 10 romlet 36 romlet 66 r 0,1 r 1,2 r 2,3 r 3,4 r 4,5 r 5,6 r 6,7 r 7, input output

14 The algorithm: down pa 2. Pa down a value romlet Root given a romlet o 0 Node take it romlet value and Pae it let child the ame romlet Pae it right child it romlet + it let child um (a tored in the up pa) At the lea or array poition i, output[i]=romlet +input[i] Thi i an eay ork-join computation: travere the tree built in tep 1 and produce no reult Leave aign to output Invariant: romlet i um o element let o the node range Analyi: O(n) work, O(log n) pan 13

15 Sequential cuto 14 Adding a equential cuto i eay a alway Up cuto: jut a um, have lea node hold the um o a range Down cuto: output[lo] = romlet + input[lo]; or(i=lo+1; i < hi; i++) output[i] = output[i-1] + input[i]

16 Parallel preix, generalized 15 Jut a um-array wa the implet example o a common pattern (reduce), preix-um illutrate a pattern that arie in many, many problem! Minimum, maximum o all element to the let o i I there an element to the let o i atiying ome property? Count o element to the let o i atiying ome property Thi lat one i perect or an eicient parallel pack Perect or building on top o the parallel preix trick We did an incluive um, but excluive i jut a eay