Software Pipelining Can we decrease the latency? Goal of SP Lecture. Seth Copen Goldstein Software Pipelining

Size: px

Start display at page:

Download "Software Pipelining Can we decrease the latency? Goal of SP Lecture. Seth Copen Goldstein Software Pipelining"

Jasmine McDonald
6 years ago
Views:

1 5-4 Ltur Sotwr Piplinin Copyrit St Copn Goltin -8 Sotwr Piplinin Sotwr piplinin i n IS tniqu tt rorr t intrution in loop. Poily movin intrution rom on itrtion to t prviou or t nxt itrtion. Vry lr improvmnt in runnin tim r poil. T irt riou ppro to otwr piplinin w prnt y Aikn & Niolu. Aikn 988 P.D. ti. Imprtil it inor rour zr (ouin only on t-pnn ontrint). But prk lr mount o ollow-on rr. (om li orrow rom T Clln & M. Vo) St Copn Goltin St Copn Goltin -5 Gol o SP Inr itn twn pnnt oprtion y movin tintion oprtion to ltr itrtion A: l [] B: * C: t [], D: + 4 Aum ll v ltny o A B C D Cn w r t ltny? Lt unroll A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A B C D A B C D St Copn Goltin St Copn Goltin -5 4 St Copn Goltin -

2 5-4 Ltur Rnm vril Sul A D A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 B C D A B C A B C D A B C D A B C D D A B C St Copn Goltin St Copn Goltin -5 6 Unroll Som Mor A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A A B C D D A B C B C St Copn Goltin -5 7 D A B C D D A B C D A B C Unroll Som Mor A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A B C D3 D A B C D A B C A D A3 B3 C3 B C St Copn Goltin -5 8 D A B C D3 D A B C D A3 B3 C3 St Copn Goltin -

3 5-4 Ltur On Mor Tim A D Cn Rrrn A D B C A B D A D B C A B D A D A B C D4 D A B C D A B C D A3 B3 C3 D3 A4 B4 C4 C B C A3 B3 C3 D3 A4 B4 C4 A B C D4 D A B C D A B C D A3 B3 C3 D3 A4 B4 C4 C B C A3 B3 C3 D3 A4 B4 C St Copn Goltin St Copn Goltin -5 A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 Rrrn A B C D A B C D A B D A3 A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 A: l [] B: * C: t [], D: + 4 Rrrn A B C D A B C D A B D A3 D3 A B C D3 D A B C D A B C C D3 B3 C3 A B C D3 D A B C D A B C C B3 C3 D A3 B3 C3 D A3 B3 C St Copn Goltin St Copn Goltin -5 St Copn Goltin - 3

4 5-4 Ltur A: l [] B: * D: + 4 A: l [] D: + 4 C: t [], B: * A: l [] D: + 4 B: * C: t [], D3: + 4 C: t [], SP Loop Prolo Boy Epilo A B C C C D3 D A B B B C D A A A B C D D D St Copn Goltin -5 3 Gol o SP Inr itn twn pnnt oprtion y movin tintion oprtion to ltr itrtion A B C pnni in initil loop tr SP A itrtion i i+ i St Copn Goltin -5 4 B C Gol o SP Inr itn twn pnnt oprtion y movin tintion oprtion to ltr itrtion But lo, to unovr ILP ro itrtion ounri! Exmpl Aum oprtin on ininit wi min A A B A A B A B C A i B i- C i- A3 B C B i C i- B3 C C3 C i St Copn Goltin St Copn Goltin -5 6 St Copn Goltin - 4

5 5-4 Ltur C i Exmpl Aum oprtin on ininit wi min A Prolo A B A i B i- C i- loop oy B i C i- pilo or (i=; i<n; i++) { } A i B i C i Dlin wit xit onition i= i (i >= N) oto on A B i (i+ == N) oto lt i= A i (i+ == N) oto pilo i= loop: A i B i- C i- i++ pilo: B i C i- lt: i (i < N) oto loop i on: St Copn Goltin St Copn Goltin -5 8 Loop Unrollin V. SP For SuprSlr or VLIW Loop Unrollin ru loop ovr Sotwr Piplinin ru ill/rin Bt i i you omin tm # o ovrlpp itrtion Sotwr Piplinin Loop Unrollin Aikn/Niolu Sulin Stp Prorm lr rplmnt to limint mmory rrn wr poil. or i:= to N o := V[i-] := := := := := U[i] : V[i] := : W[i] := := X[i] or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] St Tim Copn Goltin St Copn Goltin -5 St Copn Goltin - 5

6 5-4 Ltur Aikn/Niolu Sulin Stp Aikn/Niolu Sulin Stp, ont Unroll t loop n omput t t-pnn rp (DDG). DDG or roll loop: or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] DDG or unroll loop: or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] St Copn Goltin St Copn Goltin -5 Aikn/Niolu Sulin Stp 3 Buil tlu o itrtion numr v yl tim yl itrtion Aikn/Niolu Sulin Stp 3 illy, Buil tlu you r multin o itrtion numr v yl tim. itrtion uprlr wit ininit rour, ininit ritr rnmin, lwy pritin t loop-k rn: tu, ut pur t pnny yl St Copn Goltin St Copn Goltin -5 4 St Copn Goltin - 6

7 5-4 Ltur Aikn/Niolu Sulin Stp 4 Fin rptin pttrn o intrution. yl itrtion Aikn/Niolu Sulin Stp 4 Fin rptin pttrn o intrution. yl itrtion St Copn Goltin St Copn Goltin -5 6 Aikn/Niolu Sulin Stp 4 Aikn/Niolu Sulin Stp 5 Fin rptin pttrn o intrution. yl itrtion Go k n rlt lop to DDG yl Col t lop itrtion yl itrtion St Copn Goltin St Copn Goltin -5 8 St Copn Goltin - 7

8 5-4 Ltur Aikn/Niolu Sulin Stp 6 Fin t loop oy n rroll t loop. yl itrtion Aikn/Niolu Sulin Stp 6 Fin t loop oy n rroll t loop. yl itrtion Prolou/ntry o Loop oy Epilou/xit o St Copn Goltin St Copn Goltin -5 3 Aikn/Niolu Sulin Stp 7 Gnrt o. (Aum VLIW-lik min or ti xmpl. T intrution on lin oul iu in prlll.) := := := U[] := X[] := := := U[] := X[] := V[] := W[] := := := := 3 := U[3] 3 := X[3] := V[] := 3 := := W[] := 3 := 3 4 := U[4] 4 := X[4] 3 := V[3] := 3 4 := 3 3 i := 3 L: i := i- i i+ := i i i := i i W[i] := i V[i+] := i+ i+ := U[I+] i+ := X[i+] i+ := i i i+ := i+ i+ i := i+ i i<n- oto L Aikn/Niolu Sulin Stp 8 Sin vrl vrion o vril (.., i n i+ ) mit liv imultnouly, w n to nw tmp n mov := := := U[] := X[] := := := U[] := X[] := V[] := W[] := := := := 3 := U[3] 3 := X[3] := V[] := 3 := := W[] := 3 := 3 4 := U[4] 4 := X[4] 3 := V[3] := 3 4 := 3 3 i := 3 L: i := i- i i+ := i i i := i i W[i] := i V[i+] := i+ i+ := U[I+] i+ := X[i+] i+ := i i i+ := i+ i+ i := i+ i i<n- oto L N- := N- N- N := N N- N- := N- N- W[N-] := N- N := N- N- N := N- + N N := N N w[n] := N N- := N- N- N := N N- N- := N- N- W[N-] := N- N := N- N- N := N- + N N := N N w[n] := N v[n] := N v[n] := N St Copn Goltin St Copn Goltin -5 3 St Copn Goltin - 8

9 5-4 Ltur Aikn/Niolu Sulin Stp 8 Sin vrl vrion o vril (.., i n i+ ) mit liv imultnouly, w n to nw tmp n mov := := := U[] := X[] := := := U[] := X[] := V[] := W[] := := := := := U[3] := X[3] := V[] := 3 := := W[] := 3 := 3 4 := U[4] 4 := X[4] 3 := V[3] := 3 4 := 3 i := 3 L: i := i i+ := := ; =; = ; =; = ; = i := i W[i] := i V[i+] := i+ i+ := U[I+] i+ := X[i+] i+ := i i+ := i+ i := i+ i i<n- oto L N- := N- N- N := N N- N- := N- N- W[N-] := N- N := N- N- N := N- + N N := N N w[n] := N v[n] := N Nxt Stp in SP AN88 i not l wit rour ontrint. Moulo Sulin i SP loritm tt o. It ul t loop on rour ontrint prn ontrint Billy, it lit ulin tt tk into ount rour onlit rom ovrlppin itrtion St Copn Goltin St Copn Goltin Rour Contrint Minimlly iniviil qun, i n, n xut totr i omin rour in tp o not x vill rour. R(i) i rour oniurtion vtor R(i) i t numr o unit o rour i r(i) i rour u vtor.t. r(i) R(i) E no in G n oit r(i) Sotwr Piplinin Gol Fin t m ul or itrtion. Str y itrtion initition intrvl, Gol: minimiz St Copn Goltin St Copn Goltin St Copn Goltin - 9

10 5-4 Ltur Sotwr Piplinin Gol Fin t m ul or itrtion. Str y itrtion initition intrvl, Gol: minimiz. Sotwr Piplinin Gol Fin t m ul or itrtion. Str y itrtion initition intrvl, Gol: minimiz. rour mut witin ontrint rour mut witin ontrint St Copn Goltin St Copn Goltin Sotwr Piplinin Gol Fin t m ul or itrtion. Str y itrtion initition intrvl, Gol: minimiz. Sotwr Piplinin Gol Fin t m ul or itrtion. Str y itrtion initition intrvl, Gol: minimiz. rour mut witin ontrint rour mut witin ontrint St Copn Goltin St Copn Goltin -5 4 St Copn Goltin -

11 5-4 Ltur Sotwr Piplinin Gol Fin t m ul or itrtion. Str y itrtion initition intrvl, Gol: minimiz. Sotwr Piplinin Gol Fin t m ul or itrtion. Str y itrtion initition intrvl, Gol: minimiz. U rour mut witin ontrint rour mut witin ontrint moulo rour tl St Copn Goltin St Copn Goltin -5 4 Prn Contrint Rviw: or yli ulin, ontrint i ut t rquir ly twn two op u, v: <(u,v)> For n, u v, w mut v σ(v)-σ(u) (u,v) Prn Contrint Cyli: ontrint om tupl: <p,> p i t minimum itrtion ly (or t loop rri pnn itn) i t ly For n, u v, w mut v σ(v)-σ(u) (u,v)-*p(u,v) p I t pnn i witin n itrtion, p= loop-rri ro p itr ounri, p> St Copn Goltin St Copn Goltin St Copn Goltin -

12 5-4 Ltur Itrtiv Appro Finin minimum S tt tii t ontrint i NP-Complt. Huriti: Fin lowr n uppr oun or S or rom lowr to uppr oun? Sul rp. I u, on Otrwi try in (wit nxt ir ) Tu: Itrtiv Moulo Sulin Ru, t.l. Huriti: Itrtiv Appro Fin lowr n uppr oun or S or rom lowr to uppr oun Sul rp. I u, on Otrwi try in (wit nxt ir ) So t ky irn: AN88 o not um S wn ulin IMS mut um n S or ulin ttmpt to unrtn rour onlit St Copn Goltin St Copn Goltin Lowr Boun Ayli Exmpl Rour Contrint: S R (lo ll II r ) mximum ovr ll rour o # o u ivi y # vill roun up or own? <,> Prn Contrint: S E (lo ll II r ) mx ovr ll yl: ()/p() <,3> Lowr Boun: S R = Uppr Boun: 5 In prti, on i y, otr i r. Tim rt ppro: ut u S R lowr oun, tn o inry r or t S <,> St Copn Goltin St Copn Goltin St Copn Goltin -

13 5-4 Ltur or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Lowr Boun on Aum ALU n MU Aum ltny Op or lo i yl <,> Rour => 5 yl Dpnni => 3 yl <,> <,> <,> <,> St Copn Goltin <,> <,> <,> <,> Sulin t trutur To ul or initition intrvl : Crt rour tl wit row n R olumn Crt vtor, σ, o lnt N or n intrution in t loop σ[n] = t tim t wi n i ul, or NONE Prioritiz intrution y om uriti ritil pt (or yl) rour ritil St Copn Goltin -5 5 Sulin loritm Pik n intrution, n Clult rlit tim u to pnn ontrint For ll x=pr(n), rlit = mx(rlit, σ(x)+(x,n)-. p(x,n)) try n ul n rom rlit to (rlit+-).t. rour ontrint r oy. poil twit: ul onlitin no to mk wy or n, my rnomly, lik im nnl I w il, tn ti ul i ulty (i.. iv up on ti ) Sulin loritm ont. W now ul n t rlit, I.., σ(n) = rlit Fix up ul Suor, x, o n mut ul.t. σ(x) >= σ(n)+(n,x)-. p(n,x), otrwi ty r rmov (ul) n put k on worklit. rpt ti om numr o tim until itr u, tn ritr llot il, tn inr St Copn Goltin St Copn Goltin -5 5 St Copn Goltin - 3

14 5-4 Ltur Simplt Exmpl Simplt Exmpl or () { = + = * = *94 } <,> <,> or () { = + = * = *94 } <,> <,> Try II = Rour: Moulo Rour Tl: Wt i IIr? Wt i IIr? St Copn Goltin St Copn Goltin Simplt Exmpl Simplt Exmpl or () { = + = * = *94 } or () { = + = * = *94 } Try II = Try II = Moulo Rour Tl: Moulo Rour Tl: St Copn Goltin St Copn Goltin St Copn Goltin - 4

15 5-4 Ltur Simplt Exmpl Simplt Exmpl or () { = + = * = *94 } or () { = + = * = *94 } Try II = Try II = Moulo Rour Tl: rlit : im() + ly() - = +- = Moulo Rour Tl: rlit? ul? wt nxt? St Copn Goltin St Copn Goltin Simplt Exmpl or () { = + = * = *94 } Try II = Moulo Rour Tl: Lon: lowr oun my not ivl 3 or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Exmpl Prioriti:? <,> <,> <,> <,> <,> <,> <,> <,> <,> St Copn Goltin St Copn Goltin -5 6 St Copn Goltin - 5

16 5-4 Ltur or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Exmpl Prioriti:,,,,,,,, <,> <,> <,> <,> <,> <,> <,> <,> <,> St Copn Goltin -5 6 or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:,,,,,,,, =5 ALU MU intr σ St Copn Goltin -5 6 or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:,,,,, =5 ALU MU intr σ or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:,,,, =5 ALU MU intr σ St Copn Goltin St Copn Goltin St Copn Goltin - 6

17 5-4 Ltur or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:,,,, =5 ALU MU intr σ 3 4 or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:,,,, =5 ALU MU intr σ 3 4 u -> violtion St Copn Goltin St Copn Goltin or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:,,,, =5 ALU MU intr σ or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:,,, =5 ALU MU intr σ u -> violtion St Copn Goltin St Copn Goltin St Copn Goltin - 7

18 5-4 Ltur or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:,, =5 ALU MU intr σ or i:= to N o := := := := := := U[i] : V[i] := : W[i] := := X[i] Prioriti:, =5 ALU MU intr σ St Copn Goltin St Copn Goltin -5 7 Crtin t Loop intr σ Crt t oy rom t ul. Dtrmin wi itrtion n intrution ll into Mrk it our n t lonin to tt itrtion. A Mov to upt ritr Prolo ill in p t innin For mov w will v n intrution in prolo, n w ill in pnnt intrution Epilo ill in p t n = U[]; = X[]; FOR i = to N := U[i+] := X[i+] nop :=? :=? :=? :=? :=? : W[i] := = = : V[i] := St Copn Goltin St Copn Goltin -5 7 St Copn Goltin - 8

19 5-4 Ltur Conitionl Wt out intrnl ontrol trutur, I.., onitionl Tr ppro Sul ot i n u onitionl mov Sul i, tn mk t oy o t onitionl mro op wit pproprit rour vtor Tr ul t loop Wt to tk wy Dpnn nlyi i vry importnt Sotwr piplinin i ool Ritr r ky rour St Copn Goltin St Copn Goltin St Copn Goltin - 9

MERGE-BASED SpMV PERFECT WORKLOAD BALANCE. GUARANTEED. Duane Merrill, NVIDIA Research

MERGE-BASED SpMV PERFECT WORKLOAD BALANCE. GUARANTEED. Dun Mrrill, NVIDIA Rsr SPARSE MATRIX-VECTOR MULTIPLICATION SpMV (Ax = y) -- -- -- -- -- -- -- -- * = 2.0 0.0 2.0 4.0 sprs mtrix A ns vtor x ns vtor