EECS150 - Digital Design Lecture 23 - High-level Design and Optimization 3, Parallelism and Pipelining

Transcription

1 EECS150 - Digitl Design Lecture 23 - High-level Design nd Optimiztion 3, Prllelism nd Pipelining Nov 12, 2002 John Wwrzynek Fll 2002 EECS150 - Lec23-HL3 Pge 1 Prllelism Prllelism is the ct of doing more thn one thing t time. Optimiztion in hrdwre design often involves using prllelism to trde etween cost nd performnce. Emple, Student finl grde clcultion: red mt1, ht2, mt3, project; grde = 0.2 mt1 0.2 mt2 0.2 mt3 0.4 project; write grde; High performnce hrdwre implementtion: 0.2 mt1 0.2 mt2 0.2 mt3 0.4 proj As mny opertions s possile re done in prllel. Fll 2002 EECS150 - Lec23-HL3 Pge 2 grde

2 Prllelism Is there lower cost hrdwre implementtion? Different tree orgniztion? Cn fctor out multiply y 0.2: mt1 mt2 0.4 proj mt3 How out shring opertors (multipliers nd dders)? 0.2 grde Fll 2002 EECS150 - Lec23-HL3 Pge 3 Time multiple single ALU for ll dds nd multiplies: Attempts to minimize cost t the epense of time. Need to dd etr register, mues, control. Prllelism controller cc1 = mt1 mt2; cc1 = cc1 mt3; cc1 = 0.2 cc1; cc2 = 0.4 proj; grde = cc1 cc2; mt1 mt1 mt3 proj ALU cc1 cc2 If we dopt this pproch, we cn then consider the comintionl hrdwre circuit digrm s n strct computtion-grph. 0.2 mt1 0.2 mt2 0.2 mt3 0.4 proj grde Using other primitives, other coverings re possile. Fll 2002 EECS150 - Lec23-HL3 Pge 4 A B C A B C D This technique covers the computtion grph y performing the ction of ech node one t time. (Sort of emultes it.)

3 HW versus SW This time-multipleed ALU pproch is very similr to wht conventionl softwre version would ccomplish: CPUs time-multiple function units (ALUs, etc.) dd r2,r1,r3 dd r2,r2,r4 mult r2,r4,r5. This model mtches our tendency to epress computtion sequentilly - even though mny nturlly contin prllelism. Our progrmming lnguges lso strengthen this tendency. In hrdwre we hve the ility to eploit prolem prllelism - gives us kno on performnce/cost. Mye est to epress computtions s strct computtions grphs (rther thn progrms ) - should led to wider rnge of implementtions. Note: modern processors spend much of their cost udget ttempting to restore eecution prllelism: super-sclr eecution. Fll 2002 EECS150 - Lec23-HL3 Pge 5 Optimizing Itertive Computtions Hrdwre implementtions of computtions lmost lwys involves looping. Why? Is this true with softwre? Are there progrms without loops? Mye in through wy code. We proly would not other uilding such thing into hrdwre, would we? (FPGA my chnge this.) Fct is, our computtions re closely tied to loops. Almost ll our HW includes some looping mechnism. Wht do we use looping for? Fll 2002 EECS150 - Lec23-HL3 Pge 6

4 Optimizing Itertive Computtions Types of loops: 1) Looping over input dt (streming): e: MP3 plyer, video compressor, music synthesizer. 2) Looping over memory dt e: vector inner product, mtri multiply, list-processing These two re relly very similr. 1) is often turned into 2) y uffering up input dt, nd processing offline. Even for online processing, uffers re used to smooth out temporry rte mismtches. 3) CPUs re one ig loop. Instruction fetch eecute Instruction fetch eecute ut chnge their personlity with ech itertion. 4) Others? Loops offer opportunity for prllelism y eecuting more thn one itertion t once, through prllel itertion eecution &/or pipelining Fll 2002 EECS150 - Lec23-HL3 Pge 7 Pipelining With looping usully we re less interested in the ltency of one itertion nd more in the loop eecution rte, or throughput. These cn e different due to prllel itertion eecution &/or pipelining. Pipelining review from CS61C: Anlog to wshing clothes: step 1: wsh (20 minutes) step 2: dry step 3: fold (20 minutes) (20 minutes) 60 minutes 4 lods 4 hours wsh dry fold 20 min overlpped 2 hours Fll 2002 EECS150 - Lec23-HL3 Pge 8

5 Pipelining wsh dry fold In the limit, s we increse the numer of lods, the verge time per lod pproches 20 minutes. The ltency (time from strt to end) for one lod = 60 min. The throughput = 3 lods/hour The pipelined throughput # of pipe stges un-pipelined throughput. Fll 2002 EECS150 - Lec23-HL3 Pge 9 Generl principle: T Pipelining IN CL Cut the CL lock into pieces (stges) nd seprte with registers: T' OUT Assume T=8ns T FF (setup clk q)=1ns F = 1/9ns = 111MHz IN CL1 CL2 OUT T1 T2 T = 4ns 1ns 4ns 1ns = 10ns F = 1/(4ns 1ns) = 200MHz Assume T1 = T2 = 4ns CL lock produces new result every 5ns insted of every 9ns. Fll 2002 EECS150 - Lec23-HL3 Pge 10

6 Limits on Pipelining Without FF overhed, throughput improvement α # of stges. After mny stges re dded. FF overhed egins to dominte: 500 throughput (1/T) idel rel hlf the clock period in FF overhed FF overhed is the setup nd clk to Q times # of stges Other limiters: clock skew contriutes to clock overhed unequl stges FFs dominte cost clock distriution power consumption feedck (dependencies etween loop itertions) Fll 2002 EECS150 - Lec23-HL3 Pge 11 Emple F() = = i2 i c F() y Computtion grph: nd y re ssumed to e strems Divide into 3 (nerly) equl stges. Insert pipeline registers t dshed lines. c Cn we pipeline sic opertors? y Fll 2002 EECS150 - Lec23-HL3 Pge 12

7 Emple: Pipelined Adder c0 s3 3 3 s2 2 2 s1 s reg reg FF c0 FF FF s3 s2 s1 s0 Fll 2002 EECS150 - Lec23-HL3 Pge 13 Pipelining Loops with Feedck Loop crry dependency Emple 1: = i i unpipelined version: dd 1 i i1 dd 2 1 Cn we cut the feedck nd Cn t overlp the overlp itertions? i itertions ecuse of the dependency. The etr register Try putting register fter dd1: doesn t help the sitution (ctully dd 1 i i1 hurts). dd 2 1 y In generl, cn t i pipeline feedck loops. Fll 2002 EECS150 - Lec23-HL3 Pge 14

8 Pipelining Loops with Feedck Loop crry dependency However, we cn overlp the nonfeedck prt of the itertions: i Add is ssocitive nd communitive. Therefore we cn reorder the computtion to shorten the dely of the feedck pth: i = ( i ) = ( i ) Shorten the feedck pth. dd 1 i i1 i2 dd y y y 2 i i1 i2 Pipelining is limited to 2 stges. Fll 2002 EECS150 - Lec23-HL3 Pge 15 Pipelining Loops with Feedck Emple 2: = i i Reorder to shorten the feedck loop nd try putting register fter multiply: i dd 1 i i1 i2 mult 1 dd Still need 2 cycles/itertion Fll 2002 EECS150 - Lec23-HL3 Pge 16

9 Pipelining Loops with Feedck Emple 2: = i i Once gin, dding register doesn t help. Best solution is to overlp non-feedck prt with feedck prt. Therefore criticl pth includes multipln series with dd. Cn overlp first dd with multiply/dd opertion. Only 1 cycle/itertion. Higher performnce solution. dd 1 i i1 i2 mult 1 dd Alterntive is to move flip-flop to fter multiple, ut sme criticl pth. Fll 2002 EECS150 - Lec23-HL3 Pge 17 C-slow Technique Another pproch to incresing throughput in the presence of feedck: try to fill in holes in the chrt with nother (independent) computtion: dd 1 i i1 i2 mult 1 dd If we hve second similr computtion, cn interleve it with the first: 1 F 1 y 1 = 1 y 1 i-1 1 i 1 2 F 2 y 2 = 2 y 2 i-1 2 i 2 Use mues to direct ech strem. Ech produces 1 result / 2 cycles. Here the feedck depth=2 cycles (we sy C=2). Ech loop hs throughput of F/C. But the ggregte throughput is F. With this technique we could pipeline even deeper, ssuming we could supply C independent strems. Fll 2002 EECS150 - Lec23-HL3 Pge 18

10 C-slow Technique Essentilly this mens we go hed nd cut feedck pth: This mkes opertions in djcent pipeline stges independent nd llows full cycle for ech: i C computtions (in this cse C=2) cn use the pipeline simultneously. Must e independent. Input MUX interleves input strems. Ech strem runs t hve the pipeline frequency. Pipeline chieves full throughput. dd 1 mult y y y y y y dd 2 y y y y y y Fll 2002 EECS150 - Lec23-HL3 Pge 19 Beyond Pipelining - SIMD Prllelism An ovious wy to eploit more prllelism from loops is to mke multiple instnces of the loop eecution dt-pth nd run them in prllel shring the some controller. For P instnces, throughput improves y fctor of P. emple: = f( i ) i f i1 f i2 f i3 f Usully clled SIMD prlllism. Single Instruction Multiple Dt 1 Assumes the net 4 vlues ville t once. The vlidity of this ssumption depends on the rtio of f repet rte to input rte (or memory ndwidth). Cost α P. Usully, much higher thn for pipelining. However, potentilly provides high speedup. Often pplied fter pipelining. Limited, once gin, y loop crry dependencies. Feedck trnsltes to dependencies etween prllel dt-pths. 2 3 Fll 2002 EECS150 - Lec23-HL3 Pge 20

11 SIMD Prllelism with Feedck Emple, from erlier: = i i i1 i2 In this emple end up with crry ripple sitution. Could employ look-hed / prllel-prefi optimiztion techniques to speed up propgtion. As with pipelining, this technique is most effective in the sence of loop crry dependence. Fll 2002 EECS150 - Lec23-HL3 Pge