Computer Architecture. Pipelining and Instruction Level Parallelism An Introduction. Outline of This Lecture

Transcription

1 Compute Achitectue Pipelining and nstuction Level Paallelism An ntoduction Adapted fom COD2e by Hennessy & Patteson Slide 1 Outline of This Lectue ntoduction to the Concept of Pipelined Pocesso Pipelined Datapath and Pipelined Contol Pipeline Example: nstuctions nteaction Pipeline Hazads Fowading Stalls ntoduction to nstuction Level Paallelism Supescala, VLW Out-of-ode execution Banch Pediction Futue Chapte 6 - Pipelining Basics Slide 2

2 The Five Stages of Load F: nstuction Fetch Fetch the instuction fom the nstuction Memoy RF/D: Registes Fetch and nstuction Decode EX: Calculate the memoy addess MEM: Read the data fom the Data Memoy WB: Wite the data back to the egiste file Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Load F RF/D EX MEM WB Chapte 6 - Pipelining Basics Slide 3 Key deas Behind Pipelining Analogy Gading the mid tem exams: 6 poblems, six people gading the exam Each peson gades ONE poblem Pass exam to next peson as soon as one finishes he pat Assume each poblem takes 0.15 hou to gade Each individual exam still takes 0.9 hous to gade But with 6 people, all exams can be gaded much quicke: 100 exams: 90 hous, vs. 90 hs x 6 = 540 hous The load instuction has 5 stages: Five independent functional units to wok on each stage Each functional unit is used only once Anothe load can stat as soon as 1st finishes its F stage Each load still takes five cycles to complete The thoughput, howeve, is much highe Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 4

3 Pipelining the Load nstuction Five independent functional units in pipeline ae: nstuction Memoy fo the F stage Registe file s ead pots fo the RF/D stage fo the EX stage Data Memoy fo the MEM stage Registe File s Wite pot (bus W) fo the WB stage 1 instuction entes the pipeline evey cycle Clock 1 instuction comes out of pipeline (completes) evey cycle Effective Cycles pe nstuction (CP) is 1 Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 1st lw F RF/D EX MEM WB 2nd lw F RF/D EX MEM WB 3d lw F RF/D EX MEM WB Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 5 Fou Stages of F: nstuction Fetch Fetch the instuction fom the nstuction Memoy RF/D: Registes Fetch and nstuction Decode EX: opeates on the two egiste opeands WB: Wite the output back to the egiste file Cycle 1 Cycle 2 Cycle 3 Cycle 4 F RF/D EX WB Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 6

4 Pipelining + Load We have a poblem: Two instuctions ty to wite to egiste file at same time! Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Clock F RF/D EX WB Ops! We have a poblem! F RF/D EX WB Load F RF/D EX MEM WB F RF/D EX WB F RF/D EX WB Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 7 mpotant Obsevation A functional unit can be used once pe instuction Each functional unit must be used at same stage fo all instuctions: Load uses Registe File s Wite Pot duing its 5th stage Load F RF/D EX MEM WB uses Registe File s Wite Pot duing its 4th stage F RF/D EX WB Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 8

5 Solution: Delay WB a Cycle Delay s egiste wite by one cycle: instuctions also use Reg File s wite pot at Stage 5 MEM stage is a NOOP stage: nothing is being done F RF/D EX MEM WB Clock Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 F RF/D MEM EX WB F RF/D MEM EX WB Load F RF/D EX MEM WB F RF/D MEM EX WB F RF/D MEM EX WB Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 9 A Pipelined Datapath Clk F RF/D EX MEM WB RegW ExtOp Op Banch PC 1 0 PC+4 A Unit F/D Registe PC+4 mm16 Rs Ra Rb Rt RFile Rt Rw Di Rd D/Ex Registe 0 1 PC+4 mm16 busa busb EX Unit Ex/MEM Registe Zeo Data ME RAM Do WA Di MEM/WB Registe 1 Mux 0 RegDst Sc MemW MemtoReg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 10

6 How About Contol Signals? Contol Signals at Stage N = Func (nst. at Stage N) N = EX, MEM, o WB Example: Contols Signals at EX Stage Func(Load s EX) F RF/D EX MEM WB Op=Add RegW ExtOp=1 Banch PC 1 0 PC+4 A Unit F/D: PC+4 mm16 Rs Ra Rb Rt RFile Rt Rw Di Rd D/Ex Registe 0 1 PC+4 mm16 busa busb EX Unit Ex/MEM: Load s Addess Zeo Data ME RAM Do WA Di MEM/WB Registe 1 Mux 0 RegDst=0 Sc=1 MemW MemtoReg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 11 Pipeline Contol The Main Contol geneates the contol signals duing RF/D Contol signals fo EX (ExtOp, Sc,...) used 1 cycle late Contol signals fo MEM (MemW, Banch) used 2 cycles late Contol signals fo WB (MemtoReg MemW) used 3 cycles late RF/D EX MEM WB F/D Registe Main Contol ExtOp Sc Op RegDst MemW Banch MemtoReg RegW D/Ex Registe ExtOp Sc Op RegDst MemW Banch MemtoReg RegW Ex/MEM Registe MemW Banch MemtoReg RegW MEM/WB Registe MemtoReg RegW Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 12

7 Single Cycle, Multi-Cycle, Pipelined Clk Cycle 1 Cycle 2 Single Cycle mplementation: Load Stoe Waste Clk Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Multiple Cycle mplementation: Load F Reg EX MEM WB Stoe F Reg EX MEM F Pipeline mplementation: Load F Reg EX MEM WB Stoe F Reg EX MEM WB F Reg EX MEM WB Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 13 Hazads Challenge to Pipelining Limits to pipelining: Hazads pevent next instuction fom executing duing its designated clock cycle stuctual hazads: HW cannot suppot this combination of instuctions ealie case of load and R-typ like a stuctual hazad, but nomally cannot fix by etiming instuction. data hazads: instuction depends on esult of pio instuction still in the pipeline contol hazads: pipelining of banches & othe instuctionscommon solution is to stall the late pat of the pipeline until the hazad pipeline Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 14

8 Data Hazad on 1 Dependencies backwads in time ae hazads n s t. O d e Time (clock cycles) F D/RF EX MEM WB add 1,2,3 sub 4,1,3 and 6,1,7 o 8,1,9 xo 10,1,11 m Reg Dm Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 15 HW Stalls to Resolve Hazad Dependencies backwads in time ae hazads eliminate evese time by a stall n s t. O d e Time (clock cycles) F D/RF EX MEM WB add 1,2,3 sub 4, 1,3 and 6,1,7 o 8,1,9 xo 10,1,11 m bubble bubble bubble Reg Dm Reg m Reg Dm m Reg m Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 16

9 nsight: Data is available! n s t. O d e Pipeline egistes aleady contain needed data Fowad the data to the appopiate unit Time (clock cycles) F D/RF EX MEM WB add 1,2,3 sub 4,1,3 and 6,1,7 o 8,1,9 xo 10,1,11 m Reg Dm Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 17 HW fo Fowading (Bypassing) ncease multiplexos to add paths fom egistes Assumes egiste ead duing wite gets new value (othewise moe esults to be fowaded) Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 18

10 Fowading Cannot Hide All Hazads n s t. O d e Time (clock cycles) F D/RF EX MEM WB lw 1, 0(2) sub 4,1,6 and 6,1,7 o 8,1,9 m Reg Dm Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 19 Option: HW Stalls to Resolve Hazad ntelock : checks fo hazad & stalls n s t. O d e Time (clock cycles) F D/RF EX MEM WB lw 1, 0(2) stall sub 4,1,3 and 6,1,7 o 8,1,9 m bubble bubble bubble bubble m Reg Dm Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 20

11 Option: SW esolves hazad n s t. O d e SW insets independent instuctions Wost case: pefomance no bette/wose Time (clock cycles) F D/RF EX MEM WB lw 1, 0(2) unelated instuction sub 4,1,3 and 6,1,7 o 8,1,9 m Reg Dm Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 21 Contol Hazad on Banches Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 22

12 Hazads on Banches Time (clock cycles) F D/RF EX MEM WB beq 1,2,L sub 4,1,3 and 6,2,7 o 8,7,9 L: add 1,2,1 m Reg Dm Reg Stall fo two cycles on evey banch! Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 23 CP mpact: Banch Stall mpact f CP = 1, 30% banch, Stall 2 cycles => new CP = 1.6! Reducing the banch penalty MPS banch aleady moe aggessive than most limited eq/neq allows us to detemine banch condition ealy (afte EX), athe than late (e.g., afte MEM) doing bette use sepaate compaato athe than and move banch decision to RF (had!!!) educes penalty to 1 cycle Going futhe Vaiety of techniques: sepaating banch and destination sepaating banch condition and banch decision hadwae pediction of banche Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 24

13 When is pipelining had? nteupts: 5 instuctions executing in 5 stage pipeline How to stop the pipeline? Restat? Who caused the inteupt? Stage Poblem inteupts occuing F Page fault on instuction fetch; misaligned memoy access; memoy-potection violation D Undefined o illegal opcode EX Aithmetic inteupt MEM Page fault on data fetch; misaligned memoy access; memoy-potection violation Load with data page fault, Add with instuction page fault? Solution 1: inteupt vecto/instuction, estat eveything incomplete Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 25 Fist Geneation RSC Pipelines All instuctions: 1 pipeline ode ( static schedule ). Registe wite in last stage + eads pefomed in fist stage afte issue. Simpliy/eliminate hazads Memoy access in stage 4 Avoid all memoy hazads Contol hazads use delayed banch (with fast path) RAW hazads use bypass, except on load esults Load esolved by delayed load o stall Good pipeline pefomance at little cost/complexity. Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 26

14 Summay of Pipelining Basics Speed Up = Pipeline Depth Hazads limit pefomance on computes: stuctual: need moe HW esouces data: need fowading, compile scheduling contol: ealy evaluation & PC, delayed banch, pediction nceasing length of pipe inceases hazads since pipelining helps instuction bandwidth, not latency Compiles can educe cost of data & contol hazads load delay slots banch delay slots Exceptions (also FP, SA) make pipelining hade Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 27 Advanced Pipelining Pipelining exploits paallelism among instuctions by ovelapping them Called nstuction Level Paallelism (LP) Limited by a vaiety of things: paallelism in the pogam compile technology in exposing paallelism functional unit capability: how many ovlapping instuctions ability of hadwae to find instuctions to un in paallel Exploiting LP is hot topic in pocesso design: Lots of diffeent appoaches Multiple instuctions/cycle compile vs. HW fo scheduling instuctions Both achitectue appoaches and compile appoaches Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 28

15 Technique Pipelining Supe-scala Exploiting Available LP ssue multiple scala instuctions pe cycle VLW Each instuction specifies multiple scala opeations F D Ex M W F D Ex M W F D Ex M W F D Ex M W F D Ex M W F D Ex M W F D Ex M W F D Ex M W F D Ex M W Ex M W Ex M W Ex M W HW Limitation ssue ate, FU stalls, FU depth Hazad esolution Packing Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 29 Easy Supescala -Cache nt Reg nst ssue and Bypass FP Reg nt Unit Load / Stoe Unit FP Add FP Mul D-Cache ssue intege and FP opeations in paallel! potential hazads? expected speedup? what combinations of instuctions make sense? Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 30

16 ssuing Multiple nstuction/ Cycle Supescala: 2 instuctions, 1 FP & 1 anything else Fetch 64-bits/clock cycle; nt on left, FP on ight Can only issue 2nd instuction if 1st instuction issues Moe pots fo FP egistes to do FP load & FP op in a pai Type Pipe Stages nt. instuction F D EX MEM W FP instuction F D EX MEM WB nt. instuction F D EX MEM WB FP instuction F D EX MEM WB nt. instuction F D EX MEM WB FP instuction F D EX MEM WB 1 cycle load delay expands to 3 instuction in SS instuction in ight half can t use esult, no can eithe instuction in next slot Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 31 Dynamic Banch Pediction Pedict diection of banches on past behavio keep a cache of banch behavio, look up pediction Pefomance = f(accuacy, cost of mispediction) Banch pediction buffe: lowe bits of PC addess index table of 1-bit values says whethe o not banch taken last time evaluate actual banch condition, if pediction incoect: ecove by flushing pipeline, estating fetch eset pediction Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 32

17 Speculative Supescala Execution Get all available paallelism acoss banches in face cache misses limited only by data dependences Goal: esouces and available bandwidth ae only HW limit Banch pediction execute instuctions speculatively Hazad detection and aggessive esolution out-of-ode execution (dynamic scheduling) in-ode completion Exception handling easie handles incoect speculation nstuction Fetch Decode nstuction Window Execution Units look ahead and pefetch instuctions ssue multiple instuctions to Execution Units when inputs ae available Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 33 Vaiety of Moden Micopocesso Pocesso nstuction Completion Rate Scheduling of pipeline Banch pediction PowePC Dynamic, nonspeculative HW MPS R Dynamic, speculative HW Pentium 4 Dynamic, nonspeculative HW UltaSPARC 4 Static HW Meced? Static? Static? Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 34

18 Limits to Multi-ssue Machines nheent limitations of LP 1 banch in 5 => 5-way VLW busy? Latencies of units => many opeations must be scheduled Need about Pipeline Depth x No. Functional Units of independentdifficulties in building HW Duplicate FUs to get paallel execution ncease pots to Registe File (3 x intege/fp ate) ncease pots to memoy Decoding challenge and impact on clock ate, pipeline depth Limitations specific to eithe SS o VLW implementation Decode issue in SS VLW code size: unoll loops + wasted fields in VLW VLW lock step => 1 hazad & all instuctions stall VLW & binay compatibility Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 35 Summay nstuction Level Paallelism in SW o HW Loop level paallelism is easiest to see SW dependencies/compile sophistication detemine if compile can unoll loops SW Scheduling HW scheduling Banch Pediction SupeScala and VLW CP < 1 Dynamic issue vs. Static issue Moe instuctions issue/clock, lage penalty of hazads Futue? Stay tuned Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 36

19 Single Memoy=>Stuctual Hazad Time (clock cycles) n s t. O d e Load nst 1 nst 2 nst 3 nst 4 MEM Reg MEM Reg MEM Reg MEM Reg MEM Reg MEM Reg MEM Reg MEM Reg MEM Reg MEM Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 37 Stall to esolve Stuctual Hazad Time (clock cycles) n s t. O d e Load nst 1 nst 2 nst 3(stall) nst 4 MEM Reg MEM Reg MEM Reg MEM Reg MEM Reg MEM Reg bubble MEM Reg MEM Reg MEM Reg MEM Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 38

20 Duplicate to Resolve Hazad Sepaate nstuction Cache (m) & Data Cache (Dm) Time (clock cycles) n s t. O d e Load nst 1 nst 2 nst 3 nst 4 m Reg Dm Reg Adapted fom COD2e by Hennessy & Patteson Chapte 6 - Pipelining Basics Slide 39