Datapath Background. This Unit: (Scalar In-Order) Pipelining. CIS 501 Computer Architecture. Readings

Transcription

1 This Unit: (clr In-rer) Pipelining CI 501 Computer rchitecture Unit 6: Pipelining pp pp pp ystem softwre CPU I/ Principles of pipelining Effects of overhe n hzrs Pipeline igrms hzrs tlling n bypssing Control hzrs rnch preiction Preiction lies originlly evelope by mir Roth with contributions by Milo Mrtin t University of Pennsylvni with sources tht inclue University of Wisconsin slies by Mrk Hill, Guri ohi, Jim mith, n Dvi Woo. CI 501 (Mrtin/Roth): Pipelining 1 CI 501 (Mrtin/Roth): Pipelining 2 Reings H&P ppenix pth ckgroun CI 501 (Mrtin/Roth): Pipelining 3 CI 501 (Mrtin/Roth): Technology

2 pth n Control ingle-cycle pth I$ D$ I$ D$ control pth: implements execute portion of fetch/exec. loop Functionl units (LUs), registers, memory interfce Control: implements ecoe portion of fetch/execute loop Mux selectors, write enble signls regulte flow of t in tpth Prt of ecoe involves trnslting insn opcoe into control signls CI 501 (Mrtin/Roth): Pipelining 5 ingle-cycle tpth: true tomic fetch/execute loop Fetch, ecoe, execute one complete instruction every cycle Hrwire control : opcoe to control signls RM Low CPI: 1 by efinition Long clock perio: to ccommote slowest instruction CI 501 (Mrtin/Roth): Pipelining 6 Multi-Cycle pth I$ Multi-cycle tpth: ttcks slow clock Fetch, ecoe, execute one complete insn over multiple cycles Micro-coe control: stges control signls llows insns to tke ifferent number of cycles (min point) ± pposite of single-cycle: short clock perio, high CPI (think: CIC) CI 501 (Mrtin/Roth): Pipelining 7 D$ D ingle-cycle vs. Multi-cycle Performnce ingle-cycle Clock perio = 50ns, CPI = 1 Performnce = 50ns/insn Multi-cycle hs opposite performnce split of single-cycle horter clock perio Higher CPI Multi-cycle rnch: 20% (3 cycles), lo: 20% (5 cycles), LU: 60% ( cycles) Clock perio = 11ns, CPI = (20%*3)(20%*5)(60%*) = Why is clock perio 11ns n not 10ns? Performnce = ns/insn sie: CIC mkes perfect sense in multi-cycle tpth CI 501 (Mrtin/Roth): Pipelining 8

3 Ltency versus Throughput insn0.fetch, ec, exec ingle-cycle insn0.fetch insn0.ec Multi-cycle insn1.fetch, ec, exec insn0.exec insn1.fetch insn1.ec Cn we hve both low CPI n short clock perio? Not if tpth executes only one insn t time insn1.exec Pipelining sics Ltency vs. Throughput Ltency: no goo wy to mke single insn go fster Throughput: fortuntely, no one cres bout single insn ltency Gol is to mke progrms, not iniviul insns, go fster Progrms contin billions of insns Key: exploit inter-insn prllelism CI 501 (Mrtin/Roth): Technology 9 CI 501 (Mrtin/Roth): Pipelining 10 Pipelining insn0.fetch Multi-cycle Importnt performnce technique Improves instruction throughput rther instruction ltency egin with multi-cycle esign When insn vnces from stge 1 to 2, next insn enters t stge 1 Form of prllelism: insn-stge prllelism Mintins illusion of sequentil fetch/execute loop Iniviul instruction tkes the sme number of stges ut instructions enter n leve t much fster rte Lunry nlogy insn0.ec insn0.exec insn1.fetch insn0.fetch insn0.ec insn0.exec Pipeline insn1.fetch insn1.ec insn1.exec insn1.ec insn1.exec CI 371 (Roth/Mrtin): Pipelining 11 Five tge Pipeline pth Temporry vlues (,,,,,D) re-ltche every stge Why? 5 insns my be in pipeline t once with ifferent s Notice, not ltche fter LU stge (not neee lter) Pipeline control: one single-cycle controller Control signls themselves pipeline CI 371 (Roth/Mrtin): Pipelining 12 D

4 Five tge Pipeline Performnce Pipeline Terminology T insn-mem T regfile T LU T t-mem T regfile T singlecycle Pipelining: cut tpth into N stges (here five) ne insn in ech stge in ech cycle Clock perio = MX(T insn-mem, T regfile, T LU, T t-mem ) se CPI = 1: insn enters n leves every cycle ctul CPI > 1: pipeline must often stll Iniviul insn ltency increses (pipeline overhe), not the point CI 371 (Roth/Mrtin): Pipelining 13 Five stge: Fetch, Decoe, execute, ory, Writebck Nothing mgicl bout 5 stges (Pentium h 22 stges!) Ltches (pipeline registers) nme by stges they seprte, F/D, D/X, X/M, M/W CI 371 (Roth/Mrtin): Pipelining 1 F/D D/X X/M M/W D More Terminology & Foreshowing clr pipeline: one insn per stge per cycle lterntive: supersclr (lter) In-orer pipeline: insns enter execute stge in orer lterntive: out-of-orer (lter) Pipeline epth: number of pipeline stges Nothing mgicl bout five Tren hs been to eeper pipelines (gin, more lter) Instruction Convention Different Is use inconsistent register orers ome Is (for exmple MIP) Instruction estintion (i.e., output) on the left $1, $2, $3 mens $1!$2$3 ther Is Instruction estintion (i.e., output) on the right r1,r2,r3 mens r1r2!r3 l 0(r5),r mens mem[r58]!r st r,0(r5) mens r!mem[r58] CI 501 (Mrtin/Roth): Pipelining 15 Will try to specify to voi confusion, next slies MIP style CI 501 (Mrtin/Roth): Pipelining 16

5 Pipeline Exmple: Cycle 1 Pipeline Exmple: Cycle 2 << 2 << 2 D D $3,$2,$1 lw $,0($5) $3,$2,$1 3 instructions CI 501 (Mrtin/Roth): Pipelining 17 CI 501 (Mrtin/Roth): Pipelining 18 Pipeline Exmple: Cycle 3 Pipeline Exmple: Cycle << 2 << 2 D D sw $6,($7) lw $,0($5) $3,$2,$1 sw $6,($7) lw $,0($5) $3,$2,$1 3 instructions CI 501 (Mrtin/Roth): Pipelining 19 CI 501 (Mrtin/Roth): Pipelining 20

6 Pipeline Exmple: Cycle 5 Pipeline Exmple: Cycle 6 << 2 << 2 D D sw $6,($7) lw $,0($5) sw $6,(7) lw CI 501 (Mrtin/Roth): Pipelining 21 CI 501 (Mrtin/Roth): Pipelining 22 Pipeline Exmple: Cycle 7 Pipeline Digrm D << 2 Pipeline igrm: shorthn for wht we just sw cross: cycles Down: insns Convention: X mens lw $,0($5) finishes execute stge n writes into X/M ltch t en of cycle $3,$2,$1 F D X M W sw lw $,0($5) F D X M W sw $6,($7) F D X M W CI 501 (Mrtin/Roth): Pipelining 23 CI 501 (Mrtin/Roth): Pipelining 2

7 Exmple Pipeline Perf. Clcultion ingle-cycle Clock perio = 50ns, CPI = 1 Performnce = 50ns/insn Multi-cycle rnch: 20% (3 cycles), lo: 20% (5 cycles), LU: 60% ( cycles) Clock perio = 11ns, CPI = (20%*3)(20%*5)(60%*) = Performnce = ns/insn 5-stge pipeline Clock perio = 12ns pprox. (50ns / 5 stges) overhes CPI = 1 (ech insn tkes 5 cycles, but 1 completes ech cycle) Performnce = 12ns/insn Well ctully CPI = 1 some penlty for pipelining (next) CPI = 1.5 (on verge insn completes every 1.5 cycles) Performnce = 18ns/insn Much higher performnce thn single-cycle or multi-cycle CI 501 (Mrtin/Roth): Pipelining 25 Q1: Why Is Pipeline Clock Perio > (ely thru tpth) / (number of pipeline stges)? few resons: Ltches ely Extr bypssing logic s ely Pipeline stges hve ifferent elys, clock perio is mx ely These fctors hve implictions for iel number pipeline stges Diminishing clock frequency gins for longer (eeper) pipelines CI 501 (Mrtin/Roth): Pipelining 26 Q2: Why Is Pipeline CPI > 1? CPI for sclr in-orer pipeline is 1 stll penlties tlls use to resolve hzrs Hzr: conition tht jeoprizes sequentil illusion tll: pipeline ely introuce to restore sequentil illusion Clculting pipeline CPI Frequency of stll * stll cycles Penlties (stlls generlly on t overlp in in-orer pipelines) 1 stll-freq 1 *stll-cyc 1 stll-freq 2 *stll-cyc 2 Correctness/performnce/mke common cse fst (MCCF) Long penlties K if they hppen rrely, e.g., * 10 = 1.1 tlls lso hve implictions for iel number of pipeline stges Depenences, Pipeline Hzrs, n ypssing CI 501 (Mrtin/Roth): Pipelining 27 CI 501 (Mrtin/Roth): Technology 28

8 Depenences n Hzrs Depenence: reltionship between two insns : two insns use sme storge loction Control: one insn ffects whether nother executes t ll Not b thing, progrms woul be boring without them Enforce by mking oler insn go before younger one Hppens nturlly in single-/multi-cycle esigns ut not in pipeline Hzr: epenence & possibility of wrong insn orer Effects of wrong insn orer cnnot be externlly visible tll: for orer by keeping younger insn in sme stge Hzrs re b thing: stlls reuce performnce CI 501 (Mrtin/Roth): Pipelining 29 Why Does Every Tke 5 Cycles? D $3,$2,$1 lw $,0($5) Coul/shoul we llow to skip M n go to W? No It wouln t help: pek fetch still only 1 insn per cycle tructurl hzrs: imgine follows lw CI 501 (Mrtin/Roth): Pipelining 30 << 2 tructurl Hzrs tructurl hzrs Two insns trying to use sme circuit t sme time E.g., structurl hzr on register file write port To fix structurl hzrs: proper I/pipeline esign Ech insn uses every structure exctly once For t most one cycle lwys t sme stge reltive to F (fetch) Tolerte structure hzrs stll logic to stll pipeline when hzrs occur Exmple tructurl Hzr l r2,0(r1) F D X M W r1,r3,r F D X M W sub r1,r3,r5 F D X M W st r6,0(r1) F D X M W tructurl hzr: resource neee twice in one cycle Exmple: unifie instruction & t memories (cches) olutions: eprte instruction/t memories (cches) Reesign cche to llow 2 ccesses per cycle (slow, expensive) tll pipeline CI 501 (Mrtin/Roth): Pipelining 31 CI 501 (Mrtin/Roth): Pipelining 32

9 Hzrs F/D D/X X X/M sw $6,0($7) lw $,0($5) Let s forget bout brnches n the control for while The three insn sequence we sw erlier execute fine ut it wsn t rel progrm Rel progrms hve t epenences They pss vlues vi registers n memory M/W $3,$2,$1 D Depenent pertions Inepenent opertions $3,$2,$1 $6,$5,$ Woul this progrm execute correctly on pipeline? $3,$2,$1 $6,$5,$3 Wht bout this progrm? $3,$2,$1 lw $,0($3) i $6,1,$3 sw $3,0($7) CI 501 (Mrtin/Roth): Pipelining 33 CI 501 (Mrtin/Roth): Pipelining 3 Hzrs ory Hzrs F/D D/X X X/M D M/W F/D D/X X X/M D M/W sw $3,0($7) i $6,1,$3 lw $,0($3) $3,$2,$1 Woul this progrm execute correctly on this pipeline? Which insns woul execute with correct inputs? is writing its result into $3 in current cycle lw re $3 two cycles go! got wrong vlue i re $3 one cycle go! got wrong vlue sw is reing $3 this cycle! mybe (epening on regfile esign) CI 501 (Mrtin/Roth): Pipelining 35 lw $,0($1) sw $5,0($1) re memory t hzrs problem for this pipeline? No lw following sw to sme ress in next cycle, gets right vlue Why? mem re/write lwys tke plce in sme stge hzrs through registers? Yes (previous slie) ccur becuse register write is three stges fter register re Cn only re register vlue three cycles fter writing it CI 501 (Mrtin/Roth): Pipelining 36

10 bservtion! F/D D/X X X/M lw $,0($3) Techniclly, this sitution is broken lw $,0($3) hs lrey re $3 from regfile $3,$2,$1 hsn t yet written $3 to regfile ut funmentlly, everything is K lw $,0($3) hsn t ctully use $3 yet $3,$2,$1 hs lrey compute $3 CI 501 (Mrtin/Roth): Pipelining 37 M/W $3,$2,$1 D Reucing Hzrs: ypssing F/D D/X X X/M lw $,0($3) ypssing Reing vlue from n intermeite (µrchitecturl) source Not witing until it is vilble from primry source Here, we re bypssing the register file lso clle forwring CI 501 (Mrtin/Roth): Pipelining 38 M/W $3,$2,$1 D WX ypssing LUin ypssing F/D D/X X X/M D M/W F/D D/X X X/M D M/W lw $,0($3) $3,$2,$1 $,$2,$3 $3,$2,$1 Wht bout this combintion? nother bypss pth n MUX (multiplexor) input First one ws n MX bypss This one is WX bypss Cn lso bypss to LU input CI 501 (Mrtin/Roth): Pipelining 39 CI 501 (Mrtin/Roth): Pipelining 0

11 WM ypssing? ypss Logic D F/D D/X X X/M D M/W sw $3,0($) lw $3,0($2) Does WM bypssing mke sense? Not to the ress input (why not?) ut to the store t input, yes CI 501 (Mrtin/Roth): Pipelining 1 bypss Ech MUX hs its own, here it is for MUX LUin (D/X..Regource1 == X/M..RegDest) => 0 (D/X..Regource1 == M/W..RegDest) => 1 Else => 2 CI 501 (Mrtin/Roth): Pipelining 2 Pipeline Digrms with ypssing If bypss exists, from / to stges execute in sme cycle Exmple: full bypssing, use MX bypss r2,r3!r1 F D X M W sub r1,r!r2 F D X M W Exmple: full bypssing, use WX bypss r2,r3!r1 F D X M W l [r7]!r5 F D X M W sub r1,r!r2 F D X M W Exmple: WM bypss r2,r3!r1 F D X M W? F D X M W Cn you think of coe exmple tht uses the WM bypss? CI 501 (Mrtin/Roth): Pipelining 3 Hve We Prevente ll Hzrs? D stll nop $,$2,$3 CI 501 (Mrtin/Roth): Pipelining lw $3,($2) No. Consier lo followe by epenent insn ypssing lone isn t sufficient! Hrwre solution: etect this sitution n inject stll cycle oftwre solution: ensure compiler oesn t generte such coe

12 tlling to voi Hzrs F/D D/X X X/M hzr nop Prevent F/D insn from reing (vncing) this cycle Write nop into D/X. (effectively, insert nop in hrwre) lso reset (cler) the tpth control signls Disble F/D ltch n write enbles (why?) Re-evlute sitution next cycle CI 501 (Mrtin/Roth): Pipelining 5 M/W D tlling on Lo-To-Use Depenences D stll nop $,$2,$3 lw $3,($2) tll = (D/X..pertion == LD) && ((F/D..Regrc1 == D/X..RegDest) ((F/D..Regrc2 == D/X..RegDest) && (F/D..P!= TRE)) CI 501 (Mrtin/Roth): Pipelining 6 tlling on Lo-To-Use Depenences tlling on Lo-To-Use Depenences D nop D nop stll $,$2,$3 (stll bubble) lw $3,($2) tll = (D/X..pertion == LD) && ((F/D..Regrc1 == D/X..RegDest) ((F/D..Regrc2 == D/X..RegDest) && (F/D..P!= TRE)) CI 501 (Mrtin/Roth): Pipelining 7 stll $,$2,$3 (stll bubble) lw $3, tll = (D/X..pertion == LD) && ((F/D..Regrc1 == D/X..RegDest) ((F/D..Regrc2 == D/X..RegDest) && (F/D..P!= TRE)) CI 501 (Mrtin/Roth): Pipelining 8

13 Performnce Impct of Lo/Use Penlty ssume rnch: 20%, lo: 20%, store: 10%, other: 50% 50% of los re followe by epenent instruction require 1 cycle stll (I.e., insertion of 1 nop) Clculte CPI CPI = 1 (1 * 20% * 50%) = 1.1 Reucing Lo-Use tll Frequency $3,$2,$1 F D X M W lw $,($3) F D X M W i $6,$,1 F * D X M W sub $8,$3,$1 F D X M W Use compiler scheuling to reuce lo-use stll frequency More on compiler scheuling lter $3,$2,$1 F D X M W lw $,($3) F D X M W sub $8,$3,$1 F D X M W i $6,$,1 F D X M W CI 501 (Mrtin/Roth): Pipelining 9 CI 371 (Roth/Mrtin): Pipelining 50 Pipelining n Multi-Cycle pertions Pipeline Multiplier F/D D/X X/M D F/D D/X X/M D Wht if you wnte to multi-cycle opertion? E.g., -cycle multiply P/W: seprte output ltch connects to W stge Controlle by pipeline control finite stte mchine (FM) CI 501 (Mrtin/Roth): Pipelining 51 X Xctrl P P/W Multiplier itself is often pipeline, wht oes this men? Prouct/multiplicn register/lus/ltches replicte Cn strt ifferent multiply opertions in consecutive cycles CI 501 (Mrtin/Roth): Pipelining 52 P M P0/P1 P M P1/P2 P M P2/P3 P M P3/W

14 Pipeline Digrm with Multiplier mul $,$3,$5 F D P0 P1 P2 P3 W i $6,$,1 F D * * * X M W Wht bout Two instructions trying to write regfile in sme cycle? tructurl hzr! Must prevent: mul $,$3,$5 F D P0 P1 P2 P3 W i $6,$1,1 F D X M W $5,$6,$10 F D X M W More Multiplier Nsties Wht bout Mis-orere writes to the sme register oftwre thinks gets $ from i, ctully gets it from mul mul $,$3,$5 F D P0 P1 P2 P3 W i $,$1,1 F D X M W $10,$,$6 F D X M W Common? Not for -cycle multiply with 5-stge pipeline More common with eeper pipelines In ny cse, must be correct CI 501 (Mrtin/Roth): Pipelining 53 CI 501 (Mrtin/Roth): Pipelining 5 Correcte Pipeline Digrm With the correct stll logic Prevent mis-orere writes to the sme register Why two cycles of ely? mul $,$3,$5 F D P0 P1 P2 P3 W i $,$1,1 F * * D X M W $10,$,$6 F D X M W Multi-cycle opertions complicte pipeline logic CI 371 (Roth/Mrtin): Pipelining 55 Pipeline Functionl Units lmost ll multi-cycle functionl units re pipeline Ech opertion tkes N cycles ut cn strt initite new (inepenent) opertion every cycle Requires internl ltching n some hrwre repliction cheper wy to bnwith thn multiple non-pipeline units mulf f0,f1,f2 F D E* E* E* E* W mulf f3,f,f5 F D E* E* E* E* W ne exception: int/fp ivie: ifficult to pipeline n not worth it ivf f0,f1,f2 F D E/ E/ E/ E/ W ivf f3,f,f5 F D s* s* s* E/ E/ E/ E/ W s* = structurl hzr, two insns nee sme structure Is n pipelines esigne to hve few of these Cnonicl exmple: ll insns force to go through M stge CI 501 (Mrtin/Roth): Pipelining 56

15 Wht bout rnches? F/D D/X X << 2 X/M Control Depenences n rnch Preiction Control hzrs options Coul just stll to wit for brnch outcome (two-cycle penlty) Fetch pst brnch insns before brnch outcome is known Defult: ssume not-tken (t fetch, cn t tell it s brnch) CI 501 (Mrtin/Roth): Technology 57 CI 501 (Mrtin/Roth): Pipelining 58 rnch Recovery nop F/D nop D/X rnch recovery: wht to o when brnch is ctully tken s tht will be written into F/D n D/X re wrong Flush them, i.e., replce them with nops They hven t h written permnent stte yet (regfile, D) Two cycle penlty for tken brnches CI 501 (Mrtin/Roth): Pipelining 59 X << 2 X/M rnch Performnce ck of the envelope clcultion rnch: 20%, lo: 20%, store: 10%, other: 50% y, 75% of brnches re tken CPI = 1 20% * 75% * 2 = * 0.75 * 2 = 1.3 rnches cuse 30% slowown Even worse with eeper pipelines How o we reuce this penlty? CI 501 (Mrtin/Roth): Pipelining 60

16 ig Ie: pecultive Execution pecultion: risky trnsctions on chnce of profit pecultive execution Execute before ll prmeters known with certinty Correct specultion voi stll, improve performnce Incorrect specultion (mis-specultion) Must bort/flush/sqush incorrect insns Must uno incorrect chnges (recover pre-specultion stte) The gme : [% correct * gin] [(1 % correct ) * penlty] Control specultion: specultion ime t control hzrs Unknown prmeter: re these the correct insns to execute next? CI 501 (Mrtin/Roth): Pipelining 61 Control pecultion n Recovery i r1,1!r3 Correct: F D X M W bnez r3,trg F D X M W st r6![r7] F D X M W trg: r,r5!r F D X M W specultive Mis-specultion recovery: wht to o on wrong guess Not too pinful in n in-orer pipeline rnch resolves in X Younger insns (in F, D) hven t chnge permnent stte Flush insns currently in F/D n D/X (i.e., replce with nops) Recovery: i r1,1!r3 F D X M W bnez r3,trg F D X M W st r6![r7] F D trg: r,r5!r F trg: r,r5!r F D X M W CI 501 (Mrtin/Roth): Pipelining 62 Reucing Penlty: Fst rnches Fst brnch: trgets control-hzr penlty siclly, brnch insns tht cn resolve t D, not X Test must be comprison to zero or equlity, no time for LU New tken brnch penlty is 1 itionl comprison insns (e.g., cmplt, slt) for complex tests Must bypss into ecoe stge now, too bnez r3,trg F D X M W trg: r,r5,r F D X M W CI 501 (Mrtin/Roth): Pipelining 63 Fst rnch Performnce ssume: rnch: 20%, 75% of brnches re tken CPI = 1 20% * 75% * 1 = *0.75*1 = % slowown (better thn the 30% from before) ut wit, fst brnches ssume only simple comprisons Fine for MIP ut not fine for Is with brnch if $1 > $2 opertions In such cses, sy 25% of brnches require n extr insn CPI = 1 (20% * 75% * 1) 20%*25%*1(extr insn) = 1.2 Exmple of I n micro-rchitecture interction Type of brnch instructions nother option: Delye brnch or brnch ely slot Wht bout conition coes? CI 501 (Mrtin/Roth): Pipelining 6

17 Fewer Mispreictions: rnch Preiction P nop TG F/D D/X X/M X Dynmic brnch preiction: hrwre guesses outcome trt fetching from guesse ress Flush on mis-preiction nop CI 501 (Mrtin/Roth): Pipelining 65 TG <> << 2 rnch Preiction Performnce Prmeters rnch: 20%, lo: 20%, store: 10%, other: 50% 75% of brnches re tken Dynmic brnch preiction rnches preicte with 95% ccurcy CPI = 1 20% * 5% * 2 = 1.02 CI 501 (Mrtin/Roth): Pipelining 66 Dynmic rnch Preiction Components I$ P regfile tep #1: is it brnch? Esy fter ecoe... tep #2: is the brnch tken or not tken? Direction preictor (pplies to conitionl brnches only) Preicts tken/not-tken tep #3: if the brnch is tken, where oes it go? Esy fter ecoe CI 501 (Mrtin/Roth): Pipelining 67 D$ rnch Direction Preiction Lern from pst, preict the future Recor the pst in hrwre structure Direction preictor (DP) Mp conitionl-brnch to tken/not-tken (T/N) ecision Iniviul conitionl brnches often bise or wekly bise 90% one wy or the other consiere bise Why? Loop bck eges, checking for uncommon conitions rnch history tble (HT): simplest preictor inexes tble of bits (0 = N, 1 = T), no tgs Essentilly: brnch will go sme wy it went lst time [31:10] [9:2] 1:0 Wht bout lising? Two with the sme lower bits? No problem, just preiction! HT T or NT T or NT Preiction (tken or CI 501 (Mrtin/Roth): Pipelining not tken) 68

18 rnch History Tble (HT) rnch history tble (HT): simplest irection preictor inexes tble of bits (0 = N, 1 = T), no tgs Essentilly: brnch will go sme wy it went lst time Problem: consier inner loop brnch below (* = mis-preiction) for (i=0;i<100;i) for (j=0;j<3;j) // whtever tte/preiction N* T T T* N* T T T* N* T T T* utcome T T T N T T T N T T T N Two built-in mis-preictions per inner loop itertion rnch preictor chnges its min too quickly Two-it turting Counters (2bc) Two-bit sturting counters (2bc) [mith] Replce ech single-bit preiction (0,1,2,3) = (N,n,t,T) s hysteresis Force preictor to mis-preict twice before chnging its min tte/preiction N* n* t T* t T T T* t T T T* utcome T T T N T T T N T T T N ne mispreict ech loop execution (rther thn two) Fixes this pthology (which is not contrive, by the wy) Cn we o even better? CI 501 (Mrtin/Roth): Pipelining 69 CI 501 (Mrtin/Roth): Pipelining 70 Correlte Preictor Correlte (two-level) preictor [Ptt] Exploits observtion tht brnch outcomes re correlte Mintins seprte preiction per (, HR) rnch history register (HR): recent brnch outcomes imple working exmple: ssume progrm hs one brnch HT: one 1-bit DP entry HT2HR: 2 2 = 1-bit DP entries tte/preiction HR=NN N* T T T T T T T T T T T ctive pttern HR=NT N N* T T T T T T T T T T HR=TN N N N N N* T T T T T T T HR=TT N N N* T* N N N* T* N N N* T* utcome N N T T T N T T T N T T T N We in t mke nything better, wht s the problem? CI 501 (Mrtin/Roth): Pipelining 71 Correlte Preictor Wht hppene? HR wsn t long enough to cpture the pttern Try gin: HT3HR: 2 3 = 8 1-bit DP entries tte/preiction HR=NNN N* T T T T T T T T T T T HR=NNT N N* T T T T T T T T T T HR=NTN N N N N N N N N N N N N ctive pttern HR=NTT N N N* T T T T T T T T T HR=TNN N N N N N N N N N N N N HR=TNT N N N N N N* T T T T T T HR=TTN N N N N N* T T T T T T T HR=TTT N N N N N N N N N N N N utcome N N N T T T N T T T N T T T N No mis-preictions fter preictor lerns ll the relevnt ptterns CI 501 (Mrtin/Roth): Pipelining 72

19 Correlte Preictor Design choice I: one globl HR or one per (locl)? Ech one cptures ifferent kins of ptterns Globl is better, cptures locl ptterns for tight loop brnches Design choice II: how mny history bits (HR size)? Tricky one Given unlimite resources, longer HRs re better, but HT utiliztion ecreses Mny history ptterns re never seen Mny brnches re history inepenent (on t cre) xor HR llows multiple s to ynmiclly shre HT HR length < log 2 (HT size) Preictor tkes longer to trin Typicl length: 8 12 Hybri Preictor Hybri (tournment) preictor [McFrling] ttcks correlte preictor HT cpcity problem Ie: combine two preictors imple HT preicts history inepenent brnches Correlte preictor preicts only brnches tht nee history Chooser ssigns brnches to one preictor or the other rnches strt in simple HT, move mis-preiction threshol Correlte preictor cn be me smller, hnles fewer brnches 90 95% ccurcy HR HT HT chooser CI 501 (Mrtin/Roth): Pipelining 73 CI 501 (Mrtin/Roth): Pipelining 7 When to Perform rnch Preiction? During Decoe Look t instruction opcoe to etermine brnch instructions Cn clculte next from instruction (for -reltive brnches) ne cycle mis-fetch penlty even if brnch preictor is correct bnez r3,trg F D X M W trg: r,r5,r F D X M W During Fetch? How o we o tht? Revisiting rnch Preiction Components I$ P regfile tep #1: is it brnch? Esy fter ecoe... uring fetch: preictor tep #2: is the brnch tken or not tken? Direction preictor (s before) tep #3: if the brnch is tken, where oes it go? rnch trget preictor (T) upplies trget if brnch is tken D$ CI 501 (Mrtin/Roth): Pipelining 75 CI 501 (Mrtin/Roth): Pipelining 76

20 rnch Trget uffer (T) s before: lern from pst, preict the future Recor the pst brnch trgets in hrwre structure rnch trget buffer (T): guess the future bse on pst behvior Lst time the brnch X ws tken, it went to ress Y o, in the future, if ress X is fetche, fetch ress Y next pertion Like cche: ress =, t = trget- ccess t Fetch in prllel with instruction memory preicte-trget = T[] Upte t X whenever trget!= preicte-trget T[] = trget lising? No problem. s before, this is only preiction CI 501 (Mrtin/Roth): Pipelining 77 rnch Trget uffer (continue) t Fetch, how oes insn know it s brnch & shoul re T? It oesn t hve to ll insns ccess T in prllel with Imem Fetch Key ie: use T to preict which insn re brnches Implement by tgging ech entry with its corresponing Upte T on every tken brnch insn, recor trget : T[].tg =, T[].trget = trget of brnch ll insns ccess t Fetch in prllel with Imem Check for tg mtch, signifies insn t tht is brnch Preicte = (T[].tg == )? T[].trget : tg T trget preicte trget CI 371 (Roth/Mrtin): Pipelining 78 == Why Does T Work? ecuse most control insns use irect trgets Trget encoe in insn itself! sme tken trget every time Wht bout inirect trgets? Trget hel in register! cn be ifferent ech time Inirect conitionl jumps re not wiely supporte Two inirect cll iioms Dynmiclly linke functions (DLLs): trget lwys the sme Dynmiclly isptche (virtul) functions: hr but uncommon lso two inirect unconitionl jump iioms witches: hr but uncommon Function returns: hr n common but CI 501 (Mrtin/Roth): Pipelining 79 Return ress tck (R) T R PD I tg trget == preicte trget Return ress stck (R) Cll instruction? R[T] = Return instruction? Preicte-trget = R[--T] Q: how cn you tell if n insn is cll/return before ecoing it? ccessing R on every insn T-style oesn t work nswer: pre-ecoe bits in Imem, written when first execute Cn lso be use to signify brnches CI 501 (Mrtin/Roth): Pipelining 80

21 Putting It ll Together T & brnch irection preictor uring fetch T R PD I tg trget == is ret? preicte trget rnch Preiction Performnce Dynmic brnch preiction 20% of instruction brnches imple preictor: brnches preicte with 75% ccurcy CPI = 1 (20% * 25% * 2) = 1.1 More vnce preictor: 95% ccurcy CPI = 1 (20% * 5% * 2) = 1.02 rnch mis-preictions still big problem though Pipelines re long: typicl mis-preiction penlty is 10 cycles Pipelines re supersclr (lter) HT tken/not-tken If brnch preiction correct, no tken brnch penlty CI 501 (Mrtin/Roth): Pipelining 81 CI 501 (Mrtin/Roth): Pipelining 82 voiing rnches vi I: Preiction Conventionl control Conitionlly execute insns lso conitionlly fetche beq r3,trg F D X M W sub r6,1,r5 F D flushe: wrong pth trg: r,r5,r F flushe: why? trg: r,r5,r F D X M W If beq mis-preicts, both sub n must be flushe Wste: is inepenent of mis-preiction Preiction: not preiction, preiction I support for conitionlly-execute unconitionlly-fetche insns If beq mis-preicts, nnul sub in plce, preserve Exmple is if-then, but if-then-else cn be preicte too How is this one? How oes get correct vlue for r5 CI 501 (Mrtin/Roth): Pipelining 83 Full Preiction Full preiction Every insn cn be nnulle, nnulment controlle by Preicte registers: itionl register in ech insn (e.g., I6) setp.eq r3,p3 F D X M W sub.p r6,1,r5,p3 F D X nnulle trg: r,r5,r F D X M W Preicte coes: conition bits in ech insn (e.g., RM) setcc r3 F D X M W sub.nz r6,1,r5 F D X nnulle trg: r,r5,r F D X M W nly LU insn shown (sub), but this pplies to ll insns, even stores rnches replce with set-preicte insns CI 501 (Mrtin/Roth): Pipelining 8

22 Conitionl Moves (CMVs) Conitionl (register) moves Construct ppernce of full preiction from one primitive cmoveq r1,r2,r3 // if (r1==0) r3=r2; My require some coe upliction to chieve esire effect Pinful, potentilly impossible for some insn sequences Requires more registers nly goo wy of retro-fitting preiction onto I (e.g., I32, lph) sub r6,1,r9 D X M W cmovne r3,r9,r5 F D X M W trg: r,r5,r F D X M W Preiction Performnce Cost/benefit nlysis enefit: preiction vois brnches Thus voiing mis-preictions lso reuces pressure on preictor tble (few brnches to trck) Cost: extr (nnulle) instructions s brnch preictors re highly ccurte Might not help: 5-stge pipeline, two instruction on ech pth of if-then-else No performnce gin, likely slower if brnch preictble r even hurt! ut cn help: Deeper pipelines, hr-to-preict brnches, n few e insn Thus, preiction is useful, but not pnce CI 501 (Mrtin/Roth): Pipelining 85 CI 501 (Mrtin/Roth): Pipelining 86 Reserch: Perceptron Preictor Reserch Perceptron preictor [Jimenez] ttcks HR size problem using mchine lerning pproch HT replce by tble of function coefficients F i (signe) Preict tken if!(hr i *F i )> threshol Tble size #* HR * F (cn use long HR: ~60 bits) Equivlent correlte preictor woul be #*2 HR How oes it lern? Upte F i when brnch is tken HR i == 1? F i : F i ; on t cre F i bits sty ner 0, importnt F i bits sturte Hybri HT/perceptron ccurcy: 95 98% F! F i *HR i > thresh HR CI 501 (Mrtin/Roth): Technology 87 CI 501 (Mrtin/Roth): Pipelining 88

23 More Reserch: GEHL Preictor Problem with both correlte preictor n perceptron me HT rel-estte eicte to 1st history bit (1 column) s to 2n, 3r, 10th, 60th Not goo use of spce: 1st bit much more importnt thn 60th Chmpionship rnch Preiction CP Workshop hel in conjunction with MICR ubmitte coe is teste on stnr brnch trces Highest preiction ccurcy wins GEometric History-Length preictor [eznec, IC 05] Multiple HTs, inexe by geometriclly longer HRs (0,, 16, 32) HTs re (prtilly) tgge, not seprte chooser Preict: use mtching entry from HT with longest HR Mis-preict: crete entry in HT with longer HR nly 25% of HT use for bits (not 50%) Helps mortize cost of tgging Trins quickly 95-97% ccurte CI 501 (Mrtin/Roth): Pipelining 89 Two trcks Ielistic: preictor simultor must run in uner 2 hours Relistic: preictor must synthesize into 32K 256 bits or less 2006 winners Relistic: L-TGE (GEHL follow-on) Ielistic: GTL (nother GEHL follow-on) CI 501 (Mrtin/Roth): Pipelining 90 Reserch: Runhe Execution -regfile Reserch: Rzor regfile I$ regfile D$ I$ P == D$ In-orer writebcks essentilly imply stlls on D$ misses Cn sve power or use ile time for performnce Runhe execution [Duns] how regfile kept in sync with min regfile (write to both) D$ miss: continue executing using show regfile (isble stores) D$ miss returns: flush pipe n restrt with stlle cts like smrt prefetch engine Performs better s cche t miss grows (reltive to clock perio) CI 501 (Mrtin/Roth): Pipelining 91 Rzor [Uht, Ernst] Ientify pipeline stges with nrrow signl mrgins (e.g., X) Rzor X/M ltch: reltches X/M input signls fter sfe ely Compre X/M ltch with sfe rzor X/M ltch, ifferent? Flush F,D,X & M Restrt M using X/M rzor ltch, restrt F using D/X ltch Pipeline will not brek! reuce V DD until flush rte too high lterntively: over-clock until flush rte too high CI 501 (Mrtin/Roth): Pipelining 92

24 ummry pp pp pp ystem softwre CPU I/ Principles of pipelining Effects of overhe n hzrs Pipeline igrms hzrs tlling n bypssing Control hzrs rnch preiction Preiction CI 501 (Mrtin/Roth): Pipelining 93