Hakim Weatherspoon CS 3410 Computer Science Cornell University

Transcription

1 Hakim Weatherspoon CS 3410 Computer Science Cornell University The slides are the product of many rounds of teaching CS 3410 by Professors Weatherspoon, Bala, Bracy, McKee, and Sirer.

2 memory inst register file alu PC new pc offset target imm control extend =? cmp addr d in d out memory

3 Advantages Single cycle per instruction make logic and clock simple Disadvantages Since instructions take different time to finish, memory and functional unit are not efficiently utilized Cycle time is the longest delay Load instruction Best possible CPI is 1 (actually < 1 w parallelism) However, lower MIPS and longer clock period (lower clock frequency); hence, lower performance

4 Advantages Better MIPS and smaller clock period (higher clock frequency) Hence, better performance than Single Cycle processor Disadvantages Higher CPI than single cycle processor Pipelining: Want better Performance want small CPI (close to 1) with high MIPS and short clock period (high clock frequency)

5 Parallelism Pipelining Both!

6 Alice Bob They don t always get along

7

8 Saw Drill Glue Paint

9 N pieces, each built following same sequence: Saw Drill Glue Paint

10 Alice owns the room Bob can enter when Alice is finished Repeat for remaining tasks No possibility for conflicts

11 time Latency: Elapsed Time for 4 Alice: hours/task 4 Throughput: Elapsed Time for 1 Bob: task/4 4 hrs Concurrency: Total elapsed time: 1 4*N Can we do better? CPI = 4

12 Partition room into stages of a pipeline Dave Carol Bob Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep

13 Partition room into stages of a pipeline Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

14 Partition room into stages of a pipeline Bob Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

15 Partition room into stages of a pipeline Carol Bob Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

16 Partition room into stages of a pipeline Dave Carol Bob Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

17 Partition room into stages of a pipeline Alice Alice Alice Alice One person owns a stage at a time 4 stages 4 people working simultaneously Everyone moves right in lockstep It still takes all four stages for one job to complete

18 time Latency: 4 hrs/task Throughput: 1 task/hr Concurrency: 4 CPI = 1

19 Time What if drilling takes twice as long, but gluing and paint take ½ as long? Latency: Throughput: CPI =

20 Time Done: 4 cycles Done: 6 cycles Done: 8 cycles What if drilling takes twice as long, but gluing and paint take ½ as long? Latency: 4 cycles/task Throughput: 1 task/2 cycles CPI = CPI = 2

21 Principle: Throughput increased by parallel execution Balanced pipeline very important Else slowest stage dominates performance Pipelining: Identify pipeline stages Isolate stages from each other Resolve pipeline hazards (next lecture)

22 Single Cycle vs Pipelined Processor

23 Single-cycle insn0.fetch, dec, exec insn1.fetch, dec, exec Pipelined insn0.fetch insn0.dec insn1.fetch insn0.exec insn1.dec insn1.exec 23

24 5 stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 24

25 Review: Single cycle processor memory inst register file alu PC new pc offset target imm control extend =? cmp addr d in d out memory

26 memory inst register file alu +4 addr PC control d in d out memory new pc imm extend compute jump/branch targets Instruction Fetch Instruction Decode Execute Memory Write Back

27 memory register file alu +4 addr PC new pc control extend compute jump/branch targets d in d out memory Fetch Decode Execute Memory WB

28 memory PC +4 new pc inst register file control extend imm B A alu compute jump/branch targets B D addr d in d out memory D M Instruction Fetch Instruction Decode ctrl Execute ctrl Memory ctrl Write Back IF/ID ID/EX EX/MEM MEM/WB

29 Cycle add nand lw add sw IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB IF ID EX MEM WB Latency: 5 cycles Throughput: 1 insn/cycle Concurrency: 5 CPI = 1 29

30 Break datapath into multiple cycles (here 5) Parallel execution increases throughput Balanced pipeline very important Slowest stage determines clock rate Imbalance kills performance Add pipeline registers (flip flops) for isolation Each stage begins by reading values from latch Each stage ends by writing values to latch Resolve hazards 30

31 memory PC +4 new pc inst register file control extend imm B A alu compute jump/branch targets B D addr d in d out memory D M Instruction Fetch Instruction Decode ctrl Execute ctrl Memory ctrl Write Back IF/ID ID/EX EX/MEM MEM/WB

32 Stage Fetch Decode Execute Memory Perform Functionality Use PC to index Program Memory, increment PC Decode instruction, generate control signals, read register file Perform ALU operation Compute targets (PC+4+offset, etc.) in case this is a branch, decide if branch taken Perform load/store if needed, address is ALU result Latch values of interest Instruction bits (to be decoded) PC + 4 (to compute branch targets) Control information, Rd index, immediates, offsets, register values (Ra, Rb), PC+4 (to compute branch targets) Control information, Rd index, etc. Result of ALU operation, value in case this is a store instruction Control information, Rd index, etc. Result of load, pass result from execute Writeback Select value, write to register file 32

33 Stage 1: Instruction Fetch Fetch a new instruction every cycle Current PC is index to instruction memory Increment the PC at end of cycle (assume no branches for now) Write values of interest to pipeline register (IF/ID) Instruction bits (for later decoding) PC+4 (for later computing branch targets)

34 instruction memory addr mc +4 PC new pc PC+4 pc rel (PC relative); e.g. BEQ, BNE pc abs (PC absolute); e.g. J and JAL. (PC+4) target 00 pc reg (PC registers); e.g. JR

35 instruction memory addr mc PC = read word inst PC+4 Rest of pipeline new pc pc reg pc relpc abs pc sel IF/ID

36 instruction memory addr mc PC = read word inst PC+4 Rest of pipeline pc reg pc relpc abs PC+4 pc reg (PC registers: JR) pc rel (PC relative: BEQ, BNE) pc abs (PC absolute: J and JAL) pc sel IF/ID 36

37 Stage 2: Instruction Decode On every cycle: Read IF/ID pipeline register to get instruction bits Decode instruction, generate control signals Read from register file Write values of interest to pipeline register (ID/EX) Control information, Rd index, immediates, offsets, Contents of Ra, Rb PC+4 (for computing branch targets later)

38 ctrl PC+4 Rest of pipeline ID/EX PC+4 inst IF/ID WE Rd D register file A B Ra Rb Stage 1: Instruction Fetch imm B A

39 ctrl PC+4 Rest of pipeline ID/EX PC+4 inst IF/ID WE Rd D register file A B Ra Rb Stage 1: Instruction Fetch imm B A decode result dest extend

40 Stage 3: Execute On every cycle: Read ID/EX pipeline register to get values and control bits Perform ALU operation Compute targets (PC+4+offset, etc.) in case this is a branch Decide if jump/branch should be taken Write values of interest to pipeline register (EX/MEM) Control information, Rd index, Result of ALU operation Value in case this is a memory store instruction

41 Stage 2: Instruction Decode ctrl B D Rest of pipeline EX/MEM ctrl PC+4 imm B A target ID/EX alu

42 pcrel pcabs Stage 2: Instruction Decode ctrl B D Rest of pipeline EX/MEM ctrl PC+4 imm B A ID/EX + pcreg alu pcsel branch? target

43 Stage 4: Memory On every cycle: Read EX/MEM pipeline register to get values and control bits Perform memory load/store if needed address is ALU result Write values of interest to pipeline register (MEM/WB) Control information, Rd index, Result of memory operation Pass result of ALU operation

44 ctrl Rest of pipeline MEM/WB ctrl Stage 3: Execute B D M D EX/MEM addr d in d out memory mc target

45 ctrl Rest of pipeline MEM/WB ctrl Stage 3: Execute B D M D pcsel pcreg pcrel pcabs EX/MEM branch? addr d in d out memory mc target

46 Stage 5: Write back On every cycle: Read MEM/WB pipeline register to get values and control bits Select value and write to register file

47 ctrl Stage 4: Memory M D MEM/WB

48 result ctrl Stage 4: Memory M D dest MEM/WB

49 +4 IF/ID addr d in d out mem ID/EX EX/MEM MEM/WB PC+4 inst OP Rt Rd OP Rd OP Rd PC+4 imm B A B D M D inst mem PC Rd D A B Ra Rb 49

50 Consider a non pipelined processor with clock period C (e.g., 50 ns). If you divide the processor into N stages (e.g., 5), your new clock period will be: A. C B. N C. less than C/N D. C/N E. greater than C/N 50

51 Consider a non pipelined processor with clock period C (e.g., 50 ns). If you divide the processor into N stages (e.g., 5), your new clock period will be: A. C B. N C. less than C/N D. C/N E. greater than C/N 51

52 Pipelining is a powerful technique to mask latencies and increase throughput Logically, instructions execute one at a time Physically, instructions execute in parallel Instruction level parallelism Abstraction promotes decoupling Interface (ISA) vs. implementation (Pipeline)

53 Instructions same length 32 bits, easy to fetch and then decode 3 types of instruction formats Easy to route bits between stages Can read a register source before even knowing what the instruction is Memory access through lw and sw only Access memory after ALU 53

54 5 stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 54

55 add nand lw add sw r3 r1, r2 r6 r4, r5 r4 20(r2) r5 r2, r5 r7 12(r3) Assume 8 register machine 55

56 M U X PC 4 + PC+4 instruction rega regb Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R7 0 extend PC+4 vala valb imm Rd Rt op M U X M U X A L U target ALU result valb dest op Data mem ALU result mdata IF/ID ID/EX EX/MEM MEM/WB dest op M U X data dest

57 At time 1, Fetch add r3 r1 r2 4 0 extend data dest 0 0 M U X Time: 0 IF/ID ID/EX EX/MEM MEM/WB

58 add M U X 8 PC 4 4 Fetch: add Time: 1/ add Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend / 0 4 / 0 36 / / 03 / 02 nop / add M U X M U X A L U nop Data mem IF/ID ID/EX EX/MEM MEM/WB nop M U X data dest

59 nand add M U X 12 PC 8 4 Fetch: nand Time: 2/ nand Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend / / 18 / add / nand M U X A L U / 0 4 / 0 45 / 0 9 / 36 M 3 U / 25 / 0 3 X 0 nop / add Data mem IF/ID ID/EX EX/MEM MEM/WB nop M U X data dest

60 lw 4 20(2) nand add M U X PC 12 4 Fetch: lw 4 20(2) Time: 3/ lw 4 20(2) 4 5 Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend nand nand (18 7) 18 = = = / 18 / 9 7 M U X M U X A L U 3 / / 3 / 9 7 / 3 6 add / nand Data mem / / 0 3 M U X nop / add IF/ID ID/EX EX/MEM MEM/WB data dest

61 add lw 4 20(2) nand add M U X 20 PC 16 4 Fetch: add Time: add Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend lw M U X M U X A L U nand 45 3 Data mem IF/ID ID/EX EX/MEM MEM/WB add M U X data dest

62 sw 7 12(3) add lw 4 20 (2) nand add M U X 24 PC 20 4 Fetch: sw 7 12(3) Time: sw 7 12(3) 2 5 Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend add M U X M U X A L U lw 3 6 Data mem nand IF/ID ID/EX EX/MEM MEM/WB 45 3 M U X data dest

63 sw 7 12(3) add lw 4 20(2) nand M U X 28 PC 24 4 No more instructions Time: Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend sw M U X M U X A L U add 29 4 Data mem IF/ID ID/EX EX/MEM MEM/WB lw 3 6 M U X data dest

64 nop nop sw 7 12(3) add lw 4 20(2) M U X 32 PC 28 4 No more instructions Time: 7 + Register file Bits Bits Bits R0 R1 R2 R3 R4 R5 R6 R extend M U X M U X A L U sw 16 5 Data mem IF/ID ID/EX EX/MEM MEM/WB add 99 4 M U X data dest

65 nop nop nop sw 7 12(3) add M U X 36 PC Register file R0 R1 R2 R3 R4 R5 R6 R extend M U X A L U Data mem M U X data dest No more instructions Time: 8 Bits Bits Bits M U X IF/ID ID/EX EX/MEM MEM/WB 7 sw 5 Slides thanks to Sally McKee

66 nop nop nop nop sw 7 12(3) M U X 40 PC Register file R0 R1 R2 R3 R4 R5 R6 R extend M U X A L U Data mem M U X data dest No more instructions Bits Bits Bits M U X Time: 9 IF/ID ID/EX EX/MEM MEM/WB

67 Pipelining is great because: A. You can fetch and decode the same instruction at the same time. B. You can fetch two instructions at the same time. C. You can fetch one instruction while decoding another. D. Instructions only need to visit the pipeline stages that they require. E. C and D 67

68 Pipelining is great because: A. You can fetch and decode the same instruction at the same time. B. You can fetch two instructions at the same time. C. You can fetch one instruction while decoding another. D. Instructions only need to visit the pipeline stages that they require. E. C and D 68

69 memory PC +4 new pc inst register file control extend imm B A alu compute jump/branch targets B D addr d in d out memory D M Instruction Fetch Instruction Decode ctrl Execute ctrl Memory ctrl Write Back IF/ID ID/EX EX/MEM MEM/WB

70 5 stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 70

71 Correctness problems associated w/processor design 1. Structural hazards Same resource needed for different purposes at the same time (Possible: ALU, Register File, Memory) 2. Data hazards Instruction output needed before it s available 3. Control hazards Next instruction PC unknown at time of Fetch 71

72 Dependence: relationship between two insns Data: two insns use same storage location Control: 1 insn affects whether another executes at all Not a bad thing, programs would be boring otherwise Enforced by making older insn go before younger one Happens naturally in single /multi cycle designs But not in a pipeline Hazard: dependence & possibility of wrong insn order Effects of wrong insn order cannot be externally visible Hazards are a bad thing: most solutions either complicate the hardware or reduce performance 72

73 Data Hazards register file (RF) reads occur in stage 2 (ID) RF writes occur in stage 5 (WB) RF written in ½ half, read in second ½ half of cycle x10: add r3 r1, r2 x14: sub r5 r3, r4 1. Is there a dependence? 2. Is there a hazard? A) Yes B) No C) Cannot tell with the information given. 73

74 Data Hazards register file (RF) reads occur in stage 2 (ID) RF writes occur in stage 5 (WB) RF written in ½ half, read in second ½ half of cycle x10: add r3 r1, r2 x14: sub r5 r3, r4 1. Is there a dependence? 2. Is there a hazard? A) Yes for both B) No C) Cannot tell with the information given. 74

75 Which of the following statements is true? A. Whether there is a data dependence between two instructions depends on the machine the program is running on. B. Whether there is a data hazard between two instructions depends on the machine the program is running on. C. Both A & B D. Neither A nor B 75

76 Which of the following statements is true? A. Whether there is a data dependence between two instructions depends on the machine the program is running on. B. Whether there is a data hazard between two instructions depends on the machine the program is running on. C. Both A & B D. Neither A nor B 76

77 time Clock cycle add r3, r1, r2 IF ID MEM WB sub r5, r3, r4 IF ID MEM WB lw r6, 4(r3) IF ID MEM WB or r5, r3, r5 IF ID MEM WB sw r6, 12(r3) IF ID MEM WB

78 add r3, r1, r2 sub r5, r3, r4 lw r6, 4(r3) or r5, r3, r5 How many data hazards due to r3 only A) 1 B) 2 C) 3 D) 4 E) 5 sw r6, 12(r3)

79 time Clock cycle backwards arrows require time travel add r3, r1, r2 IF ID X MEM WB sub r5, r3, r4 IF ID X MEM WB lw r6, 4(r3) IF ID X MEM WB or r5, r3, r5 IF ID X MEM WB sw r6, 12(r3) IF ID X MEM WB 79

80 time Clock cycle add r3, r1, r2 IF ID X MEM WB sub r5, r3, r4 IF ID X MEM WB lw r6, 4(r3) IF ID X MEM WB or r5, r3, r5 IF ID X MEM WB sw r6, 12(r3) IF ID X MEM WB 80

81 time Clock cycle add r3, r1, r2 IF ID X MEM WB sub r5, r3, r4 IF ID X MEM WB lw r6, 4(r3) IF ID X MEM WB or r5, r3, r5 IF ID X MEM WB sw r6, 12(r3) IF ID X MEM WB 81

82 Data Hazards register file reads occur in stage 2 (ID) register file writes occur in stage 5 (WB) next instructions may read values about to be written i.e. add r3, r1, r2 sub r5, r3, r4 How to detect?

83 Detecting Data Hazards inst mem PC +4 inst PC+4 IF/ID Rd D A B Ra Rb IF/ID.Ra 0 && (IF/ID.Ra==ID/Ex.Rd IF/ID.Ra==Ex/M.Rd IF/ID.Ra==M/W.Rd) sub r5,r3,r4 OP Rt Rd PC+4 imm B A add r3, r1, r2 B D OP Rd d in addr d out mem M D OP Rd ID/EX EX/MEM MEM/WB

84 Data Hazards register file reads occur in stage 2 (ID) register file writes occur in stage 5 (WB) next instructions may read values about to be written How to detect? Logic in ID stage: stall = (IF/ID.Ra!= 0 && (IF/ID.Ra == ID/EX.Rd IF/ID.Ra == EX/M.Rd IF/ID.Ra == M/WB.Rd)) (same for Rb)

85 Detecting Data Hazards inst mem PC +4 inst PC+4 Rd D A B Ra Rb detect hazard OP Rt Rd PC+4 imm B A B D OP Rd addr d in d out mem M D OP Rd IF/ID ID/EX EX/MEM MEM/WB

86 Data hazards occur when a operand (register) depends on the result of a previous instruction that may not be computed yet. A pipelined processor needs to detect data hazards.

87 What to do if data hazard detected?

88 What to do if data hazard detected? A) Wait/Stall B) Reorder in Software (SW) C) Forward/Bypass D) All the above E) None. We will use some other method

89 1. Do Nothing Change the ISA to match implementation Hey compiler: don t create code w/data hazards! (We can do better than this) 2. Stall Pause current and subsequent instructions till safe 3. Forward/bypass Forward data value to where it is needed (Only works if value actually exists already) 89

90 How to stall an instruction in ID stage prevent IF/ID pipeline register update stalls the ID stage instruction convert ID stage instr into nop for later stages innocuous bubble passes through pipeline prevent PC update stalls the next (IF stage) instruction

91 add r3, r1, r2 sub inst r5, r3, r5 or r6, r3, r4 mem add r6, r3, r8 PC +4 WE=0 inst PC+4 IF/ID Rd A D B Ra Rb detect hazard If detect hazard MemWr=0 RegWr=0 OP Rt Rd PC+4 imm B A B D OP Rd d in addr d out mem M D OP Rd ID/EX EX/MEM MEM/WB

92 time Clock cycle add r3, r1, r2 sub r5, r3, r5 or r6, r3, r4 add r6, r3, r8

93 time r3 = 10 add r3, r1, r2 r3 = 20 sub r5, r3, r5 Clock cycle IF ID Ex M W 3 Stalls IF ID ID ID ID Ex M W or r6, r3, r4 IF IF IF IF ID Ex M add r6, r3, r8 IF ID Ex

94 inst mem +4 inst D rd ra rb A B A B D B data mem D M PC (MemWr=0 RegWr=0) nop Op WE Rd Op WE Rd Op WE Rd sub r5,r3,r5 add r3,r1,r2 or r6,r3,r4 (WE=0) /stall NOP = If(IF/ID.rA 0 && (IF/ID.rA==ID/Ex.Rd IF/ID.rA==Ex/M.Rd IF/ID.rA==M/W.Rd)) STALL CONDITION MET

95 inst mem PC +4 or r6,r3,r4 inst (WE=0) D rd ra rb A B (MemWr=0 RegWr=0) nop sub r5,r3,r5 /stall A B Rd WE Op NOP = If(IF/ID.rA 0 && (IF/ID.rA==ID/Ex.Rd IF/ID.rA==Ex/M.Rd IF/ID.rA==M/W.Rd)) (MemWr=0 RegWr=0) nop D B Rd WE Op data mem add r3,r1,r2 STALL CONDITION MET D M Op WE Rd

96 inst mem +4 inst D rd ra rb A B A B D B data mem D M PC (MemWr=0 RegWr=0) nop Rd WE Op (MemWr=0 RegWr=0) Rd WE Op (MemWr=0 RegWr=0) sub r5,r3,r5 nop nop Rd WE Op add r3,r1,r2 or r6,r3,r4 (WE=0) /stall NOP = If(IF/ID.rA 0 && (IF/ID.rA==ID/Ex.Rd IF/ID.rA==Ex/M.Rd IF/ID.rA==M/W.Rd)) STALL CONDITION MET

97 time r3 = 10 add r3, r1, r2 r3 = 20 sub r5, r3, r5 Clock cycle IF ID Ex M W 3 Stalls IF ID ID ID ID Ex M W or r6, r3, r4 IF IF IF IF ID Ex M add r6, r3, r8 IF ID Ex

98 How to stall an instruction in ID stage prevent IF/ID pipeline register update stalls the ID stage instruction convert ID stage instr into nop for later stages innocuous bubble passes through pipeline prevent PC update stalls the next (IF stage) instruction

99 Data hazards occur when a operand (register) depends on the result of a previous instruction that may not be computed yet. A pipelined processor needs to detect data hazards. Stalling, preventing a dependent instruction from advancing, is one way to resolve data hazards. Stalling introduces NOPs ( bubbles ) into a pipeline. Introduce NOPs by (1) preventing the PC from updating, (2) preventing writes to IF/ID registers from changing, and (3) preventing writes to memory and register file. *Bubbles in pipeline significantly decrease performance.

100 1. Do Nothing Change the ISA to match implementation Compiler: don t create code with data hazards! (Nice try, we can do better than this) 2. Stall Pause current and subsequent instructions till safe 3. Forward/bypass Forward data value to where it is needed (Only works if value actually exists already) 100

101 Forwarding bypasses some pipelined stages forwarding a result to a dependent instruction operand (register). Three types of forwarding/bypass Forwarding from Ex/Mem registers to Ex stage (M Ex) Forwarding from Mem/WB register to Ex stage (W Ex) RegisterFile Bypass

102 inst mem D A B A B imm D B data mem D M detect hazard Rb Ra forward unit Rd WE MC Rd WE MC IF/ID ID/Ex Ex/Mem Mem/WB 102

103 inst mem D A B A B imm D B data mem D M detect hazard Rb Ra forward unit Rd WE MC Rd WE MC IF/ID ID/Ex Ex/Mem Mem/WB Three types of forwarding/bypass Forwarding from Ex/Mem registers to Ex stage (M Ex) Forwarding from Mem/WB register to Ex stage (W Ex) RegisterFile Bypass 103

104 Ex/Mem inst mem D A B data mem sub r5, r3, r1 add r3, r1, r2 add r3, r1, r2 sub r5, r3, r1 IF ID Ex M W IF ID Ex M W Problem: EX needs ALU result that is in MEM stage Solution: add a bypass from EX/MEM.D to start of EX 104

105 Ex/Mem A inst mem D B data mem sub r5, r3, r1 add r3, r1, r2 Detection Logic in Ex Stage: forward = (Ex/M.WE && EX/M.Rd!= 0 && ID/Ex.Ra == Ex/M.Rd) (same for Rb) 105

106 Mem/WB A inst mem D B data mem add r3, r1, r2 sub r5, r3, r1 or r6, r3, r4 or r6, r3, r4 IF ID Ex M W IF ID IF Ex ID M Ex W M sub r5, r3, r1 W add r3, r1,r2 Problem: EX needs value being written by WB Solution: Add bypass from WB final value to start of EX 106

107 Mem/WB inst mem D A B data mem or r6, r3, r4 sub r5, r3, r1 Detection Logic: forward = (M/WB.WE && M/WB.Rd!= 0 && ID/Ex.Ra == M/WB.Rd && not (ID/Ex.WE && Ex/M.Rd!= 0 && ID/Ex.Ra == Ex/M.Rd) (same for Rb) add r3, r1,r2 107

108 A inst mem D B data mem add r6, r3, r8 or r6, r3, r4 sub r5, r3, r1 add r3, r1,r2 Problem: Reading a value that is currently being written Solution: just negate register file clock writes happen at end of first half of each clock cycle reads happen during second half of each clock cycle

109 inst mem D A B data mem add r6, r3, r8 or r6, r3, r4 sub r5, r3, r1 add r3, r1,r2 add r3, r1, r2 IF ID Ex M W sub r5, r3, r1 or r6, r3, r4 add r6, r3, r8 IF ID IF Ex M W ID Ex M W IF ID Ex M W

110 5 stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 110

111 time Clock cycle add r3, r1, r2 sub r5, r3, r4 lw r6, 4(r3) or r5, r3, r5 sw r6, 12(r3) 111

112 time add r3, r1, r2 Clock cycle IF ID Ex M W sub r5, r3, r4 IF ID Ex M W lw r6, 4(r3) IF ID Ex M W or r5, r3, r6 IF ID Ex M W sw r6, 12(r3) IF ID Ex M W

113 time add r3, r1, r2 Clock cycle IF ID Ex M W backwards arrows require time travel sub r5, r3, r4 IF ID Ex M W lw r6, 4(r3) IF ID Ex M W or r5, r3, r5 IF ID Ex M W sw r6, 12(r3) IF ID Ex M W

114 A inst mem D B data mem or r6, r3, r4 lw r4, 20(r8) Data dependency after a load instruction: Value not available until after the M stage Next instruction cannot proceed if dependent THE KILLER HAZARD 114

115 inst mem A D B or r6,r4,r1 lw r4, 20(r8) data mem lw r4, 20(r8) or r6, r3, r4 115

116 inst mem A D B or r6,r4,r1 lw r4, 20(r8) data mem lw r4, 20(r8) IF ID Ex or r6, r3, r4 IF ID 116

117 inst mem A D B or r6,r4,r1 NOP data mem lw r4, 20(r8) lw r4, 20(r8) or r6, r3, r4 IF ID Ex M W Stall IF ID* ID Ex M W 117

118 inst mem D A B data mem or r6,r4,r1 NOP lw r4, 20( lw r4, 20(r8) or r6, r3, r4 IF ID Ex M W Stall IF ID* ID Ex Ex M W 118

119 inst mem D A B A B imm D B data mem D M detect hazard Rd Rb Ra MC forward unit IF/ID ID/Ex Ex/Mem Mem/WB Rd WE MC Rd WE MC Stall = If(ID/Ex.MemRead && IF/ID.Ra == ID/Ex.Rd 119

120 inst mem D A B A B imm D B data mem D M detect hazard Rd Rb Ra MC forward unit IF/ID ID/Ex Ex/Mem Mem/WB Rd WE MC Rd WE MC Most frequent 3410 non solution to load use hazards Why is this solution so so so so so so awful? 120

121 Forwarding values directly from Memory to the Execute stage without storing them in a register first: A. Does not remove the need to stall. B. Adds one too many possible inputs to the ALU. C. Will cause the pipeline register to have the wrong value. D. Halves the frequency of the processor. E. Both A & D 121

122 Forwarding values directly from Memory to the Execute stage without storing them in a register first: A. Does not remove the need to stall. B. Adds one too many possible inputs to the ALU. C. Will cause the pipeline register to have the wrong value. D. Halves the frequency of the processor. E. Both A & D 122

123 Two MIPS Solutions: MIPS 2000/3000: delay slot ISA says results of loads are not available until one cycle later Assembler inserts nop, or reorders to fill delay slot MIPS 4000 onwards: stall But really, programmer/compiler reorders to avoid stalling in the load delay slot 123

124 Data hazards occur when a operand (register) depends on the result of a previous instruction that may not be computed yet. A pipelined processor needs to detect data hazards. Stalling, preventing a dependent instruction from advancing, is one way to resolve data hazards. Stalling introduces NOPs ( bubbles ) into a pipeline. Introduce NOPs by (1) preventing the PC from updating, (2) preventing writes to IF/ID registers from changing, and (3) preventing writes to memory and register file. Bubbles (nops) in pipeline significantly decrease performance. Forwarding bypasses some pipelined stages forwarding a result to a dependent instruction operand (register). Better performance than stalling.

125 Find all hazards, and say how they are resolved: add r3, r1, r2 nand r5, r3, r4 add r2, r6, r3 lw r6, 24(r3) sw r6, 12(r2)

126 Find all hazards, and say how they are resolved: add r3, r1, r2 nand r5, r3, r4 add r2, r6, r3 lw r6, 24(r3) sw r6, 12(r2) 5 Hazards

127 Find all hazards, and say how they are resolved: add r3, r1, r2 nand r5, r3, r4 add r2, r6, r3 lw r6, 24(r3) sw r6, 12(r2) Forwarding from Ex/M ID/Ex (M Ex) Forwarding from M/W ID/Ex (W Ex) RegisterFile (RF) Bypass Forwarding from M/W ID/Ex (W Ex) Stall + Forwarding from M/W ID/Ex (W Ex) 5 Hazards

128 Find all hazards, and say how they are resolved: add r3, r1, r2 sub r3, r2, r1 nand r4, r3, r1 or r0, r3, r4 xor r1, r4, r3 sb r4, 1(r0)

129 Find all hazards, and say how they are resolved: add r3, r1, r2 sub r3, r2, r1 nand r4, r3, r1 or r0, r3, r4 xor r1, r4, r3 sb r4, 1(r0) Hours and hours of debugging!

130 Delay Slot(s) Modify ISA to match implementation Stall Pause current and all subsequent instructions Forward/Bypass Try to steal correct value from elsewhere in pipeline Otherwise, fall back to stalling or require a delay slot Tradeoffs?

131 5 stage Pipeline Implementation Working Example Hazards Structural Data Hazards Control Hazards 131

132 i = 0; do { n += 2; i++; } while(i < max) i = 7; n ; i r1 Assume: n r2 max r3 x10 addiu r1, r0, 0 # i=0 x14 Loop: addiu r2, r2, 2 # n += 2 x18 addiu r1, r1, 1 # i++ x1c blt r1, r3, Loop # i<max? x20 addiu r1, r0, 7 # i = 7 x24 subi r2, r2, 1 # n 132

133 Control Hazards instructions are fetched in stage 1 (IF) branch and jump decisions occur in stage 3 (EX) next PC not known until 2 cycles after branch/jump x1c blt r1, r3, Loop x20 addiu r1, r0, 7 x24 subi r2, r2, 1 Branch not taken? No Problem! Branch taken? Just fetched 2 addi s Zap & Flush 133

134 inst mem PC +4 D A B New PC = 14 1C blt r1,r3,l 20 addiu r1,r0,7 24 subi r2,r2,1 14 L:addi r2,r2,2 branch calc decide branch data mem IF ID Ex M W IF ID NOP NOP NOP prevent PC update clear IF/ID latch branch continues If branch Taken Zap IF NOP NOP NOP NOP IF ID Ex M W 134

135 prevent PC update clear IF/ID latch branch continues inst mem PC +4 D A B New PC = 1C 1C blt r1,r3,l 20 addiu r1,r0,7 24 subi r2,r2,1 14 L:addi r2,r2,2 branch calc decide branch data mem If branch Taken Zap IF ID Ex M W IF ID NOP NOP NOP IF NOP NOP NOP NOP For every taken branch? OUCH!!! IF ID Ex M W 135

136 1. Delay Slot You MUST do this MIPS ISA: 1 insn after ctrl insn always executed Whether branch taken or not 2. Resolve Branch at Decode Some groups do this for Project 3, your choice Move branch calc from EX to ID Alternative: just zap 2 nd instruction when branch taken 3. Branch Prediction Not in 3410, but every processor worth anything does this (no offense!) 136

137 inst mem PC +4 D A B New PC = 1C branch calc decide branch 1C blt r1, r3, Loop F D X 20 addiu r1, r0, 7 F D 24 subi r2, r2, 1 F data mem If branch Taken Zap

138 i = 0; do { n += 2; i++; } while(i < max) i = 7; n ; x10 addiu r1, r0, 0 # i=0 i r1 Assume: n r2 max r3 x14 Loop: addiu r2, r2, 2 # n += 2 x18 addiu r1, r1, 1 # i++ x1c blt r1, r3, Loop # i<max? x20 nop x24 addiu r1, r0, 7 # i = 7 x28 subi r2, r2, 1 # n++ 138

139 inst mem PC +4 D A B New PC = 1C branch calc decide branch 1C blt r1, r3, Loop F D X 20 nop F D 24 addiu r1, r0, 7 F data mem If branch Taken Zap

140 inst mem PC +4 D A B branch calc decide data mem New PC = 1C branch 1C blt r1, r3, Loop F D X 20 nop F D 14 Loop:addiu r2,r2,2 F If branch Taken No Zapping 140

141 x10 addiu r1, r0, 0 # i=0 x14 Loop: addiu r2, r2, 2 # n += 2 x18 addiu r1, r1, 1 # i++ x1c blt r1, r3, Loop # i<max? x20 nop Compiler transforms code x10 addiu r1, r0, 0 # i=0 x14 Loop: addiu r1, r1, 1 # i++ x18 blt r1, r3, Loop # i<max? x1c addiu r2, r2, 2 # n += 2 141

142 inst mem PC +4 D A B branch calc decide data mem New PC = 1C branch 1C blt r1, r3, Loop F D X 20 addi r2,r2,2 F D 14 Loop:addi r1,r1,1 F Note: Insn in delay slot will always be executed whether branch take or not 142

143 Most processor support Speculative Execution Guess direction of the branch Allow instructions to move through pipeline Zap them later if guess turns out to be wrong A must for long pipelines 143

144 Pipeline so far Guess (predict) that the branch will not be taken We can do better! Make prediction based on last branch Predict take branch if last branch taken Or Predict do not take branch if last branch not taken Need one bit to keep track of last branch

145 What is accuracy of branch predictor? Wrong twice per loop! Once on loop enter and exit We can do better with 2 bits While (r3 0) {. r3 ;} Top: BEQZ r3, End J Top End: While (r3 0) {. r3 ;} Top2: BEQZ r3, End2 End2: J Top

146 Branch Not Taken (NT) Predict Taken 2 (PT2) Predict Taken 1 (PT1) Branch Taken (T) Branch Taken (T) Branch Not Taken (NT) Branch Taken (T) Predict Not Taken 2 (PT2) Predict Not Taken 1 (PT1) Branch Not Taken (NT)

147 Control hazards Is branch taken or not? Performance penalty: stall and flush Reduce cost of control hazards Move branch decision from Ex to ID 2 nops to 1 nop Delay slot Compiler puts useful work in delay slot. ISA level. Branch prediction Correct. Great! Wrong. Flush pipeline. Performance penalty

148 Data hazards Control hazards Structural hazards resource contention so far: impossible because of ISA and pipeline design

149 Data hazards register file reads occur in stage 2 (IF) register file writes occur in stage 5 (WB) next instructions may read values soon to be written Control hazards branch instruction may change the PC in stage 3 (EX) next instructions have already started executing Structural hazards resource contention so far: impossible because of ISA and pipeline design

150 Data hazards occur when a operand (register) depends on the result of a previous instruction that may not be computed yet. Pipelined processors need to detect data hazards. Stalling, preventing a dependent instruction from advancing, is one way to resolve data hazards. Stalling introduces NOPs ( bubbles ) into a pipeline. Introduce NOPs by (1) preventing the PC from updating, (2) preventing writes to IF/ID registers from changing, and (3) preventing writes to memory and register file. Nops significantly decrease performance. Forwarding bypasses some pipelined stages forwarding a result to a dependent instruction operand (register). Better performance than stalling. 150

151 Control hazards occur because the PC following a control instruction is not known until control instruction is executed. If branch is taken need to zap instructions. 1 cycle performance penalty. Delay Slots can potentially increase performance due to control hazards. The instruction in the delay slot will always be executed. Requires software (compiler) to make use of delay slot. Put nop in delay slot if not able to put useful instruction in delay slot. We can reduce cost of a control hazard by moving branch decision and calculation from Ex stage to ID stage. With a delay slot, this removes the need to flush instructions on taken branches. 151