Processor: Superscalars Dynamic Scheduling

Processor: Superscalars Dynamic Scheduling Z. Jerry Shi Assistant Professor of Computer Science and Engineering University of Connecticut * Slides adapted from Blumrich&Gschwind/ELE475 03, Peh/ELE475 (Princeton), Brooks/CS146 (Harvard)* Recap: Out-of-order execution 1

Dynamic scheduling Hardware solution to reduce data hazards In-order issue Decode and check structural hazards of instructions in turn Out-of-order execution Executes an instruction when it has all its operands (data dependencies satisfied) Allows out-of-order completion Goals of scheduling Goal of static scheduling Compiler tries to avoids/reduce dependencies Goal of dynamic scheduling Hardware tries to avoid stalling when present Why hardware and not compiler? Code portability Some information are not available at compile time ISA can limit registers ID space Speculations sometimes needs hardware to work well t everyone uses gcc O3 2

Types of dependences Data Dependences (true data dependences): instruction j is data dependent on i if Instruction i produces a result that my be used by instruction j, or Instruction j is data dependent on instruction k and instruction k is data dependent on i Name dependences: instruction i precedes instruction j An antidependence: when instruction j writes a register or memory location that instruction i reads An output dependence: when instruction i and instruction j write to the same register or memory location Control dependence Data hazards Consider instruction i and instruction j. Instruction i is before instruction j. RAW (read after write): j tries to read a source before i writes it WAW (write after write): j tries to write an operand before i writes it WAR (write after read): j tries to write a destination before it is read by i 3

Dynamic Scheduling Key Idea: Hardware allows instructions behind stall to proceed <OP out, in0, in1> DIVD F0,F2,F4 ADDD F10,F0,F8 SUBD F12,F8,F14 What type of dependency on F0? data? anti? output-dependency? What type of hazard can this dependency cause? RAW? WAR? WAW? Which instruction can be executed out-of-order? Scoreboarding Basics Instruction decode and operation fetch are in a single cycle in the basic pipeline Two things are needed to support multiple instructions in ID Split ID into two stages ue Decode instructions, check for structural hazards Read operands Wait until no data hazards, then read operands Buffered storage (Instruction buffer/window/ queue) IF ID EX M WB Standard Pipeline I-Buffer/Scoreboard IF IS Rd EX M WB New Pipeline 4

Scoreboarding Originally proposed in CDC6000 (S. Cray, 1964) Out of order execution Centralized scheme bypassing WAR/WAW hazards are problems Scoreboarding Stages ue Fetch Same as before ue Check structural hazards If a functional unit is free and no other active instruction has the same destination registers (WAW), then issue instruction and update internal data structure Do not issue until structural and WAW hazards are cleared Stalled instruction stays in I-Buffer further instruction will be issued Size of buffer is also a structural hazard Have to stall Fetch if buffer fills te: ue is in-order, stalls stop younger instructions 5

Scoreboarding Stages: Read Operands Read Operands (Check Data Hazards) Check scoreboard for whether source operands are available Read operands and start execution when all the operands are ready A operand is available if earlier issued active instructions will write it currently active functional unit is going to write it Dynamically avoid RAW hazards Instructions may be sent into execution out of order Scoreboarding Stages: Execution/Write Result Execution Execute/update scoreboard Write result Scoreboard checks for WAR stalls and stalls completing instruction, if necessary Before, stalls only occur at the beginning of instructions. w it can be at the end as well Happens if Completing instruction s destination register matches an older instruction that has not yet read its source operands DIV.D F0, F2, F4 ADD.D F10, F0, F8 SUB.D F8, F8, F14 6

Scoreboarding Control Hardware Three main parts Instruction status bits Indicate which of the four stages instructions are in Functional unit status bits Busy when operations are being performed Op: Operations to perform in the unit F i : destination register F j, F k : source registers Q j, Q k : functional units producing F j, F k R j, R k : flags indicating F j and F k are ready but not yet read Set to after oprands are read Register result status Which functional unit will write each register Scoreboard Example Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 LD F2 45+ R3 MULTD F0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Mult2 Add Divide FU 7

Scoreboard Example: Cycle 1 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 LD F2 45+ R3 MULTD F0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F6 R2 Yes Mult1 Mult2 Add Divide 1 FU Integer Scoreboard Example: Cycle 2 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 LD F2 45+ R3 MULTD F0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F6 R2 Yes Mult1 Mult2 Add Divide 2 FU Integer ue 2nd LD? 8

Scoreboard Example: Cycle 3 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 LD F2 45+ R3 MULTD F0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F6 R2 Mult1 Mult2 Add Divide 3 FU Integer ue MULT? Scoreboard Example: Cycle 4 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 MULTD F0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Mult2 Add Divide 4 FU Integer 9

Scoreboard Example: Cycle 5 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 MULTD F0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F2 R3 Yes Mult1 Mult2 Add Divide 5 FU Integer Scoreboard Example: Cycle 6 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 MULTD F0 F2 F4 6 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F2 R3 Yes Mult1 Yes Mult F0 F2 F4 Integer Yes Mult2 Add Divide 6 FU Mult1 Integer 10

Scoreboard Example: Cycle 7 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 MULTD F0 F2 F4 6 SUBD F8 F6 F2 7 DIVD F10 F0 F6 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F2 R3 Mult1 Yes Mult F0 F2 F4 Integer Yes Mult2 Add Yes Sub F8 F6 F2 Integer Yes Divide 7 FU Mult1 Integer Add Read multiply operands? Scoreboard Example: Cycle 8a (First half of clock cycle) Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 MULTD F0 F2 F4 6 SUBD F8 F6 F2 7 DIVD F10 F0 F6 8 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F2 R3 Mult1 Yes Mult F0 F2 F4 Integer Yes Mult2 Add Yes Sub F8 F6 F2 Integer Yes Divide Yes Div F10 F0 F6 Mult1 Yes 8 FU Mult1 Integer Add Divide 11

Scoreboard Example: Cycle 8b (Second half of clock cycle) Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 SUBD F8 F6 F2 7 DIVD F10 F0 F6 8 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Yes Mult F0 F2 F4 Yes Yes Mult2 Add Yes Sub F8 F6 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 Yes 8 FU Mult1 Add Divide Scoreboard Example: Cycle 9 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 DIVD F10 F0 F6 8 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk te Remaining Integer 10 Mult1 Yes Mult F0 F2 F4 Yes Yes Mult2 2Add Yes Sub F8 F6 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 Yes 9 FU Mult1 Add Divide Read operands for MULT & SUB? ue ADDD? 12

Scoreboard Example: Cycle 10 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 DIVD F10 F0 F6 8 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 9 Mult1 Yes Mult F0 F2 F4 Mult2 1Add Yes Sub F8 F6 F2 Divide Yes Div F10 F0 F6 Mult1 Yes 10 FU Mult1 Add Divide Scoreboard Example: Cycle 11 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 11 DIVD F10 F0 F6 8 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 8 Mult1 Yes Mult F0 F2 F4 Mult2 0Add Yes Sub F8 F6 F2 Divide Yes Div F10 F0 F6 Mult1 Yes 11 FU Mult1 Add Divide 13

Scoreboard Example: Cycle 12 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 7 Mult1 Yes Mult F0 F2 F4 Mult2 Add Divide Yes Div F10 F0 F6 Mult1 Yes 12 FU Mult1 Divide Read operands for DIVD? Scoreboard Example: Cycle 13 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 13 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 6 Mult1 Yes Mult F0 F2 F4 Mult2 Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 Yes 13 FU Mult1 Add Divide 14

Scoreboard Example: Cycle 14 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 13 14 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 5 Mult1 Yes Mult F0 F2 F4 Mult2 2 Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 Yes 14 FU Mult1 Add Divide Scoreboard Example: Cycle 15 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 13 14 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 4 Mult1 Yes Mult F0 F2 F4 Mult2 1 Add Yes Add F6 F8 F2 Divide Yes Div F10 F0 F6 Mult1 Yes 15 FU Mult1 Add Divide 15

Scoreboard Example: Cycle 16 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 13 14 16 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 3 Mult1 Yes Mult F0 F2 F4 Mult2 0 Add Yes Add F6 F8 F2 Divide Yes Div F10 F0 F6 Mult1 Yes 16 FU Mult1 Add Divide Scoreboard Example: Cycle 17 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 13 14 16 WAR Hazard! Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 2 Mult1 Yes Mult F0 F2 F4 Mult2 Add Yes Add F6 F8 F2 Divide Yes Div F10 F0 F6 Mult1 Yes 17 FU Mult1 Add Divide Why not write result of ADD??? 16

Scoreboard Example: Cycle 18 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 13 14 16 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 1 Mult1 Yes Mult F0 F2 F4 Mult2 Add Yes Add F6 F8 F2 Divide Yes Div F10 F0 F6 Mult1 Yes 18 FU Mult1 Add Divide Scoreboard Example: Cycle 19 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 19 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 13 14 16 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer 0 Mult1 Yes Mult F0 F2 F4 Mult2 Add Yes Add F6 F8 F2 Divide Yes Div F10 F0 F6 Mult1 Yes 19 FU Mult1 Add Divide 17

Scoreboard Example: Cycle 20 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 19 20 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 ADDD F6 F8 F2 13 14 16 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Mult2 Add Yes Add F6 F8 F2 Divide Yes Div F10 F0 F6 Yes Yes 20 FU Add Divide Scoreboard Example: Cycle 21 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 19 20 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 21 ADDD F6 F8 F2 13 14 16 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Mult2 Add Yes Add F6 F8 F2 Divide Yes Div F10 F0 F6 Yes Yes 21 FU Add Divide WAR Hazard is now gone... 18

Scoreboard Example: Cycle 22 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 19 20 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 21 ADDD F6 F8 F2 13 14 16 22 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Mult2 Add 39 Divide Yes Div F10 F0 F6 22 FU Divide Faster than light computation (skip a couple of cycles) 19

Scoreboard Example: Cycle 61 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 19 20 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 21 61 ADDD F6 F8 F2 13 14 16 22 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Mult2 Add 0 Divide Yes Div F10 F0 F6 61 FU Divide Scoreboard Example: Cycle 62 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 19 20 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 21 61 62 ADDD F6 F8 F2 13 14 16 22 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Mult2 Add Divide 62 FU 20

Review: Scoreboard Example: Cycle 62 Instruction status: Read Exec Write Instruction j k ue Oper Comp Result LD F6 34+ R2 1 2 3 4 LD F2 45+ R3 5 6 7 8 MULTD F0 F2 F4 6 9 19 20 SUBD F8 F6 F2 7 9 11 12 DIVD F10 F0 F6 8 21 61 62 ADDD F6 F8 F2 13 14 16 22 Functional unit status: dest S1 S2 FU FU Fj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Mult1 Mult2 Add Divide 62 FU In-order issue; out-of-order execute & commit Scoreboarding Review LD F6 34+ R2 LD F2 45+ R3 MULTDF0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 1 2 3 4 5 6 7 8 9 10 11 12 13 LD F6, 34(R2) Rd Ex Wb LD F2, 45(R3) Rd Ex Wb MULTD F0, F2, F4 Rd M1 M2 M3 M4 SUBD F8, F6, F2 Rd A1 A2 Wb DIVD F10, F0, F6 ADDD F6, F8, F2 21

Scoreboarding Review LD F6 34+ R2 LD F2 45+ R3 MULTDF0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 11 12 13 14 15 16 17 18 19 20 21 22. 62 LD F6, 34(R2) LD F2, 45(R3) MULTD F0, F2, F4 M2 M3 M4 M5 M6 M7 M8 M9 M10 Wb SUBD F8, F6, F2 A2 Wb DIVD F10, F0, F6 Rd D1 Wb ADDD F6, F8, F2 Rd A1 A2 A2 A2 A2 A2 A2 Wb Scoreboarding Limitations Number and type of functional units Number of instruction buffer entries (scoreboard size) Amount of application ILP (RAW hazards) Presence of antidependences (WAR) and output dependences (WAW) In-order issue for WAW/structural hazards limits scheduler WAR stalls are critical for loops (hardware loop unrolling) 22

A Dynamic Scheduling Algorithm: Tomasulo s For IBM 360/91 (before caches!) Goal: High Performance without special compilers Small number of floating point registers (4 in 360) prevented interesting compiler scheduling of operations This led Tomasulo to try to figure out how to get more effective registers renaming in hardware! Why Study 1966 Computer? The descendants of this have flourished! Alpha 21264, HP 8000, MIPS 10000, Pentium III, PowerPC 604, Gist of Tomasulo s Keeps track of when operands are available Minimizes RAW hazards Renames registers reservation stations Dynamically change register name to avoid WAR/WAW hazards Compiler can eliminate some WAR/WAW false hazards, but not all t enough registers Hazards across branches (common!). Eliminate on taken, or fall through, but not both Architectural registers are names not locations Many more locations (reservation stations or physical registers) than names (logical or architectural registers) Dynamically map names to locations 23

Gist of Tomasulo s A * B C A, B: True data dependencies *: Structural dependency C: Output/anti dependency due to register reuse A, B: Gets them once they are ready *: Tracks if they are available C: Rename through reservation stations Tomasulo Organization From Mem FP Op Queue Load Buffers Load1 Load2 Load3 Load4 Load5 Load6 FP Registers Store Buffers Add1 Add2 Add3 Mult1 Mult2 FP FP adders adders Reservation Stations FP FP multipliers To Mem Common Data Bus (CDB) 24

Tomasulo s stages ue (Get instruction from FP operation Queue) If empty reservation station available (eliminates WAR, WAW) Else, stall (structural hazard) Execute (Operate on operands) If operands are not yet ready (as reflected in the reservations table) Monitor common data bus until it is produced, then execute (stalls for RAW hazards) Writes results (Execution done) When FU is done, write results onto common data bus, into any waiting reservation stations and registers What s different with a common data bus rmal data bus data + destination ( go to bus) Common data bus data + source ( come from bus) 64 bits of data + 4 bits of Functional Unit source address Write if matches expected Functional Unit (produces result) 25

Walkthrough Example LD LD MULD SUBD DIVD ADDD F6, 34+, R2 F2, 45+, R3 F0, F2, F4 F8, F6, F2 F10, F0, F6 F6, F8, F2 Tomasulo s builds dependency graph on the fly Reservation Station Components Op: Operation that this unit performs (+, *,..) Vj, Vk: Value of Source operands Store buffers has V field, result to be stored Qj, Qk: Reservation stations producing the values te: Qj,Qk=0 ready Store buffers only have Qi for the res. station producing result Busy: Indicates reservation station or FU is busy Register result status Indicates which functional unit will write each register, if one exists. Blank when no pending instructions that will write that register. 26

Assumed latencies ADD.D 2 cycles in FU L.D 1 cycle AGEN 1 cycle memory access MUL.D 10 cycles in FU DIV.D 40 cycles in FU Tomasulo Example Instruction stream Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 Load1 LD F2 45+ R3 Load2 MULTD F0 F2 F4 Load3 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk FU count down Add1 Add2 Add3 Mult1 Mult2 0 FU Clock cycle counter 3 Load/Buffers 3 FP Adder R.S. 2 FP Mult R.S. 27

Tomasulo Example Cycle 1 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 Load1 Yes 34+R2 LD F2 45+ R3 Load2 MULTD F0 F2 F4 Load3 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 Mult2 1 FU Load1 Tomasulo Example Cycle 2 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 Load1 Yes 34+R2 LD F2 45+ R3 2 Load2 Yes 45+R3 MULTD F0 F2 F4 Load3 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 Mult2 2 FU Load2 Load1 te: Can have multiple loads outstanding 28

Tomasulo Example Cycle 3 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 Load1 Yes 34+R2 LD F2 45+ R3 2 Load2 Yes 45+R3 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 Yes MULTD R(F4) Load2 Mult2 3 FU Mult1 Load2 Load1 te: registers names are removed ( renamed ) in Reservation Stations; MULT issued Load1 completing; what is waiting for Load1? Tomasulo Example Cycle 4 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 Load2 Yes 45+R3 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Yes SUBD M(A1) Load2 Add2 Add3 Mult1 Yes MULTD R(F4) Load2 Mult2 4 FU Mult1 Load2 M(A1) Add1 Load2 completing; what is waiting for Load2? 29

Tomasulo Example Cycle 5 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 DIVD F10 F0 F6 5 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk 2 Add1 Yes SUBD M(A1) M(A2) Add2 Add3 10 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 5 FU Mult1 M(A2) M(A1) Add1 Mult2 Timer starts down for Add1, Mult1 Tomasulo Example Cycle 6 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk 1 Add1 Yes SUBD M(A1) M(A2) Add2 Yes ADDD M(A2) Add1 Add3 9Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 6 FU Mult1 M(A2) Add2 Add1 Mult2 ue ADDD here despite name dependency on F6? 30

Tomasulo Example Cycle 7 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 7 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk 0 Add1 Yes SUBD M(A1) M(A2) Add2 Yes ADDD M(A2) Add1 Add3 8Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 7 FU Mult1 M(A2) Add2 Add1 Mult2 Add1 (SUBD) completing; what is waiting for it? Tomasulo Example Cycle 8 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 2 Add2 Yes ADDD (M-M) M(A2) Add3 7Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 8 FU Mult1 M(A2) Add2 (M-M) Mult2 31

Tomasulo Example Cycle 9 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 1 Add2 Yes ADDD (M-M) M(A2) Add3 6Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 9 FU Mult1 M(A2) Add2 (M-M) Mult2 Tomasulo Example Cycle 10 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 10 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 0 Add2 Yes ADDD (M-M) M(A2) Add3 5Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 10 FU Mult1 M(A2) Add2 (M-M) Mult2 Add2 (ADDD) completing; what is waiting for it? 32

Tomasulo Example Cycle 11 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 4Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 11 FU Mult1 M(A2) (M-M+M(M-M) Mult2 Write result of ADDD here? All quick instructions complete in this cycle! Tomasulo Example Cycle 12 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 3Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 12 FU Mult1 M(A2) (M-M+M(M-M) Mult2 thing happens, waiting for Mult1 to complete 33

Tomasulo Example Cycle 13 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 2Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 13 FU Mult1 M(A2) (M-M+M(M-M) Mult2 thing happens, waiting for Mult1 to complete Tomasulo Example Cycle 14 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 1Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 14 FU Mult1 M(A2) (M-M+M(M-M) Mult2 thing happens, waiting for Mult1 to complete 34

Tomasulo Example Cycle 15 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 15 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 0Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 15 FU Mult1 M(A2) (M-M+M(M-M) Mult2 Mult1 (MULTD) completing; what is waiting for it? Tomasulo Example Cycle 16 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 15 16 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 40 Mult2 Yes DIVD M*F4 M(A1) 16 FU M*F4 M(A2) (M-M+M(M-M) Mult2 Just waiting for Mult2 (DIVD) to complete 35

Faster than light computation (skip a couple of cycles) Tomasulo Example Cycle 55 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 15 16 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 1Mult2 Yes DIVD M*F4 M(A1) 55 FU M*F4 M(A2) (M-M+M(M-M) Mult2 36

Tomasulo Example Cycle 56 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 15 16 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 56 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 0Mult2 Yes DIVD M*F4 M(A1) 56 FU M*F4 M(A2) (M-M+M(M-M) Mult2 Mult2 (DIVD) is completing; what is waiting for it? Tomasulo Example Cycle 57 Instruction status: Exec Write Instruction j k ue Comp Result Busy Address LD F6 34+ R2 1 3 4 Load1 LD F2 45+ R3 2 4 5 Load2 MULTD F0 F2 F4 3 15 16 Load3 SUBD F8 F6 F2 4 7 8 DIVD F10 F0 F6 5 56 57 ADDD F6 F8 F2 6 10 11 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 Mult2 Yes DIVD M*F4 M(A1) 56 FU M*F4 M(A2) (M-M+M(M-M) Result Once again: In-order issue, out-of-order execution and out-of-order completion. 37

Tomasulo Review LD F6 34+ R2 LD F2 45+ R3 MULTDF0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 1 2 3 4 5 6 7 8 9 10 11 12 13 LD F6, 34(R2) Ex M Wb LD F2, 45(R3) Ex M Wb MULTD F0, F2, F4 M1 M2 M3 M4 M5 M6 M7 M8 SUBD F8, F6, F2 A1 A2 Wb DIVD F10, F0, F6 ADDD F6, F8, F2 A1 A2 Wb Tomasulo Review LD F6 34+ R2 LD F2 45+ R3 MULTDF0 F2 F4 SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 11 12 13 14 15 16 17 18 19 20 21 22. 57 LD F6, 34(R2) LD F2, 45(R3) MULTD F0, F2, F4 M6 M7 M8 M9 M10 Wb SUBD F8, F6, F2 DIVD F10, F0, F6 D1 D2 D3 D4 D5 D6 Wb ADDD F6, F8, F2 38

Tomasulo s scheme offers two major advantages Distribution of the hazard detection logic Distributed reservation stations and the CDB If multiple instructions waiting on single result, and each instruction has other operands, then instructions can be released simultaneously by broadcast on CDB Else if a centralized register file were used, the units would have to read their results from the registers when register buses are available. Elimination of stalls for WAW and WAR hazards How? Tomasulo Loop Example Loop: LD F0 0 R1 MULTD F4 F0 F2 SD F4 0 R1 SUBI R1 R1 #8 BNEZ R1 Loop This time assume Multiply takes 4 clocks Assume 1st load takes 8 clocks (L1 cache miss), 2nd load takes 1 clock (hit) To be clear, will show clocks for SUBI, BNEZ Reality: integer instructions ahead of floating point instructions Show 2 iterations 39

Loop Example Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 Load1 1 MULTD F4 F0 F2 Load2 1 SD F4 0 R1 Load3 Iteration 2 LD F0 0 R1 Store1 2 MULTD F4 F0 F2 Store2 Count 2 SD F4 0 R1 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Added Store Buffers Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Instruction Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 0 80 Fu Value of Register used for address, iteration control Loop Example Cycle 1 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 Load2 Load3 Store1 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 1 80 Fu Load1 40

Loop Example Cycle 2 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Load3 Store1 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 2 80 Fu Load1 Mult1 Loop Example Cycle 3 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 1 SD F4 0 R1 3 Load3 Store1 Yes 80 Mult1 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 3 80 Fu Load1 Mult1 Implicit renaming sets up data flow graph 41

Loop Example Cycle 4 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 1 SD F4 0 R1 3 Load3 Store1 Yes 80 Mult1 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 4 80 Fu Load1 Mult1 Dispatching SUBI Instruction (not in FP queue) Loop Example Cycle 5 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 1 SD F4 0 R1 3 Load3 Store1 Yes 80 Mult1 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 5 72 Fu Load1 Mult1 And, BNEZ instruction (not in FP queue) 42

Loop Example Cycle 6 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 2 LD F0 0 R1 6 Store1 Yes 80 Mult1 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 6 72 Fu Load2 Mult1 tice that F0 never sees Load from location 80 Loop Example Cycle 7 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 2 LD F0 0 R1 6 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 Yes Multd R(F2) Load2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 7 72 Fu Load2 Mult2 Register file completely detached from computation First and Second iteration completely overlapped 43

Loop Example Cycle 8 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 2 LD F0 0 R1 6 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 Yes Multd R(F2) Load2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 8 72 Fu Load2 Mult2 Loop Example Cycle 9 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 2 LD F0 0 R1 6 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 Yes Multd R(F2) Load2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 9 72 Fu Load2 Mult2 Load1 completing: who is waiting? te: Dispatching SUBI 44

Loop Example Cycle 10 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 2 LD F0 0 R1 6 10 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 4 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 Mult2 Yes Multd R(F2) Load2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 10 64 Fu Load2 Mult2 Load2 completing: who is waiting? te: Dispatching BNEZ Loop Example Cycle 11 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 Load2 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 3 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 4 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 11 64 Fu Load3 Mult2 Next load in sequence 45

Loop Example Cycle 12 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 Load2 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 2 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 3 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 12 64 Fu Load3 Mult2 Why not issue third multiply? Loop Example Cycle 13 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 Load2 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 1 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 2 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 13 64 Fu Load3 Mult2 Why not issue third store? 46

Loop Example Cycle 14 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 14 Load2 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 0 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 1 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 14 64 Fu Load3 Mult2 Mult1 completing. Who is waiting? Loop Example Cycle 15 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 14 15 Load2 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 Yes 80 [80]*R2 2 MULTD F4 F0 F2 7 15 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 SUBI R1 R1 #8 0 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 15 64 Fu Load3 Mult2 Mult2 completing. Who is waiting? 47

Loop Example Cycle 16 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 14 15 Load2 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 Yes 80 [80]*R2 2 MULTD F4 F0 F2 7 15 16 Store2 Yes 72 [72]*R2 2 SD F4 0 R1 8 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 4 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 16 64 Fu Load3 Mult1 Loop Example Cycle 17 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 14 15 Load2 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 Yes 80 [80]*R2 2 MULTD F4 F0 F2 7 15 16 Store2 Yes 72 [72]*R2 2 SD F4 0 R1 8 Store3 Yes 64 Mult1 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 17 64 Fu Load3 Mult1 48

Loop Example Cycle 18 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 14 15 Load2 1 SD F4 0 R1 3 18 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 Yes 80 [80]*R2 2 MULTD F4 F0 F2 7 15 16 Store2 Yes 72 [72]*R2 2 SD F4 0 R1 8 Store3 Yes 64 Mult1 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 18 64 Fu Load3 Mult1 Loop Example Cycle 19 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 1 MULTD F4 F0 F2 2 14 15 Load2 1 SD F4 0 R1 3 18 19 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 2 MULTD F4 F0 F2 7 15 16 Store2 Yes 72 [72]*R2 2 SD F4 0 R1 8 19 Store3 Yes 64 Mult1 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 19 56 Fu Load3 Mult1 49

Loop Example Cycle 20 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 10 Load1 Yes 56 1 MULTD F4 F0 F2 2 14 15 Load2 1 SD F4 0 R1 3 18 19 Load3 Yes 64 2 LD F0 0 R1 6 10 11 Store1 2 MULTD F4 F0 F2 7 15 16 Store2 2 SD F4 0 R1 8 19 20 Store3 Yes 64 Mult1 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MULTD F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 20 56 Fu Load1 Mult1 Once again: In-order issue, out-of-order execution and out-of-order completion. Why can Tomasulo overlap iterations of loops? Register renaming Replace static register names from code with dynamic register locations Load/store buffers or reservation stations instead of registers Increases effective size of register file Multiple iterations use different physical destinations for registers (dynamic loop unrolling) Permit instruction issue to advance past integer control flow operations Also buffer old values of registers Crucial: integer units must get ahead of floating point unit so we can issue multiple iterations 50

Tomasulo review Reservation Stations Distribute RAW hazard detection Renaming eliminates WAW hazards Buffering values in Reservation Stations removes WARs Tag match in CDB requires many associative compares Common Data Bus Achilles heal of Tomasulo Multiple writebacks (multiple CDBs) expensive Load/Store reordering Load address compared with store address in store buffer Tomasulo drawbacks Key: Complexity Many associative comparisons at reservation stations Many associative stores (CDB) at high speed Performance limited by Common Data Bus Each CDB must go to multiple functional units high capacitance, high wiring density Number of functional units that can complete per cycle limited to one! Multiple CDBs are very expensive n-precise interrupts! Let s look at speculative tomasulo s next 51

Tomasulo vs. Scoreboarding explicit checking for WAW or WAR hazards CDB broadcasts results rather than waiting on registers Loads/Store are treated like basic FUs Distributed vs. Centralized control Dynamic Scheduling: Multiple ue Assumptions Both a floating-point and an integer operation can be issued in every cycle Even if they are dependent Two CDBs Branches single issue but branch prediction is perfect 52

Multi-ue Example Cycle 1 Instruction status: Exec Write ITER Instruction j k uecompresult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MUL F4 F0 F2 1 Load2 Load3 Store1 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 LD F0 0 R1 Add2 MUL F4 F0 F2 Add3 SD F4 0 R1 Mult1 Yes Add R(F2) Load1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 1 80 Fu Load1 Mult1 Multi-ue Example Cycle 2 Instruction status: Exec Write ITER Instruction j k uecompresult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MUL F4 F0 F2 1 Load2 1 SD F4 0 R1 2 Load3 1 SUBI R1 R1 #8 2 Store1 Yes 80 Mult1 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1-80 8 LD F0 0 R1 Add2 MUL F4 F0 F2 Add3 SD F4 0 R1 Mult1 * R(F2) Load1 SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 2 80 Fu Load1 Mult1 53

Multi-ue Example Cycle 3 Instruction status: Exec Write ITER Instruction j k uecompresult Busy Addr Fu 1 LD F0 0 R1 1 3 Load1 80 1 MUL F4 F0 F2 1 Load2 1 SD F4 0 R1 2 Load3 1 SUBI R1 R1 #8 2 Store1 Yes 80 Mult1 1 BNEZ 3 Store2 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: 1 Add1 Yes - 80 8 LD F0 0 R1 Add2 MUL F4 F0 F2 Add3 SD F4 0 R1 10 Mult1 Yes * M(80) R(F2) SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 3 80 Fu M(80) Mult1 Load1 completed; MULT and SUB executing; BNEZ issued Multi-ue Example Cycle 4 Instruction status: Exec Write ITER Instruction j k ue CompResult Busy Addr Fu 1 LD F0 0 R1 1 3 4 Load1 80 1 MUL F4 F0 F2 1 Load2 1 SD F4 0 R1 2 Load3 1 SUBI R1 R1 #8 2 4 Store1 Yes 80 Mult1 1 BNEZ 3 Store2 2 LD F0 0 R1 4 Store3 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: 0 Add1 Yes - 80 8 LD F0 0 R1 Add2 MUL F4 F0 F2 Add3 SD F4 0 R1 9 Mult1 Yes * M(80) R(F2) SUBI R1 R1 #8 Mult2 BNEZ R1 Loop Register result status Clock R1 F0 F2 F4 F6 F8 F10 F12... F30 4 80 Fu M(80) Mult1 LD in Iteration 2 can be issued but not executed 54