CPE 631 Lecture 10: Instruction Level Parallelism and Its Dynamic Exploitation
|
|
- Felicity Snow
- 6 years ago
- Views:
Transcription
1 Lecture 10: Instruction Level Parallelism and Its Dynamic Exploitation Aleksandar Milenkovic, Electrical and Computer Engineering University of Alabama in Huntsville Outline Instruction Level Parallelism (ILP) Recap: Data Dependencies Extended MIPS Pipeline and Hazards Dynamic scheduling with a scoreboard 16/02/2004 UAH- 2 Aleksandar Milenkovich 1
2 ILP: Concepts and Challenges ILP (Instruction Level Parallelism) overlap execution of unrelated instructions Techniques that increase amount of parallelism exploited among instructions reduce impact of data and control hazards increase processor ability to exploit parallelism Pipeline CPI = Ideal pipeline CPI + Structural stalls + RAW stalls + WAR stalls + WAW stalls + Control stalls Reducing each of the terms of the right-hand side minimize CPI and thus increase instruction throughput 16/02/2004 UAH- 3 Two approaches to exploit parallelism Dynamic techniques largely depend on hardware to locate the parallelism Static techniques relay on software 16/02/2004 UAH- 4 Aleksandar Milenkovich 2
3 Techniques to exploit parallelism Technique (Section in the textbook) Reduces Forwarding and bypassing (Section A.2) Delayed branches (A.2) Basic dynamic scheduling (A.8) Dynamic scheduling with register renaming (3.2) Dynamic branch prediction (3.4) Issuing multiple instruction per cycle (3.6) Speculation (3.7) Dynamic memory disambiguation (3.2, 3.7) Loop Unrolling (4.1) Basic compiler pipeline scheduling (A.2, 4.1) Compiler dependence analysis (4.4) Software pipelining and trace scheduling (4.3) Data hazard (DH) stalls Control hazard stalls DH stalls (RAW) WAR and WAW stalls CH stalls Ideal CPI Data and control stalls RAW stalls w. memory CH stalls DH stalls Ideal CPI, DH stalls Ideal CPI and DH stalls Compiler speculation (4.4) Ideal CPI, and D/CH stalls 16/02/2004 UAH- 5 Dynamically Scheduled Pipelines Aleksandar Milenkovich 3
4 Overcoming Data Hazards with Dynamic Scheduling Why in HW at run time? Works when can t know real dependence at compile time Simpler compiler Code for one machine runs well on another Example DIV.D F0,F2,F4 ADD.D F10,F0,F8 SUB.D F12,F8,F12 SUB.D cannot execute because the dependence of ADD.D on DIV.D causes the pipeline to stall; yet SUBD is not data dependent on anything! Key idea: Allow instructions behind stall to proceed 16/02/2004 UAH- 7 Overcoming Data Hazards with Dynamic Scheduling (cont d) Enables out-of-order execution => out-of-order completion Out-of-order execution divides ID stage: 1. Issue decode instructions, check for structural hazards 2. Read operands wait until no data hazards, then read operands Scoreboarding technique for allowing instructions to execute out of order when there are sufficient resources and no data dependencies (CDC 6600, 1963) 16/02/2004 UAH- 8 Aleksandar Milenkovich 4
5 Scoreboarding Implications Out-of-order completion => WAR, WAW hazards? DIV.D F0,F2,F4 ADD.D F10,F0,F8 SUB.D F8,F8,F12 DIV.D F0,F2,F4 ADD.D F10,F0,F8 SUB.D F10,F8,F12 Solutions for WAR Queue both the operation and copies of its operands Read registers only during Read Operands stage For WAW, must detect hazard: stall until other completes Need to have multiple instructions in execution phase => multiple execution units or pipelined execution units Scoreboard keeps track of dependencies, state or operations Scoreboard replaces ID, EX, WB with 4 stages 16/02/2004 UAH- 9 Four Stages of Scoreboard Control ID1: Issue decode instructions & check for structural hazards ID2: Read operands wait until no data hazards, then read operands EX: Execute operate on operands; when the result is ready, it notifies the scoreboard that it has completed execution WB: Write results finish execution; the scoreboard checks for WAR hazards. If none, it writes results. If WAR, then it stalls the instruction DIV.D F0,F2,F4 ADD.D F10,F0,F8 SUB.D F8,F8,F12 Scoreboarding stalls the the SUBD in its write result stage until ADDD reads its operands 16/02/2004 UAH- 10 Aleksandar Milenkovich 5
6 Four Stages of Scoreboard Control 1. Issue decode instructions & check for structural hazards (ID1) If a functional unit for the instruction is free and no other active instruction has the same destination register (WAW), the scoreboard issues the instruction to the functional unit and updates its internal data structure. If a structural or WAW hazard exists, then the instruction issue stalls, and no further instructions will issue until these hazards are cleared. 2. Read operands wait until no data hazards, then read operands (ID2) A source operand is available if no earlier issued active instruction is going to write it, or if the register containing the operand is being written by a currently active functional unit. When the source operands are available, the scoreboard tells the functional unit to proceed to read the operands from the registers and begin execution. The scoreboard resolves RAW hazards dynamically in this step, and instructions may be sent into execution out of order. 16/02/2004 UAH- 11 Four Stages of Scoreboard Control 3. Execution operate on operands (EX) The functional unit begins execution upon receiving operands. When the result is ready, it notifies the scoreboard that it has completed execution. 4. Write result finish execution (WB) Once the scoreboard is aware that the functional unit has completed execution, the scoreboard checks for WAR hazards. If none, it writes results. If WAR, then it stalls the instruction. Example: DIV.D ADD.D SUB.D F0,F2,F4 F10,F0,F8 F8,F8,F14 CDC 6600 scoreboard would stall SUBD until ADD.D reads operands 16/02/2004 UAH- 12 Aleksandar Milenkovich 6
7 Three Parts of the Scoreboard 1. Instruction status which of 4 steps the instruction is in (Capacity = window size) 2. Functional unit status Indicates the state of the functional unit (FU). 9 fields for each functional unit Busy Indicates whether the unit is busy or not Op Operation to perform in the unit (e.g., + or ) Fi Destination register Fj, Fk Source-register numbers Qj, Qk Functional units producing source registers Fj, Fk Rj, Rk Flags indicating when Fj, Fk are ready 3. Indicates which functional unit will write each register, if one exists. Blank when no pending instructions will write that register 16/02/2004 UAH- 13 MIPS with a Scoreboard Registers FP Mult FP Mult FP Div FP Div Add1 Add2 Add3 FP Div Control/ Status Scoreboard Control/ Status 16/02/2004 UAH- 14 Aleksandar Milenkovich 7
8 Detailed Scoreboard Pipeline Control Instruction status Issue Read operands Execution complete Write result Wait until Not busy (FU) and not result (D) Rj and Rk Functional unit done f((fj( f )?Fi(FU) or Rj( f )=No) & (Fk( f )?Fi(FU) or Rk( f )=No)) Bookkeeping Busy(FU) yes; Op(FU) op; Fi(FU) D ; Fj(FU) S1 ; Fk(FU) S2 ; Qj Result( S1 ); Qk Result( S2 ); Rj not Qj; Rk not Qk; Result( D ) FU; Rj No; Rk No f(if Qj(f)=FU then Rj(f) Yes); f(if Qk(f)=FU then Rj(f) Yes); Result(Fi(FU)) 0; Busy(FU) No 16/02/2004 UAH- 15 Scoreboard Example Instruction status Read ExecutionWrite Instruction j k Issue operandcomplete Result L.D F6 34+ R2 L.D F2 45+ R3 MUL.D F0 F2 F4 SUB.D F8 F6 F2 DIV.D F10 F0 F6 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for FU for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No Mult1 No Add No Divide No FU 16/02/2004 UAH- 16 Aleksandar Milenkovich 8
9 Scoreboard Example: Cycle 1 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R2 1 L.D F2 45+ R3 MUL.D F0 F2 F4 SUB.D F8 F6 F2 DIV.D F10 F0 F6 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F6 R2 Yes Mult1 No Add No Divide No 1 FU Integer Issue 1 st L.D! 16/02/2004 UAH- 17 Scoreboard Example: Cycle 2 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R2 1 2 L.D F2 45+ R3 MUL.D F0 F2 F4 SUB.D F8 F6 F2 DIV.D F10 F0 F6 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F6 R2 Yes Mult1 No Add No Divide No 2 FU Integer Issue 2 nd L.D? Structural hazard! No further instructions will issue! 16/02/2004 UAH- 18 Aleksandar Milenkovich 9
10 Scoreboard Example: Cycle 3 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R3 MUL.D F0 F2 F4 SUB.D F8 F6 F2 DIV.D F10 F0 F6 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F6 R2 Yes Mult1 No Add No Divide No 3 FU Integer Issue MUL.D? 16/02/2004 UAH- 19 Scoreboard Example: Cycle 4 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R3 MUL.D F0 F2 F4 SUB.D F8 F6 F2 DIV.D F10 F0 F6 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F6 R2 Yes Mult1 No Add No Divide No 4 FU Integer Check for WAR hazards! If none, write result! 16/02/2004 UAH- 20 Aleksandar Milenkovich 10
11 Scoreboard Example: Cycle 5 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R3 5 MUL.D F0 F2 F4 SUB.D F8 F6 F2 DIV.D F10 F0 F6 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F2 R3 Yes Mult1 No Add No Divide No 5 FU Integer Issue 2 nd L.D! 16/02/2004 UAH- 21 Scoreboard Example: Cycle 6 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R3 5 6 MUL.D F0 F2 F4 6 SUB.D F8 F6 F2 DIV.D F10 F0 F6 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F2 R3 Yes Mult1 Yes Mult F0 F2 F4 Integer No Yes Add No Divide No 6 FU Mult1 Integer Issue MUL.D! 16/02/2004 UAH- 22 Aleksandar Milenkovich 11
12 Scoreboard Example: Cycle 7 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 SUB.D F8 F6 F2 7 DIV.D F10 F0 F6 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F2 R3 Yes Mult1 Yes Mult F0 F2 F4 Integer No Yes Add Yes Sub F8 F6 F2 Integer Yes No Divide No 7 FU Mult1 Integer Add Issue SUB.D! 16/02/2004 UAH- 23 Scoreboard Example: Cycle 8 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 SUB.D F8 F6 F2 7 DIV.D F10 F0 F6 8 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Yes Load F2 R3 Yes Mult1 Yes Mult F0 F2 F4 Integer No Yes Add Yes Sub F8 F6 F2 Integer Yes No Divide Yes Div F10 F0 F6 Mult1 No Yes 8 FU Mult1 Integer Add Divide Issue DIV.D! 16/02/2004 UAH- 24 Aleksandar Milenkovich 12
13 Scoreboard Example: Cycle 9 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 9 SUB.D F8 F6 F2 7 9 DIV.D F10 F0 F6 8 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 10 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes 2 Add Yes Sub F8 F6 F2 Integer Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 9 FU Mult1 Add Divide Read operands for MUL.D and SUB.D! Assume we can feed Mult1 and Add units in the same clock cycle. Issue ADD.D? Structural Hazard (unit is busy)! 16/02/2004 UAH- 25 Scoreboard Example: Cycle 11 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 9 SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 8 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes 0 Add Yes Sub F8 F6 F2 Integer Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 11 FU Mult1 Add Divide Last cycle of SUB.D execution. 16/02/2004 UAH- 26 Aleksandar Milenkovich 13
14 Scoreboard Example: Cycle 12 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 9 SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F2 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 7 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes Add Yes Sub F8 F6 F2 Integer Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 12 FU Mult1 Add Divide Check WAR on F8. Write F8. 16/02/2004 UAH- 27 Scoreboard Example: Cycle 13 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 9 SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F2 13 Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 6 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 13 FU Mult1 Add Divide Issue ADD.D! 16/02/2004 UAH- 28 Aleksandar Milenkovich 14
15 Scoreboard Example: Cycle 14 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 9 SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F Functional unit status dest S1 S2 FU for FU for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 5 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes 2 Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 14 FU Mult1 Add Divide Read operands for ADD.D! 16/02/2004 UAH- 29 Scoreboard Example: Cycle 15 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 9 SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 4 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes 1 Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 14 FU Mult1 Add Divide Read operands for ADD.D! 16/02/2004 UAH- 30 Aleksandar Milenkovich 15
16 Scoreboard Example: Cycle 16 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 9 SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 3 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes 0 Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 16 FU Mult1 Add Divide 16/02/2004 UAH- 31 Scoreboard Example: Cycle 17 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F4 6 9 SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 2 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 17 FU Mult1 Add Divide Why cannot write F6? 16/02/2004 UAH- 32 Aleksandar Milenkovich 16
17 Scoreboard Example: Cycle 19 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No 0 Mult1 Yes Mult F0 F2 F4 Integer Yes Yes Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 17 FU Mult1 Add Divide 16/02/2004 UAH- 33 Scoreboard Example: Cycle 20 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F SUB.D F8 F6 F DIV.D F10 F0 F6 8 ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No Mult1 Yes Mult F0 F2 F4 Integer Yes Yes Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 No Yes 20 FU Mult1 Add Divide 16/02/2004 UAH- 34 Aleksandar Milenkovich 17
18 Scoreboard Example: Cycle 21 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F SUB.D F8 F6 F DIV.D F10 F0 F ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No Mult1 No Add Yes Add F6 F8 F2 Yes Yes Divide Yes Div F10 F0 F6 Mult1 Yes Yes 21 FU Add Divide 16/02/2004 UAH- 35 Scoreboard Example: Cycle 22 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F SUB.D F8 F6 F DIV.D F10 F0 F ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No Mult1 No Add Yes Add F6 F8 F2 Yes Yes 40 Divide Yes Div F10 F0 F6 Mult1 Yes Yes 22 FU Add Divide Write F6? 16/02/2004 UAH- 36 Aleksandar Milenkovich 18
19 Scoreboard Example: Cycle 61 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F SUB.D F8 F6 F DIV.D F10 F0 F ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No Mult1 No Add No 0 Divide Yes Div F10 F0 F6 Mult1 Yes Yes 61 FU Divide Write F6? 16/02/2004 UAH- 37 Scoreboard Example: Cycle 62 Instruction status Read ExecutioWrite Instruction j k Issue operandcompletresult L.D F6 34+ R L.D F2 45+ R MUL.D F0 F2 F SUB.D F8 F6 F DIV.D F10 F0 F ADD.D F6 F8 F Functional unit status dest S1 S2 FU for jfu for kfj? Fk? Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer No Mult1 No Add No Divide Yes Div F10 F0 F6 Mult1 Yes Yes 62 FU Divide Write F6? 16/02/2004 UAH- 38 Aleksandar Milenkovich 19
20 Scoreboard Results For the CDC % improvement for Fortran 150% improvement for hand coded assembly language cost was similar to one of the functional units surprisingly low bulk of cost was in the extra busses Still this was in ancient time no caches & no main semiconductor memory no software pipelining compilers? So, why is it coming back performance via ILP 16/02/2004 UAH- 39 Scoreboard Limitations Amount of parallelism among instructions can we find independent instructions to execute Number of scoreboard entries how far ahead the pipeline can look for independent instructions (we assume a window does not extend beyond a branch) Number and types of functional units avoid structural hazards Presence of antidependences and output dependences WAR and WAW stalls become more important 16/02/2004 UAH- 40 Aleksandar Milenkovich 20
21 Tomasulo s Algorithm Used in IBM 360/91 FPU (before caches) Goal: high FP performance without special compilers Conditions: Small number of floating point registers (4 in 360) prevented interesting compiler scheduling of operations Long memory accesses and long FP delays This led Tomasulo to try to figure out how to get more effective registers renaming in hardware! Why Study 1966 Computer? The descendants of this have flourished! Alpha 21264, HP 8000, MIPS 10000, Pentium III, PowerPC 604, 16/02/2004 UAH- 41 Tomasulo s Algorithm (cont d) Control & buffers distributed with Function Units (FU) FU buffers called reservation stations => buffer the operands of instructions waiting to issue; Registers in instructions replaced by values or pointers to reservation stations (RS) => register renaming avoids WAR, WAW hazards More reservation stations than registers, so can do optimizations compilers can t Results to FU from RS, not through registers, over Common Data Bus that broadcasts results to all FUs Load and Stores treated as FUs with RSs as well Integer instructions can go past branches, allowing FP ops beyond basic block in FP queue 16/02/2004 UAH- 42 Aleksandar Milenkovich 21
22 Tomasulo-based FPU for MIPS From Mem FP Op Queue Load Buffers Load1 Load2 Load3 Load4 Load5 Load6 Add1 Add2 Add3 FP FP adders adders From Instruction Unit Mult1 Mult2 Reservation Stations FP Registers FP FP multipliers Store Buffers Store1 Store2 Store3 To Mem Common Data Bus (CDB) 16/02/2004 UAH- 43 Reservation Station Components Op: Operation to perform in the unit (e.g., + or ) Vj, Vk: Value of Source operands Store buffers has V field, result to be stored Qj, Qk: Reservation stations producing source registers (value to be written) Note: Qj/Qk=0 => source operand is already available in Vj /Vk Store buffers only have Qi for RS producing result Busy: Indicates reservation station or FU is busy Indicates which functional unit will write each register, if one exists. Blank when no pending instructions that will write that register. 16/02/2004 UAH- 44 Aleksandar Milenkovich 22
23 Three Stages of Tomasulo Algorithm 1. Issue get instruction from FP Op Queue If reservation station free (no structural hazard), control issues instr & sends operands (renames registers) 2. Execute operate on operands (EX) When both operands ready then execute; if not ready, watch Common Data Bus for result 3. Write result finish execution (WB) Write it on Common Data Bus to all awaiting units; mark reservation station available Normal data bus: data + destination ( go to bus) Common data bus: data + source ( come from bus) 64 bits of data + 4 bits of Functional Unit source address Write if matches expected Functional Unit (produces result) Does the broadcast Example speed: 2 clocks for Fl.pt. +,-; 10 for * ; 40 clks for / 16/02/2004 UAH- 45 Tomasulo Example Instruction stream Instruction status: Exec Write Instruction j k Issue Comp Result Busy Address LD F6 34+ R2 Load1 No LD F2 45+ R3 Load2 No MULTD F0 F2 F4 Load3 No SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 3 Load/Buffers Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk FU count down Add1 Add2 Add3 Mult1 Mult2 No No No No No : 0 FU Clock cycle counter 3 FP Adder R.S. 2 FP Mult R.S. 16/02/2004 UAH- 46 Aleksandar Milenkovich 23
24 Instruction status: Exec Write Tomasulo Example Cycle 1 Instruction j k Issue Comp Result Busy Address LD F6 34+ R2 1 Load1 Yes 34+R2 LD F2 45+ R3 Load2 No MULTD F0 F2 F4 Load3 No SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 Mult2 No No No No No : 1 FU Load1 16/02/2004 UAH- 47 Instruction status: Exec Write Tomasulo Example Cycle 2 Instruction j k Issue Comp Result Busy Address LD F6 34+ R2 1 Load1 Yes 34+R2 LD F2 45+ R3 2 Load2 Yes 45+R3 MULTD F0 F2 F4 Load3 No SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Add2 Add3 Mult1 Mult2 No No No No No : 2 FU Load2 Load1 Note: Can have multiple loads outstanding 16/02/2004 UAH- 48 Aleksandar Milenkovich 24
25 Instruction status: Exec Write Tomasulo Example Cycle 3 Instruction j k Issue Comp Result Busy Address LD F6 34+ R2 1 3 Load1 Yes 34+R2 LD F2 45+ R3 2 Load2 Yes 45+R3 MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F2 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No Mult1 Yes MULTD R(F4) Load2 : 3 FU Mult1 Load2 Load1 Note: registers names are removed ( renamed ) in Reservation Stations; MULT issued Load1 completing; what is waiting for Load1? 16/02/2004 UAH- 49 Instruction status: Exec Write Tomasulo Example Cycle 4 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R3 2 4 Load2 Yes 45+R3 MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F2 4 DIVD F10 F0 F6 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 Yes SUBD M(A1) Load2 Add2 No Add3 No Mult1 Yes MULTD R(F4) Load2 : 4 FU Mult1 Load2 M(A1) Add1 Load2 completing; what is waiting for Load2? 16/02/2004 UAH- 50 Aleksandar Milenkovich 25
26 Instruction status: Exec Write Tomasulo Example Cycle 5 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F2 4 DIVD F10 F0 F6 5 ADDD F6 F8 F2 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk 2 Add1 Yes SUBD M(A1) M(A2) Add2 No Add3 No 10 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 5 FU Mult1 M(A2) M(A1) Add1 Mult2 Timer starts down for Add1, Mult1 16/02/2004 UAH- 51 Instruction status: Exec Write Tomasulo Example Cycle 6 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F2 4 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk 1 Add1 Yes SUBD M(A1) M(A2) Add2 Yes ADDD M(A2) Add1 Add3 No 9 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 6 FU Mult1 M(A2) Add2 Add1 Mult2 Issue ADDD here despite name dependency on F6? 16/02/2004 UAH- 52 Aleksandar Milenkovich 26
27 Instruction status: Exec Write Tomasulo Example Cycle 7 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F2 4 7 DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk 0 Add1 Yes SUBD M(A1) M(A2) Add2 Yes ADDD M(A2) Add1 Add3 No 8 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 7 FU Mult1 M(A2) Add2 Add1 Mult2 Add1 (SUBD) completing; what is waiting for it? 16/02/2004 UAH- 53 Instruction status: Exec Write Tomasulo Example Cycle 8 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No 2 Add2 Yes ADDD (M-M) M(A2) Add3 No 7 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 8 FU Mult1 M(A2) Add2 (M-M) Mult2 16/02/2004 UAH- 54 Aleksandar Milenkovich 27
28 Instruction status: Exec Write Tomasulo Example Cycle 9 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F2 6 Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No 1 Add2 Yes ADDD (M-M) M(A2) Add3 No 6 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 9 FU Mult1 M(A2) Add2 (M-M) Mult2 16/02/2004 UAH- 55 Instruction status: Exec Write Tomasulo Example Cycle 10 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No 0 Add2 Yes ADDD (M-M) M(A2) Add3 No 5 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 10 FU Mult1 M(A2) Add2 (M-M) Mult2 Add2 (ADDD) completing; what is waiting for it? 16/02/2004 UAH- 56 Aleksandar Milenkovich 28
29 Instruction status: Exec Write Tomasulo Example Cycle 11 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No 4 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 11 FU Mult1 M(A2) (M-M+M(M-M) Mult2 Write result of ADDD here? All quick instructions complete in this cycle! 16/02/2004 UAH- 57 Instruction status: Exec Write Tomasulo Example Cycle 12 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No 3 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 12 FU Mult1 M(A2) (M-M+M(M-M) Mult2 16/02/2004 UAH- 58 Aleksandar Milenkovich 29
30 Instruction status: Exec Write Tomasulo Example Cycle 13 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No 2 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 13 FU Mult1 M(A2) (M-M+M(M-M) Mult2 16/02/2004 UAH- 59 Instruction status: Exec Write Tomasulo Example Cycle 14 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F4 3 Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No 1 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 14 FU Mult1 M(A2) (M-M+M(M-M) Mult2 16/02/2004 UAH- 60 Aleksandar Milenkovich 30
31 Instruction status: Exec Write Tomasulo Example Cycle 15 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No 0 Mult1 Yes MULTD M(A2) R(F4) Mult2 Yes DIVD M(A1) Mult1 : 15 FU Mult1 M(A2) (M-M+M(M-M) Mult2 Mult1 (MULTD) completing; what is waiting for it? 16/02/2004 UAH- 61 Instruction status: Exec Write Tomasulo Example Cycle 16 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No Mult1 No 40 Mult2 Yes DIVD M*F4 M(A1) : 16 FU M*F4 M(A2) (M-M+M(M-M) Mult2 Just waiting for Mult2 (DIVD) to complete 16/02/2004 UAH- 62 Aleksandar Milenkovich 31
32 Instruction status: Exec Write Tomasulo Example Cycle 55 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F Load3 No SUBD F8 F6 F DIVD F10 F0 F6 5 ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No Mult1 No 1 Mult2 Yes DIVD M*F4 M(A1) : 55 FU M*F4 M(A2) (M-M+M(M-M) Mult2 16/02/2004 UAH- 63 Instruction status: Exec Write Tomasulo Example Cycle 56 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F Load3 No SUBD F8 F6 F DIVD F10 F0 F ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No Mult1 No 0 Mult2 Yes DIVD M*F4 M(A1) : 56 FU M*F4 M(A2) (M-M+M(M-M) Mult2 Mult2 (DIVD) is completing; what is waiting for it? 16/02/2004 UAH- 64 Aleksandar Milenkovich 32
33 Instruction status: Exec Write Tomasulo Example Cycle 57 Instruction j k Issue Comp Result Busy Address LD F6 34+ R Load1 No LD F2 45+ R Load2 No MULTD F0 F2 F Load3 No SUBD F8 F6 F DIVD F10 F0 F ADDD F6 F8 F Reservation Stations: S1 S2 RS RS Time Name Busy Op Vj Vk Qj Qk Add1 No Add2 No Add3 No Mult1 No Mult2 Yes DIVD M*F4 M(A1) : 56 FU M*F4 M(A2) (M-M+M(M-M) Result Once again: In-order issue, out-of-order execution and out-of-order completion. 16/02/2004 UAH- 65 Tomasulo Drawbacks Complexity delays of 360/91, MIPS 10000, Alpha 21264, IBM PPC 620 in CA:AQA 2/e, but not in silicon! Many associative stores (CDB) at high speed Performance limited by Common Data Bus Each CDB must go to multiple functional units high capacitance, high wiring density Number of functional units that can complete per cycle limited to one! Multiple CDBs more FU logic for parallel assoc stores Non-precise interrupts! We will address this later 16/02/2004 UAH- 66 Aleksandar Milenkovich 33
34 Tomasulo Loop Example Loop: LD F0 0(R1) MULTD F4 F0 F2 SD F4 0 R1 SUBI R1 R1 #8 BNEZ R1 Loop This time assume Multiply takes 4 clocks Assume 1st load takes 8 clocks (L1 cache miss), 2nd load takes 1 clock (hit) To be clear, will show clocks for SUBI, BNEZ Reality: integer instructions ahead of Fl. Pt. Instructions Show 2 iterations 16/02/2004 UAH- 67 Loop Example Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 Load1 No 1 MULTD F4 F0 F2 Load2 No 1 SD F4 0 R1 Load3 No Iteration 2 LD F0 0 R1 Store1 No 2 MULTD F4 F0 F2 Store2 No Count 2 SD F4 0 R1 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Added Store Buffers Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 No SUBI R1 R1 #8 BNEZ R1 Loop Instruction Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Value of Register used for address, iteration control 16/02/2004 UAH- 68 Aleksandar Milenkovich 34
35 Loop Example Cycle 1 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 Load2 No Load3 No Store1 No Store2 No Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 No SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load1 16/02/2004 UAH- 69 Loop Example Cycle 2 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 No Load3 No Store1 No Store2 No Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load1 Mult1 16/02/2004 UAH- 70 Aleksandar Milenkovich 35
36 Loop Example Cycle 3 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 No 1 SD F4 0 R1 3 Load3 No Store1 Yes 80 Mult1 Store2 No Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load1 Mult1 Implicit renaming sets up data flow graph 16/02/2004 UAH- 71 Loop Example Cycle 4 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 No 1 SD F4 0 R1 3 Load3 No Store1 Yes 80 Mult1 Store2 No Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load1 Mult1 16/02/2004 UAH- 72 Aleksandar Milenkovich 36
37 Loop Example Cycle 5 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 No 1 SD F4 0 R1 3 Load3 No Store1 Yes 80 Mult1 Store2 No Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load1 Mult1 16/02/2004 UAH- 73 Loop Example Cycle 6 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 No 2 LD F0 0 R1 6 Store1 Yes 80 Mult1 Store2 No Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load2 Mult1 16/02/2004 UAH- 74 Aleksandar Milenkovich 37
38 Loop Example Cycle 7 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 No 2 LD F0 0 R1 6 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 No Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 Yes Multd R(F2) Load2 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load2 Mult2 16/02/2004 UAH- 75 Loop Example Cycle 8 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 No 2 LD F0 0 R1 6 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 Yes Multd R(F2) Load2 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load2 Mult2 16/02/2004 UAH- 76 Aleksandar Milenkovich 38
39 Loop Example Cycle 9 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R1 1 9 Load1 Yes 80 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 No 2 LD F0 0 R1 6 Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load1 SUBI R1 R1 #8 Mult2 Yes Multd R(F2) Load2 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load2 Mult2 16/02/2004 UAH- 77 Loop Example Cycle 10 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F2 2 Load2 Yes 72 1 SD F4 0 R1 3 Load3 No 2 LD F0 0 R Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 4 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 Mult2 Yes Multd R(F2) Load2 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load2 Mult2 16/02/2004 UAH- 78 Aleksandar Milenkovich 39
40 Loop Example Cycle 11 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F2 2 Load2 No 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 3 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 4 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult2 16/02/2004 UAH- 79 Loop Example Cycle 12 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F2 2 Load2 No 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 2 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 3 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult2 16/02/2004 UAH- 80 Aleksandar Milenkovich 40
41 Loop Example Cycle 13 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F2 2 Load2 No 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 1 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 2 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult2 16/02/2004 UAH- 81 Loop Example Cycle 14 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F Load2 No 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R Store1 Yes 80 Mult1 2 MULTD F4 F0 F2 7 Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 0 Mult1 Yes Multd M[80] R(F2) SUBI R1 R1 #8 1 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult2 16/02/2004 UAH- 82 Aleksandar Milenkovich 41
42 Loop Example Cycle 15 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F Load2 No 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R Store1 Yes 80 [80]*R2 2 MULTD F4 F0 F Store2 Yes 72 Mult2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 No SUBI R1 R1 #8 0 Mult2 Yes Multd M[72] R(F2) BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult2 16/02/2004 UAH- 83 Loop Example Cycle 16 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F Load2 No 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R Store1 Yes 80 [80]*R2 2 MULTD F4 F0 F Store2 Yes 72 [72]*R2 2 SD F4 0 R1 8 Store3 No Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 4 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult1 16/02/2004 UAH- 84 Aleksandar Milenkovich 42
43 Loop Example Cycle 17 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F Load2 No 1 SD F4 0 R1 3 Load3 Yes 64 2 LD F0 0 R Store1 Yes 80 [80]*R2 2 MULTD F4 F0 F Store2 Yes 72 [72]*R2 2 SD F4 0 R1 8 Store3 Yes 64 Mult1 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult1 16/02/2004 UAH- 85 Loop Example Cycle 18 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F Load2 No 1 SD F4 0 R Load3 Yes 64 2 LD F0 0 R Store1 Yes 80 [80]*R2 2 MULTD F4 F0 F Store2 Yes 72 [72]*R2 2 SD F4 0 R1 8 Store3 Yes 64 Mult1 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult1 16/02/2004 UAH- 86 Aleksandar Milenkovich 43
44 Loop Example Cycle 19 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 No 1 MULTD F4 F0 F Load2 No 1 SD F4 0 R Load3 Yes 64 2 LD F0 0 R Store1 No 2 MULTD F4 F0 F Store2 Yes 72 [72]*R2 2 SD F4 0 R Store3 Yes 64 Mult1 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load3 Mult1 16/02/2004 UAH- 87 Loop Example Cycle 20 Instruction status: Exec Write ITER Instruction j k Issue CompResult Busy Addr Fu 1 LD F0 0 R Load1 Yes 56 1 MULTD F4 F0 F Load2 No 1 SD F4 0 R Load3 Yes 64 2 LD F0 0 R Store1 No 2 MULTD F4 F0 F Store2 No 2 SD F4 0 R Store3 Yes 64 Mult1 Reservation Stations: S1 S2 RS Time Name Busy Op Vj Vk Qj Qk Code: Add1 No LD F0 0 R1 Add2 No MULTD F4 F0 F2 Add3 No SD F4 0 R1 Mult1 Yes Multd R(F2) Load3 SUBI R1 R1 #8 BNEZ R1 Loop Clock R1 F0 F2 F4 F6 F8 F10 F12... F Fu Load1 Mult1 Once again: In-order issue, out-of-order execution and out-of-order completion. 16/02/2004 UAH- 88 Aleksandar Milenkovich 44
45 Why can Tomasulo overlap iterations of loops? Register renaming Multiple iterations use different physical destinations for registers (dynamic loop unrolling) Reservation stations Permit instruction issue to advance past integer control flow operations Also buffer old values of registers - totally avoiding the WAR stall that we saw in the scoreboard Other perspective: Tomasulo building data flow dependency graph on the fly 16/02/2004 UAH- 89 Tomasulo s scheme offers 2 major advantages (1) the distribution of the hazard detection logic distributed reservation stations and the CDB If multiple instructions waiting on single result, & each instruction has other operand, then instructions can be released simultaneously by broadcast on CDB If a centralized register file were used, the units would have to read their results from the registers when register buses are available. (2) the elimination of stalls for WAW and WAR hazards 16/02/2004 UAH- 90 Aleksandar Milenkovich 45
46 What about Precise Interrupts? Tomasulo had: In-order issue, out-of-order execution, and out-of-order completion Need to fix the out-of-order completion aspect so that we can find precise breakpoint in instruction stream 16/02/2004 UAH- 91 Relationship between precise interrupts and speculation Speculation is a form of guessing Important for branch prediction: Need to take our best shot at predicting branch direction If we speculate and are wrong, need to back up and restart execution to point at which we predicted incorrectly: This is exactly same as precise exceptions! Technique for both precise interrupts/exceptions and speculation: in-order completion or commit 16/02/2004 UAH- 92 Aleksandar Milenkovich 46
47 HW support for precise interrupts Need HW buffer for results of uncommitted instructions: reorder buffer 3 fields: instr, destination, value Use reorder buffer number instead of reservation station when execution completes Supplies operands between execution complete & commit (Reorder buffer can be operand source => more registers like RS) Instructions commit Once instruction commits, result is put into register As a result, easy to undo speculated instructions on mispredicted branches or exceptions FP Op Queue Res Stations FP Adder Reorder Buffer FP Regs Res Stations FP Adder 16/02/2004 UAH- 93 Four Steps of Speculative Tomasulo Algorithm 1. Issue get instruction from FP Op Queue If reservation station and reorder buffer slot free, issue instr & send operands & reorder buffer no. for destination (this stage sometimes called dispatch ) 2. Execution operate on operands (EX) When both operands ready then execute; if not ready, watch CDB for result; when both in reservation station, execute; checks RAW (sometimes called issue ) 3. Write result finish execution (WB) Write on Common Data Bus to all awaiting FUs & reorder buffer; mark reservation station available. 4. Commit update register with reorder result When instr. at head of reorder buffer & result present, update register with result (or store to memory) and remove instr from reorder buffer. Mispredicted branch flushes reorder buffer (sometimes called graduation ) 16/02/2004 UAH- 94 Aleksandar Milenkovich 47
48 What are the hardware complexities with reorder buffer (ROB)? How do you find the latest version of a register? (As specified by Smith paper) need associative comparison network Could use future file or just use the register result status buffer to track which specific reorder buffer has received the value Need as many ports on ROB as register file Dest Reg Result Exceptions? Valid Program Counter FP Op Queue Compar network Reorder Buffer FP Regs Reorder Table Res Stations FP Adder Res Stations FP Adder 16/02/2004 UAH- 95 Summary Reservations stations: implicit register renaming to larger set of registers + buffering source operands Prevents registers as bottleneck Avoids WAR, WAW hazards of Scoreboard Allows loop unrolling in HW Not limited to basic blocks (integer units gets ahead, beyond branches) Today, helps cache misses as well Don t stall for L1 Data cache miss (insufficient ILP for L2 miss?) Lasting Contributions Dynamic scheduling Register renaming Load/store disambiguation 360/91 descendants are Pentium III; PowerPC 604; MIPS R10000; HP-PA 8000; Alpha /02/2004 UAH- 96 Aleksandar Milenkovich 48
CPE 631 Lecture 11: Instruction Level Parallelism and Its Dynamic Exploitation
Lecture 11: Instruction Level Parallelism and Its Dynamic Exploitation Aleksandar Milenkovic, milenka@ece.uah.edu Electrical and Computer Engineering University of Alabama in Huntsville Outline Instruction
More informationCPE 631 Lecture 10: Instruction Level Parallelism and Its Dynamic Exploitation
Lecture 10: Instruction Level Parallelism and Its Dynamic Exploitation Aleksandar Milenković, milenka@ece.uah.edu Electrical and Computer Engineering University of Alabama in Huntsville Outline Tomasulo
More informationPage 1. Recall from Pipelining Review. Lecture 16: Instruction Level Parallelism and Dynamic Execution #1: Ideas to Reduce Stalls
CS252 Graduate Computer Architecture Recall from Pipelining Review Lecture 16: Instruction Level Parallelism and Dynamic Execution #1: March 16, 2001 Prof. David A. Patterson Computer Science 252 Spring
More informationRecall from Pipelining Review. Lecture 16: Instruction Level Parallelism and Dynamic Execution #1: Ideas to Reduce Stalls
CS252 Graduate Computer Architecture Recall from Pipelining Review Lecture 16: Instruction Level Parallelism and Dynamic Execution #1: March 16, 2001 Prof. David A. Patterson Computer Science 252 Spring
More informationProcessor: Superscalars Dynamic Scheduling
Processor: Superscalars Dynamic Scheduling Z. Jerry Shi Assistant Professor of Computer Science and Engineering University of Connecticut * Slides adapted from Blumrich&Gschwind/ELE475 03, Peh/ELE475 (Princeton),
More informationPage 1. Recall from Pipelining Review. Lecture 15: Instruction Level Parallelism and Dynamic Execution
CS252 Graduate Computer Architecture Recall from Pipelining Review Lecture 15: Instruction Level Parallelism and Dynamic Execution March 11, 2002 Prof. David E. Culler Computer Science 252 Spring 2002
More information吳俊興高雄大學資訊工程學系. October Example to eleminate WAR and WAW by register renaming. Tomasulo Algorithm. A Dynamic Algorithm: Tomasulo s Algorithm
EEF011 Computer Architecture 計算機結構 吳俊興高雄大學資訊工程學系 October 2004 Example to eleminate WAR and WAW by register renaming Original DIV.D ADD.D S.D SUB.D MUL.D F0, F2, F4 F6, F0, F8 F6, 0(R1) F8, F10, F14 F6,
More informationCPE 631 Lecture 09: Instruction Level Parallelism and Its Dynamic Exploitation
Lecture 09: Instruction Level Parallelism and Its Dynamic Exploitation Aleksandar Milenkovic, milenka@ece.uah.edu Electrical and Computer Engineering University of Alabama in Huntsville Outline Instruction
More informationCISC 662 Graduate Computer Architecture. Lecture 10 - ILP 3
CISC 662 Graduate Computer Architecture Lecture 10 - ILP 3 Michela Taufer http://www.cis.udel.edu/~taufer/teaching/cis662f07 Powerpoint Lecture Notes from John Hennessy and David Patterson s: Computer
More informationCS252 Graduate Computer Architecture Lecture 6. Recall: Software Pipelining Example
CS252 Graduate Computer Architecture Lecture 6 Tomasulo, Implicit Register Renaming, Loop-Level Parallelism Extraction Explicit Register Renaming John Kubiatowicz Electrical Engineering and Computer Sciences
More informationReduction of Data Hazards Stalls with Dynamic Scheduling So far we have dealt with data hazards in instruction pipelines by:
Reduction of Data Hazards Stalls with Dynamic Scheduling So far we have dealt with data hazards in instruction pipelines by: Result forwarding (register bypassing) to reduce or eliminate stalls needed
More informationReview: Compiler techniques for parallelism Loop unrolling Ÿ Multiple iterations of loop in software:
CS152 Computer Architecture and Engineering Lecture 17 Dynamic Scheduling: Tomasulo March 20, 2001 John Kubiatowicz (http.cs.berkeley.edu/~kubitron) lecture slides: http://www-inst.eecs.berkeley.edu/~cs152/
More informationELEC 5200/6200 Computer Architecture and Design Fall 2016 Lecture 9: Instruction Level Parallelism
ELEC 5200/6200 Computer Architecture and Design Fall 2016 Lecture 9: Instruction Level Parallelism Ujjwal Guin, Assistant Professor Department of Electrical and Computer Engineering Auburn University,
More informationInstruction Level Parallelism
Instruction Level Parallelism Dynamic scheduling Scoreboard Technique Tomasulo Algorithm Speculation Reorder Buffer Superscalar Processors 1 Definition of ILP ILP=Potential overlap of execution among unrelated
More informationGraduate Computer Architecture. Chapter 3. Instruction Level Parallelism and Its Dynamic Exploitation
Graduate Computer Architecture Chapter 3 Instruction Level Parallelism and Its Dynamic Exploitation 1 Overview Instruction level parallelism Dynamic Scheduling Techniques Scoreboarding (Appendix A.8) Tomasulo
More informationMetodologie di Progettazione Hardware-Software
Metodologie di Progettazione Hardware-Software Advanced Pipelining and Instruction-Level Paralelism Metodologie di Progettazione Hardware/Software LS Ing. Informatica 1 ILP Instruction-level Parallelism
More informationDetailed Scoreboard Pipeline Control. Three Parts of the Scoreboard. Scoreboard Example Cycle 1. Scoreboard Example. Scoreboard Example Cycle 3
1 From MIPS pipeline to Scoreboard Lecture 5 Scoreboarding: Enforce Register Data Dependence Scoreboard design, big example Out-of-order execution divides ID stage: 1.Issue decode instructions, check for
More informationInstruction-Level Parallelism and Its Exploitation
Chapter 2 Instruction-Level Parallelism and Its Exploitation 1 Overview Instruction level parallelism Dynamic Scheduling Techniques es Scoreboarding Tomasulo s s Algorithm Reducing Branch Cost with Dynamic
More informationPipelining: Issue instructions in every cycle (CPI 1) Compiler scheduling (static scheduling) reduces impact of dependences
Dynamic Scheduling Pipelining: Issue instructions in every cycle (CPI 1) Compiler scheduling (static scheduling) reduces impact of dependences Increased compiler complexity, especially when attempting
More informationSuperscalar Architectures: Part 2
Superscalar Architectures: Part 2 Dynamic (Out-of-Order) Scheduling Lecture 3.2 August 23 rd, 2017 Jae W. Lee (jaewlee@snu.ac.kr) Computer Science and Engineering Seoul NaMonal University Download this
More informationCOSC4201 Instruction Level Parallelism Dynamic Scheduling
COSC4201 Instruction Level Parallelism Dynamic Scheduling Prof. Mokhtar Aboelaze Parts of these slides are taken from Notes by Prof. David Patterson (UCB) Outline Data dependence and hazards Exposing parallelism
More informationStatic vs. Dynamic Scheduling
Static vs. Dynamic Scheduling Dynamic Scheduling Fast Requires complex hardware More power consumption May result in a slower clock Static Scheduling Done in S/W (compiler) Maybe not as fast Simpler processor
More informationReview: Evaluating Branch Alternatives. Lecture 3: Introduction to Advanced Pipelining. Review: Evaluating Branch Prediction
Review: Evaluating Branch Alternatives Lecture 3: Introduction to Advanced Pipelining Two part solution: Determine branch taken or not sooner, AND Compute taken branch address earlier Pipeline speedup
More informationAdvantages of Dynamic Scheduling
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Computer Architecture ECE 568 Part 5 Dynamic scheduling with Scoreboards Israel Koren ECE568/Koren Part.5.1 Advantages of Dynamic
More informationWhat is ILP? Instruction Level Parallelism. Where do we find ILP? How do we expose ILP?
What is ILP? Instruction Level Parallelism or Declaration of Independence The characteristic of a program that certain instructions are, and can potentially be. Any mechanism that creates, identifies,
More informationScoreboard information (3 tables) Four stages of scoreboard control
Scoreboard information (3 tables) Instruction : issued, read operands and started execution (dispatched), completed execution or wrote result, Functional unit (assuming non-pipelined units) busy/not busy
More informationAdapted from David Patterson s slides on graduate computer architecture
Mei Yang Adapted from David Patterson s slides on graduate computer architecture Introduction Basic Compiler Techniques for Exposing ILP Advanced Branch Prediction Dynamic Scheduling Hardware-Based Speculation
More informationLecture 4: Introduction to Advanced Pipelining
Lecture 4: Introduction to Advanced Pipelining Prepared by: Professor David A. Patterson Computer Science 252, Fall 1996 Edited and presented by : Prof. Kurt Keutzer Computer Science 252, Spring 2000 KK
More informationWebsite for Students VTU NOTES QUESTION PAPERS NEWS RESULTS
Advanced Computer Architecture- 06CS81 Hardware Based Speculation Tomasulu algorithm and Reorder Buffer Tomasulu idea: 1. Have reservation stations where register renaming is possible 2. Results are directly
More informationHardware-based speculation (2.6) Multiple-issue plus static scheduling = VLIW (2.7) Multiple-issue, dynamic scheduling, and speculation (2.
Instruction-Level Parallelism and its Exploitation: PART 2 Hardware-based speculation (2.6) Multiple-issue plus static scheduling = VLIW (2.7) Multiple-issue, dynamic scheduling, and speculation (2.8)
More informationILP concepts (2.1) Basic compiler techniques (2.2) Reducing branch costs with prediction (2.3) Dynamic scheduling (2.4 and 2.5)
Instruction-Level Parallelism and its Exploitation: PART 1 ILP concepts (2.1) Basic compiler techniques (2.2) Reducing branch costs with prediction (2.3) Dynamic scheduling (2.4 and 2.5) Project and Case
More informationInstruction Level Parallelism (ILP)
Instruction Level Parallelism (ILP) Pipelining supports a limited sense of ILP e.g. overlapped instructions, out of order completion and issue, bypass logic, etc. Remember Pipeline CPI = Ideal Pipeline
More informationEITF20: Computer Architecture Part3.2.1: Pipeline - 3
EITF20: Computer Architecture Part3.2.1: Pipeline - 3 Liang Liu liang.liu@eit.lth.se 1 Outline Reiteration Dynamic scheduling - Tomasulo Superscalar, VLIW Speculation ILP limitations What we have done
More informationCS252 Graduate Computer Architecture Lecture 8. Review: Scoreboard (CDC 6600) Explicit Renaming Precise Interrupts February 13 th, 2010
CS252 Graduate Computer Architecture Lecture 8 Explicit Renaming Precise Interrupts February 13 th, 2010 John Kubiatowicz Electrical Engineering and Computer Sciences University of California, Berkeley
More informationChapter 3 Instruction-Level Parallelism and its Exploitation (Part 1)
Chapter 3 Instruction-Level Parallelism and its Exploitation (Part 1) ILP vs. Parallel Computers Dynamic Scheduling (Section 3.4, 3.5) Dynamic Branch Prediction (Section 3.3) Hardware Speculation and Precise
More informationLoad1 no Load2 no Add1 Y Sub Reg[F2] Reg[F6] Add2 Y Add Reg[F2] Add1 Add3 no Mult1 Y Mul Reg[F2] Reg[F4] Mult2 Y Div Reg[F6] Mult1
Instruction Issue Execute Write result L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F2, F6 DIV.D F10, F0, F6 ADD.D F6, F8, F2 Name Busy Op Vj Vk Qj Qk A Load1 no Load2 no Add1 Y Sub Reg[F2]
More informationCompiler Optimizations. Lecture 7 Overview of Superscalar Techniques. Memory Allocation by Compilers. Compiler Structure. Register allocation
Lecture 7 Overview of Superscalar Techniques CprE 581 Computer Systems Architecture, Fall 2013 Reading: Textbook, Ch. 3 Complexity-Effective Superscalar Processors, PhD Thesis by Subbarao Palacharla, Ch.1
More informationHandout 2 ILP: Part B
Handout 2 ILP: Part B Review from Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level Parallelism Loop unrolling by compiler to increase ILP Branch prediction to increase ILP
More informationOutline Review: Basic Pipeline Scheduling and Loop Unrolling Multiple Issue: Superscalar, VLIW. CPE 631 Session 19 Exploiting ILP with SW Approaches
Session xploiting ILP with SW Approaches lectrical and Computer ngineering University of Alabama in Huntsville Outline Review: Basic Pipeline Scheduling and Loop Unrolling Multiple Issue: Superscalar,
More informationThe basic structure of a MIPS floating-point unit
Tomasulo s scheme The algorithm based on the idea of reservation station The reservation station fetches and buffers an operand as soon as it is available, eliminating the need to get the operand from
More informationInstruction Level Parallelism. Taken from
Instruction Level Parallelism Taken from http://www.cs.utsa.edu/~dj/cs3853/lecture5.ppt Outline ILP Compiler techniques to increase ILP Loop Unrolling Static Branch Prediction Dynamic Branch Prediction
More informationInstruction Level Parallelism
Instruction Level Parallelism The potential overlap among instruction execution is called Instruction Level Parallelism (ILP) since instructions can be executed in parallel. There are mainly two approaches
More informationTopics. Digital Systems Architecture EECE EECE Predication, Prediction, and Speculation
Digital Systems Architecture EECE 343-01 EECE 292-02 Predication, Prediction, and Speculation Dr. William H. Robinson February 25, 2004 http://eecs.vanderbilt.edu/courses/eece343/ Topics Aha, now I see,
More informationFour Steps of Speculative Tomasulo cycle 0
HW support for More ILP Hardware Speculative Execution Speculation: allow an instruction to issue that is dependent on branch, without any consequences (including exceptions) if branch is predicted incorrectly
More informationCSE 820 Graduate Computer Architecture. week 6 Instruction Level Parallelism. Review from Last Time #1
CSE 820 Graduate Computer Architecture week 6 Instruction Level Parallelism Based on slides by David Patterson Review from Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level
More informationChapter 3: Instruction Level Parallelism (ILP) and its exploitation. Types of dependences
Chapter 3: Instruction Level Parallelism (ILP) and its exploitation Pipeline CPI = Ideal pipeline CPI + stalls due to hazards invisible to programmer (unlike process level parallelism) ILP: overlap execution
More information5008: Computer Architecture
5008: Computer Architecture Chapter 2 Instruction-Level Parallelism and Its Exploitation CA Lecture05 - ILP (cwliu@twins.ee.nctu.edu.tw) 05-1 Review from Last Lecture Instruction Level Parallelism Leverage
More informationEECC551 Exam Review 4 questions out of 6 questions
EECC551 Exam Review 4 questions out of 6 questions (Must answer first 2 questions and 2 from remaining 4) Instruction Dependencies and graphs In-order Floating Point/Multicycle Pipelining (quiz 2) Improving
More informationLecture-13 (ROB and Multi-threading) CS422-Spring
Lecture-13 (ROB and Multi-threading) CS422-Spring 2018 Biswa@CSE-IITK Cycle 62 (Scoreboard) vs 57 in Tomasulo Instruction status: Read Exec Write Exec Write Instruction j k Issue Oper Comp Result Issue
More informationDynamic Scheduling. Better than static scheduling Scoreboarding: Tomasulo algorithm:
LECTURE - 13 Dynamic Scheduling Better than static scheduling Scoreboarding: Used by the CDC 6600 Useful only within basic block WAW and WAR stalls Tomasulo algorithm: Used in IBM 360/91 for the FP unit
More informationChapter 4 The Processor 1. Chapter 4D. The Processor
Chapter 4 The Processor 1 Chapter 4D The Processor Chapter 4 The Processor 2 Instruction-Level Parallelism (ILP) Pipelining: executing multiple instructions in parallel To increase ILP Deeper pipeline
More informationLatencies of FP operations used in chapter 4.
Instruction-Level Parallelism (ILP) ILP: refers to the overlap execution of instructions. Pipelined CPI = Ideal pipeline CPI + structural stalls + RAW stalls + WAR stalls + WAW stalls + Control stalls.
More informationHardware-based Speculation
Hardware-based Speculation M. Sonza Reorda Politecnico di Torino Dipartimento di Automatica e Informatica 1 Introduction Hardware-based speculation is a technique for reducing the effects of control dependences
More informationNOW Handout Page 1. Review from Last Time #1. CSE 820 Graduate Computer Architecture. Lec 8 Instruction Level Parallelism. Outline
CSE 820 Graduate Computer Architecture Lec 8 Instruction Level Parallelism Based on slides by David Patterson Review Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level Parallelism
More informationMulticycle ALU Operations 2/28/2011. Diversified Pipelines The Path Toward Superscalar Processors. Limitations of Our Simple 5 stage Pipeline
//11 Limitations of Our Simple stage Pipeline Diversified Pipelines The Path Toward Superscalar Processors HPCA, Spring 11 Assumes single cycle EX stage for all instructions This is not feasible for Complex
More informationComputer Architectures. Chapter 4. Tien-Fu Chen. National Chung Cheng Univ.
Computer Architectures Chapter 4 Tien-Fu Chen National Chung Cheng Univ. chap4-0 Advance Pipelining! Static Scheduling Have compiler to minimize the effect of structural, data, and control dependence "
More informationHardware-based Speculation
Hardware-based Speculation Hardware-based Speculation To exploit instruction-level parallelism, maintaining control dependences becomes an increasing burden. For a processor executing multiple instructions
More informationCOSC 6385 Computer Architecture - Pipelining (II)
COSC 6385 Computer Architecture - Pipelining (II) Edgar Gabriel Spring 2018 Performance evaluation of pipelines (I) General Speedup Formula: Time Speedup Time IC IC ClockCycle ClockClycle CPI CPI For a
More informationFunctional Units. Registers. The Big Picture: Where are We Now? The Five Classic Components of a Computer Processor Input Control Memory
The Big Picture: Where are We Now? CS152 Computer Architecture and Engineering Lecture 18 The Five Classic Components of a Computer Processor Input Control Dynamic Scheduling (Cont), Speculation, and ILP
More informationCISC 662 Graduate Computer Architecture Lecture 11 - Hardware Speculation Branch Predictions
CISC 662 Graduate Computer Architecture Lecture 11 - Hardware Speculation Branch Predictions Michela Taufer http://www.cis.udel.edu/~taufer/teaching/cis6627 Powerpoint Lecture Notes from John Hennessy
More informationINSTITUTO SUPERIOR TÉCNICO. Architectures for Embedded Computing
UNIVERSIDADE TÉCNICA DE LISBOA INSTITUTO SUPERIOR TÉCNICO Departamento de Engenharia Informática Architectures for Embedded Computing MEIC-A, MEIC-T, MERC Lecture Slides Version 3.0 - English Lecture 09
More informationCSE 502 Graduate Computer Architecture. Lec 8-10 Instruction Level Parallelism
CSE 502 Graduate Computer Architecture Lec 8-10 Instruction Level Parallelism Larry Wittie Computer Science, StonyBrook University http://www.cs.sunysb.edu/~cse502 and ~lw Slides adapted from David Patterson,
More informationUniversity of Southern California Department of Electrical Engineering EE557 Fall 2001 Instructor: Michel Dubois Homework #3.
University of Southern California Department of Electrical Engineering EE557 Fall 2001 Instructor: Michel Dubois Homework #3. SOLUTIONS Problem 1 (20pts). There are seven dependences in the C loop presented
More information332 Advanced Computer Architecture Chapter 3. Dynamic scheduling, out-of-order execution, register renaming and speculative execution
332 Advanced Computer Architecture Chapter 3 Dynamic scheduling, out-of-order execution, register renaming and speculative execution February 2017 Paul H J Kelly These lecture notes are partly based on
More informationInstruction Frequency CPI. Load-store 55% 5. Arithmetic 30% 4. Branch 15% 4
PROBLEM 1: An application running on a 1GHz pipelined processor has the following instruction mix: Instruction Frequency CPI Load-store 55% 5 Arithmetic 30% 4 Branch 15% 4 a) Determine the overall CPI
More informationEI 338: Computer Systems Engineering (Operating Systems & Computer Architecture)
EI 338: Computer Systems Engineering (Operating Systems & Computer Architecture) Dept. of Computer Science & Engineering Chentao Wu wuct@cs.sjtu.edu.cn Download lectures ftp://public.sjtu.edu.cn User:
More informationNOW Handout Page 1. Outline. Csci 211 Computer System Architecture. Lec 4 Instruction Level Parallelism. Instruction Level Parallelism
Outline Csci 211 Computer System Architecture Lec 4 Instruction Level Parallelism Xiuzhen Cheng Department of Computer Sciences The George Washington University ILP Compiler techniques to increase ILP
More informationHigh Performance Computer Architecture Prof. Ajit Pal Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur
High Performance Computer Architecture Prof. Ajit Pal Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture - 18 Dynamic Instruction Scheduling with Branch Prediction
More informationExploiting ILP with SW Approaches. Aleksandar Milenković, Electrical and Computer Engineering University of Alabama in Huntsville
Lecture : Exploiting ILP with SW Approaches Aleksandar Milenković, milenka@ece.uah.edu Electrical and Computer Engineering University of Alabama in Huntsville Outline Basic Pipeline Scheduling and Loop
More informationNOW Handout Page 1. Review from Last Time. CSE 820 Graduate Computer Architecture. Lec 7 Instruction Level Parallelism. Recall from Pipelining Review
Review from Last Time CSE 820 Graduate Computer Architecture Lec 7 Instruction Level Parallelism Based on slides by David Patterson 4 papers: All about where to draw line between HW and SW IBM set foundations
More informationESE 545 Computer Architecture Instruction-Level Parallelism (ILP) and Static & Dynamic Instruction Scheduling Instruction level parallelism
Computer Architecture ESE 545 Computer Architecture Instruction-Level Parallelism (ILP) and Static & Dynamic Instruction Scheduling 1 Outline ILP Compiler techniques to increase ILP Loop Unrolling Static
More informationAdvanced Computer Architecture
Advanced Computer Architecture Instruction Level Parallelism Myung Hoon, Sunwoo School of Electrical and Computer Engineering Ajou University Outline ILP Compiler techniques to increase ILP Loop Unrolling
More informationSuper Scalar. Kalyan Basu March 21,
Super Scalar Kalyan Basu basu@cse.uta.edu March 21, 2007 1 Super scalar Pipelines A pipeline that can complete more than 1 instruction per cycle is called a super scalar pipeline. We know how to build
More informationEEC 581 Computer Architecture. Instruction Level Parallelism (3.6 Hardware-based Speculation and 3.7 Static Scheduling/VLIW)
1 EEC 581 Computer Architecture Instruction Level Parallelism (3.6 Hardware-based Speculation and 3.7 Static Scheduling/VLIW) Chansu Yu Electrical and Computer Engineering Cleveland State University Overview
More informationComputer Architecture 计算机体系结构. Lecture 4. Instruction-Level Parallelism II 第四讲 指令级并行 II. Chao Li, PhD. 李超博士
Computer Architecture 计算机体系结构 Lecture 4. Instruction-Level Parallelism II 第四讲 指令级并行 II Chao Li, PhD. 李超博士 SJTU-SE346, Spring 2018 Review Hazards (data/name/control) RAW, WAR, WAW hazards Different types
More informationChapter 3 (CONT II) Instructor: Josep Torrellas CS433. Copyright J. Torrellas 1999,2001,2002,2007,
Chapter 3 (CONT II) Instructor: Josep Torrellas CS433 Copyright J. Torrellas 1999,2001,2002,2007, 2013 1 Hardware-Based Speculation (Section 3.6) In multiple issue processors, stalls due to branches would
More informationCISC 662 Graduate Computer Architecture Lecture 13 - CPI < 1
CISC 662 Graduate Computer Architecture Lecture 13 - CPI < 1 Michela Taufer http://www.cis.udel.edu/~taufer/teaching/cis662f07 Powerpoint Lecture Notes from John Hennessy and David Patterson s: Computer
More informationRecall from Pipelining Review. Instruction Level Parallelism and Dynamic Execution
332 Advanced Computer Architecture Chapter 4 Instruction Level Parallelism and Dynamic Execution January 2004 Paul H J Kelly These lecture notes are partly based on the course text, Hennessy and Patterson
More informationGood luck and have fun!
Midterm Exam October 13, 2014 Name: Problem 1 2 3 4 total Points Exam rules: Time: 90 minutes. Individual test: No team work! Open book, open notes. No electronic devices, except an unprogrammed calculator.
More informationEEC 581 Computer Architecture. Lec 7 Instruction Level Parallelism (2.6 Hardware-based Speculation and 2.7 Static Scheduling/VLIW)
EEC 581 Computer Architecture Lec 7 Instruction Level Parallelism (2.6 Hardware-based Speculation and 2.7 Static Scheduling/VLIW) Chansu Yu Electrical and Computer Engineering Cleveland State University
More informationAdvanced issues in pipelining
Advanced issues in pipelining 1 Outline Handling exceptions Supporting multi-cycle operations Pipeline evolution Examples of real pipelines 2 Handling exceptions 3 Exceptions In pipelined execution, one
More informationCOSC 6385 Computer Architecture - Instruction Level Parallelism (II)
COSC 6385 Computer Architecture - Instruction Level Parallelism (II) Edgar Gabriel Spring 2016 Data fields for reservation stations Op: operation to perform on source operands S1 and S2 Q j, Q k : reservation
More informationOutline EEL 5764 Graduate Computer Architecture. Chapter 2 - Instruction Level Parallelism. Recall from Pipelining Review
Outline EEL 5764 Graduate Computer Architecture Chapter 2 - Instruction Level Parallelism Ann Gordon-Ross Electrical and Computer Engineering University of Florida ILP Compiler techniques to increase ILP
More informationDYNAMIC INSTRUCTION SCHEDULING WITH SCOREBOARD
DYNAMIC INSTRUCTION SCHEDULING WITH SCOREBOARD Slides by: Pedro Tomás Additional reading: Computer Architecture: A Quantitative Approach, 5th edition, Chapter 3, John L. Hennessy and David A. Patterson,
More informationAdvanced Computer Architecture CMSC 611 Homework 3. Due in class Oct 17 th, 2012
Advanced Computer Architecture CMSC 611 Homework 3 Due in class Oct 17 th, 2012 (Show your work to receive partial credit) 1) For the following code snippet list the data dependencies and rewrite the code
More informationCS433 Homework 2 (Chapter 3)
CS433 Homework 2 (Chapter 3) Assigned on 9/19/2017 Due in class on 10/5/2017 Instructions: 1. Please write your name and NetID clearly on the first page. 2. Refer to the course fact sheet for policies
More informationChapter 4. Advanced Pipelining and Instruction-Level Parallelism. In-Cheol Park Dept. of EE, KAIST
Chapter 4. Advanced Pipelining and Instruction-Level Parallelism In-Cheol Park Dept. of EE, KAIST Instruction-level parallelism Loop unrolling Dependence Data/ name / control dependence Loop level parallelism
More informationCS 252 Graduate Computer Architecture. Lecture 4: Instruction-Level Parallelism
CS 252 Graduate Computer Architecture Lecture 4: Instruction-Level Parallelism Krste Asanovic Electrical Engineering and Computer Sciences University of California, Berkeley http://wwweecsberkeleyedu/~krste
More informationMulti-cycle Instructions in the Pipeline (Floating Point)
Lecture 6 Multi-cycle Instructions in the Pipeline (Floating Point) Introduction to instruction level parallelism Recap: Support of multi-cycle instructions in a pipeline (App A.5) Recap: Superpipelining
More informationComplex Pipelining: Out-of-order Execution & Register Renaming. Multiple Function Units
6823, L14--1 Complex Pipelining: Out-of-order Execution & Register Renaming Laboratory for Computer Science MIT http://wwwcsglcsmitedu/6823 Multiple Function Units 6823, L14--2 ALU Mem IF ID Issue WB Fadd
More informationDonn Morrison Department of Computer Science. TDT4255 ILP and speculation
TDT4255 Lecture 9: ILP and speculation Donn Morrison Department of Computer Science 2 Outline Textbook: Computer Architecture: A Quantitative Approach, 4th ed Section 2.6: Speculation Section 2.7: Multiple
More informationNOW Handout Page 1. COSC 5351 Advanced Computer Architecture
COSC 5351 Advanced Computer Slides modified from Hennessy CS252 course slides ILP Compiler techniques to increase ILP Loop Unrolling Static Branch Prediction Dynamic Branch Prediction Overcoming Data Hazards
More informationTomasulo s Algorithm
Tomasulo s Algorithm Architecture to increase ILP Removes WAR and WAW dependencies during issue WAR and WAW Name Dependencies Artifact of using the same storage location (variable name) Can be avoided
More informationCS 614 COMPUTER ARCHITECTURE II FALL 2004
CS 64 COMPUTER ARCHITECTURE II FALL 004 DUE : October, 005 HOMEWORK II READ : - Portions of Chapters 5, 7, 8 and 9 of the Sima book and - Portions of Chapter 3, 4 and Appendix A of the Hennessy book ASSIGNMENT:
More informationPage # CISC 662 Graduate Computer Architecture. Lecture 8 - ILP 1. Pipeline CPI. Pipeline CPI (I) Michela Taufer
CISC 662 Graduate Computer Architecture Lecture 8 - ILP 1 Michela Taufer http://www.cis.udel.edu/~taufer/teaching/cis662f07 Powerpoint Lecture Notes from John Hennessy and David Patterson s: Computer Architecture,
More informationComputer Architecture A Quantitative Approach, Fifth Edition. Chapter 3. Instruction-Level Parallelism and Its Exploitation
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 3 Instruction-Level Parallelism and Its Exploitation Introduction Pipelining become universal technique in 1985 Overlaps execution of
More informationPage 1. CISC 662 Graduate Computer Architecture. Lecture 8 - ILP 1. Pipeline CPI. Pipeline CPI (I) Pipeline CPI (II) Michela Taufer
CISC 662 Graduate Computer Architecture Lecture 8 - ILP 1 Michela Taufer Pipeline CPI http://www.cis.udel.edu/~taufer/teaching/cis662f07 Powerpoint Lecture Notes from John Hennessy and David Patterson
More informationMultiple Instruction Issue and Hardware Based Speculation
Multiple Instruction Issue and Hardware Based Speculation Soner Önder Michigan Technological University, Houghton MI www.cs.mtu.edu/~soner Hardware Based Speculation Exploiting more ILP requires that we
More informationELE 818 * ADVANCED COMPUTER ARCHITECTURES * MIDTERM TEST *
ELE 818 * ADVANCED COMPUTER ARCHITECTURES * MIDTERM TEST * SAMPLE 1 Section: Simple pipeline for integer operations For all following questions we assume that: a) Pipeline contains 5 stages: IF, ID, EX,
More informationComplex Pipelining COE 501. Computer Architecture Prof. Muhamed Mudawar
Complex Pipelining COE 501 Computer Architecture Prof. Muhamed Mudawar Computer Engineering Department King Fahd University of Petroleum and Minerals Presentation Outline Diversified Pipeline Detecting
More information