INSTITUTO SUPERIOR TÉCNICO. Architectures for Embedded Computing

Size: px

Start display at page:

Download "INSTITUTO SUPERIOR TÉCNICO. Architectures for Embedded Computing"

Katrina Daniel
5 years ago
Views:

UNIVERSIDADE TÉCNICA DE LISBOA INSTITUTO SUPERIOR TÉCNICO Departamento de Engenharia Informática Architectures for Embedded Computing MEIC-A, MEIC-T, MERC Lecture Slides Version 3.

1 UNIVERSIDADE TÉCNICA DE LISBOA INSTITUTO SUPERIOR TÉCNICO Departamento de Engenharia Informática Architectures for Embedded Computing MEIC-A, MEIC-T, MERC Lecture Slides Version English Lecture 09 Title: Dynamic and Summary: Dynamic scheduling (centralized control - scoreboard; distributed control - algorithm); scheduling. 2010/2011 Nuno.Roma@ist.utl.pt

.. Pipelining - implementation problems: Multi-cycle instructions;

2 Architectures for Embedded Computing Dynamic and Prof. Nuno Roma ACE 2010/11 - DEI-IST 1 / 45 Previous Class In the previous class... Pipelining - implementation problems: Multi-cycle instructions; Super-pipelining; Interruptions; Code optimization for pipeline execution. Prof. Nuno Roma ACE 2010/11 - DEI-IST 2 / 45

3 Road Map Prof. Nuno Roma ACE 2010/11 - DEI-IST 3 / 45 Summary Today: Dynamic scheduling: Centralized control: Distributed control: algorithm scheduling. Bibliography: Computer Architecture: a Quantitative Approach, Sections A.7, 2.4 and 2.6 Prof. Nuno Roma ACE 2010/11 - DEI-IST 4 / 45

4 Prof. Nuno Roma ACE 2010/11 - DEI-IST 5 / 45 Static scheduling: Instructions are read from memory and executed in the pipeline in the same order as they occur in the program; If there is a data dependency that cannot be hidden, the hazard detection hardware stalls the pipeline by suspending the fetching of new instructions; To avoid this penalization, the compiler can schedule/intercalate other instructions which do not interfere with the dependency/hazard: Compiler/Static solutions: Interleaving of instructions; Loop-unrolling, etc. Prof. Nuno Roma ACE 2010/11 - DEI-IST 6 / 45

5 : The hardware (on its own) re-arranges the instruction scheduling in order to: Reduce hazards and dependencies that were unknown at compile time: simplification of the compiler; Mask unpredictable delays (e.g.: cache misses), by executing other operations that do not impose any dependencies: performance improve; Prof. Nuno Roma ACE 2010/11 - DEI-IST 7 / 45 : The hardware (on its own) re-arranges the instruction scheduling in order to: Reduce hazards and dependencies that were unknown at compile time: simplification of the compiler; Mask unpredictable delays (e.g.: cache misses), by executing other operations that do not impose any dependencies: performance improve; The structural and data hazards are checked in ID stage: As soon as the necessary conditions to execute each instruction are assured, it is released from the ID stage. Prof. Nuno Roma ACE 2010/11 - DEI-IST 7 / 45

: The execution can be done out-of-order : each instruction is executed as soon as the corresponding operands are available. Out-of-order execution implies Out-of-order completion! Prof.

6 : The execution can be done out-of-order : each instruction is executed as soon as the corresponding operands are available. Out-of-order execution implies Out-of-order completion! Prof. Nuno Roma ACE 2010/11 - DEI-IST 8 / 45 : The hardware may change the execution order of instructions in order to keep its functionality. Particularly useful whenever the execution stage is implemented by several concurrent processing units. E.g.: floating-point computational units. Prof. Nuno Roma ACE 2010/11 - DEI-IST 9 / 45

7 : The ID stage is now separated in 2 sub-steps: Issue: decoding and check for structural hazards; Read Operands: wait until no data hazards, then read operands. F I R X M W All instructions must pass through the issue step in their original order (in-order issue); Instructions may be stalled or bypassed by others that were blocked in the read operands step (out-of-order read). Pending instructions queue. Prof. Nuno Roma ACE 2010/11 - DEI-IST 10 / 45 Prof. Nuno Roma ACE 2010/11 - DEI-IST 11 / 45

8 Centralized : ing: allows an out-of-order execution of instructions as soon as the required resources are available and there isn t any data dependency. Potential WAR and WAW hazards: DIV.D F0, F2, F4 ADD.D F10, F0, F8 SUB.D F8, F8, F14 DIV.D F0, F2, F4 ADD.D F10, F0, F8 SUB.D F10, F8, F14 Prof. Nuno Roma ACE 2010/11 - DEI-IST 12 / 45 Centralized : Several instructions may simultaneously be in the EX stage (in distinct execution units or different phases of the same execution unit). Prof. Nuno Roma ACE 2010/11 - DEI-IST 13 / 45

9 Centralized : The scoreboard records all dependencies and hazards that occur along the program, determining when each instruction can: Prof. Nuno Roma ACE 2010/11 - DEI-IST 14 / 45 Centralized : The scoreboard records all dependencies and hazards that occur along the program, determining when each instruction can: Issue, by checking the structural and the WAW data hazards, introducing stalls whenever necessary; Example of a WAW hazard: DIV.D F0, F2, F4 ADD.D F10, F0, F8 SUB.D F10, F8, F14 Prof. Nuno Roma ACE 2010/11 - DEI-IST 14 / 45

10 Centralized : The scoreboard records all dependencies and hazards that occur along the program, determining when each instruction can: Read operands, by checking the RAW data hazards and introducing stalls whenever necessary; Example of a RAW hazards: DIV.D F0, F2, F4 ADD.D F10, F0, F8 SUB.D F8, F8, F14 Prof. Nuno Roma ACE 2010/11 - DEI-IST 14 / 45 Centralized : The scoreboard records all dependencies and hazards that occur along the program, determining when each instruction can: Execute - when the result is ready, each EX stage notifies the scoreboard that it has completed execution; This step replaces the EX step in the original pipeline and takes multiple cycles in the FP pipeline; Prof. Nuno Roma ACE 2010/11 - DEI-IST 14 / 45

11 Centralized : The scoreboard records all dependencies and hazards that occur along the program, determining when each instruction can: Write/store, by checking the WAR data hazards and introducing stalls whenever necessary; Example of a WAR hazard: DIV.D F0, F2, F4 ADD.D F10, F0, F8 SUB.D F8, F8, F14 Prof. Nuno Roma ACE 2010/11 - DEI-IST 14 / 45 Centralized : The scoreboard records all dependencies and hazards that occur along the program, determining when each instruction can: Issue, by checking the structural and the WAW data hazards, introducing stalls whenever necessary; Read operands, by checking the RAW data hazards and introducing stalls whenever necessary; Execute - when the result is ready, each EX stage notifies the scoreboard that it has completed execution; Write/store, by checking the WAR data hazards and introducing stalls whenever necessary; The operands are read only when all operands are available at the register bank: there is NO forwarding!!! Prof. Nuno Roma ACE 2010/11 - DEI-IST 14 / 45

12 Centralized : The scoreboard is structured in 3 tables: Instruction-Status - indicates in which of the four steps the instruction is in; Prof. Nuno Roma ACE 2010/11 - DEI-IST 15 / 45 Centralized : The scoreboard is structured in 3 tables: Instruction-Status - indicates in which of the four steps the instruction is in; Functional Units Status - indicates the status of the functional unit, using the following 9 fields: Busy - the unit is busy or not; Op - type of operation; F i - destination register; F j, F k - source-register numbers; Q j, Q k - functional units producing source registers F j and F k ; R j, R k - flags indicating when F j and F k are ready and not yet read; set to No after operands are read. Prof. Nuno Roma ACE 2010/11 - DEI-IST 15 / 45

13 Centralized : The scoreboard is structured in 3 tables: Instruction-Status - indicates in which of the four steps the instruction is in; Functional Units Status - indicates the status of the functional unit, using the following 9 fields: Busy - the unit is busy or not; Op - type of operation; F i - destination register; F j, F k - source-register numbers; Q j, Q k - functional units producing source registers F j and F k ; R j, R k - flags indicating when F j and F k are ready and not yet read; set to No after operands are read. - indicates which functional unit will write each register. Prof. Nuno Roma ACE 2010/11 - DEI-IST 15 / 45 Centralized : Example: Instruction Status Instruction Issue Read Ops Ex Complete Write Res L.D F6,34(R2) L.D F2,45(R3) MUL.D F0,F2,F4 SUB.D F8,F6,F2 DIV.D F10,F0,F6 ADD.D F6,F8,F2 Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer Y Load F2 R3 Y Mult1 Y Mult F0 F2 F4 Int N Y Mult2 N Add Y Sub F8 F6 F2 Int Y N Div Y Div F10 F0 F6 Mult1 N Y F0 F2 F4 F6 F8 F10 F12 F30 Unit Mult1 Int Add Div Prof. Nuno Roma ACE 2010/11 - DEI-IST 16 / 45

14 Centralized : Example: Instruction Status Instruction Issue Read Ops Ex Complete Write Res L.D F6,34(R2) L.D F2,45(R3) MUL.D F0,F2,F4 SUB.D F8,F6,F2 DIV.D F10,F0,F6 ADD.D F6,F8,F2 What is the scoreboard status immediately before instruction MUL.D writes its result? Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer Y Load F2 R3 Y Mult1 Y Mult F0 F2 F4 Int N Y Mult2 N Add Y Sub F8 F6 F2 Int Y N Div Y Div F10 F0 F6 Mult1 N Y F0 F2 F4 F6 F8 F10 F12 F30 Unit Mult1 Int Add Div Prof. Nuno Roma ACE 2010/11 - DEI-IST 16 / 45 Centralized : Example: Instruction Status Instruction Issue Read Ops Ex Complete Write Res L.D F6,34(R2) L.D F2,45(R3) MUL.D F0,F2,F4 SUB.D F8,F6,F2 DIV.D F10,F0,F6 ADD.D F6,F8,F2 What is the scoreboard status immediately before instruction MUL.D writes its result? Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer YDIV.D Loadinstruction F2 R3 still didn t read its Y Mult1 Yoperands Mult due F0 tof2 itsf4data Intdependency N Y Mult2 N Add Ywith Sub MUL.D. F8 F6 F2 Int Y N Div Y Div F10 F0 F6 Mult1 N Y F0 F2 F4 F6 F8 F10 F12 F30 Unit Mult1 Int Add Div Prof. Nuno Roma ACE 2010/11 - DEI-IST 16 / 45

15 Centralized : Example: Instruction Status Instruction Issue Read Ops Ex Complete Write Res L.D F6,34(R2) L.D F2,45(R3) MUL.D F0,F2,F4 SUB.D F8,F6,F2 DIV.D F10,F0,F6 ADD.D F6,F8,F2 What is the scoreboard status immediately before instruction MUL.D writes its result? Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer YDIV.D Loadinstruction F2 R3 still didn t read its Y Mult1 Yoperands Mult due F0 tof2 itsf4data Intdependency N Y Mult2 N Add Ywith Sub MUL.D; F8 F6 F2 Int Y N Div Y Div F10 F0 F6 Mult1 N Y ADD.D instruction already read its operands and is under execution. F0 F2 F4 F6 F8 F10 F12 F30 Unit Mult1 Int Add Div Prof. Nuno Roma ACE 2010/11 - DEI-IST 16 / 45 Centralized : Example: Instruction Status Instruction Issue Read Ops Ex Complete Write Res L.D F6,34(R2) L.D F2,45(R3) MUL.D F0,F2,F4 SUB.D F8,F6,F2 DIV.D F10,F0,F6 ADD.D F6,F8,F2 Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer N Mult1 Y Mult F0 F2 F4 N N Mult2 N Add Y Add F6 F8 F2 N N Div Y Div F10 F0 F6 Mult1 N Y F0 F2 F4 F6 F8 F10 F12 F30 Unit Mult1 Add Div Prof. Nuno Roma ACE 2010/11 - DEI-IST 17 / 45

16 Centralized : Example: Instruction Status Instruction Issue Read Ops Ex Complete Write Res L.D F6,34(R2) L.D F2,45(R3) MUL.D F0,F2,F4 SUB.D F8,F6,F2 DIV.D F10,F0,F6 ADD.D F6,F8,F2 What is the scoreboard status immediately before instruction DIV.D writes its result? Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer N Mult1 Y Mult F0 F2 F4 N N Mult2 N Add Y Add F6 F8 F2 N N Div Y Div F10 F0 F6 Mult1 N Y F0 F2 F4 F6 F8 F10 F12 F30 Unit Mult1 Add Div Prof. Nuno Roma ACE 2010/11 - DEI-IST 17 / 45 Centralized : Example: Instruction Status Instruction Issue Read Ops Ex Complete Write Res L.D F6,34(R2) L.D F2,45(R3) MUL.D F0,F2,F4 SUB.D F8,F6,F2 DIV.D F10,F0,F6 ADD.D F6,F8,F2 What is the scoreboard status immediately before instruction DIV.D writes its result? Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer NInstruction ADD.D completed its execu- Mult1 Mult as soon F0 asf2instruction F4 DIV.D got N Ytion N Mult2 N Add Yinto the Add read F6 operands F8 F2 stage. N N Div Y Div F10 F0 F6 Mult1 N Y F0 F2 F4 F6 F8 F10 F12 F30 Unit Mult1 Add Div Prof. Nuno Roma ACE 2010/11 - DEI-IST 17 / 45

D F6,F8,F2 Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer N Mult1 N Mult Mult2 N Add N Div Y Div F10 F0

17 Centralized : Example: Instruction Status Instruction Issue Read Ops Ex Complete Write Res L.D F6,34(R2) L.D F2,45(R3) MUL.D F0,F2,F4 SUB.D F8,F6,F2 DIV.D F10,F0,F6 ADD.D F6,F8,F2 Functional Unit Status Unit Busy Op F i F j F k Q j Q k R j R k Integer N Mult1 N Mult Mult2 N Add N Div Y Div F10 F0 F6 Mult1 N N F0 F2 F4 F6 F8 F10 F12 F30 Unit Mult1 Add Div Prof. Nuno Roma ACE 2010/11 - DEI-IST 18 / 45 Centralized : First implementation of a scoreboard: CDC 6600 (1964) Prof. Nuno Roma ACE 2010/11 - DEI-IST 19 / 45

18 Centralized : Achieved speedup: 1.7 with Fortran applications; 2.5 with applications that were programmed and optimized in assembly. Prof. Nuno Roma ACE 2010/11 - DEI-IST 20 / 45 Centralized : Achieved speedup: 1.7 with Fortran applications; 2.5 with applications that were programmed and optimized in assembly. Problems: Large number of buses. Prof. Nuno Roma ACE 2010/11 - DEI-IST 20 / 45

19 Centralized : Achieved speedup: 1.7 with Fortran applications; 2.5 with applications that were programmed and optimized in assembly. Problems: Large number of buses. Limitations: The number of scoreboard entries - determines how far ahead the pipeline can look for independent instructions; The number and types of functional units. Prof. Nuno Roma ACE 2010/11 - DEI-IST 20 / 45 Prof. Nuno Roma ACE 2010/11 - DEI-IST 21 / 45

Distributed : Tomasulo : First implementation: IBM 360/91 (1964) Motivation: Only 4 floating-point registers; Very long memory accesses; Very long latency in the floating-point units. Prof.

20 Distributed : Tomasulo : First implementation: IBM 360/91 (1964) Motivation: Only 4 floating-point registers; Very long memory accesses; Very long latency in the floating-point units. Prof. Nuno Roma ACE 2010/11 - DEI-IST 22 / 45 Distributed : Tomasulo Minimization of data hazards: RAW: Each instruction is executed only when all its operands are available; WAR/WAW: Application of register renaming techniques: Renames all destination registers, including those with a pending read or write for an earlier instruction, so that the out-of-order write does not affect any instructions that depend on an earlier value of an operand. Prof. Nuno Roma ACE 2010/11 - DEI-IST 23 / 45

21 Distributed : Tomasulo Example (using temporary registers S and T): DIV.D F0, F2, F4 ADD.D F6, F0, F8 S.D F6, 0(R1) SUB.D F8, F10, F14 MUL.D F6, F10, F8 DIV.D F0, F2, F4 ADD.D S, F0, F8 S.D S, 0(R1) SUB.D T, F10, F14 MUL.D F6, F10, T With this renaming, the MUL.D instruction can write over F6 before the S.D instruction is finalized. Prof. Nuno Roma ACE 2010/11 - DEI-IST 24 / 45 Distributed : Tomasulo Solution to minimize the hazards: register renaming, accomplished through the use of reservation stations; Prof. Nuno Roma ACE 2010/11 - DEI-IST 25 / 45

22 Distributed : Tomasulo Each reservation station stores the information corresponding to a given instruction which has passed to the Issued status and is currently waiting for the release of the required execution unit or the availability of the required operands; Prof. Nuno Roma ACE 2010/11 - DEI-IST 25 / 45 Distributed : Tomasulo Once in the Issued status, each instruction no longer indexes the number of the register or the memory address of each operand; from now on, they are indexed using the reservation station number corresponding to the instruction which will produce the required operand value; Prof. Nuno Roma ACE 2010/11 - DEI-IST 25 / 45

23 Distributed : Tomasulo Memory writes, pending in the Store buffer, wait until the value which will be written and the corresponding effective address are available; Prof. Nuno Roma ACE 2010/11 - DEI-IST 25 / 45 Distributed : Tomasulo Each reservation station monitors the common data bus (CDB), where the several results are being transferred, looking for the required operands; Data forwarding mechanism, in order to eliminate the RAW hazards; Prof. Nuno Roma ACE 2010/11 - DEI-IST 25 / 45

24 Distributed : Tomasulo The reservation stations represent an extension to the register bank, providing an easy solution to most WAW and WAR hazards; Since the number of reservation stations is greater than the number of registers, the WAR and WAW are easily solved. Prof. Nuno Roma ACE 2010/11 - DEI-IST 25 / 45 Distributed : Tomasulo Control hazards??? Any instruction is NOT initiated until all previous branch predictions are confirmed. Prof. Nuno Roma ACE 2010/11 - DEI-IST 25 / 45

25 Distributed : Tomasulo Information accommodated in each reservation station: Op - type of operation; Q j, Q k - reservation stations that will produce the corresponding source operands (a value of zero indicates that the source operand is already available in V j and V k ); V j, V k - value of the source operands; A - effective address (load/store); Busy - reservation station is busy/free. Prof. Nuno Roma ACE 2010/11 - DEI-IST 26 / 45 Distributed : Tomasulo Example 1: L.D L.D MUL.D SUB.D DIV.D ADD.D F6, 32(R2) F2, 44(R3) F0, F2, F4 F8, F2, F6 F10, F0, F6 F6, F8, F2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 27 / 45

26 Distributed : Tomasulo Instruction Status Instruction Issue Execute Write Res L.D F6,32(R2) L.D F2,44(R3) MUL.D F0,F2,F4 SUB.D F8,F2,F6 DIV.D F10,F0,F6 ADD.D F6,F8,F2 (1) - After the first Load has Reservation finished its Station execution: Name Busy Op V j V k Q j Q k A Load1 N Load2 Y LOAD 44+Regs[R3] Add1 Y SUB Mem[32+Regs[R2]] Load2 Add2 Y ADD Add1 Load2 Add3 N Mult1 Y MUL Regs[F4] Load2 Mult2 Y DIV Mem[32+Regs[R2]] Mult1 F0 F2 F4 F6 F8 F10 F12 F30 Q i Mult1 Load2 Add2 Add1 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 28 / 45 Distributed : Tomasulo Instruction Status Instruction Issue Execute Write Res L.D F6,32(R2) L.D F2,44(R3) MUL.D F0,F2,F4 SUB.D F8,F2,F6 DIV.D F10,F0,F6 ADD.D F6,F8,F2 Reservation Station Name Busy Op V j V k Q j Q k A Load1 N Load2 Y LOAD 44+Regs[R3] Add1 Y SUB Mem[32+Regs[R2]] Load2 Add2 Y ADD Add1 Load2 Add3 N Mult1 Y MUL Regs[F4] Load2 Mult2 Y DIV Mem[32+Regs[R2]] Mult1 F0 F2 F4 F6 F8 F10 F12 F30 Q i Mult1 Load2 Add2 Add1 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 28 / 45

27 Distributed : Tomasulo Instruction Status Instruction Issue Execute Write Res L.D F6,32(R2) L.D F2,44(R3) MUL.D F0,F2,F4 SUB.D F8,F2,F6 DIV.D F10,F0,F6 ADD.D F6,F8,F2 (1) - After the first Load has Reservation finished its Station execution: Name Busy Op V j V k Q j Q k A Load1 N ADD.D instruction, associated to a WAR hazard in Load2 Y LOAD the WB phase, can finish even before the DIV instruction starts Add1 Y SUB Mem[32+Regs[R2]] its execution!!! Load2 44+Regs[R3] Add2 Y ADD Add1 Load2 Add3 N Mult1 Y MUL Regs[F4] Load2 Mult2 Y DIV Mem[32+Regs[R2]] Mult1 F0 F2 F4 F6 F8 F10 F12 F30 Q i Mult1 Load2 Add2 Add1 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 28 / 45 Distributed : Tomasulo Instruction Status Instruction Issue Execute Write Res L.D F6,32(R2) L.D F2,44(R3) MUL.D F0,F2,F4 SUB.D F8,F2,F6 DIV.D F10,F0,F6 ADD.D F6,F8,F2 Reservation Station Name Busy Op V j V k Q j Q k A Load1 N Load2 Y LOAD 44+Regs[R3] Add1 Y SUB Mem[32+Regs[R2]] Load2 Add2 Y ADD Add1 Load2 Add3 N Mult1 Y MUL Regs[F4] Load2 Mult2 Y DIV Mem[32+Regs[R2]] Mult1 F0 F2 F4 F6 F8 F10 F12 F30 Q i Mult1 Load2 Add2 Add1 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 28 / 45

28 Distributed : Tomasulo Instruction Status Instruction Issue Execute Write Res L.D F6,32(R2) L.D F2,44(R3) MUL.D F0,F2,F4 SUB.D F8,F2,F6 DIV.D F10,F0,F6 ADD.D F6,F8,F2 (2) - MUL.D is ready to write Reservation its result: Station Name Busy Op V j V k Q j Q k A Load1 N Load2 N Add1 N Add2 N Add3 N Mult1 Y MUL Mem[44+Regs[R3]] Regs[F4] Mult2 Y DIV Mem[32+Regs[R2]] Mult1 F0 F2 F4 F6 F8 F10 F12 F30 Q i Mult1 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 29 / 45 Distributed : Tomasulo Instruction Status Instruction Issue Execute Write Res L.D F6,32(R2) L.D F2,44(R3) MUL.D F0,F2,F4 SUB.D F8,F2,F6 DIV.D F10,F0,F6 ADD.D F6,F8,F2 Reservation Station Name Busy Op V j V k Q j Q k A Load1 N Load2 N Add1 N Add2 N Add3 N Mult1 Y MUL Mem[44+Regs[R3]] Regs[F4] Mult2 Y DIV Mem[32+Regs[R2]] Mult1 F0 F2 F4 F6 F8 F10 F12 F30 Q i Mult1 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 29 / 45

29 Distributed : Tomasulo Instruction Status Instruction Issue Execute Write Res L.D F6,32(R2) L.D F2,44(R3) MUL.D F0,F2,F4 SUB.D F8,F2,F6 DIV.D F10,F0,F6 ADD.D F6,F8,F2 (2) - MUL.D is ready to write Reservation its result: Station Name Busy Op V j V k Q j Q k A Load1 N ADD.D instruction has already finished!!! Load2 N Add1 N Add2 N Add3 N Mult1 Y MUL Mem[44+Regs[R3]] Regs[F4] Mult2 Y DIV Mem[32+Regs[R2]] Mult1 F0 F2 F4 F6 F8 F10 F12 F30 Q i Mult1 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 29 / 45 Distributed : Tomasulo Instruction Status Instruction Issue Execute Write Res L.D F6,32(R2) L.D F2,44(R3) MUL.D F0,F2,F4 SUB.D F8,F2,F6 DIV.D F10,F0,F6 ADD.D F6,F8,F2 Reservation Station Name Busy Op V j V k Q j Q k A Load1 N Load2 N Add1 N Add2 N Add3 N Mult1 Y MUL Mem[44+Regs[R3]] Regs[F4] Mult2 Y DIV Mem[32+Regs[R2]] Mult1 F0 F2 F4 F6 F8 F10 F12 F30 Q i Mult1 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 29 / 45

30 Distributed : Tomasulo Example 2 (loop): Loop: L.D F0, 0(R1) MUL.D F4, F0, F2 S.D F4, 0(R1) DADDIU R1, R1, -8 BNE R1, R2, Loop; Branches if R1 R2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 30 / 45 Distributed : Tomasulo Instruction Status Instruction Iteration Issue Execute Write Res L.D F0, 0(R1) 1 MUL.D F4, F0, F2 1 S.D F4, 0(R1) 1 L.D F0, 0(R1) 2 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 2 Reservation Station Name Busy Op V j V k Q j Q k A Load1 Y Load Regs[R1]+0 Load2 Y Load Regs[R1]-8 Add1 N Add2 N Add3 N Mult1 Y MUL Regs[F2] Load1 Mult2 Y MUL Regs[F2] Load2 Store1 Y Store Regs[R1] Mult1 Store2 Y Store Regs[R1]-8 Mult2 F0 F2 F4 F6 F8 F10 F12 F30 Q i Load2 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 31 / 45

31 Distributed : Tomasulo Instruction Status Instruction Iteration Issue Execute Write Res L.D F0, 0(R1) 1 MUL.D F4, F0, F2 1 S.D F4, 0(R1) 1 L.D F0, 0(R1) 2 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 2 Reservation Station Name Busy Op V j V k Q j Q k A (1) - The loop is dynamically unrolled by Load1 Y Load Regs[R1]+0 Load2 they HW, Load by renaming the operands using the Regs[R1]-8 Add1 N reservation stations as extra registers: Add2 N Add3 N Mult1 Y MUL None of the instructions Regs[F2] Load1 has already Mult2 Y MUL Regs[F2] Load2 finished... Store1 Y Store Regs[R1] Mult1 Store2 Y Store Regs[R1]-8 Mult2 F0 F2 F4 F6 F8 F10 F12 F30 Q i Load2 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 31 / 45 Distributed : Tomasulo Instruction Status Instruction Iteration Issue Execute Write Res L.D F0, 0(R1) 1 MUL.D F4, F0, F2 1 S.D F4, 0(R1) 1 L.D F0, 0(R1) 2 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 2 Reservation Station Name Busy Op V j V k Q j Q k A Load1 Y Load Regs[R1]+0 Load2 Y Load Regs[R1]-8 Add1 N Add2 N Add3 N Mult1 Y MUL Regs[F2] Load1 Mult2 Y MUL Regs[F2] Load2 Store1 Y Store Regs[R1] Mult1 Store2 Y Store Regs[R1]-8 Mult2 F0 F2 F4 F6 F8 F10 F12 F30 Q i Load2 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 31 / 45

32 Distributed : Tomasulo Instruction Status Instruction Iteration Issue Execute Write Res L.D F0, 0(R1) 1 MUL.D F4, F0, F2 1 S.D F4, 0(R1) 1 L.D F0, 0(R1) 2 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 2 Reservation Station Name Busy Op V j V k Q j Q k A (2) - The memory accesses with instructions Load/Store can be executed out-of-order pro- Load1 Y Load Regs[R1]+0 Load2 Y Load Regs[R1]-8 Add1 vided N that they use distinct addresses. Add2 N Add3 N Mult1 Y MUL Regs[F2] Load1 Mult2 Y MUL Regs[F2] Load2 Store1 Y Store Regs[R1] Mult1 Store2 Y Store Regs[R1]-8 Mult2 F0 F2 F4 F6 F8 F10 F12 F30 Q i Load2 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 31 / 45 Distributed : Tomasulo Instruction Status Instruction Iteration Issue Execute Write Res L.D F0, 0(R1) 1 MUL.D F4, F0, F2 1 S.D F4, 0(R1) 1 L.D F0, 0(R1) 2 MUL.D F4, F0, F2 2 S.D F4, 0(R1) 2 Reservation Station Name Busy Op V j V k Q j Q k A Load1 Y Load Regs[R1]+0 Load2 Y Load Regs[R1]-8 Add1 N Add2 N Add3 N Mult1 Y MUL Regs[F2] Load1 Mult2 Y MUL Regs[F2] Load2 Store1 Y Store Regs[R1] Mult1 Store2 Y Store Regs[R1]-8 Mult2 F0 F2 F4 F6 F8 F10 F12 F30 Q i Load2 Mult2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 31 / 45

33 Distributed : Tomasulo Implementation of algorithm: Advantages: The hazard detection and the execution control is now distributed, contrary to what happens with the scoreboard: The information that is stored in the reservation station of each functional unit determines when the instruction can start its execution. The results are directly passed to the functional units through the reservation stations, instead of being passed through the registers: Elimination of WAW and WAR hazards Prof. Nuno Roma ACE 2010/11 - DEI-IST 32 / 45 Prof. Nuno Roma ACE 2010/11 - DEI-IST 33 / 45

34 consists in: Allowing the anticipated execution of a given instruction that is still conditioned (e.g.: previous branch condition); Passing of its result to other instructions, but preventing that they irreversibly commit their final result until the instruction is no longer speculative. Prof. Nuno Roma ACE 2010/11 - DEI-IST 34 / 45 consists in: Allowing the anticipated execution of a given instruction that is still conditioned (e.g.: previous branch condition); Passing of its result to other instructions, but preventing that they irreversibly commit their final result until the instruction is no longer speculative. Combines three techniques: Dynamic branch prediction, to select the instructions that should be executed; Speculation, to allow the execution of instructions before their control dependencies have been resolved and assuring the ability to rollback the effect of incorrectly speculated code sequences; Dynamic scheduling, to schedule the several pipeline stages. Prof. Nuno Roma ACE 2010/11 - DEI-IST 34 / 45

35 consists in: Allowing the anticipated execution of a given instruction that is still conditioned (e.g.: previous branch condition); Passing of its result to other instructions, but preventing that they irreversibly commit their final result until the instruction is no longer speculative. Combines three techniques: Dynamic branch prediction, to select the instructions that should be executed; Speculation, to allow the execution of instructions before their control dependencies have been resolved and assuring the ability to rollback the effect of incorrectly speculated code sequences; Dynamic scheduling, to schedule the several pipeline stages. Although the instructions execute out-of-order, the final commit step is executed in the correct order. Prof. Nuno Roma ACE 2010/11 - DEI-IST 34 / 45 Contrasting with dynamic scheduling techniques, with speculative scheduling the instructions continue through the fetch, issue and execute stages, assuming that the previous branch prediction is correct. Prof. Nuno Roma ACE 2010/11 - DEI-IST 35 / 45

36 Contrasting with dynamic scheduling techniques, with speculative scheduling the instructions continue through the fetch, issue and execute stages, assuming that the previous branch prediction is correct. Extension to algorithm to allow speculation: How: distinction between by-passing the results between instructions (using the reservation stations) and the effective write in the register bank; Why: allow speculative execution of instructions (with by-pass of their results), but preventing that such results are effectively written until the instruction is no longer speculative, i.e., until the previous branch predictions have been confirmed (instruction commit). Prof. Nuno Roma ACE 2010/11 - DEI-IST 35 / 45 Contrasting with dynamic scheduling techniques, with speculative scheduling the instructions continue through the fetch, issue and execute stages, assuming that the previous branch prediction is correct. Extension to algorithm to allow speculation: How: distinction between by-passing the results between instructions (using the reservation stations) and the effective write in the register bank; Why: allow speculative execution of instructions (with by-pass of their results), but preventing that such results are effectively written until the instruction is no longer speculative, i.e., until the previous branch predictions have been confirmed (instruction commit). The instructions are executed out-of-order, but they finish (commit) in the original order. Prof. Nuno Roma ACE 2010/11 - DEI-IST 35 / 45

37 Dynamic scheduling: after the fetch and issue stages, the instructions are blocked in the reservation stations until their operand values are available; Implementation of branches: the instructions after the branch are not executed while the branch condition is not evaluated. Prof. Nuno Roma ACE 2010/11 - DEI-IST 36 / 45 Reorder Buffer scheduling: all instructions whose operands values or origins are already known pass through the fetch, issue and execute stages; Implementation of branches: the instructions after the branch are executed and their results are temporarily stored in the Reorder Buffer, until the branch condition is known. Prof. Nuno Roma ACE 2010/11 - DEI-IST 36 / 45

38 Reorder Buffer: Assume a role quite similar to the reservation stations, by supplying partial results to subsequent instructions; Each entry contains 4 fields: Type of instruction - e.g.: branch/store/register operation; Destination - target register or memory address; Value - instruction result; Ready - conclusion of the operation. Prof. Nuno Roma ACE 2010/11 - DEI-IST 37 / 45 Commit phase: occurs when the instruction reaches the top of the Reorder Buffer and its result is ready; Prof. Nuno Roma ACE 2010/11 - DEI-IST 38 / 45

39 Commit phase: occurs when the instruction reaches the top of the Reorder Buffer and its result is ready; If the branch prediction was correct: the target register/memory position is updated with the new value; Prof. Nuno Roma ACE 2010/11 - DEI-IST 38 / 45 Commit phase: occurs when the instruction reaches the top of the Reorder Buffer and its result is ready; If the branch prediction was incorrect: the speculation is wrong and the Reorder Buffer is completely discarded, restarting the execution at the instruction that corresponds to the correct branch target. Prof. Nuno Roma ACE 2010/11 - DEI-IST 38 / 45

40 Commit phase: occurs when the instruction reaches the top of the Reorder Buffer and its result is ready; In-Order Commit!!!! Precise Interruptions!!! Prof. Nuno Roma ACE 2010/11 - DEI-IST 38 / 45 Example 1: L.D L.D MUL.D SUB.D DIV.D ADD.D F6, 32(R2) F2, 44(R3) F0, F2, F4 F8, F6, F2 F10, F0, F6 F6, F8, F2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 39 / 45

41 Reorder Buffer Busy Instruction Status Destination Value 1 N L.D F6, 32(R2) Commit F6 Mem[32+Regs[R2]] 2 N L.D F2, 44(R3) Commit F2 Mem[44+Regs[R3]] 3 Y MUL.D F0, F2, F4 Write result F0 #2 Regs[F4] 4 Y SUB.D F8, F6, F2 Write result F8 #2 - #1 5 Y DIV.D F10, F0, F6 Execute F10 6 Y ADD.D F6, F8, F2 Write result F6 #4 - #2 Reservation Stations Name Busy Op V j V k Q j Q k Dest A Load1 N Load2 N Add1 N Add2 N Add3 N Mult1 N MUL.D Mem[44+Regs[R3]] Regs[F4] #3 Mult2 Y DIV.D Mem[32+Regs[R2]] #3 #5 F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 Re-Order # Busy Y N N N N N Y N Y N Y Prof. Nuno Roma ACE 2010/11 - DEI-IST 40 / 45 Reorder Buffer Busy Instruction Status Destination Value 1 N L.D F6, 32(R2) Commit F6 Mem[32+Regs[R2]] 2 N L.D F2, 44(R3) Commit F2 Mem[44+Regs[R3]] 3 Y MUL.D F0, F2, F4 Write result F0 #2 Regs[F4] 4 Y SUB.D F8, F6, F2 Write result F8 #2 - #1 5 Y DIV.D F10, F0, F6 Execute F10 6 Y ADD.D F6, F8, F2 Write result F6 #4 - #2 By the time MUL.D instruction is ready to Commit, only the two LD.D instructions have already Committed, although several other have already finished their execution. Reservation Stations Name Busy Op V j V k Q j Q k Dest A Load1 N Load2 N Add1 N Add2 N Add3 N Mult1 N MUL.D Mem[44+Regs[R3]] Regs[F4] #3 Mult2 Y DIV.D Mem[32+Regs[R2]] #3 #5 F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 Re-Order # Busy Y N N N N N Y N Y N Y Prof. Nuno Roma ACE 2010/11 - DEI-IST 40 / 45

42 Reorder Buffer Busy Instruction Status Destination Value 1 N L.D F6, 32(R2) Commit F6 Mem[32+Regs[R2]] 2 N L.D F2, 44(R3) Commit F2 Mem[44+Regs[R3]] 3 Y MUL.D F0, F2, F4 Write result F0 #2 Regs[F4] 4 Y SUB.D F8, F6, F2 Write result F8 #2 - #1 5 Y DIV.D F10, F0, F6 Execute F10 6 Y ADD.D F6, F8, F2 Write result F6 #4 - #2 Reservation Stations Name Busy Op V j V k Q j Q k Dest A Load1 N Load2 N Add1 N Add2 N Add3 N Mult1 N MUL.D Mem[44+Regs[R3]] Regs[F4] #3 Mult2 Y DIV.D Mem[32+Regs[R2]] #3 #5 F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 Re-Order # Busy Y N N N N N Y N Y N Y Prof. Nuno Roma ACE 2010/11 - DEI-IST 40 / 45 Example 2 (loop): Loop: L.D F0, 0(R1) MUL.D F4, F0, F2 S.D F4, 0(R1) DADDIU R1, R1, -8 BNE R1, R2, Loop; Branches if R1 R2 Prof. Nuno Roma ACE 2010/11 - DEI-IST 41 / 45

43 Reorder Buffer Busy Instruction Status Destination Value 1 N L.D F0, 0(R1) Commit F0 Mem[0+Regs[R1]] 2 N MUL.D F4, F0, F2 Commit F4 #1 Regs[F2] 3 Y S.D F4, 0(R1) Write result 0+Regs[R1] #2 4 Y DADDIU R1, R1, -8 Write result R1 Regs[R1]-8 5 Y BNE R1, R2, Loop Write result 6 N L.D F0, 0(R1) Write result F0 Mem[#4] 7 N MUL.D F4, F0, F2 Write result F4 #6 Regs[F2] 8 Y S.D F4, 0(R1) Write result 0+Regs[R1] #7 9 Y DADDIU R1, R1, -8 Write result R1 # Y BNE R1, R2, Loop Write result Reservation Station Name Busy Op V j V k Q j Q k Dest A empty F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 Re-Order # 6 7 Busy Y N N N Y N N N N N N Prof. Nuno Roma ACE 2010/11 - DEI-IST 42 / 45 Reorder Buffer Busy Instruction Status Destination Value 1 N L.D F0, 0(R1) Commit F0 Mem[0+Regs[R1]] 2 N MUL.D F4, F0, F2 Commit F4 #1 Regs[F2] 3 Y S.D F4, 0(R1) Write result 0+Regs[R1] #2 4 Y DADDIU R1, R1, -8 Write result R1 Regs[R1]-8 5 Y BNE R1, R2, Loop Write result 6 N L.D F0, 0(R1) Write result F0 Mem[#4] 7 N MUL.D F4, F0, F2 Write result F4 #6 Regs[F2] 8 Y S.D F4, 0(R1) Write result 0+Regs[R1] #7 Only LD.D and MUL.D Committed, although 9 Y DADDIU R1, R1, -8 result R1 # Y all BNE other R1, R2, instructions Loop Writehave result already finished their execution. Reservation Station Name Busy Op V j V k Q j Q k Dest A empty F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 Re-Order # 6 7 Busy Y N N N Y N N N N N N Prof. Nuno Roma ACE 2010/11 - DEI-IST 42 / 45

44 Reorder Buffer Busy Instruction Status Destination Value 1 N L.D F0, 0(R1) Commit F0 Mem[0+Regs[R1]] 2 N MUL.D F4, F0, F2 Commit F4 #1 Regs[F2] 3 Y S.D F4, 0(R1) Write result 0+Regs[R1] #2 4 Y DADDIU R1, R1, -8 Write result R1 Regs[R1]-8 5 Y BNE R1, R2, Loop Write result 6 N L.D F0, 0(R1) Write result F0 Mem[#4] 7 N MUL.D F4, F0, F2 Write result F4 #6 Regs[F2] 8 Y S.D F4, 0(R1) Write result 0+Regs[R1] #7 9 Y DADDIU R1, R1, -8 Write result R1 # Y BNE R1, R2, Loop Write result Reservation Station Name Busy Op V j V k Q j Q k Dest A empty F0 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 Re-Order # 6 7 Busy Y N N N Y N N N N N N Prof. Nuno Roma ACE 2010/11 - DEI-IST 42 / 45 Interrupt handling? The exception is recorded in the entry of the Reorder Buffer corresponding to the instruction under execution; It is only treated, in the Commit phase, if the instruction reveals to be NOT speculative. Prof. Nuno Roma ACE 2010/11 - DEI-IST 43 / 45

45 Prof. Nuno Roma ACE 2010/11 - DEI-IST 44 / 45 Multiple-issue processors; Superscalar processors; Very Long Instrucion Word (VLIW) processors; Code optimization for multiple-issue processors; Multi-threading. Prof. Nuno Roma ACE 2010/11 - DEI-IST 45 / 45

The basic structure of a MIPS floating-point unit

The basic structure of a MIPS floating-point unit Tomasulo s scheme The algorithm based on the idea of reservation station The reservation station fetches and buffers an operand as soon as it is available, eliminating the need to get the operand from