CPU Organization Datapath Design:

Transcription

1 CPU Organization Datapath Design: Capabilities & performance characteristics of principal Functional Units (FUs): (e.g., Registers, ALU, Shifters, Logic Units,...) Ways in which these components are interconnected (buses connections, multiplexors, etc.). How information flows between components. Control Unit Design: Logic and means by which such information flow is controlled. Control and coordination of FUs operation to realize the targeted Instruction Set Architecture to be implemented (can either be implemented using a finite state machine or a microprogram). Hardware description with a suitable language, possibly using Register Transfer Notation (RTN). #1 Final Review Winter

2 Hierarchy of Computer Architecture High-Level Language Programs Software Machine Language Program Software/Hardware Boundary Hardware Logic Diagrams Application Compiler Instr. Set Proc. Operating System Digital Design Circuit Design Firmware I/O system Datapath & Control Layout Assembly Language Programs Instruction Set Architecture Microprogram Register Transfer Notation (RTN) Circuit Diagrams #2 Final Review Winter

3 Computer Performance Measures: Program Execution Time For a specific program compiled to run on a specific machine A, the following parameters are provided: The total instruction count of the program. The average number of cycles per instruction (average CPI). Clock cycle of machine A How can one measure the performance of this machine running this program? Intuitively the machine is said to be faster or has better performance running this program if the total execution time is shorter. Thus the inverse of the total measured program execution time is a possible performance measure or metric: Performance A = 1 / Execution Time A How to compare performance of different machines? What factors affect performance? How to improve performance? #3 Final Review Winter

4 CPU Execution Time: The CPU Equation A program is comprised of a number of instructions Measured in: instructions/program The average instruction takes a number of cycles per instruction (CPI) to be completed. Measured in: cycles/instruction CPU has a fixed clock cycle time = 1/clock rate Measured in: seconds/cycle CPU execution time is the product of the above three parameters as follows: CPU CPU time time = Seconds = Instructions x Cycles Cycles x Seconds Program Program Instruction Cycle Cycle #4 Final Review Winter

5 Factors Affecting CPU Performance CPU CPU time time = Seconds = Instructions x Cycles Cycles x Seconds Program Program Instruction Cycle Cycle Program Compiler Instruction Set Architecture (ISA) Organization Technology Instruction Count X X X CPI X X X X Clock Rate X X #5 Final Review Winter

6 Aspects of CPU Execution Time CPU Time = Instruction count x CPI x Clock cycle Depends on: Program Used Compiler ISA Instruction Count Depends on: Program Used Compiler ISA CPU Organization CPI Clock Cycle Depends on: CPU Organization Technology #6 Final Review Winter

7 Instruction Types & CPI: An Example An instruction set has three instruction classes: Instruction class CPI A 1 B 2 C 3 Two code sequences have the following instruction counts: Instruction counts for instruction class Code Sequence A B C CPU cycles for sequence 1 = 2 x x x 3 = 10 cycles CPI for sequence 1 = clock cycles / instruction count = 10 /5 = 2 CPU cycles for sequence 2 = 4 x x x 3 = 9 cycles CPI for sequence 2 = 9 / 6 = 1.5 #7 Final Review Winter

8 Instruction Frequency & CPI Given a program with n types or classes of instructions with the following characteristics: C i = Count of instructions of type i CPI i = Average cycles per instruction of type i F i = Frequency of instruction type i = C i / total instruction count Then: CPI = n ( ) i= 1 CPI i F i #8 Final Review Winter

9 Instruction Type Frequency & CPI: A RISC Example Base Machine (Reg / Reg) Op Freq Cycles CPI(i) % Time ALU 50% % Load 20% % Store 10% % Branch 20% % CPI = Typical Mix n ( ) i = 1 CPI i F i CPI =.5 x x x x 2 = 2.2 #9 Final Review Winter

10 Metrics of Computer Performance Application Execution time: Target workload, SPEC95, etc. Programming Language Compiler ISA (millions) of Instructions per second MIPS (millions) of (F.P.) operations per second MFLOP/s Datapath Control Function Units Transistors Wires Pins Megabytes per second. Cycles per second (clock rate). Each metric has a purpose, and each can be misused. #10 Final Review Winter

11 Performance Enhancement Calculations: Amdahl's Law The performance enhancement possible due to a given design improvement is limited by the amount that the improved feature is used Amdahl s Law: Performance improvement or speedup due to enhancement E: Execution Time without E Performance with E Speedup(E) = = Execution Time with E Performance without E Suppose that enhancement E accelerates a fraction F of the execution time by a factor S and the remainder of the time is unaffected then: Execution Time with E = ((1-F) + F/S) X Execution Time without E Hence speedup is given by: Execution Time without E 1 Speedup(E) = = ((1 - F) + F/S) X Execution Time without E (1 - F) + F/S Note: All fractions here refer to original execution time. #11 Final Review Winter

12 Pictorial Depiction of Amdahl s Law Enhancement E accelerates fraction F of execution time by a factor of S Before: Execution Time without enhancement E: Unaffected, fraction: (1- F) Affected fraction: F Unchanged Unaffected, fraction: (1- F) F/S After: Execution Time with enhancement E: Execution Time without enhancement E 1 Speedup(E) = = Execution Time with enhancement E (1 - F) + F/S #12 Final Review Winter

13 Amdahl's Law With Multiple Enhancements: Example Three CPU performance enhancements are proposed with the following speedups and percentage of the code execution time affected: Speedup 1 = S 1 = 10 Percentage 1 = F 1 = 20% Speedup 2 = S 2 = 15 Percentage 1 = F 2 = 15% Speedup 3 = S 3 = 30 Percentage 1 = F 3 = 10% While all three enhancements are in place in the new design, each enhancement affects a different portion of the code and only one enhancement can be used at a time. What is the resulting overall speedup? Speedup = ( ( 1 ) i F 1 i + i F S i i ) Speedup = 1 / [( ) +.2/ /15 +.1/30)] = 1 / [ ] = 1 /.5833 = 1.71 #13 Final Review Winter

14 MIPS Instruction Formats I-Type: R-Type ALU Load/Store, Branch J-Type: Jumps op rs rt rd shamt funct 6 bits 5 bits 5 bits 5 bits 5 bits 6 bits op rs rt immediate 6 bits 5 bits 5 bits 16 bits 26 op target address 6 bits 26 bits op: Opcode, operation of the instruction. rs, rt, rd: The source and destination register specifiers. shamt: Shift amount. funct: selects the variant of the operation in the op field. address / immediate: Address offset or immediate value. target address: Target address of the jump instruction. #14 Final Review Winter

15 A Single Cycle MIPS Datapath Inst Memory Adr Rs <21:25> Rt <16:20> Rd <11:15> <0:15> Imm16 Instruction<31:0> imm16 4 PC Ext Adder Adder npc_sel PC Mux Clk 00 RegDst RegWr busw 32 Clk 1 imm16 Rd 0 Rt 5 5 Rs 5 Rw Ra Rb bit Registers 16 Rt busa busb 32 Extender Equal Mux 1 ALUctr = ALU MemWr WrEn Adr Data In Data Memory Clk MemtoReg 0 Mux 1 ExtOp ALUSrc #15 Final Review Winter

16 Instruction Memory Adr Instruction<31:0> <0:15> <11:15> <16:20> <21:25> <21:25> <0:25> Op Fun Rt Rs Rd Imm16 Jump_target Control Unit npc_sel RegWr RegDst ExtOp ALUSrc ALUctr MemWr MemtoReg Jump Equal DATA PATH #16 Final Review Winter

17 The Truth Table For The Main Control op R-type ori lw sw beq jump RegDst ALUSrc MemtoReg RegWrite MemWrite Branch Jump ExtOp ALUop (Symbolic) x R-type Or Add x 1 x Add x 0 x x Subtract x x x x xxx ALUop <2> x ALUop <1> x ALUop <0> x #17 Final Review Winter

18 Clk PC Old Value Rs, Rt, Rd, Op, Func ALUctr Worst Case Timing (Load) Clk-to-Q New Value Old Value Old Value Instruction Memoey Access Time New Value Delay through Control Logic New Value ExtOp Old Value New Value ALUSrc Old Value New Value MemtoReg Old Value New Value RegWr Old Value New Value busa busb Old Value Delay through Extender & Mux Old Value Register Write Occurs Register File Access Time New Value New Value ALU Delay Address Old Value New Value Data Memory Access Time busw Old Value New #18 Final Review Winter

19 MIPS Single Cycle Instruction Timing Comparison Arithmetic & Logical PC Inst Memory Reg File mux ALU mux setup Load PC Inst Memory Reg File mux ALU Data Mem mux Critical Path Store PC Inst Memory Reg File mux ALU Data Mem Branch PC Inst Memory Reg File cmp mux setup #19 Final Review Winter

20 CPU Design Steps 1. Analyze instruction set operations using independent RTN => datapath requirements. 2. Select set of datapath components & establish clock methodology. 3. Assemble datapath meeting the requirements. 4. Analyze implementation of each instruction to determine setting of control points that effects the register transfer. 5. Assemble the control logic. #20 Final Review Winter

21 Drawback of Single Cycle Processor Long cycle time. All instructions must take as much time as the slowest: Cycle time for load is longer than needed for all other instructions. Real memory is not as well-behaved as idealized memory Cannot always complete data access in one (short) cycle. #21 Final Review Winter

22 Reducing Cycle Time: Multi-Cycle Design Cut combinational dependency graph by inserting registers / latches. The same work is done in two or more fast cycles, rather than one slow cycle. storage element storage element Acyclic Combinational Logic => Acyclic Combinational Logic (A) storage element storage element Acyclic Combinational Logic (B) storage element #22 Final Review Winter

23 Instruction Processing Cycles Instruction Fetch Next Instruction Obtain instruction from program storage Update program counter to address of next instruction }Common steps for all instructions Instruction Decode Determine instruction type Obtain operands from registers Execute Compute result value or status Result Store Store result in register/memory if needed (usually called Write Back). #23 Final Review Winter

24 Partitioning The Single Cycle Datapath Add registers between smallest steps Next PC PC Instruction Fetch npc_sel Operand Fetch ExtOp ALUSrc ALUctr MemRd MemWr RegDst RegWr MemWr Exec Mem Access Reg. File Data Mem Result Store #24 Final Review Winter

25 npc_sel Next PC PC Example Multi-cycle Datapath Instruction Fetch IR Reg File Operand Fetch A B ExtOp ALUSrc ALUctr Ext ALU R MemRd MemWr Mem Access M MemToReg Data Mem RegDst RegWr Reg. File Registers added: IR: Instruction register A, B: Two registers to hold operands read from register file. R: or ALUOut, holds the output of the ALU M: or Memory data register (MDR) to hold data read from data memory Result Store Equal #25 Final Review Winter

26 Operations In Each Cycle R-Type Logic Immediate Load Store Branch Instruction Fetch IR Mem[PC] IR Mem[PC] IR Mem[PC] IR Mem[PC] IR Mem[PC] Instruction Decode A R[rs] B R[rt] A R[rs] A R[rs] A R[rs] B R[rt] A R[rs] B R[rt] If Equal = 1 Execution R A + B R A OR ZeroExt[imm16] R A + SignEx(Im16) R A + SignEx(Im16) PC PC (SignExt(imm16) x4) else PC PC + 4 Memory M Mem[R] Mem[R] B PC PC + 4 Write Back R[rd] R PC PC + 4 R[rt] R PC PC + 4 R[rd] M PC PC + 4 #26 Final Review Winter

27 Finite State Machine (FSM) Control Model State specifies control points for Register Transfer. Transfer occurs upon exiting state (same falling edge). inputs (conditions) Next State Logic Control State State X Register Transfer Control Points Depends on Input Output Logic outputs (control points) #27 Final Review Winter

28 Control Specification For Multi-cycle CPU Finite State Machine (FSM) IR MEM[PC] instruction fetch A R[rs] B R[rt] decode / operand fetch Execute R-type R A fun B ORi R A or ZX LW R A + SX SW R A + SX BEQ & ~Equal PC PC + 4 BEQ & Equal PC PC + SX 00 Memory M MEM[R] MEM[R] B PC PC + 4 To instruction fetch R[rd] R PC PC + 4 R[rt] R PC PC + 4 R[rt] M PC PC + 4 Write-back To instruction fetch To instruction fetch #28 Final Review Winter

29 Traditional FSM Controller datapath + state diagram => control Translate RTN statements into control points. Assign states. Implement the controller. #29 Final Review Winter

30 Mapping RTNs To Control Points Examples & State Assignments IR MEM[PC] instruction fetch 0000 imem_rd, IRen Execute ALUfun, Sen R-type R A fun B 0100 Aen, Ben ORi R A or ZX 0110 A R[rs] B R[rt] 0001 LW R A + SX 1000 SW decode / operand fetch R A + SX 1011 BEQ & ~Equal PC PC BEQ & Equal PC PC + SX Memory RegDst, RegWr, PCen M MEM[S] 1001 MEM[S] B PC PC To instruction fetch state 0000 R[rd] R PC PC R[rt] R PC PC R[rt] M PC PC Write-back To instruction fetch state 0000 To instruction fetch state 0000 #30 Final Review Winter

31 R Detailed Control Specification State Op field Eq Next IR PC Ops Exec Mem Write-Back en sel A B Ex Sr ALU S R W M M-R Wr Dst 0000??????? BEQ BEQ R-type x ori x LW x SW x xxxxxx x xxxxxx x xxxxxx x fun xxxxxx x xxxxxx x or xxxxxx x xxxxxx x add xxxxxx x xxxxxx x xxxxxx x add xxxxxx x #31 Final Review Winter

32 Alternative Multiple Cycle Datapath (In Textbook) Shared instruction/data memory unit A single ALU shared among instructions Shared units require additional or widened multiplexors Temporary registers to hold data between clock cycles of the instruction: Additional registers: Instruction Register (IR), Memory Data Register (MDR), A, B, ALUOut #32 Final Review Winter

33 Operations In Each Cycle R-Type Logic Immediate Load Store Branch Instruction Fetch IR Mem[PC] PC PC + 4 IR Mem[PC] PC PC + 4 IR Mem[PC] PC PC + 4 IR Mem[PC] PC PC + 4 IR Mem[PC] PC PC + 4 Instruction Decode A R[rs] B R[rt] ALUout PC + (SignExt(imm16) x4) A R[rs] B R[rt] ALUout PC + (SignExt(imm16) x4) A R[rs] B R[rt] ALUout PC + (SignExt(imm16) x4) A R[rs] B R[rt] ALUout PC + (SignExt(imm16) x4) A B R[rs] R[rt] ALUout PC + (SignExt(imm16) x4) Execution ALUout A + B ALUout A OR ZeroExt[imm16] ALUout A + SignEx(Im16) ALUout A + SignEx(Im16) If Equal = 1 PC ALUout Memory M Mem[ALUout] Mem[ALUout] B Write Back R[rd] ALUout R[rt] ALUout R[rd] Mem #33 Final Review Winter

34 Finite State Machine (FSM) Specification IR MEM[PC] PC PC instruction fetch A R[rs] B R[rt] ALUout PC +SX 0001 decode Execute R-type ALUout A fun B 0100 ORi ALUout A op ZX 0110 LW ALUout A + SX 1000 SW ALUout A + SX 1011 BEQ If A = B then PC ALUout 0010 Memory R[rd] ALUout 0101 To instruction fetch R[rt] ALUout 0111 M MEM[ALUout] 1001 R[rt] M 1010 To instruction fetch MEM[ALUout] B 1100 To instruction fetch Write-back #34 Final Review Winter

35 MIPS Multi-cycle Datapath Performance Evaluation What is the average CPI? State diagram gives CPI for each instruction type Workload below gives frequency of each type Type CPI i for type Frequency CPI i x freqi i Arith/Logic 4 40% 1.6 Load 5 30% 1.5 Store 4 10% 0.4 branch 3 20% 0.6 Average CPI: 4.1 Better than CPI = 5 if all instructions took the same number of clock cycles (5). #35 Final Review Winter

36 Control Implementation Alternatives Control may be designed using one of several initial representations. The choice of sequence control, and how logic is represented, can then be determined independently; the control can then be implemented with one of several methods using a structured logic technique. Initial Representation Finite State Diagram Microprogram Sequencing Control Explicit Next State Microprogram counter Function + Dispatch ROMs Logic Representation Logic Equations Truth Tables Implementation PLA ROM Technique hardwired control microprogrammed control #36 Final Review Winter

37 Microprogrammed Control Finite state machine control for a full set of instructions is very complex, and may involve a very large number of states: Slight microoperation changes require new FSM controller. Microprogramming: Designing the control as a program that implements the machine instructions. A microprogam for a given machine instruction is a symbolic representation of the control involved in executing the instruction and is comprised of a sequence of microinstructions. Each microinstruction defines the set of datapath control signals that must asserted (active) in a given state or cycle. The format of the microinstructions is defined by a number of fields each responsible for asserting a set of control signals. Microarchitecture: Logical structure and functional capabilities of the hardware as seen by the microprogrammer. #37 Final Review Winter

38 Microprogrammed Control Unit Microprogram Storage ROM/PLA Outputs Control Signal Fields Multicycle Datapath 1 Adder Microinstruction Address Inputs State Reg Sequencing Control Field Types of branching Set state to 0 (fetch) Dispatch i (state 1) Use incremented address (seq) state number 2 Address Select Logic Microprogram Counter, MicroPC Opcode #38 Final Review Winter

39 Single Bit Control Multiple Bit Control List of control Signals Grouped Into Fields Signal name Effect when deasserted Effect when asserted ALUSelA 1st ALU operand = PC 1st ALU operand = Reg[rs] RegWrite None Reg. is written MemtoReg Reg. write data input = ALU Reg. write data input = memory RegDst Reg. dest. no. = rt Reg. dest. no. = rd MemRead None Memory at address is read, MDR Mem[addr] MemWrite None Memory at address is written IorD Memory address = PC Memory address = S IRWrite None IR Memory PCWrite None PC PCSource PCWriteCond None IF ALUzero then PC PCSource PCSource PCSource = ALU PCSource = ALUout Signal name Value Effect ALUOp 00 ALU adds 01 ALU subtracts 10 ALU does function code 11 ALU does logical OR ALUSelB 000 2nd ALU input = Reg[rt] 001 2nd ALU input = nd ALU input = sign extended IR[15-0] 011 2nd ALU input = sign extended, shift left 2 IR[15-0] 100 2nd ALU input = zero extended IR[15-0] #39 Final Review Winter

40 Microinstruction Field Values Field Name Values for Field Function of Field with Specific Value ALU Add ALU adds Subt. ALU subtracts Func code ALU does function code Or ALU does logical OR SRC1 PC 1st ALU input = PC rs 1st ALU input = Reg[rs] SRC2 4 2nd ALU input = 4 Extend 2nd ALU input = sign ext. IR[15-0] Extend0 2nd ALU input = zero ext. IR[15-0] Extshft 2nd ALU input = sign ex., sl IR[15-0] rt 2nd ALU input = Reg[rt] destination rd ALU Reg[rd] ALUout rt ALU Reg[rt] ALUout rt Mem Reg[rt] Mem Memory Read PC Read memory using PC Read ALU Read memory using ALU output Write ALU Write memory using ALU output, value B Memory register IR IR Mem PC write ALU PC ALU ALUoutCond IF ALU Zero then PC ALUout Sequencing Seq Go to sequential µinstruction Fetch Go to the first microinstruction Dispatch i Dispatch using ROM. #40 Final Review Winter

41 Microprogram for The Control Unit Label ALU SRC1 SRC2 Dest. Memory Mem. Reg. PC Write Sequencing Fetch: Add PC 4 Read PC IR ALU Seq Add PC Extshft Dispatch Lw: Add rs Extend Seq Read ALU Seq rt MEM Fetch Sw: Add rs Extend Seq Write ALU Fetch Rtype: Func rs rt Seq rd ALU Fetch Beq: Subt. rs rt ALUoutCond. Fetch Ori: Or rs Extend0 Seq rt ALU Fetch #41 Final Review Winter

42 MIPS Integer ALU Requirements (1) Functional Specification: inputs: 2 x 32-bit operands A, B, 4-bit mode outputs: 32-bit result S, 1-bit carry, 1 bit overflow, 1 bit zero operations: add, addu, sub, subu, and, or, xor, nor, slt, sltu (2) Block Diagram: c A zero ovf ALU S 32 B m 10 operations thus 4 control bits 4 00 add 01 addu 02 sub 03 subu 04 and 05 or 06 xor 07 nor 12 slt 13 sltu #42 Final Review Winter

43 Building Block: 1-bit ALU Performs: AND, OR, addition on A, B or A, B inverted A invertb CarryIn and Operation or Mux Result B 1-bit Full Adder add CarryOut #43 Final Review Winter

44 32-Bit ALU Using 32 1-Bit ALUs A0 B0 A31 B31 CarryIn0 CarryIn1 A1 B1 CarryIn2 A2 B2 CarryIn3 CarryIn31 1-bit ALU 1-bit ALU 1-bit ALU : : 1-bit ALU CarryOut0 CarryOut1 CarryOut30 Result0 Result1 Result2 Result31 32-bit rippled-carry adder (operation/invertb lines not shown) Addition/Subtraction Performance: Assume gate delay = T Total delay = 32 x (1-Bit ALU Delay) = 32 x 2 x gate delay = 64 x gate delay = 64 T C CarryOut31 #44 Final Review Winter

45 MIPS ALU With SLT Support Added CarryIn0 Less A0 B0 A1 B1 Less = 0 A2 B2 Less = 0 1-bit Result0 ALU CarryIn1 CarryOut0 1-bit Result1 ALU CarryIn2 CarryOut1 CarryIn3 CarryIn31 1-bit ALU A31 B31 1-bit Less = 0 ALU CarryOut31 C : : CarryOut30 Result2 : : : : Result31 Zero Overflow #45 Final Review Winter

46 Improving ALU Performance: Carry Look Ahead (CLA) A0 B1 A B A B A B G P G P G P G P Cin S S S S C1 =G0 + C0 P0 C2 = G1 + G0 P1 + C0 P0 P1 A B C-out kill 0 1 C-in propagate 1 0 C-in propagate generate G = A and B P = A xor B C3 = G2 + G1 P2 + G0 P1 P2 + C0 P0 P1 P2 G P #46 Final Review Winter

47 C L A 4-bit Adder 4-bit Adder 4-bit Adder Cascaded Carry Look-ahead C0 G0 P0 C1 =G0 + C0 P0 C2 = G1 + G0 P1 + C0 P0 P1 16-Bit Example Delay = = 5 gate delays = 5T { C3 = G2 + G1 P2 + G0 P1 P2 + C0 P0 P1 P2 G P Assuming all gates have equal delay T C4 =... #47 Final Review Winter

48 Unsigned Multiplication Example Paper and pencil example (unsigned): Multiplicand 1000 Multiplier Product m bits x n bits = m + n bit product, m = 32, n = 32, 64 bit product. The binary number system simplifies multiplication: 0 => place 0 ( 0 x multiplicand). 1 => place a copy ( 1 x multiplicand). We will examine 4 versions of multiplication hardware & algorithm: Successive refinement of design. #48 Final Review Winter

49 Operation of Combinational Multiplier A 3 A 2 A 1 A 0 B 0 A 3 A 2 A 1 A 0 B 1 A 3 A 2 A 1 A 0 B 2 A 3 A 2 A 1 A 0 B 3 P 7 P 6 P 5 P 4 P 3 P 2 P 1 P 0 At each stage shift A left ( x 2). Use next bit of B to determine whether to add in shifted multiplicand. Accumulate 2n bit partial product at each stage. #49 Final Review Winter

50 MULTIPLY HARDWARE Version 3 Combine Multiplier register and Product register: 32-bit Multiplicand register. 32 -bit ALU. 64-bit Product register, (0-bit Multiplier register). Multiplicand 32 bits 32-bit ALU Product (Multiplier) 64 bits Shift Right Write Control #50 Final Review Winter

51 Multiply Algorithm Version 3 Product0 = 1 Start 1. Test Product0 Product0 = 0 1a. Add multiplicand to the left half of product & place the result in the left half of Product register 2. Shift the Product register right 1 bit. 32nd repetition? No: < 32 repetitions Yes: 32 repetitions Done #51 Final Review Winter

52 Combinational Shifter from MUXes Basic Building Block sel A B bit right shifter D A 7 A 6 A 5 A 4 A 3 A 2 A 1 A 0 S 2 S 1 S R 7 R 6 R 5 R 4 R 3 R 2 R 1 R 0 What comes in the MSBs? How many levels for 32-bit shifter? #52 Final Review Winter

53 Division 1001 Quotient Divisor Dividend Remainder (or Modulo result) See how big a number can be subtracted, creating quotient bit on each step: Binary => 1 * divisor or 0 * divisor Dividend = Quotient x Divisor + Remainder => Dividend = Quotient + Divisor 3 versions of divide, successive refinement #53 Final Review Winter

54 DIVIDE HARDWARE Version 3 32-bit Divisor register. 32 -bit ALU. 64-bit Remainder regegister (0-bit Quotient register). Divisor 32 bits 32-bit ALU HI LO Remainder (Quotient) 64 bits Shift Left Write Control #54 Final Review Winter

55 Divide Algorithm Version 3 Start: Place Dividend in Remainder 1. Shift the Remainder register left 1 bit. 2. Subtract the Divisor register from the left half of the Remainder register, & place the result in the left half of the Remainder register. Remainder >= 0 Test Remainder Remainder < 0 3a. Shift the Remainder register to the left setting the new rightmost bit to 1. 3b. Restore the original value by adding the Divisor register to the left half of the Remainderregister, &place the sum in the left half of the Remainder register. Also shift the Remainder register to the left, setting the new least significant bit to 0. nth repetition? No: < n repetitions Yes: n repetitions (n = 4 here) Done. Shift left half of Remainder right 1 bit. #55 Final Review Winter

56 Representation of Floating Point Numbers in Single Precision IEEE 754 Standard Value = N = (-1) S X 2 E-127 X (1.M) 0 < E < 255 Actual exponent is: e = E sign S E M exponent: excess 127 binary integer added mantissa: sign + magnitude, normalized binary significand with a hidden integer bit: 1.M Example: 0 = = Magnitude of numbers that can be represented is in the range: (1.0) to ( ) Which is approximately: 1.8 x to 3.40 x #56 Final Review Winter

57 Representation of Floating Point Numbers in Single Precision IEEE 754 Standard Value = N = (-1) S X 2 E-127 X (1.M) 0 < E < 255 Actual exponent is: e = E sign S E M exponent: excess 127 binary integer added mantissa: sign + magnitude, normalized binary significand with a hidden integer bit: 1.M Example: 0 = = Magnitude of numbers that can be represented is in the range: (1.0) to ( ) Which is approximately: 1.8 x to 3.40 x #57 Final Review Winter

58 Representation of Floating Point Numbers in Double Precision IEEE 754 Standard Value = N = (-1) S X 2 E-1023 X (1.M) 0 < E < 2047 Actual exponent is: e = E sign S E M exponent: excess 1023 binary integer added Mantissa: sign + magnitude, normalized binary significand with a hidden integer bit: 1.M Example: 0 = = Magnitude of numbers that can be represented is in the range: (1.0) to ( ) Which is approximately: 2.23 x to 1.8 x #58 Final Review Winter

59 Floating Point Conversion Example The decimal number is to be represented in the IEEE bit single precision format: = (converted to binary) Hidden = x 2 11 (normalized binary) The mantissa is negative so the sign S is given by: S = 1 The biased exponent E is given by E = e E = = = Fractional part of mantissa M: M = (in 23 bits) The IEEE 754 single precision representation is given by: S E M 1 bit 8 bits 23 bits #59 Final Review Winter

60 (1) Start Compare the exponents of the two numbers shift the smaller number to the right until its exponent matches the larger exponent Floating Point Addition Flowchart (2) Add the significands (mantissas) (3) Normalize the sum, either shifting right and incrementing the exponent or shifting left and decrementing the exponent No Still normalized? Done yes (4) Overflow or Underflow? Generate exception or return error Round the significand to the appropriate number of bits If mantissa = 0, set exponent to 0 (5) No Yes #60 Final Review Winter

61 Floating Point Addition Hardware #61 Final Review Winter

62 Floating Point Multiplication Flowchart (1) Start Is one/both operands =0? Set the result to zero: exponent = 0 (2) (3) (4) (5) Compute exponent: biased exp.(x) + biased exp.(y) - bias Compute sign of result: Xs XOR Ys Multiply the mantissas Normalize mantissa if needed Generate exception or return error Yes Overflow or Underflow? (6) No Still Normalized? Yes No Round or truncate the result mantissa Done (7) #62 Final Review Winter

63 Extra Bits for Rounding Extra bits used to prevent or minimize rounding errors. How many extra bits? IEEE: As if computed the result exactly and rounded. Addition: 1.xxxxx 1.xxxxx 1.xxxxx + 1.xxxxx 0.001xxxxx 0.01xxxxx 1x.xxxxy 1.xxxxxyyy 1x.xxxxyyy post-normalization pre-normalization pre and post Guard Digits: digits to the right of the first p digits of significand to guard against loss of digits can later be shifted left into first P places during normalization. Addition: carry-out shifted in Subtraction: borrow digit and guard Multiplication: carry and guard, Division requires guard #63 Final Review Winter

64 Rounding Digits Normalized result, but some non-zero digits to the right of the significand --> the number should be rounded E.g., B = 10, p = 3: = * 10 2-bias 2-bias = * 10 2-bias = * 10 One round digit must be carried to the right of the guard digit so that after a normalizing left shift, the result can be rounded, according to the value of the round digit. IEEE Standard: four rounding modes: round to nearest (default) round towards plus infinity round towards minus infinity round towards 0 round to nearest: round digit < B/2 then truncate > B/2 then round up (add 1 to ULP: unit in last place) = B/2 then round to nearest even digit it can be shown that this strategy minimizes the mean error introduced by rounding. #64 Final Review Winter

65 Sticky Bit Additional bit to the right of the round digit to better fine tune rounding d0. d1 d2 d3... dp X... X X X S X X S d0. d1 d2 d3... dp X... X X X 0 Rounding Summary: Sticky bit: set to 1 if any 1 bits fall off the end of the round digit d0. d1 d2 d3... dp X... X X X 1 generates a borrow Radix 2 minimizes wobble in precision. Normal operations in +,-,*,/ require one carry/borrow bit + one guard digit. One round digit needed for correct rounding. Sticky bit needed when round digit is B/2 for max accuracy. Rounding to nearest has mean error = 0 if uniform distribution of digits are assumed. #65 Final Review Winter

66 Pipelining: Design Goals The length of the machine clock cycle is determined by the time required for the slowest pipeline stage. An important pipeline design consideration is to balance the length of each pipeline stage. If all stages are perfectly balanced, then the time per instruction on a pipelined machine (assuming ideal conditions with no stalls): Time per instruction on unpipelined machine Number of pipe stages Under these ideal conditions: Speedup from pipelining = the number of pipeline stages = k One instruction is completed every cycle: CPI = 1. #66 Final Review Winter

67 Pipelined Instruction Processing Representation Clock cycle Number Time in clock cycles Instruction Number Instruction I IF ID EX MEM WB Instruction I+1 IF ID EX MEM WB Instruction I+2 IF ID EX MEM WB Instruction I+3 IF ID EX MEM WB Instruction I +4 IF ID EX MEM WB Time to fill the pipeline Pipeline Stages: IF = Instruction Fetch ID = Instruction Decode EX = Execution MEM = Memory Access WB = Write Back First instruction, I Completed Last instruction, I+4 completed #67 Final Review Winter

68 Single Cycle, Multi-Cycle, Pipeline: Performance Comparison Example For 1000 instructions, execution time: Single Cycle Machine: 8 ns/cycle x 1 CPI x 1000 inst = 8000 ns Multicycle Machine: 2 ns/cycle x 4.6 CPI (due to inst mix) x 1000 inst = 9200 ns Ideal pipelined machine, 5-stages: 2 ns/cycle x (1 CPI x 1000 inst + 4 cycle fill) = 2008 ns #68 Final Review Winter

69 Clk Single Cycle, Multi-Cycle, Vs. Pipeline Single Cycle Implementation: Cycle 1 Cycle 2 8 ns Load Store Waste 2ns Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10 Clk Multiple Cycle Implementation: Load IF ID EX MEM WB Store IF ID EX MEM R-type IF Pipeline Implementation: Load IF ID EX MEM WB Store IF ID EX MEM WB R-type IF ID EX MEM WB #69 Final Review Winter

70 MIPS: A Pipelined Datapath 0 M u x 1 IF/ID ID/EX EX/MEM MEM/WB Add 4 Shift left 2 Add result Add PC Address Instruction memory Instruction Read register 1 Read data 1 Read register 2 Registers Read data 2 Write register Write data 0 M ux 1 Zero ALU ALU result Address Write data Data memory Read data 1 M ux 0 16 Sign extend 32 IF ID EX MEM WB Instruction Fetch Instruction Decode Execution Memory Write Back #70 Final Review Winter

71 Pipeline Control Pass needed control signals along from one stage to the next as the instruction travels through the pipeline just like the data Execution/Address Calculation stage control lines Memory access stage control lines Write-back stage control lines Instruction Reg Dst ALU Op1 ALU Op0 ALU Src Branch Mem Read Mem Write Reg write Mem to Reg R-format lw sw X X beq X X WB Instruction Control M WB EX M WB IF/ID ID/EX EX/MEM MEM/WB #71 Final Review Winter

72 Basic Performance Issues In Pipelining Pipelining increases the CPU instruction throughput: The number of instructions completed per unit time. Under ideal condition instruction throughput is one instruction per machine cycle, or CPI = 1 Pipelining does not reduce the execution time of an individual instruction: The time needed to complete all processing steps of an instruction (also called instruction completion latency). It usually slightly increases the execution time of each instruction over unpipelined implementations due to the increased control overhead of the pipeline and pipeline stage registers delays. #72 Final Review Winter

73 Pipelining Performance Example Example: For an unpipelined machine: Clock cycle = 10ns, 4 cycles for ALU operations and branches and 5 cycles for memory operations with instruction frequencies of 40%, 20% and 40%, respectively. If pipelining adds 1ns to the machine clock cycle then the speedup in instruction execution from pipelining is: Non-pipelined Average instruction execution time = Clock cycle x Average CPI = 10 ns x ((40% + 20%) x %x 5) = 10 ns x 4.4 = 44 ns In the pipelined five implementation five stages are used with an average instruction execution time of: 10 ns + 1 ns = 11 ns Speedup from pipelining = Instruction time unpipelined Instruction time pipelined = 44 ns / 11 ns = 4 times #73 Final Review Winter

74 Pipeline Hazards Hazards are situations in pipelining which prevent the next instruction in the instruction stream from executing during the designated clock cycle resulting in one or more stall cycles. Hazards reduce the ideal speedup gained from pipelining and are classified into three classes: Structural hazards: Arise from hardware resource conflicts when the available hardware cannot support all possible combinations of instructions. Data hazards: Arise when an instruction depends on the results of a previous instruction in a way that is exposed by the overlapping of instructions in the pipeline. Control hazards: Arise from the pipelining of conditional branches and other instructions that change the PC. #74 Final Review Winter

75 Structural hazard Example: Single Memory For Instructions & Data Time (clock cycles) I n s t r. O r d e r Load Instr 1 Instr 2 Instr 3 Instr 4 ALU Mem Reg Mem Reg Mem Reg Mem Reg ALU Mem ALU Mem Reg Mem Reg ALU Reg Mem Reg ALU Mem Reg Mem Reg Detection is easy in this case (right half highlight means read, left half write) #75 Final Review Winter

76 Data Hazards Example Problem with starting next instruction before first is finished Data dependencies here that go backward in time create data hazards. sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2) Time (in clock cycles) Value of register $2: Program execution order (in instructions) sub $2, $1, $3 CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 IM Reg CC 7 CC 8 CC / DM Reg and $12, $2, $5 IM Reg DM Reg or $13, $6, $2 IM Reg DM Reg add $14, $2, $2 IM Reg DM Reg sw $15, 100($2) IM Reg DM Reg #76 Final Review Winter

77 Data Hazard Resolution: Stall Cycles Stall the pipeline by a number of cycles. The control unit must detect the need to insert stall cycles. In this case two stall cycles are needed. Time (in clock cycles) Value of register $2: Program execution order (in instructions) sub $2, $1, $3 CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 IM Reg CC 7 CC / DM Reg CC 9 20 CC CC and $12, $2, $5 IM STALL STALL Reg DM Reg or $13, $6, $2 STALL STALL IM Reg DM Reg add $14, $2, $2 IM Reg DM Reg sw $15, 100($2) IM Reg DM Reg #77 Final Review Winter

78 Data Hazard Resolution: Stall Cycles Stall the pipeline by a number of cycles. The control unit must detect the need to insert stall cycles. In this case two stall cycles are needed. Time (in clock cycles) Value of register $2: Program execution order (in instructions) sub $2, $1, $3 CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 IM Reg CC 7 CC / DM Reg CC 9 20 CC CC and $12, $2, $5 IM STALL STALL Reg DM Reg or $13, $6, $2 STALL STALL IM Reg DM Reg add $14, $2, $2 IM Reg DM Reg sw $15, 100($2) IM Reg DM Reg #78 Final Review Winter

79 Performance of Pipelines with Stalls Hazards in pipelines may make it necessary to stall the pipeline by one or more cycles and thus degrading performance from the ideal CPI of 1. CPI pipelined = Ideal CPI + Pipeline stall clock cycles per instruction If pipelining overhead is ignored and we assume that the stages are perfectly balanced then: Speedup = CPI unpipelined / (1 + Pipeline stall cycles per instruction) When all instructions take the same number of cycles and is equal to the number of pipeline stages then: Speedup = Pipeline depth / (1 + Pipeline stall cycles per instruction) #79 Final Review Winter

80 Data Hazard Resolution: Forwarding Register file forwarding to handle read/write to same register ALU forwarding #80 Final Review Winter

81 Data Hazard Example With Forwarding Value of register $2 : Value of EX/MEM : Value of MEM/WB : Time (in clock cycles) CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 CC 7 CC 8 CC / X X X 20 X X X X X X X X X 20 X X X X Program execution order (in instructions) sub $2, $1, $3 IM Reg DM Reg and $12, $2, $5 IM Reg DM Reg or $13, $6, $2 IM Reg DM Reg add $14, $2, $2 IM Reg DM Reg sw $15, 100($2) IM Reg DM Reg #81 Final Review Winter

82 A Data Hazard Requiring A Stall A load followed by an R-type instruction that uses the loaded value Program execution order (in instructions) lw $2, 20($1) Time (in clock cycles) CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 IM Reg DM Reg CC 7 CC 8 CC 9 and $4, $2, $5 IM Reg DM Reg or $8, $2, $6 IM Reg DM Reg add $9, $4, $2 IM Reg DM Reg slt $1, $6, $7 IM Reg DM Reg Even with forwarding in place a stall cycle is needed This condition must be detected by hardware #82 Final Review Winter

83 Compiler Scheduling Example Reorder the instructions to avoid as many pipeline stalls as possible: lw $15, 0($2) lw $16, 4($2) sw $16, 0($2) sw $15, 4($2) The data hazard occurs on register $16 between the second lw and the first sw resulting in a stall cycle With forwarding we need to find only one independent instructions to place between them, swapping the lw instructions works: lw $15, 0($2) lw $16, 4($2) sw $15, 0($2) sw $16, 4($2) Without forwarding we need three independent instructions to place between them, so in addition two nops are added. lw $15, 0($2) lw $16, 4($2) nop nop sw $15, 0($2) sw $16, 4($2) #83 Final Review Winter

84 Control Hazards: Example Three other instructions are in the pipeline before branch instruction target decision is made when BEQ is in MEM stage. Program execution order (in instructions) Time (in clock cycles) CC 1 CC 2 CC 3 CC 4 CC 5 CC 6 CC 7 CC 8 CC 9 40 beq $1, $3, 7 IM Reg DM Reg 44 and $12, $2, $5 IM Reg DM Reg 48 or $13, $6, $2 IM Reg DM Reg 52 add $14, $2, $2 IM Reg DM Reg 72 lw $4, 50($7) IM Reg DM Reg In the above diagram, we are predicting branch not taken Need to add hardware for flushing the three following instructions if we are wrong losing three cycles. #84 Final Review Winter

85 Reducing Delay of Taken Branches Next PC of a branch known in MEM stage: Costs three lost cycles if taken. If next PC is known in EX stage, one cycle is saved. Branch address calculation can be moved to ID stage using a register comparator, costing only one cycle if branch is taken. IF.Flush Hazard detection unit M u x ID/EX WB EX/MEM Control 0 M u x M WB MEM/WB IF/ID EX M WB PC 4 Instruction memory Shift left 2 Registers = M u x M u x ALU Data memory M u x Sign extend M u x Forwarding unit #85 Final Review Winter

86 Pipeline Performance Example Assume the following MIPS instruction mix: Type Frequency Arith/Logic 40% Load 30% of which 25% are followed immediately by an instruction using the loaded value Store 10% branch 20% of which 45% are taken What is the resulting CPI for the pipelined MIPS with forwarding and branch address calculation in ID stage? CPI = Ideal CPI + Pipeline stall clock cycles per instruction = 1 + stalls by loads + stalls by branches = 1 +.3x.25x1 +.2 x.45x1 = = #86 Final Review Winter

87 Memory Hierarchy: Motivation Processor-Memory (DRAM) Performance Gap 1000 Performance CPU Processor-Memory Performance Gap: (grows 50% / year) DRAM µproc 60%/yr. DRAM 7%/yr. #87 Final Review Winter

88 Processor-DRAM Performance Gap Impact: Example To illustrate the performance impact, assume a pipelined RISC CPU with CPI = 1 using non-ideal memory. Over an 10 year period, ignoring other factors, the cost of a full memory access in terms of number of wasted instructions: CPU CPU Memory Minimum CPU cycles or Year speed cycle Access instructions wasted MHZ ns ns 1986: /125 = : /30 = : /13.3 = : /5 = : /3.33 = 33 #88 Final Review Winter

89 Memory Hierarchy: Motivation The Principle Of Locality Programs usually access a relatively small portion of their address space (instructions/data) at any instant of time (program working set). Two Types of locality: Temporal Locality: If an item is referenced, it will tend to be referenced again soon. Spatial locality: If an item is referenced, items whose addresses are close will tend to be referenced soon. The presence of locality in program behavior, makes it possible to satisfy a large percentage of program access needs using memory levels with much less capacity than program address space. #89 Final Review Winter

90 Levels of The Memory Hierarchy Part of The On-chip CPU Datapath Registers One or more levels (Static RAM): Level 1: On-chip 16-64K Level 2: On or Off-chip K Level 3: Off-chip 128K-8M Dynamic RAM (DRAM) 16M-16G Interface: SCSI, RAID, IDE, G-100G Registers Cache Main Memory Magnetic Disc Farther away from The CPU Lower Cost/Bit Higher Capacity Increased Access Time/Latency Lower Throughput Optical Disk or Magnetic Tape #90 Final Review Winter

91 Cache Design & Operation Issues Q1: Where can a block be placed cache? (Block placement strategy & Cache organization) Fully Associative, Set Associative, Direct Mapped Q2: How is a block found if it is in cache? (Block identification) Tag/Block Q3: Which block should be replaced on a miss? (Block replacement) Random, LRU Q4: What happens on a write? (Cache write policy) Write through, write back #91 Final Review Winter

92 Cache Organization Example #92 Final Review Winter

93 Locating A Data Block in Cache Each block frame in cache has an address tag. The tags of every cache block that might contain the required data are checked or searched in parallel. A valid bit is added to the tag to indicate whether this entry contains a valid address. The address from the CPU to cache is divided into: A block address, further divided into: An index field to choose a block set in cache. (no index field when fully associative). A tag field to search and match addresses in the selected set. A block offset to select the data from the block. Tag Block Address Index Block Offset #93 Final Review Winter

94 Address Field Sizes Physical Address Generated by CPU Tag Block Address Index Block Offset Block offset size = log2(block size) Index size = log2(total number of blocks/associativity) Tag size = address size - index size - offset size #94 Final Review Winter

95 Four-Way Set Associative Cache: MIPS Implementation Example Tag Field Address Index Field Index V Tag Data V Tag Data V Tag Data V Tag Data sets 1024 block frames 4-to-1 m ultiplexo r Hit Da ta #95 Final Review Winter

96 Cache Organization/Addressing Example Given the following: A single-level L 1 cache with 128 cache block frames Each block frame contains four words (16 bytes) 16-bit memory addresses to be cached (64K bytes main memory or 4096 memory blocks) Show the cache organization/mapping and cache address fields for: Fully Associative cache. Direct mapped cache. 2-way set-associative cache. #96 Final Review Winter

97 Cache Example: Fully Associative Case V V All 128 tags must be checked in parallel by hardware to locate a data block V Valid bit Block Address = 12 bits Tag = 12 bits Block offset = 4 bits #97 Final Review Winter

98 Cache Example: Direct Mapped Case V V V Only a single tag must be checked in parallel to locate a data block V Valid bit Block Address = 12 bits Tag = 5 bits Index = 7 bits Block offset = 4 bits Main Memory #98 Final Review Winter

99 Cache Example: 2-Way Set-Associative Two tags in a set must be checked in parallel to locate a data block Valid bits not shown Block Address = 12 bits Tag = 6 bits Index = 6 bits Block offset = 4 bits Main Memory #99 Final Review Winter

100 Cache Replacement Policy When a cache miss occurs the cache controller may have to select a block of cache data to be removed from a cache block frame and replaced with the requested data, such a block is selected by one of two methods: Random: Any block is randomly selected for replacement providing uniform allocation. Simple to build in hardware. The most widely used cache replacement strategy. Least-recently used (LRU): Accesses to blocks are recorded and and the block replaced is the one that was not used for the longest period of time. LRU is expensive to implement, as the number of blocks to be tracked increases, and is usually approximated. #100 Final Review Winter

101 Miss Rates for Caches with Different Size, Associativity & Replacement Algorithm Sample Data Associativity: 2-way 4-way 8-way Size LRU Random LRU Random LRU Random 16 KB 5.18% 5.69% 4.67% 5.29% 4.39% 4.96% 64 KB 1.88% 2.01% 1.54% 1.66% 1.39% 1.53% 256 KB 1.15% 1.17% 1.13% 1.13% 1.12% 1.12% #101 Final Review Winter

102 Single Level Cache Performance For a CPU with a single level (L1) of cache and no stalls for cache hits: With ideal memory CPU time = (CPU execution clock cycles + Memory stall clock cycles) x clock cycle time Memory stall clock cycles = (Reads x Read miss rate x Read miss penalty) + (Writes x Write miss rate x Write miss penalty) If write and read miss penalties are the same: Memory stall clock cycles = Memory accesses x Miss rate x Miss penalty #102 Final Review Winter

103 Single Level Cache Performance CPUtime = IC x CPI x C CPI execution = CPI with ideal memory CPI = CPI execution + Mem Stall cycles per instruction CPUtime = IC x (CPI execution + Mem Stall cycles per instruction) x C Mem Stall cycles per instruction = Mem accesses per instruction x Miss rate x Miss penalty CPUtime = IC x (CPI execution + Mem accesses per instruction x Miss rate x Miss penalty ) x C Misses per instruction = Memory accesses per instruction x Miss rate CPUtime = IC x (CPI execution + Misses per instruction x Miss penalty) x C #103 Final Review Winter

104 Cache Performance Example Suppose a CPU executes at Clock Rate = 200 MHz (5 ns per cycle) with a single level of cache. CPI execution = 1.1 Instruction mix: 50% arith/logic, 30% load/store, 20% control Assume a cache miss rate of 1.5% and a miss penalty of 50 cycles. CPI = CPI execution + mem stalls per instruction Mem Stalls per instruction = Mem accesses per instruction x Miss rate x Miss penalty Mem accesses per instruction = = 1.3 Instruction fetch Load/store Mem Stalls per instruction = 1.3 x.015 x 50 = CPI = = The ideal CPU with no misses is 2.075/1.1 = 1.88 times faster #104 Final Review Winter

105 Cache Performance Example Suppose for the previous example we double the clock rate to 400 MHZ, how much faster is this machine, assuming similar miss rate, instruction mix? Since memory speed is not changed, the miss penalty takes more CPU cycles: Miss penalty = 50 x 2 = 100 cycles. CPI = x.015 x 100 = = 3.05 Speedup = (CPI old x C old )/ (CPI new x C new ) = x 2 / 3.05 = 1.36 The new machine is only 1.36 times faster rather than 2 times faster due to the increased effect of cache misses. CPUs with higher clock rate, have more cycles per cache miss and more memory impact on CPI #105 Final Review Winter