Chapter 4. Advanced Pipelining and Instruction-Level Parallelism. In-Cheol Park Dept. of EE, KAIST
|
|
- Philip Harmon
- 5 years ago
- Views:
Transcription
1 Chapter 4. Advanced Pipelining and Instruction-Level Parallelism In-Cheol Park Dept. of EE, KAIST
2 Instruction-level parallelism Loop unrolling Dependence Data/ name / control dependence Loop level parallelism Dynamic scheduling Scoreboarding / Tomasulo approach Hardware branch prediction Branch prediction buffer / Branch target buffer Multiple issue Compiler support for ILP Software pipelining / Trace schduling Hardware support for parallelism Studies of ILP Real example, PowerPC 620 2
3 ILP : parallelism among instruction sequences How to utilize parallelism Pipeline Multiple issue processor How much parallelism # of instructions in a basic block Branch frequency is about 15~20% Exploit ILP across multiple basic blocks Loop-level parallelism Loop unrolling either statically or dynamically Vector instruction vector processor 3
4 Dynamic scheduling Scoreboarding / Tomasulo approach Hardware Branch Prediction Branch prediction buffer/ Branch target buffer Multiple Issue Superscalar / VLIW Compiler Support for ILP Software Pipelining / Trace Scheduling Hardware Support for Parallelism Conditional instruction Poison bit / Boosting Tomasulo + Reorder buffer 4
5 for (i=1 ; i<=1000 ; i++) x[i] = x[i] +s; Loop : LD F0, 0(R1) ; F0 = array element ADDD F4, F0, F2 ; add scalar in F2 SD 0(R1), F4 ; store result SUBI R1, R1, 8 ; decrement pointer ; 8 Byte (per DW) BNEZ R1, Loop ; Branch R1!= zero clock cycle issued Loop : LD F0, 0(R1) 1 Stall 2 ADDD F4, F0, F2 3 Stall 4 Stall 5 SD 0(R1), F4 6 SUBI R1, R1, #8 7 BNEZ R1, Loop 8 Stall 9 scheduling Loop : LD F0, 0(R1) Stall ADDD F4, F0, F2 SUBI R1, R1, #8 BNEZ R1, Loop ; delayed branch SD 8(R1), F4 ; altered & interchanged with SUBI 5
6 Unrolling 4 iterations Loop : LD F0, 0(R1) ADDD F4, F0, F2 SD 0(R1), F4 ; drop SUBI & BNEZ LD F6, -8(R1) ADDD F8, F6, F2 SD -8(R1), F8 ; drop SUBI & BNEZ LD F10, -16(R1) ADDD F12, F10, F2 SD -16(R1), F12 ; drop SUBI & BNEZ LD F14, -24(R1) ADDD F16, F14, F2 SD -24(R1), F16 ; drop SUBI & BNEZ SUBI R1, R1, #32 BNEZ R1, Loop scheduling Loop : LD F0, 0(R1) LD F6, -8(R1) LD F10, -16(R1) LD F14, -24(R1) ADDD F4, F0, F2 ADDD F8, F6, F2 ADDD F12, F10, F2 ADDD F16, F14, F2 SD 0(R1), F4 SD -8(R1), F8 SD -16(R1), F12 SUBI R1, R1, #32 BNEZ R1, Loop SD 8(R1), F16 ; 8-32 = -24 6
7 To reduce loop overhead Eliminate branches Improve scheduling Allow instructions from different iterations to be scheduled Exposes more computations that can be scheduled to minimize stalls Different registers for each iteration Increases the required count of registers When the upper bound of the loop is not known k copies of the loop body (n mod k) + (n / k) iterations 7
8 Data dependence (RAW) Data dependent instructions can not execute simultaneously hazard stall is a property of the pipeline To avoid Maintain the dependence but avoid a harzard Data forwarding Scheduling = code rearrangement Eliminate the dependence by transforming the code Data dependence that flows through memory is difficult to detect. 8
9 Name dependence (WAR, WAW) There is no value being transmitted between the instructions Two types : anti-dependence (WAR) output dependence (WAW) Register renaming Either by compiler or in hardware Only true dependence remains Control dependence Every instruction except for the first basic block is control dependent on some set of branches An instruction dependent on a branch cannot be moved before or after the branch To avoid it, we can do delayed branch, speculation, and loop unrolling 9
10 Loop carried dependence Dependence exists between different iterations for ( i = 1 ; i <= 100; i++) x[i] = x[i] + s; Loop is parallel for ( i = 1 ; i <= 100; i++){ a[i] = a[i] + b[i]; b[i+1] = c[i] +d[i]; } a[1] = a[1] + b[1]; for ( i = 1 ; i <= 99; i++){ b[i+1] = c[i] +d[i] a[i+1] = a[i+1] + b[i+1]; } b[101] = c[100] + d[100]; b[i] : loop carried dependence 10
11 Static scheduling : compiler techniques to reduce hazards and stalls Dynamic scheduling : hardware rearranges instruction execution to reduce stalls Out-of-order execution, out-of-order completion Instructions begin execution as soon as their operands are available Imprecise execution ID stage is split into two stages Issue : decode instructions, check for structural hazards Read operands : wait until no data hazards, then read operands Two techniques Scoreboarding Tomasulo approach (Reservation Station) 11
12 12
13 Used in CDC6600 The goal is to execute an instruction as early as possible Instructions can be issued and executed if they do not dependent on any active or stalled instruction Performance improvement of 1.7 for FORTRAN programs and 2.5 for hand-coded assembly programs Scoreboard All hazard detection and resolution is centralized Constructs a record of data dependences Monitors every change in the hardware Determines when the instruction can read operands and begin execution Controls when the instruction can write its result into the destination 13
14 Issue Check for structural and WAW hazards If so, issue stalls Read operands Read source operands when available and begin execution Resolves RAW hazards dynamically No forwarding of data Execute Notify its completion to the scoreboard Write result Checks for WAR hazards, and stalls the instruction if necessary 14
15 15
16 16
17 Amount of parallelism available among the instructions Parallelism within a basic block Scoreboard size Instruction window : the set of instructions examined as candidates for potential execution The number and types of functional units The presence of anti-dependence and output dependence 17
18 Used by IBM360/91 floating point unit [1967] Scoreboarding + Register renaming Eliminates WAW and WAR hazards by register renaming Reservation station Located in front of each functional unit to hold an instruction issued Hazard detection and execution control are distributed Fetches and buffers an operand as soon as available The result is directly passed to reservation stations waiting for it Common data bus (CDB) 18
19 19
20 Issue Get an instruction from instruction queue Issue if there is an empty reservation station Sends the operands if they are in registers, otherwise a tag field denoting the reservation station that will produce the operand If no empty reservation station, stall until a station is free Execute If operands are not yet available, monitor CDB while waiting for the register to be computed When the operand becomes available, place it into reservation station When both operands are available, execute the operation Write result Write it into CDB, into registers, and any reservation stations waiting for the result 20
21 21
22 22
23 Dynamic prediction at run time The prediction will change if the branch changes its behavior while the program is running Predicts branch direction and branch target address Usually implemented based on the branch prediction buffers 23
24 A small memory indexed by low portion of the address of the branch instruction Aliasing between branch instructions having the same low portion Accessed with the instruction address during IF stage Contains previous branch history One bit history mis-predicts two times for a change Actual : T T T T T T N T T T Predict :? T T T T T T N T T N-bit saturating up-down counter 2-bit counter is enough for almost all applications 24
25 25
26 26
27 Integer programs have higher branch frequency but lower prediction accuracy than FP programs Buffer hit rate is not the limiting factor Increasing number of bits per predictor has little impact Correlating predictor = Two-level predictor The current branch prediction is heavily affected by the direction of the last branches. If ( d == 0 ) d=1; If ( d == 1 ) branch branch 27
28 If ( d == 0 ) d=1; If ( d == 1 ) BNEZ R1, L1 ; branch b1 (d!=0) ADDI R1, R0, #1 ; d==0, so d=1 L1: SUBI R3, R1, #1 BNEZ R3, L2 ; branch b2 (d!=1) L2: 28
29 Two-level Branch address + direction history of last branches (m,n) predictor m : last m branches are used Usually m > 10 The direction history is called a pattern n : n-bit counter Usually n = 2 Pattern-based predictor, which does not use branch address to index the predictor, produces quite good results gshare : widely used (branch address XOR history pattern) is used to index the predictor 29
30 30
31 31
32 BTB A cache that stores the predicted address for the next instruction after a branch Access during the IF stage, that is, before decoding the instruction Must know the instruction to be fetched is a branch Store only predict-taken branches Must know whether the fetched instruction is a taken branch For 2-bit counter predictor, use both a target buffer and a prediction buffer 32
33 33
34 34
35 Store one or more target instructions instead of target PC Branch folding if the only function of branch is to change PC Return stack The accuracy of predicting the target address for a return instruction is low Indirect jump : destination address varies at runtime Majority of the indirect jumps come from procedure returns Small stack for return addresses Caches most recent return addresses 35
36 36
37 Superscalar Issue a varying number of independent instructions per clock Satisfy some hardware constraints Simplest organization : one integer + one floating-point More ports in register file More difficult in a CISC whose instruction length is variable The effect of stalls or delay is more severe than scalar machines Statically scheduled by compiler Dynamically scheduled by hardware Issue two or more instructions to reservation stations Pipeline the issue stage so that it runs two or more times faster For load/store instructions, we do not want out-of-order execution Queue Cf. Decoupled architecture 37
38 VLIW (Very Long Instruction Word) Issue a fixed number of instructions formatted as one large instruction or as a fixed instruction packet Extremely difficult to determine in hardware whether multiple instructions are independent An efficient compiler is essential to make a long sequence of instructions that can be executed in parallel Limitations Large number of functional units; much larger than the number of instructions in a VLIW Large number of ports on the register file and memory Large memory bandwidth Instructions are not full Waste of instruction bits and functional units Binary code compatibility 38
39 Parallel Loop Recurrence : a variable is defined based on the value of the variable in an earlier iteration Dependence distance The larger the distance, the more potential parallelism Array Assume indices are affine : affine if index = a*i+b For two indices, ai+b and cj+d, These is no dependence if GCD(a,c) does not divide (d-b) Renaming Dependence analysis is challenging Pointers, indirect indexing, Lacks of runtime information: Possible dependence, but not encountered in runtime. 39
40 Reorganizing loops such that each iteration in the software-pipelined code is made from instructions chosen from different iterations of the original loop Interleaves instructions from different iterations without unrolling the loop Prolog / Start-up code Epilog / Clean-up code 40
41 41
42 42
43 43
44 Find parallelism across branches Run frequent paths faster Trace selection Likely sequence of basic blocks Trace compaction Global code scheduling Code motion speculative execution Often insert compensation code (or book-keeping code) to off-trace code to ensure correctness One example of compensation code : inverse operation 44
45 45
46 Conditional instruction If condition is true, the instruction is executed normally, otherwise the execution continues as if a NOP. Conditional move instructions are employed in recent processors 46
47 Compiler speculation Ignore executions for speculative instructions Simply return an undefined value for any exception that would cause termination Still need the renaming in software Poison bits A poison bit is added to every register, and a bit is added to every instruction to indicate whether the instruction is speculative Renaming in software Boosting Providing the renaming and buffering in hardware, much as Tomasulo s algolithm Record (boosted + predicted branch direction) The result of boosted instructions are forwarded to and used by other boosted instructions 47
48 48
49 49
50 50
51 Hardware based Speculation Dynamic branch prediction + speculation + dynamic scheduling Advantages Disambiguate memory reference Hardware branch prediction is superior Completely precise exception Does not require compensation Good performance for different implementation of an architecture binary compatibility Tomasulo s algolithm + Reorder buffer 51
52 52
53 Issue (dispatch) Issue if there is an empty reservations station and an empty slot in reorder buffer Execute Monitor CDB while waiting for the register to be computed, and then execute Write Result Write the result on CDB and into reorder buffer Commit Update the register with the result when the instruction reaches the head of the reorder buffer If mis-predicted branch reached the head of the reorder buffer, the reorder buffer is flushed and execution is started at the correct successor of the branch. 53
54 54
55 55
56 Ideal / Perfect Processor Perfect register renaming Perfect branch prediction Perfect jump instruction Perfect memory-address alias analysis Unlimited issue : Enough functional units to allow the ready instructions to issue Look arbitrarily far ahead to find instructions to issue One cycle execution 56
57 57
58 Limitations on the window size 2K window + issue of 64 instructions Realistic branch and jump prediction + aggressive predictor with 8K entries + 2K jump and return predictors Finite renaming registers renaming registers Imperfect alias analysis 58
59 59
60 60
61 2K window + issue of 64 instructions 61
62 62
63 2K window + issue of 64 instructions + aggressive predictor with 8K entries + 2K jump and return predictors 63
64 64
65 2K window + issue of 64 instructions + aggressive predictor with 8K entries + 2K jump and return predictors renaming registers 65
66 66
67 64 instruction issues A selective predictor with 1K entries and 16 entry return predictor Perfect disambiguation of memory references done dynamically Register renaming with additional 64 integer registers and 64 FP registers No cache misses + unit latencies 67
68 68
69 69
70 64 bit advanced superscalar processor Can fetch, issue, complete up to 4 instructions per cycle Speculative execution past 4 unresolved branches Register renaming 8 extra integer registers, 8 extra FP registers Reservation stations with 16-entry reorder buffer Execution unit Six execution unit 3 integer units with 2 reservation stations each Two simple integer units, XSU0, XSU1 One complex integer function unit, XCFXU 1 branch unit, BPU, with 2 reservation stations 1 Load/Store unit, LSU with 3 reservation stations 1 FP unit, FPU 70
71 8 entry instruction queue Static / Dynamic Branch prediction Branch prediction in fetch and dispatch stages 256-entry 2-way branch target address cache 2048-entry branch history table Caches 32KB 8-way set associative non-blocking data cache 32KB 8-way set associative instruction cache BUS interface 40-bit address bus and 128 bit data bus Split transaction Pipelined snoop bus protocol MESI 71
72 72
73 73
74 74
75 Branch misprediction 256 entry BTB, 2 way set associative 2K entry branch prediction buffer Return stack Instruction cache miss Not serious because of a perfect off-chip cache Partial cache line fill 75
76 76
77 No reservation station available No rename registers Reorder buffer is full The same functional unit Misc. Shortage of read ports Special registers serialization 77
78 78
79 79
80 80
81 81
82 Source operand unavailable ILP is insufficient fewer buffers Functional unit unavailable Increase the number of FUs Increase pipelining in the un-pipelined units Out-of-order disallowed Serialization Overall 1.limitation of FU LSU 2.losses in fetch, issue, and execution 3.ILP limitation and finite buffering 82
83 83
Hardware-based Speculation
Hardware-based Speculation Hardware-based Speculation To exploit instruction-level parallelism, maintaining control dependences becomes an increasing burden. For a processor executing multiple instructions
More informationInstruction-Level Parallelism and Its Exploitation
Chapter 2 Instruction-Level Parallelism and Its Exploitation 1 Overview Instruction level parallelism Dynamic Scheduling Techniques es Scoreboarding Tomasulo s s Algorithm Reducing Branch Cost with Dynamic
More informationHardware-based speculation (2.6) Multiple-issue plus static scheduling = VLIW (2.7) Multiple-issue, dynamic scheduling, and speculation (2.
Instruction-Level Parallelism and its Exploitation: PART 2 Hardware-based speculation (2.6) Multiple-issue plus static scheduling = VLIW (2.7) Multiple-issue, dynamic scheduling, and speculation (2.8)
More informationEITF20: Computer Architecture Part3.2.1: Pipeline - 3
EITF20: Computer Architecture Part3.2.1: Pipeline - 3 Liang Liu liang.liu@eit.lth.se 1 Outline Reiteration Dynamic scheduling - Tomasulo Superscalar, VLIW Speculation ILP limitations What we have done
More informationCase Study IBM PowerPC 620
Case Study IBM PowerPC 620 year shipped: 1995 allowing out-of-order execution (dynamic scheduling) and in-order commit (hardware speculation). using a reorder buffer to track when instruction can commit,
More informationEECC551 Exam Review 4 questions out of 6 questions
EECC551 Exam Review 4 questions out of 6 questions (Must answer first 2 questions and 2 from remaining 4) Instruction Dependencies and graphs In-order Floating Point/Multicycle Pipelining (quiz 2) Improving
More informationILP concepts (2.1) Basic compiler techniques (2.2) Reducing branch costs with prediction (2.3) Dynamic scheduling (2.4 and 2.5)
Instruction-Level Parallelism and its Exploitation: PART 1 ILP concepts (2.1) Basic compiler techniques (2.2) Reducing branch costs with prediction (2.3) Dynamic scheduling (2.4 and 2.5) Project and Case
More informationHardware-Based Speculation
Hardware-Based Speculation Execute instructions along predicted execution paths but only commit the results if prediction was correct Instruction commit: allowing an instruction to update the register
More informationPage # CISC 662 Graduate Computer Architecture. Lecture 8 - ILP 1. Pipeline CPI. Pipeline CPI (I) Michela Taufer
CISC 662 Graduate Computer Architecture Lecture 8 - ILP 1 Michela Taufer http://www.cis.udel.edu/~taufer/teaching/cis662f07 Powerpoint Lecture Notes from John Hennessy and David Patterson s: Computer Architecture,
More informationComputer Architecture A Quantitative Approach, Fifth Edition. Chapter 3. Instruction-Level Parallelism and Its Exploitation
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 3 Instruction-Level Parallelism and Its Exploitation Introduction Pipelining become universal technique in 1985 Overlaps execution of
More informationCSE 820 Graduate Computer Architecture. week 6 Instruction Level Parallelism. Review from Last Time #1
CSE 820 Graduate Computer Architecture week 6 Instruction Level Parallelism Based on slides by David Patterson Review from Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level
More informationCPE 631 Lecture 11: Instruction Level Parallelism and Its Dynamic Exploitation
Lecture 11: Instruction Level Parallelism and Its Dynamic Exploitation Aleksandar Milenkovic, milenka@ece.uah.edu Electrical and Computer Engineering University of Alabama in Huntsville Outline Instruction
More information5008: Computer Architecture
5008: Computer Architecture Chapter 2 Instruction-Level Parallelism and Its Exploitation CA Lecture05 - ILP (cwliu@twins.ee.nctu.edu.tw) 05-1 Review from Last Lecture Instruction Level Parallelism Leverage
More informationCopyright 2012, Elsevier Inc. All rights reserved.
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 3 Instruction-Level Parallelism and Its Exploitation 1 Branch Prediction Basic 2-bit predictor: For each branch: Predict taken or not
More informationPage 1. CISC 662 Graduate Computer Architecture. Lecture 8 - ILP 1. Pipeline CPI. Pipeline CPI (I) Pipeline CPI (II) Michela Taufer
CISC 662 Graduate Computer Architecture Lecture 8 - ILP 1 Michela Taufer Pipeline CPI http://www.cis.udel.edu/~taufer/teaching/cis662f07 Powerpoint Lecture Notes from John Hennessy and David Patterson
More informationChapter 3 Instruction-Level Parallelism and its Exploitation (Part 1)
Chapter 3 Instruction-Level Parallelism and its Exploitation (Part 1) ILP vs. Parallel Computers Dynamic Scheduling (Section 3.4, 3.5) Dynamic Branch Prediction (Section 3.3) Hardware Speculation and Precise
More informationCS425 Computer Systems Architecture
CS425 Computer Systems Architecture Fall 2018 Static Instruction Scheduling 1 Techniques to reduce stalls CPI = Ideal CPI + Structural stalls per instruction + RAW stalls per instruction + WAR stalls per
More informationCPE 631 Lecture 10: Instruction Level Parallelism and Its Dynamic Exploitation
Lecture 10: Instruction Level Parallelism and Its Dynamic Exploitation Aleksandar Milenković, milenka@ece.uah.edu Electrical and Computer Engineering University of Alabama in Huntsville Outline Tomasulo
More informationPage 1. Recall from Pipelining Review. Lecture 16: Instruction Level Parallelism and Dynamic Execution #1: Ideas to Reduce Stalls
CS252 Graduate Computer Architecture Recall from Pipelining Review Lecture 16: Instruction Level Parallelism and Dynamic Execution #1: March 16, 2001 Prof. David A. Patterson Computer Science 252 Spring
More informationInstruction Level Parallelism
Instruction Level Parallelism The potential overlap among instruction execution is called Instruction Level Parallelism (ILP) since instructions can be executed in parallel. There are mainly two approaches
More informationExploitation of instruction level parallelism
Exploitation of instruction level parallelism Computer Architecture J. Daniel García Sánchez (coordinator) David Expósito Singh Francisco Javier García Blas ARCOS Group Computer Science and Engineering
More informationNOW Handout Page 1. Review from Last Time #1. CSE 820 Graduate Computer Architecture. Lec 8 Instruction Level Parallelism. Outline
CSE 820 Graduate Computer Architecture Lec 8 Instruction Level Parallelism Based on slides by David Patterson Review Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level Parallelism
More informationRecall from Pipelining Review. Lecture 16: Instruction Level Parallelism and Dynamic Execution #1: Ideas to Reduce Stalls
CS252 Graduate Computer Architecture Recall from Pipelining Review Lecture 16: Instruction Level Parallelism and Dynamic Execution #1: March 16, 2001 Prof. David A. Patterson Computer Science 252 Spring
More informationCISC 662 Graduate Computer Architecture Lecture 13 - CPI < 1
CISC 662 Graduate Computer Architecture Lecture 13 - CPI < 1 Michela Taufer http://www.cis.udel.edu/~taufer/teaching/cis662f07 Powerpoint Lecture Notes from John Hennessy and David Patterson s: Computer
More informationEE 4683/5683: COMPUTER ARCHITECTURE
EE 4683/5683: COMPUTER ARCHITECTURE Lecture 4A: Instruction Level Parallelism - Static Scheduling Avinash Kodi, kodi@ohio.edu Agenda 2 Dependences RAW, WAR, WAW Static Scheduling Loop-carried Dependence
More informationSuper Scalar. Kalyan Basu March 21,
Super Scalar Kalyan Basu basu@cse.uta.edu March 21, 2007 1 Super scalar Pipelines A pipeline that can complete more than 1 instruction per cycle is called a super scalar pipeline. We know how to build
More informationCMSC411 Fall 2013 Midterm 2 Solutions
CMSC411 Fall 2013 Midterm 2 Solutions 1. (12 pts) Memory hierarchy a. (6 pts) Suppose we have a virtual memory of size 64 GB, or 2 36 bytes, where pages are 16 KB (2 14 bytes) each, and the machine has
More informationCS433 Midterm. Prof Josep Torrellas. October 19, Time: 1 hour + 15 minutes
CS433 Midterm Prof Josep Torrellas October 19, 2017 Time: 1 hour + 15 minutes Name: Instructions: 1. This is a closed-book, closed-notes examination. 2. The Exam has 4 Questions. Please budget your time.
More informationStatic vs. Dynamic Scheduling
Static vs. Dynamic Scheduling Dynamic Scheduling Fast Requires complex hardware More power consumption May result in a slower clock Static Scheduling Done in S/W (compiler) Maybe not as fast Simpler processor
More informationComputer Science 246 Computer Architecture
Computer Architecture Spring 2009 Harvard University Instructor: Prof. dbrooks@eecs.harvard.edu Compiler ILP Static ILP Overview Have discussed methods to extract ILP from hardware Why can t some of these
More informationLecture 8 Dynamic Branch Prediction, Superscalar and VLIW. Computer Architectures S
Lecture 8 Dynamic Branch Prediction, Superscalar and VLIW Computer Architectures 521480S Dynamic Branch Prediction Performance = ƒ(accuracy, cost of misprediction) Branch History Table (BHT) is simplest
More informationPage # Let the Compiler Do it Pros and Cons Pros. Exploiting ILP through Software Approaches. Cons. Perhaps a mixture of the two?
Exploiting ILP through Software Approaches Venkatesh Akella EEC 270 Winter 2005 Based on Slides from Prof. Al. Davis @ cs.utah.edu Let the Compiler Do it Pros and Cons Pros No window size limitation, the
More informationDynamic Scheduling. Better than static scheduling Scoreboarding: Tomasulo algorithm:
LECTURE - 13 Dynamic Scheduling Better than static scheduling Scoreboarding: Used by the CDC 6600 Useful only within basic block WAW and WAR stalls Tomasulo algorithm: Used in IBM 360/91 for the FP unit
More informationPreventing Stalls: 1
Preventing Stalls: 1 2 PipeLine Pipeline efficiency Pipeline CPI = Ideal pipeline CPI + Structural Stalls + Data Hazard Stalls + Control Stalls Ideal pipeline CPI: best possible (1 as n ) Structural hazards:
More informationILP: Instruction Level Parallelism
ILP: Instruction Level Parallelism Tassadaq Hussain Riphah International University Barcelona Supercomputing Center Universitat Politècnica de Catalunya Introduction Introduction Pipelining become universal
More informationExploiting ILP with SW Approaches. Aleksandar Milenković, Electrical and Computer Engineering University of Alabama in Huntsville
Lecture : Exploiting ILP with SW Approaches Aleksandar Milenković, milenka@ece.uah.edu Electrical and Computer Engineering University of Alabama in Huntsville Outline Basic Pipeline Scheduling and Loop
More informationCISC 662 Graduate Computer Architecture. Lecture 10 - ILP 3
CISC 662 Graduate Computer Architecture Lecture 10 - ILP 3 Michela Taufer http://www.cis.udel.edu/~taufer/teaching/cis662f07 Powerpoint Lecture Notes from John Hennessy and David Patterson s: Computer
More informationUG4 Honours project selection: Talk to Vijay or Boris if interested in computer architecture projects
Announcements UG4 Honours project selection: Talk to Vijay or Boris if interested in computer architecture projects Inf3 Computer Architecture - 2017-2018 1 Last time: Tomasulo s Algorithm Inf3 Computer
More informationMetodologie di Progettazione Hardware-Software
Metodologie di Progettazione Hardware-Software Advanced Pipelining and Instruction-Level Paralelism Metodologie di Progettazione Hardware/Software LS Ing. Informatica 1 ILP Instruction-level Parallelism
More informationAdvanced Pipelining and Instruction- Level Parallelism 4
4 Advanced Pipelining and Instruction- Level Parallelism 4 Who s first? America. Who s second? Sir, there is no second. Dialog between two observers of the sailing race later named The America s Cup and
More informationProcessor: Superscalars Dynamic Scheduling
Processor: Superscalars Dynamic Scheduling Z. Jerry Shi Assistant Professor of Computer Science and Engineering University of Connecticut * Slides adapted from Blumrich&Gschwind/ELE475 03, Peh/ELE475 (Princeton),
More informationUNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering. Computer Architecture ECE 568
UNIVERSITY OF MASSACHUSETTS Dept. of Electrical & Computer Engineering Computer Architecture ECE 568 Part 10 Compiler Techniques / VLIW Israel Koren ECE568/Koren Part.10.1 FP Loop Example Add a scalar
More informationComputer Architectures. Chapter 4. Tien-Fu Chen. National Chung Cheng Univ.
Computer Architectures Chapter 4 Tien-Fu Chen National Chung Cheng Univ. chap4-0 Advance Pipelining! Static Scheduling Have compiler to minimize the effect of structural, data, and control dependence "
More informationChapter 4 The Processor 1. Chapter 4D. The Processor
Chapter 4 The Processor 1 Chapter 4D The Processor Chapter 4 The Processor 2 Instruction-Level Parallelism (ILP) Pipelining: executing multiple instructions in parallel To increase ILP Deeper pipeline
More informationPipelining and Exploiting Instruction-Level Parallelism (ILP)
Pipelining and Exploiting Instruction-Level Parallelism (ILP) Pipelining and Instruction-Level Parallelism (ILP). Definition of basic instruction block Increasing Instruction-Level Parallelism (ILP) &
More informationAdapted from David Patterson s slides on graduate computer architecture
Mei Yang Adapted from David Patterson s slides on graduate computer architecture Introduction Basic Compiler Techniques for Exposing ILP Advanced Branch Prediction Dynamic Scheduling Hardware-Based Speculation
More informationFour Steps of Speculative Tomasulo cycle 0
HW support for More ILP Hardware Speculative Execution Speculation: allow an instruction to issue that is dependent on branch, without any consequences (including exceptions) if branch is predicted incorrectly
More informationAdvanced Computer Architecture CMSC 611 Homework 3. Due in class Oct 17 th, 2012
Advanced Computer Architecture CMSC 611 Homework 3 Due in class Oct 17 th, 2012 (Show your work to receive partial credit) 1) For the following code snippet list the data dependencies and rewrite the code
More informationMultiple Instruction Issue and Hardware Based Speculation
Multiple Instruction Issue and Hardware Based Speculation Soner Önder Michigan Technological University, Houghton MI www.cs.mtu.edu/~soner Hardware Based Speculation Exploiting more ILP requires that we
More informationDYNAMIC AND SPECULATIVE INSTRUCTION SCHEDULING
DYNAMIC AND SPECULATIVE INSTRUCTION SCHEDULING Slides by: Pedro Tomás Additional reading: Computer Architecture: A Quantitative Approach, 5th edition, Chapter 3, John L. Hennessy and David A. Patterson,
More informationEECC551 Review. Dynamic Hardware-Based Speculation
EECC551 Review Recent Trends in Computer Design. Computer Performance Measures. Instruction Pipelining. Branch Prediction. Instruction-Level Parallelism (ILP). Loop-Level Parallelism (LLP). Dynamic Pipeline
More informationChapter 3: Instruction Level Parallelism (ILP) and its exploitation. Types of dependences
Chapter 3: Instruction Level Parallelism (ILP) and its exploitation Pipeline CPI = Ideal pipeline CPI + stalls due to hazards invisible to programmer (unlike process level parallelism) ILP: overlap execution
More informationCISC 662 Graduate Computer Architecture Lecture 11 - Hardware Speculation Branch Predictions
CISC 662 Graduate Computer Architecture Lecture 11 - Hardware Speculation Branch Predictions Michela Taufer http://www.cis.udel.edu/~taufer/teaching/cis6627 Powerpoint Lecture Notes from John Hennessy
More informationReduction of Data Hazards Stalls with Dynamic Scheduling So far we have dealt with data hazards in instruction pipelines by:
Reduction of Data Hazards Stalls with Dynamic Scheduling So far we have dealt with data hazards in instruction pipelines by: Result forwarding (register bypassing) to reduce or eliminate stalls needed
More informationLoad1 no Load2 no Add1 Y Sub Reg[F2] Reg[F6] Add2 Y Add Reg[F2] Add1 Add3 no Mult1 Y Mul Reg[F2] Reg[F4] Mult2 Y Div Reg[F6] Mult1
Instruction Issue Execute Write result L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F2, F6 DIV.D F10, F0, F6 ADD.D F6, F8, F2 Name Busy Op Vj Vk Qj Qk A Load1 no Load2 no Add1 Y Sub Reg[F2]
More informationMulti-cycle Instructions in the Pipeline (Floating Point)
Lecture 6 Multi-cycle Instructions in the Pipeline (Floating Point) Introduction to instruction level parallelism Recap: Support of multi-cycle instructions in a pipeline (App A.5) Recap: Superpipelining
More informationMulticycle ALU Operations 2/28/2011. Diversified Pipelines The Path Toward Superscalar Processors. Limitations of Our Simple 5 stage Pipeline
//11 Limitations of Our Simple stage Pipeline Diversified Pipelines The Path Toward Superscalar Processors HPCA, Spring 11 Assumes single cycle EX stage for all instructions This is not feasible for Complex
More informationHandout 2 ILP: Part B
Handout 2 ILP: Part B Review from Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level Parallelism Loop unrolling by compiler to increase ILP Branch prediction to increase ILP
More informationCPE 631 Lecture 10: Instruction Level Parallelism and Its Dynamic Exploitation
Lecture 10: Instruction Level Parallelism and Its Dynamic Exploitation Aleksandar Milenkovic, milenka@ece.uah.edu Electrical and Computer Engineering University of Alabama in Huntsville Outline Instruction
More informationPage 1. Recall from Pipelining Review. Lecture 15: Instruction Level Parallelism and Dynamic Execution
CS252 Graduate Computer Architecture Recall from Pipelining Review Lecture 15: Instruction Level Parallelism and Dynamic Execution March 11, 2002 Prof. David E. Culler Computer Science 252 Spring 2002
More informationWhat is ILP? Instruction Level Parallelism. Where do we find ILP? How do we expose ILP?
What is ILP? Instruction Level Parallelism or Declaration of Independence The characteristic of a program that certain instructions are, and can potentially be. Any mechanism that creates, identifies,
More informationCS252 Graduate Computer Architecture Midterm 1 Solutions
CS252 Graduate Computer Architecture Midterm 1 Solutions Part A: Branch Prediction (22 Points) Consider a fetch pipeline based on the UltraSparc-III processor (as seen in Lecture 5). In this part, we evaluate
More informationFloating Point/Multicycle Pipelining in DLX
Floating Point/Multicycle Pipelining in DLX Completion of DLX EX stage floating point arithmetic operations in one or two cycles is impractical since it requires: A much longer CPU clock cycle, and/or
More informationProcessor (IV) - advanced ILP. Hwansoo Han
Processor (IV) - advanced ILP Hwansoo Han Instruction-Level Parallelism (ILP) Pipelining: executing multiple instructions in parallel To increase ILP Deeper pipeline Less work per stage shorter clock cycle
More informationLecture-13 (ROB and Multi-threading) CS422-Spring
Lecture-13 (ROB and Multi-threading) CS422-Spring 2018 Biswa@CSE-IITK Cycle 62 (Scoreboard) vs 57 in Tomasulo Instruction status: Read Exec Write Exec Write Instruction j k Issue Oper Comp Result Issue
More informationELEC 5200/6200 Computer Architecture and Design Fall 2016 Lecture 9: Instruction Level Parallelism
ELEC 5200/6200 Computer Architecture and Design Fall 2016 Lecture 9: Instruction Level Parallelism Ujjwal Guin, Assistant Professor Department of Electrical and Computer Engineering Auburn University,
More informationReorder Buffer Implementation (Pentium Pro) Reorder Buffer Implementation (Pentium Pro)
Reorder Buffer Implementation (Pentium Pro) Hardware data structures retirement register file (RRF) (~ IBM 360/91 physical registers) physical register file that is the same size as the architectural registers
More informationDynamic Control Hazard Avoidance
Dynamic Control Hazard Avoidance Consider Effects of Increasing the ILP Control dependencies rapidly become the limiting factor they tend to not get optimized by the compiler more instructions/sec ==>
More informationInstruction-Level Parallelism (ILP)
Instruction Level Parallelism Instruction-Level Parallelism (ILP): overlap the execution of instructions to improve performance 2 approaches to exploit ILP: 1. Rely on hardware to help discover and exploit
More informationCPI < 1? How? What if dynamic branch prediction is wrong? Multiple issue processors: Speculative Tomasulo Processor
1 CPI < 1? How? From Single-Issue to: AKS Scalar Processors Multiple issue processors: VLIW (Very Long Instruction Word) Superscalar processors No ISA Support Needed ISA Support Needed 2 What if dynamic
More informationHiroaki Kobayashi 12/21/2004
Hiroaki Kobayashi 12/21/2004 1 Loop Unrolling Static Branch Prediction Static Multiple Issue: The VLIW Approach Software Pipelining Global Code Scheduling Trace Scheduling Superblock Scheduling Conditional
More informationOutline Review: Basic Pipeline Scheduling and Loop Unrolling Multiple Issue: Superscalar, VLIW. CPE 631 Session 19 Exploiting ILP with SW Approaches
Session xploiting ILP with SW Approaches lectrical and Computer ngineering University of Alabama in Huntsville Outline Review: Basic Pipeline Scheduling and Loop Unrolling Multiple Issue: Superscalar,
More informationCPI IPC. 1 - One At Best 1 - One At best. Multiple issue processors: VLIW (Very Long Instruction Word) Speculative Tomasulo Processor
Single-Issue Processor (AKA Scalar Processor) CPI IPC 1 - One At Best 1 - One At best 1 From Single-Issue to: AKS Scalar Processors CPI < 1? How? Multiple issue processors: VLIW (Very Long Instruction
More informationLecture 10: Static ILP Basics. Topics: loop unrolling, static branch prediction, VLIW (Sections )
Lecture 10: Static ILP Basics Topics: loop unrolling, static branch prediction, VLIW (Sections 4.1 4.4) 1 Static vs Dynamic Scheduling Arguments against dynamic scheduling: requires complex structures
More informationReview: Compiler techniques for parallelism Loop unrolling Ÿ Multiple iterations of loop in software:
CS152 Computer Architecture and Engineering Lecture 17 Dynamic Scheduling: Tomasulo March 20, 2001 John Kubiatowicz (http.cs.berkeley.edu/~kubitron) lecture slides: http://www-inst.eecs.berkeley.edu/~cs152/
More informationCS433 Homework 2 (Chapter 3)
CS Homework 2 (Chapter ) Assigned on 9/19/2017 Due in class on 10/5/2017 Instructions: 1. Please write your name and NetID clearly on the first page. 2. Refer to the course fact sheet for policies on collaboration..
More informationEECC551 - Shaaban. 1 GHz? to???? GHz CPI > (?)
Evolution of Processor Performance So far we examined static & dynamic techniques to improve the performance of single-issue (scalar) pipelined CPU designs including: static & dynamic scheduling, static
More informationCS433 Homework 2 (Chapter 3)
CS433 Homework 2 (Chapter 3) Assigned on 9/19/2017 Due in class on 10/5/2017 Instructions: 1. Please write your name and NetID clearly on the first page. 2. Refer to the course fact sheet for policies
More informationLecture 9: More ILP. Today: limits of ILP, case studies, boosting ILP (Sections )
Lecture 9: More ILP Today: limits of ILP, case studies, boosting ILP (Sections 3.8-3.14) 1 ILP Limits The perfect processor: Infinite registers (no WAW or WAR hazards) Perfect branch direction and target
More informationHardware Speculation Support
Hardware Speculation Support Conditional instructions Most common form is conditional move BNEZ R1, L ;if MOV R2, R3 ;then CMOVZ R2,R3, R1 L: ;else Other variants conditional loads and stores nullification
More informationAdvanced d Instruction Level Parallelism. Computer Systems Laboratory Sungkyunkwan University
Advanced d Instruction ti Level Parallelism Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu ILP Instruction-Level Parallelism (ILP) Pipelining:
More informationInstruction Level Parallelism (ILP)
1 / 26 Instruction Level Parallelism (ILP) ILP: The simultaneous execution of multiple instructions from a program. While pipelining is a form of ILP, the general application of ILP goes much further into
More information計算機結構 Chapter 4 Exploiting Instruction-Level Parallelism with Software Approaches
4.1 Basic Compiler Techniques for Exposing ILP 計算機結構 Chapter 4 Exploiting Instruction-Level Parallelism with Software Approaches 吳俊興高雄大學資訊工程學系 To avoid a pipeline stall, a dependent instruction must be
More informationDynamic Scheduling. CSE471 Susan Eggers 1
Dynamic Scheduling Why go out of style? expensive hardware for the time (actually, still is, relatively) register files grew so less register pressure early RISCs had lower CPIs Why come back? higher chip
More informationHY425 Lecture 09: Software to exploit ILP
HY425 Lecture 09: Software to exploit ILP Dimitrios S. Nikolopoulos University of Crete and FORTH-ICS November 4, 2010 ILP techniques Hardware Dimitrios S. Nikolopoulos HY425 Lecture 09: Software to exploit
More informationHardware-based Speculation
Hardware-based Speculation M. Sonza Reorda Politecnico di Torino Dipartimento di Automatica e Informatica 1 Introduction Hardware-based speculation is a technique for reducing the effects of control dependences
More informationInstruction Level Parallelism (ILP)
Instruction Level Parallelism (ILP) Pipelining supports a limited sense of ILP e.g. overlapped instructions, out of order completion and issue, bypass logic, etc. Remember Pipeline CPI = Ideal Pipeline
More informationHY425 Lecture 09: Software to exploit ILP
HY425 Lecture 09: Software to exploit ILP Dimitrios S. Nikolopoulos University of Crete and FORTH-ICS November 4, 2010 Dimitrios S. Nikolopoulos HY425 Lecture 09: Software to exploit ILP 1 / 44 ILP techniques
More informationINSTITUTO SUPERIOR TÉCNICO. Architectures for Embedded Computing
UNIVERSIDADE TÉCNICA DE LISBOA INSTITUTO SUPERIOR TÉCNICO Departamento de Engenharia Informática Architectures for Embedded Computing MEIC-A, MEIC-T, MERC Lecture Slides Version 3.0 - English Lecture 09
More informationDonn Morrison Department of Computer Science. TDT4255 ILP and speculation
TDT4255 Lecture 9: ILP and speculation Donn Morrison Department of Computer Science 2 Outline Textbook: Computer Architecture: A Quantitative Approach, 4th ed Section 2.6: Speculation Section 2.7: Multiple
More informationCS252 Graduate Computer Architecture Lecture 6. Recall: Software Pipelining Example
CS252 Graduate Computer Architecture Lecture 6 Tomasulo, Implicit Register Renaming, Loop-Level Parallelism Extraction Explicit Register Renaming John Kubiatowicz Electrical Engineering and Computer Sciences
More informationWebsite for Students VTU NOTES QUESTION PAPERS NEWS RESULTS
Advanced Computer Architecture- 06CS81 Hardware Based Speculation Tomasulu algorithm and Reorder Buffer Tomasulu idea: 1. Have reservation stations where register renaming is possible 2. Results are directly
More informationCourse on Advanced Computer Architectures
Surname (Cognome) Name (Nome) POLIMI ID Number Signature (Firma) SOLUTION Politecnico di Milano, July 9, 2018 Course on Advanced Computer Architectures Prof. D. Sciuto, Prof. C. Silvano EX1 EX2 EX3 Q1
More informationLecture 6 MIPS R4000 and Instruction Level Parallelism. Computer Architectures S
Lecture 6 MIPS R4000 and Instruction Level Parallelism Computer Architectures 521480S Case Study: MIPS R4000 (200 MHz, 64-bit instructions, MIPS-3 instruction set) 8 Stage Pipeline: first half of fetching
More informationGetting CPI under 1: Outline
CMSC 411 Computer Systems Architecture Lecture 12 Instruction Level Parallelism 5 (Improving CPI) Getting CPI under 1: Outline More ILP VLIW branch target buffer return address predictor superscalar more
More informationInstruction Level Parallelism. Taken from
Instruction Level Parallelism Taken from http://www.cs.utsa.edu/~dj/cs3853/lecture5.ppt Outline ILP Compiler techniques to increase ILP Loop Unrolling Static Branch Prediction Dynamic Branch Prediction
More informationLecture 8: Instruction Fetch, ILP Limits. Today: advanced branch prediction, limits of ILP (Sections , )
Lecture 8: Instruction Fetch, ILP Limits Today: advanced branch prediction, limits of ILP (Sections 3.4-3.5, 3.8-3.14) 1 1-Bit Prediction For each branch, keep track of what happened last time and use
More informationEE382A Lecture 7: Dynamic Scheduling. Department of Electrical Engineering Stanford University
EE382A Lecture 7: Dynamic Scheduling Department of Electrical Engineering Stanford University http://eeclass.stanford.edu/ee382a Lecture 7-1 Announcements Project proposal due on Wed 10/14 2-3 pages submitted
More informationTopics. Digital Systems Architecture EECE EECE Predication, Prediction, and Speculation
Digital Systems Architecture EECE 343-01 EECE 292-02 Predication, Prediction, and Speculation Dr. William H. Robinson February 25, 2004 http://eecs.vanderbilt.edu/courses/eece343/ Topics Aha, now I see,
More informationEEC 581 Computer Architecture. Instruction Level Parallelism (3.6 Hardware-based Speculation and 3.7 Static Scheduling/VLIW)
1 EEC 581 Computer Architecture Instruction Level Parallelism (3.6 Hardware-based Speculation and 3.7 Static Scheduling/VLIW) Chansu Yu Electrical and Computer Engineering Cleveland State University Overview
More information