Prefetching. An introduction to and analysis of Hardware and Software based Prefetching. Jun Yi Lei Robert Michael Allen Jr.
|
|
|
- Morgan Skinner
- 6 years ago
- Views:
Transcription
1 Prefetching An introduction to and analysis of Hardware and Software based Prefetching Jun Yi Lei Robert Michael Allen Jr. 11/12/2010 EECC-551: Shaaban 1
2 Outline What is Prefetching Background Software vs Hardware Hardware Implementation Current and Future Implications Summary Questions 11/12/2010 EECC-551: Shaaban 2
3 What is Prefetching? Process of bringing data into a higher level of memory before it is actually needed for processing Algorithms are designed to predict future data requests Reduces miss rate AMD's Bulldozer, an x86 core processor 11/12/2010 EECC-551: Shaaban 3
4 Background (History) Intel's i486 in 1989 was the first chip to have an internal L1 cache. Prefetching however was not utilized until some years later (in the traditional sense with caches) Prefetching uses unused memory bandwith to perform operations. 11/12/2010 EECC-551: Shaaban 4
5 Background (Requirements to Prefetch) Present working set must not be replaced by future set Must disjoint from the data cache avoids increasing bandwidth and complexity of the cache Must be expandable Must keep the data until it is unnecessary 11/12/2010 EECC-551: Shaaban 5
6 Background (When to Prefetch) Data should arrive to its destination before its needed Prefetch on miss? Prefetch at regular intervals? Prefetch dynamically based on program history? 11/12/2010 EECC-551: Shaaban 6
7 Background (How to Prefetch) 11/12/2010 EECC-551: Shaaban 7
8 Background (Who Wants to Prefetch) Push-based prefetching (simultaneous thread requests for prefetching) Processor Side Memor Side 11/12/2010 EECC-551: Shaaban 8
9 Software vs Hardware (Software Definition) Software Prefetching Prefetching techniques performed by the compiler or by the programmer Usually can prefetch instructions Utilizes prefetch input queue (PIQ) in certain architectures Compiler assisted prefetching in loops Stanford University Intermediate Form (SUIF) 11/12/2010 EECC-551: Shaaban 9
10 Software vs Hardware (Hardware Definition) Hardware Prefetching Hardware detect access patterns and decide When to perform prefetching Usually can prefetch future data Prefetch into the prefetch buffer or free cache 11/12/2010 EECC-551: Shaaban 10
11 Hardware Implementation (Overview) Two of many algorithms Sequential Prefetching Dependence Based Prefetching for Linked Data Structures (LDS) 11/12/2010 EECC-551: Shaaban 11
12 Hardware Implementation (Sequential Prefetching) Utlizes principle of spatial locality to prefetch data Fetches number of cache blocks each time cache is accessed Benefits from spatial locality w/o problems associated with large cache blocks Simplest scheme is the One Block Look-ahead (OBL) approach Fetch block n+1 when block n is accessed 11/12/2010 EECC-551: Shaaban 12
13 Hardware Implementation (Sequential Prefetching) OBL Prefetch on Miss Fetches block n + 1 when block n is requested from memory Requires a cache miss to prefetch 11/12/2010 EECC-551: Shaaban 13
14 Hardware Implementation (Sequential Prefetching) OBL Tagged Prefetch Utilizes a tag bit to detect when a block is demand fetched or prefetched. Does not require cache miss to trigger prefetch Allows for history based performance evaluation Can keep track of number of block demand fetches vs block prefetches 11/12/2010 EECC-551: Shaaban 14
15 Hardware Implementation (Sequential Prefetching) OBL - Sequential Dynamic Prefetch Instead of fetching block n+1, fetch block n+1 to n+k K is highly dependent upon the degree of spatial locality Small K's insufficient performance improvement Large K's cache pollution and collision 11/12/2010 EECC-551: Shaaban 15
16 Hardware Implementation (Sequential Prefetching) Analysis When considering the access pattern shown in earlier slides... Prefetch-on-Miss: Results in a cache miss for every other block... only improves performance by 50% at most Tagged Prefetch or Sequential Prefetch with K: Results in only one cache miss 11/12/2010 EECC-551: Shaaban 16
17 Hardware Implementation (Dependence Based Prefetching for LDS) Background Growing number of important, nonnumeric applications employ LDSs Databases commonly use index trees and hash tables Programs also allow for dynamic allocation and de-allocation LDSs can be linear, binary trees, multi-way... 11/12/2010 EECC-551: Shaaban 17
18 Hardware Implementation (Dependence Based Prefetching for LDS) Concept Based on finding pointer loads and establishing dependence relationships Producer Loads vs Consumer Loads First load (producer) gets the address which the second load (consumer) uses to retrieve the actual value Prefetching would occur on the first load and get the address and the value that the second load will need. 11/12/2010 EECC-551: Shaaban 18
19 Hardware Implementation (Dependence Based Prefetching for LDS) Main Components Correlation Table (CT) Stores Dependce information about loads Potential Producer Window (PPW) Maintains the list of most recently loaded values and the corresponding load instruction Prefetch Request Queue (PQR) Buffers requests for prefetch Prefetch Buffer (PB) Temporarily holds prefetch blocks 11/12/2010 EECC-551: Shaaban 19
20 Hardware Implementation (Dependence Based Prefetching for LDS) Prefetching Process Requests enqueued in the PRQ are dequeued when they can be serviced by the memory system The PB (not shown) attempts to extract the block from L1 cache, issuing a request to L2 cache on a miss 11/12/2010 EECC-551: Shaaban 20
21 Hardware Implementation (Dependence Based Prefetching for LDS) Dependency Information Dependency information is stored in the CT prod.add, con.add, tmpl prod.add = producing load instruction's PC address con.add = consuming load instruction's PC address tmpl = condensed form of the consuming load Need to create producer-consumer relationship... maintain a list of loaded values and their corresponding load instructions... stored in the PPW addrval, prod.add addrval = the base address used by the consumer 11/12/2010 EECC-551: Shaaban 21
22 Hardware Implementation (Dependence Based Prefetching for LDS) Creating Correlations in CT Correlations are created at instruction commit time As a load commits, check to see if there's a potential producer in the PPW If a match exists, an entry in the CT is created The value loaded by the current load is entered along with the current PC into the PPW as a new addrval-prod.add pair 11/12/2010 EECC-551: Shaaban 22
23 Hardware Implementation (Dependence Based Prefetching for LDS) Creating Correlations in CT Piece of code that is responsible for traversing a LDS a) Source code (C code) b) Machine code equivalent c) List layout in memory 11/12/2010 EECC-551: Shaaban 23
24 Hardware Implementation (Dependence Based Prefetching for LDS) Creating Correlations in CT 11/12/2010 EECC-551: Shaaban 24
25 Hardware Implementation (Dependence Based Prefetching for LDS) Creating Correlations in CT The average speedup for a system based on Dependence Based Prefetching for a 1KB PB is 10% This significantly outperforms a basic system with an extra 32KB of data cache. Increasing the PB past a certain point (~32KB) shows diminishing returns. CT however is implemented as a multi-way set associative cache and not all pointer loads are actually address loads. Unnecessary prefetches maybe initiated in this scheme. 11/12/2010 EECC-551: Shaaban 25
26 Several types of prefetchers are included in Core:
27 Several types of prefetchers are included in Core: the instruction prefetcher pre loads instructions in the instruction L1 cache based on branching prediction results. Each of the two cores has one. the IP prefetcher scrutinizes historical reading in order to have an overall diagram and loads "foreseeable" data in L1 cache. Each core also has one. The DCU prefetcher detects multiple reading from a single cache line for a determined period of time and decides to load the following line in the L1 cache. One per core as well. the DPL prefetcher has a similar functioning to the DCU. The only difference is that it detects requests on two successive cache lines (N and N+1) and is triggered if the reading of the line N+2 moves from the central memory to cache L2. The cache L2 has two of them, which are shared dynamically between the two cores.
28 Performance Evaluation on a baseline system
29 Performance Evaluation Cont.
30 Citations 1. J-L. Baer and T-F. Chen. An effective on-chip preloading scheme to reduce data access penalty. In Proceedings of Supercomputing ' Amir Roth, Andreas Moshovos and Gurindar S. Sohi. Dependence Based Prefetching for Linked Data Structures. University of Wisconsin. 3. Thomas Lexander and Gershon Kedem, Distributed Prefetch-buffer / Cache Design for High Performance Memory Systems /12/2010 EECC-551: Shaaban 30
31 Summary What is Prefetching Background Software vs Hardware Hardware Implementation Current and Future Implications Summary Questions 11/12/2010 EECC-551: Shaaban 31
32 Questions Questions? 11/12/2010 EECC-551: Shaaban 32
Reducing Miss Penalty: Read Priority over Write on Miss. Improving Cache Performance. Non-blocking Caches to reduce stalls on misses
Improving Cache Performance 1. Reduce the miss rate, 2. Reduce the miss penalty, or 3. Reduce the time to hit in the. Reducing Miss Penalty: Read Priority over Write on Miss Write buffers may offer RAW
Memory Hierarchy Computing Systems & Performance MSc Informatics Eng. Memory Hierarchy (most slides are borrowed)
Computing Systems & Performance Memory Hierarchy MSc Informatics Eng. 2011/12 A.J.Proença Memory Hierarchy (most slides are borrowed) AJProença, Computer Systems & Performance, MEI, UMinho, 2011/12 1 2
Memory Hierarchy Computing Systems & Performance MSc Informatics Eng. Memory Hierarchy (most slides are borrowed)
Computing Systems & Performance Memory Hierarchy MSc Informatics Eng. 2012/13 A.J.Proença Memory Hierarchy (most slides are borrowed) AJProença, Computer Systems & Performance, MEI, UMinho, 2012/13 1 2
Assignment 2: Understanding Data Cache Prefetching
Assignment 2: Understanding Data Cache Prefetching Computer Architecture Due: Monday, March 27, 2017 at 4:00 PM This assignment represents the second practical component of the Computer Architecture module.
10/16/2017. Miss Rate: ABC. Classifying Misses: 3C Model (Hill) Reducing Conflict Misses: Victim Buffer. Overlapping Misses: Lockup Free Cache
Classifying Misses: 3C Model (Hill) Divide cache misses into three categories Compulsory (cold): never seen this address before Would miss even in infinite cache Capacity: miss caused because cache is
Analyzing Memory Access Patterns and Optimizing Through Spatial Memory Streaming. Ogün HEPER CmpE 511 Computer Architecture December 24th, 2009
Analyzing Memory Access Patterns and Optimizing Through Spatial Memory Streaming Ogün HEPER CmpE 511 Computer Architecture December 24th, 2009 Agenda Introduction Memory Hierarchy Design CPU Speed vs.
Computer Architecture Spring 2016
omputer Architecture Spring 2016 Lecture 09: Prefetching Shuai Wang Department of omputer Science and Technology Nanjing University Prefetching(1/3) Fetch block ahead of demand Target compulsory, capacity,
Execution-based Prediction Using Speculative Slices
Execution-based Prediction Using Speculative Slices Craig Zilles and Guri Sohi University of Wisconsin - Madison International Symposium on Computer Architecture July, 2001 The Problem Two major barriers
Advanced Caching Techniques (2) Department of Electrical Engineering Stanford University
Lecture 4: Advanced Caching Techniques (2) Department of Electrical Engineering Stanford University http://eeclass.stanford.edu/ee282 Lecture 4-1 Announcements HW1 is out (handout and online) Due on 10/15
EECS 470. Lecture 15. Prefetching. Fall 2018 Jon Beaumont. History Table. Correlating Prediction Table
Lecture 15 History Table Correlating Prediction Table Prefetching Latest A0 A0,A1 A3 11 Fall 2018 Jon Beaumont A1 http://www.eecs.umich.edu/courses/eecs470 Prefetch A3 Slides developed in part by Profs.
Topics to be covered. EEC 581 Computer Architecture. Virtual Memory. Memory Hierarchy Design (II)
EEC 581 Computer Architecture Memory Hierarchy Design (II) Department of Electrical Engineering and Computer Science Cleveland State University Topics to be covered Cache Penalty Reduction Techniques Victim
Memory Hierarchy Basics
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 2 Memory Hierarchy Design 1 Memory Hierarchy Basics Six basic cache optimizations: Larger block size Reduces compulsory misses Increases
Advanced cache optimizations. ECE 154B Dmitri Strukov
Advanced cache optimizations ECE 154B Dmitri Strukov Advanced Cache Optimization 1) Way prediction 2) Victim cache 3) Critical word first and early restart 4) Merging write buffer 5) Nonblocking cache
EITF20: Computer Architecture Part4.1.1: Cache - 2
EITF20: Computer Architecture Part4.1.1: Cache - 2 Liang Liu liang.liu@eit.lth.se 1 Outline Reiteration Cache performance optimization Bandwidth increase Reduce hit time Reduce miss penalty Reduce miss
Based Prefetching for Linked Data Structures
Dependence Based Prefetching for Linked Data Structures Amir Roth, Andreas Moshovos and Gurindar S. Sohi Computer Sciences Department University of Wisconsin, Madison 1210 W. Dayton St. Madison, WI 53706
Lecture: Large Caches, Virtual Memory. Topics: cache innovations (Sections 2.4, B.4, B.5)
Lecture: Large Caches, Virtual Memory Topics: cache innovations (Sections 2.4, B.4, B.5) 1 Techniques to Reduce Cache Misses Victim caches Better replacement policies pseudo-lru, NRU Prefetching, cache
CSE 820 Graduate Computer Architecture. week 6 Instruction Level Parallelism. Review from Last Time #1
CSE 820 Graduate Computer Architecture week 6 Instruction Level Parallelism Based on slides by David Patterson Review from Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level
Handout 2 ILP: Part B
Handout 2 ILP: Part B Review from Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level Parallelism Loop unrolling by compiler to increase ILP Branch prediction to increase ILP
Hardware-Based Speculation
Hardware-Based Speculation Execute instructions along predicted execution paths but only commit the results if prediction was correct Instruction commit: allowing an instruction to update the register
Storage Efficient Hardware Prefetching using Delta Correlating Prediction Tables
Storage Efficient Hardware Prefetching using Correlating Prediction Tables Marius Grannaes Magnus Jahre Lasse Natvig Norwegian University of Science and Technology HiPEAC European Network of Excellence
Way-associative cache
Advance Caching 1 Way-associative cache blocks sharing the same index are a set block/line address tag index offset block / cacheline valid tag data valid tag data =? =? hit? hit? 2 Speeding up Memory
Computer Systems Architecture
Computer Systems Architecture Lecture 12 Mahadevan Gomathisankaran March 4, 2010 03/04/2010 Lecture 12 CSCE 4610/5610 1 Discussion: Assignment 2 03/04/2010 Lecture 12 CSCE 4610/5610 2 Increasing Fetch
Energy Characterization of Hardware-Based Data Prefetching
Energy Characterization of Hardware-Based Data Prefetching Yao Guo, Saurabh Chheda, Israel Koren, C. Mani Krishna, and Csaba Andras Moritz Electrical and Computer Engineering, University of Massachusetts,
Computer Systems Architecture I. CSE 560M Lecture 17 Guest Lecturer: Shakir James
Computer Systems Architecture I CSE 560M Lecture 17 Guest Lecturer: Shakir James Plan for Today Announcements and Reminders Project demos in three weeks (Nov. 23 rd ) Questions Today s discussion: Improving
LRU. Pseudo LRU A B C D E F G H A B C D E F G H H H C. Copyright 2012, Elsevier Inc. All rights reserved.
LRU A list to keep track of the order of access to every block in the set. The least recently used block is replaced (if needed). How many bits we need for that? 27 Pseudo LRU A B C D E F G H A B C D E
A Survey of prefetching techniques
A Survey of prefetching techniques Nir Oren July 18, 2000 Abstract As the gap between processor and memory speeds increases, memory latencies have become a critical bottleneck for computer performance.
CS7810 Prefetching. Seth Pugsley
CS7810 Prefetching Seth Pugsley Predicting the Future Where have we seen prediction before? Does it always work? Prefetching is prediction Predict which cache line will be used next, and place it in the
Chapter 2: Memory Hierarchy Design Part 2
Chapter 2: Memory Hierarchy Design Part 2 Introduction (Section 2.1, Appendix B) Caches Review of basics (Section 2.1, Appendix B) Advanced methods (Section 2.3) Main Memory Virtual Memory Fundamental
Lecture: Large Caches, Virtual Memory. Topics: cache innovations (Sections 2.4, B.4, B.5)
Lecture: Large Caches, Virtual Memory Topics: cache innovations (Sections 2.4, B.4, B.5) 1 More Cache Basics caches are split as instruction and data; L2 and L3 are unified The /L2 hierarchy can be inclusive,
Cache Memory COE 403. Computer Architecture Prof. Muhamed Mudawar. Computer Engineering Department King Fahd University of Petroleum and Minerals
Cache Memory COE 403 Computer Architecture Prof. Muhamed Mudawar Computer Engineering Department King Fahd University of Petroleum and Minerals Presentation Outline The Need for Cache Memory The Basics
Lecture notes for CS Chapter 2, part 1 10/23/18
Chapter 2: Memory Hierarchy Design Part 2 Introduction (Section 2.1, Appendix B) Caches Review of basics (Section 2.1, Appendix B) Advanced methods (Section 2.3) Main Memory Virtual Memory Fundamental
NOW Handout Page 1. Review from Last Time #1. CSE 820 Graduate Computer Architecture. Lec 8 Instruction Level Parallelism. Outline
CSE 820 Graduate Computer Architecture Lec 8 Instruction Level Parallelism Based on slides by David Patterson Review Last Time #1 Leverage Implicit Parallelism for Performance: Instruction Level Parallelism
Introduction. Stream processor: high computation to bandwidth ratio To make legacy hardware more like stream processor: We study the bandwidth problem
Introduction Stream processor: high computation to bandwidth ratio To make legacy hardware more like stream processor: Increase computation power Make the best use of available bandwidth We study the bandwidth
Lecture 16: Checkpointed Processors. Department of Electrical Engineering Stanford University
Lecture 16: Checkpointed Processors Department of Electrical Engineering Stanford University http://eeclass.stanford.edu/ee382a Lecture 18-1 Announcements Reading for today: class notes Your main focus:
Advanced optimizations of cache performance ( 2.2)
Advanced optimizations of cache performance ( 2.2) 30 1. Small and Simple Caches to reduce hit time Critical timing path: address tag memory, then compare tags, then select set Lower associativity Direct-mapped
MEMORY HIERARCHY DESIGN. B649 Parallel Architectures and Programming
MEMORY HIERARCHY DESIGN B649 Parallel Architectures and Programming Basic Optimizations Average memory access time = Hit time + Miss rate Miss penalty Larger block size to reduce miss rate Larger caches
EXAM 1 SOLUTIONS. Midterm Exam. ECE 741 Advanced Computer Architecture, Spring Instructor: Onur Mutlu
Midterm Exam ECE 741 Advanced Computer Architecture, Spring 2009 Instructor: Onur Mutlu TAs: Michael Papamichael, Theodoros Strigkos, Evangelos Vlachos February 25, 2009 EXAM 1 SOLUTIONS Problem Points
Introduction to OpenMP. Lecture 10: Caches
Introduction to OpenMP Lecture 10: Caches Overview Why caches are needed How caches work Cache design and performance. The memory speed gap Moore s Law: processors speed doubles every 18 months. True for
Why memory hierarchy? Memory hierarchy. Memory hierarchy goals. CS2410: Computer Architecture. L1 cache design. Sangyeun Cho
Why memory hierarchy? L1 cache design Sangyeun Cho Computer Science Department Memory hierarchy Memory hierarchy goals Smaller Faster More expensive per byte CPU Regs L1 cache L2 cache SRAM SRAM To provide
CS 152 Computer Architecture and Engineering. Lecture 8 - Memory Hierarchy-III
CS 152 Computer Architecture and Engineering Lecture 8 - Memory Hierarchy-III Krste Asanovic Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~krste
Lec 11 How to improve cache performance
Lec 11 How to improve cache performance How to Improve Cache Performance? AMAT = HitTime + MissRate MissPenalty 1. Reduce the time to hit in the cache.--4 small and simple caches, avoiding address translation,
Lecture: Cache Hierarchies. Topics: cache innovations (Sections B.1-B.3, 2.1)
Lecture: Cache Hierarchies Topics: cache innovations (Sections B.1-B.3, 2.1) 1 Types of Cache Misses Compulsory misses: happens the first time a memory word is accessed the misses for an infinite cache
5008: Computer Architecture
5008: Computer Architecture Chapter 2 Instruction-Level Parallelism and Its Exploitation CA Lecture05 - ILP (cwliu@twins.ee.nctu.edu.tw) 05-1 Review from Last Lecture Instruction Level Parallelism Leverage
Superscalar Processors
Superscalar Processors Increasing pipeline length eventually leads to diminishing returns longer pipelines take longer to re-fill data and control hazards lead to increased overheads, removing any a performance
Chapter 14 - Processor Structure and Function
Chapter 14 - Processor Structure and Function Luis Tarrataca luis.tarrataca@gmail.com CEFET-RJ L. Tarrataca Chapter 14 - Processor Structure and Function 1 / 94 Table of Contents I 1 Processor Organization
EECS 470. Lecture 14 Advanced Caches. DEC Alpha. Fall Jon Beaumont
Lecture 14 Advanced Caches DEC Alpha Fall 2018 Instruction Cache BIU Jon Beaumont www.eecs.umich.edu/courses/eecs470/ Data Cache Slides developed in part by Profs. Austin, Brehob, Falsafi, Hill, Hoe, Lipasti,
Memory Hierarchy Basics. Ten Advanced Optimizations. Small and Simple
Memory Hierarchy Basics Six basic cache optimizations: Larger block size Reduces compulsory misses Increases capacity and conflict misses, increases miss penalty Larger total cache capacity to reduce miss
Wenisch Final Review. Fall 2007 Prof. Thomas Wenisch EECS 470. Slide 1
Final Review Fall 2007 Prof. Thomas Wenisch http://www.eecs.umich.edu/courses/eecs470 Slide 1 Announcements Wenisch 2007 Exam is Monday, 12/17 4 6 in this room I recommend bringing a scientific calculator
Chapter 9 Pipelining. Jin-Fu Li Department of Electrical Engineering National Central University Jungli, Taiwan
Chapter 9 Pipelining Jin-Fu Li Department of Electrical Engineering National Central University Jungli, Taiwan Outline Basic Concepts Data Hazards Instruction Hazards Advanced Reliable Systems (ARES) Lab.
EITF20: Computer Architecture Part4.1.1: Cache - 2
EITF20: Computer Architecture Part4.1.1: Cache - 2 Liang Liu liang.liu@eit.lth.se 1 Outline Reiteration Cache performance optimization Bandwidth increase Reduce hit time Reduce miss penalty Reduce miss
SUPERSCALAR AND VLIW PROCESSORS
Datorarkitektur I Fö 10-1 Datorarkitektur I Fö 10-2 What is a Superscalar Architecture? SUPERSCALAR AND VLIW PROCESSORS A superscalar architecture is one in which several instructions can be initiated
Chapter 2: Memory Hierarchy Design Part 2
Chapter 2: Memory Hierarchy Design Part 2 Introduction (Section 2.1, Appendix B) Caches Review of basics (Section 2.1, Appendix B) Advanced methods (Section 2.3) Main Memory Virtual Memory Fundamental
Speculative Parallelization in Decoupled Look-ahead
Speculative Parallelization in Decoupled Look-ahead Alok Garg, Raj Parihar, and Michael C. Huang Dept. of Electrical & Computer Engineering University of Rochester, Rochester, NY Motivation Single-thread
TDT Coarse-Grained Multithreading. Review on ILP. Multi-threaded execution. Contents. Fine-Grained Multithreading
Review on ILP TDT 4260 Chap 5 TLP & Hierarchy What is ILP? Let the compiler find the ILP Advantages? Disadvantages? Let the HW find the ILP Advantages? Disadvantages? Contents Multi-threading Chap 3.5
Memory Technology. Caches 1. Static RAM (SRAM) Dynamic RAM (DRAM) Magnetic disk. Ideal memory. 0.5ns 2.5ns, $2000 $5000 per GB
Memory Technology Caches 1 Static RAM (SRAM) 0.5ns 2.5ns, $2000 $5000 per GB Dynamic RAM (DRAM) 50ns 70ns, $20 $75 per GB Magnetic disk 5ms 20ms, $0.20 $2 per GB Ideal memory Average access time similar
Chapter 4. Advanced Pipelining and Instruction-Level Parallelism. In-Cheol Park Dept. of EE, KAIST
Chapter 4. Advanced Pipelining and Instruction-Level Parallelism In-Cheol Park Dept. of EE, KAIST Instruction-level parallelism Loop unrolling Dependence Data/ name / control dependence Loop level parallelism
Efficient Prefetching with Hybrid Schemes and Use of Program Feedback to Adjust Prefetcher Aggressiveness
Journal of Instruction-Level Parallelism 13 (11) 1-14 Submitted 3/1; published 1/11 Efficient Prefetching with Hybrid Schemes and Use of Program Feedback to Adjust Prefetcher Aggressiveness Santhosh Verma
Feedback Directed Prefetching: Improving the Performance and Bandwidth-Efficiency of Hardware Prefetchers
Feedback Directed Prefetching: Improving the Performance and Bandwidth-Efficiency of Hardware Prefetchers Microsoft ssri@microsoft.com Santhosh Srinath Onur Mutlu Hyesoon Kim Yale N. Patt Microsoft Research
Superscalar Processors
Superscalar Processors Superscalar Processor Multiple Independent Instruction Pipelines; each with multiple stages Instruction-Level Parallelism determine dependencies between nearby instructions o input
Memory Hierarchy. Slides contents from:
Memory Hierarchy Slides contents from: Hennessy & Patterson, 5ed Appendix B and Chapter 2 David Wentzlaff, ELE 475 Computer Architecture MJT, High Performance Computing, NPTEL Memory Performance Gap Memory
Cray XE6 Performance Workshop
Cray XE6 Performance Workshop Mark Bull David Henty EPCC, University of Edinburgh Overview Why caches are needed How caches work Cache design and performance. 2 1 The memory speed gap Moore s Law: processors
The Memory Hierarchy. Cache, Main Memory, and Virtual Memory (Part 2)
The Memory Hierarchy Cache, Main Memory, and Virtual Memory (Part 2) Lecture for CPSC 5155 Edward Bosworth, Ph.D. Computer Science Department Columbus State University Cache Line Replacement The cache
Memory Hierarchy. Reading. Sections 5.1, 5.2, 5.3, 5.4, 5.8 (some elements), 5.9 (2) Lecture notes from MKP, H. H. Lee and S.
Memory Hierarchy Lecture notes from MKP, H. H. Lee and S. Yalamanchili Sections 5.1, 5.2, 5.3, 5.4, 5.8 (some elements), 5.9 Reading (2) 1 SRAM: Value is stored on a pair of inerting gates Very fast but
arxiv: v1 [cs.ar] 20 Jan 2018
![arxiv: v1 [cs.ar] 20 Jan 2018](/thumbs/78/76969016.jpg "arxiv: v1 [cs.ar] 20 Jan 2018")
Pointer-Chase Prefetcher for Linked Data Structures arxiv:1801.08088v1 [cs.ar] 20 Jan 2018 ABSTRACT Nitish Kumar Srivastava Cornell University nks45@cornell.edu Caches only exploit spatial and temporal
LIMITS OF ILP. B649 Parallel Architectures and Programming
LIMITS OF ILP B649 Parallel Architectures and Programming A Perfect Processor Register renaming infinite number of registers hence, avoids all WAW and WAR hazards Branch prediction perfect prediction Jump
CS3350B Computer Architecture
CS335B Computer Architecture Winter 25 Lecture 32: Exploiting Memory Hierarchy: How? Marc Moreno Maza wwwcsduwoca/courses/cs335b [Adapted from lectures on Computer Organization and Design, Patterson &
Like scalar processor Processes individual data items Item may be single integer or floating point number. - 1 of 15 - Superscalar Architectures
Superscalar Architectures Have looked at examined basic architecture concepts Starting with simple machines Introduced concepts underlying RISC machines From characteristics of RISC instructions Found
Memory Systems and Compiler Support for MPSoC Architectures. Mahmut Kandemir and Nikil Dutt. Cap. 9
Memory Systems and Compiler Support for MPSoC Architectures Mahmut Kandemir and Nikil Dutt Cap. 9 Fernando Moraes 28/maio/2013 1 MPSoC - Vantagens MPSoC architecture has several advantages over a conventional
Instruction Level Parallelism (Branch Prediction)
Instruction Level Parallelism (Branch Prediction) Branch Types Type Direction at fetch time Number of possible next fetch addresses? When is next fetch address resolved? Conditional Unknown 2 Execution
Lecture 2: Caches. belongs to Milo Martin, Amir Roth, David Wood, James Smith, Mikko Lipasti
Lecture 2: Caches http://wwwcssfuca/~ashriram/cs885/ belongs to Milo Martin, Amir Roth, David Wood, James Smith, Mikko Lipasti 1 Why focus on caches and memory? CPU can only compute as fast as memory Add
Computer Architecture A Quantitative Approach, Fifth Edition. Chapter 3. Instruction-Level Parallelism and Its Exploitation
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 3 Instruction-Level Parallelism and Its Exploitation Introduction Pipelining become universal technique in 1985 Overlaps execution of
Portland State University ECE 587/687. Memory Ordering
Portland State University ECE 587/687 Memory Ordering Copyright by Alaa Alameldeen and Haitham Akkary 2012 Handling Memory Operations Review pipeline for out of order, superscalar processors To maximize
Lecture-18 (Cache Optimizations) CS422-Spring
Lecture-18 (Cache Optimizations) CS422-Spring 2018 Biswa@CSE-IITK Compiler Optimizations Loop interchange Merging Loop fusion Blocking Refer H&P: You need it for PA3 and PA4 too. CS422: Spring 2018 Biswabandan
Chapter-5 Memory Hierarchy Design
Chapter-5 Memory Hierarchy Design Unlimited amount of fast memory - Economical solution is memory hierarchy - Locality - Cost performance Principle of locality - most programs do not access all code or
Pipelining. Parts of these slides are from the support material provided by W. Stallings
Pipelining Raul Queiroz Feitosa Parts of these slides are from the support material provided by W. Stallings Objective To present the Pipelining concept, its limitations and the techniques for performance
Branch-directed and Pointer-based Data Cache Prefetching
Branch-directed and Pointer-based Data Cache Prefetching Yue Liu Mona Dimitri David R. Kaeli Department of Electrical and Computer Engineering Northeastern University Boston, M bstract The design of the
Announcements. ! Previous lecture. Caches. Inf3 Computer Architecture
Announcements! Previous lecture Caches Inf3 Computer Architecture - 2016-2017 1 Recap: Memory Hierarchy Issues! Block size: smallest unit that is managed at each level E.g., 64B for cache lines, 4KB for
Exploiting Incorrectly Speculated Memory Operations in a Concurrent Multithreaded Architecture (Plus a Few Thoughts on Simulation Methodology)
Exploiting Incorrectly Speculated Memory Operations in a Concurrent Multithreaded Architecture (Plus a Few Thoughts on Simulation Methodology) David J Lilja lilja@eceumnedu Acknowledgements! Graduate students
Cache Designs and Tricks. Kyle Eli, Chun-Lung Lim
Cache Designs and Tricks Kyle Eli, Chun-Lung Lim Why is cache important? CPUs already perform computations on data faster than the data can be retrieved from main memory and microprocessor execution speeds
15-740/ Computer Architecture Lecture 22: Superscalar Processing (II) Prof. Onur Mutlu Carnegie Mellon University
15-740/18-740 Computer Architecture Lecture 22: Superscalar Processing (II) Prof. Onur Mutlu Carnegie Mellon University Announcements Project Milestone 2 Due Today Homework 4 Out today Due November 15
COMPILER-ASSISTED HARDWARE-BASED DATA PREFETCHING FOR NEXT GENERATION PROCESSORS
COMPILER-ASSISTED HARDWARE-BASED DATA PREFETCHING FOR NEXT GENERATION PROCESSORS A Dissertation Presented by YAO GUO Submitted to the Graduate School of the University of Massachusetts Amherst in partial
Review on ichat: Inter Cache Hardware Assistant Data Transfer for Heterogeneous Chip Multiprocessors. By: Anvesh Polepalli Raj Muchhala
Review on ichat: Inter Cache Hardware Assistant Data Transfer for Heterogeneous Chip Multiprocessors By: Anvesh Polepalli Raj Muchhala Introduction Integrating CPU and GPU into a single chip for performance
h Coherence Controllers
High-Throughput h Coherence Controllers Anthony-Trung Nguyen Microprocessor Research Labs Intel Corporation 9/30/03 Motivations Coherence Controller (CC) throughput is bottleneck of scalable systems. CCs
Memory systems. Memory technology. Memory technology Memory hierarchy Virtual memory
Memory systems Memory technology Memory hierarchy Virtual memory Memory technology DRAM Dynamic Random Access Memory bits are represented by an electric charge in a small capacitor charge leaks away, need
Getting CPI under 1: Outline
CMSC 411 Computer Systems Architecture Lecture 12 Instruction Level Parallelism 5 (Improving CPI) Getting CPI under 1: Outline More ILP VLIW branch target buffer return address predictor superscalar more
CS Memory System
CS-2002-05 Building DRAM-based High Performance Intermediate Memory System Junyi Xie and Gershon Kedem Department of Computer Science Duke University Durham, North Carolina 27708-0129 May 15, 2002 Abstract
ECE/CS 752 Final Project: The Best-Offset & Signature Path Prefetcher Implementation. Qisi Wang Hui-Shun Hung Chien-Fu Chen
ECE/CS 752 Final Project: The Best-Offset & Signature Path Prefetcher Implementation Qisi Wang Hui-Shun Hung Chien-Fu Chen Outline Data Prefetching Exist Data Prefetcher Stride Data Prefetcher Offset Prefetcher
Superscalar Machines. Characteristics of superscalar processors
Superscalar Machines Increasing pipeline length eventually leads to diminishing returns longer pipelines take longer to re-fill data and control hazards lead to increased overheads, removing any performance
The Stanford Hydra CMP. Lance Hammond, Ben Hubbert, Michael Siu, Manohar Prabhu, Mark Willey, Michael Chen, Maciek Kozyrczak*, and Kunle Olukotun
The Stanford Hydra CMP Lance Hammond, Ben Hubbert, Michael Siu, Manohar Prabhu, Mark Willey, Michael Chen, Maciek Kozyrczak*, and Kunle Olukotun Computer Systems Laboratory Stanford University http://www-hydra.stanford.edu
Dynamic Speculative Precomputation
In Proceedings of the 34th International Symposium on Microarchitecture, December, 2001 Dynamic Speculative Precomputation Jamison D. Collins y, Dean M. Tullsen y, Hong Wang z, John P. Shen z y Department
Outline. 1 Reiteration. 2 Cache performance optimization. 3 Bandwidth increase. 4 Reduce hit time. 5 Reduce miss penalty. 6 Reduce miss rate
Outline Lecture 7: EITF20 Computer Architecture Anders Ardö EIT Electrical and Information Technology, Lund University November 21, 2012 A. Ardö, EIT Lecture 7: EITF20 Computer Architecture November 21,
CACHE MEMORIES ADVANCED COMPUTER ARCHITECTURES. Slides by: Pedro Tomás
CACHE MEMORIES Slides by: Pedro Tomás Additional reading: Computer Architecture: A Quantitative Approach, 5th edition, Chapter 2 and Appendix B, John L. Hennessy and David A. Patterson, Morgan Kaufmann,
18-447: Computer Architecture Lecture 23: Tolerating Memory Latency II. Prof. Onur Mutlu Carnegie Mellon University Spring 2012, 4/18/2012
18-447: Computer Architecture Lecture 23: Tolerating Memory Latency II Prof. Onur Mutlu Carnegie Mellon University Spring 2012, 4/18/2012 Reminder: Lab Assignments Lab Assignment 6 Implementing a more
CS 838 Chip Multiprocessor Prefetching
CS 838 Chip Multiprocessor Prefetching Kyle Nesbit and Nick Lindberg Department of Electrical and Computer Engineering University of Wisconsin Madison 1. Introduction Over the past two decades, advances
ANALYSIS OF CACHE ARCHITECTURES
ANALYSIS OF CACHE ARCHITECTURES Nikolay Pavlovich Laptev Department of Computer Science University of California Santa Barbara Santa Barbara, CA, 93030 nlaptev@cs.ucsb.edu ABSTRACT The growing processor
Fall 2011 Prof. Hyesoon Kim. Thanks to Prof. Loh & Prof. Prvulovic
Fall 2011 Prof. Hyesoon Kim Thanks to Prof. Loh & Prof. Prvulovic Reading: Data prefetch mechanisms, Steven P. Vanderwiel, David J. Lilja, ACM Computing Surveys, Vol. 32, Issue 2 (June 2000) If memory
Cache Optimisation. sometime he thought that there must be a better way
Cache sometime he thought that there must be a better way 2 Cache 1. Reduce miss rate a) Increase block size b) Increase cache size c) Higher associativity d) compiler optimisation e) Parallelism f) prefetching
Chapter 2. Parallel Hardware and Parallel Software. An Introduction to Parallel Programming. The Von Neuman Architecture
An Introduction to Parallel Programming Peter Pacheco Chapter 2 Parallel Hardware and Parallel Software 1 The Von Neuman Architecture Control unit: responsible for deciding which instruction in a program
Architecture-Conscious Database Systems
Architecture-Conscious Database Systems 2009 VLDB Summer School Shanghai Peter Boncz (CWI) Sources Thank You! l l l l Database Architectures for New Hardware VLDB 2004 tutorial, Anastassia Ailamaki Query
EC 513 Computer Architecture
EC 513 Computer Architecture Cache Organization Prof. Michel A. Kinsy The course has 4 modules Module 1 Instruction Set Architecture (ISA) Simple Pipelining and Hazards Module 2 Superscalar Architectures